The request path
Illustrative deployment · informed by vLLM’s documentation
A client application connects through an authenticated endpoint to an inference runtime on a rented server. Model files and application data sit on attached storage. A separate administrator connection maintains the host. Logs or backups may be written elsewhere. This is an explanatory setup, not a report of a client system.
The request can touch several readers and stores: the application, endpoint, runtime, application logs, host diagnostics and backup systems. A privacy specification must address the actual path.
| Boundary | Question |
|---|---|
| Client to endpoint | Who authenticates, where does encryption terminate, and what content can the endpoint record? |
| Endpoint to runtime | Which services are reachable, and are internal ports exposed beyond the intended network? |
| Runtime to storage | What is written, who can read it, and what survives a restart? |
| Administrator to host | Who has access, under what conditions, and what can that access reveal? |
| Host to external services | Are downloads, updates, telemetry, model tools or APIs permitted? |
| Storage to backup | Which copies exist, who holds them, and when are they deleted? |
Endpoint and administrator access
Protecting an inference endpoint does not automatically protect every management or internal service. For example, vLLM’s security documentation describes additional network and management-interface protections beyond its API-key setting.
An administrator may be able to read storage or view a running process. A client API key cannot prevent host or provider access.
Logging and retention
Content logging and metadata logging have different consequences. Timing, token counts or client identifiers can still be sensitive when prompts are omitted. Other services may log requests even if the application does not.
A logging review covers locations, readers, retention, backups and deletion at rental end across the proxy, runtime, host and provider. A zero-logging claim needs checks against deployed settings.
Model behaviour and isolation
Model selection, system prompts and output filters affect behaviour. Network isolation and retention depend on the deployment. An unfiltered model can still call external tools or run on a host with administrator access and persistent logs.
For a rental, we agree model, runtime, compute allocation, location, access paths, external connections and support responsibilities. Capacity needs a workload test on that configuration. Privacy needs checks of its access and storage.