The request path

Illustrative deployment · informed by vLLM’s documentation

A client application connects through an authenticated endpoint to an inference runtime on a rented server. Model files and application data sit on attached storage. A separate administrator connection maintains the host. Logs or backups may be written elsewhere. This is an explanatory setup, not a report of a client system.

The request can touch several readers and stores: the application, endpoint, runtime, application logs, host diagnostics and backup systems. A privacy specification must address the actual path.

Boundaries and questions to resolve
BoundaryQuestion
Client to endpointWho authenticates, where does encryption terminate, and what content can the endpoint record?
Endpoint to runtimeWhich services are reachable, and are internal ports exposed beyond the intended network?
Runtime to storageWhat is written, who can read it, and what survives a restart?
Administrator to hostWho has access, under what conditions, and what can that access reveal?
Host to external servicesAre downloads, updates, telemetry, model tools or APIs permitted?
Storage to backupWhich copies exist, who holds them, and when are they deleted?

Endpoint and administrator access

Protecting an inference endpoint does not automatically protect every management or internal service. For example, vLLM’s security documentation describes additional network and management-interface protections beyond its API-key setting.

An administrator may be able to read storage or view a running process. A client API key cannot prevent host or provider access.

Logging and retention

Content logging and metadata logging have different consequences. Timing, token counts or client identifiers can still be sensitive when prompts are omitted. Other services may log requests even if the application does not.

A logging review covers locations, readers, retention, backups and deletion at rental end across the proxy, runtime, host and provider. A zero-logging claim needs checks against deployed settings.

Model behaviour and isolation

Model selection, system prompts and output filters affect behaviour. Network isolation and retention depend on the deployment. An unfiltered model can still call external tools or run on a host with administrator access and persistent logs.

For a rental, we agree model, runtime, compute allocation, location, access paths, external connections and support responsibilities. Capacity needs a workload test on that configuration. Privacy needs checks of its access and storage.