No Lock on the GPU

The setup

The target was an AI voice-agent platform: the kind of company that gives other businesses a synthetic voice that answers phones, books appointments, and talks to customers in real time. The product is built on speech models - text-to-speech to speak, speech-to-text to listen, and a voice-cloning capability so a customer can have an agent that sounds like a specific person.

All of that runs on GPUs. Speech synthesis at conversational latency is expensive compute, and a company like this typically has a fleet of high-end accelerators doing nothing but inference. That fleet is the most costly thing the company owns and one of the most powerful, because the models running on it can generate a convincing human voice from a short sample. So it was worth understanding exactly how that compute was fenced off.

Recon

Working black-box, from the open internet with no account, we mapped the platform’s hosts and watched how each one answered. The main API gateway behaved exactly as it should. Every route we touched came back asking for a bearer token, and an absent or wrong token was rejected. The front door was uniformly authenticated.

Then we noticed the gateway was not the only thing reachable. There were separate model-server hosts - the GPU-backed machines that actually run the speech models - answering on a public hostname of their own. They were a different service from the gateway, sitting on the same public internet.

The hypothesis

The reasoning was straightforward. Authentication on a platform like this almost always lives in one place: the API gateway that fronts the product. That gateway checks the token, then forwards the real work to internal model servers that do the GPU inference. Those model servers are usually treated as trusted sidecars - the assumption is that only the gateway ever talks to them, so they do not need their own lock.

If that was the design here, and the model servers were also directly reachable on a public hostname, then the gateway’s authentication would be protecting nothing. The hypothesis was that the sidecars did not inherit the gateway’s auth - that we could talk to the GPUs directly and skip the front door entirely.

What we tried

The model server spoke its own API, and the service published a spec describing every route it offered. We read the spec first to understand the surface, then tested the cheapest, most reversible thing on it: synthesis.

We sent two text-to-speech requests directly to the model server with no credentials. Both returned valid audio - the model spoke our text back to us as a wav stream, rendered on the company’s GPUs, billed to the company, requested by an anonymous caller. The authentication setting on the route was none, and it was none on every route the spec described.

That mattered because of what the other routes were. The spec listed a speech-to-text route (unauthenticated transcription, more free GPU time), a voice-cloning route that turns an audio sample into a synthetic voice, and dictionary-mutation routes that change how the models pronounce specific words - a global, persistent configuration store shared across the platform. There were also telemetry routes that reported real-time call volume.

What we deliberately did not do is the important part. We confirmed synthesis with two calls because audio generation is self-contained and changes no state. We did not run the voice-cloning endpoint - generating a synthetic voice, even of ourselves, is not something to do on someone else’s exposed infrastructure to prove a point. We did not touch the dictionary-mutation routes, because a write there is global and persistent: mutating one entry would change pronunciation for every customer on the platform. We confirmed those two capabilities existed and were callable from the service’s own published spec, and we stopped there.

What we found

The finding was that the GPU model servers had authentication set to none on every route, on a public hostname, while the gateway in front of them was fully locked. Grouped by what an anonymous caller could reach:

• Free inference on the company’s GPUs. Unauthenticated text-to-speech and speech-to-text ran on H100-class accelerators. Anyone could point a script at the host and consume the platform’s most expensive resource at will. This is straightforward resource abuse and direct cost exposure - the compute bill has no owner attached to it.

-> POST /<synthesis-route>            (no Authorization header)
   { "text": "...", "voice": "<id>" }

<- 200 OK
   audio/wav  ... valid synthesized speech, rendered on the platform GPUs ...

• A deepfake vector. The voice-cloning route was reachable without authentication. A capability meant to let a paying customer build one specific agent voice was, in this posture, an open synthetic-voice generator - an impersonation and fraud tool anyone could drive.
• Global, persistent config tampering. The dictionary-mutation routes let an unauthenticated caller change how the models pronounce words, platform-wide and permanently. That is a write into shared state that every customer inherits.
• Operational telemetry leak. The telemetry routes reported real-time call volume, handing an outsider a live view of the platform’s traffic.

The shape of the problem: the front door was locked and the compute behind it was not. The gateway’s bearer check was real, but the GPUs it was supposed to protect answered directly to anyone who knew the hostname.

Why it happens

No one decided to leave the GPUs open. This is what a normal, sensible architecture looks like when one deployment detail slips.

The clean design is a gateway that authenticates every request and internal model servers that trust the gateway. Inside a private network, model servers with auth: none are fine - they are supposed to be unreachable except through the front door, so adding auth to them feels redundant. The gap opens when one of those sidecars also gets a public hostname, usually for a legitimate reason: a health check, a direct-access path for testing, a load balancer wired one host too wide. The moment that happens, the “only the gateway talks to us” assumption is false, and the auth that was skipped because it seemed redundant is the only thing that was holding.

The result is a capability as powerful as anything the company owns - its GPUs and its voice-cloning model - sitting on the internet, because authentication was designed to live one layer up and the layer below was never meant to be reachable.

For developers

If you run GPU-backed model servers behind a gateway, four controls close this:

1. Give every model server its own authentication. Do not rely on the gateway alone. Each inference host should reject an unauthenticated request itself, so that reaching it directly gains nothing. Defense in depth is the whole point: the sidecar’s lock is what saves you when the network boundary leaks.
2. Never put GPU inference on a public host. Bind model servers to the private network and let only the gateway reach them. A directly reachable inference endpoint is a standing invitation to run free compute on your most expensive hardware.
3. Treat cloning and other high-capability routes as privileged, always. Voice cloning is a deepfake generator; it should sit behind authentication, authorization, and rate limiting even inside your own network. Assume it will one day be reachable and make that survivable.
4. Protect config mutators as writes. Dictionary and pronunciation routes change shared, persistent state. They are writes, not reads, and belong behind the same authentication and audit trail you would put on any state-changing action - not left open because they look like tuning knobs.

And one detection step: audit your model servers’ access logs for inference calls that did not arrive through the gateway. If a request reached a GPU host directly, the boundary you were counting on is already porous.

The takeaway

A locked front door protects nothing if the compute behind it answers to anyone who walks around the building. Authentication that lives only on the gateway leaves every model server one deployment slip away from being an anonymous capability - free GPU time, a deepfake generator, and a global config store, all reachable without a credential. GPU inference is too expensive and too powerful to guard at one layer only. Every service that can do real work needs its own lock.

This teardown is one instance of a pattern we see repeatedly - a secured main service with an unsecured sibling beside it. We wrote up the general case, and how to defend against it, in The Sibling Service That Skipped Authentication.