Self-hosting moved
the boundary, not the work.
You moved the model in-house so customer data would stay inside a boundary you control. The server holding that boundary shipped with no authentication, reachable the moment you followed the setup instructions.
The reasoning is sound, and most teams reach it independently: send prompts to OpenAI or Anthropic and that data leaves your network on every call, so you pull the model in-house instead — Ollama on a GPU box, vLLM behind your own API, maybe a gateway in front of both so the app code doesn't have to care which one answers. The data-boundary problem looks solved. Then the setup guide tells you to bind the server so your app can actually reach it, and you follow that instruction exactly once, without reading past the line that makes it work.
A loopback default, and the line that switches it off.
Ollama's default bind is documented and deliberate: per Ollama's own FAQ, the server listens on 127.0.0.1 port 11434 until the OLLAMA_HOST variable says otherwise — and nearly every "run this on your GPU box" guide sets it to 0.0.0.0 so the app server elsewhere on the network can reach it. The variable does exactly what its name says, and nothing more; it doesn't say anything about who else can now reach the port. Ollama's own authentication documentation turns out to be about signing in to ollama.com's cloud product — the local server it fronts has no login built in.
The firewall was supposed to be the second layer, and for a container it can silently not be consulted at all. Where the model runner sits behind Docker, Docker's own documentation on packet filtering is direct about why: a published container port is routed through the nat table ahead of the INPUT and OUTPUT chains a host tool like ufw inspects. An operator can run ufw status, see "active," and still be running an internet-reachable server — the two systems were never asked about the same port. One config decided the address; nothing decided whether the network should carry it.
What waits behind that gap has a documented shape too. In February 2026, Cyera researchers reported CVE-2026-7482, nicknamed "Bleeding Llama": an unchecked tensor-shape field on Ollama's /api/create upload endpoint that reads past the file's declared size into whatever sits next to it in heap memory. Cyera's writeup shows the conversion trick that keeps the leaked bytes intact, and a follow-on call that uploads the result somewhere the finder controls. Three unauthenticated requests, on a server whose only misstep was answering the address it was told to answer.
What a routine internet sweep already sees.
None of that requires anyone to target you specifically. Internet-wide scanners sweep every routable IPv4 address on a running schedule, the way a search engine crawls pages, and a model runner's response looks nothing like a generic web server's. Cisco Talos's own case study, published in September 2025, worked from Shodan's already-indexed results rather than active probing and still logged over 1,000 exposed Ollama instances inside the first ten minutes — findable by the default port, an "Ollama" string in the response banner, and a "Server: uvicorn" header common to the framework these runners are usually built on. Censys' own scan of all 65,535 TCP ports, not only the default one, turned up roughly 10,600 high-confidence instances, more than a quarter of them sitting on a port other than 11434 — obscurity by port number buys almost nothing against a scanner that checks every one.
The fingerprint is distinctive because the endpoint responds with structured JSON naming the models it runs, content no generic web server returns unprompted. The Cloud Security Alliance's own count put the number of reachable, unauthenticated Ollama instances at roughly 175,000 in June 2026, and by the middle of that month had already watched one of them folded into an automated attack pipeline with no operator at the keyboard. Being reachable and being found stop being two separate events at that scale.
Fixed points before the checklist.
Three things are worth separating from the diagnostic list that follows: what the model runner actually defaults to, when the vulnerability behind this specific gap became public record, and what OOPUO will only touch with your say-so.
OLLAMA_HOST is set to anything elseWhere the four gaps actually sit.
Is the bind address actually private?
Check what OLLAMA_HOST or vLLM's --host flag is actually set to in the running config, since deployment docs are written for the demo and rarely get revisited afterward. Loopback only ever reaches the process itself; 0.0.0.0 reaches anything that can route to the box — the public internet, if nothing else stops it.
Passes whenThe value is loopback or a private segment, and you can name who set it that way and why.
Does the firewall actually see this traffic?
Where a container runtime is involved, confirm the block at the layer that actually handles the packet — a cloud security group, or the runtime's own filtering rules — rather than a host tool's status flag, which can read "active" while the traffic routes around it entirely.
Passes whenThe block is verified at the layer that actually sees the traffic: the security group, or the runtime's own filtering rule.
Does the auth flag cover the route that matters?
vLLM's own security documentation lists endpoints its --api-key flag does not protect even when configured — /invocations among them, which is also where CVE-2026-22778's unauthenticated remote-code-execution chain landed. "I set an API key" isn't proof of coverage on its own; the excluded-endpoint list is.
Passes whenYou've read that list rather than assumed the flag is total, and the excluded paths are blocked some other way.
How would you actually know a version is behind?
Ollama's fix for CVE-2026-7482 shipped inside v0.17.1's release notes, which listed Nemotron model support, MLX memory improvements, and a web-search toggle — no CVE number, no security label, nothing that would surface in a changelog-only check.
Passes whenSomething other than the release notes tells you when to upgrade — a subscribed advisory feed, running on its own schedule.
Bind narrow, patch on a schedule.
Bind to nothing but what has to reach it
Loopback, or a private network segment — never a public interface. If the app and the model share a box, they don't need a routable address between them at all.
Give the firewall something to actually enforce
Where a container runtime routes around the host firewall, apply the block at the layer that really sees the traffic: the platform's security group, or the runtime's own filtering rules.
Close the gaps a flag's own docs admit to
Where the server's built-in auth has documented exceptions — vLLM's excluded endpoints are the example — put a reverse proxy in front that allowlists exactly the paths you serve and denies the rest by default.
Subscribe to the security-advisory feed directly
A quiet changelog told you nothing about v0.17.1. Watching GitHub Security Advisories on the project puts the next one on record the day it's filed.
This describes checking infrastructure you operate, against sources anyone can read — not testing a server you don't run. Where self-hosted inference is part of what OOPUO reviews, that happens on surfaces named in advance, with a disclosure path agreed first, and with your permission AI-assisted review runs only against those approved surfaces.
The boundary is only as real as the box holding it.
Moving a model in-house is an architecture decision with an operational half nobody assigns to anyone by default — reachability, authentication, and a patch cadence that doesn't wait on a changelog label. Private AI is where OOPUO does that half: self-hosted models, GPU boxes, and the data boundary they're supposed to hold. Send what you're running.