Private AI
and GPU.
For teams where the useful question is not "which model?" but "where is this data allowed to go?" — customer records, health or financial data, legal documents, internal knowledge, anything under a contract that says it stays put.
Shared AI APIs are the right default for most products. They stop being the right default the moment the data crossing them is not yours to send, the volume makes per-token pricing absurd, or a customer contract requires an answer about where processing happens. This is the work of deciding which case you are in, and building for it.
Most teams do not need this.
Private inference is more expensive to run and more work to operate than calling an API. It is worth it when a specific constraint makes the shared path wrong — and when no such constraint exists, the honest recommendation is to keep using the API and spend the money elsewhere.
If none of these apply to you, that is the finding, and it is delivered as plainly as any other. Selling infrastructure nobody needs is a bad trade for both sides.
Four decisions that matter.
Data boundaries
The map before the machinery: which categories of data exist, which are allowed to leave your infrastructure, and which are not. Then the enforcement — what is stripped or masked before a prompt is built, what may be retained by whom, and what your processing terms actually commit you to when a customer asks.
DeliverableA written data-flow boundary: what may cross, what may not, and where it is enforced in code.
Private inference
Running open-weight models on infrastructure you control, or in a dedicated tenancy with contractual guarantees about retention and training. Which model sizes actually fit the task, what quality you give up against a frontier API, how to keep a fallback, and how to evaluate quality before it reaches customers.
DeliverableA serving architecture with the model choice justified against measured quality, cost, and latency.
Dedicated GPU capacity
What hardware the workload genuinely needs, and whether to rent, reserve, or own it. Sizing against real throughput rather than a spec sheet, keeping utilisation high enough that dedicated capacity beats per-token pricing, and what happens to your service when a single GPU host fails.
DeliverableA sizing and cost model against your actual volume, with the break-even against API pricing shown.
Internal assistants and retrieval
Assistants over internal knowledge, built so the permission model survives contact with retrieval. Whose documents a given employee's answer may draw on, how access changes when someone moves team or leaves, what gets logged, and how staff can tell a grounded answer from a fluent guess.
DeliverableA retrieval design where document permissions are enforced at query time, not hoped for.
Decide, then build.
- 01
The boundary comes before the model
Choosing a model first and asking about data later is how teams end up rebuilding. The boundary is the constraint everything else is designed against, so it gets written down first.
- 02
Measured against your task, not a leaderboard
Benchmark rankings say little about whether a model handles your documents. Model choice is argued from evaluation on your own cases, with the quality gap against a frontier API stated rather than glossed over.
- 03
Cost modelled honestly, including the boring parts
Dedicated capacity has real costs that quotes tend to omit: idle time, redundancy, storage, and the hours someone spends operating it. The comparison against API pricing includes them.
- 04
Operable by your team
Private infrastructure that only its author can run is a liability. Deployment, monitoring, and the failure path are documented so your team owns it.
How an engagement runs.
Constraint
A short call to establish which constraint is driving this — boundary, volume, or control — and whether it is real. Sometimes this is where it stops.
Design
The data boundary written down, then a serving architecture and sizing model built against it.
Evaluate
Candidate models measured on your own cases, with quality, cost, and latency compared against your current setup.
Build
The chosen architecture deployed, monitored, documented, and handed over to your team.
Start with the constraint.
Tell me what data is involved and what you are not allowed to do with it. If the answer is that a shared API is fine, you will get that answer. The other two layers are application hardening and infrastructure hardening. Once it is built, continuous operations is what keeps it running.