Hosted AI API vs self-hosted models: cost and control compared

Every AI feature starts with the same fork: call a provider’s API, or run a model you control. The argument for each is usually made by someone selling one of them.

Here’s the version with the incentives removed, and the hybrid most teams actually end up running.

The real comparison

	Hosted API	Self-hosted
Capability ceiling	The frontier	Best open weights
Cost shape	Per token, scales with use	Fixed hardware, marginal near zero
Data leaves your network	Yes	No
Operations burden	Theirs	Yours entirely
Model stability	Deprecated on their schedule	A file that never changes
Time to first request	Minutes	Days to weeks

The cost argument is usually wrong in both directions

Self-hosting advocates compare hardware cost against API list price at peak volume, and forget the salary of whoever keeps it running.

Serving efficiency complicates it further, since a well-run vLLM deployment gets several times the throughput of a naive one from the same card. Kwon and colleagues’ PagedAttention work reported 2-4x throughput gains, which means the self-hosted cost per token depends heavily on how good you are at self-hosting.

The API bill is visible on an invoice. The self-hosting bill is spread across salaries, on-call rotas and the project that slipped while someone debugged CUDA drivers.

API advocates make the opposite error, projecting today’s usage forward when the whole point of a successful feature is that usage grows. Per-token pricing that was trivial at launch becomes a line item the finance team notices, and our piece on feature costs works the arithmetic properly.

The honest framing is that hosted converts capital cost into marginal cost. That’s good while you’re small or spiky, and progressively worse as volume becomes large and steady.

Capability is a narrower argument than it was

Two years ago self-hosting meant accepting a large quality gap. That gap narrowed to the point where two of the six labs at the top of the 2026 AI Index ratings publish open weights.

Meta shipped a 30B model built to run agents on a single consumer GPU, per TechCrunch, and researchers now test open models against closed ones in specialist professional domains and publish competitive results.

The gap that remains is at the very top: hard multi-step reasoning, the longest documents, the most ambiguous instructions. If your workload lives there, hosted frontier models are still the only option, and no amount of infrastructure enthusiasm changes that.

Stability cuts the other way

Hosted models change underneath you, and lately they change every few weeks. Google shipped an entry-level model three weeks after its predecessor, and deprecation eventually forces every user forward.

A file on your disk behaves identically next year, which matters if you’ve tuned prompts against its specific quirks or documented its behaviour for a compliance file. Our piece on the release treadmill covers who absorbs that churn.

The counterweight is improvement. A pinned model also never gets better, and the hosted fleet does, so stability is a benefit precisely in proportion to how much you’ve invested in the current behaviour.

Data residency is the argument that ends discussions

For some organisations this whole comparison is decided by one row of the table. If the data cannot leave the network, the data does not leave the network.

Regulation has sharpened that. The erasure and accuracy duties in our piece on GDPR and model weights are simpler to answer when you control every copy, and enterprise contracts increasingly ask the question directly.

Check the weaker version too: hosted providers differ on whether they train on your inputs, and the enterprise tier often behaves differently from the consumer one. That single setting has decided more procurement outcomes than any benchmark.

The hybrid everyone converges on

Run a capable open model for the high-volume, low-complexity and privacy-sensitive work. Send the hard reasoning and long documents to a hosted frontier model.

Pricing pressure makes the hosted half cheaper than it was, too. Grok 4.6 matched GPT-5.6 Sol at a fifth the output price, and with a five-times spread across providers at similar quality, the frontier tier is itself a shopping decision.

Most workloads split naturally along those lines, and because self-hosted engines expose OpenAI-compatible APIs, the routing is a base URL rather than an architecture.

Our guides to the self-hosted stack and choosing an inference engine cover the local half of that arrangement.

The failure mode on each side

Both choices have a characteristic way of going wrong, and knowing them is more useful than the comparison table.

The hosted failure is quiet dependence. The provider changes a model, adjusts a price, or tightens a rate limit, and your product’s behaviour shifts on somebody else’s schedule while your team debugs a change they didn’t make.

The self-hosted failure is quiet neglect. The deployment works on day one, nobody owns it by month six, and it slowly becomes the least-patched, least-monitored system in the company while still handling your most sensitive data.

Neither failure appears in the evaluation that chose the approach, because both are organisational rather than technical. Plan for the one your organisation is prone to.

How to decide in an afternoon

Write down your monthly token volume, honestly, including the growth you actually expect rather than the growth in the pitch deck. Below a few hundred million tokens a month, hosted wins on cost alone and the rest of this is moot.

Then check whether any of your data is contractually or legally confined. If it is, that portion self-hosts regardless of economics, and you’re designing the hybrid rather than choosing a side.

Quantisation is what makes the local side realistic on ordinary hardware, with GPTQ-class methods running models at 4 bits per weight with negligible degradation.

Then run ten real tasks against the best open model that fits your hardware and the hosted model you’d otherwise buy. The measured gap on your own work, priced at your own volume, is the entire decision, and it takes one afternoon to produce.