Claude Haiku 4.5 and the Economics of Small, Fast Models in Production

The interesting shift worth understanding is what small models have become capable of. A small fast model is no longer a weak model you tolerate for cheap tasks. It is a genuinely capable model for a wide band of real work, fast enough to serve interactive traffic and cheap enough to operate at volume without the bill dictating your architecture. That changes which model belongs at the centre of your system, and the organisations still defaulting to the largest model for everything are optimising for a hard-problem profile that most of their traffic does not have.

Most production traffic is not hard

The case for small models starts with an honest look at what production systems actually process. Strip a real workload down and the distribution is lopsided: a large volume of straightforward requests, a smaller band of moderately complex ones, and a thin tail of genuinely hard cases. The large model earns its cost only on that thin tail. On everything else, it produces the same answer a small model would have produced, more slowly and at several times the price.

Seeing this clearly reframes the default. The question is not whether a small model can handle your hardest cases, because it usually cannot, and that is fine. The question is what fraction of your traffic is actually hard, and the answer is almost always a minority. Sizing your model choice to the hard tail, when the hard tail is a fraction of the volume, means overpaying on the majority to be ready for the exception. A production AI system designed around the real traffic distribution operates the bulk on a small fast model and reserves the large one for the cases that need it.

The economics compound at volume

Here is the position worth holding. For high-volume production workloads, a small fast model as the default, with escalation to a larger model for the hard cases, is usually the right economic architecture, and the saving is not marginal, it compounds with every request. At scale, the difference between operating your volume on a small model versus a large one is not a line-item optimisation. It is a structural difference in what the system costs to operate.

The compounding is the point. A small per-request saving, multiplied across the full volume of production traffic, becomes a large operating difference, and it recurs every day the system operates. Speed compounds too: a faster model serves interactive workloads that a slower large model cannot, opening use cases that latency would otherwise have closed. The small model is not a compromise you accept to save money. For the right workload it is the better engineering choice, cheaper and faster on the traffic it handles, with the large model available for the minority that exceeds its reach. A governed AI deployment can show, per request, why the small or large model was used, which is a cost position and an audit position at once.

The escalation is the load-bearing part

The small-model-default architecture works only if the escalation works, and this is where teams under-invest. A small model past its competence does not announce the problem. It produces a plausible, cheaper, worse answer, and because the answer looks fine, the degradation is easy to miss until it shows up in outcomes. The saving from a small default is real only if you have engineered the mechanism that catches the cases the small model cannot handle and routes them to the larger one.

That escalation needs a signal for when the small model is out of its depth, whether a confidence measure, a difficulty classifier, or a validation step that catches weak outputs and reroutes them. Building that signal well is the real engineering of a small-model architecture, and getting it wrong in the cheap direction quietly degrades your hard cases while getting it wrong in the expensive direction sends too much to the large model and erodes the saving. Owning the deployment means owning the escalation as carefully as the default, because the small model is only as safe as the mechanism that knows when to override it.

The trade-off worth naming

Defaulting to a small model trades a capability ceiling for cost and speed, and the ceiling is real. The small model handles most of the traffic well and cannot handle all of it, and a workload that pushes past its ceiling without a working escalation gets quietly worse answers at a lower price, which is a bad trade dressed as a saving. The economics favour the small default only when the escalation is engineered to protect the cases beyond its reach. Without that, the saving is borrowed against quality you will pay back later, in outcomes you stopped watching.

Where the economics almost broke

A team defaulted a high-volume workload to a small fast model to cut cost, and the cost did fall sharply, which looked like a clear win. Then a class of complex requests that the previous large-model setup had handled well started getting worse answers, because the routing had been simplified to send everything to the small model and nothing was catching the cases that exceeded it. The degradation was subtle enough that it took a while to notice, and by then some of those worse answers had reached users.

The small default had been the right instinct and an incomplete implementation. The fix added the escalation the design had skipped: a difficulty signal that routed the genuinely hard requests to the large model while the bulk stayed on the small one. Cost stayed low, because most traffic was still cheap, and quality on the hard cases recovered, because those cases now reached the model that could handle them. The small model had been the right workhorse. It had needed a defined boundary and a path across it.

The assumption that the biggest model is the safe default is expensive precisely because it feels responsible, and it survives because the overpayment is invisible on a per-request basis and only becomes large when multiplied by volume you were not examining. Small fast models have earned a central role in production, and the organisations still routing everything to the largest model are sizing their entire system to a hard-problem profile that most of their traffic does not match.

The teams that get the economics right make the capable small model their workhorse and engineer a disciplined escalation to the large model for the cases that exceed it. They size the architecture to the real traffic distribution, not to the hardest case, and they protect quality with a boundary the small model is not asked to cross. If your production workloads default to a large model, the question worth asking is what fraction of that traffic is genuinely hard, because that fraction is smaller than the default assumes, and the gap between it and your current spend is where the saving lives.