In June 2025, roughly 70% of all tokens on OpenRouter — the router developers use to send requests to different models — went to three American labs: Google, OpenAI, and Anthropic. A year later, in June 2026, their share had dropped to about 30%. The rest went to open and cheap models, mostly Chinese.
The models at the top got noticeably stronger over that year. Demand moved the other way.
Six weeks ago, in «AI Is Turning Into Expensive Infrastructure», we argued that access to the strongest models was getting pricier and splitting into tiers, and that knowing which model to use for which job would become a skill of its own. At the time that was a forecast. Now it shows up in the numbers, and the shift is running faster than we expected. The explanation, though, isn’t the one that jumps to mind first.
Where the Demand Went
On that same OpenRouter, by mid-June 2026 the split between open and closed models had flipped: 60 to 40 in favor of open. Three months earlier, in March, it was exactly the reverse. Chinese-origin models (DeepSeek, Tencent, Alibaba, MiniMax, Xiaomi, Moonshot) now take more than 45% of total token volume; a year before that they held under 2%. DeepSeek alone doubled its share in six months, from 9% to roughly 18%, putting it ahead of any other provider.
One caveat matters. OpenRouter is a slice of developer traffic, not the whole market. Enterprise contracts with Microsoft, Google, and Anthropic never show up in it, and in revenue terms the picture looks different, because cheap tokens deliver volume, not income. As an indicator of how people behave when every request comes out of their own pocket, the slice still tells you something.
The easy explanation is that everyone chased the low price. The gap is real: a cheap open tier runs about $0.14 per million input tokens, while a closed vendor’s flagship runs $5 for that same million. Thirty times the cost for the same request. But price alone explains nothing. Cheap models existed a year ago, and two years ago. You just couldn’t build anything on them.
What Actually Changed: the Floor Came Up
That’s the real shift. The top didn’t get cheaper — the middle got good enough to work.
The clearest way to see it is by intelligence class. GPT-4-level capability, which was the ceiling in 2023 and cost around $30 per million tokens, now costs under a dollar — by some estimates, cents. That’s roughly a 95% drop in two years, and orders of magnitude over three. A model people once queued up for now sits in every provider’s budget tier, and on plenty of tasks it still does the job.
Distillation amplified the effect. The idea is that a large model trains a small one, and the small one inherits the behavior you care about while staying far cheaper. Phi-4-reasoning-plus, at 14 billion parameters, scores 77.7% on the AIME 2025 math olympiad — the same as DeepSeek-R1 at 671 billion on that benchmark. It’s a narrow slice and you can’t generalize it to everything, but the direction is clear: “big versus small” stopped meaning “smart versus dumb.”
For businesses and agents, something else matters more, and benchmarks barely capture it. The boring capabilities got better — the ones everything used to fall apart on:
- following instructions. A weak model a year ago would drift off-format by the third paragraph. A mid-tier model today holds the frame;
- structured output. Returning valid JSON against a schema became the norm rather than a lucky break;
- tool calls. An agent is mostly a loop: think, call a function, look at the result. A year ago a correct function call from a cheap model was a coin flip. Today it’s table stakes, and some mid-tier models tuned for tool use beat general-purpose flagships at orchestration.
That’s why demand moved. Agent work isn’t made of olympiad problems — it’s careful execution of well-defined steps, many times over. Once mid-tier models stopped breaking on that routine, paying flagship rates at every step got hard to justify.
The other half of the explanation is that real workflows are simpler than they look from the outside. Our agent for as-built construction documentation reads site paperwork, checks it for completeness, and drafts acceptance certificates. That sounds like one big task. In practice it’s a chain: identify what kind of document arrived, pull the fields out of it, check them against a list of requirements, draft the certificate, and ask a question wherever the data is missing. Half those steps are recognition and shuffling, where a strong model and a cheap one give the same answer and the bill differs several times over. Difficulty isn’t spread evenly along the chain — it’s concentrated in two or three places: where you have to interpret an ambiguous phrase, reconcile conflicting data, or decide what happens next.
Harvey, the legal AI company — high cost of error, not a place anyone cuts corners on cheap tooling — landed in the same spot. They built a hybrid setup: heavy tasks go to a flagship model, routine work to a cheap open-weight one. Inference costs dropped threefold with no loss of quality. Their framing is sharper than ours: the definition of quality moved from “use the most powerful model for everything” to “use the one that gets to the right answer most efficiently.”
Where Cutting Model Costs Will Burn You
Stop here and you get a genuinely bad piece of advice: put a mid-tier model everywhere and enjoy the invoice. There’s math that breaks it.
When an agent runs a chain of dependent steps, the probabilities multiply. A step that’s 95% accurate, repeated ten times, gets you to about 59% end-to-end. At 90% per step you land at 35%. At 85% you’re at 20% — the chain completes once in five attempts. A model gap that looks like “eh, slightly worse” on a single step turns into “doesn’t work” across a long chain.
Hence the rule: the longer the autonomous stretch where nobody checks the intermediate result, the more expensive your model needs to be.
| Stretch of work | Who catches the error | Safe to economize? |
|---|---|---|
| Single step, a human reviews the output | the human, immediately | Yes. The gap between models is barely visible |
| Chain of 3–5 steps, checked at the end | the human, at the end | Partly: yes on recognition and data transfer, no on decisions |
| 10+ steps with no stops | nobody | No. Errors compound |
| Free-form input — a document everyone fills in their own way | nobody, until the very end | No |
| Long context that has to be held in full | nobody | No |
| Cost of a mistake exceeds a year’s difference in spend | depends on the process | No — tokens aren’t the number to optimize |
The flip side of the rule is friendlier: wherever review is genuine, your requirements on the model get a lot softer. We wrote about that separately when we looked at the human-plus-AI pairing — a mid-tier model with review regularly beats a flagship without one.
One more thing, usually forgotten in the slide deck about savings. A router that decides where to send each request is code: you write it, debug it, and maintain it. Plus a second provider, a second key, a second failure mode. Consultants put average savings for projects that got routing into production at around 38%, ranging from 30% to 70% depending on how uniform the tasks are. At volumes where the monthly bill is measured in tens of dollars, that work will never pay for itself.
Organizations whose data legally cannot leave a closed perimeter are a separate story — banks, healthcare, government, critical infrastructure. There the choice doesn’t start with the price list but with the question of where the data is allowed to be processed at all, and a stronger middle tier has opened a door for them. That’s a conversation of its own.
Why Swapping the Model Usually Doesn’t Help
Now the uncomfortable part, the one that spoils all the neat arithmetic about savings.
We fell for the obvious temptation. Torgi, our tender agent, worked unevenly: sometimes it produced a solid breakdown of a procurement, sometimes it lost half a document or mixed up the client. A stronger model came out, we plugged it in, and waited for quality to lift on its own. What we got was slightly smoother phrasing and exactly the same missing documents. The model had nothing to do with it — half the text never reached it in the first place.
While the middle was weak, model choice really did decide the outcome: pick a flagship and it ran, pick a cheap one and it collapsed. Now that the middle has evened out, the gap between models on routine work is smaller than the gap between what’s been built around them.
Around the model sits the scaffolding: how context gets assembled, which tools are available, what happens on an error, where it stops and asks a human, what persists between runs, how the result gets checked. The model determines how smart an answer is possible at all. The scaffolding determines whether the right question reaches the model and what happens to the answer afterward. At volume, the second one usually costs more.
That gives you a practical test that saves months. When an agent performs badly, go through the failures one at a time. If the model had everything it needed and still answered stupidly, that’s on the model, and it’s worth replacing. If it answered badly because it got half a document, contradictory instructions, or no tool to ask for clarification, no model swap will save you. We paid for that lesson with our own time, and in our experience the second case is far more common.
There’s good news in it too. Scaffolding that works survives a model change. You write it once, and after that you can swap providers to fit the task and the budget — which is exactly what a market demands when the best choice goes stale within a quarter. The mid-tier segment moves fastest of all: what was the obvious pick in March has been displaced by July. If changing models means rewriting half your code, that rigidity will cost you more than you’ll ever save on tokens.
What to Do About It
The practical conclusion isn’t to rush everything onto cheap models. It’s that “which model should we use” is a question asked at the wrong level. A model isn’t chosen for a company, or even for a product. It’s chosen for a step.
Revisit any decision older than a year. If you picked your model in 2025, you picked it in a different market. Back then the middle genuinely couldn’t handle agent routine, and betting on a flagship was right. Today the same bet can cost several times more with nothing to show for it.
Go through the failures before you change the model. Swapping models is the most visible move available and almost never the most useful one. First look at what actually reached the model, and what was done with its answer afterward.
Break the process into steps and measure. Don’t argue in a meeting about which model is smarter. Take fifty real cases, run every step through a mid-tier model and a flagship, and compare both against a reference. Usually it turns out the two differ on two steps out of seven. After that the question is settled by numbers instead of opinions.
Perimeter first, price second. If you’re handling personal data, medical records, financial data, or anything under critical-infrastructure rules, start with where that data is legally allowed to be processed. The shortlist narrows itself, and half the arguments about cost disappear.
Cost your operations, not your subscriptions. What does one analyzed tender cost? One document package? One customer reply? And what happens to that number when volume grows tenfold?
Instead of a Conclusion
A year ago the main question was whether we’d get access to the strongest model. Now it’s a different one: at which steps do we actually need it, and what have we built around it?
The top tier hasn’t gone anywhere, and for the jobs it’s meant for it’s irreplaceable. It’s just that the floor came up so far that almost all of an ordinary company’s ordinary work now sits below it. The routine is handled by a model that costs several times less, fits in your own data center, and never makes headlines for a benchmark record.
The headlines go to the top. The work gets done by the middle. And whether that turns into a working tool or an expensive toy is no longer decided by the model, but by what you’ve built around it.