For most of the last decade, "AI is expensive" was a fair summary. Running a capable model cost real money per query, so autonomous agents — the kind that chain many steps together rather than answering one question — were confined to research demos and well-funded labs. Two things changed at once in 2025 and 2026. Agents got good enough to trust, and the cost of running them fell off a cliff. Put those together and a category of software that was economically impossible eighteen months ago is now within reach of a two-person startup.
The price of intelligence fell off a cliff
The numbers are striking. According to Stanford's 2026 AI Index, the cost of querying a model with GPT-3.5-level performance dropped from roughly $20 per million tokens in late 2022 to about $0.07 per million tokens by late 2024 — a decline of more than 280× in roughly 23 months. Separate industry data cited by Perplexity AI Magazine puts the fall in enterprise token costs at about 67% in a single year. A customer-service chatbot that would have cost hundreds of dollars a month in compute two years ago now costs a few dollars for the same quality of responses.
None of this is magic. It is the compounding of better hardware (Nvidia's H100 and B200 GPUs), smarter serving software (paged attention, speculative decoding, mixture-of-experts routing and aggressive quantization of open-weight models), and the scale economies that come from processing dramatically higher volumes of tokens. As inference got cheaper, more use cases crossed the line from "too expensive to automate" to "obviously worth it."
Then agents arrived — and started eating tokens
Here is the catch that surprises first-time builders. A chatbot answers once. An agent loops: it plans, retrieves information, calls tools, checks the result, and retries until the task is done. Industry analysis of enterprise deployments finds that moving from a chatbot to an agent can raise token consumption by one or two orders of magnitude — ten to a hundred times. In agentic workloads the final visible output can be only 5–15% of the tokens consumed; the rest is context, including retrieved documents, previous steps and the conversation history that accumulates with every tool call.
So cheaper tokens do not automatically mean cheaper bills. McKinsey argued in 2026 that per-token pricing has "stopped being a useful measure" of what enterprises actually pay, because the bill is now dominated by architecture — retrieval, repeated refinement, evaluation and the systems wrapped around the model. In its own May 2026 Enterprise AI FinOps survey, 93% of respondents said they had exceeded their AI budgets. The lesson is uncomfortable: the model is no longer the expensive part. The way you wire the agent is.
Why agents got good enough to trust
The reliability curve is the other half of the story. On real-world agentic benchmarks tracked by the 2026 AI Index, success on terminal tasks climbed from about 20% in early 2025 to 77.3% in early 2026, while desktop-driving accuracy rose from about 12% to 66.3% over the same window. A year ago these tools failed four times out of five; now they succeed three times in four. That is the difference between a demo and a deployable feature.
The business proof point is Klarna. Within a month of launching its OpenAI-powered assistant in February 2024, the fintech handled about 2.3 million customer conversations — roughly two-thirds of all its support chats — at the equivalent workload of 700 full-time agents, cutting average resolution time from eleven minutes to under two and projecting a $40 million profit improvement for the year. The nuance matters: by mid-2025 Klarna had rehired humans for complex cases, settling into a hybrid model. Agents excel at the high-volume, well-defined long tail; humans own the emotionally charged, compliance-sensitive edge. The $40 million was largely avoided hiring, not headcount slashed — an important distinction when communicating AI economics.
The real unlock is architectural, not just cheaper models
Because the bill is now driven by how an agent is built, the biggest savings come from design choices rather than waiting for the next price cut. Three levers dominate. First, model routing: send each task to the smallest model that can do it well. Classification, extraction and simple chats do not need a frontier reasoning engine, and the price spread between the cheapest capable model and the most expensive one now spans hundreds of times, so routing can cut high-volume simple-task costs by 95% or more. Second, open-source and self-hosted models: open-weight models rose from about 11% of enterprise token volume in early 2025 to roughly 38% by early 2026, as their quality crossed the threshold where they handle a large share of production work at a fraction of API price. Third, context discipline — compressing what you send to the model, caching repeated lookups, and putting a hard budget cap on tool calls per task so a misbehaving loop cannot quietly spend a fortune.
The shift is visible in integration data, too: agent-pattern API calls — sequences with multi-turn reasoning, tool use and iterative refinement — made up 41% of new integration use cases in the first quarter of 2026, up from 18% a year earlier. Agents are no longer a forward-looking thesis; they are the current production reality.
What it means for builders
The practical implication is that the cost ceiling which blocked agentic products in 2024 has effectively dissolved. Analysis drawing on Epoch AI's cost curves suggests a multi-agent workflow that was uneconomic at about one cent per task in early 2024 is on track to cost a fraction of a cent by the end of 2026. The scarce resource is no longer the dollar cost of intelligence — it is latency, evaluation quality and reliability over long sessions. Builders who treat agents as systems to be observed, cached, routed and capped will ship features that were enterprise-only eighteen months ago. Those who "token-max" by defaulting every query to the most expensive frontier model will keep getting surprise bills.
For small teams and indie builders, the window is open in a way it has not been. Start with one narrow, well-bounded workflow. Default to small or cheap models, add routing so only hard tasks reach expensive ones, cache what repeats, and measure cost per completed task rather than cost per token. The model price will keep falling on its own. The architecture is the part you actually control.
