Token prices for large language models are dropping, but the overall cost of running AI workloads is climbing quickly.
Cheaper tokens, more expensive agents
Research from Gartner shows token costs will fall by 95% by 2030. Meanwhile, inference costs for agentic workflows will rise more than fivefold in the next two years. The increase stems from AI applications using more, and often more expensive, tokens as models become more complex. This is what Gartner calls the “inference paradox.” Buyers dangerously assume that as AI providers improve token economics, these savings will be reflected in their roadmaps—but they will not. The market is captured by a “token-deflation illusion.”
The move from simple chatbots to autonomous agents is fueling the rise. A chatbot reads and interprets a request and delivers a “probabilistically reasonable” answer. Agents, however, must reason, question, and adapt when something goes wrong, often running continuously and invisibly in the background. Some coordinate with others in what Gartner describes as “swarms.”
That coordination carries a price. Advanced agents can cost up to 150 times more on a single task than basic chatbots. Training a medium-sized agentic model with advanced reasoning capabilities costs 2.5 times more than training a similarly sized chatbot. Inference costs are five times greater, and agents require 5 to 30 times more tokens to handle equivalent tasks.
The math behind the surge
Gartner built a Tokenomics Model to analyze the impacts of agentic systems. The firm tested 12 types of AI models with various capabilities. Basic workflows cost around $0.05 per inference token. Summarization and knowledge retrieval cost roughly $0.10. More complex workflows cost around $0.30, while planning and learning reach roughly $0.40 per token—an 8x to 10x increase over basic workflows.
Related: IT Executives Struggle in Silence
Running hundreds of agents, each performing dozens or hundreds of tasks hourly, drives up costs. Agents break problems into smaller tasks, call higher-order models, and require multimodal data. The volume of compute required is substantial.
Costs will inevitably escalate, and there is no guarantee that value will grow commensurately. Enterprises will need ways to measure ROI and improvement across completed workflows. Orchestration will be the differentiator.
Agentic systems demand efficient management. A poorly orchestrated swarm can incur unnecessary calls to expensive models. Gartner’s research highlights that the computing power required for these capabilities is “mind-bending.”
Returns are possible but won’t come easily. Companies must rethink workflows and invest in tools to measure value per outcome. Letting agentic systems run without oversight could lead to high bills with little benefit.
As agents handle more tasks, their responses raise new questions about reliability. Untraceable answers can create risks when decisions depend on them.
