The Agentic Token Explosion

Agents are the most powerful - and most expensive - pattern in modern AI. Learn why agentic workloads burn 10-100× the tokens of simple chat, and the patterns that tame them.

Why Agents Multiply Token Spend

A simple chat call is one round trip. An agent is a loop: the model thinks, calls a tool, reads the result, thinks again - often dozens of times for a single user request. Token costs compound in three dimensions:

1. Iterations. Each loop step is a fresh model call.
2. Context re-reading. On every step, the agent re-sends the system prompt, tool definitions, and the entire conversation so far. A 20-step agent run re-reads early context 20 times.
3. Tool payloads. Tool results (search output, file contents, API responses) enter the context and are then re-read on every subsequent step.

The Five Patterns That Tame Agentic Spend

1. Cache the Static Prefix

The system prompt and tool definitions are identical on every step - exactly what prompt caching is built for. With caching enabled, those 5,000 tokens cost roughly 10% of full price after the first step. This is the single highest-impact fix for agents, and it is covered in depth in Lesson 5 and the Prompt Caching course.

2. Compress Tool Results Before They Enter Context

Do not dump a 10,000-token API response into context when the agent needs 200 tokens of it. Summarize, truncate, or extract the relevant fields before appending the result. Remember: every token a tool returns is re-read on every later step, so tool-result bloat compounds quadratically.

3. Compact Long Runs

When the loop history grows past a threshold, summarize completed steps into a short progress note and drop the raw transcript. Modern agent frameworks support context compaction natively - turn it on and tune the threshold rather than building your own.

4. Use Sub-Agents to Isolate Context

Delegate a research or search task to a sub-agent that reads 50k tokens of material and reports back a 500-token answer. The main agent never carries those 50k tokens forward. This is token optimization at the architecture level - the same job gets done, but the expensive reading happens once in a disposable context instead of being re-read in the main loop forever.

5. Bound the Loop

Set maximum iterations, per-run token budgets, and abort criteria. A stuck agent retrying in a loop is the AI equivalent of an infinite loop in billing.

Right-Sizing Models Inside an Agent

Not every step needs the frontier model. A common production split:

Many teams cut agent costs 40-70% with this split alone - before caching and compaction are even applied.

A Worked Example: Taming One Agent

Take the 300k-token run from the example above and apply the patterns:

• Cache the 5k preamble: ~95k of those tokens drop to one-tenth price.
• Compress tool results from ~1,000 to ~300 tokens each: the quadratic term shrinks by two-thirds.
• Compact at step 10: the second half of the run reads a summary, not the full transcript.

Result: the same task completes on roughly 60-80k effective full-price tokens instead of 300k - a 75%+ reduction with no loss of capability. Stack model right-sizing on top and the cost falls further still.