Cost control
Agent costs surprise people because a single run is many calls, and one careless design choice multiplies across every one. The good news is that four levers explain almost all of it, and three are free.
How it works
- Measure cost per unit of work: per run, per ticket, per document. Total spend tells you nothing.
- Cut steps: most runs use more steps than they need because the tools are badly described.
- Cut context: the biggest single lever, because every step pays for the whole prompt.
- Cache the stable prefix and repeated tool results.
- Right-size the model per step, using the cheap one for classification and the expensive one for the hard step only.
Optional: load a real model
See it work
When it pays, and when it does not
| Use it when | Skip it when |
|---|---|
| Cost per run is above what the work is worth | The agent runs ten times a day, where engineering time costs more than the tokens |
| Volume is growing faster than value | You are still proving the thing works, where speed of iteration matters more |
| One step dominates the bill | Cost is spread evenly, which usually means context size is the culprit |
| You can measure per unit | You have no attribution at all, in which case build that first |
How it fails
Optimising the wrong step
Teams shave the cheap classification step and leave the expensive one untouched. Measure per step first.
Cheaper model, more steps
A weaker model loops more and can cost more in total. Compare cost per completed run, not per call.
Ignoring the human cost
An agent that saves two cents and costs a person a minute of review is a loss.
What it costs
| Engineering time | The levers are cheap. Attribution is the work. |
|---|---|
| Quality risk | Shorter context and smaller models trade accuracy for cost, so measure both together. |
| Ongoing attention | Costs drift upward with every prompt edit unless someone watches. |
Cost per unit of work, always. Total spend causes panic and produces bad decisions. Cost per resolved ticket produces engineering.
Related: Caching · Budgets and limits · Choosing a model · all patterns · agent jobs
Related: Caching · Budgets and limits · Choosing a model · all patterns · agent jobs
Free from AI School - no signup, everything runs in your browser.