Choosing a model
Most agent systems use one model for everything, which means either paying frontier prices for classification or asking a small model to do planning it cannot do. Choosing per step is the single largest quality and cost lever available.
How it works
- List the steps: classify, plan, call tools, write, verify.
- Assign the cheapest model that passes your eval set for each step.
- Use a strong model where the step requires holding several constraints at once, which is usually planning.
- Use a small or local model where the step is classification, extraction, or formatting.
- Re-run the eval set whenever a provider ships a new version, because the right answer changes.
Optional: load a real model
See it work
When it pays, and when it does not
| Use it when | Skip it when |
|---|---|
| Steps differ in difficulty | Every step is equally hard, which is rare and usually means the steps are too coarse |
| Volume is high on the easy steps | Everything is low volume, where engineering time dominates cost |
| You have an eval set | You have no way to measure, in which case pick the strong model and revisit later |
| Latency matters on one step | The whole thing is batch, where cost is the only variable |
How it fails
Benchmarks as evidence
Public benchmarks predict your task poorly. Your twenty real cases predict it well.
Downgrading without measuring
A cheaper model that loops twice costs more and reads worse.
Never revisiting
Model quality per dollar moves every few months, and last year's choice is rarely still right.
What it costs
| Complexity | Several models means several failure modes and several sets of quirks. |
|---|---|
| Evaluation | You need the eval set before this is anything other than guessing. |
| Savings | Often the largest single cost reduction available, and usually with no quality loss. |
Verification should not use a model at all. That one line saves more money than every other choice on this page.
Related: Evaluation harness · Cost control · Verification · all patterns · agent jobs
Related: Evaluation harness · Cost control · Verification · all patterns · agent jobs
Free from AI School - no signup, everything runs in your browser.