Failure modes
Agent failures repeat across every team and every framework. Learning to name them is most of the debugging, because each one has a known control and reaching for the wrong control wastes a week.
How it works
- Describe what went wrong in terms of the trace, not the output.
- Name the failure mode. There are about eleven that account for nearly everything.
- Apply the control that matches, which is rarely a better prompt.
- Add the case to your eval set so the fix stays fixed.
- Check whether the same control is missing elsewhere, because failures cluster.
Optional: load a real model
See it work
When it pays, and when it does not
| Use it when | Skip it when |
|---|---|
| You have a trace to read | You only have the final output, in which case fix your logging first |
| Failures repeat | It happened once and never again, which is still worth an eval case |
| Someone wants to fix it with a prompt | The fix is genuinely a prompt, which is about a third of cases |
| You are choosing what to build next | Everything works, which means you are not measuring |
How it fails
Prompting around a missing control
Adding please always check to a prompt instead of writing the check. It works until it does not.
Fixing the symptom
Increasing the step limit because it hit the limit, when the real problem is a loop.
No eval case after the fix
The same failure returns in three weeks and nobody remembers the diagnosis.
What it costs
| Time | Naming failures is fast once you know the list. Guessing is what takes weeks. |
|---|---|
| Controls | Most cost almost nothing in tokens, because they are code. |
| Discipline | The expensive part is adding the eval case every time, and it is the part that compounds. |
Most agent failures are missing controls, not weak models. A stronger model raises the failure rate you can tolerate; it never removes the need for the check.
Related: Testing agents · Verification · Budgets and limits · all patterns · agent jobs
Related: Testing agents · Verification · Budgets and limits · all patterns · agent jobs
Free from AI School - no signup, everything runs in your browser.