Retries and idempotency
Agents fail halfway through more often than they fail at the start, which puts them in a state no single call ever reaches. Retries fix transient failures and create duplicates, and which one you get depends entirely on whether the action was idempotent.
How it works
- Classify every tool as read, idempotent write, or non-idempotent write.
- Retry reads freely with backoff.
- Retry idempotent writes with the same idempotency key, so a duplicate is a no-op.
- Never retry a non-idempotent write. Escalate and let a person check the state.
- Log every retry, because a rising retry rate is a dependency telling you something.
Optional: load a real model
See it work
When it pays, and when it does not
| Use it when | Skip it when |
|---|---|
| Calls cross a network | Everything is local and deterministic |
| You can add idempotency keys | The API has no idempotency support, where retries need human confirmation |
| Failures are transient | Failures are deterministic, where retrying wastes time and money |
| Duplicates are detectable | Duplicates are invisible, which is when a retry becomes a silent data problem |
How it fails
Retry storms
Every agent retrying a struggling service in lockstep. Backoff plus jitter, always.
Duplicate sends
The email went out, the response was lost, the retry sent it again. Idempotency keys or nothing.
Retrying the model
A model that produced a bad tool call will often produce the same one. Change something before retrying.
What it costs
| Latency | Backoff means waiting, which users feel. Cap total wait, not just attempts. |
|---|---|
| Tokens | Model retries cost a full call each. Retry the tool, not the reasoning, where you can. |
| Engineering | Idempotency keys are a day of work and prevent an entire class of incident. |
Classify your tools before you write any retry code. Read, idempotent write, dangerous write. That table is the retry policy, and everything else follows from it.
Related: Budgets and limits · Guardrails · Pipeline repair · all patterns · agent jobs
Related: Budgets and limits · Guardrails · Pipeline repair · all patterns · agent jobs
Free from AI School - no signup, everything runs in your browser.