Tracing and logging

An agent that fails once in fifty runs is impossible to debug from a screenshot. Traces are not optional infrastructure here, they are the only way anyone ever finds out what happened.

How it works

Log every step: the prompt, the tool call, the arguments, the raw result, the timing, and the token count.
Give every run an id, and put that id in front of the user when something goes wrong.
Store the model and prompt version with the run, or you cannot compare last week to this week.
Sample and read traces weekly, by hand. Dashboards show you what you already suspected.
Redact before storing. Traces contain everything the agent saw, which is a compliance surface.

Optional: load a real model

See it work

When it pays, and when it does not

Use it when	Skip it when
The agent runs unattended	You watch every run yourself, which is only true in week one
Failures are intermittent	Every failure is reproducible on demand
Several people change the prompts	One person owns everything and remembers every change
You need to explain a decision later	Nothing the agent does has consequences worth explaining

How it fails

Logging the output only

The output tells you what went wrong and never why. Log the steps.

No version stamps

Comparing this week's failure rate against last week is meaningless if the prompt changed and nobody recorded it.

Traces as a data breach

Full traces contain customer data. Redact at write time, not at read time.

What it costs

Storage	Traces are large. Sample the successful ones, keep every failure.
Engineering	A day to instrument, and it pays back the first time something breaks in production.
Privacy	Real. Traces are the most sensitive artifact an agent produces.

Read traces by hand every week. Every team that does this finds a failure mode their dashboard was not designed to show.

Related: Observability · Tracing · Logging · Metrics · Monitoring agents · all patterns · agent jobs

Free from AI School - no signup, everything runs in your browser.