Testing Frameworks for Agents

Every agent testing framework is a way of writing down the same handful of assertions. Build one here by ticking boxes, run it against a real agent in this tab, and then read it back as pytest, promptfoo, DeepEval and vitest. Once you can see that it is the same test each time, choosing a framework stops being difficult.

What a testing framework actually gives you

You can test an agent with nothing but a for loop and a list of assertions, and plenty of good teams do exactly that. A framework earns its place by supplying five things you would otherwise write yourself.

1. A place to declare cases

Goal, tools, expected behaviour, in a file anyone can add to without touching the runner.

2. A runner that repeats

Agents are not deterministic, so every case runs several times and the result is a pass rate, not a boolean.

3. Assertions that fit agents

Which tool ran, with what arguments, how many steps, what the answer contained. Not string equality.

4. A judge for prose

Where exact matching cannot work, a model scores the answer against a rubric, and the rubric is version controlled.

5. A report and a gate

A number CI can fail on, and a diff against the last run so you can see what your change actually did.

What none of them give you

The cases. Your golden set comes from your own traffic and your own incidents, and it is the part that matters most.

Optional: load a real model

Live: build a test with clicks, then run it

Pick a case, tick the assertions you want, and run it against a real agent in this tab. Every assertion below is one you would genuinely write at work, and the code panel underneath rewrites itself as you tick.

Case Repeat times

Results appear here, one row per assertion.

The trace of the last run appears here. Assertions are made about this, not about the wording of the answer.

The same test, written in four frameworks

This is the whole lesson of this page: the frameworks differ in syntax and reporting, not in what a good agent test asserts. Tick a different assertion above and every tab below changes.

The landscape, and what each thing is for

Tool	Shape	Reach for it when
pytest or vitest, plus your own harness	Ordinary test files calling your agent behind a thin wrapper that records the trace.	Always start here. It runs in the CI you already have, and most teams never need more.
promptfoo	Cases in YAML, assertions declarative, side by side comparison of prompts and models.	You are comparing two prompts or two models over the same fifty cases and want a table.
DeepEval	pytest-native, with metrics for tool correctness, task completion and hallucination built in.	You want agent-specific metrics without writing the scoring code yourself.
LangSmith, Langfuse, Braintrust	Hosted tracing plus datasets and evals over the traces you already collect.	Production traces are the thing you want to turn into your test set. Note the vendor lock on the trace format.
Ragas	Retrieval-specific scores: faithfulness, answer relevance, context precision.	The agent's main failure mode is retrieval, not tool use.
Inspect, OpenAI Evals	Structured eval suites aimed at model capability rather than one application.	You are evaluating models to choose one, not testing your own system.

Three things to try, in this order

1. Run the same case twice

With no model loaded the result is identical every time, because a keyword router is standing in. Load a model and run it again: some cases will pass three times out of three, and some will not. The pass rate is the real result.

2. Turn on "never calls the forbidden tool"

This is the assertion that quietly becomes a security control. A read-only agent that calls a write tool once in a hundred runs is a production incident, and only a repeated test finds it.

3. Turn off every trace assertion and keep only "produced an answer"

Everything passes. That is what a green suite looks like when it is testing nothing, and it is the most common state of a real agent test suite.

Picking one without wasting a month

Does it run in your existing CI without a hosted account	If not, your suite will stop running the first week someone is busy.
Can it assert on the trace, not just the output	Output-only assertions cannot catch the wrong tool, the extra step, or the forbidden call.
Does it repeat and report a rate	A framework that reports the last run as pass or fail is hiding the flake you most need to see.
Can you export your cases	Your golden set outlives every framework you will use. Keep it in your own repo, in a plain format.
What does it cost per run	Model-graded assertions cost a model call each. A hundred cases, three repeats, four judged assertions is 1200 calls per commit.

Start with twenty cases in a plain file. Ten from real traffic, five from incidents that already happened, five that should be refused. Run them three times each on every change. That suite, in whatever framework, beats a sophisticated setup that nobody maintains.

Related: Testing agents · Evaluation harness · Monitoring agents · Guardrails · all patterns

Free from AI School - no signup, everything runs in your browser.