Data quality watch

Bad data is worse than no data, because dashboards keep rendering. A quality agent is the cheapest insurance in a data team, and it is almost entirely rules with a model only writing the explanation.

The job on one screen

Runs when	After each pipeline run, and nightly for everything else.
Reads	Row counts, null rates, value ranges, last updated timestamps, and the expectations for each table.
Decides	Which failures are real, which are expected variation, and which block downstream jobs.
Produces	A pass or fail per check, plus one written explanation of the worst failure.
Stops when	All checks run. It never fixes data.
Tools it needs	Warehouse (read only), scheduler, chat webhook.
Autonomy to start at	Auto, once it has a logged track record. Start lower.

Optional: load a real model

Run it

What it watches, and how far is too far

Tolerances are the whole design. Too tight and everyone mutes it, too loose and it misses the thing you built it for.

Signal	Baseline	Tolerance	First thing to check
Rows loaded	48000	25%	A near empty load usually means an upstream export failed silently rather than a business collapse.
Null customer_id, %	0	1%	Check whether a join key changed type or a new source system started writing to this table.
Hours since last update	6	50%	Freshness failures are usually the scheduler, not the query.
Duplicate primary keys	0	1%	A retried load without idempotency. Look for a job that ran twice last night.

Where this one goes wrong

Alerting on every check

Forty failing checks is one incident. Group by table and report the root table, not the leaves.

Static thresholds on seasonal data

Monday volume is not Sunday volume. Baseline per weekday or accept weekly false alarms.

Fixing data automatically

An agent that deletes duplicate rows to make a check pass destroys the evidence of the real bug.

How you would know it is working

Measure	Why that one
Failures caught before a human noticed	The whole value. Ask analysts whether they found it first.
False failure rate	Above 10% and the channel gets muted, which returns you to zero.
Time to detection	Compare to the old answer, which was usually a Slack message from a confused executive.

Earning more rope

Assist	Run checks and post a daily summary.
Approve	Alert in real time, and mark downstream dashboards as stale automatically.
Auto	Block dependent jobs on critical check failures. Powerful and safe, since blocking is reversible.

Never let it fix the data. Detection and explanation are the job. A pipeline that silently repairs itself hides the bug that will eventually matter.

Related: Anomaly explainer · Pipeline repair triage · Testing agents · all agent jobs · Agent Lab home

Free from AI School - no signup, everything runs in your browser.