Intermediate

Guardrails and Human-in-the-Loop Gates

The most important design decision in a production AI fleet is not which model to use or which framework to build on. It is where humans stay in the loop - and making sure those loops are enforced, not just requested.

AI School Editorial Team · Lilly Tech Systems Published Jun 25, 2026 · Reviewed Jun 25, 2026

Why Fully Autonomous Agents Are Premature

Fully autonomous operation - agents that take real-world actions without any human checkpoint - is the long-term vision for AI agent systems. It is not the right design for a production system today, and particularly not for one that deploys to a live public website. The reasons are not theoretical:

Agent outputs are probabilistic. Even a well-specified agent produces the wrong output occasionally. Without a review checkpoint, that wrong output goes live. The question is not if this will happen but when, and whether you catch it before or after it affects real users.
Specs drift from reality. The environment the agent operates in changes over time. File structures change, cross-link targets move, quality bars evolve. The spec does not always keep up. Human review catches cases where an agent followed the spec correctly but the spec was outdated.
Consequences are asymmetric. When an agent does a good job, the human gate adds thirty minutes to the timeline. When an agent does a bad job, the human gate prevents a content error from going live on a site that is actively monitored for quality. The expected value of the review is very high relative to its cost.

The PR Gate: The Primary Human Checkpoint

Every agent in our fleet produces a pull request as its output. The PR gate is the primary human-in-the-loop checkpoint: the human reviews the PR diff, the PR description, and any specific concerns the agent flagged, then decides to merge, request changes, or close. The PR has been live in git infrastructure for years and comes with a rich review interface, comment thread, CI integration, and merge history. It is not the most sophisticated gate possible, but it is extremely well-tested and universally understood.

The PR gate provides several properties that are hard to replicate with custom tooling:

Atomic review. The PR diff shows all changes at once. The reviewer can see the full scope of what the agent did in a single view.
Rich annotation. The reviewer can comment on specific lines, request changes, and ask the agent (or a human) to address them before merging.
Rollback point. If a merged PR turns out to have a problem, a revert PR restores the previous state in minutes.
CI integration. CI checks can run automated validation - link checkers, HTML validators, accessibility scanners - before the human sees the PR, reducing the reviewer’s manual burden.

💡

The PR gate is a quality gate, not just a rubber stamp. For our fleet, the reviewer is expected to check: Does the course index match the lesson structure? Do all internal links resolve? Is the writing quality at the gold-standard bar? Are the bylines and dates correct? Does the searchData in main.js match the lesson titles? The PR description from the agent includes a self-check section - the reviewer uses it as a starting point, not a substitute for their own review.

Branch Naming as a Guardrail

Branch naming conventions do more than organize the repository - they act as a first-line safety layer. Our ContentCreation-Draft agent always creates branches named contentcreation/draft-<slug>. This convention serves three functions:

Identity. Anyone looking at the branch list can immediately see which branches are agent-created versus human-created.
Deduplication. When the agent checks for existing in-progress work, it scans for branches matching the pattern contentcreation/draft-*. If a branch already exists for the slug it wants to build, it skips that item. This prevents duplicate builds when the previous PR has not been reviewed yet.
Protection rules. Branch protection rules on the repository target master. The contentcreation/draft-* namespace is allowed to be pushed to, but cannot be merged without going through the PR interface. This is the hard enforcement layer that makes "NEVER push to master" in the spec into a real technical constraint.

Scope Guards in Specs: The Soft Constraint Layer

While tool design (hard constraints) is the primary safety layer, spec constraints (soft constraints) serve as the second layer. Spec constraints make the desired behavior explicit in a way the agent can reason about, even in edge cases that the tool grants do not directly cover.

"NEVER push to master" - Reinforces the branch protection at the agent-reasoning level. Even if branch protection were somehow bypassed (tool misconfiguration, test environment, etc.), the agent knows this is a hard constraint.

"Do NOT modify unrelated files" - Prevents scope creep where the agent makes "helpful" changes to files outside its assignment. In practice, ContentCreation-Draft sometimes notices issues with existing pages and wants to fix them. The spec explicitly says this is not its job.

"Do NOT merge the PR" - Explicitly prohibits the agent from using any merge-adjacent tools that might be technically available. The agent creates the PR; merging is reserved for the human.

"If the backlog is empty, open no PR" - Prevents the agent from inventing work when there is none. Without this constraint, an agent might create a course on a topic not in the backlog, potentially duplicating existing content or wasting review time.

"Skip items marked in-progress or that already have an open PR" - Prevents duplicate work and protects the in-flight state of items another session is handling.

The Lesson from the Committed Draft

One early failure illustrates why scope guards matter. Before the "keep draft content outside the deployed tree" constraint was in the spec, the ContentCreation runbook described writing a content draft as a Markdown file before building the HTML, so the human could review the content before the build phase. On a local session, this worked fine - the human reviewed the draft, gave approval, and the Markdown was deleted before the HTML was committed.

But the cloud agent session had no concept of the local draft workflow. It wrote the Markdown draft to the repository directly, committed it, and pushed it. The Markdown file deployed to the live site as duplicate content - an unfinished draft, publicly visible, which had to be urgently removed. The spec now explicitly prohibits committing draft Markdown files. The constraint was added because the failure happened.

⚠

Every constraint in a production spec should have a failure story behind it. If you are writing a constraint that has never been violated, you are speculating about failure modes. If you are writing a constraint that was added after a real failure, you are encoding hard-won operational knowledge. Both types are valid; the latter are usually more important.

When Can Human Gates Be Relaxed?

As fleet agents mature and accumulate a track record, it becomes reasonable to ask: when is it safe to reduce human oversight? The answer depends on the risk profile of the action and the observed error rate of the agent:

Low-risk, high-track-record actions (e.g., adding a new sitemap entry, updating a counter) can be automated without review after the agent has done them correctly many times.
High-risk actions (e.g., publishing content to the live site, sending messages to real users) should retain human gates even after strong track records, because the cost of a single failure is high enough to justify the review overhead.
Novel actions always require human review, regardless of track record. A well-performing agent in familiar territory can still behave unexpectedly when the context changes.

The practical implication: do not design gates as temporary scaffolding to be removed as soon as the agent is stable. Design them as permanent quality checkpoints whose overhead is justified by the value of the review they provide.