Intermediate

Guardrails & Safety Pattern

Guardrails are protective layers that wrap around AI models to ensure inputs are safe, outputs are appropriate, and the system operates within defined boundaries. They are the safety nets that make AI systems production-ready.

The Three Layers of Guardrails

A robust guardrails architecture operates at three distinct layers, each catching different categories of problems:

Input guardrails: Validate and sanitize user inputs before they reach the model
Output guardrails: Check and filter model outputs before they reach the user
Structural guardrails: System-level controls on resource usage, cost, and access

💡

Defense in depth: Like security, guardrails work best when layered. No single guardrail catches everything. A prompt injection might bypass the input filter but get caught by the output validator. Always implement multiple layers.

Input Guardrails

Input guardrails protect the model from malicious, inappropriate, or problematic inputs:

PII Detection and Redaction

Personally Identifiable Information (PII) - names, email addresses, phone numbers, social security numbers, credit card numbers - should be detected and redacted before being sent to the model, especially when using third-party APIs:

Pattern matching: Regex-based detection for structured PII (emails, phone numbers, SSNs, credit cards)
Named Entity Recognition (NER): ML-based detection for unstructured PII (names, addresses, medical conditions)
Redaction strategies: Replace with placeholder tokens ([EMAIL], [PHONE]), hash, or remove entirely
Reversible redaction: For workflows where the original PII needs to be restored in the output

Prompt Injection Defense

Prompt injection attacks attempt to override the system prompt by embedding instructions in user input. Defense strategies include:

Input sanitization: Strip or escape known injection patterns ("ignore previous instructions", "you are now", role-play attempts)
Delimiter-based isolation: Wrap user input in clear delimiters that the model can distinguish from system instructions
Classifier-based detection: Train a lightweight classifier to flag inputs that look like injection attempts
Dual-LLM approach: Use a separate, smaller model to screen inputs before passing them to the main model

Content Filtering

Screen inputs for inappropriate content categories: hate speech, violence, sexual content, self-harm, and other policy violations. Use classification models or API-based services like OpenAI's Moderation API or Perspective API.

Output Guardrails

Output guardrails ensure the model's responses meet quality and safety standards:

Hallucination Detection

Source grounding: Compare the model's claims against the source documents provided via RAG. Flag statements not supported by the sources.
Self-consistency checking: Ask the model to answer the same question multiple times and flag inconsistencies
Confidence scoring: Use the model's token probabilities to identify low-confidence passages that may be hallucinated
Fact-checking pipeline: Route factual claims through a verification step using search or knowledge bases

Toxicity Filtering

Even well-prompted models can occasionally generate toxic, biased, or inappropriate output. Post-processing filters catch these cases:

Toxicity classifiers: Run outputs through models like Perspective API, Detoxify, or custom toxicity classifiers
Bias detection: Check for demographic biases in recommendations, descriptions, or decisions
Tone analysis: Verify the output matches the expected tone (professional, friendly, neutral)

Format Validation

Ensure the model's output conforms to the expected structure, especially for API responses:

JSON schema validation: Validate structured outputs against a defined schema
Length constraints: Enforce minimum and maximum response lengths
Required field checks: Verify all expected fields are present in structured responses
Type checking: Ensure numbers are numbers, dates are valid dates, and so on

Input/Output Guardrails Pipeline

Here is a complete implementation of a guardrails pipeline that wraps any LLM call with input and output validation:

Python - Guardrails Pipeline

import re
from dataclasses import dataclass
from typing import Callable, Optional
from enum import Enum


class GuardrailAction(Enum):
    ALLOW = "allow"
    BLOCK = "block"
    MODIFY = "modify"
    FLAG = "flag"


@dataclass
class GuardrailResult:
    action: GuardrailAction
    original: str
    modified: Optional[str] = None
    violations: list[str] = None
    metadata: dict = None

    def __post_init__(self):
        self.violations = self.violations or []
        self.metadata = self.metadata or {}


class GuardrailsPipeline:
    """Composable guardrails pipeline for input and output."""

    def __init__(self):
        self.input_guards: list[Callable] = []
        self.output_guards: list[Callable] = []

    def add_input_guard(self, guard: Callable):
        self.input_guards.append(guard)
        return self

    def add_output_guard(self, guard: Callable):
        self.output_guards.append(guard)
        return self

    def check_input(self, text: str) -> GuardrailResult:
        """Run all input guardrails sequentially."""
        current = text
        all_violations = []

        for guard in self.input_guards:
            result = guard(current)
            if result.action == GuardrailAction.BLOCK:
                return result
            if result.action == GuardrailAction.MODIFY:
                current = result.modified
            all_violations.extend(result.violations)

        return GuardrailResult(
            action=GuardrailAction.ALLOW,
            original=text,
            modified=current if current != text else None,
            violations=all_violations
        )

    def check_output(self, text: str) -> GuardrailResult:
        """Run all output guardrails sequentially."""
        current = text
        all_violations = []

        for guard in self.output_guards:
            result = guard(current)
            if result.action == GuardrailAction.BLOCK:
                return result
            if result.action == GuardrailAction.MODIFY:
                current = result.modified
            all_violations.extend(result.violations)

        return GuardrailResult(
            action=GuardrailAction.ALLOW,
            original=text,
            modified=current if current != text else None,
            violations=all_violations
        )

    async def guarded_call(self, user_input: str,
                           llm_fn: Callable) -> dict:
        """Full guarded LLM call with input + output checks."""
        # Check input
        input_result = self.check_input(user_input)
        if input_result.action == GuardrailAction.BLOCK:
            return {
                "blocked": True,
                "reason": input_result.violations,
                "stage": "input"
            }

        # Use modified input if guardrails changed it
        clean_input = input_result.modified or user_input

        # Call the LLM
        llm_output = await llm_fn(clean_input)

        # Check output
        output_result = self.check_output(llm_output)
        if output_result.action == GuardrailAction.BLOCK:
            return {
                "blocked": True,
                "reason": output_result.violations,
                "stage": "output"
            }

        return {
            "blocked": False,
            "output": output_result.modified or llm_output,
            "input_violations": input_result.violations,
            "output_violations": output_result.violations
        }

PII Detection and Redaction

This implementation detects and redacts common PII patterns, providing reversible redaction so original values can be restored when needed:

Python - PII Detection and Redaction

import re
import hashlib
from typing import Dict, Tuple


class PIIDetector:
    """Detect and redact Personally Identifiable Information."""

    PII_PATTERNS = {
        "email": r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b',
        "phone_us": r'\b(?:\+1[-.\s]?)?\(?\d{3}\)?[-.\s]?\d{3}[-.\s]?\d{4}\b',
        "ssn": r'\b\d{3}-\d{2}-\d{4}\b',
        "credit_card": r'\b(?:\d{4}[-\s]?){3}\d{4}\b',
        "ip_address": r'\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b',
        "date_of_birth": r'\b(?:0[1-9]|1[0-2])/(?:0[1-9]|[12]\d|3[01])/(?:19|20)\d{2}\b',
    }

    def __init__(self):
        self.redaction_map: Dict[str, str] = {}

    def detect(self, text: str) -> list[dict]:
        """Find all PII instances in text."""
        findings = []
        for pii_type, pattern in self.PII_PATTERNS.items():
            for match in re.finditer(pattern, text):
                findings.append({
                    "type": pii_type,
                    "value": match.group(),
                    "start": match.start(),
                    "end": match.end()
                })
        return findings

    def redact(self, text: str,
               reversible: bool = False) -> Tuple[str, dict]:
        """Redact PII from text. Optionally reversible."""
        findings = self.detect(text)
        redacted = text
        mapping = {}

        # Process in reverse to preserve positions
        for finding in sorted(findings,
                              key=lambda x: x["start"],
                              reverse=True):
            placeholder = f"[{finding['type'].upper()}]"

            if reversible:
                token = hashlib.md5(
                    finding["value"].encode()
                ).hexdigest()[:8]
                placeholder = f"[{finding['type'].upper()}_{token}]"
                mapping[placeholder] = finding["value"]

            redacted = (redacted[:finding["start"]]
                       + placeholder
                       + redacted[finding["end"]:])

        return redacted, mapping

    def restore(self, text: str, mapping: dict) -> str:
        """Restore redacted PII using the mapping."""
        restored = text
        for placeholder, original in mapping.items():
            restored = restored.replace(placeholder, original)
        return restored


# Usage as a guardrail
def pii_guard(text: str) -> GuardrailResult:
    detector = PIIDetector()
    findings = detector.detect(text)

    if not findings:
        return GuardrailResult(
            action=GuardrailAction.ALLOW,
            original=text
        )

    redacted, mapping = detector.redact(text, reversible=True)
    return GuardrailResult(
        action=GuardrailAction.MODIFY,
        original=text,
        modified=redacted,
        violations=[f"PII detected: {f['type']}" for f in findings],
        metadata={"pii_mapping": mapping}
    )

Prompt Injection Detection

This guardrail uses pattern matching and heuristics to detect common prompt injection techniques:

Python - Prompt Injection Detector

class PromptInjectionDetector:
    """Detect prompt injection attempts."""

    INJECTION_PATTERNS = [
        r"ignore (?:all )?(?:previous|above|prior) instructions",
        r"you are now",
        r"forget (?:everything|all|your)",
        r"disregard (?:all|your|the)",
        r"new (?:instructions|rules|persona)",
        r"act as (?:if you are|a)",
        r"pretend (?:you are|to be)",
        r"override (?:your|the|system)",
        r"system prompt",
        r"jailbreak",
        r"\[system\]",
        r"<system>",
    ]

    ROLE_PLAY_PATTERNS = [
        r"you are (?:DAN|evil|uncensored)",
        r"developer mode",
        r"do anything now",
    ]

    def __init__(self, sensitivity: float = 0.7):
        self.sensitivity = sensitivity

    def check(self, text: str) -> dict:
        """Check text for injection patterns."""
        text_lower = text.lower()
        matches = []
        score = 0.0

        for pattern in self.INJECTION_PATTERNS:
            if re.search(pattern, text_lower):
                matches.append(pattern)
                score += 0.3

        for pattern in self.ROLE_PLAY_PATTERNS:
            if re.search(pattern, text_lower):
                matches.append(pattern)
                score += 0.5

        # Heuristic: excessive use of special characters
        special_ratio = sum(
            1 for c in text if c in '{}[]<>|\\/'
        ) / max(len(text), 1)
        if special_ratio > 0.1:
            matches.append("high_special_char_ratio")
            score += 0.2

        return {
            "is_injection": score >= self.sensitivity,
            "score": min(score, 1.0),
            "matched_patterns": matches
        }


def injection_guard(text: str) -> GuardrailResult:
    """Guardrail function for prompt injection detection."""
    detector = PromptInjectionDetector()
    result = detector.check(text)

    if result["is_injection"]:
        return GuardrailResult(
            action=GuardrailAction.BLOCK,
            original=text,
            violations=[
                f"Prompt injection detected "
                f"(score: {result['score']:.2f})"
            ],
            metadata=result
        )

    return GuardrailResult(
        action=GuardrailAction.ALLOW,
        original=text
    )

Structural Guardrails

Structural guardrails operate at the system level to control resource usage and prevent abuse:

Guardrail Type	What It Controls	Implementation
Token limits	Maximum input/output token count per request	Count tokens before API call; truncate or reject if over limit
Rate limiting	Requests per user per time window	Token bucket or sliding window algorithm
Cost caps	Maximum spend per user, per day, or per month	Track token usage and cost per request; disable when budget exceeded
Model access control	Which users can access which models	Role-based access control (RBAC) with model tiers
Session limits	Maximum conversation length or duration	Count turns or elapsed time; prompt user to start new session
Concurrent request limits	Maximum simultaneous requests per user	Semaphore or queue-based throttling

Guardrails Frameworks

Several open-source and commercial frameworks provide pre-built guardrails:

Guardrails AI

An open-source Python framework that validates LLM outputs against structured specifications. It uses RAIL (Reliable AI Language) specs to define expected output formats and constraints, automatically re-prompting the model when outputs fail validation.

NeMo Guardrails (NVIDIA)

A toolkit for adding programmable guardrails to LLM-based conversational systems. It uses Colang, a modeling language for defining conversation flows, topic boundaries, and safety rules. NeMo Guardrails can prevent off-topic conversations, enforce factual accuracy, and block harmful outputs.

Constitutional AI (Anthropic)

An approach where the model is trained to follow a set of principles (a "constitution") that guide its behavior. The model learns to self-critique and revise its outputs to align with the constitution, reducing the need for external guardrails.

Comparison of Guardrails Frameworks

Framework	Type	Strengths	Best For
Guardrails AI	Output validation	Schema enforcement, auto-retry, composable validators	Structured output, API responses
NeMo Guardrails	Conversation control	Topic steering, flow control, Colang DSL	Chatbots, virtual assistants
LLM Guard	Input/output scanning	PII detection, toxicity, injection detection	API endpoints, general safety
Rebuff	Prompt injection	Multi-layer injection detection, canary tokens	Security-focused applications
Custom pipeline	Flexible	Full control, domain-specific rules	Unique requirements, regulated industries

Layered Defense Strategy

The most robust guardrails implementations use a layered approach where each layer catches what the others miss:

Layer 1 - Pre-processing: Input sanitization, PII redaction, content classification, and injection detection run before the request reaches the model
Layer 2 - System prompt: Clear instructions in the system prompt about what the model should and should not do, including boundary definitions and refusal protocols
Layer 3 - Model-level: Constitutional AI training, RLHF safety alignment, and model-specific safety features built into the model itself
Layer 4 - Post-processing: Output validation, toxicity filtering, hallucination checks, and format verification before the response reaches the user
Layer 5 - Monitoring: Continuous logging, anomaly detection, and alerting to catch novel attack patterns and safety issues in production

⚠️

No guardrail is perfect: Even with all five layers, determined adversaries may find ways through. Guardrails reduce risk; they do not eliminate it. Pair guardrails with monitoring, incident response procedures, and regular red-teaming exercises to continuously improve your defenses.

Monitoring and Alerting for Safety Violations

Production guardrails systems need robust monitoring to detect emerging threats and measure effectiveness:

Violation dashboards: Real-time visibility into blocked inputs, flagged outputs, and guardrail trigger rates
Anomaly detection: Alert when violation rates spike suddenly (may indicate a coordinated attack) or drop unexpectedly (may indicate a guardrail failure)
False positive tracking: Monitor cases where guardrails incorrectly block legitimate inputs, as overly aggressive guardrails degrade user experience
Attack pattern analysis: Cluster and analyze blocked injection attempts to identify new attack vectors and update detection rules
Regular red-teaming: Schedule periodic adversarial testing where security researchers attempt to bypass your guardrails, then use findings to strengthen them

✅

Log everything: Every guardrail trigger should be logged with the full input, the violation type, and the action taken. This data is invaluable for improving your guardrails over time and for incident investigation. Make sure logs are stored securely, especially when they contain PII or attack payloads.

← Previous Human-in-the-Loop Next → Cache & Optimization

Ready to Go Deeper?

Live instructor-led courses from our partners. Affiliate disclosure.

🧠

AI & ML Courses - 30% Off

Live instructor-led AI, machine learning, data science, and cloud courses for working professionals. Use code Limited30 at checkout.

Edureka

📊

DataCamp - AI & Data Science

Hands-on Python, machine learning, and AI courses with interactive exercises and real projects.

DataCamp

🎓

edX - Top AI Courses

University-level AI courses from MIT, Harvard, Stanford. Earn certificates that employers recognize.

edX