Advanced

Output Sanitization Best Practices

Deploy production-grade output sanitization using guardrails frameworks, layered filtering architectures, continuous monitoring, and incident response procedures.

Guardrails Frameworks

Purpose-built frameworks simplify deploying output sanitization at scale:

NVIDIA NeMo Guardrails

Colang - NeMo Guardrails Configuration

# config.yml - Define guardrail rails
models:
  - type: main
    engine: openai
    model: gpt-4

rails:
  output:
    flows:
      - check output safety
      - check pii leakage
      - check factual accuracy

# rails.co - Define output checking flows
define flow check output safety
  $is_safe = execute check_safety(output=$bot_message)
  if not $is_safe
    bot refuse to respond
    "I cannot provide that information. Let me help
    you with something else."

Guardrails AI

Python - Guardrails AI Validation

from guardrails import Guard
from guardrails.hub import ToxicLanguage, DetectPII, NSFWText

# Create a guard with multiple validators
guard = Guard().use_many(
    ToxicLanguage(threshold=0.5, on_fail="fix"),
    DetectPII(pii_entities=["EMAIL", "PHONE", "SSN"], on_fail="fix"),
    NSFWText(threshold=0.8, on_fail="refrain"),
)

# Validate LLM output
result = guard.validate(llm_output)
if result.validation_passed:
    return result.validated_output
else:
    return "I'm sorry, I cannot provide that response."

Production Architecture

Production Sanitization Architecture

User Request
    ↓
[Input Sanitization]  → Prompt injection detection
    ↓
LLM Processing
    ↓
[Layer 1: Sync Filters]     → PII regex, blocklist (<5ms)
    ↓
[Layer 2: Local ML]         → Toxicity, topic classification (10-50ms)
    ↓
[Layer 3: Code Scanner]     → Vulnerability patterns (if code) (10ms)
    ↓
[Layer 4: Policy Engine]    → Business rules, compliance checks (5ms)
    ↓
Sanitized Response → User
    ↓ (async)
[Layer 5: Deep Analysis]    → Cloud moderation API, logging, analytics

Monitoring and Metrics

Metric	Target	Alert Threshold
Block rate	1-5% of outputs	>10% (possible model issue)
False positive rate	<0.1%	>0.5% (filter too aggressive)
PII detection rate	Track trend	Sudden spike (possible memorization)
Sanitization latency	<100ms p95	>200ms (performance regression)
Human escalation rate	<0.01%	>0.1% (filters need tuning)

Incident Response

Immediate: Block the specific output pattern, notify the on-call team
Short-term: Add the pattern to blocklists, retrain classifiers if needed
Medium-term: Root cause analysis - was this a model failure, filter gap, or new attack?
Long-term: Update policies, improve training data for classifiers, share learnings

Frequently Asked Questions

A well-optimized pipeline adds 20-100ms. Regex and blocklist checks take under 5ms. Local ML models add 10-50ms. Cloud API calls (if used synchronously) add 100-300ms but can be run asynchronously. Since LLM generation itself takes 500ms-5s, the sanitization overhead is a small fraction of total response time.

Yes, but it requires a different approach. Buffer tokens until you have enough context for meaningful analysis (typically a sentence or paragraph). Run fast filters on each buffer, and run deeper analysis on the accumulated output. Some systems buffer the entire response before delivery for maximum safety, trading real-time streaming for security.

NeMo Guardrails is best for conversational AI with complex dialog flows and topical control. Guardrails AI excels at structured output validation with its Hub ecosystem of validators. For simple use cases, you may not need a framework at all - a custom pipeline with regex, an ML classifier, and a moderation API can be sufficient and easier to debug.