AI Threat Mitigation Intermediate

Identifying threats is only half the battle - you must also implement effective countermeasures. This lesson covers defense-in-depth strategies for AI systems, from input validation and adversarial training to runtime monitoring and incident response. The goal is to build layered defenses that reduce risk even when individual controls fail.

Defense-in-Depth for AI

A defense-in-depth strategy for AI systems should include multiple layers of protection:

Input-Level Mitigations

Input Validation and Sanitization

• Schema validation - Enforce strict input schemas (data types, ranges, dimensions) before processing
• Statistical anomaly detection - Flag inputs that fall outside the expected data distribution
• Input preprocessing - Apply transformations (JPEG compression, spatial smoothing, feature squeezing) that destroy adversarial perturbations
• Confidence thresholds - Reject predictions below a confidence threshold and route to human review

Adversarial Input Detection

• Detector networks - Train a secondary model to detect adversarial examples
• Input transformation detection - Compare model outputs on original and transformed versions of the input
• Ensemble disagreement - Use multiple models and flag inputs where they disagree

import numpy as np

def validate_input(input_data, expected_shape, value_range):
    """Validate ML model input before inference."""
    # Check shape
    if input_data.shape != expected_shape:
        raise ValueError(f"Invalid shape: {input_data.shape}")

    # Check value range
    if input_data.min() < value_range[0] or input_data.max() > value_range[1]:
        raise ValueError("Input values out of expected range")

    # Check for statistical anomalies
    z_scores = np.abs((input_data - input_data.mean()) / input_data.std())
    if z_scores.max() > 10:
        raise ValueError("Statistical anomaly detected in input")

    return True

Model-Level Mitigations

Adversarial Training

Train the model on both clean and adversarial examples to improve robustness. This is currently the most effective defense against evasion attacks, though it comes at the cost of increased training time and potentially reduced accuracy on clean data.

Differential Privacy

Add calibrated noise during training to provide mathematical guarantees about information leakage. Differential privacy limits what an attacker can learn about individual training examples from the model's outputs.

Model Ensemble Defenses

• Diverse ensembles - Use models with different architectures that are unlikely to share the same vulnerabilities
• Randomized smoothing - Add random noise to inputs and aggregate predictions for provable robustness guarantees
• Defensive distillation - Train a smoother model that is harder to attack with gradient-based methods

Runtime Monitoring and Detection

Continuous Monitoring

• Data drift detection - Monitor input distributions for shifts that may indicate poisoning or adversarial campaigns
• Prediction drift - Track model output distributions and alert on unexpected changes
• Performance monitoring - Compare ongoing accuracy against held-out validation sets
• Query pattern analysis - Detect systematic probing indicative of model extraction attempts

Incident Response

Prepare an AI-specific incident response plan that covers:

1. Detection

   Automated alerts from monitoring systems trigger investigation.

2. Containment

   Ability to quickly roll back to a known-good model version or disable the endpoint.

3. Analysis

   Forensic tools to analyze adversarial inputs, identify the attack vector, and assess impact.

4. Recovery

   Retrain the model with cleaned data, deploy updated defenses, and restore service.

5. Lessons Learned

   Update the threat model, improve monitoring, and document the incident.

Risk Assessment Matrix

Use this matrix to prioritize mitigations based on threat likelihood and impact: