Jailbreak Detection
Build real-time detection systems that identify jailbreak attempts before they succeed, using ML classifiers, heuristic rules, perplexity analysis, and behavioral anomaly detection.
Detection Architecture
A robust jailbreak detection system operates as a pipeline that screens both inputs and outputs:
User Input ↓ [Stage 1: Pattern Matching] → Known attack signatures ↓ [Stage 2: ML Classifier] → Learned jailbreak patterns ↓ [Stage 3: Perplexity Check] → Unusual token distributions ↓ [Stage 4: Context Analysis] → Multi-turn behavior patterns ↓ LLM Processing ↓ [Stage 5: Output Validation] → Response safety check ↓ User Response
Pattern-Based Detection
The first detection layer uses pattern matching to catch known attack signatures:
import re from typing import List, Tuple class JailbreakPatternDetector: def __init__(self): self.patterns = [ # DAN-style attacks (r"do anything now|DAN mode", "dan_attack", 0.9), (r"pretend you (are|have) no (restrictions|rules)", "persona_override", 0.85), # Role-play exploits (r"act as .* (without|no) (restrictions|limits)", "roleplay_exploit", 0.8), # Authority claims (r"(as|i am) (your|the|an?) (developer|creator|admin)", "authority_claim", 0.7), # Encoding requests (r"(decode|translate) (this )?(base64|rot13)", "encoding_bypass", 0.75), ] def detect(self, text: str) -> List[Tuple[str, float]]: results = [] text_lower = text.lower() for pattern, label, confidence in self.patterns: if re.search(pattern, text_lower): results.append((label, confidence)) return results
ML-Based Classification
Pattern matching catches known attacks. ML classifiers generalize to detect novel jailbreak attempts:
from transformers import pipeline class JailbreakClassifier: def __init__(self, model_name="jailbreak-detector-v1"): self.classifier = pipeline( "text-classification", model=model_name ) self.threshold = 0.7 def is_jailbreak(self, text: str) -> dict: result = self.classifier(text)[0] return { "is_jailbreak": result["label"] == "JAILBREAK" and result["score"] > self.threshold, "confidence": result["score"], "label": result["label"] }
Perplexity-Based Detection
Jailbreak prompts often have unusual statistical properties. Perplexity analysis can flag these anomalies:
- High perplexity: Encoded text, random characters, or unusual token patterns
- Low perplexity with high danger score: Well-crafted but harmful prompts
- Sudden perplexity shifts: Normal conversation that suddenly shifts to jailbreak patterns
Multi-Turn Behavior Analysis
Detecting multi-turn manipulation requires tracking conversation patterns over time:
| Signal | Description | Detection Method |
|---|---|---|
| Topic escalation | Gradually moving from benign to sensitive topics | Topic classifier per turn + trend analysis |
| Boundary testing | Repeated requests near safety boundaries | Count near-refusal responses |
| Context manipulation | Trying to establish false context for later exploitation | Context consistency checks |
| Rapid rephrasing | Same request rephrased many times | Semantic similarity between turns |
Ready to Go Deeper?
Live instructor-led courses from our partners. Affiliate disclosure.
AI & ML Courses - 30% Off
Live instructor-led AI, machine learning, data science, and cloud courses for working professionals. Use code Limited30 at checkout.
EdurekaDataCamp - AI & Data Science
Hands-on Python, machine learning, and AI courses with interactive exercises and real projects.
DataCampedX - Top AI Courses
University-level AI courses from MIT, Harvard, Stanford. Earn certificates that employers recognize.
edX