Incident Response for LLM Systems

Traditional incident response was built for systems with clear failure signals. LLM incidents often have neither - requiring a different playbook designed for probabilistic, quality-based degradation.

Why LLM Incidents Need Special Playbooks

When a database goes down, you know it instantly: the error rate spikes to 100%, the system is clearly broken, and the path to resolution is well-understood. LLM incidents rarely work this way. The most common LLM incidents are quality degradation events: the system continues to operate, serves responses, and looks healthy on every infrastructure metric - but the quality of those responses has degraded below an acceptable threshold.

This creates three unique challenges for incident response:

Ambiguity of impact. How bad is it? With a database outage, impact is binary. With LLM quality degradation, you need to estimate what fraction of responses are affected, how severely, and for which user segments. This estimation is harder, takes longer, and is more uncertain.

No obvious fix. Restarting a service fixes a crashed process. For an LLM quality incident, the "fix" might be rolling back to a previous prompt version, reverting a model change, adjusting a temperature parameter, or waiting for a vendor to revert a model update. The correct action depends on root cause, which takes time to determine.

Gradual onset. LLM incidents often have a slow onset: quality degrades by 5% per week after a model update, and by the time anyone notices, the cause was 30 days ago. This makes timeline reconstruction harder and post-mortems more complex.

Severity Tier Definitions for LLM Incidents

Standard P0-P3 severity tiers need LLM-specific definitions because the usual "is the system up?" criterion doesn’t capture quality failures.

The LLM Incident Response Playbook

The following six-step playbook applies to P0 and P1 incidents. For P2, steps 1-3 and 5-6 apply; containment options are more limited. For P3, a simplified version of steps 1 and 6 is sufficient.

Step 1: Detect. The incident begins when a monitoring alert fires, a user report comes in, or a team member notices anomalous behavior. The first responder’s job in the first 5 minutes is to confirm the incident is real (not a fluke), estimate scope (how many users, how many requests, since when), and open an incident channel. Check the golden signals dashboard immediately: latency, error rate, token usage, format compliance, and quality score. Determine which signals are anomalous and by how much.

Step 2: Assess Severity. Using the severity tier definitions above, assign a severity level. This determines who gets paged, how quickly action is required, and what level of communication is needed. When in doubt, assign the higher severity and downgrade once you have more information - upgrading severity mid-incident wastes time.

Step 3: Contain. Containment for LLM incidents means limiting further exposure to the degraded system. Options, in order of speed: (a) Enable the degraded-mode fallback that returns a static or simplified response instead of calling the LLM. (b) Roll back to the previous prompt version if a recent prompt change is suspected. (c) Switch to the fallback model or provider if the vendor API is the issue. (d) Reduce traffic to the affected path via feature flag while investigation continues. The goal is to stop the bleeding, not to fix the root cause - that comes in step 5.

Step 4: Communicate. Internal communication: post an update in the incident channel every 15-30 minutes during a P0/P1. Assign an incident commander who owns the timeline and coordinates between the investigating engineer and stakeholders. External communication: for user-facing incidents, follow your organization’s standard communication protocols. LLM quality incidents often do not require public status page updates (unlike outages), but affected user segments may need direct communication if specific wrong outputs were delivered.

Step 5: Remediate. Remediation is the root-cause fix. Common LLM remediations: revert the prompt to the last known-good version; roll back to the previous model configuration; apply a temporary patch to the prompt to handle the specific failure pattern; contact the vendor about a model update that changed behavior. Validate the fix before lifting containment: send 100 test requests through the fixed system and confirm quality signals are back within SLO.

Step 6: Post-Mortem. Every P0 and P1 incident requires a post-mortem within 5 business days. Use the LLM-specific template below.

The LLM Incident Post-Mortem Template

Five Common Mistakes in LLM Incident Response

1. Waiting for user complaints. If your detection depends on users telling you something is wrong, you are days behind the actual incident start. Build quality monitoring that detects degradation before users notice.
2. Treating it like a code incident. The first instinct is to roll back the last code deploy. But the cause of LLM quality incidents is usually a prompt change, a model update, or an input distribution shift - not application code. Check the prompt version and model configuration first.
3. Skipping containment and going straight to root cause. Root cause analysis takes time. During that time, users continue to receive degraded responses. Contain first (enable fallback, roll back prompt), then investigate.
4. Declaring the incident closed before validating the fix. The fix feels obvious, you apply it, and you close the incident. But you didn’t run test traffic through the fixed system to confirm quality signals recovered. The incident reopens 30 minutes later.
5. Writing post-mortems that identify causes but don’t create action items. A post-mortem that ends with "we should improve our monitoring" has accomplished nothing. Every post-mortem must end with exactly three assigned, time-bound action items that close a specific gap revealed by the incident.