Intermediate

The Production Patterns Checklist

Everything in this course, compressed into a pre-deploy checklist, a maturity model, a decision tree, and ten rules to keep on your wall.

The 25-Point Pre-Deploy Checklist

Run this checklist before shipping any prompt to production. Items marked (Critical) are blocking; the others are high-value but context-dependent.

Design (check before building)

• ☐ The task has a clear, written success definition. You know what “correct output” means.
• ☐ The output contract is defined: format, schema, length bounds.
• ☐ The failure modes are anticipated: what are the 3 most likely ways this breaks?
• ☐ The scope boundary is explicit: what questions this answers and what it does not.
• ☐ The input distribution is understood: what does the realistic range of inputs look like?

System Prompt (Critical)

• ☐ (Critical) System prompt has all four layers: Identity, Capabilities, Constraints, Format.
• ☐ (Critical) All behavioral constraints are explicit, not implied by context.
• ☐ Critical constraints are stated at both the beginning and end of the system prompt.
• ☐ System prompt has been tested for internal conflicts: no two instructions logically contradict.
• ☐ Scope boundary is enforced with both positive (what to do) and negative (what not to do) instructions.

Output Format (Critical)

• ☐ (Critical) Output format is enforced at the API level (tool use / JSON mode) wherever possible, not just in the prompt.
• ☐ (Critical) Server-side schema validation is in place before the output is used downstream.
• ☐ (Critical) max_tokens is set as a hard limit; the value is tested to avoid mid-sentence cutoffs.
• ☐ “No preamble” instruction is included where preamble would break parsing.
• ☐ Parser handles the five failure modes: preamble, markdown fencing, schema drift, string escaping, truncation.

Examples and Coverage

• ☐ If few-shot examples are used, they cover at least 3 distinct input archetypes.
• ☐ At least one example demonstrates correct handling of an edge case.
• ☐ Examples are stored in a versioned file, not hardcoded in the prompt string.
• ☐ Prompt has been red-teamed: tested with at least 10 adversarial or unexpected inputs.

Testing and Monitoring (Critical)

• ☐ (Critical) A regression test set exists with at least 20 labeled (input, expected_output) pairs.
• ☐ (Critical) The test set has been run against the prompt and a passing rate is established as the baseline.
• ☐ A process exists to detect production degradation (logging + sampling, or automated monitoring).
• ☐ A rollback plan exists: previous version is accessible and deployable in under 10 minutes.

Lifecycle

• ☐ The prompt is stored in version control with a CHANGELOG.
• ☐ The model version is documented; impact of a model update has been considered.
• ☐ Owner and review process are documented: who can change this prompt, and how?

The Prompt Maturity Model

Where does your prompt sit on the maturity curve? Each level describes a different set of practices in place.

Level	Name	What’s In Place	Typical Failure Mode
1	Ad-hoc	Prompt written and deployed; no tests, no versioning	Silent degradation discovered via user complaints
2	Structured	Four-layer system prompt; explicit output constraints; server-side validation	Format surprises are caught; behavior regressions are not
3	Tested	Level 2 + regression test set; changelog; red-teaming before changes	Regressions caught before production; distribution drift still a blind spot
4	Hardened	Level 3 + CI gate; production monitoring; model-version pinning; rollback process	Resilient to model updates, distribution shift, and adversarial use

Most teams ship Level 1 prompts in production. Level 2 is the minimum viable production standard for any prompt that affects user experience. Level 3 is the target for prompts in critical workflows. Level 4 is appropriate for high-stakes applications where failure has significant business or user impact.

Is This Prompt Production-Ready? Decision Tree

Ten Anti-Patterns to Avoid

1. The 2 AM deploy: Shipping a prompt change without running the regression suite. Every prompt change needs a test run, no matter how minor it looks.
2. Implicit behavior dependence: Relying on model defaults (“it always returns JSON”) rather than explicit constraints. Model updates will break this.
3. The everything system prompt: Stuffing every possible edge case into the system prompt until it is thousands of tokens long. Dilutes core instructions. Simplify.
4. Hope-based parsing: Parsing model output without exception handling and schema validation. The 3% of outputs that break format will cause runtime errors.
5. No rollback plan: Deploying a prompt without knowing how to revert in under 10 minutes. Prompt regressions hit production fast.
6. Happy-path-only test sets: Regression suites with only easy, expected inputs. They pass everything and catch nothing real.
7. Single-shot red-teaming: Red-teaming a prompt once at launch and never again. New users find new vectors; re-red-team when the user population changes.
8. Format instruction without enforcement: Asking the model to “return only JSON” without using tool use or JSON mode. The instruction reduces format failures; it does not eliminate them.
9. Stale few-shot examples: Few-shot examples that were correct when written but no longer reflect current correct behavior. Silently teach the wrong thing.
10. Prompt changes without changelogs: Editing prompt text in-place with no record of what changed, why, or when. Makes debugging regressions and auditing decisions impossible.

Ten Rules to Remember

1. Explicit beats implicit. Every assumption you leave implicit is a regression waiting for a model update.
2. Show the model what not to do, not just what to do. Negative constraints close the behavioral gaps that positive instructions leave open.
3. Validate the output at the application layer, not just in the prompt.
4. Set max_tokens as a hard limit. Prompt-level length instructions are guidance, not guarantees.
5. Add a test case before you fix a bug. The test case proves the fix and prevents regression.
6. A regression suite with 20 good test cases is more valuable than 200 mediocre ones. Coverage over quantity.
7. Log 1% of production inputs and review them weekly. You will catch degradation before your users do.
8. Prompts have owners. If anyone can change any prompt without review, everyone will regret it.
9. Red-team before you ship. The 15 minutes you spend trying to break your own prompt saves hours of incident response.
10. The failure mode you find in testing is the one that doesn’t reach users. The one you skip is the one that does.