Skip to content
Tutorials Center
My learning

Topics

Lesson 33 of 38 ยท Prompt Patterns in Production

Prompt Libraries in Production

Move prompts out of scattered strings into a versioned registry: templates with typed variables, release labels, safe rollouts, rollbacks and tests in your pipeline.

In a demo, the prompt is a string in the middle of a function. In production that stops working quickly: the same prompt is copied into three services, someone edits one copy, nobody knows which version produced yesterday's bad answer, and a "small wording fix" ships without any test. A prompt library (or registry) treats prompts like code and configuration: named, versioned, validated, tested and released on purpose.

This lesson builds a miniature registry in Python so you can see each idea working, then lists what to look for in real tools. It builds on delimiters and templates and iterating and versioning.

What a prompt record contains

A prompt in production is more than text:

Field Why
name Stable identifier your code asks for (ticket-classifier)
version Immutable: once published, text never changes
template The text with named variables
variables Declared inputs, so missing or extra values are caught
model and settings Temperature, max tokens: they change behaviour as much as words
labels Pointers such as production and staging to a version
notes Why it changed, who approved it

The key rule: versions are immutable, labels move. Your application asks for ticket-classifier@production. To release, you move the label; to roll back, you move it back. No deploy of application code is needed for a prompt-only change (although many teams still keep prompts in the repository and deploy with the code, which is simpler and just as valid).

A minimal registry

Python A tiny versioned prompt registry

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Three useful behaviours appear already: a new version never overwrites the old one, production keeps serving v1 while v2 is tested in staging, and a call with missing variables fails loudly instead of sending the literal text $audience to a model.

Typed and validated variables

Variables are an attack and bug surface. Validate them before rendering:

  • Required variables present, no unknown ones.
  • Length limits (a pasted novel in $text destroys your token budget).
  • Untrusted values wrapped in delimiters (defending against injection).
  • Braces and special characters handled by the template engine, not by string concatenation.

Release flow

flowchart LR
    E["Edit in a branch or draft"] --> T["Run test set"]
    T --> G{"Passes thresholds?"}
    G -- no --> E
    G -- yes --> S["Promote to staging"]
    S --> C["Canary: a small % of traffic"]
    C --> M{"Metrics healthy?"}
    M -- yes --> P["Promote to production"]
    M -- no --> R["Roll back the label"]

A prompt change goes through the same gates as code.

The gates reuse what you built earlier: your test set, metrics and regression tests. The release check below compares a candidate with the current production version on the same cases.

Python Gate a release on a test set

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Note the extra rule: no case that passed before may fail now, even if the average improves. Averages hide regressions.

Rollouts and rollbacks

  • Canary: send a small share of traffic (say 5%) to the new version and compare metrics (error rate, thumbs-down rate, cost, latency) before widening.
  • A/B test: split traffic for a fixed period to measure real user impact, when offline tests cannot answer the question.
  • Instant rollback: moving a label back should take seconds and need no code deploy.
  • Pin versions in logs: every logged request records prompt name, version and model, so any bad answer can be traced to its exact prompt.

Model upgrades are prompt changes

A prompt is tuned to a model. When the provider retires a model or you switch to a new one, rerun the whole test set before moving the production label. Pin to dated model versions where the provider offers them; floating aliases can change behaviour under you.

Organizing a library

  • One prompt per task, named by task (extract-invoice-fields), not by author or date.
  • Share common blocks (tone, safety rules, output format) as partials included by several prompts, so one fix reaches all of them.
  • Keep examples (few-shot) next to the prompt that uses them, versioned together.
  • Add a short README per prompt: purpose, inputs, expected output, known failures, owner.
  • Review prompt changes in pull requests, with the test results attached.

Tools

Whether you build a small registry like the one above, keep prompt files in your repository, or adopt a hosted prompt-management platform, look for the same features: immutable versions, labels, diffs, variable validation, test integration, access control and audit trail, and logging that records which version served each request.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Why are prompt versions immutable?

    Reproducibility: you can trace and replay any answer to its exact prompt.
  2. 2. What does moving the 'production' label back to an older version do?

    Labels are pointers; changing the pointer changes what is served.
  3. 3. Why should a release check look at per-case regressions and not only average accuracy?

    A prompt can gain on average while breaking important cases.
  4. 4. What should happen when a required template variable is missing?

    A missing variable is a bug; failing early avoids sending a broken prompt.
  5. 5. When switching to a new model version you should

    Prompts are tuned to a model; behaviour can differ, so test before releasing.

Summary

  • Treat prompts like code: named, versioned (immutable), tested and released through gates.
  • Labels such as production and staging point to versions, enabling canaries and instant rollbacks.
  • Validate variables, record model and settings with the template and log the version of every request.
  • A new model is a prompt change: rerun the test set first.

Next: Cost, Latency and Caching