Skip to content
Tutorials Center
My learning

Topics

Lesson 31 of 38 ยท Safety, Injection and Guardrails

Guardrails and Output Checking

Build guardrails around a language model: input checks, output validators, topic limits, moderation, safe fallbacks, and how to measure false positives and misses.

A guardrail is a check that sits around the model, not inside the prompt. The prompt asks the model to behave; the guardrail verifies that it did, and does something safe when it did not. Prompts are requests. Guardrails are enforcement.

Typical guardrails run before the model (input checks), after it (output checks) or around tool calls (permission checks). This lesson builds a small pipeline and shows how to measure it. Everything here is ordinary Python, so it runs for real in the editors below; the model itself is replaced by canned replies.

The guardrail pipeline

flowchart LR
    I["User input"] --> IC{"Input checks"}
    IC -- blocked --> F1["Safe refusal"]
    IC -- ok --> M["Model"]
    M --> OC{"Output checks"}
    OC -- fails --> F2["Retry or fallback"]
    OC -- ok --> U["User"]

Each check can end the request early with a safe response.

Design rules:

  1. Fail closed. If a check cannot decide, take the safe path (refuse, ask a human), not the permissive one.
  2. Cheap checks first. Length limits and regexes cost nothing; a second model call costs money and time.
  3. Keep the reason. Log which check fired, so you can tune it.
  4. Return something useful. "I can only help with orders and returns" beats a blank error.

Input checks

Typical input guardrails:

  • Size limits. Reject or truncate inputs beyond a limit to protect your token budget.
  • Topic scope. A tax-help bot should not write poems. A cheap keyword or classifier gate keeps it on topic.
  • Sensitive data. Detect card numbers, national ids or API keys before they enter your logs or reach a third-party API (see privacy and PII).
  • Known attack patterns. Phrases such as "ignore previous instructions" are cheap to flag, though never enough alone (see defending against injection).
Python Input guardrail with reasons

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The keyword gate is blunt. "Can I get my money back?" would be refused because it contains none of the topic words. That is a false positive, and we will measure it below.

Output checks

Output guardrails verify what comes back before a user (or another program) sees it:

  • Format. Valid JSON, matches the schema, enum values allowed (see validating and repairing JSON).
  • Content rules. No forbidden words, no unknown links, no leaked secrets or prompt text.
  • Grounding. Every number or quote must appear in the supplied context, or the answer cites a real source.
  • Length and tone. Within bounds, no insults.

On failure you choose: retry (once or twice, with the error message added), repair (strip the offending part), or fall back (a fixed safe answer, or hand off to a human).

Python Output guardrail with retry and fallback

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

In a real system you would add the problem list to the retry prompt ("Your last answer was rejected because ...") so the model can correct itself.

Model-based guardrails

Some rules cannot be written as regexes ("is this reply respectful?", "does it give medical advice?"). A second model call can classify the first one's output, much like LLM-as-judge. Rules of thumb:

  • Ask for a small enum (safe / unsafe) with a short reason, and validate it.
  • Use a cheaper, faster model for the check if quality allows.
  • Run it only after the cheap checks pass.
  • Test it like any prompt: it has its own error rate.

Providers also offer moderation endpoints that score text for categories such as hate or self-harm. They are a good baseline layer, but they do not know your policy (for example "never discuss competitors"), so keep your own rules too.

Measuring a guardrail

A guardrail makes two kinds of mistakes:

  • False positive: blocks a good request (annoys users).
  • False negative: lets a bad one through (the harm you wanted to stop).

Tightening one usually loosens the other. Measure both on a labelled set, using the metrics you already know: precision and recall.

Python Precision and recall of a guardrail

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

This crude rule blocks an innocent "ignore the typo" message and misses a role-play attack. Real tuning means adding examples of both kinds to your test set and trying again.

Design advice

  • Put guardrails in code you control, not in a prompt the user can talk around.
  • Layer several cheap checks rather than one clever one.
  • Always have a fallback path, and make it good.
  • Review the blocked traffic regularly: it shows both attacks and where you annoy real users.
  • Keep policies versioned and tested like prompts (see versioning prompts).

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What is the difference between a prompt instruction and a guardrail?

    A guardrail is code or a separate check that enforces rules even when the model ignores the prompt.
  2. 2. What does 'fail closed' mean?

    Failing closed prefers a safe refusal over a risky permissive default.
  3. 3. A guardrail blocks 'Can I get my money back?' as off-topic. What is this?

    A good request was blocked, which is a false positive.
  4. 4. Which check is best done first?

    Cheap deterministic checks run first so you only pay for expensive checks when needed.
  5. 5. Why are provider moderation endpoints not enough alone?

    Moderation covers general harm categories; your policy (topics, competitors, claims) needs your own rules.

Summary

  • Guardrails enforce rules around the model: input checks, output checks, tool permission checks.
  • Fail closed, run cheap checks first, log reasons and give users a helpful fallback.
  • On a failed output check: retry with the error, repair, or fall back.
  • Measure false positives and false negatives; tune with a labelled set.

Next: Privacy and PII Handling