Skip to content
Tutorials Center
My learning

Topics

Lesson 25 of 38 ยท Evaluation and Testing Prompts

Building a Prompt Test Set

Stop judging prompts by gut feeling. Build a small, representative test set with expected answers, edge cases and tags, and run it against any prompt.

You change one sentence in a prompt. Three examples you tried by hand now look better. Did the prompt improve, or did you just look at three lucky cases? Without a test set you cannot tell, and prompts have a nasty habit of fixing one case while quietly breaking five others.

An evaluation set (or eval set, test set, "golden set") is a list of inputs with a description of what a good answer looks like. Running it takes seconds, gives a number, and turns prompt writing from guessing into engineering. This chapter builds the whole loop; this lesson creates the data.

No real model runs on this site, so the examples use a stand-in function in place of the model call. Swap in your own API call and the harness code is unchanged.

What goes into a test case

A case needs three things: an input, a way to judge the output, and tags.

Field Purpose Example
id Stable name for reporting refund-typo-01
input What you send to the prompt "my order never arived"
expected The correct answer, or the rule it must meet "shipping"
tags Groups for slicing results ["typo", "short"]

Judging can be exact (a category label), a rule ("contains a date, under 50 words, valid JSON") or a rubric scored by a person or by a model (later in this chapter). Prefer the strictest judge that still accepts every genuinely good answer.

Where cases come from

Good sets mix four sources:

  1. Real inputs from logs, tickets or user research, anonymized. These matter most because they show how people really write.
  2. Typical cases that cover each main category evenly.
  3. Edge cases: empty input, very long input, mixed languages, typos, ambiguity, contradictory requests.
  4. Past failures. Every bug report becomes a permanent test. This is how a set grows into a safety net.

Include cases where the right behaviour is to refuse or ask a question. A prompt that always answers confidently will pass a set full of answerable questions and still be unsafe.

Size and balance

Start with 20 to 50 cases; that is enough to catch big regressions. Grow toward a few hundred where mistakes are costly. Balance categories so a majority class does not hide failures: if 90 percent of cases are "billing", a prompt that only understands billing scores 90 percent.

Split the set in two. Use the development set while you tune the prompt. Keep a held-out set you look at rarely, to check you have not just fitted the prompt to the examples you stared at.

A minimal harness

Python A test set and a harness (stand-in model)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The harness reports failing case ids, so every number comes with something to investigate. The hand-tested "typical" cases pass in both versions; only the typo case reveals the difference, which is exactly why edge cases belong in the set.

Slice results by tag

An overall score hides patterns. Group by tag to find where a prompt is weak:

Python Pass rate per tag

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Keep the set healthy

  • Store it in version control as JSON or CSV beside the prompt, so changes to either are reviewed together.
  • Never tune on the held-out set. The moment you do, it becomes a development set.
  • Check that the labels are right. Wrong expectations make good prompts look bad; review disagreements by hand.
  • Watch for leakage. If an example from the set also appears as a few-shot example in the prompt, the test measures memorization.
  • Refresh it. User behaviour drifts, so add recent real inputs regularly (see monitoring).

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why do three hand-tested examples fail to prove a prompt is better?

    A small hand-picked sample hides regressions; a set covering many cases reveals them.
  2. 2. Which source of test cases is usually the most valuable?

    Real inputs show how people actually write, including typos, slang and ambiguity.
  3. 3. What is a held-out set for?

    A set you rarely look at gives an honest score, since you did not shape the prompt around it.
  4. 4. Why include cases where the right behaviour is to refuse or ask a question?

    Without such cases, a prompt that always answers confidently looks perfect while being unsafe.

Summary

  • An eval set is inputs plus a way to judge outputs; it replaces gut feeling with a number.
  • Mix real inputs, typical cases, edge cases and past failures; include refusals.
  • Start with 20 to 50 cases, tag them, and report failures by id and by tag.
  • Keep a held-out set, version the data with the prompt and fix wrong labels.

Next: Metrics for Prompt Outputs