Skip to content
Tutorials Center
My learning

Topics

Lesson 32 of 53 ยท Evaluation

Evaluating LLM Applications

Learn why vibe checks fail and how to build LLM evaluations: test sets, golden answers, exact, regex and semantic checks, regression suites and sampling noise.

You change a prompt, try three questions, and the answers look better. You ship it. A week later users complain about a case you never tried. This is the "vibes" approach to LLM development, and it breaks down quickly, because model output is open-ended, varies from run to run, and a fix for one input often breaks another.

The alternative is an evaluation (an "eval"): a fixed set of test cases plus a way to score the answers automatically. Evals do for LLM apps what unit tests do for normal code. They tell you whether a change helped, whether a new model is worth switching to, and whether last month's quality is still there. This lesson builds a small eval harness with a fake model so you can see every moving part.

Why vibes fail

  • Tiny samples. Three examples cannot represent thousands of real inputs.
  • Memory. You remember the answer you liked, not the ones you skimmed past.
  • Regressions. Improving one behavior silently damages another.
  • Noise. Sampling with a temperature above zero means the same prompt can produce different answers, so one good run proves little.
  • Disagreement. Two teammates "feel" different things about the same output.

The eval loop

Test set Inputs with known good outcomes.
Run Call the app on every case.
Score Check each answer automatically.
Compare Did the change help or hurt?

Run it after every prompt, model or retrieval change.

1 / 2

Building a test set

A test case is an input plus whatever you need to judge the output. Good sources:

  • Real traffic. Sample actual user questions (with privacy rules applied), including the messy ones.
  • Known failures. Every bug report becomes a permanent test case.
  • Edge cases. Empty input, very long input, other languages, requests the app should refuse.
  • Hand-written examples covering each feature you promise.

Thirty to a hundred well-chosen cases is a solid start. Label each with a category so you can see that, say, refusals fail while math is fine. Keep a separate group of cases you never tune against, so you can check you did not overfit the prompt to your own tests.

Golden answers and checks

A golden answer is the expected output for a case. How you compare depends on the task:

Check Use it when Example
Exact match Output is a label or short value positive equals positive
Normalized match Case and spacing should not matter " Paris. " equals paris
Regex / contains Key facts must appear, wording is free answer contains 1969
Structure check Output must be valid JSON with certain fields parse and validate
Code check Answer is code run it against unit tests
Semantic similarity Meaning matters, wording varies embedding cosine above a threshold
LLM judge or human Open-ended quality see LLM-as-Judge and Human Review

Use the cheapest check that is reliable. A deterministic check is fast, free and never disagrees with itself. Reach for model-based grading only for qualities code cannot measure.

Python A tiny eval harness with a fake model

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The overall score hides detail; the per-category breakdown and the list of failures are what you actually act on. Always read the failures, because sometimes the test (not the model) is wrong, for example a golden answer that is too strict.

Regression suites

An eval is most valuable as a regression suite: run it before and after every change and compare. The question is not "is 83% good?" but "did this edit move 83% to 90% or to 70%, and which cases flipped?" Store results per case, and diff them:

  • Fixed cases failed before and pass now.
  • Broken cases passed before and fail now. These deserve the most attention.
  • Unchanged cases tell you nothing new.

You can wire the suite into continuous integration and block a merge when the score drops below a threshold or when any must-pass case breaks.

Sampling noise

When temperature is above zero, the same input can pass once and fail the next time. If you run each case a single time, a score difference of a few points may be pure luck. Two habits help:

  • Run each case several times and report the pass rate per case, or lower the temperature to near zero for evals when your production setting allows it.
  • Keep the sample large enough. With 20 cases, one flipped case moves the score by 5 points. Never celebrate a 3-point gain on a small set. Metrics: Retrieval, Benchmarks and Statistics shows how to put a confidence interval around a score.
Python Noise: the same system, scored many times

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The true quality never changed, yet small test sets produce wildly different scores. This is why a small eval can only detect big changes.

A practical workflow

  1. Write 20 to 50 cases from real inputs and known failures.
  2. Pick the cheapest reliable check for each.
  3. Record a baseline score for the current prompt and model.
  4. Change one thing at a time and re-run.
  5. Read the broken and fixed cases, not only the total.
  6. Add every new production failure as a test case.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why is trying three questions by hand a weak way to judge a prompt change?

    A small, hand-picked sample misses regressions and is shaped by what you happen to remember.
  2. 2. Which check is best for a classifier that must output exactly 'positive' or 'negative'?

    When the output is a fixed label, a deterministic exact match is cheap and reliable.
  3. 3. After a prompt edit, the score rises from 80% to 82%, but 4 cases that used to pass now fail. What should you do?

    Comparing per-case results shows what the change broke, which a single average hides.
  4. 4. Why run each case several times when temperature is above zero?

    Sampling makes outputs vary, so repeated runs separate real quality from luck.

Summary

  • Evals replace vibes with a fixed test set and automatic scoring.
  • Build cases from real inputs, past failures and edge cases; tag them by category.
  • Use the cheapest reliable check: exact, normalized, regex, structure, code tests, semantic similarity, then judges.
  • Treat the suite as regression tests: compare per-case before and after every change.
  • Sampling noise and small sets make small score differences meaningless.

Next: grade open-ended answers with LLM-as-Judge and Human Review.