Skip to content
Tutorials Center
My learning

Topics

Lesson 14 of 38 ยท Techniques: Few-shot, Chain-of-Thought and Roles

Self-Consistency and Prompt Chaining

Combine several model runs by majority vote (self-consistency) and split big jobs into a chain of small prompts. Runnable Python simulations included.

A single model call is one sample from a distribution. Two techniques build reliability on top of that: run the same question several times and combine the answers (self-consistency), or split a large task into a sequence of smaller prompts (prompt chaining). Both trade extra calls for better quality.

Self-consistency

If you ask a question with a non-zero temperature, the reasoning path and sometimes the answer will vary between runs. Self-consistency samples several answers and picks the one that appears most often (a majority vote). The idea: a correct answer can be reached by many routes, while mistakes tend to scatter.

flowchart TB
  Q["Question"] --> R1["Run 1 -> 8"]
  Q --> R2["Run 2 -> 8"]
  Q --> R3["Run 3 -> 9"]
  Q --> R4["Run 4 -> 8"]
  Q --> R5["Run 5 -> 7"]
  R1 --> V["Majority vote -> 8"]
  R2 --> V
  R3 --> V
  R4 --> V
  R5 --> V

Sample several reasoning paths, then vote on the final answers.

Requirements: the answer must be short and comparable (a number, a label, a choice), and you need a temperature above zero (at zero the runs would be identical). Cost grows linearly with the number of samples, so use it where accuracy matters more than price.

Simulating the vote

The simulation below does not call a model. It draws noisy "answers" from a distribution where the correct answer is simply the most likely one (60 percent), which is the assumption that makes voting work. It shows how accuracy grows with the number of samples.

Python Why majority voting helps (simulation)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

One sample is right about 60 percent of the time; voting over several samples pushes this higher. The effect depends on the assumption: if the model is consistently wrong (its most likely answer is incorrect), voting will not rescue it.

Prompt chaining

Prompt chaining breaks a task into stages. The output of one prompt becomes (part of) the input of the next. Each prompt is short, focused and testable.

Example: producing a customer reply from a long email thread.

  1. Extract the customer's problem and requested outcome (JSON).
  2. Look up the relevant policy (code or retrieval).
  3. Draft a reply using the extracted data and the policy.
  4. Check the draft against rules (tone, length, no promises).

Advantages over one giant prompt: you can see and fix a failing stage, you can use a cheap model for easy steps, you can add code between steps (validation, database calls), and each step is easier to evaluate.

Costs: more calls, more latency, and errors can compound, since a mistake in step 1 flows into steps 2 to 4. Validate between stages.

Python A three-stage chain with stand-in steps

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Note how stage 2 is plain code, not a model, and how each stage passes structured data forward. This is the main reason chaining is popular: you get to choose, per stage, whether a model or ordinary code is the better tool.

Other combination ideas

  • Generate, then critique, then revise: one prompt writes, a second reviews against a checklist, a third fixes.
  • Decompose, then solve parts: a plan prompt lists sub-questions; each is answered separately; a final prompt merges them.
  • Routing: a cheap classifier prompt decides which specialized prompt handles the request.
  • Best of n with a judge: sample several drafts, have a grader prompt pick one.

Guarding a chain

A few habits prevent chain failures:

  1. Validate every hand-off (types, required keys, allowed values).
  2. Fail early and visibly: log the stage and input that failed.
  3. Cap retries and total calls to control cost.
  4. Test stages separately and the whole chain end to end.
Python Validate a hand-off between stages

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does self-consistency do?

    It combines several sampled answers by majority vote.
  2. 2. When will majority voting NOT help?

    Voting only helps if the correct answer is the most common one; consistent errors win the vote.
  3. 3. What is an advantage of prompt chaining?

    Splitting a task lets you inspect, test and fix individual stages.
  4. 4. What is a risk of chaining?

    Validate hand-offs so one bad stage does not poison the rest.

Summary

  • Self-consistency samples several answers (temperature above zero) and takes a majority vote; it costs extra calls and fails if the model is consistently wrong.
  • Prompt chaining splits a task into stages, mixing models and plain code; each stage is simpler and testable.
  • Validate every hand-off, cap retries and calls, and test stages separately.

Next: Roles, Personas and System Prompts