Skip to content
Tutorials Center
My learning

Topics

Lesson 15 of 39 ยท Planning and Reasoning

Chain-of-Thought and Reasoning Models

Understand why step-by-step reasoning improves LLM answers, how reasoning models differ, when to ask for thinking, and how agents use and verify intermediate steps.

A language model produces one token at a time, and each token can only depend on the tokens before it. If a question needs several steps, such as working out a total or comparing three options, the model does better when it writes the steps down first. The written steps become part of its context, so later tokens can build on earlier ones. This is chain-of-thought (CoT) reasoning, and it is the foundation of everything in this chapter: planning, ReAct loops and reflection all start from "let the model think before it acts".

This lesson uses small Python stand-ins so you can run the ideas on the site. They are not a real language model: the "model" is a scripted function. The real prompts are shown as read-only text.

Answering directly versus reasoning first

Compare two prompts to a real model. The first asks for the answer only; the second invites reasoning.

TEXT
Direct:   "A shop sells pens at 3 for $2.40. How much do 14 pens cost? Answer with a number only."
Reasoned: "A shop sells pens at 3 for $2.40. How much do 14 pens cost? Think step by step, then give the final answer on a line starting with 'Answer:'."

For simple questions both work. For multi-step ones, the reasoned version is usually more accurate because the model computes the unit price (0.80) before multiplying. Always ask for a clearly marked final line, so your code can extract it reliably.

Reasoning models

Many providers now offer reasoning models that think before answering: they generate an internal chain of reasoning (often hidden or summarized) and then the visible reply. For these models, you usually do not need to say "think step by step"; you may instead control a thinking budget or effort level. Practical consequences for agents:

  • Reasoning improves hard planning, math and code decisions, but costs more tokens and time. Use it for the planning step, and a faster model for routine formatting.
  • Do not rely on the visible reasoning text being a perfect explanation of why the model acted. Check outcomes, not just narratives.
  • Keep hidden reasoning out of logs shown to end users unless your provider's rules allow it.

Extracting the final answer

Your code must separate the reasoning from the result. A marker line is the simplest contract; JSON is the sturdiest (see structured outputs).

Python Parse reasoning and the final answer

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The failure branch matters. Real models sometimes skip the marker, so a robust agent retries with a reminder instead of crashing.

Why steps help, in code

You can see the benefit with a tiny experiment. A "quick guesser" that approximates is wrong more often than a solver that shows its steps, which is the same effect CoT has on a model: intermediate results are computed and then reused.

Python Steps versus a one-shot estimate

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Reasoning patterns you will meet

flowchart LR
  Q["Question"] --> A["Direct answer"]
  Q --> B["Chain of thought"]
  Q --> C["Self-consistency: sample several chains"]
  C --> V["Majority vote"]
  Q --> D["Tree of thoughts: explore and prune branches"]

From cheapest to most thorough.

  • Zero-shot CoT: just ask for steps.
  • Few-shot CoT: include one or two worked examples in the prompt, showing the style of reasoning you want.
  • Self-consistency: sample several independent reasoning chains and take the most common final answer. It costs multiple calls but reduces random slips.
  • Tree of thoughts: explore several partial solutions, score them and continue from the best. It is covered further in reflection and self-critique and replanning.

Using reasoning inside an agent

In an agent loop, the model usually reasons briefly before each action: what do I know, what is missing, which tool answers that? You can keep this reasoning in the message history (it helps later steps) or drop it to save tokens. A good default: keep the last few steps of reasoning, summarize older ones. Memory management is detailed in short-term memory.

Remember three limits. Reasoning can be confidently wrong: verify with tools or code where possible (compute totals in Python, do not trust mental arithmetic). It is not free: long chains cost tokens and add latency. And it is not a guarantee of faithfulness: the text the model writes may not be the real cause of its answer.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why does writing intermediate steps often improve answers?

    Each generated token conditions the next, so computed intermediate results can be reused.
  2. 2. What is a good way to make the final answer easy to extract?

    A marker or structured format gives your code a stable contract to parse.
  3. 3. What does self-consistency do?

    Voting across independent samples reduces the chance of a single bad chain deciding the result.
  4. 4. How should you treat reasoning text that a model shows?

    Reasoning can be wrong or unfaithful, so check outcomes.

Summary

  • Step-by-step reasoning lets a model reuse intermediate results, which helps multi-step tasks.
  • Reasoning models think before answering and are controlled with effort or budget settings.
  • Always extract the final answer through a marker or structured output, with a retry for failures.
  • Self-consistency and tree search trade extra calls for reliability.
  • Verify important numbers and facts with tools; reasoning text is not proof.

Next: The ReAct Pattern: Reason, Act, Observe