Skip to content
Tutorials Center
My learning

Topics

Lesson 19 of 37 ยท Fine-tuning and Alignment

Fine-tuning vs Prompting

Learn when to prompt, retrieve or fine-tune a large language model, what full fine-tuning costs in memory, and why a pretrained start beats training from scratch.

A pretrained large language model already knows a lot of grammar, facts and code. What it does not know is your task, your tone or your data. You have three levers for closing that gap: change the prompt, give the model documents at request time, or change its weights. This lesson helps you pick the lever and shows what changing the weights costs.

Fine-tuning means continuing training of a pretrained model on a smaller, task-specific dataset. It is the most powerful lever and also the most expensive one, so it is rarely the first thing to try.

Real LLM training needs GPUs and billions of parameters. The examples here run in your browser with NumPy and scikit-learn, using small models that behave the same way, so you can see the ideas rather than just read about them.

The three levers

flowchart TD
    A["Model gives wrong or off-style answers"] --> B{"Can a better prompt fix it?"}
    B -->|"yes"| C["Prompt engineering: instructions, examples, format"]
    B -->|"no"| D{"Is knowledge missing or changing?"}
    D -->|"yes"| E["Retrieval: put documents into the prompt"]
    D -->|"no, behavior or style"| F["Fine-tuning: change the weights"]

Start at the top and move down only when the cheaper lever fails.

  • Prompting costs nothing to set up and can be changed in seconds. It is limited by the context window and by how well the model follows instructions. See Prompting Patterns.
  • Retrieval keeps facts outside the model so they can be updated and cited. See Retrieval-Augmented Generation.
  • Fine-tuning bakes behavior into the weights: a consistent output format, a house style, a narrow skill such as classifying support tickets, or a smaller model that imitates a larger one.

Fine-tuning is a poor way to teach a model new facts. Facts seen in a small tuning set are learned unreliably and cannot be updated without training again. It is a good way to teach behavior.

What fine-tuning changes

Pretraining produced a weight set theta. Full fine-tuning runs gradient descent on a new dataset, starting from theta, and updates every weight. Variants differ in which weights move:

Method Weights updated Typical use
Full fine-tuning all large datasets, maximum quality
Head or linear probe a new output layer only classification on frozen features
Adapters, LoRA small added matrices cheap, many tasks on one base (LoRA and PEFT)
Prompt or prefix tuning learned input vectors very small storage

What it costs: memory

Training memory is dominated by more than the weights. With mixed precision and the Adam optimizer, each parameter needs the weight itself, a gradient, and two optimizer moments, and often a full-precision master copy. A common rule of thumb is about 16 bytes per parameter, before activations.

Python Memory for full fine-tuning

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

A 7B model needs roughly 13 GiB just to run in 16-bit, but over 100 GiB to fully fine-tune. That gap is the main reason parameter-efficient methods exist.

Why start from a pretrained model

Why not train a fresh model on your small dataset? Because pretraining supplies representations that took enormous data to learn. We can show the effect at toy scale with scikit-learn. We treat PCA fitted on lots of unlabeled digit images as "pretraining", then train a small classifier on only a handful of labeled examples, with and without those learned features.

Python Pretrained features vs raw pixels

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The classifier on pretrained features is usually at least as accurate with the same 30 labels, and it trains on 20 numbers instead of 64. An LLM works the same way at a vastly larger scale: its pretraining is the "free" knowledge, and fine-tuning only has to steer it.

A practical decision checklist

Ask these in order:

  1. Did you write a clear prompt with two or three examples, and measure the result on a held-out set? If not, do that first (Evaluation).
  2. Is the failure about missing or fresh information? Use retrieval.
  3. Is the failure about format, tone or a narrow skill that prompting cannot make reliable, or is latency/cost the issue because prompts are long? Consider fine-tuning, possibly a small model.
  4. Do you have at least a few hundred to a few thousand clean examples? Quality beats quantity: noisy labels teach noisy behavior.
  5. Can you evaluate before and after? Without a test set you cannot tell whether tuning helped.

Common mistakes

  • Fine-tuning to inject facts, then being surprised by hallucinations.
  • Training on the test set, or on data that overlaps it (contamination).
  • Over-training on a tiny set, which erases general ability (catastrophic forgetting). Use a low learning rate and few epochs.
  • Skipping a baseline: a good prompt on the base model is the number your tuned model must beat.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which problem is fine-tuning best suited for?

    Fine-tuning shapes behavior. Fresh facts belong in retrieval, and context length is an architecture property.
  2. 2. Why does full fine-tuning need far more memory than inference?

    Adam keeps two moments per weight, plus gradients and often fp32 master weights, so about 16 bytes per parameter instead of 2.
  3. 3. In the digits example, what role did PCA on unlabeled images play?

    Unlabeled data taught a useful representation, so the classifier needed fewer labeled examples.
  4. 4. What is catastrophic forgetting?

    Large updates on a narrow dataset can overwrite useful pretrained behavior; low learning rates and mixing in general data help.

Summary

  • Prompt first, retrieve for knowledge, fine-tune for behavior.
  • Full fine-tuning updates all weights and costs roughly 16 bytes per parameter with Adam.
  • Pretrained representations let small labeled datasets go a long way.
  • Always keep a prompt-only baseline and a held-out test set.

Next: Supervised Fine-Tuning