Skip to content
Tutorials Center
My learning

Topics

Lesson 23 of 38 ยท Efficient Fine-tuning and Quantization

Parameter-Efficient Fine-Tuning (PEFT) Overview

Why full fine-tuning gets expensive and how parameter-efficient methods (adapters, LoRA, prefix tuning, prompt tuning, BitFit) cut trainable parameters, with runnable counting stand-ins.

Full fine-tuning updates every weight of the model. For a 110-million-parameter encoder that is fine. For a 7-billion-parameter language model it means holding the weights, the gradients and the optimizer state for 7 billion numbers each, which does not fit on a normal GPU. It also leaves you with a full-size copy of the model for every task you fine-tune.

Parameter-efficient fine-tuning (PEFT) freezes the pretrained weights and trains a small number of extra or selected parameters, typically 0.1% to 2% of the total. The result is much less memory, much smaller files (megabytes instead of gigabytes) and often accuracy close to full fine-tuning. The Hugging Face peft library implements the main methods behind one interface.

The peft library, PyTorch and GPUs are not available on this site. Plain code blocks are real and read-only. The TryIt stand-ins count parameters and memory with plain Python, which is exactly the arithmetic that motivates PEFT.

The families of methods

PEFT methods at a glance

Adapters Small bottleneck layers inserted inside each block.
LoRA Low-rank updates added to chosen weight matrices.
Prefix tuning Learned vectors prepended to attention keys and values.
Prompt tuning Learned soft tokens added to the input embeddings.

The pretrained weights stay frozen in all four.

1 / 2
  • Adapters insert a tiny two-layer network (down-projection, nonlinearity, up-projection) after attention and feed-forward sublayers. Only the adapters train. A drawback is that they add a little latency at inference because they are extra layers.
  • LoRA (low-rank adaptation) adds a trainable low-rank matrix product to selected weight matrices and can be merged into the original weights afterwards, so there is zero extra latency. It is the most popular method; see LoRA explained.
  • Prefix tuning learns "virtual" key and value vectors that every attention layer sees in front of the real sequence. Prompt tuning is the simpler version that learns only extra input embeddings (soft prompt). P-tuning adds a small encoder to produce those vectors.
  • BitFit trains only the bias terms: extremely few parameters, modest quality.
  • IA3 learns tiny scaling vectors that rescale keys, values and feed-forward activations.

Counting trainable parameters

The best way to feel the difference is to count. This stand-in models a BERT-base-like encoder (12 layers, hidden size 768, feed-forward size 3072, 30,522-token vocabulary) and counts parameters under each strategy. The formulas are the standard ones for this architecture.

Python Stand-in: trainable parameters per method

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice that even the largest PEFT choice here trains only a few percent of the weights, and LoRA with rank 8 on two matrices trains well under one percent (plus the head you must train anyway for classification). For a 7B model, the ratio is even more dramatic because the adapters grow with hidden size, not with the full parameter count.

Where the memory goes

Parameter count is not the only cost. During training with the Adam optimizer in mixed precision you store the weights, one gradient per trainable weight, and two optimizer moments per trainable weight, usually in 32-bit. Freezing a weight removes its gradient and its optimizer state, which is where most of the saving comes from. The frozen weights are only read in the forward pass.

Python Stand-in: training memory, full versus PEFT

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Full fine-tuning needs on the order of a hundred gigabytes before activations; LoRA needs roughly the cost of just holding the model. The weights themselves can then be compressed further with quantization, which is QLoRA.

Using the peft library

The workflow is the same for every method: build a config, wrap the model with get_peft_model, and train with the usual Trainer.

Python
from peft import LoraConfig, TaskType, get_peft_model
from transformers import AutoModelForSequenceClassification

base = AutoModelForSequenceClassification.from_pretrained(
    "roberta-base", num_labels=2)

config = LoraConfig(
    task_type=TaskType.SEQ_CLS,
    r=8, lora_alpha=16, lora_dropout=0.05,
    target_modules=["query", "value"],
)
model = get_peft_model(base, config)
model.print_trainable_parameters()
# prints something like: trainable params: 0.9M || all params: 125M || trainable%: 0.7

Other configs follow the same shape: PrefixTuningConfig, PromptTuningConfig, IA3Config. When you save, only the small adapter weights are written (model.save_pretrained("out/lora")), and you load them later on top of the base model with PeftModel.from_pretrained(base, "out/lora"). One base model can serve many tasks by swapping tiny adapter files.

Choosing a method

Situation Reasonable choice
Small encoder, plenty of GPU, maximum accuracy Full fine-tuning
Large language model, limited GPU LoRA, or QLoRA if memory is very tight
Many tasks sharing one base LoRA adapters swapped at serving time
Extremely little data Head only, or a few layers unfrozen
You cannot touch inference code LoRA merged into the weights

Start with LoRA at rank 8 to 16 on the attention projections. If quality is not enough, raise the rank, add more target modules, or try full fine-tuning to find out whether PEFT is the bottleneck.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What is common to all PEFT methods?

    PEFT trains few parameters on top of frozen pretrained weights; quality is often close, not always identical.
  2. 2. Why does PEFT save so much training memory?

    Gradients and Adam moments exist only for trainable parameters, which dominates training memory.
  3. 3. Which method can be merged into the base weights, adding no inference latency?

    A LoRA update is a matrix that can be added into the original weight matrix.
  4. 4. What gets saved by model.save_pretrained() on a PEFT model?

    Adapter files are megabytes; you reload them on top of the base model.

Summary

  • PEFT freezes the pretrained model and trains a tiny fraction of parameters, saving memory and storage.
  • Main methods: adapters, LoRA, prefix tuning, prompt tuning, IA3 and BitFit; LoRA is the usual first choice.
  • Memory savings come from not storing gradients and optimizer state for frozen weights.
  • The peft library wraps a model with get_peft_model and saves only the adapter.
  • One base model can serve many tasks by swapping adapters.

Next: LoRA explained