Skip to content
Tutorials Center
My learning

Topics

Lesson 32 of 45 ยท Saving, Speed and Hardware

Saving and Loading Models

Save and load PyTorch models safely: state_dict, torch.save and torch.load, weights_only, resumable checkpoints with optimizer state, ONNX and TorchScript.

Training a model can take hours or weeks, so you need to be able to keep the result, load it later for inference, and resume a run that was interrupted. PyTorch's answer is simple: a model's learned numbers live in a dictionary called the state_dict, and torch.save / torch.load write and read dictionaries to disk.

There is one catch that makes this lesson more than a recipe: PyTorch's default file format is built on Python's pickle, and loading a pickle file from a stranger can run arbitrary code. PyTorch itself cannot run on this site, so its code here is read-only. The live editors use a NumPy stand-in that follows the same ideas: save only plain arrays and numbers, restore the optimizer too, and refuse anything that is not on an allow-list.

What goes in a checkpoint

Model weights model.state_dict()
Optimizer state Momentum and Adam statistics.
Progress Epoch, step, best score.
Settings Config needed to rebuild the model.

Weights alone are enough for inference, but not for resuming training.

1 / 2

The state_dict

Every nn.Module can describe itself as an ordered dictionary that maps parameter names to tensors. Optimizers have one too. A state_dict holds numbers and nothing else: no code, no class definition.

Python
import torch
from torch import nn

model = nn.Sequential(nn.Linear(4, 8), nn.ReLU(), nn.Linear(8, 2))

for name, tensor in model.state_dict().items():
    print(name, tuple(tensor.shape))
# 0.weight (8, 4)
# 0.bias (8,)
# 2.weight (2, 8)
# 2.bias (2,)

The keys come from the module tree (0 and 2 are the positions in the Sequential). Besides parameters, a state_dict also contains buffers such as the running mean of a BatchNorm layer, which are not learned by gradient descent but are part of the model's state (see Initialization, Normalization and Dropout).

Saving and loading weights

The recommended workflow saves the state_dict, not the whole model object:

Python
torch.save(model.state_dict(), "model.pt")          # write

model = nn.Sequential(nn.Linear(4, 8), nn.ReLU(), nn.Linear(8, 2))  # rebuild the same architecture
state = torch.load("model.pt", weights_only=True)    # read
model.load_state_dict(state)
model.eval()                                         # switch off dropout, use BatchNorm statistics

You must create the model in code first, because the file contains only numbers. That is a feature: the weights stay valid even if you refactor the surrounding code, as long as the parameter names and shapes still match. load_state_dict is strict by default and raises an error listing missing or unexpected keys. Pass strict=False only when you deliberately load part of a model (for example, pretrained layers into a bigger network, see Transfer Learning) and always inspect what it returns.

Two habits save headaches. Call model.eval() before inference, because dropout and BatchNorm behave differently in training mode. And pass map_location when the file was written on a GPU but you load it on a machine without one:

Python
state = torch.load("model.pt", map_location="cpu", weights_only=True)

Saving the whole model with torch.save(model, "model.pt") is possible, but it pickles the class by its import path. The file then breaks if you rename the module or move the file, and it is the riskiest option to load.

Why weights_only matters

torch.load uses pickle. A pickle file is a small program for building objects, and it can contain instructions such as "import os and call system". Loading an untrusted file with the full unpickler is therefore the same as running untrusted code. This is not theoretical: model hubs have hosted malicious pickle files.

weights_only=True makes the loader accept only a small allow-list of types: tensors, dictionaries, lists, numbers and strings. Anything else raises an error instead of running. In recent PyTorch releases this is the default, but being explicit in your own code costs nothing. The rule of thumb:

  • Your own checkpoints, saved as plain dictionaries of tensors and numbers: weights_only=True.
  • Files from other people: use weights_only=True or a format that cannot execute code at all, such as safetensors, a simple tensor file format used by many model hubs.
  • If a load fails because of an unknown class, do not just turn the flag off. Find out what the class is and whether you trust the file.

The stand-in below shows the difference. A pickle that tries to call a function runs it with the default loader, and is refused by an allow-list loader that works like weights_only=True.

Python Pickle runs code, an allow-list loader refuses

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Checkpoints for resuming training

To resume a run exactly where it stopped you need more than the weights. Optimizers such as Adam keep running averages for every parameter; if you restart them from zero, the first steps behave differently (learning-rate schedules also have their own state). A checkpoint is just a dictionary:

Python
checkpoint = {
    "epoch": epoch,
    "model": model.state_dict(),
    "optimizer": optimizer.state_dict(),
    "scheduler": scheduler.state_dict(),
    "best_val_loss": best_val_loss,
    "config": {"hidden": 64, "layers": 4},
}
torch.save(checkpoint, "last.ckpt")

# later, in a new process
ckpt = torch.load("last.ckpt", map_location=device, weights_only=True)
model.load_state_dict(ckpt["model"])
optimizer.load_state_dict(ckpt["optimizer"])
scheduler.load_state_dict(ckpt["scheduler"])
start_epoch = ckpt["epoch"] + 1

Practical advice: build the model and the optimizer first, then load their states (the optimizer needs the parameters to attach its statistics to). Write to a temporary name and rename when finished (os.replace), so a crash during saving never leaves you with a half-written file. Keep the best checkpoint separately from the latest one. Random number generators have state too; for bit-exact resumption also save torch.get_rng_state() (see Debugging PyTorch Code).

The proof that this matters is easy to run. The stand-in trains a tiny model with Adam written in NumPy, interrupts it half-way, and compares three runs: uninterrupted, resumed with the saved optimizer state, and resumed with weights only. The checkpoint is stored without pickle at all, as a NumPy .npz archive plus JSON text, which is the same idea as saving a dictionary of tensors.

Python Resume with and without optimizer state

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Resuming with the full state reproduces the uninterrupted run, while the weights-only restart drifts away from it. In a long training run that difference can show up as a sudden bump in the loss curve right after a restart.

Exporting: TorchScript and ONNX

A state_dict needs your Python class to be useful. When the consumer is not a Python program (a C++ service, a phone, a browser), you export the computation itself.

Python
example = torch.randn(1, 4)

# TorchScript: trace records what the model did for the example input
scripted = torch.jit.trace(model, example)
scripted.save("model_ts.pt")
loaded = torch.jit.load("model_ts.pt")      # no class definition needed

# ONNX: an open format that many runtimes understand
torch.onnx.export(
    model, example, "model.onnx",
    input_names=["features"], output_names=["scores"],
    dynamic_axes={"features": {0: "batch"}},   # allow any batch size
)

Tracing runs the model once and records the operations, so Python if statements and loops that depend on the data are frozen into whatever path the example took; scripting (torch.jit.script) compiles the code and keeps control flow. For new projects, check which export path your target runtime recommends, since the PyTorch export tooling keeps evolving. Always compare the exported model's outputs to the original on a few inputs before trusting it. Testing and Deploying Models covers that check.

Common mistakes

  • Saving the whole model object, then renaming the file that defines the class. Save the state_dict and rebuild in code.
  • Loading before building: load_state_dict is a method of an existing model, it does not create one.
  • Forgetting model.eval() after loading, then wondering why predictions change between calls.
  • Loading GPU-saved files on a CPU machine without map_location.
  • Disabling weights_only to silence an error on a file you did not create.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does model.state_dict() contain?

    It holds numbers only: parameters and buffers such as BatchNorm running statistics. The class itself is not stored.
  2. 2. Why is loading an untrusted pickle-based checkpoint dangerous?

    Pickle rebuilds objects by calling functions named in the file. weights_only=True restricts loading to plain data types.
  3. 3. Which of these is needed in a checkpoint to resume Adam training faithfully?

    Adam's running averages affect every update. Restarting them from zero changes the next steps, as the NumPy comparison showed.
  4. 4. You saved a model on a GPU machine. What should you pass to torch.load on a laptop with no GPU?

    map_location remaps tensors saved on a CUDA device onto the CPU. Without it, loading tries to restore them on the GPU and fails.

Summary

  • A state_dict maps names to tensors; save model.state_dict(), rebuild the model in code, then load_state_dict.
  • Pickle-based loading can execute code: use weights_only=True for your own plain checkpoints and safer formats for files from others.
  • A resumable checkpoint also holds the optimizer (and scheduler) state, the epoch and any configuration.
  • Call model.eval() for inference and use map_location when moving between devices.
  • TorchScript and ONNX export the computation for non-Python consumers; always verify outputs after exporting.

Next, see how to run all of this on faster hardware in GPUs, Devices and Mixed Precision.