Skip to content
Tutorials Center
My learning

Topics

Lesson 26 of 39 ยท Collaboration and Deployment

Version Control for Notebooks

Why .ipynb files make noisy Git diffs, and how to fix it: strip outputs with nbstripout, review changes with nbdime, pair notebooks with scripts using Jupytext.

Git is built for line-based text, and a notebook is a JSON document that mixes your code with outputs, images and bookkeeping. Commit a notebook naively and every run produces a huge, unreadable diff: execution counts change, a plot is re-encoded into a different base64 blob, and a single edited line becomes hundreds of changed lines. Teams then stop reviewing notebooks, which is the opposite of what version control is for.

This lesson explains what is inside an .ipynb file, why it diffs badly, and three practical fixes. Git, nbstripout and friends cannot run on this site, so those commands are read-only; the runnable cells work on a notebook as a Python dictionary, which is exactly what the tools do.

What an .ipynb file contains

A notebook is JSON with a list of cells and some metadata. Here is a minimal one:

JSON
{
  "nbformat": 4,
  "nbformat_minor": 5,
  "metadata": {
    "kernelspec": {"name": "python3", "display_name": "Python 3"},
    "language_info": {"name": "python", "version": "3.12.4"}
  },
  "cells": [
    {
      "cell_type": "code",
      "execution_count": 3,
      "id": "a1b2c3d4",
      "metadata": {},
      "source": ["print(1 + 1)"],
      "outputs": [
        {"output_type": "stream", "name": "stdout", "text": ["2\n"]}
      ]
    }
  ]
}

The parts that change without you editing any logic are execution_count, outputs and parts of metadata. The parts that carry your work are the cells' source and the Markdown text.

Python Read a notebook as data

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice the execution counts 4 and 7. If you open and re-run the notebook, they change, and so does the file, even when no code changed.

Fix 1: do not commit outputs

The simplest fix is to commit notebooks without outputs. A tool called nbstripout removes outputs and execution counts when you commit, using a Git filter, so your working copy keeps results while the repository does not.

Terminal
pip install nbstripout
nbstripout --install        # registers a Git filter for this repository

Alternatively use pre-commit with an nbstripout or nbQA hook, or clear outputs from the interface before committing (Edit, Clear Outputs of All Cells).

The trade-off: readers on GitHub no longer see results. If the outputs are the point (a report, a tutorial), commit an exported HTML or PDF next to the notebook (see exporting with nbconvert), or keep outputs and accept noisier diffs.

Python What nbstripout does (stand-in)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The real tool does the same and also removes some volatile metadata. Run it before every commit and a re-run of the notebook no longer shows up as a change.

Fix 2: review with nbdime

nbdime understands the notebook structure. It shows a diff cell by cell, with source changes, output changes and metadata changes separated, and it can merge notebooks in a three-way merge without corrupting the JSON.

Terminal
pip install nbdime
nbdime config-git --enable --global     # use nbdime for .ipynb in git diff and merge
nbdiff old.ipynb new.ipynb              # terminal diff
nbdiff-web old.ipynb new.ipynb          # visual diff in a browser

After config-git, git diff notebook.ipynb prints a readable summary instead of raw JSON. The JupyterLab Git extension (see JupyterLab extensions) can also show nbdime diffs in the interface.

Fix 3: pair the notebook with a text file (Jupytext)

Jupytext keeps a plain-text twin of every notebook: a Python script, a Markdown file, or a percent-format script. You commit and review the text file, which diffs like ordinary code, and the .ipynb is regenerated when needed.

Terminal
pip install jupytext
jupytext --set-formats ipynb,py:percent notebook.ipynb   # pair them
jupytext --sync notebook.ipynb                            # update the other file

The percent format looks like this:

Python
# %% [markdown]
# # Sales report

# %%
import pandas as pd
df = pd.read_csv("sales.csv")

Each # %% line starts a cell. Opening the .py file in JupyterLab (right-click, Open With, Notebook) shows it as a notebook. Code reviewers see a small, ordinary diff.

Python Convert cells to percent format (stand-in)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

A workable team policy

  1. Choose: outputs-free .ipynb (nbstripout) or paired text files (Jupytext). Do not mix per person.
  2. Add the tool to pre-commit so nobody forgets.
  3. Use nbdime for review and for merge conflicts.
  4. Pull requests change cells, not outputs. Put long-lived results in exported files.
  5. Keep notebooks small. Move reusable code into modules (see moving code into modules); those are easy to diff, test and review.

Merge conflicts

Two people editing the same notebook produce JSON conflicts that cannot be fixed by hand reliably. With nbdime installed as the merge driver, Git merges cell by cell, and only genuine conflicts remain, shown in a web view. With Jupytext, conflicts occur in the text file and are resolved like normal code.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why do notebooks produce noisy Git diffs?

    The file mixes code with volatile run data, so re-running creates changes even without edits.
  2. 2. What does nbstripout do?

    It is a Git filter that keeps the repository copy free of outputs.
  3. 3. Which tool shows a cell-aware diff and can merge notebooks?

    nbdime understands notebook structure for diffs and three-way merges.
  4. 4. What does Jupytext give you?

    You commit and review a .py or .md file kept in sync with the .ipynb.

Summary

  • An .ipynb file is JSON with code, outputs and volatile metadata; it diffs poorly by default.
  • Strip outputs with nbstripout, review with nbdime, or pair notebooks with text via Jupytext.
  • Automate with pre-commit and agree one policy for the team.
  • Move reusable code out of notebooks into modules.

Next: Exporting with nbconvert