Skip to content
Tutorials Center
My learning

Topics

Lesson 8 of 39 ยท Notebooks and Kernels

The .ipynb File Format

Open a Jupyter notebook file and read it: the nbformat 4 JSON structure, cell types, output types, metadata, cell ids, and how to inspect or clean a notebook with Python.

A Jupyter notebook is not a binary file. It is plain JSON text with the extension .ipynb (originally "IPython notebook"). Knowing its structure lets you read a notebook in any text editor, write a script that inspects or cleans notebooks, understand why notebooks produce noisy diffs in Git, and see where an outputs or metadata problem comes from.

The format is specified as nbformat; the current major version is 4. The Python package nbformat reads, validates and writes it. In this lesson we use only Python's standard json module, which is enough to work with the structure.

The examples build real nbformat 4 documents in plain Python and process them. They run on this site because they need no Jupyter software, only JSON.

The top level

A notebook has four top-level keys:

Key Meaning
nbformat Major version, 4
nbformat_minor Minor version, such as 5 (adds required cell ids)
metadata Notebook-wide information: kernel, language, authors
cells The list of cells in order

A typical metadata section stores the kernelspec (which kernel the notebook asked for) and language_info (details of the language the last kernel reported):

JSON
"metadata": {
  "kernelspec": {
    "display_name": "Python 3 (ipykernel)",
    "language": "python",
    "name": "python3"
  },
  "language_info": {
    "name": "python",
    "version": "3.12.3"
  }
}

The name (python3) is the id of an installed kernel; it is what Choosing and managing kernels lets you register. If a colleague does not have a kernel with that name, Jupyter asks them to pick another one.

Cells

Each element of cells is an object with a cell_type of markdown, code or raw.

A Markdown cell

JSON
{
  "cell_type": "markdown",
  "id": "7f1c2a9e",
  "metadata": {},
  "source": ["# Title\n", "Some text."]
}

A code cell

JSON
{
  "cell_type": "code",
  "execution_count": 3,
  "id": "b81d0c34",
  "metadata": {},
  "outputs": [],
  "source": ["total = 2 + 3\n", "total"]
}

Things to notice:

  • source is either one string or a list of lines. Lists are the default because each line ends with \n, which makes line-based diffs more readable.
  • id is a unique short string for each cell (required since nbformat 4.5). It lets tools refer to a cell by something other than its position.
  • execution_count is the number in In [3]. It is null if the cell never ran.
  • metadata is a free dictionary for tags ("tags": ["parameters"]), collapsed state, and extension data.

Outputs

The outputs list of a code cell holds the results from the last run. Every output has an output_type:

output_type Content
stream Text from print: name (stdout or stderr) and text
execute_result The last expression's value: data (a MIME bundle), metadata, execution_count
display_data Rich output shown with display(), plots: data and metadata
error An exception: ename, evalue and traceback (a list of strings with colour codes)

A MIME bundle maps content types to representations of the same object, for example:

JSON
"data": {
  "text/plain": "   a  b\n0  1  3\n1  2  4",
  "text/html": "<table>...</table>",
  "image/png": "iVBORw0KGgo... (base64 text)"
}

The front end picks the richest type it can show. Because images are stored as base64 text inside the JSON, a notebook with many plots can reach many megabytes, and every re-run changes those long text lines. That is the main reason why diffs of notebooks are noisy, and why teams often strip outputs before committing (see below).

flowchart TD
    NB["Notebook (.ipynb)"] --> META["metadata: kernelspec, language_info"]
    NB --> CELLS["cells[]"]
    CELLS --> MD["markdown: source"]
    CELLS --> CODE["code: source, execution_count, outputs[]"]
    CODE --> O1["stream"]
    CODE --> O2["execute_result"]
    CODE --> O3["display_data"]
    CODE --> O4["error"]

The notebook tree: a JSON document with a clear hierarchy.

Build and read a notebook

The next stand-in assembles a notebook with a printed output, a result and an image placeholder, then reads it back the way a tool would. Everything is plain JSON:

Python Stand-in: write a notebook and summarise it

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Cleaning outputs

Tools such as nbstripout and the menu Edit โ†’ Clear Outputs of All Cells remove outputs, which gives small diffs and avoids committing data by accident (a printed password, a table of personal records). The logic is straightforward, as you can see:

Python Stand-in: strip outputs and execution counts

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Reading and validating with real tools

With the nbformat package installed in your environment you can read and check notebooks:

Python
import nbformat

nb = nbformat.read("analysis.ipynb", as_version=4)
nbformat.validate(nb)                    # raises ValidationError if the structure is wrong
for cell in nb.cells:
    print(cell.cell_type, len(cell.source))
nbformat.write(nb, "copy.ipynb")

This block is read-only on the site. nbformat.validate checks the document against the JSON schema, which is a good step in a build pipeline.

Version control and security notes

  • JSON with base64 images makes Git diffs unreadable. Options: strip outputs on commit, use nbdime (a notebook-aware diff and merge tool), or pair the notebook with a plain-text version using Jupytext.
  • Outputs can leak secrets or personal data. Clear them before publishing.
  • Notebooks can contain HTML and JavaScript in outputs. Jupyter marks a notebook as trusted only if it was last saved by you; for untrusted notebooks it refuses to run output scripts. Never click "Trust" on a file you did not examine.
  • Metadata is shared by every cell and the notebook; tags in it drive tools like papermill (covered later in the reproducibility chapter).

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What kind of file is an .ipynb notebook?

    Notebooks are JSON text defined by the nbformat specification.
  2. 2. Where do the results of the last run of a code cell live in the file?

    Each code cell has an outputs list holding stream, execute_result, display_data and error items.
  3. 3. Why are notebook diffs in Git often noisy?

    Embedded outputs and counters change with every run, so even tiny edits produce large diffs.
  4. 4. What does execution_count: null mean for a code cell?

    null appears when the cell has not been executed, or after Clear Outputs.

Summary

  • A notebook is JSON with nbformat, nbformat_minor, metadata and cells.
  • Cells are markdown, code or raw; code cells carry source, execution_count, outputs and metadata.
  • Outputs are stream, execute_result, display_data or error; rich data is a MIME bundle, with images as base64.
  • Outputs make files big and diffs noisy, so strip them or use nbdime or Jupytext.
  • You can read and clean notebooks with plain json, or validate them with the nbformat package.
  • Never trust a notebook you have not inspected: outputs can contain scripts.

Next: Choosing and managing kernels