Skip to content
Tutorials Center
My learning

Topics

Lesson 32 of 39 ยท Best Practices

Moving Code from Notebooks into Modules

Turn repeated notebook code into functions and importable .py modules: when to extract, project layout, autoreload, editable installs and keeping notebooks thin.

Notebooks are excellent for exploring and explaining. They are poor places to keep logic you rely on: code in cells cannot be imported by another notebook, is hard to test, hides in diffs, and tends to get copied and pasted between files until nobody knows which copy is right. The remedy is simple and powerful: let notebooks call code that lives in modules.

This lesson shows when and how to move code out, a layout that works, and the tools that make the round trip comfortable. The runnable cells practice the refactoring itself: from a long cell to a function, and then to a module written and imported inside the Pyodide sandbox.

The signals that it is time

  • You copy a cell from one notebook into another.
  • A cell is longer than a screen, or has a block you can describe with a verb ("clean the dates").
  • You want to test the logic, or run it from a script or a scheduled job.
  • The same bug has to be fixed in more than one place.

Start with functions inside the notebook. Only when a function is stable, or needed elsewhere, does it graduate to a file.

Step 1: from cell to function

Here is typical exploratory code:

Python
df = pd.read_csv("orders.csv")
df["date"] = pd.to_datetime(df["date"])
df = df[df["amount"] > 0]
df["month"] = df["date"].dt.to_period("M")
monthly = df.groupby("month")["amount"].sum()

Turn the repeated, meaningful part into a function with explicit inputs and outputs. Avoid reading globals and avoid mutating the argument.

Python Extract a pure function

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The function takes data in and returns a new value. It has a docstring and handles the empty case. You can now call it twice, test it, and read the notebook as "totals = monthly_totals(orders)".

Step 2: a project layout

A common layout keeps notebooks and code side by side:

TEXT
project/
  pyproject.toml
  src/
    sales/
      __init__.py
      cleaning.py
      reports.py
  notebooks/
    01-explore.ipynb
    02-report.ipynb
  tests/
    test_cleaning.py
  data/            # raw data, often not in Git
  README.md

Install your package in editable mode inside the project's environment, so notebooks can import sales from anywhere:

Terminal
pip install -e .

With pyproject.toml configured for the src layout, edits to src/sales/cleaning.py take effect immediately, with no copy step. This is far better than sys.path.append("../src") at the top of every notebook, which breaks when the notebook moves.

Step 3: autoreload while you work

Python caches imports for the life of the kernel. To see edits to your module without restarting, add to the first notebook cell:

Python
%load_ext autoreload
%autoreload 2
from sales.cleaning import monthly_totals

Reload mode 2 re-imports changed modules before each cell. It is a convenience for development; for a final run, always restart and run all (see hidden state).

A module and its import, runnable

The sandbox here has no editable installs, but we can write a real module into a temporary folder and import it, which is exactly what happens on your machine.

Python Create a module and import it

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The second call shows the edited behavior after importlib.reload, the mechanism behind %autoreload.

Keep notebooks thin

After the move, the notebook should read like a report:

Python
from sales.cleaning import load_orders, monthly_totals
from sales.plots import revenue_chart

orders = load_orders("data/orders.csv")
totals = monthly_totals(orders)
revenue_chart(totals)

Each line is a step, and the narrative text between cells explains what it means. The logic is tested in tests/ (see testing notebook code) and the notebook adds the story.

Good function design for notebooks

  • Pass data in, return data out. Do not rely on globals such as df defined in another cell.
  • Do not print inside functions that compute; return values and let the notebook display them.
  • Parameters over constants. monthly_totals(rows, min_amount=0) is more reusable than a hardcoded threshold.
  • One job per function, and a name that is a verb phrase.
  • Add a docstring with one line saying what it returns.
  • Plotting functions return the figure (fig), so the notebook can show or save it.

What to leave in the notebook

Not everything should move. Exploratory scratch work, one-off charts, the narrative text and decisions for this specific analysis belong in the notebook. Moving a function early freezes it; moving it too late leaves duplication. A good rule: on the second copy, extract.

Tools that help

  • jupytext pairs a notebook with a .py file, and some teams develop in the percent-format script and open it as a notebook (see version control).
  • Editors (VS Code, PyCharm) can extract functions from selected code and open notebooks with the same tooling as scripts.
  • ruff or black format and lint modules; nbqa runs them on notebooks.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why is code in modules easier to maintain than code in notebook cells?

    Modules work with imports, test runners and version control diffs; cell code does not.
  2. 2. What does pip install -e . do for a project?

    Editable installs link to the source tree, so notebooks import the current code.
  3. 3. What does %autoreload 2 do?

    It reloads edited modules so you need not restart the kernel while developing.
  4. 4. Which function design is best for reuse?

    Explicit inputs and outputs make a function independent of notebook state.

Summary

  • Extract repeated or testable code: first to functions, then to modules.
  • Use a src/ layout and pip install -e ., not path hacks; use autoreload while developing.
  • Functions take data in and return data out; notebooks stay thin and tell the story.
  • Extract on the second copy, not before.

Next: Testing Notebook Code