Skip to content
Tutorials Center
My learning

Topics

Lesson 20 of 39 ยท Reproducibility

Reproducible Environments for Notebooks

Give every notebook project its own virtual environment and register it as a kernel, so imports, versions and results stay the same on every machine and every day.

A notebook that works today and fails next month is almost always an environment problem, not a code problem. A library was upgraded, a different Python answered the call, or a package you installed "just once" is missing on your colleague's laptop. Reproducibility starts by making the environment a first-class part of the project instead of an accident of your machine.

This lesson shows how to create an isolated environment per project, connect it to Jupyter as a kernel, and prove inside the notebook which Python and which packages are really running. Jupyter itself cannot run on this site, so the commands below are read-only. The runnable examples are small Python stand-ins that teach the same checks.

One project, one environment

An environment is a folder that holds its own Python interpreter and its own installed packages. Two projects can then use different versions of the same library without fighting.

Terminal
# create and activate (macOS/Linux)
python -m venv .venv
source .venv/bin/activate

# Windows PowerShell
python -m venv .venv
.venv\Scripts\Activate.ps1

pip install jupyterlab pandas matplotlib

With conda the same idea reads:

Terminal
conda create -n sales-analysis python=3.12 pandas matplotlib jupyterlab
conda activate sales-analysis

Keep the environment folder next to the notebooks (.venv) and never commit it to Git. What you commit is a list of requirements, covered in the next lesson.

Kernels: how Jupyter finds your Python

A notebook does not run Python itself. It talks to a kernel, a separate process that executes your cells. Each kernel is described by a small kernel.json file (an interpreter path plus a display name). Jupyter lists the kernels it knows in the kernel picker.

flowchart LR
  A["Notebook (.ipynb)"] --> B["Jupyter server"]
  B --> C["Kernel: sales-analysis"]
  C --> D[".venv/bin/python"]
  D --> E["Packages in .venv"]

The notebook only stores the kernel name. Which interpreter that name points to is decided on your machine.

If you start Jupyter from inside an activated environment that has ipykernel installed, that environment's Python is available automatically. To make an environment selectable from a Jupyter you started elsewhere, register it:

Terminal
pip install ipykernel
python -m ipykernel install --user --name sales-analysis --display-name "Python (sales-analysis)"

jupyter kernelspec list     # shows every registered kernel and its folder
jupyter kernelspec remove sales-analysis   # unregister it again

Use python -m ipykernel, not a bare ipykernel command: the -m form guarantees the module comes from the Python you are currently using.

Prove which Python is running

When something is odd, ask the kernel directly. These two lines belong at the top of any notebook that has ever misbehaved:

Python
import sys
print(sys.executable)   # path of the interpreter running this kernel
print(sys.version)

If the path is not inside your project's .venv, the notebook is using the wrong kernel. Change it from the kernel menu (in JupyterLab: Kernel, Change Kernel).

Python Which Python is this? (runs in Pyodide)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

In a real virtual environment sys.prefix differs from sys.base_prefix, which is the standard test for "am I inside a venv". Here the interpreter is Pyodide (Python compiled for the browser), so the answer is just a demo of the calls.

Installing packages from a notebook

You will want to install a package without leaving the notebook. Use the magic %pip, not !pip:

Python
%pip install seaborn

%pip installs into the environment of the running kernel. !pip install runs whatever pip is first on the shell PATH, which may belong to a different Python. That mismatch is one of the classic causes of "I installed it but the import fails". After installing a package that was already imported, restart the kernel so the new version is loaded.

Do not leave %pip install cells scattered through a finished notebook. Installing belongs in the requirements file; the notebook should only import.

Check versions at the top

A tiny cell that prints the versions used documents the run and makes bug reports easier:

Python Print the versions a notebook depends on

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

importlib.metadata reads the installed package list without importing the packages themselves, so the cell is fast and safe. The NOT INSTALLED line shows what a missing dependency looks like before it turns into a confusing ModuleNotFoundError forty cells later.

A compact checklist

One environment per project

Create .venv Next to the notebooks
Register kernel ipykernel install
Verify Print sys.executable
1 / 2

Common mistakes

  • Installing Jupyter globally and every library globally too. Versions collide across projects.
  • Running pip in a terminal where a different environment is active than the kernel you selected.
  • Sharing a notebook that depends on a kernel name only your machine has. Others need to pick their own kernel, which is fine as long as the requirements file is in the repository.
  • Forgetting that conda and pip can both install into the same environment. Prefer one tool per environment, and use pip inside conda only for packages conda lacks.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does a Jupyter kernel do?

    The notebook is only a document. A kernel process runs the code and sends results back.
  2. 2. Why is %pip install preferred over !pip install inside a notebook?

    !pip runs the first pip on the shell PATH, which may belong to a different Python than the kernel.
  3. 3. Which command registers the current environment as a kernel?

    ipykernel's install command writes a kernel.json that points to the current interpreter.
  4. 4. How can you check which interpreter the kernel is using?

    sys.executable is the path of the Python running the cell.

Summary

  • Give each project its own virtual or conda environment and keep it out of Git.
  • A kernel connects a notebook to one interpreter; register environments with ipykernel.
  • Verify with sys.executable and a version printout at the top of the notebook.
  • Install with %pip, then restart the kernel if the package was already imported.

Next: Pinning Dependencies