Skip to content
Tutorials Center
My learning

Topics

Lesson 6 of 39 ยท Notebooks and Kernels

Kernels Explained

Understand the Jupyter kernel: the separate process that runs your code, how the front end talks to it with messages, what lives in kernel memory, and kernel lifecycle.

When you press Shift+Enter, something has to actually run your Python. That something is the kernel: a separate program that waits for code, executes it, keeps all your variables in memory, and sends the results back. The notebook in your browser is only a display and editing surface. This split is the most important architectural idea in Jupyter, and understanding it explains many behaviours that otherwise feel mysterious: why variables survive between cells, why closing a tab does not stop code, and why a notebook can use a different language or even a different computer.

Jupyter and real kernels cannot run on this site. The code below simulates the protocol with plain Python dictionaries so you can see what travels between the front end and the kernel. The message names and fields follow the real Jupyter messaging specification, but the stand-in does not use sockets.

Front end, server and kernel

flowchart LR
    UI["Front end: JupyterLab or Notebook (browser)"]
    SRV["Jupyter Server"]
    K1["Kernel A: Python 3.12"]
    K2["Kernel B: R"]
    UI <-->|"WebSocket"| SRV
    SRV <-->|"ZeroMQ sockets"| K1
    SRV <-->|"ZeroMQ sockets"| K2

One server can host many kernels, each started by a different notebook.

  • The front end shows cells and displays results. It does not understand Python.
  • The server manages files, sessions and kernels. It translates between WebSocket messages from the browser and ZeroMQ messages to kernels.
  • The kernel is a normal operating-system process. For Python it is usually ipykernel, which wraps the Python interpreter and adds IPython features (magics, rich display, tab completion).

Each open notebook is attached to its own kernel by a session. Two notebooks do not share variables unless you attach both to the same kernel on purpose.

The messaging protocol

Every interaction is a message: a JSON-like object with a header, a parent header (which request this answers), metadata and content. When you run a cell, this exchange happens:

  1. The front end sends an execute_request with the cell's code.
  2. The kernel publishes status: busy.
  3. It publishes execute_input (echoing the code and counter).
  4. While running, it may publish stream (text printed), display_data (rich output such as an image) or error.
  5. If the cell's last line was an expression, it publishes execute_result.
  6. It publishes status: idle and sends back an execute_reply saying ok or error.

Because the kernel publishes messages as they happen, you see printed output appear line by line while a long cell still runs.

Here is a stand-in kernel that produces the same sequence of message types. It really executes the code with Python's exec, and wraps the output in messages:

Python Stand-in: a toy kernel and its messages

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The same kernel object lives on between calls, so x defined in the first request would still exist in a third. That persistent namespace is exactly the "kernel memory".

What the kernel remembers

The kernel holds:

  • every variable, function, class and imported module you created;
  • open files, database connections and running threads;
  • large objects such as DataFrames or trained models;
  • the history of inputs and outputs (In and Out lists, and the special names _, __, ___ for the last outputs in IPython kernels).

It does not remember anything once it stops. The notebook file keeps your code and last outputs, but not the objects.

Python Stand-in: kernel memory exists only while the process lives

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Kernel lifecycle

A kernel goes through these states, shown by the small circle next to the kernel name:

State Meaning
Starting The process is being launched.
Idle Ready for the next cell.
Busy Running code (the prompt shows [*]).
Dead or Disconnected The process crashed or the connection to it broke.

Typical actions in the Kernel menu:

  • Interrupt sends SIGINT: it raises KeyboardInterrupt in your code, like pressing Ctrl+C in a terminal. Memory is kept.
  • Restart kills the process and starts a new one. All variables are lost, outputs stay in the file.
  • Restart and Run All does the above then executes every cell from the top.
  • Shut Down ends the kernel without starting a new one.
  • Change Kernel attaches the notebook to a different kernel, for example a different Python environment.

See Restart, interrupt and debug for what to do when a kernel hangs.

Why "which Python?" matters

The kernel is a Python process from one specific environment. If you installed pandas in a virtual environment but the notebook uses a kernel from another Python, import pandas fails even though it works in your terminal. You can check which interpreter the kernel uses:

Python
import sys
print(sys.executable)
print(sys.version)

The path printed tells you the environment. Choosing and managing kernels shows how to register kernels for each environment.

Kernels do not need to be local

Because the server talks to kernels through sockets, a kernel can run on another machine: a powerful remote server, a container, or a cloud service. Your browser only sends and receives small messages, while the heavy computation happens elsewhere. This is how hosted notebook services work.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What is a Jupyter kernel?

    The kernel is the computational engine. The front end only displays cells and results.
  2. 2. Which message type carries text that your code printed?

    print() output is published as stream messages. execute_result is the value of the final expression.
  3. 3. What does Restart do to variables?

    A restart creates a fresh process with empty memory. The notebook file keeps the old outputs as text.
  4. 4. Imports work in your terminal but fail in the notebook. What is a likely cause?

    The kernel runs one specific interpreter. If the package is installed in another environment, the import fails.

Summary

  • The kernel is a separate process that executes code and keeps variables in memory.
  • The browser front end, the Jupyter Server and the kernel communicate with messages (execute_request, stream, execute_result, status, execute_reply).
  • Kernel memory lives only while the process runs; the file keeps code and output text.
  • Interrupt keeps memory, Restart clears it, Shut Down ends the kernel.
  • The kernel's interpreter decides which packages you can import.
  • Kernels can run remotely because the protocol is just messages.

Next: Execution order and hidden state