Skip to content
Tutorials Center
My learning

Topics

Lesson 12 of 39 ยท Cells and Magics

Line and Cell Magics

Learn IPython magic commands in Jupyter: line magics with %, cell magics with %%, the most useful built-ins, help with ? and ??, and how to write your own magic.

Magic commands, or just magics, are special instructions that the IPython kernel understands in addition to normal Python. They start with a percent sign and give you shortcuts for things that are awkward in plain code: timing a statement, running a script, listing variables, writing a file, switching a plotting backend, running a cell as Bash. Magics are not Python syntax, so they work only in IPython-based kernels, and exporting a notebook to a .py file turns them into calls such as get_ipython().run_line_magic(...).

Magics need a real IPython kernel, which cannot run here. Examples with % are therefore read-only. The runnable stand-ins implement a small magic dispatcher in plain Python so you can see how a cell is split into magic and Python parts.

Line magics and cell magics

  • A line magic starts with one % and applies to the rest of that line:

    Python
    %pwd
    %timeit sum(range(1000))
    files = %ls
    

    A line magic can appear anywhere a statement can, and its return value can be assigned, as in the last line.

  • A cell magic starts with %%, must be the first line of the cell, and applies to the whole cell:

    Python
    %%time
    total = 0
    for i in range(1_000_000):
        total += i
    

By default automagic is on, which means you may leave out the % of a line magic when no variable has the same name (pwd works, but %pwd is safer, because a variable called pwd would shadow it).

List what is available:

Python
%lsmagic        # every line and cell magic
%magic          # a long manual

Getting help: ? and ??

Not a magic but just as useful. Add ? after a name to see its documentation in a pane, and ?? to also see its source code if Python has it:

Python
len?
import statistics
statistics.mean??
%timeit?

Put the cursor inside a call and press Shift+Tab for the same help as a tooltip (see Modes and keyboard shortcuts).

The built-in magics you will use most

Magic What it does
%who, %whos List variables in the namespace; %whos adds types and sizes.
%pwd, %cd path, %ls Show, change and list the working directory.
%run script.py Run a Python file (or another notebook) in the current namespace.
%load script.py Replace the cell's content with a file's source.
%%writefile name.py Write the cell's text to a file instead of running it.
%env NAME=value Read or set environment variables of the kernel.
%history -n 1-5 Print past inputs.
%reset Delete all variables (asks first; -f forces).
%store name Save a variable so another notebook can restore it with %store -r.
%matplotlib inline Show Matplotlib figures in the notebook (see Plots in notebooks).
%autoreload 2 After %load_ext autoreload: reload changed modules automatically.
%%time, %timeit, %%timeit, %prun, %lprun Time and profile (Timing and profiling).
%%bash, %%sh, !cmd Run shell commands (Shell commands).
%%html, %%javascript, %%latex, %%markdown Render the cell as that kind of content (Rich output).
%%capture out Capture the cell's output into a variable instead of showing it.

%load_ext name loads an extension that adds more magics, for example %load_ext autoreload or %load_ext sql (from the jupysql package).

Two magics worth a closer look

%%writefile turns a notebook into a small code workshop: write a module, then import it.

Python
%%writefile helpers.py
def square(x):
    return x * x
Python
%load_ext autoreload
%autoreload 2
from helpers import square

Edit helpers.py (in an editor, or by running the %%writefile cell again) and the next call uses the new code without restarting the kernel. This is the bridge between notebooks and a real code base: functions get a proper home as modules, and the notebook imports them.

%%capture hides noisy output but keeps it for later:

Python
%%capture log
print("lots of messages")
Python
log.show()       # print it now
log.stdout       # the text as a string

How a magic is dispatched (stand-in)

Before executing a cell, IPython looks at each line. A first line starting with %% makes the whole cell a cell magic; lines starting with % become line magics; everything else is Python. The following stand-in implements three magics in about thirty lines so you can watch the dispatching:

Python Stand-in: a tiny magic dispatcher

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Writing your own magic

You can register a function as a magic. With IPython you decorate it; the stand-in mirrors the same registry idea, so you can run the logic here, and the real decorator is shown below it:

Python Stand-in: a magic registry with decorators

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

In a real notebook the equivalent is:

Python
from IPython.core.magic import register_line_magic

@register_line_magic
def shout(line):
    return line.upper() + "!"

and then %shout good morning returns GOOD MORNING!.

Portability warning

Magics make notebooks convenient but they are not plain Python. If you export to a script (jupyter nbconvert --to script), magic lines are translated to get_ipython() calls that only work inside IPython. Prefer ordinary functions for logic you want to reuse, and keep magics for interactive conveniences and setup.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What distinguishes a cell magic from a line magic?

    A cell magic begins with %% on the first line of the cell and acts on all the lines below.
  2. 2. What does %%writefile helpers.py do?

    The cell content is saved to a file; combined with %autoreload you can import and refine it.
  3. 3. How do you see the documentation and source of a function named f?

    f? shows the docstring and f?? also shows the source when available.
  4. 4. Why can exporting a notebook with magics to a .py script cause trouble?

    Magics are IPython features, not Python syntax, so the exported script needs IPython to run them.

Summary

  • Magics are IPython commands: % for one line, %% for the whole cell (first line only).
  • ? and ?? show help and source; Shift+Tab shows a tooltip.
  • Everyday magics: %who, %pwd, %run, %load, %%writefile, %env, %history, %store, %%capture.
  • %load_ext adds extensions, and %autoreload 2 lets you edit modules without restarting.
  • You can register your own magics with register_line_magic and its cell and combined variants.
  • Magics are not plain Python, so keep reusable logic in functions and modules.

Next: Rich output and display