Skip to content
Tutorials Center
My learning

Topics

Lesson 14 of 39 ยท Cells and Magics

Timing and Profiling Code

Measure and speed up code in notebooks: %time versus %timeit, how timeit repeats runs, profiling with %prun and cProfile, line profiling and memory tools, plus honest benchmarking habits.

"Which version is faster?" is a question you ask often in a notebook, and guessing is a bad way to answer. IPython gives you magics that measure it for you: %time for one run, %timeit for a careful repeated measurement, and %prun and friends to find where the time goes. This lesson explains each, shows what they do under the hood using standard Python modules you can run here, and ends with rules for benchmarks you can trust.

The magics themselves run only in a real IPython kernel, so they appear in read-only blocks. The runnable stand-ins use time, timeit and cProfile from the standard library, which are exactly what the magics wrap. Timings differ from machine to machine, so we print results that do not depend on speed (such as which option is faster) or label them as examples.

%time and %%time: one run

Python
%time sum(range(1_000_000))

prints CPU time and wall time (the time on a clock) of a single execution, for example "CPU times: user ... Wall time: ...". The cell version times the whole cell:

Python
%%time
data = [i * i for i in range(1_000_000)]
total = sum(data)

One run is noisy: the first run may include caches warming up, another program may steal the CPU. Use it for slow code (seconds), where noise matters less.

%timeit and %%timeit: many runs

Python
%timeit sum(range(1_000_000))

%timeit repeats the statement many times, picks a suitable loop count automatically (so one measurement takes a fair fraction of a second), repeats that several times, and reports the mean and standard deviation per loop, for example 25.1 ms ยฑ 0.4 ms per loop (mean ยฑ std. dev. of 7 runs, 10 loops each). Options:

Option Meaning
-n 100 Number of loops per run
-r 5 Number of runs
-o Return the result object (so you can use .best, .average)
-q Quiet: do not print

The cell version times the whole cell body. Any code you write on the same line as the magic, after the options, is setup: it runs once and is not timed:

Python
%%timeit -n 100 -r 5 numbers = list(range(1000))
sorted(numbers, reverse=True)

Here only the sorted call is measured; building the list is excluded. Without the setup part, everything in the cell would be timed, including building the data.

What %timeit does: the timeit module

Under the hood %timeit uses Python's timeit module. You can use it in any script, which makes the stand-in below directly useful in your own projects. Here we compare building a list with a loop against a comprehension and report which is faster:

Python Stand-in: compare two implementations with timeit

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Why the minimum? Noise can only make a run slower, never faster than the true cost, so the minimum is the best estimate of the code's real speed. The mean and standard deviation that %timeit prints tell you how steady the measurement was.

A hand-made %timeit makes the mechanics clear:

Python Stand-in: write your own mini %timeit

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Finding the slow part: %prun

Timing tells you how long; profiling tells you where. %prun runs a statement under Python's profiler, cProfile, and shows a table of functions sorted by time:

Python
%prun -l 10 -s cumulative my_pipeline()

-s cumulative sorts by cumulative time (including called functions) and -l 10 keeps ten lines. Columns: ncalls (number of calls), tottime (time inside the function itself), cumtime (including sub-calls), percall. The functions at the top are your suspects.

The stand-in profiles a small program, and we print the call counts, which are deterministic, rather than the times:

Python Stand-in: profile with cProfile and read call counts

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Expect 300 calls each to parse and score and one to pipeline. In a real profile you would then look at which of them has the largest tottime, and optimise only that.

Line by line: %lprun and memory

Two extensions give finer tools. Both are installed separately (python -m pip install line_profiler memory_profiler), then loaded with %load_ext:

Python
%load_ext line_profiler
%lprun -f score pipeline(lines)       # time per line inside function `score`

%load_ext memory_profiler
%memit big = [0] * 10_000_000         # peak memory of a statement

%lprun marks each line with its share of the time, which finds a single expensive line inside a long function. %memit reports memory in MiB; use it when kernels die from running out of memory (see Restart, interrupt and debug).

Benchmarking habits that give honest numbers

  1. Compare like with like: the same inputs, same size, same machine, nothing else heavy running.
  2. Measure before optimising. The slow part is rarely where you think. Profile first.
  3. Beware the first run: imports, caching and JIT warm-up slow it. %timeit handles this by repeating.
  4. Do not time print or plotting unless that is the thing you want to measure; output rendering in a notebook is slow.
  5. Use realistic data sizes. An algorithm that wins at 100 items may lose at 10 million.
  6. State your findings in a Markdown cell next to the numbers: results without context age badly (see Cell types and Markdown).

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why is %timeit usually better than %time for fast statements?

    One run of a fast statement is too noisy. %timeit chooses a loop count and repeats the measurement.
  2. 2. Which tool tells you which function in a pipeline takes the most time?

    A profiler shows per-function call counts and times, pointing to what to optimise.
  3. 3. Why is the minimum of several timing runs a good estimate of the code's real speed?

    Other processes and cache effects add time but never subtract it, so the fastest run is least disturbed.
  4. 4. You want line-by-line timing inside one function. What do you use?

    %lprun -f function statement reports time per line of that function.

Summary

  • %time and %%time measure a single run; %timeit and %%timeit repeat and give mean ยฑ standard deviation.
  • %timeit wraps the standard timeit module, which you can use in scripts too.
  • %prun (cProfile) shows where time goes by function; %lprun shows it by line; %memit shows memory.
  • Use the minimum of several runs when comparing implementations.
  • Profile before optimising, use realistic data, and do not time printing.
  • Write down what you measured and what you concluded.

Next: Shell commands and the system