Skip to content
Tutorials Center
My learning

Topics

Lesson 38 of 49 ยท Performance and Internals

Writing Fast NumPy Code

Make NumPy code fast: replace Python loops, preallocate, use in-place operations and smaller dtypes, avoid temporaries, measure with timeit and chunk big data.

NumPy is fast because its loops run in compiled C code over a block of memory, not because the Python you write around it is clever. Fast NumPy code therefore means one thing above all: keep the work inside NumPy calls and out of the Python interpreter. A single a + b on a million numbers takes about a millisecond. A Python for loop doing the same job takes around a hundred times longer.

This lesson collects the habits that matter: removing loops, allocating memory once, choosing a smaller dtype, avoiding hidden temporary arrays, and measuring correctly so you do not optimise the wrong thing. It ends with chunking for data that does not fit in memory and a short, read-only look at Numba and Cython for the rare cases where array code is not enough.

Measure first, then change

Guessing where code is slow is a reliable way to waste an afternoon. Write the obvious version, check that it is correct, find out whether it is actually too slow, and only then optimise. Use timeit for small snippets (in a notebook, the %timeit magic does the same). Follow these rules:

  • Repeat and take the minimum. Other programs, garbage collection and CPU frequency changes only ever make a run slower, so the fastest of several runs is the best estimate of the true cost.
  • Time the work, not the setup. Create the input arrays outside the timed function.
  • Use realistic sizes. A trick that wins on 100 elements can lose on 10 million, and the reverse.
  • Compare results too. A fast wrong answer is worthless; assert equality with np.allclose.

The timings you will see on your machine differ from the ones in books. The exercises below print comparisons (is A at least 20 times faster than B?) rather than raw seconds, because ratios are stable even when the clock is not.

flowchart TD
    A["Code is too slow"] --> B["Measure: where is the time spent?"]
    B --> C{"Python loop over array elements?"}
    C -->|"yes"| D["Vectorize with ufuncs, slicing, where"]
    C -->|"no"| E{"Many temporaries or huge copies?"}
    E -->|"yes"| F["Use out=, in-place ops, smaller dtype, chunks"]
    E -->|"no"| G{"Still too slow and not expressible as array code?"}
    G -->|"yes"| H["Numba, Cython or another compiled language"]
    G -->|"no"| I["Good enough: stop"]
    D --> B
    F --> B

A tuning loop: measure, fix the biggest cost, measure again.

Replace Python loops with array operations

The biggest win is nearly always to stop looping over elements. Compare a loop with np.dot, which sums the products of two arrays in C:

Python Loop versus vectorized sum of squares

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The last two lines surprise people. A for loop over a NumPy array is slower than over a list, because each step wraps a raw number in a new NumPy scalar object. If you must loop in Python, convert with .tolist() first. Better still, express the work with the tools from earlier lessons: ufuncs, broadcasting, boolean masks and aggregations. Typical rewrites:

Loop pattern Array version
if inside a loop np.where(cond, a, b) or a boolean mask
accumulate a total a.sum(), np.dot, np.cumsum
compare every pair broadcasting with newaxis
apply a function to every row an axis= argument

Preallocate instead of growing

np.append, np.concatenate and np.vstack create a brand new array every time and copy all the old data into it. Growing an array inside a loop therefore costs O(nยฒ) in total. Allocate the final array once with np.empty or np.zeros and fill it, or better, compute everything in one expression.

The out= parameter of every ufunc lets you reuse an existing array for the result. The in-place operators (+=, *=) do the same, and are worth understanding exactly.

Python Preallocation, out= and in-place operators

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice the last block. y *= 10 modifies the array that both y and alias point to, while y = y * 10 builds a new array and rebinds y. In-place operations save memory and time, but they also change data that other variables may share, including views (see views and copies). Do not use them on an argument that your caller still needs, and remember that an in-place operation cannot change the dtype: int_array *= 1.5 raises an error.

Temporary arrays

An expression like a * b + c first computes a * b into a hidden temporary array, then adds c into another new array. For small arrays that is invisible. For arrays of 100 million elements each temporary is 800 MB, and allocating and touching it costs real time, often more than the arithmetic itself.

Ways to reduce temporaries:

  • Use out= or in-place operators on a buffer you own, as above.
  • Process the data in chunks (see below) so temporaries stay small and fit in the CPU cache.
  • Split a long expression into steps only when you reuse intermediate results.
  • For long chains of arithmetic, the optional numexpr library compiles the whole expression and evaluates it block by block.

NumPy also optimises some simple cases itself: it can reuse a temporary for chained operations when it is safe. Do not count on that for correctness, only for a small speed gain.

Choose the dtype deliberately

Memory traffic is a large part of the cost of array code. A smaller dtype moves fewer bytes, so it is often faster and it always fits more data in memory and in the cache. float32 halves the footprint of float64; uint8 is one eighth of int64.

Python Memory per dtype

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The last line is the price of small types: uint8 arithmetic wraps around silently (200 + 100 becomes 44). Use small dtypes for storage and when you know the value range, and widen before arithmetic that could overflow. float32 has only about 7 decimal digits of precision, which is plenty for images and many models but not for long financial sums. See data types and type conversion for the rules.

The np.vectorize myth

np.vectorize has a name that promises speed, but the documentation says plainly that it is a convenience for broadcasting, not a performance tool. It still calls your Python function once per element.

Python np.vectorize is a loop in disguise

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

All three give the same answers, but np.vectorize takes about as long as the explicit loop, while the real array expression is dozens of times faster. Use np.vectorize when the function is too complicated to express with array operations and you want automatic broadcasting; never use it to make code fast.

Chunking large data

When the data is larger than memory, or when temporaries would be huge, process it in blocks and combine partial results. Many statistics can be built from running sums: the mean needs the sum and the count, the variance also needs the sum of squares.

Python Mean and standard deviation in chunks

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Only 100,000 values exist in memory at any moment, yet the result matches the all-at-once computation. For arrays stored on disk, np.load(..., mmap_mode="r") gives a memory-mapped array you can slice in blocks like this (see saving and loading arrays). A chunk size of a few hundred thousand elements is usually a good balance between Python overhead and cache efficiency.

The running-sum formula for the standard deviation is simple but can lose precision when the mean is huge compared to the spread. For serious work, combine chunk means and variances with the pairwise update formula or use Welford's algorithm.

When array code is not enough

Some algorithms are inherently sequential: each step depends on the previous result (a recursive filter, a state machine, a tight numeric loop with many branches). You cannot express them with a ufunc. Then you have options that keep Python as the language:

  • Numba compiles a Python function with a decorator, working directly on NumPy arrays:
Python
# Read-only: Numba is not available in this page's runner
from numba import njit

@njit
def running_max_drop(prices):
    peak = prices[0]
    worst = 0.0
    for p in prices:
        if p > peak:
            peak = p
        worst = min(worst, p / peak - 1.0)
    return worst

The first call compiles; later calls run at C speed.

  • Cython lets you add C type declarations to Python-like code and compile it into an extension module.
  • Rewriting the hot part in C, C++ or Rust and calling it through a binding is the heaviest option, used by many libraries.

Try them last. First check whether scipy, pandas or a different algorithm already does the job, because a better algorithm beats a faster loop every time.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Why is growing an array with np.append inside a loop slow?

    NumPy arrays have a fixed size. Appending builds a new array and copies the old one into it, so n appends cost O(n squared) in total.
  2. 2. What does np.vectorize do for performance?

    np.vectorize is a convenience wrapper for broadcasting. The loop still runs in Python, so it is about as slow as an explicit loop.
  3. 3. Which line avoids creating a new result array?

    out= tells the ufunc to write the result into an existing array instead of allocating one.
  4. 4. You time a function once and get 12 ms, then again and get 9 ms. What is the best practice?

    Interference only makes runs slower, so the minimum of several repeats is the closest estimate of the code's true cost.
  5. 5. What is a risk of storing data as uint8 to save memory?

    Small integer types wrap around on overflow without any error, so 200 + 100 becomes 44 in uint8.

Summary

  • Fast NumPy code keeps the loops inside compiled calls: ufuncs, slicing, where, aggregations and np.dot.
  • Measure with timeit, repeat, take the minimum, and compare ratios and results, not single runs.
  • Preallocate (np.empty, np.zeros) or compute in one expression; never grow arrays with np.append in a loop.
  • out= and in-place operators avoid temporary arrays, but they change shared data and cannot change the dtype.
  • Smaller dtypes save memory and bandwidth, but watch for overflow and precision.
  • np.vectorize is for convenience, not speed.
  • Process huge data in chunks and combine partial sums; use Numba or Cython only for inherently sequential code.

Next: Advanced Techniques