Skip to content
Tutorials Center
My learning

Topics

Lesson 39 of 49 ยท Performance and Internals

Advanced Techniques

Go beyond the basics: apply_along_axis, frompyfunc, vectorize, ufunc.reduce, accumulate, outer and at, polynomials with poly1d and vander, and cumulative tricks.

Once you know slicing, broadcasting and the usual aggregations, a second layer of NumPy tools lets you solve problems that look awkward at first. How do you apply your own function to every row? How do you build a running maximum without a loop? What happens when an index appears twice in an assignment? How do you fit and evaluate a polynomial?

This lesson is a toolbox of those techniques. Each one is short, but together they let you stay in array code in many situations where a loop looks tempting. We will also be honest about which tools are real speed-ups and which are only conveniences, using what you learned in writing fast NumPy code.

Applying your own function along an axis

np.apply_along_axis(func, axis, arr) takes every 1D slice of arr along axis, passes it to func, and stacks the results. It is the straightforward answer to "I have a function that works on one row; run it on all rows."

np.frompyfunc(func, nin, nout) and np.vectorize(func) solve a different problem: calling a function written for scalars on whole arrays, with broadcasting. frompyfunc is the bare tool and always returns arrays of dtype object; np.vectorize wraps it, guesses an output dtype (or uses otypes=) and adds options such as signature for functions that take small arrays.

Python apply_along_axis, frompyfunc and vectorize

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Key points from the output:

  • apply_along_axis returns one entry per slice. If func returns an array (like top2), that array becomes a row of the result.
  • The results of frompyfunc have dtype object, so convert with .astype(int) if you want a numeric array.
  • With signature="(n)->()", np.vectorize hands your function a whole 1D vector and expects one number back, so it works like apply_along_axis with automatic broadcasting.

All three tools call your Python function once per element or slice, so they are conveniences, not accelerators. If a built-in does the same thing, such as grid.max(axis=1) - grid.min(axis=1) for the spread, use the built-in; it is vastly faster.

Ufunc methods: reduce, accumulate, outer, at

Every binary ufunc, such as np.add, np.multiply or np.maximum, has extra methods. They turn a simple operation into a family of useful ones.

Method What it does Example with np.add on [1, 2, 3, 4]
reduce Combine all elements into one value 10
accumulate Keep every intermediate result [1, 3, 6, 10]
reduceat Reduce over slices that start at given indices
outer Apply to all pairs of two arrays a 4 by n table
at Unbuffered in-place operation at indices see below
flowchart LR
    A["np.add on [1, 2, 3, 4]"] --> R["reduce<br/>10"]
    A --> C["accumulate<br/>[1, 3, 6, 10]"]
    A --> O["outer with [10, 20]<br/>table of all pair sums"]

One ufunc, three methods: collapse to a single value, keep the running values, or combine every pair.

np.add.reduce(a) is what a.sum() calls, and np.multiply.accumulate is np.cumprod. The value of the method form is that it works for any ufunc: np.maximum.accumulate is a running maximum, np.logical_or.reduce is any, np.gcd.reduce is the greatest common divisor of a whole array.

Python reduce, accumulate, reduceat and outer

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

outer builds a two-dimensional table from two one-dimensional arrays, exactly like the broadcasting trick a[:, np.newaxis] * b, but with any ufunc and without having to remember newaxis (see broadcasting). reduceat computes several reductions over consecutive chunks in a single call, which is handy for grouped sums when the groups are contiguous.

Repeated indices and ufunc.at

There is one famous trap. Buffered fancy-index assignment does not accumulate when an index repeats:

Python counts[idx] += 1 versus np.add.at

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

counts[idx] += 1 is evaluated as read, add, write: it reads counts[idx], adds 1 to the copy, and writes back, so a repeated index is written several times with the same value. np.add.at applies the operation once for every occurrence. It is correct but slower than np.bincount, which is the best choice for plain counting. Use ufunc.at for operations bincount cannot do, such as np.maximum.at.

Polynomials

NumPy has two polynomial APIs. The older one is np.poly1d together with np.polyfit, np.polyval and np.roots, which store coefficients with the highest power first. The newer numpy.polynomial package (class Polynomial) stores coefficients with the lowest power first and is numerically better behaved. Mixing the two orders is a classic source of bugs, so check twice.

Python poly1d, polyfit, Polynomial and vander

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

np.vander builds the Vandermonde matrix, whose columns are powers of the input. Fitting a polynomial is nothing more than solving a least-squares problem with that matrix, which is what the last lines do by hand and what solving linear systems explains. For high degrees (above roughly 10), plain power bases become numerically unstable. Use Polynomial.fit, which rescales the input automatically, or a Chebyshev basis from the same package.

Polynomial arithmetic, roots and fitting in detail

The first example showed the basics. Here are the operations you need when polynomials are the actual subject: adding, dividing, building a polynomial from its roots, judging a fit, and switching to a better-conditioned basis.

The function forms np.polyadd, np.polysub, np.polymul, np.polydiv, np.polyder and np.polyint take plain coefficient arrays (highest power first) and return arrays, which is handy when you do not want a poly1d object. np.poly(roots) goes the other way and returns the coefficients of the monic polynomial that has those roots. np.roots(coeffs) finds the roots as the eigenvalues of the companion matrix, so complex roots appear when needed.

Python Arithmetic, division, roots and residuals

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

What to take from it:

  • np.polydiv returns two arrays: the quotient and the remainder. A remainder of zero means the divisor is a factor (here x - 1 divides p exactly because 1 is a root).
  • A higher degree always lowers the training error, but degree 8 on 30 noisy points is overfitting: it follows the noise. Judge a fit on data it has not seen, for example with a train and test split as in the linear regression project.
  • Polynomial.fit maps your x values into a standard window (-1 to 1) before solving, which keeps the maths stable. .convert() converts back to the original x units. The Chebyshev class gives the same curve but with a basis whose terms stay well-behaved even at high degrees.
  • np.polyfit also takes w= weights and cov=True to return the covariance matrix of the coefficients.

Cumulative tricks

cumsum, diff and maximum.accumulate are surprisingly powerful when combined. diff and cumsum are inverses of each other, which makes them ideal for range sums, moving sums and finding where something changes.

Python Prefix sums, running records and run lengths

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The prefix-array idea answers many range questions with one pass over the data and constant time per query. The run-length example works by finding the positions where consecutive values differ (diff != 0), then measuring the gaps between those positions. If you want windows without a cumulative sum, np.lib.stride_tricks.sliding_window_view gives free overlapping views; its rules and dangers are in memory layout and strides.

Choosing the right tool

  • A built-in with an axis argument exists? Use it. It is the fastest and clearest.
  • Need a running result or a table of all pairs? Use a ufunc method (accumulate, outer).
  • Counting with repeated indices? np.bincount, else np.add.at.
  • A scalar Python function must run over an array and you can't rewrite it? np.vectorize or frompyfunc, accepting the loop cost.
  • A function that works on one row at a time? np.apply_along_axis.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What is the dtype of the array returned by np.frompyfunc(f, 2, 1)(x, y)?

    frompyfunc always produces object arrays. Convert with astype if you need a numeric dtype.
  2. 2. idx = np.array([1, 1, 1]); counts = np.zeros(3, int); counts[idx] += 1. What is counts?

    The right-hand side reads counts[idx] once, adds 1 to each copy, and writes back the same value three times. Use np.add.at or np.bincount to count repeats.
  3. 3. Which expression gives the running maximum of an array a?

    accumulate keeps every intermediate result of repeatedly applying np.maximum.
  4. 4. np.poly1d([2, -3, 1]) represents which polynomial?

    poly1d takes coefficients from the highest power down, so [2, -3, 1] is 2x^2 - 3x + 1. The newer Polynomial class uses the opposite order.
  5. 5. Why is np.apply_along_axis(np.sum, 1, grid) a poor choice compared with grid.sum(axis=1)?

    apply_along_axis is a Python-level loop over slices. The built-in axis argument runs in compiled code.

Summary

  • np.apply_along_axis runs a function over every 1D slice; np.frompyfunc and np.vectorize call scalar functions on arrays. They are conveniences with loop-like speed, and frompyfunc returns object arrays.
  • Every binary ufunc has reduce, accumulate, reduceat, outer and at; they give running maxima, all-pairs tables and more.
  • a[idx] += 1 does not count repeated indices. Use np.bincount or np.add.at.
  • np.poly1d and np.polyfit use highest-power-first; numpy.polynomial.Polynomial uses lowest-power-first. np.vander builds the matrix used for polynomial fitting.
  • cumsum, diff and maximum.accumulate combine into prefix sums, moving sums, running records and run-length encodings.

Next: Common Pitfalls and Debugging Arrays