Skip to content
Tutorials Center
My learning

Topics

Lesson 11 of 49 ยท Array Math

Vectorized Operations and ufuncs

Replace Python loops with NumPy vectorized arithmetic and universal functions: np.add, sqrt, exp, log, rounding, the out= argument and in-place operators.

If you come from plain Python, your first instinct for "do this to every number" is a for loop. NumPy offers something better: you write the operation once and it is applied to every element by fast compiled code. This style is called vectorization, and it is the single biggest reason NumPy programs are both shorter and much faster than loop-based ones.

Behind the operators sits a family of functions called universal functions, or ufuncs. a + b is really np.add(a, b). This lesson covers elementwise arithmetic, the most useful ufuncs, rounding, the out= argument for saving memory, in-place operators, and a fair comparison of loops against vectorized code. Combining arrays of different shapes is the subject of the next lesson, Broadcasting.

Elementwise arithmetic

Operators on arrays work element by element, and the arrays must have compatible shapes (identical shapes for now). A scalar is combined with every element.

flowchart LR
    A["a = [1 2 3]"] --> P["a + b"]
    B["b = [10 20 30]"] --> P
    P --> R["[11 22 33]"]

Elementwise addition pairs the elements at the same position; no loop appears in your code.

Python Arithmetic on whole arrays

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Note that a * b multiplies matching elements. Matrix multiplication has its own operator, @, covered in Vectors, Dot Products and the @ Operator.

Operators are ufuncs

Every arithmetic operator has a named ufunc behind it. The names are useful because ufuncs take extra arguments (like out=) and have extra methods such as reduce and accumulate.

Operator Ufunc
a + b np.add(a, b)
a - b np.subtract(a, b)
a * b np.multiply(a, b)
a / b np.divide(a, b)
a ** b np.power(a, b)
a % b np.mod(a, b)
-a np.negative(a)
Python Operators and ufunc names

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Common mathematical ufuncs

NumPy ships ufuncs for most of the maths you would otherwise import from the math module, but they work on whole arrays and are not limited to single numbers.

Group Functions
Roots and powers np.sqrt, np.cbrt, np.square, np.power
Exponentials np.exp, np.expm1, np.log, np.log2, np.log10, np.log1p
Sign and size np.abs, np.sign, np.negative
Trigonometry np.sin, np.cos, np.tan, np.arctan2, np.deg2rad
Python Mathematical ufuncs

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The last line produces nan (not a number) instead of an exception, because the square root of a negative real number is undefined. Depending on settings, NumPy may also print a warning; see Comparing Arrays and Floating Point for how to control that.

Rounding

Four ufunc-style functions turn floats into whole-number floats, and they differ for negative numbers and halves. np.round (also np.around) rounds to the nearest value and sends exact halves to the nearest even number, so 0.5 becomes 0 and 1.5 becomes 2.

Python floor, ceil, trunc and round

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The results stay as floats (-3. rather than -3). If you need integers afterwards, convert them with astype(int); see Converting and Casting Types.

The out= parameter

Normally a + b allocates a fresh array for the result. Every ufunc accepts out=, naming an existing array to write into instead. It saves allocations in tight loops and is the building block of in-place operators.

Python Writing into an existing array

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The final example fails with a casting error: dividing integers gives floats, which cannot be stored safely in an integer array. The out array must have a dtype that can hold the answer.

In-place operators

+=, -=, *= and friends modify an array without creating a new one. They are out= in operator form.

Python In-place versus new array

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

In-place operators save memory on large arrays, but they affect every name and view that shares the data (see Views vs Copies), and they refuse to silently change the dtype.

Vectorized versus loop

A Python loop over a NumPy array is slow: every iteration creates a Python object and goes through the interpreter. A vectorized expression runs one tight loop in compiled code. You can confirm two things without any timing: the results are the same, and the vectorized code is much shorter.

Python Same result, one line

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

To measure the speed difference yourself, run this on your own machine (timings depend on the computer, so we do not run it here):

Python
import timeit
import numpy as np

a = np.arange(1_000_000)

loop = timeit.timeit(lambda: [x * 2 for x in a], number=5)
vec = timeit.timeit(lambda: a * 2, number=5)
print(f"loop: {loop:.3f}s  vectorized: {vec:.4f}s")

On a typical laptop the vectorized line is around 50 to 100 times faster. Vectorized code also tends to be easier to read, because it states the formula rather than the bookkeeping.

Tip: If you find yourself writing for i in range(len(a)) over a NumPy array, stop and ask whether a ufunc, a mask or an aggregation can do the job. Loops belong in the places where each step depends on the previous result.

Trigonometry, hyperbolic functions and angles

All the trigonometric ufuncs work in radians. Convert with np.deg2rad (alias np.radians) and np.rad2deg (alias np.degrees). Two functions deserve special attention because they save you from classic mistakes:

  • np.arctan2(y, x) returns the angle of the point (x, y) in the full range from -pi to pi. Plain np.arctan(y / x) cannot tell (1, 1) from (-1, -1), and it divides by zero when x is 0.
  • np.hypot(a, b) computes sqrt(a**2 + b**2) without overflowing for huge inputs. It is the length of the hypotenuse, or of a 2-D vector.
Python Angles, arctan2 and hypot

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

np.unwrap is the tool for phase angles that wrap around. It adds multiples of 2*pi so that consecutive values never jump by more than pi, which makes a rotation measured every few milliseconds a smooth curve instead of a saw tooth.

The hyperbolic functions np.sinh, np.cosh and np.tanh and their inverses np.arcsinh, np.arccosh and np.arctanh follow the same pattern. np.tanh squeezes any number into the range (-1, 1) and is a classic activation function in neural networks; see the neural network project.

Careful exponentials and logarithms

For tiny numbers, np.exp(x) - 1 loses almost all its precision, because exp(x) is very close to 1 and the subtraction cancels the digits that matter. np.expm1(x) computes the same value accurately, and np.log1p(x) does the same for log(1 + x). Finance and statistics code uses them for small rates and probabilities.

np.logaddexp(a, b) computes log(exp(a) + exp(b)) without overflow, which is the safe way to add probabilities that are stored as logarithms.

Python expm1, log1p and logaddexp

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Rounding, signs and division

Besides the four rounding functions from the section above, three more are worth knowing. np.rint rounds to the nearest integer (ties to even, like np.round), np.fix rounds toward zero like trunc, and np.sign returns -1, 0 or 1. np.heaviside(x, h0) is the step function: 0 for negative numbers, 1 for positive numbers and h0 at exactly zero.

Division has more variants than you might expect:

  • np.floor_divide(a, b) (a // b) rounds the quotient toward minus infinity.
  • np.remainder(a, b) (a % b) takes the sign of the divisor, matching Python.
  • np.fmod(a, b) takes the sign of the dividend, matching C.
  • np.divmod(a, b) returns both the quotient and the remainder in one call.
  • np.float_power(a, b) always computes in float64, so integer powers never overflow silently the way np.power does with int64.
Python Remainders, divmod and power

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Putting it together, the angle between a compass heading and a destination, the length of a displacement and a clean sign for each step come from three calls: arctan2, hypot and sign. Reach for the specialised function before you write a formula by hand.

Reciprocals, powers of two and element-wise minimum and maximum

A few more small ufuncs round out the family:

  • np.reciprocal(a) computes 1 / a for every element. With a float array it is the same as 1 / a. With an integer array it truncates the result toward zero, so np.reciprocal(np.array([1, 2, 4])) is [1, 0, 0]. Convert to float first when you want fractions. A zero input gives inf (with a divide-by-zero warning) for floats.
  • np.exp2(x) computes 2 ** x. It is exact for whole-number exponents and handy for bit counts and octaves.
  • np.maximum(a, b) and np.minimum(a, b) compare two arrays (or an array and a scalar) element by element and keep the larger or smaller value at each position. This is different from a.max(), which collapses one array to a single number. The pair broadcasts, so np.maximum(a, 0) is the classic ReLU, and np.minimum(np.maximum(a, lo), hi) is what np.clip(a, lo, hi) does in one call.
  • If either value is nan, maximum and minimum return nan. Their siblings np.fmax and np.fmin ignore a nan and return the other value instead.
Python reciprocal, exp2, maximum and minimum

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Real or complex?

NumPy treats complex numbers as a normal dtype (complex128). Three helpers tell you what you have: np.isreal(z) and np.iscomplex(z) test each element (an element counts as real when its imaginary part is exactly zero), while np.isrealobj(z) and np.iscomplexobj(z) look at the dtype of the whole array. np.real(z) and np.imag(z) extract the two parts, and np.real_if_close(z) drops the imaginary part when it is only rounding noise, which is common after FFTs and eigenvalue calculations.

Python isreal, iscomplex and real_if_close

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

np.isreal answers "is this value real?" for each element, so an array with the dtype complex128 can still return True for some positions. np.isrealobj answers "is the dtype real?" for the whole array, so it returns False for z above even though three of its values have no imaginary part.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does a * b do when a and b are NumPy arrays of the same shape?

    * is elementwise (it calls np.multiply). Matrix multiplication uses the @ operator or np.matmul.
  2. 2. What is np.round(np.array([0.5, 1.5, 2.5]))?

    NumPy rounds exact halves to the nearest even number, so 0.5 becomes 0, 1.5 becomes 2 and 2.5 becomes 2.
  3. 3. What is the purpose of the out= argument of a ufunc?

    out= supplies the destination array, which avoids a temporary allocation. The array must be able to hold the result dtype.
  4. 4. Why does counts += 0.5 fail when counts is an integer array?

    In-place operators keep the array's dtype and refuse a result that would need an unsafe cast, raising a casting error.

Summary

  • Vectorization applies an operation to a whole array in compiled code, replacing explicit Python loops.
  • Operators such as +, * and ** are ufuncs (np.add, np.multiply, np.power) that work elementwise.
  • Common ufuncs include sqrt, exp, log, abs, sign, and the trigonometric functions; invalid inputs give nan or inf.
  • floor, ceil and trunc differ for negatives; np.round sends halves to the nearest even number.
  • out= and in-place operators write into existing arrays and need a compatible dtype.
  • Vectorized code is shorter and typically orders of magnitude faster than a loop over elements.

Next: combine arrays of different shapes in Broadcasting.