Skip to content
Tutorials Center
My learning

Topics

Lesson 60 of 62 ยท Practice and Review

Common Mistakes and Debugging

The most common linear algebra bugs in NumPy: shape errors, * versus @, misusing inv, comparing floats with ==, and eigenvector sign ambiguity, with fixes.

Most linear algebra bugs in code are not deep. The math is fine; the program is quietly doing something slightly different from what you meant. A product that should be a matrix product is elementwise. A vector that should be a column is a flat array. A check that should allow rounding error demands exact equality. This lesson collects the mistakes that show up again and again, shows each one failing in a live editor, and gives you a fix and a habit.

Many of these bugs do not raise an error at all, which is what makes them dangerous. The best defense is a short debugging routine: print shapes, test small cases by hand, check results against a defining equation, and compare floats with a tolerance.

Five classic traps

Wrong shapes Flat arrays versus columns.
  • is not @
Elementwise is not the matrix product.
Inverse habit Solve systems, do not invert.
Float equality Rounding breaks ==.

Four of the five give wrong answers without any error message.

1 / 2

Mistake 1: shape confusion

NumPy has three different things that all look like "a vector": a flat array of shape (3,), a column of shape (3, 1), and a row of shape (1, 3). Linear algebra on paper has only columns. In code, mixing them up causes errors, or worse, silent broadcasting.

The classic silent bug: subtracting a flat array from a column. Broadcasting happily produces a 3 by 3 matrix instead of a vector of three differences.

Python Flat arrays versus columns

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The habit: print .shape whenever a result looks odd, and decide early whether your vectors are flat or columns. Flat arrays are fine for @ with a matrix (A @ x works with x of shape (n,) and returns shape (m,)), but be careful when you combine them with columns through + or -.

Mistake 2: * instead of @

In NumPy, * multiplies elementwise. The matrix product is @ (or np.matmul). When both matrices have the same shape, using * raises no error, so the bug hides.

Python Elementwise versus matrix product

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The identity test is a nice sanity check: if your "product" with the identity matrix changes the matrix, you used *. The same trap exists for powers: A ** 2 squares each entry, while np.linalg.matrix_power(A, 2) is A @ A.

Mistake 3: reaching for the inverse

Textbooks write x = A^-1 b, so code often says np.linalg.inv(A) @ b. That is almost always the wrong tool. Computing the inverse costs more, is usually less accurate, and for a singular matrix it either raises an error or returns enormous garbage values. np.linalg.solve(A, b) solves the same system directly, using an LU factorization (see Solving Systems with NumPy).

Python solve instead of inv

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

When you truly need the inverse itself (rare), compute it, but check np.linalg.cond(A) first. When you need A^-1 times many different vectors, factor once with LU instead of inverting.

Mistake 4: comparing floats with ==

Floating point numbers are binary approximations. Results that are equal in exact arithmetic often differ in the last bit. This breaks ==, np.array_equal and assert x == y for computed values.

Python Never compare computed floats with ==

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Use np.allclose(a, b) or np.isclose for comparisons, and judge "is this zero" by a tolerance that fits your problem. Functions like matrix_rank already do this for you: they count singular values above a tolerance rather than testing for exact zeros. Tests of the form det(M) == 0 are doubly fragile, because even the sign of a tiny determinant is meaningless.

Mistake 5: expecting a unique eigenvector

If v is an eigenvector of A, so is -v, and so is 5v. NumPy returns eigenvectors normalized to length 1, but the sign is chosen by the algorithm and may differ between libraries, versions or machines. A test that expects a specific sign, or a plot that depends on it, is fragile. The same is true for the columns of U and rows of V^T in the SVD, which can flip together.

Python Eigenvectors are only defined up to sign

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Compare eigenvectors through a sign-independent quantity, such as abs(v1 @ v2) (which is 1 when two unit vectors are parallel or opposite), or normalize the sign first as above. Also remember that np.linalg.eig makes no promise about the order of the eigenvalues; sort them yourself if the order matters, or use eigh, which returns them ascending.

Other bugs worth knowing

  • Integer dtypes. np.array([[1, 2], [3, 4]]) has an integer dtype. Most linalg functions convert to float, but your own in-place operations (like A[0] /= 3) will fail or truncate. Create matrices with floats when you will do row operations.
  • Using eig on a symmetric matrix. It works, but eigh is faster, returns real values, and guarantees orthonormal eigenvectors. With eig, tiny complex parts can appear in the output.
  • Forgetting the transpose of Vt. np.linalg.svd returns V^T, not V. The reconstruction is U @ np.diag(s) @ Vt, with no extra transpose.
  • Views and copies. Slicing a matrix gives a view. If you modify a slice during elimination, you modify the original too. Use .copy() when you need to keep the input.
  • np.matrix. An old class with different * semantics; avoid it and use plain arrays.

A debugging routine

When a result looks wrong, go through the same steps each time:

  1. Print the shape and dtype of every input.
  2. Shrink the problem to 2 by 2 or 3 by 3 and compute the answer by hand.
  3. Check the defining equation: A @ x against b, A @ v against lambda * v, Q.T @ Q against I, U @ S @ Vt against A.
  4. Compare with np.allclose, never with ==.
  5. If the numbers look huge or unstable, check np.linalg.cond(A).

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. A has shape (3, 3) and B has shape (3, 3). What does A * B compute?

    The * operator is always elementwise in NumPy arrays. The matrix product is A @ B.
  2. 2. Why is np.linalg.solve(A, b) usually preferred over np.linalg.inv(A) @ b?

    Both work with floating point, but solve uses a factorization directly, costs less and loses less accuracy. Neither gives exact results.
  3. 3. How should you test whether a computed matrix equals the identity?

    Rounding error makes exact equality unreliable. allclose accepts small differences. A determinant of 1 does not imply the identity either.
  4. 4. np.linalg.eigh returns an eigenvector v for a matrix. Which statement is true?

    Any nonzero multiple of an eigenvector is an eigenvector. The sign NumPy returns is an implementation detail, so do not rely on it.
  5. 5. You compute np.linalg.svd(A) and get U, s, Vt. What is the correct reconstruction?

    NumPy already returns V transposed, so no extra transpose is needed. Mixing this up is a very common bug.

Summary

  • Print shapes. Flat (n,), column (n, 1) and row (1, n) behave differently, and broadcasting can hide mistakes.
  • * and ** are elementwise; use @ and np.linalg.matrix_power for matrix products and powers.
  • Solve systems with np.linalg.solve; avoid inv, and check np.linalg.cond for sensitive problems.
  • Compare floats with np.allclose; judge zeros by a tolerance, which is what matrix_rank does.
  • Eigenvectors (and SVD vectors) are defined only up to sign; normalize the sign or compare something sign-independent.
  • Verify every decomposition by reconstructing the matrix and checking the defining equation.

Next, test yourself on the classic questions in Interview Questions.