Skip to content
Tutorials Center
My learning

Topics

Lesson 16 of 62 ยท Linear Systems

Solving Systems with NumPy

Use np.linalg.solve to solve Ax = b, learn why you should not compute the inverse, check results with residuals, and meet conditioning warnings.

You now know how to solve a system by hand: eliminate, reduce, read off the answer. In practice you let a library do it. NumPy's np.linalg.solve takes a square matrix A and a right-hand side b and returns the vector x with A @ x == b. Under the hood it runs the same elimination you have already seen, written in compiled code and organised as an LU decomposition.

This lesson covers the everyday workflow: calling solve, solving for many right-hand sides at once, checking that the answer is trustworthy, and understanding why textbooks write x = A^-1 b while real code almost never computes inv(A). You will also see your first example of a system that is nearly singular and what that does to the answer.

Calling np.linalg.solve

For a square system Ax = b, pass A with shape (n, n) and b with shape (n,), or (n, k) to solve k systems that share the same A.

Python Your first solve

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice np.allclose instead of ==. Floating-point answers are rarely bit-for-bit equal to the target, so you compare with a tolerance (see Common Mistakes for more on float equality).

solve insists on a square, non-singular A. A non-square A always raises numpy.linalg.LinAlgError. For an exactly singular A, desktop NumPy raises LinAlgError: Singular matrix, but some builds (including the in-browser one used by the examples here) quietly return nan instead. Never assume an error will warn you: check that the result is finite, or use scipy.linalg.solve, which reports singular matrices with an exception on every build.

Python When solve refuses

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

For a rectangular system (more equations than unknowns, the usual situation with real data) there is usually no exact solution, and you want the best compromise instead. That is least squares, solved with np.linalg.lstsq.

Choosing the right NumPy call

Square, full rank Use np.linalg.solve(A, b).
Tall or noisy Use np.linalg.lstsq(A, b).
Singular solve raises LinAlgError.

Reach for inv only when you truly need the matrix itself.

1 / 2

Why not compute the inverse?

On paper, x = A^-1 b is the cleanest way to write the solution. In code, np.linalg.inv(A) @ b does extra work: it solves n systems (one per column of the identity) to build the whole inverse, and then multiplies. The answer is the same in exact arithmetic, but with floats the inverse route is slower and typically a little less accurate.

Python solve versus inv

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Both errors are very small for this well-behaved random matrix (the exact digits depend on the library build); the gap grows for harder matrices. The inverse route also does roughly three times the arithmetic, which you notice on large systems. A simple rule of thumb: if you only need x, call solve. If you need to apply the same A to many right-hand sides, still call solve with a matrix B, or factor A once with LU. Compute inv(A) only when the inverse matrix itself is the thing you want, as in the next lesson.

Checking a solution: the residual

How do you know an answer is good? Compute the residual r = b - A @ x. A tiny residual means x satisfies the equations. It is the cheapest and most honest check you have, and you can run it on any solver's output.

Python Residual checks

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

A small residual proves that A @ x is close to b. It does not always prove that x is close to the true solution, and the next section shows why.

Conditioning: when the answer is fragile

Some matrices are almost singular. Their rows are nearly parallel, so tiny changes in b (even rounding error) cause large changes in x. The condition number measures this sensitivity: roughly, it is the factor by which relative errors in the input can be magnified in the output. np.linalg.cond(A) computes it. A condition number near 1 is perfect, and a value around 1e16 means double precision has nothing left to offer.

Python A nearly singular system

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

A shift of 0.0001 in one entry of b moved the solution from (1, 1) to (0, 2). That is the hallmark of an ill-conditioned system: the solver did nothing wrong, but the problem itself amplifies noise. If your real data have measurement error of that size, the "solution" is mostly noise. More on this in Numerical Linear Algebra Pitfalls.

NumPy only raises an error for exactly singular matrices. For a matrix that is singular up to rounding, solve may happily return enormous numbers, and sometimes LinAlgError appears and sometimes it does not. Check np.linalg.cond(A) when results look suspicious.

A tiny residual does not mean a trustworthy x. For an ill-conditioned A, many very different vectors have almost the same A @ x. Look at the condition number as well as the residual.

scipy.linalg.solve

SciPy offers scipy.linalg.solve, which does the same job with more options. The most useful one is assume_a, which tells the solver about the structure of A (for example "sym" or "pos" for symmetric positive-definite matrices) so it can use a faster, more specialised algorithm.

Python scipy.linalg.solve with structure

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Use NumPy's version for quick work and SciPy's when you know your matrix is symmetric, triangular or banded, or when you want reliable errors for singular matrices. Plain np.linalg.solve is enough for most of this course.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which call solves the square system Ax = b in the recommended way?

    solve factors A and substitutes, which is faster and usually more accurate than building the full inverse first.
  2. 2. What does np.linalg.solve do when A is exactly singular?

    An exactly singular matrix has no unique solution. NumPy raises LinAlgError or, in some builds, returns nan, so check for finite values. For least squares, use np.linalg.lstsq.
  3. 3. A solver returns x with a residual of about 1e-16, but cond(A) is about 1e15. What should you conclude?

    A tiny residual only says A @ x is close to b. With a huge condition number, very different vectors give nearly the same A @ x, so x can be inaccurate.
  4. 4. How should you pass three right-hand sides to np.linalg.solve with the same A?

    solve accepts B of shape (n, k) and returns X of the same shape, with column j solving A x = B[:, j].

Summary

  • np.linalg.solve(A, b) solves square, non-singular systems; pass a matrix B to solve several right-hand sides at once.
  • Prefer solve over inv(A) @ b: it is faster and generally more accurate.
  • Verify with the residual b - A @ x and np.allclose, never with ==.
  • The condition number np.linalg.cond(A) shows how much noise the problem amplifies; huge values mean the answer is fragile.
  • scipy.linalg.solve adds structure hints such as assume_a="pos"; rectangular systems belong to lstsq.

Next: learn what the inverse is, and when it is worth computing, in Inverse Matrices.