Skip to content
Tutorials Center
My learning

Topics

Lesson 5 of 38 ยท Linear Algebra

Matrices, Solving Systems and Inverses

Learn scipy.linalg basics: solve linear systems, compute norms and determinants, understand why you should not invert matrices, and read the condition number.

Linear algebra is the engine room of scientific computing. Fitting a line to data, simulating a bridge, ranking web pages and training a model all end up asking the same question: find the vector x that satisfies A x = b. scipy.linalg gives you fast, reliable solvers built on the LAPACK library for exactly this.

In this lesson you will solve linear systems, compute norms and determinants, see when an inverse is a bad idea and learn the number that tells you how much to trust a solution: the condition number. You should be comfortable with NumPy arrays.

scipy.linalg versus numpy.linalg

NumPy ships a small numpy.linalg module, and SciPy has a bigger scipy.linalg. Both call LAPACK. SciPy's version is a superset: it always uses LAPACK, offers many extra functions (decompositions, matrix functions, special matrices), and lets you pass hints such as assume_a="pos" for symmetric positive-definite matrices so it can pick a faster algorithm. For simple cases they return the same numbers, so use scipy.linalg whenever you are already importing SciPy.

Solving A x = b

linalg.solve(A, b) returns the vector x. If b is a 2-D array, each column is a separate right-hand side and you get one solution column for each.

Python Solve a linear system

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The check A @ x reproducing b is the best habit you can build: always look at the residual b - A x. A small residual shows the solver did its job. (It does not by itself prove x is accurate; the condition number covers that.)

Behind the scenes solve does not compute any inverse. It factors the matrix into triangular pieces and substitutes, which we study in the decompositions lesson.

Norms and determinants

A norm measures the size of a vector or matrix with one number. linalg.norm(v, ord) supports several kinds:

  • ord=2 (default for vectors): the usual length, the square root of the sum of squares.
  • ord=1: the sum of absolute values. For matrices, the largest column sum.
  • ord=np.inf: the largest absolute value for vectors, the largest row sum for matrices.
  • For matrices the default is the Frobenius norm (square root of the sum of all squared entries), while ord=2 is the spectral norm, the largest singular value.

The determinant linalg.det(A) is a single number that is zero exactly when the matrix is singular, meaning it has no inverse. In practice, do not use it to test for singularity: it can be huge or tiny for perfectly fine matrices. Use the condition number instead.

Python Norms, determinant and a singular matrix

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The second matrix has a second row that is exactly twice the first, so its rows are not independent. It has determinant 0 and inv and solve raise LinAlgError.

Why you should not invert

Textbooks write x = A^-1 b, and beginners copy that into code as linalg.inv(A) @ b. Avoid it. Computing the inverse costs about three times more work than solving, and it is usually less accurate, because it finds all n columns of the inverse when you only wanted one answer. You also lose the ability to exploit structure.

The rule: never form an inverse just to multiply by it. Call solve. If you need the same matrix with many right-hand sides, factor it once (see the next lesson). Only compute inv(A) when you truly need the inverse's entries, which is rare.

The condition number

Some systems are intrinsically touchy: tiny changes in the data cause large changes in the answer. The condition number cond(A) measures this. As a rule of thumb, you lose about log10(cond(A)) decimal digits of accuracy when solving. A double has roughly 16 digits, so a condition number near 1e16 means the answer may have no correct digits at all.

The Hilbert matrix is the classic ill-conditioned example. linalg.hilbert(n) builds it.

flowchart LR
    A["Input data<br/>b with tiny error"] --> S["Solve A x = b"]
    C["cond(A) large"] --> S
    S --> E["Error in x<br/>up to cond(A) times larger"]

A large condition number amplifies small input errors into large errors in the solution.

Python Hilbert matrices: solve versus inv

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

We built b from a known answer of all ones, so we can measure the error exactly. Even with the best algorithm, the 10-by-10 Hilbert matrix (condition number around 1e13) destroys most of the digits: both errors are large. The lesson is not that solve is perfect, but that the matrix itself is the problem, and no algorithm can fix that. When the condition number is huge, reformulate the problem (different basis, regularization) instead of hoping.

A smaller version of the same effect appears with nearly identical rows:

Python A nearly singular system

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The two right-hand sides differ in the fifth significant digit, yet the solutions are [1, 1] and [0, 2]: completely different. That is what a condition number of 4e4 looks like.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which call is the preferred way to get x from A x = b?

    solve factors the matrix and substitutes. It is faster and usually more accurate than forming the inverse.
  2. 2. What does a large condition number tell you?

    The condition number is the amplification factor for errors. It says nothing about size or symmetry, and it can be huge even when the determinant is not zero.
  3. 3. What is the default linalg.norm(M) for a 2-D matrix?

    With no ord argument, a matrix norm is the Frobenius norm, the square root of the sum of squared entries.
  4. 4. What happens when you call linalg.solve on an exactly singular matrix?

    solve raises LinAlgError for a singular matrix. For rank-deficient problems use least squares, covered in a later lesson.

Summary

  • scipy.linalg wraps LAPACK and is a superset of numpy.linalg.
  • solve(A, b) solves linear systems, accepts several right-hand sides and takes structure hints such as assume_a="pos".
  • Check the residual A @ x - b; use norm with the right ord to measure vectors and matrices.
  • Do not invert a matrix just to multiply by it.
  • The condition number predicts how many digits you lose; the determinant is not a good singularity test.

Next: LU, QR, Cholesky and Schur Decompositions.