Skip to content
Tutorials Center
My learning

Topics

Lesson 17 of 62 ยท Linear Systems

Inverse Matrices

Understand what a matrix inverse is, when it exists, how to compute it by hand and with np.linalg.inv, its algebra rules, and why singular matrices fail.

A number a has a reciprocal 1/a that undoes multiplication by a. A square matrix A can have a similar partner, the inverse A^-1, which undoes whatever A does to a vector. If A turns x into b, then A^-1 turns b back into x. This single idea explains why some linear systems have a unique solution, why a few have none, and why certain transformations can be reversed while others destroy information.

In this lesson you will define the inverse precisely, learn exactly when it exists, compute it by hand for a 2 by 2 matrix, use np.linalg.inv for larger ones, and memorise the rules for combining inverses. You will also see what goes wrong with a matrix that has no inverse. If you want to solve Ax = b, remember the advice from Solving Systems with NumPy: the inverse is a wonderful concept, but solve is usually the right tool.

Definition

A square matrix A is invertible (also called nonsingular) if there is a matrix A^-1 of the same size such that

A @ A^-1 = A^-1 @ A = I

where I is the identity matrix. When such a matrix exists it is unique. A matrix with no inverse is called singular.

Only square matrices can have a two-sided inverse. For a rectangular matrix, the closest relatives are one-sided inverses and the pseudoinverse that you will meet later.

Python Verifying an inverse

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The last two lines are the heart of it: multiply by A, then by A^-1, and you are back where you started. In particular, if Ax = b and A is invertible, then x = A^-1 b is the only solution.

When does the inverse exist?

For a square n by n matrix, all of these statements are equivalent. Either every one is true, or every one is false:

  • A is invertible.
  • Ax = b has exactly one solution for every b.
  • Ax = 0 has only the zero solution.
  • The columns of A are linearly independent.
  • The reduced row echelon form of A is the identity (there is a pivot in every column).
  • The rank of A is n.
  • The determinant of A is not zero.

This list is one of the central results of linear algebra. You can already justify some of it: if every column has a pivot, RREF reaches the identity, and the elimination steps are exactly the work of building the inverse.

One idea, many faces

Undoes A A^-1 reverses the transformation.
Unique answers Ax = b has exactly one solution.
Full rank A pivot in every column.
Nonzero det The volume is not flattened.
1 / 2

Computing the inverse by hand

For a 2 by 2 matrix there is a formula. If

A = [[a, b], [c, d]]

and ad - bc is not zero, then A^-1 = 1/(ad - bc) * [[d, -b], [-c, a]]. The number ad - bc is the determinant. When it is zero, the formula divides by zero, which matches the fact that the matrix is singular.

For larger matrices you use Gauss-Jordan elimination: write [A | I] and row-reduce until the left half becomes I. The right half is then A^-1. You saw a two-by-two version at the end of Reduced Row Echelon Form. Here is a general function.

Python Gauss-Jordan inverse from scratch

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice how the singular matrix is detected: after clearing the first column, no usable pivot remains in the second, so the loop stops. Checking for a "small" pivot with a tolerance, rather than comparing to exactly 0, is the standard defensive habit.

Why [A | I] works: augmented matrices and elementary matrices

The [A | I] method looks like a trick, so it deserves an explanation. An augmented matrix just writes two matrices side by side with a bar between them, so that every row operation you apply is applied to both halves at once. Each row operation (swap two rows, scale a row, add a multiple of one row to another) is the same as multiplying on the left by an elementary matrix: the identity matrix with that one operation applied to it.

Suppose a sequence of row operations Eโ‚, Eโ‚‚, ..., E_k turns A into I:

TEXT
E_k โ‹ฏ Eโ‚‚ Eโ‚ ยท A = I

The product E_k โ‹ฏ Eโ‚ is a matrix that, multiplied by A, gives I, which is exactly what the inverse is. So Aโปยน = E_k โ‹ฏ Eโ‚. And the right half of [A | I], which starts as I and receives the very same operations, becomes E_k โ‹ฏ Eโ‚ ยท I = Aโปยน. That is the whole proof. Two further consequences follow:

  • Every invertible matrix is a product of elementary matrices (invert each E_i and rearrange: A = Eโ‚โปยน Eโ‚‚โปยน โ‹ฏ E_kโปยน).
  • If the left half cannot be reduced to I (a zero row appears), A is singular and has no inverse.

The three kinds of elementary matrices, for a 3 ร— 3 matrix:

TEXT
swap rows 1 and 2:      scale row 2 by 5:       add 3 ร— row 1 to row 3:
[0 1 0]                 [1 0 0]                 [1 0 0]
[1 0 0]                 [0 5 0]                 [0 1 0]
[0 0 1]                 [0 0 1]                 [3 0 1]

Each one is easy to invert: swap back, scale by 1/5, subtract the multiple.

Python Record the elementary matrices and multiply them to get the inverse

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Each printed matrix is the augmented matrix [A | I] after one more row operation. The left half walks toward I and the right half ends as Aโปยน. The final check shows the product of all recorded elementary matrices is the inverse.

Rules for inverses

Assume A and B are invertible square matrices of the same size and c is a nonzero number.

Rule Meaning
(A^-1)^-1 = A Undoing an undo gives the original.
(AB)^-1 = B^-1 A^-1 The order reverses.
(A^T)^-1 = (A^-1)^T Transpose and inverse commute.
(cA)^-1 = (1/c) A^-1 Scalars come out as reciprocals.
det(A^-1) = 1 / det(A) Volume scaling is reversed.

The reversal in (AB)^-1 = B^-1 A^-1 has an everyday explanation: to undo "put on socks, then shoes" you take off the shoes first, then the socks. A common mistake is to write A^-1 B^-1, which is a different matrix in general.

Python Checking the inverse rules

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The third line prints False for these matrices, which confirms that the "wrong order" version really is wrong.

Singular matrices

What does it look like when a matrix has no inverse? Take the columns (1, 2) and (2, 4). The second is just twice the first, so the matrix squashes the whole plane onto a single line. Many different inputs map to the same output, so no function can recover the original input.

Python A matrix that cannot be undone

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Two different inputs, (2, 0) and (0, 1), both land on (2, 4). An inverse would have to send (2, 4) back to both of them at once, which is impossible.

np.linalg.inv does not always raise an error for a singular matrix. Depending on the build it may raise LinAlgError, return a matrix of nan, or, for matrices that are singular only up to rounding, return a matrix of gigantic numbers like 1e16. Check np.linalg.matrix_rank(A) or np.linalg.cond(A) first when you are unsure, and scipy.linalg.inv for a reliable exception.

Near-singular matrices and numerics

In floating point, "singular" is a spectrum. The matrix [[1, 2, 3], [4, 5, 6], [7, 8, 9]] is singular in exact arithmetic (its rows are in arithmetic progression), but the computer cannot see exact zero after rounding, and inv returns entries of size about 1e15. Those numbers are meaningless. The condition number tells you how close to singular a matrix is.

Python Reading the condition number

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

When should you compute the inverse at all? Typical honest reasons: you need the inverse matrix's entries (some statistics formulas use them), you are working out a small symbolic example, or you must apply the same inverse to a long stream of vectors where a stored factorization would not be convenient. For plain "solve this system", use np.linalg.solve. For many right-hand sides, use LU decomposition.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. If A and B are invertible, what is (AB)^-1?

    The inverse of a product reverses the order, like undoing shoes before socks.
  2. 2. Which of these guarantees that a square matrix A is NOT invertible?

    Rank below n means some column has no pivot, so the matrix squashes space and cannot be undone. Symmetric matrices can be either invertible or singular.
  3. 3. For A = [[a, b], [c, d]], when does the 2 by 2 inverse formula fail?

    The formula divides by the determinant ad - bc. A zero determinant means the matrix is singular.
  4. 4. Why is np.linalg.inv(A) @ b usually a worse way to solve Ax = b than np.linalg.solve(A, b)?

    Both agree in exact arithmetic, but building the full inverse costs extra and lets rounding errors accumulate. solve factors A once and substitutes.

Summary

  • A^-1 undoes A: A @ A^-1 = A^-1 @ A = I. Only square matrices have one, and when it exists it is unique.
  • A square matrix is invertible exactly when it has full rank, a pivot in every column, independent columns, and a nonzero determinant.
  • The 2 by 2 formula divides by ad - bc; larger inverses come from row-reducing [A | I].
  • (AB)^-1 = B^-1 A^-1, and inverse and transpose commute.
  • Singular matrices squash space, so many inputs share an output and nothing can undo them; near-singular ones give meaningless huge numbers.
  • Use np.linalg.solve for systems and reserve inv for when you truly need the inverse matrix.

Next: factor a matrix once and solve many systems cheaply in LU Decomposition.