Skip to content
Tutorials Center
My learning

Topics

Lesson 12 of 62 ยท Matrices

Block Matrices and Partitioned Products

Split big matrices into blocks, multiply and invert them block by block, and use the Schur complement to get determinants and inverses, checked in NumPy.

A large matrix rarely has to be treated as one huge grid of numbers. You can draw horizontal and vertical lines through it and see a small matrix whose entries are themselves matrices. This is a block matrix (or partitioned matrix), and the lines are not decoration: matrix operations work on blocks almost exactly as they work on numbers, as long as the block sizes fit.

Block thinking explains why algorithms split big problems into small ones, why a system with two kinds of unknowns can be solved in two stages, and where the Schur complement comes from. It builds on matrix multiplication and the inverse.

Blocks multiply like numbers

Write a matrix as a grid of blocks:

TEXT
M = [ A  B ]        N = [ E  F ]
    [ C  D ]            [ G  H ]

Then, as long as every block product is defined (the column count of a block on the left matches the row count of the block under it),

TEXT
M N = [ A E + B G    A F + B H ]
      [ C E + D G    C F + D H ]

Addition works blockwise too. The one rule to remember is that block products keep their order: AE is not EA. This is the same row-by-column rule as before with blocks in place of numbers, which is easy to see because the ordinary rule already sums products of pieces.

Python Block multiplication equals ordinary multiplication

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Block diagonal and block triangular matrices

When the off-diagonal blocks are zero, the matrix splits into independent problems:

TEXT
[ A  0 ]            [ A  B ]
[ 0  D ]    and     [ 0  D ]     (block upper triangular)

For both shapes (with square A and D):

  • Determinant: det = det(A) ยท det(D).
  • Eigenvalues: the eigenvalues of the whole matrix are the eigenvalues of A together with those of D.
  • Block diagonal inverse: [[A, 0], [0, D]]^(-1) = [[A^(-1), 0], [0, D^(-1)]], so inverting costs one inverse per block.
  • Powers: [[A, 0], [0, D]]^k = [[A^k, 0], [0, D^k]].

This is why a Jordan form is so useful: it is block diagonal, so you can understand the whole by understanding each small block.

The Schur complement

For a general 2 ร— 2 block matrix with invertible A, eliminate C the same way you would clear a number below a pivot. Multiply the top block row by C A^(-1) and subtract it from the bottom one. What is left in the lower right corner is the Schur complement of A:

TEXT
S = D - C A^(-1) B

This block elimination gives a factorization

TEXT
[ A  B ]   [ I        0 ] [ A  0 ] [ I  A^(-1) B ]
[ C  D ] = [ C A^(-1) I ] [ 0  S ] [ 0  I        ]

and two consequences come straight out of it:

  • Determinant: det [[A, B], [C, D]] = det(A) ยท det(S).
  • Invertibility and inverse: the whole matrix is invertible exactly when A and S are, and the lower-right block of the inverse is S^(-1). Solving M x = b reduces to solving with A and with S.

The Schur complement shows up in least squares, Gaussian processes, Kalman filters and the conditional distribution of a covariance matrix. In an ordinary n ร— n elimination, each step that clears a column leaves a Schur complement behind.

Python Block determinant, inverse and the Schur complement

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

All three checks agree to floating-point precision. A useful sanity habit: when you derive a block formula, test it on a random matrix whose blocks you generate yourself, as above.

The block inverse formulas assume A (or D) is invertible. A matrix can be invertible even if A is not, in which case you pick the other Schur complement or reorder the rows and columns. Never invert A literally in production code; solve linear systems with it instead (np.linalg.solve).

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. For a block upper triangular matrix [[A, B], [0, D]] with square A and D, what is the determinant?

    The off-diagonal block B does not matter: det is the product of the determinants of the diagonal blocks.
  2. 2. How is the Schur complement of A in [[A, B], [C, D]] defined?

    S = D - C A^-1 B is what remains in the lower right corner after block elimination clears C. (Option 3 is the Schur complement of D instead.)
  3. 3. Which condition is needed to multiply two partitioned matrices block by block?

    Each block product such as A E must be defined, so column partitions of the left matrix must match row partitions of the right one.
  4. 4. What is the inverse of the block diagonal matrix diag(A, D)?

    Invert each diagonal block in place.
  5. 5. If M = [[A, B], [C, D]] and A is invertible, M is invertible exactly when...

    det(M) = det(A) det(S), so M is invertible if and only if S is (given A is).

Summary

  • A block matrix is a matrix of matrices, and products work blockwise when the block sizes match and the order is kept.
  • Block diagonal and block triangular matrices have det = det(A) ยท det(D); block diagonal ones invert and exponentiate block by block, and their eigenvalues are the union of the blocks' eigenvalues.
  • The Schur complement S = D - C A^(-1) B comes from block elimination and gives det(M) = det(A) det(S), block inverse formulas and two-stage solving.

Next: Systems of Linear Equations, where matrices start solving problems.