Skip to content
Tutorials Center
My learning

Topics

Lesson 25 of 62 ยท Vector Spaces

Column Space and Null Space

Meet the four fundamental subspaces of a matrix, compute bases for the column space and null space with SciPy, and see how solutions to Ax = b are structured.

Every matrix A quietly defines two important sets of vectors. The column space is everything A can produce as an output: all vectors of the form A x. The null space is everything A crushes to zero: all vectors x with A x = 0. Together they answer the two big questions about a linear system. Does A x = b have a solution? And if so, how many?

Both sets are subspaces: they contain the zero vector and are closed under addition and scaling. That means they have bases and dimensions, so you can describe each one with a handful of vectors. This lesson builds on basis and dimension and the column view of matrix-vector products.

Two spaces from one matrix

Column space All outputs A x. Lives in R^m.
Null space All inputs sent to zero. Lives in R^n.

For an m by n matrix, outputs live in R^m and inputs in R^n.

1 / 2

The column space

Write A x as a combination of the columns of A, with the entries of x as weights. The set of all such combinations is the column space C(A), which is just the span of the columns. So:

  • A x = b has a solution exactly when b lies in the column space of A.
  • A basis for C(A) is a maximal independent set of columns. Its size is the rank of A, so dim C(A) = rank(A).

The columns that matter are the pivot columns you met in Reduced Row Echelon Form: the columns of the original A that sit in pivot positions form a basis of C(A). In NumPy, scipy.linalg.orth(A) returns an orthonormal basis for the column space, which is often more convenient than picking columns by hand.

The null space

The null space N(A) is the set of all x with A x = 0. It always contains the zero vector. If it contains anything else, then A loses information: different inputs can give the same output, because A(x + n) = A x for any n in the null space.

  • A basis for N(A) has one vector per free variable of the system, so dim N(A) = n - rank(A). This number is the nullity.
  • If the nullity is 0, the columns of A are independent and A x = 0 only has the trivial solution.

scipy.linalg.null_space(A) returns an orthonormal basis for N(A) as the columns of a matrix. If the null space is just , you get an array with zero columns.

Python Bases for the column space and the null space

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The second row of A is twice the first, so only two rows are independent and the rank is 2. The column space is a plane inside R^3 (two basis vectors, each with three entries), and the null space is a line, a single direction in R^3 that A maps to zero. The printed null vector has length 1, which is how null_space normalizes its output. Check it by hand: -1 - 2 + 3 = 0 in the first row, and similarly for the others.

Solutions of A x = b have a clear structure

Suppose A x = b has at least one solution x_p, called a particular solution. Then every other solution has the form

x = x_p + n, where n is any vector in the null space.

Adding a null-space vector does not change the output, so you can slide along the null space freely. The solution set is a shifted copy of the null space. If the null space is a line, the solutions form a line; if it is a plane, they form a plane. If the null space is , the solution is unique. And if b is not in the column space, the solution set is empty.

Python One particular solution plus the null space

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

This system has 2 equations and 3 unknowns, so the null space has dimension at least 1. Here it is exactly 1: a line of directions. We take one solution from lstsq (it returns the shortest one when many exist) and slide along n by different amounts t. Every choice still hits b. This is the same "free variable" story told in Reduced Row Echelon Form, now in the language of subspaces.

The four fundamental subspaces

A matrix A with m rows and n columns and rank r has four natural subspaces.

Subspace Lives in Dimension Spanned by
Column space C(A) R^m r columns of A
Row space C(A^T) R^n r rows of A
Null space N(A) R^n n - r solutions of A x = 0
Left null space N(A^T) R^m m - r solutions of A^T y = 0

Two striking facts hold. First, the row space and the column space have the same dimension: the number of independent rows equals the number of independent columns. Second, the null space is perpendicular to the row space, because A x = 0 says every row of A has dot product zero with x. The left null space is perpendicular to the column space for the same reason. You will see this orthogonality put to work in Orthogonality and Projections.

Python Dimensions of all four subspaces

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The third row of A is the sum of the first two, so the rank is 2 out of a possible 3. The counts line up with the table: the column and row spaces have dimension 2, the null space has dimension 4 - 2 = 2, and the left null space has dimension 3 - 2 = 1.

To decide whether b is in the column space of A, compare np.linalg.matrix_rank(A) with the rank of the augmented matrix np.column_stack([A, b]). If appending b raises the rank, b was outside the column space and A x = b has no solution.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. A is a 5 by 3 matrix with rank 3. What is the dimension of its null space?

    The nullity is the number of columns minus the rank: 3 - 3 = 0. The columns are independent, so only x = 0 is sent to zero.
  2. 2. When does A x = b have a solution?

    A x is a combination of the columns of A, so b must lie in their span, the column space. A square matrix can still be singular, so square alone does not guarantee a solution.
  3. 3. x_p solves A x = b and n is in the null space of A. What is A (x_p + n)?

    A (x_p + n) = A x_p + A n = b + 0 = b. This is why the full solution set is x_p plus the null space.
  4. 4. A is 3 by 4 with rank 2. What is the dimension of the left null space?

    The left null space lives in R^3 (one entry per row) and has dimension m - r = 3 - 2 = 1.

Summary

  • The column space is the span of the columns: all outputs A x. Its dimension is the rank.
  • The null space is all x with A x = 0. Its dimension is n - rank.
  • Solutions of A x = b are one particular solution plus anything in the null space.
  • The four fundamental subspaces are column, row, null and left null; the row space is perpendicular to the null space.
  • scipy.linalg.orth and null_space give orthonormal bases for the column and null spaces.

Next, look at the number that ties these dimensions together in Rank.