Skip to content
Tutorials Center
My learning

Topics

Lesson 22 of 62 ยท Vector Spaces

Linear Combinations and Span

Learn linear combinations and the span of vectors, see what they fill in 2D and 3D, and test with NumPy whether a vector b lies in the span of others.

So far you have treated vectors as arrows or lists of numbers, and you have solved systems of equations. This lesson connects the two with one phrase: linear combination. A linear combination is what you get when you scale some vectors and add the results. Everything a matrix does to a vector is built from linear combinations, and the question "does Ax = b have a solution?" turns into "can b be built from the columns of A?"

The set of everything you can build from a collection of vectors is called their span. Span is the first concept of the vector-spaces chapter and the foundation for independence, basis, dimension and rank in the next lessons. By the end of this one you will be able to picture spans in two and three dimensions, and decide with code whether a given vector lies in one.

Linear combinations

Given vectors v1, v2, ..., vk and numbers c1, c2, ..., ck (the coefficients or weights), the vector

c1*v1 + c2*v2 + ... + ck*vk

is a linear combination of the vs. Only two operations are allowed: scaling and adding. No squaring, no multiplying vectors together.

Python Building vectors from two ingredients

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The last two lines state the key identity of the whole subject: A @ c is the linear combination of the columns of A with weights c. You first met this as the column view in Three Ways to See Matrix-Vector Products. Notice also that choosing all weights equal to zero always produces the zero vector, so the zero vector is a combination of anything.

The span

The span of v1, ..., vk is the set of all linear combinations of them, over every possible choice of weights. It is written span{v1, ..., vk}. Three facts make spans easy to picture:

  • The span of one nonzero vector is a line through the origin.
  • The span of two vectors that point in different directions (in 3D or higher) is a plane through the origin. In 2D it is the whole plane.
  • Every span contains the origin, and stays closed under adding and scaling. A set with those properties is called a subspace.

What a span can be

A point The span of only the zero vector.
A line One nonzero vector, all its multiples.
A plane Two vectors in different directions.
All of 3D Three vectors that do not lie in one plane.

Every span passes through the origin.

1 / 2

Let us see a span instead of imagining it. We draw many random combinations of two vectors in the plane. When the vectors point in different directions, the points cover the whole plane. When one vector is a multiple of the other, they collapse onto a line.

Python Span of two vectors in 2D

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

In 3D the same story has one more level. Two independent vectors fill a plane floating in space, a flat sheet through the origin, and a third vector that sticks out of that plane gives you all of 3D. Whether the third vector "sticks out" is exactly what the next lesson formalises.

Does b lie in the span?

This is the practical question: given vectors v1, ..., vk and a target b, can you find weights c with c1*v1 + ... + ck*vk = b? Put the vectors into the columns of a matrix V and you are asking whether V c = b has a solution, a system you already know how to handle.

A neat numerical test uses rank: b is in the span of the columns exactly when appending b as an extra column does not raise the rank. You can also ask np.linalg.lstsq for the best weights and see whether the leftover residual is zero.

Python Testing membership in a span

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

When the target is outside the span, the least-squares weights give the point of the plane that is closest to b. The nonzero residual measures how far b sticks out. This is the seed of orthogonal projection and least squares.

Span and the columns of a matrix

Everything above says one thing about a matrix A with columns a1, ..., an:

  • The span of the columns is the set of all possible outputs A @ x. Its name is the column space of A (a later lesson).
  • Ax = b is solvable exactly when b is in the column space.
  • If the columns span the entire space R^m, then every b is reachable. Having a pivot in every row is the matching condition.
Python Which right-hand sides are reachable?

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

A plane through the origin in 3D can be described two ways: as a span of two vectors (a parametric description) or as all b perpendicular to a normal vector (an implicit description, one equation). Moving between the two descriptions is a skill you will practise throughout this track.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Which of these is a linear combination of u and v?

    Linear combinations only allow scaling vectors and adding the results. 3u - 2v does exactly that.
  2. 2. What is the span of the two vectors (1, 2) and (3, 6) in the plane?

    (3, 6) is 3 times (1, 2), so both lie on the same line. The span is that line.
  3. 3. Which statement about the matrix-vector product A @ c is correct?

    Each entry of c scales the matching column, and the scaled columns are added. That is the column view of a matrix-vector product.
  4. 4. Appending b as a new column of V does not change the rank. What does that tell you?

    If b were outside the span it would add a new direction and raise the rank. An unchanged rank means b is already a combination of the columns.
  5. 5. Three vectors in 3D all lie in one plane through the origin. What is their span?

    Every combination of vectors in a plane through the origin stays in that plane, so the span is that plane or something smaller.

Summary

  • A linear combination scales vectors and adds them; A @ c is a combination of the columns of A.
  • The span of a set of vectors is every combination of them; it always contains the origin.
  • One vector spans a line, two independent vectors span a plane, three independent vectors fill 3D.
  • A vector that is already in the span adds no new direction.
  • b lies in the span of the columns of V exactly when V c = b is solvable, which you can test through rank or a zero least-squares residual.

Next: decide which vectors are redundant in Linear Independence.