Skip to content
Tutorials Center
My learning

Topics

Lesson 9 of 62 ยท Matrices

Three Ways to See Matrix-Vector Products

See the product Ax three ways: as row dot products, as a combination of the columns of A, and as a linear map that transforms vectors, with NumPy examples.

The product of a matrix and a vector, Ax, is the most important operation in this whole track. Solving equations, transforming shapes, fitting models and running neural network layers all reduce to it. The calculation is always the same, but there are three useful ways to think about it, and different problems become easy depending on which view you choose.

Here you will meet all three, check with NumPy that they agree, and see when each one helps. Along the way you will find that "does Ax = b have a solution" turns into a question about columns.

Throughout, A is an m by n matrix and x is a vector with n entries. The result Ax has m entries.

One product, three views

Rows Each output entry is a row dotted with x.
Columns The output mixes the columns of A.
Map A takes the vector x to a new vector.

Same numbers, different stories.

1 / 2

View 1: dot products with the rows

This is the view straight from the multiplication rule. Entry i of Ax is the dot product of row i of A with x.

TEXT
[1 2 3]   [1]   [1*1 + 2*0 + 3*2]   [7]
[4 5 6] * [0] = [4*1 + 5*0 + 6*2] = [16]
          [2]

This view is best for computing a single output value, and for interpreting each row as a measuring device. A row [0.2, 0.3, 0.5] applied to a vector of exam scores gives a weighted average. In a dataset, one row of A can be one data point and x a set of weights, so Ax predicts every point at once.

Python Row view: one dot product per row

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

View 2: a combination of the columns

Write A as a row of column vectors a1, a2, ..., an. Then

TEXT
A x = x1 * a1 + x2 * a2 + ... + xn * an

The entries of x are weights (coefficients) and the output is the weighted sum of the columns. With the same numbers as before, x = (1, 0, 2) means "take one copy of column 1, none of column 2 and two copies of column 3":

TEXT
1 * [1]  +  0 * [2]  +  2 * [3]   =  [7]
    [4]         [5]         [6]      [16]

This view explains a lot. The set of all possible outputs Ax is exactly the set of all linear combinations of the columns. So the equation Ax = b has a solution if and only if b can be built from the columns of A. That set is called the column space. If x has a zero entry, the matching column is simply ignored.

Python Column view: mixing the columns

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

View 3: a linear map

The third view treats A as a function. It accepts a vector x from n-dimensional space and returns a vector Ax in m-dimensional space. Two properties hold for every matrix:

  • A(x + y) = Ax + Ay
  • A(cx) = c(Ax)

A function with those two properties is called linear. Every matrix gives one, and (as you will see in Linear Transformations) every linear function between these spaces comes from a matrix. The map view is the language of geometry: rotating a picture, stretching it, or projecting it onto a plane are all done by choosing the right A.

The columns tell you exactly what the map does. Feed in the standard basis vector e1 = (1, 0, ..., 0) and Ae1 is the first column of A. Column j is where the j-th axis direction lands.

Python Map view: where do the axes go?

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Choosing a view

All three views produce identical numbers, so choose by the question you are asking.

  • Row view: compute one particular output; interpret rows as equations, scores or sensors. Each row of Ax = b is one equation (see Systems of Linear Equations).
  • Column view: decide whether a target is reachable; understand rank, span and dependent columns.
  • Map view: reason about geometry, composition and what happens to shapes.

The same split continues to full matrix products. Each column of AB is A times the corresponding column of B, which is the column view applied once per column. And AB as a map means "first apply B, then A", which is why the order matters.

Python A product is a matrix-vector product per column

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Reachable and unreachable targets

Here is the column view put to work. Suppose the two columns of A point in the same direction. Then every combination of them lies on one line, and most targets b are unreachable.

Python Is b in the span of the columns?

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Here lstsq finds the closest possible combination. If even that best combination misses b, no exact solution exists. You will study it properly in Least Squares.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. In the column view, what do the entries of x do in Ax?

    Ax = x1a1 + x2a2 + ... where a1, a2, ... are the columns of A, so x supplies the weights.
  2. 2. The set of all possible products Ax is...

    By the column view, every Ax is a combination of columns, and every combination arises from some x.
  3. 3. What is A times the standard basis vector e2?

    e2 has a 1 in position 2 and zeros elsewhere, so it selects column 2 and ignores the others.
  4. 4. A is 4 by 3 and x has 3 entries. How many entries does Ax have?

    Each of the 4 rows of A produces one output entry.

Summary

  • Row view: each entry of Ax is a row of A dotted with x.
  • Column view: Ax is a weighted sum of the columns of A, with the entries of x as weights.
  • Map view: A is a linear function; its columns show where the axis directions go.
  • Ax = b is solvable exactly when b lies in the span of the columns.
  • A full product AB multiplies A onto each column of B and means "apply B, then A".

Next: Transpose and Symmetric Matrices.