Skip to content
Tutorials Center
My learning

Topics

Lesson 4 of 62 ยท Getting Started

The Dot Product

Understand the dot product as a sum of products and as a measure of alignment, then find angles, cosine similarity, projections and orthogonal vectors in NumPy.

The dot product takes two vectors of the same length and returns a single number. That number tells you how much the two vectors point in the same direction. It is the most important operation in this whole track: matrix multiplication is made of dot products, projections are built from them, and the machine-learning notion of "similarity" is a dot product in disguise.

You already know how to add vectors and measure their length. Now you will learn two faces of the dot product, the algebraic one (multiply and add) and the geometric one (lengths and an angle), and see how to use them to find angles, similarity, projections and right angles.

The algebraic definition

Multiply the matching components and add up the results:

TEXT
(a1, a2, a3) . (b1, b2, b3) = a1*b1 + a2*b2 + a3*b3

For example, (1, 2, 3) . (4, 5, 6) = 4 + 10 + 18 = 32. The result is a plain number (a scalar), not a vector. In NumPy there are three equivalent ways to write it:

Python Three ways to compute a dot product

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

All three print 32. Use @ or np.dot in practice; they are faster and clearer. Note the difference from a * b, which is only the elementwise product [4, 10, 18] with no summing.

The geometric meaning

The same number can be computed from lengths and the angle theta between the vectors:

TEXT
a . b = ||a|| * ||b|| * cos(theta)

This formula explains the sign of the dot product, which is often all you need:

The sign tells the story

Positive Angle under 90 degrees. They lean the same way.
Zero Exactly 90 degrees. They are perpendicular.
Negative Angle over 90 degrees. They lean opposite ways.

The size of the number also depends on the lengths.

1 / 2

Two consequences are worth memorizing. First, a . a = ||a||^2, so the length of a vector is sqrt(a . a). Second, a . b = 0 for non-zero vectors means they are orthogonal (perpendicular). Orthogonality is a core idea, developed in Orthogonality and Projections.

Finding the angle between vectors

Rearrange the geometric formula to solve for the angle:

TEXT
cos(theta) = (a . b) / (||a|| * ||b||)
theta = arccos(that value)
Python The angle between two vectors

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The first angle is 45 degrees: (1, 0) points along the x axis and (1, 1) points diagonally. Here is a practical warning. Rounding can push the cosine slightly outside [-1, 1] (for example 1.0000000000000002), and arccos then returns nan. Real code clips first: np.arccos(np.clip(c, -1.0, 1.0)).

Cosine similarity

The cosine itself is a useful score. Cosine similarity is the dot product of the two vectors after normalizing both to length 1. It ranges from 1 (same direction) through 0 (unrelated) to -1 (opposite). Because it ignores length, it compares direction only, which is what you want when comparing documents, user tastes or word embeddings.

Python Comparing documents by word counts

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Document b has roughly double the counts of a, so the raw dot product is large, but the cosine of about 0.997 shows they are almost the same topic. Document c shares no words with a, so both its dot product and its cosine are 0. Cosine similarity separates "about the same thing" from "just bigger".

Projection: the shadow of one vector on another

Imagine shining a light straight down onto the direction of b from above a. The shadow of a on the line through b is the projection. Its signed length is (a . b) / ||b||, and the projected vector is:

TEXT
proj_b(a) = ((a . b) / (b . b)) * b

Whatever is left over, a - proj_b(a), is perpendicular to b. So every vector splits into a part along b and a part at right angles to it.

Python Projection and the leftover part

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Projecting (3, 4) onto the x direction keeps (3, 0) and leaves (0, 4); the two pieces add back to the original and are orthogonal (their dot product with b is zero). Projection is the seed of least squares, where we project data onto the best-fitting model.

Rules of the dot product

  • Symmetric: a . b = b . a
  • Distributes over addition: a . (b + c) = a . b + a . c
  • Plays well with scalars: (c*a) . b = c * (a . b)

Unlike ordinary multiplication it combines two vectors into a number, so you cannot "divide" by a vector, and (a . b) . c makes no sense (it would be a number dotted with a vector).

In math the dot product is also written a^T b (a row times a column). That is why @ is the natural NumPy operator: the dot product is just the smallest case of matrix multiplication.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What is (2, -1, 3) . (4, 5, 1)?

    2*4 + (-1)5 + 31 = 8 - 5 + 3 = 6. The elementwise product (8, -5, 3) has not been summed yet.
  2. 2. Two non-zero vectors have a dot product of 0. What does that tell you?

    cos(theta) = 0 exactly when theta is 90 degrees, so the vectors are orthogonal.
  3. 3. Which value of cosine similarity means two vectors point in the same direction?

    The cosine of a zero angle is 1. A value of 0 means perpendicular and -1 means opposite directions.
  4. 4. Why should you clip the cosine to [-1, 1] before calling np.arccos?

    Floating-point error can make the ratio 1.0000000000000002, which is outside the domain of arccos.
  5. 5. In the projection formula, what is the leftover part a - proj_b(a)?

    Subtracting the shadow along b leaves exactly the component at right angles to b.

Summary

  • The dot product multiplies matching components and adds them, giving one number; in NumPy use a @ b or np.dot(a, b).
  • Geometrically a . b = ||a|| ||b|| cos(theta), so its sign shows whether vectors lean together, apart or at right angles.
  • a . a is the squared length, and a zero dot product means orthogonal vectors.
  • cos(theta) = (a . b) / (||a|| ||b||) gives angles; clip before arccos.
  • Cosine similarity compares direction and ignores length, which is ideal for text and embeddings.
  • Projection onto b is ((a . b) / (b . b)) * b, and the remainder is perpendicular to b.

Next up: a product that returns another vector in The Cross Product.