Skip to content
Tutorials Center
My learning

Topics

Lesson 36 of 62 · Eigenvalues and Decompositions

Eigenvalues and Eigenvectors

Learn what eigenvalues and eigenvectors are, find them with the characteristic polynomial and np.linalg.eig, see the geometry, and verify Av = lambda v in code.

Most vectors change direction when a matrix acts on them. A few special ones do not: they are only stretched, shrunk or flipped, staying on the same line through the origin. These are the eigenvectors of the matrix, and the factor by which each is scaled is its eigenvalue. The German prefix eigen means "own" or "characteristic": these are the matrix's own directions.

Eigenvalues explain how a system evolves, which directions carry the most variance in data, why some iterations converge, and how a matrix can be simplified through a change of basis. This lesson defines them, shows how to find them by hand and with np.linalg.eig, draws the geometric picture and lists the things that can go wrong.

A special direction

Most vectors A turns them to a new direction.
Eigenvectors A keeps them on their own line.
Eigenvalue The stretch factor along that line.
1 / 2

The definition

A nonzero vector v is an eigenvector of the square matrix A, with eigenvalue λ, if

TEXT
A v = λ v

The matrix acting on v gives the same result as multiplying v by a plain number. Two details matter:

  • v must be nonzero. The zero vector satisfies the equation for every λ, which would say nothing.
  • Any nonzero multiple of an eigenvector is also an eigenvector, so eigenvectors are really directions. Programs usually normalize them to length 1.

An eigenvalue can be negative (the direction flips), zero (the direction is squashed to nothing, so A is singular) or even complex, as we will see for rotations.

Try it on a simple matrix. For A = [[2, 1], [1, 2]] the vector (1, 1) is sent to (3, 3), which is 3·(1, 1).

Python Checking A v = lambda v by hand

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Finding eigenvalues: the characteristic polynomial

Rewrite A v = λ v as (A − λI) v = 0. We want a nonzero solution v, so A − λI must be singular, meaning its determinant is zero (Determinants):

TEXT
det(A − λI) = 0

This is the characteristic equation. For an n × n matrix, the left side is a polynomial of degree n in λ, so there are n eigenvalues, counting repeats and complex ones. For a 2 by 2 matrix:

TEXT
det(A − λI) = λ² − (trace A) λ + det A

Take A = [[4, 1], [2, 3]]. The trace is 7 and the determinant is 10, so λ² − 7λ + 10 = 0, which factors as (λ − 5)(λ − 2). The eigenvalues are 5 and 2. Once you have an eigenvalue, substitute it into (A − λI) v = 0 and solve for v (a null space problem). For λ = 5 the equations give v = (1, 1); for λ = 2 they give v = (1, −2).

Two shortcuts follow from the polynomial and are good sanity checks:

  • the sum of the eigenvalues equals the trace,
  • the product of the eigenvalues equals the determinant.
Python The characteristic polynomial in NumPy

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Solving the polynomial is fine for 2 by 2 and 3 by 3, but it is not how libraries work: roots of high-degree polynomials are very sensitive to rounding. NumPy uses iterative matrix algorithms (variants of the QR algorithm) that never form the polynomial.

np.linalg.eig

np.linalg.eig(A) returns a pair: a 1D array of eigenvalues and a matrix V whose columns are the matching eigenvectors, normalized to length 1. A common slip is to read the rows of V.

Python np.linalg.eig and the check A v = lambda v

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The order of eigenvalues is not guaranteed, and the sign of each eigenvector is arbitrary: v and −v are equally good. Never write a test that expects a specific sign or order. Compare through A v = λ v instead.

The geometric picture

A matrix transforms the plane. Picture the unit circle: its image is an ellipse. The eigenvectors are the directions that stay on their own line, and the eigenvalues say how far each is stretched. For this A, points on the (1, 1) direction get stretched by 5 and points on the (1, −2) direction by 2. Everything else is pulled toward the stronger direction.

Python The unit circle and the eigen-directions

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Each eigenvector's dashed arrow is a stretched copy of the solid one lying on the same line. That is exactly what A v = λ v says.

Things that can happen

Complex eigenvalues. A rotation moves every direction, so a real matrix with no real eigenvectors must have complex eigenvalues. The 90-degree rotation has λ = ±i.

Repeated eigenvalues and missing eigenvectors. A shear such as [[1, 1], [0, 1]] has eigenvalue 1 twice but only one independent eigenvector direction. Such a matrix is called defective, and it cannot be diagonalized (see Diagonalization).

Zero eigenvalue. A singular matrix has λ = 0, and its eigenvectors are exactly the nonzero vectors in the null space.

Python Rotation, shear and a singular matrix

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

For symmetric matrices, use np.linalg.eigh instead of eig. It is faster, returns real eigenvalues in ascending order and gives orthogonal eigenvectors. The spectral theorem lesson explains why.

Eigenspaces and algebraic versus geometric multiplicity

An eigenvalue does not come with just one eigenvector. If v is an eigenvector for λ, so is 2v, -v, and every nonzero multiple, and sometimes entirely different directions work too. The complete collection, together with the zero vector, forms a subspace called the eigenspace:

TEXT
E_λ = null(A - λI) = { v : A v = λ v }

Finding it is a null space computation, so the recipe has three steps:

  1. Find the eigenvalues: solve det(A - λI) = 0.
  2. For each eigenvalue λ, form the matrix A - λI.
  3. Row-reduce and solve (A - λI)v = 0. A basis for the solutions is a basis for the eigenspace.

Two different numbers describe how "big" an eigenvalue is:

  • The algebraic multiplicity is how many times λ is a root of the characteristic polynomial.
  • The geometric multiplicity is the dimension of the eigenspace, dim null(A - λI) = n - rank(A - λI).

The fundamental inequality is 1 ≤ geometric ≤ algebraic. When they are equal for every eigenvalue, the eigenvectors fill the whole space and A is diagonalizable. When for some eigenvalue the geometric multiplicity is strictly smaller, the matrix is defective: it is short of eigenvectors, and the best you can do is a Jordan form with generalized eigenvectors, which satisfy (A - λI)^k v = 0 for some k > 1 rather than k = 1.

The three 3 × 3 matrices below all have the single eigenvalue 2 with algebraic multiplicity 3, yet behave very differently:

Matrix rank(A - 2I) Geometric multiplicity Diagonalizable?
2I (diagonal) 0 3 yes
one off-diagonal 1 1 2 no
two off-diagonal 1s (Jordan block) 2 1 no
Python Eigenspaces and multiplicities, exactly with SymPy

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The symmetric matrix with repeated eigenvalue 1 (algebraic multiplicity 2) still has a two-dimensional eigenspace, so it is diagonalizable: a general fact about symmetric matrices. The Jordan block has only one genuine eigenvector direction; the chain v → (J - 2I)v → (J - 2I)²v → 0 shows the "generalized" eigenvectors that fill the gaps.

Check your understanding

Quick check · 4 questions

of 4 answered You scored / 4 · perfect!

  1. 1. What does it mean that v is an eigenvector of A with eigenvalue 4?

    Multiplying by A only scales v by the factor 4; v stays on its own line, and v must be nonzero.
  2. 2. A 2 by 2 matrix has trace 7 and determinant 10. What are its eigenvalues?

    The characteristic polynomial is lambda^2 - 7 lambda + 10 = (lambda - 5)(lambda - 2). The sum is the trace and the product is the determinant.
  3. 3. In v, V = np.linalg.eig(A), where are the eigenvectors?

    np.linalg.eig returns eigenvectors as columns, so the eigenvector for values[i] is V[:, i].
  4. 4. Why does a 2D rotation by 90 degrees have no real eigenvalues?

    An eigenvector must stay on its own line. A 90-degree rotation moves every real direction, so the eigenvalues come out as the complex pair i and -i.

Summary

  • An eigenvector v ≠ 0 satisfies A v = λ v: A only scales it by the eigenvalue λ.
  • Eigenvalues are the roots of det(A − λI) = 0; their sum is the trace and their product is the determinant.
  • np.linalg.eig returns eigenvalues and a matrix whose columns are the eigenvectors; order and signs are arbitrary, so verify with A v = λ v.
  • Geometrically, eigenvectors are the directions a transformation only stretches, shrinks or flips.
  • Rotations give complex eigenvalues, shears can be defective, and singular matrices have a zero eigenvalue.

Next up: Diagonalization and Matrix Powers, where eigenvectors turn hard matrix problems into easy ones.