Skip to content
Tutorials Center
My learning

Topics

Lesson 62 of 62 ยท Practice and Review

Linear Algebra Cheat Sheet

A one-page linear algebra cheat sheet with key concepts, formulas and the matching NumPy and SciPy calls for vectors, matrices, systems, eigenvalues and the SVD.

This page is the quick-reference version of the whole track: the ideas, the formulas, and the function you call in NumPy or SciPy. Use it when you remember that something exists but not its exact name or shape rules. Each table links back to the lesson that explains the details.

Tables assume import numpy as np, import scipy.linalg as sla, vectors u and v, a square matrix A, and b as a right-hand side.

The whole track in one view

Vectors Length, angle, projection.
Matrices Maps and products.
Systems Solve, factor, invert.
Spaces Span, basis, rank.

The first four chapters build the vocabulary.

1 / 2

Vectors

Idea Formula NumPy
Dot product u . v = sum(u_i v_i) = |u| |v| cos(theta) u @ v or np.dot(u, v)
Length (L2 norm) |v| = sqrt(v . v) np.linalg.norm(v)
L1 / max norm sum |v_i|, max |v_i| np.linalg.norm(v, 1), np.linalg.norm(v, np.inf)
Unit vector v / |v| v / np.linalg.norm(v)
Distance |u - v| np.linalg.norm(u - v)
Cosine similarity u . v / (|u| |v|) u @ v / (np.linalg.norm(u) * np.linalg.norm(v))
Projection of u onto v (u . v / v . v) v (u @ v) / (v @ v) * v
Cross product (3D) vector perpendicular to both, length = parallelogram area np.cross(u, v)
Orthogonal u . v = 0 np.isclose(u @ v, 0)

See Vectors, Length, Distance and Unit Vectors and The Dot Product.

Matrices

Idea Formula or rule NumPy
Shape rule for products (m, k) @ (k, n) -> (m, n) A.shape
Matrix product (AB)_ij = sum_t A_it B_tj A @ B
Elementwise product same shape, entry by entry A * B
Matrix power A @ A @ ... np.linalg.matrix_power(A, k)
Transpose (AB)^T = B^T A^T A.T
Identity AI = IA = A np.eye(n)
Diagonal matrix entries on the diagonal only np.diag(d)
Trace sum of the diagonal np.trace(A)
Symmetric A^T = A np.allclose(A, A.T)
Orthogonal Q^T Q = I, so Q^-1 = Q^T np.allclose(Q.T @ Q, np.eye(n))

Remember: matrix multiplication is associative but not commutative, and A * B is never the matrix product. See Matrix Multiplication and Transpose and Symmetric Matrices.

Systems of equations

Task Formula or idea Call
Solve Ax = b (square) LU factorization inside np.linalg.solve(A, b)
Inverse A A^-1 = I; exists only if det(A) != 0 np.linalg.inv(A) (rarely needed)
Least squares (tall A) minimize |Ax - b|; normal equations A^T A x = A^T b np.linalg.lstsq(A, b, rcond=None)
LU with pivoting A = P L U P, L, U = sla.lu(A)
Reuse a factorization factor once, solve many b lu, piv = sla.lu_factor(A), sla.lu_solve((lu, piv), b)
Pseudoinverse works for any shape np.linalg.pinv(A)
Condition number sigma_max / sigma_min np.linalg.cond(A)

A system Ax = b has zero, one or infinitely many solutions, never exactly two. See Solving Systems with NumPy, Inverse Matrices and LU Decomposition.

Vector spaces and rank

Concept Meaning Call
Span all linear combinations of the vectors test: rank of [A | b] equals rank of A
Linear independence no vector is a combination of the others np.linalg.matrix_rank(A) == A.shape[1]
Basis, dimension independent spanning set; number of vectors in it
Column space span of the columns; dimension = rank scipy.linalg.orth(A)
Null space solutions of Ax = 0; dimension = nullity scipy.linalg.null_space(A)
Rank number of independent columns (or rows) np.linalg.matrix_rank(A)
Rank-nullity rank + nullity = number of columns

See Linear Independence, Column Space and Null Space and Rank.

Determinant and invertibility

An n by n matrix is invertible exactly when all of these hold: det(A) != 0, rank(A) = n, the columns are independent, the null space is , no eigenvalue is zero, no singular value is zero.

Fact Formula
Meaning area or volume scale factor; sign = orientation
Product rule det(AB) = det(A) det(B)
Transpose det(A^T) = det(A)
Inverse det(A^-1) = 1 / det(A)
Eigenvalues det(A) = product of eigenvalues
Large matrices sign, logdet = np.linalg.slogdet(A) avoids overflow

Transformations

Transformation (2D) Matrix
Rotation by angle t [[cos t, -sin t], [sin t, cos t]]
Scaling by sx, sy [[sx, 0], [0, sy]]
Shear along x [[1, k], [0, 1]]
Reflection across the x-axis [[1, 0], [0, -1]]
Translation (homogeneous 3 by 3) [[1, 0, tx], [0, 1, ty], [0, 0, 1]]

The columns of a matrix are the images of the basis vectors. To apply A first and then B, multiply B @ A: the order reads right to left. See Rotation, Scaling, Shear and Reflection and Homogeneous Coordinates and Translation.

Orthogonality and QR

Idea Formula Call
Projection onto the column space of A P = A (A^T A)^-1 A^T A @ np.linalg.solve(A.T @ A, A.T)
QR decomposition A = Q R, Q orthonormal columns, R upper triangular Q, R = np.linalg.qr(A)
Gram-Schmidt build orthonormal vectors one at a time modified version is the stable one
Least squares via QR solve R x = Q^T b sla.solve_triangular(R, Q.T @ b)

Eigenvalues and decompositions

Decomposition Form Call
Eigen A v = lambda v; A = P D P^-1 vals, vecs = np.linalg.eig(A)
Symmetric eigen S = Q diag(lambda) Q^T, real lambda, orthonormal Q vals, Q = np.linalg.eigh(S)
SVD A = U Sigma V^T U, s, Vt = np.linalg.svd(A)
Characteristic polynomial det(A - lambda I) = 0 np.poly(A)
Cholesky (positive definite) S = L L^T np.linalg.cholesky(S)
Low-rank approximation keep top k singular values U[:, :k] * s[:k] @ Vt[:k]

Rules of thumb: the trace is the sum of the eigenvalues; singular values are the square roots of the eigenvalues of A^T A; the best rank-k approximation (in the 2-norm and Frobenius norm) comes from the truncated SVD; positive definite means all eigenvalues are positive. Remember that eig returns eigenvectors as columns, and svd returns V^T, not V. See Eigenvalues and Eigenvectors and Singular Value Decomposition.

Try it: identities in one run

This example checks a handful of the identities from the tables on random matrices. If you ever doubt a formula, test it like this.

Python Checking identities numerically

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Try it: every decomposition reconstructs the matrix

A decomposition is only useful if multiplying the factors gives back the original. This example factors one matrix four ways and verifies each one.

Python Four decompositions, one matrix

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Common pitfalls at a glance

  • * is elementwise; use @ for matrix products and matrix_power for powers.
  • Flat (n,), column (n, 1) and row (1, n) are different shapes; print .shape.
  • Prefer solve and lstsq to inv; check np.linalg.cond(A) for sensitive problems.
  • Compare floats with np.allclose, and judge rank with a tolerance (matrix_rank does).
  • Eigenvectors are columns and defined up to sign; eig does not sort, eigh sorts ascending.
  • svd returns V^T; rebuild with U @ np.diag(s) @ Vt.

More detail on each is in Common Mistakes and Debugging.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which call solves A x = b for a square, invertible A?

    solve uses a factorization directly. inv(A) * b would be an elementwise product with the wrong meaning, and the other two options are unrelated.
  2. 2. You apply transformation A first, then transformation B, to a column vector x. Which expression is correct?

    The matrix closest to x acts first, so applying A then B is B @ (A @ x), which equals (B @ A) @ x.
  3. 3. Which pair of facts about a 3 by 3 matrix is consistent?

    Rank below n means singular, so the determinant is zero and some eigenvalue is zero. Full rank means a nonzero determinant.
  4. 4. Which function should you use for the eigen-decomposition of a symmetric matrix?

    eigh is designed for symmetric matrices: real eigenvalues in ascending order and orthonormal eigenvectors.

Summary

  • The tables on this page cover vectors, matrices, systems, spaces, determinants, transformations, orthogonality and decompositions with the exact NumPy and SciPy calls.
  • Invertibility has many equivalent faces: determinant, rank, independence, null space, eigenvalues, singular values.
  • Composition reads right to left, and matrix multiplication is not commutative.
  • Verify any formula by testing it on random matrices with np.allclose.
  • Reconstruct after every decomposition: P @ L @ U, Q @ R, U @ diag(s) @ Vt, Q @ diag(lambda) @ Q.T.

You have reached the end of the Linear Algebra track. Revisit What Is Linear Algebra? for the big picture, or keep practicing with the eigenvalue and decomposition exercises.