Skip to content
Tutorials Center
My learning

Topics

Lesson 10 of 45 ยท Operators and Math

Linear Algebra Essentials

Use MATLAB and Octave for determinants, inverses, rank, norms, solving Ax=b with backslash, eigenvalues, SVD, LU, QR and Cholesky, with Python stand-ins.

The name MATLAB is short for "matrix laboratory", and linear algebra is the reason it exists. Systems of equations, least-squares fits, vibration modes, principal components and image compression all come down to a handful of matrix operations that MATLAB and Octave provide as one-line commands, backed by the same LAPACK routines that numerical libraries in every language use.

This lesson covers the properties you will ask of a matrix (det, rank, trace, norm, inv), the backslash operator for solving linear systems, and the main factorizations (eig, svd, lu, qr, chol). It assumes you know the difference between * and .* from Arithmetic: Matrix vs Element-wise.

MATLAB and Octave are not installed on this site, so matlab blocks are read-only. The TryIt blocks are NumPy and SciPy stand-ins that call the same LAPACK routines, so the numbers match what MATLAB prints, though NumPy formats output differently. Where a function's conventions differ (for example lu), the text says so.

Ask a matrix questions

det Zero means the matrix is singular.
rank How many independent rows it has.
trace The sum of the diagonal.
norm A measure of size.
1 / 2

Properties of a square matrix

MATLAB
A = [2 1; 1 3];

det(A)      % 5       the determinant
trace(A)    % 5       sum of the diagonal: 2 + 3
rank(A)     % 2       number of independent rows or columns
inv(A)      % [0.6 -0.2; -0.2 0.4]
A * inv(A)  % the identity matrix, up to rounding error

A matrix is invertible exactly when its determinant is nonzero, which is the same as having full rank. The matrix S = [1 2; 2 4] has a second row that is twice the first, so det(S) is 0 and rank(S) is 1; inv(S) prints "Warning: Matrix is singular to working precision" and returns Inf entries.

norm measures size. For a vector, norm(v) is the usual Euclidean length (norm([3 4]) is 5), norm(v, 1) is the sum of absolute values (7) and norm(v, Inf) the largest absolute value (4). For a matrix, norm(A) is the 2-norm (largest singular value) and norm(A, 'fro') the Frobenius norm, the square root of the sum of all squared entries. cond(A) measures how sensitive a linear system is to small changes; large values mean trouble.

Python det, rank, trace, inv and norm

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Solving Ax = b with backslash

To solve a system of linear equations A*x = b, you could compute x = inv(A) * b. Do not. The backslash operator is faster and more accurate, because it picks a suitable factorization instead of forming the inverse:

MATLAB
% 2x +  y -  z =   8
% -3x -  y + 2z = -11
% -2x +  y + 2z =  -3
A = [2 1 -1; -3 -1 2; -2 1 2];
b = [8; -11; -3];

x = A \ b       % approximately [2; 3; -1]
norm(A*x - b)   % residual, close to 0

Read A \ b as "A divides b from the left". The matrix must have as many rows as b has entries. If A is square and well conditioned, backslash solves the system exactly (to rounding). If A has more rows than columns, the system is over-determined and backslash returns the least-squares solution that minimizes norm(A*x - b). That makes it the standard tool for fitting a line or any model that is linear in its coefficients:

MATLAB
t = [0 1 2 3]';
y = [1.1 2.9 5.2 6.8]';
X = [ones(size(t)), t];   % columns: constant and slope term
coef = X \ y              % [1.09; 1.94]  ->  y is about 1.09 + 1.94 t

If A is singular, backslash warns and the answer is unreliable; check rank(A) or cond(A) first. The companion A / B solves x*B = A and is rarely needed.

Python Solve a system and fit a line

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

NumPy has no backslash operator: np.linalg.solve handles square systems and np.linalg.lstsq handles the least-squares case, while MATLAB uses the single operator \ for both.

Eigenvalues and eigenvectors

An eigenvector of a square matrix A is a direction that A only stretches: A*v = lambda*v, where the scalar lambda is the eigenvalue. They describe vibration modes, stability of dynamical systems and principal axes of data.

MATLAB
A = [4 1; 2 3];
lambda = eig(A)        % 2 and 5
[V, D] = eig(A);       % columns of V are eigenvectors, D is diagonal
A * V - V * D          % zero matrix, up to rounding

Two handy checks: the eigenvalues add up to trace(A) and multiply to det(A). For a symmetric matrix the eigenvalues are real and the eigenvectors orthogonal. The order and the sign of the eigenvectors can differ between MATLAB, Octave and NumPy, so never rely on a particular sign.

The singular value decomposition

The SVD writes any matrix, square or not, as A = U * S * V' with orthogonal U and V and a diagonal S of non-negative singular values in descending order. The singular values tell you how much "energy" lives in each direction, which is the basis of compression and principal component analysis.

MATLAB
s = svd([3 0; 4 5])     % 6.7082 and 2.2361
cond([3 0; 4 5])        % 3.0000 = largest / smallest singular value
rank(A)                 % number of singular values above a tolerance

LU, QR and Cholesky

Solving, least squares and positive-definite problems each have a favorite factorization. You rarely call these directly, but they are what backslash uses internally, so it helps to know their names:

  • [L, U, P] = lu(A) gives P*A = L*U with a lower triangular L, an upper triangular U and a permutation matrix P. Two cheap triangular solves then finish the job.
  • [Q, R] = qr(A) gives an orthogonal Q (so Q'*Q is the identity) and an upper triangular R. It is the stable way to solve least-squares problems.
  • R = chol(H) works for symmetric positive-definite H and returns an upper triangular R with R'*R = H. It is about twice as fast as LU and fails with an error if H is not positive definite, which makes it a handy positive-definiteness test.
Python eig, svd, chol, qr and lu

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Note the one convention difference: SciPy returns P with C = P @ L @ U, while MATLAB's P satisfies P*A = L*U, so MATLAB's P is SciPy's P.T. Eigenvectors and QR factors may differ in sign from MATLAB's output; the checks printed above are the part that is guaranteed.

Practical advice

  • Prefer A \ b to inv(A) * b: it is faster and more accurate, and the same holds for b / A over b * inv(A).
  • Never compare det(A) == 0. Rounding makes it a tiny nonzero number; use rank(A) or cond(A) instead.
  • Know your dimensions. Most "dimension mismatch" errors in this area come from a row where a column was needed; check size.
  • Large sparse matrices (mostly zeros) should be stored with sparse(...), which saves memory and makes \ much faster.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Which is the recommended way to solve the linear system A*x = b?

    The backslash operator chooses a stable factorization, so it is more accurate and faster than computing the inverse first.
  2. 2. What does X \ y return when X has more rows than columns?

    For an over-determined system there is usually no exact solution, so backslash returns the coefficients with the smallest residual.
  3. 3. A matrix has rank(S) = 1 and size 2-by-2. What follows?

    A square matrix is invertible only if its rank equals its size. Rank 1 in a 2-by-2 matrix means dependent rows, so the determinant is 0.
  4. 4. Which identity holds for the eigenvalues of a square matrix?

    The sum of the eigenvalues is the trace and the product is the determinant. Eigenvalues can be negative or even complex.
  5. 5. What does R = chol(H) require?

    Cholesky factorization only exists for symmetric positive-definite matrices; otherwise chol raises an error.

Summary

  • det, rank, trace, norm and cond describe a matrix; a square matrix is invertible only when its rank equals its size.
  • Solve A*x = b with x = A \ b, not with inv; for tall matrices backslash gives the least-squares answer.
  • eig returns eigenvalues (and eigenvectors); their sum is the trace and their product the determinant.
  • svd gives singular values, which measure how much each direction matters and define cond.
  • lu, qr and chol are the factorizations behind backslash; chol needs a symmetric positive-definite matrix.
  • NumPy and SciPy stand-ins reproduce the numbers, but sign and permutation conventions can differ from MATLAB.

Next: summarize data with sums, means and extremes in Built-in Math and Statistics.