Skip to content
Tutorials Center
My learning

Topics

Lesson 61 of 62 ยท Practice and Review

Interview Questions

Classic linear algebra interview questions for data science and machine learning roles, with clear answers and runnable NumPy coding solutions.

Linear algebra comes up in interviews for data science, machine learning, graphics and quantitative roles. The questions fall into two groups: conceptual ones that test whether you understand what the objects mean, and short coding ones that test whether you can turn the math into NumPy. This page has both, with answers you can say out loud in a minute or two and code you can run.

A good interview answer has three parts: a one-sentence definition, the geometric or practical meaning, and a small example or a caveat. Practice giving all three.

What interviewers look for

Meaning Can you explain it without formulas?
Connections Rank, determinant and invertibility are one story.
Code Vectorized NumPy, not loops.
Caveats Floating point, conditioning, signs.

Answer in three parts: definition, picture, caveat.

1 / 2

Conceptual questions

1. What does the dot product tell you?

The dot product of two vectors is the sum of the products of matching entries. Geometrically it equals |a| |b| cos(theta), so it measures how much two vectors point the same way. It is zero for perpendicular vectors, positive when the angle is acute and negative when it is obtuse. Dividing by both lengths gives the cosine similarity. Related: The Dot Product.

2. How is the cross product different?

The dot product returns a number; the cross product (3D only) returns a vector perpendicular to both inputs, with length equal to the area of the parallelogram they span. It is anticommutative: a x b = -(b x a). Related: The Cross Product.

3. List equivalent conditions for an n by n matrix to be invertible.

All of these say the same thing: the determinant is nonzero; the rank is n; the columns are linearly independent; the columns span R^n; the null space contains only the zero vector; Ax = b has exactly one solution for every b; zero is not an eigenvalue; all singular values are positive. Being able to move between these statements is exactly what the interviewer wants to see. Related: Inverse Matrices.

4. State the rank-nullity theorem.

For an m by n matrix, rank(A) + nullity(A) = n, where nullity is the dimension of the null space. Intuitively, each of the n input directions either survives (contributes to the rank) or is crushed to zero (contributes to the null space). Related: Rank.

5. What does the determinant mean?

It is the factor by which the linear map scales area (2D) or volume (3D), with the sign recording whether orientation flips. A determinant of zero means the map squashes space into a lower dimension, so it is not invertible. Also: det(AB) = det(A) det(B) and the determinant is the product of the eigenvalues. Related: Determinants.

6. What is an eigenvector?

A nonzero vector that the matrix only stretches (or flips), never turns: Av = lambda v. The eigenvalue lambda is the stretch factor. Eigenvectors reveal the natural axes of a transformation and make powers of a matrix easy. Related: Eigenvalues and Eigenvectors.

7. What do symmetric matrices guarantee?

A real symmetric matrix has real eigenvalues and an orthonormal set of eigenvectors (the spectral theorem), so S = Q diag(lambda) Q^T. Covariance matrices and A^T A are always symmetric, which is why they behave so nicely. Related: Symmetric Matrices and the Spectral Theorem.

The SVD A = U Sigma V^T exists for every matrix, even rectangular ones. The singular values are the square roots of the eigenvalues of A^T A, the columns of V are its eigenvectors, and the columns of U are eigenvectors of A A^T. For a symmetric positive semi-definite matrix the two decompositions coincide. Related: Singular Value Decomposition.

9. How does PCA use linear algebra?

PCA centers the data, forms the covariance matrix (or takes the SVD of the centered data matrix directly), and uses the leading eigenvectors as new axes. The eigenvalues give the variance captured along each axis. Related: Project: Principal Component Analysis.

10. Why should you avoid computing the inverse?

Forming A^-1 costs more than solving, is generally less accurate, and breaks down for ill-conditioned matrices. To solve Ax = b use np.linalg.solve; for least squares use lstsq. Related: Solving Systems with NumPy.

11. What is the condition number?

The ratio of the largest to the smallest singular value. It measures how much relative error in the input can be amplified in the solution of Ax = b. A huge condition number means the problem is nearly singular, and you should not trust the digits of the answer. Related: Numerical Linear Algebra Pitfalls.

12. Why is matrix multiplication not commutative?

Composing transformations depends on order: rotating and then shearing is not the same as shearing and then rotating. Algebraically, AB and BA may not even have the same shape. Matrix multiplication is, however, associative, and distributes over addition.

Coding questions

13. Pairwise cosine similarity without loops

Given a matrix X with one vector per row, compute all pairwise cosine similarities. The trick: normalize each row, then one matrix product does all the dot products.

Python Pairwise cosine similarity

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

14. Is this matrix singular?

Do not test the determinant against zero. Use the rank (which relies on a tolerance), and look at the condition number for "nearly singular".

Python Detecting singular matrices

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

15. Fit a line with the normal equations

Fit y = c0 + c1 x to noisy points, first with the normal equations A^T A c = A^T y, then with np.linalg.lstsq. The two agree on a well-conditioned problem, but lstsq is the safer default.

Python Least squares two ways

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

16. Dominant eigenvalue by power iteration

Repeatedly multiply a vector by A and renormalize. It converges to the eigenvector of the largest-magnitude eigenvalue, and the Rayleigh quotient v^T A v gives the eigenvalue. This is the idea behind PageRank.

Python Power iteration

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

17. Matrix multiplication from scratch

Interviewers sometimes ask for the triple loop. Know it, know it costs O(n^3), and know that NumPy's @ calls an optimized library that is far faster.

Python Triple-loop matrix product

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

18. Is a matrix a rotation?

A 2D or 3D rotation matrix is orthogonal (R^T R = I) with determinant +1. A determinant of -1 means an orthogonal matrix that includes a reflection.

Python Checking for a rotation

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Quick-fire answers

  • Trace? Sum of the diagonal and of the eigenvalues.
  • Rank of a product? rank(AB) <= min(rank(A), rank(B)).
  • Is (AB)^-1 = A^-1 B^-1? No: (AB)^-1 = B^-1 A^-1. The same reversal holds for transposes.
  • What is an orthogonal matrix? One with Q^T Q = I, so it preserves lengths and angles; its inverse is its transpose.
  • Positive definite? A symmetric matrix with x^T A x > 0 for every nonzero x, equivalently all eigenvalues positive.
  • How many solutions can Ax = b have? Zero, one, or infinitely many; never exactly two.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. An n by n matrix has a zero eigenvalue. Which statement must be true?

    The determinant is the product of the eigenvalues, so a zero eigenvalue gives determinant zero, which means the matrix is not invertible.
  2. 2. What is (AB)^-1, if A and B are invertible?

    Undoing a composition reverses the order: you undo B first, then A. Socks and shoes: the last thing you put on comes off first.
  3. 3. A 5 by 8 matrix has rank 3. What is the dimension of its null space?

    By rank-nullity, nullity = n - rank where n is the number of columns: 8 - 3 = 5.
  4. 4. How many solutions can a linear system Ax = b have?

    If two different solutions exist, every point on the line through them is also a solution, so there are infinitely many.

Summary

  • Know the equivalent invertibility conditions and move between them: determinant, rank, independence, null space, eigenvalues.
  • Give answers in three parts: definition, geometric meaning, caveat.
  • Rank-nullity: rank + nullity = number of columns.
  • Eigenvalues multiply to the determinant and add to the trace; singular values are square roots of eigenvalues of A^T A.
  • Code answers should be vectorized (normalize then @), use solve/lstsq instead of inv, and compare floats with tolerances.
  • (AB)^-1 = B^-1 A^-1 and (AB)^T = B^T A^T: the order reverses.

Keep everything on one page with the Linear Algebra Cheat Sheet.