Skip to content
Tutorials Center
My learning

Topics

Lesson 58 of 62 ยท Practice and Review

Practice: Systems and Vector Spaces

Twelve exercises on linear systems, elimination, inverses, LU, span, independence, basis, null space and rank. Every solution runs live in your browser.

This page continues the practice set with the middle of the course: solving systems of equations and understanding the vector spaces behind them. The twelve exercises start with plain solves and elimination, move through inverses and factorizations, and finish with span, independence, basis, null space and rank. Each has a runnable solution with assert lines that fail loudly if a change breaks the answer.

How to use this page

Try each task before you open the solution: sketch the equations, say how many solutions you expect, and guess the rank. Then run the code and compare. When a result surprises you, go back to the related lesson. If you have not done the first set yet, start with Practice: Vectors and Matrices.

Questions to ask about Ax = b

What is the rank? Compare rank of A with rank of the augmented matrix.
Square and invertible? Then one unique solution exists.
Free variables? Then there are infinitely many solutions.
Inconsistent? Then b is outside the column space.
1 / 2

Part 1: Solving systems

Related lessons: Systems of Linear Equations, Gaussian Elimination and Solving Systems with NumPy.

Exercise 1. Solve x + 2y + z = 4, 2x - y + 3z = 3, x + y - z = 0. Then check the answer by computing the residual A @ x - b.

Python Exercise 1: solve and check the residual

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Exercise 2. Without solving, decide for each system whether it has one solution, none or infinitely many, by comparing the rank of A with the rank of the augmented matrix [A | b].

Python Exercise 2: classify by ranks

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Exercise 3. Write your own Gaussian elimination with partial pivoting: forward elimination on the augmented matrix, then back substitution. Compare it with np.linalg.solve on a system that needs a row swap.

Python Exercise 3: Gaussian elimination from scratch

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Exercise 4. The system x + 2y + 3z = 6 and 2x + 4y + 7z = 15 has fewer equations than unknowns. Find one particular solution with lstsq, a vector that spans the null space, and write the general solution. Check that two different members of the family both solve the system.

Python Exercise 4: free variables and the general solution

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The SVD supplies an orthonormal basis for the null space in the last rows of Vt (the rows that correspond to singular values that do not exist or are zero). With rank 2 and 3 unknowns there is exactly one free direction, so the solutions form a line.

Part 2: Inverses and factorizations

Related lessons: Inverse Matrices and LU Decomposition.

Exercise 5. Invert A = [[4, 7], [2, 6]] with the 2 by 2 formula 1/(ad - bc) * [[d, -b], [-c, a]] and compare it with np.linalg.inv. Then show what happens for the singular matrix [[1, 2], [2, 4]].

Python Exercise 5: the 2 by 2 inverse

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Exercise 6. Factor A into P L U with SciPy, check that P @ L @ U rebuilds A, and solve the same matrix against three different right-hand sides while factoring only once.

Python Exercise 6: LU once, solve many times

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Part 3: Spaces, bases and rank

Related lessons: Linear Combinations and Span, Linear Independence, Basis and Dimension, Column Space and Null Space and Rank.

Exercise 7. Is b = (3, 3, 3) in the span of v1 = (1, 0, 1) and v2 = (0, 1, 1)? What about c = (1, 1, 5)? Test by solving a least-squares problem and checking whether the residual is zero.

Python Exercise 7: is it in the span?

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Exercise 8. Decide whether each set of vectors is linearly independent: {(1, 2, 3), (4, 5, 6), (7, 8, 9)}, {(1, 0, 0), (1, 1, 0), (1, 1, 1)} and four vectors in the plane. Use the rank, and for the dependent set find the actual relation.

Python Exercise 8: independence by rank

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Exercise 9. The columns of B = [[1, 1], [1, -1]] form a basis of the plane. Find the coordinates of v = (5, 1) in that basis, then rebuild v from them. What changes if you use the standard basis?

Python Exercise 9: coordinates in a basis

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Exercise 10. For A = [[1, 2, 0, 1], [0, 0, 1, 3], [1, 2, 1, 4]], compute the rank and the dimension of the null space, and check the rank-nullity theorem. Then verify that every basis vector of the null space is sent to zero.

Python Exercise 10: rank-nullity

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Exercise 11. Any matrix of the form outer(u, v) has rank 1. Build a 4 by 5 matrix as the sum of two such outer products, check its rank is 2, and then show that adding a third outer product that lies in the span of the first two keeps the rank at 2.

Python Exercise 11: rank from outer products

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Exercise 12. The system x + y = 2, 2x + 2y = 4 is consistent with infinitely many solutions, so np.linalg.solve refuses it. Find the minimum-norm solution with the pseudoinverse, check it, and show that inv fails.

Python Exercise 12: the minimum-norm solution

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. A has rank 2 and the augmented matrix [A | b] has rank 3. What does that mean?

    A larger rank for [A | b] means b adds a new direction, so b is not in the column space of A and the system is inconsistent.
  2. 2. A is 3 by 6 with rank 3. What is the dimension of its null space?

    Rank-nullity: rank plus nullity equals the number of columns. Here 3 + nullity = 6, so the nullity is 3.
  3. 3. Why is it better to call lu_factor once and lu_solve many times than np.linalg.solve each time?

    Factoring is cubic in the matrix size, while each solve with L and U is only quadratic. Reuse the factors when A stays the same.
  4. 4. Three vectors in 3D have rank 2 when stacked as columns. What does this tell you?

    The rank counts independent columns. Rank 2 with three columns means one is a combination of the others, and the span is a plane.

Summary

  • Compare rank(A) with rank([A | b]) to classify a system without solving it; use np.linalg.solve rather than an inverse when it has one solution.
  • Elimination with pivoting is a short loop; LU factors it once so many right-hand sides are cheap.
  • A set of vectors is independent exactly when the rank of the matrix of columns equals their number; the span test is a least-squares residual of zero.
  • Coordinates in a basis come from solving B @ c = v.
  • Rank plus nullity equals the number of columns; a rank-deficient consistent system has a whole family of solutions, and pinv picks the shortest.

Next: Practice: Eigenvalues and Decompositions.