Skip to content
Tutorials Center
My learning

Topics

Lesson 40 of 49 ยท Performance and Internals

Sparse Matrices with SciPy and NumPy

Learn when a sparse matrix beats a NumPy array, how scipy.sparse formats (COO, CSR, CSC, LIL, DOK, DIA) store data, and how to convert, multiply and densify them.

A normal NumPy array stores every element, including all the zeros. For many real problems that is wasteful. A word-count table for a thousand documents and fifty thousand words has fifty million cells, yet each document uses only a few hundred of the words. A graph with a million nodes where each node has ten neighbours has a trillion possible edges but only ten million real ones. Storing the zeros would cost gigabytes for no benefit.

A sparse matrix stores only the non-zero values together with where they are. The scipy.sparse package provides several sparse formats, and each is good at a different job. NumPy itself has no sparse type, so this lesson shows how to move between NumPy arrays and SciPy's sparse matrices, when sparse is worth it, and which format to pick. You should know creating arrays and matrix multiplication first.

When sparse wins

Sparse storage pays off when most entries are zero (often more than 90 to 95 percent) and the matrix is big. It does not pay off for small or mostly full matrices: the bookkeeping costs more than the zeros it saves, and operations are slower than NumPy's tightly optimised dense code.

flowchart TD
    A["Is the matrix large?"] -->|no| D["Use a dense ndarray"]
    A -->|yes| B["Are more than ~90% of entries zero?"]
    B -->|no| D
    B -->|yes| C["Use scipy.sparse"]
    C --> E["Build with COO / LIL / DOK"]
    C --> F["Compute with CSR / CSC"]

The usual decision path: build in a flexible format, convert once, then compute in a compressed format.

Python Dense versus sparse memory

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The six formats at a glance

Every format answers the question "where are the non-zeros?" differently.

Format Class Best for Weak at
COO (coordinate) coo_matrix building from lists of (row, col, value) arithmetic and slicing
CSR (compressed sparse row) csr_matrix fast row slicing, matrix-vector products adding new non-zeros
CSC (compressed sparse column) csc_matrix fast column slicing, some solvers adding new non-zeros
LIL (list of lists) lil_matrix building a matrix row by row, changing entries arithmetic
DOK (dictionary of keys) dok_matrix setting individual entries one at a time arithmetic
DIA (diagonal) dia_matrix matrices with a few filled diagonals everything else

A good habit: build in COO, LIL or DOK, then convert to CSR or CSC for calculations.

COO: three parallel arrays

COO stores a list of row numbers, a list of column numbers and a list of values. Duplicate positions are allowed and are summed when you convert, which makes COO perfect for assembling counts.

Python Building a matrix from coordinates

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

CSR and CSC: compressed storage

CSR keeps three arrays. data holds the non-zero values in row order, indices holds the column number of each one, and indptr says where each row starts and ends inside data. That makes row access and matrix-vector multiplication very fast. CSC is the same idea with the roles of rows and columns swapped.

Python Inside a CSR matrix

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Because indptr has one entry per row plus one, an all-zero row (row 2 here) simply has equal neighbours in indptr.

LIL and DOK: changing entries

CSR and CSC are slow to modify. Setting a new non-zero forces SciPy to reshuffle the arrays, and it warns about this. When you must fill a matrix gradually, use LIL (a list of rows) or DOK (a dictionary keyed by (row, col)) and convert afterwards.

Python Filling a matrix step by step

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

DIA: banded matrices

Matrices that are mostly zero except along a few diagonals, such as the finite-difference matrices behind simulations, fit DIA perfectly. sparse.diags builds one from the diagonals you give it.

Python A tridiagonal matrix

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Operations and conversions

Sparse matrices support +, -, @ (matrix product), .T, .multiply() (elementwise product) and aggregations such as sum(axis=0). Two traps are worth remembering:

  • With scipy.sparse matrix classes, * means matrix multiplication, as in the legacy np.matrix. Use @ for clarity and .multiply for elementwise products. (The newer *_array classes such as sparse.csr_array follow NumPy rules, where * is elementwise.)
  • .toarray() and .todense() build a full matrix. On a large sparse matrix that can exhaust memory, so only call them on small results. .todense() returns an np.matrix, which you usually do not want; prefer .toarray().
Python Arithmetic, reductions and conversion

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Real uses

  • Text features: scikit-learn's vectorizers return CSR matrices of word counts, and most of its models accept them directly.
  • Graphs: an adjacency matrix with an edge per non-zero entry; scipy.sparse.csgraph has shortest-path and connected-component routines.
  • Linear systems and PDEs: scipy.sparse.linalg solves large systems whose matrix is mostly zeros.
  • Recommendation data: a user-by-item matrix where each user rated a handful of items.

If a function you call needs a dense array, convert only what is needed (for example one row with csr[i].toarray()), not the whole matrix.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. You need to build a large matrix by repeatedly assigning individual entries, then use it for many matrix-vector products. Which plan is best?

    LIL and DOK handle gradual changes well, while CSR is fast for products but slow to modify. Convert once after building.
  2. 2. What happens to duplicate (row, col) entries when a COO matrix is converted to CSR?

    COO allows duplicates and sums them on conversion, which is handy for counting.
  3. 3. In CSR, what does the indptr array tell you?

    Row i occupies data[indptr[i]:indptr[i+1]]. The column numbers are in indices.
  4. 4. Why should you avoid calling toarray() on a huge sparse matrix?

    The dense array stores every zero, so a matrix that fitted in megabytes as sparse can need gigabytes as dense.

Summary

  • Sparse matrices store only the non-zeros and win when the matrix is large and mostly zeros.
  • Build with COO (from coordinates), LIL or DOK (entry by entry), then convert to CSR (rows) or CSC (columns) for calculations.
  • CSR uses data, indices and indptr; DIA suits banded matrices; sparse.diags and sparse.eye build common ones.
  • Use @ for matrix products; with the legacy matrix classes * is also a matrix product.
  • Avoid toarray() on big matrices; convert only small pieces.

Next: Writing fast NumPy code.