Skip to content
Tutorials Center
My learning

Topics

Lesson 23 of 38 ยท Regression and Matrices

Correlation Heatmaps and Triangle Masks

Compute a correlation matrix with pandas, draw it as a seaborn heatmap, hide the redundant half with a mask, and read the result without fooling yourself.

When a dataset has a dozen numeric columns, you cannot inspect every scatter plot. A correlation matrix compresses all pairs into one table of numbers between -1 and +1, and a heatmap turns that table into a picture you can scan in seconds. It is one of the most common first steps in exploratory analysis.

This lesson builds a correlation heatmap from scratch, hides the duplicate upper triangle, ranks the strongest pairs, and covers the traps: correlation is not causation, and Pearson's coefficient only sees straight-line relationships.

Computing the matrix

DataFrame.corr() returns a square table. Each cell is the correlation between two columns; the diagonal is always 1 because a column correlates perfectly with itself. Only numeric columns are used, so pass numeric_only=True if your frame also has text columns.

Python A basic correlation heatmap

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Setting vmin=-1 and vmax=1 pins the colour scale to the whole possible range, so a weak 0.2 is not painted as vivid as a strong 0.9. A diverging map such as coolwarm then reads naturally: red positive, blue negative, pale near zero. If you prefer center=0, the effect is similar when the data is balanced, but fixing both ends is the safer habit.

The triangle mask

The matrix is symmetric: the correlation of A with B equals B with A. Half the picture is repeated, and the diagonal is boring. Hide them with a mask built by np.triu():

Python Triangle correlation heatmap

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

np.triu(np.ones_like(corr, dtype=bool)) builds a matrix of True above and on the diagonal. mask= blanks those cells. Use np.triu(..., k=1) instead to keep the diagonal of ones.

Ranking the strongest pairs

A picture is for scanning; a ranked list is for reporting. Unstack the matrix, drop the duplicates and sort by absolute value:

Python Top correlations as a table

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Choosing a method

corr() defaults to Pearson, which measures straight-line association. Two alternatives are often better:

Method Use when
"pearson" Relationships are roughly linear, no big outliers.
"spearman" Relationships are monotonic but curved, or you have ranks.
"kendall" Small samples, many ties.
Python Pearson versus Spearman

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The exponential relationship is perfectly ordered but not straight, so Spearman scores it higher than Pearson.

Reading it honestly

A high correlation is not a cause. Ice-cream sales and drownings correlate because both rise in summer. A correlation near zero does not mean "unrelated" either: a U-shaped relationship can score zero. Check any interesting cell with a scatter or regression plot.

  • Many columns multiply the chance of a fluke. With 20 columns you have 190 pairs; some will look strong by luck.
  • Highly correlated predictors (multicollinearity) make regression coefficients unstable. A heatmap is a fast way to spot them before you model.
  • Correlations are sensitive to outliers; look at the data first.

For clustering similar variables together, continue to clustermaps. To compute correlations in more depth, see pandas correlation analysis.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why is the diagonal of a correlation matrix always 1?

    A variable compared with itself has correlation exactly 1.
  2. 2. What does np.triu(np.ones_like(corr, dtype=bool)) create?

    triu keeps the upper triangle including the diagonal. Pass it as mask= to hide those cells.
  3. 3. Which method suits a curved but always-increasing relationship?

    Spearman uses ranks, so any monotonic relationship scores high.
  4. 4. Why set vmin=-1 and vmax=1?

    Otherwise the colours stretch to the data's own range and exaggerate weak correlations.

Summary

  • df.corr() gives a symmetric matrix; sns.heatmap() makes it visible.
  • Pin the scale with vmin=-1, vmax=1 and use a diverging colormap.
  • Mask the redundant triangle with np.triu(); rank pairs with unstack/stack.
  • Pearson sees straight lines, Spearman sees monotonic trends; neither proves causation.

Next: Clustermaps