Skip to content
Tutorials Center
My learning

Topics

Lesson 39 of 57 ยท Regression

Correlation and Covariance

Learn covariance, Pearson and Spearman correlation, how to read a correlation coefficient, and why correlation does not prove causation, with runnable code.

So far you have described one variable at a time. Real questions usually involve two: does more study time go with higher exam scores? Do hotter days go with more ice cream sales? Covariance and correlation are the first tools for measuring how two numeric variables move together. They also lead directly into regression, the topic of this whole chapter.

In this lesson you will compute covariance and the Pearson correlation by hand, learn when to use Spearman instead, see what a scatter plot can reveal that one number cannot, and meet the most famous warning in statistics.

Covariance: do they move together?

Take paired observations (xโ‚, yโ‚), โ€ฆ, (xโ‚™, yโ‚™). The sample covariance is

cov(x, y) = ฮฃ (xแตข โˆ’ xฬ„)(yแตข โˆ’ ศณ) / (n โˆ’ 1)

Each term is positive when both values are above their means (or both below), and negative when one is above and the other below. A positive total means the variables tend to rise together; a negative total means one rises as the other falls; near zero means no linear pattern.

The problem is the units. Covariance of height (cm) and weight (kg) is in cmยทkg, so you cannot tell whether 120 is "strong" or "weak". Change height to meters and the number shrinks a hundredfold even though nothing real changed.

Python Covariance by hand and with NumPy

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The covariance with minutes is exactly 60 times larger, yet the relationship is identical. That is why we standardize.

Pearson correlation coefficient

Divide the covariance by both standard deviations and the units cancel:

r = cov(x, y) / (sโ‚“ ยท s_y)

Equivalently, r is the average product of the two z-scores. It always lies between โˆ’1 and +1.

r Reading
+1 perfect increasing straight line
0.7 to 0.9 strong positive linear relationship
0.3 to 0.5 moderate
around 0 no linear relationship
โˆ’1 perfect decreasing straight line

The labels are rules of thumb, not laws. What counts as "strong" depends on the field: 0.6 is excellent in psychology and weak in physics.

Two key properties: r is unchanged if you rescale or shift either variable (minutes vs hours gives the same r), and r(x, y) = r(y, x). It measures only straight-line association.

The square of the correlation, rยฒ, is the fraction of the variance in one variable that a straight line through the other explains. It returns in Simple Linear Regression as Rยฒ.

Always plot first

One number can hide the shape. The next example builds three datasets: a straight line, a perfect curve, and a line wrecked by one outlier.

Python Same statistic, different stories

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The parabola has r = 0 even though y is a function of x. The outlier changes r dramatically. Correlation is sensitive to outliers because it is built from means and standard deviations.

Spearman rank correlation

Spearman's ฯ applies Pearson's formula to the ranks of the data. It detects any monotonic relationship (always rising or always falling, not necessarily straight) and is much less affected by outliers. Use it for ordinal data (survey ratings), skewed data, or when the pattern is curved but consistently increasing. Kendall's ฯ„ is a similar rank measure that works well with small samples and many ties.

Python Pearson vs Spearman on a curved trend

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Spearman and Kendall are exactly 1 here because the order is perfectly preserved, while Pearson is lower because the data is not straight.

Testing a correlation

A sample correlation is an estimate. To test Hโ‚€: ฯ = 0 (no linear relationship in the population), the statistic t = rยทโˆš(nโˆ’2) / โˆš(1โˆ’rยฒ) follows a t-distribution with n โˆ’ 2 degrees of freedom. scipy.stats.pearsonr returns this p-value for you. With a large sample even a tiny r such as 0.05 is "significant", and with a small sample even r = 0.5 may not be. Report the size of r together with a confidence interval, not the p-value alone.

Correlation is not causation

Two variables can be correlated because:

  • X causes Y,
  • Y causes X,
  • a third variable (a confounder) drives both, such as hot weather raising both ice cream sales and swimming accidents,
  • or by pure chance, especially when you test many pairs.

Only a well-designed experiment with random assignment (see the track overview) can support a causal claim. Everything else needs careful reasoning about alternatives.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What is the main advantage of correlation over covariance?

    Dividing by both standard deviations cancels the units, so r is comparable across datasets. It is still sensitive to outliers and measures only linear association.
  2. 2. Two variables have Pearson r = 0. What can you conclude?

    r = 0 means no straight-line trend. A parabola such as y = x squared over a symmetric range gives r = 0 although y depends on x completely.
  3. 3. When is Spearman's rank correlation a better choice than Pearson's?

    Spearman works on ranks, so it captures any consistently increasing or decreasing pattern and is robust to extreme values.
  4. 4. A study finds that cities with more ice cream shops have more drownings. The best explanation is:

    Hot weather increases both ice cream sales and swimming, so the two outcomes correlate without either causing the other.

Summary

  • Covariance shows the direction of joint movement but depends on the units.
  • Pearson r standardizes covariance to the range โˆ’1 to +1 and measures straight-line association only.
  • Always draw a scatter plot; outliers and curves can fool r.
  • Spearman and Kendall use ranks and handle monotonic and ordinal data.
  • A significant correlation is not proof of causation: think about confounders.

Next: Simple Linear Regression