Skip to content
Tutorials Center
My learning

Topics

Lesson 39 of 76 ยท Statistics for Data Science

ANOVA and F-Tests - One-Way, Two-Way and the F-Ratio

Understand the F-distribution, the F-test for variances, one-way ANOVA by hand and in SciPy, two-way ANOVA with interaction, and post-hoc tests, with runnable Python.

A t-test compares two means. What if you have three page designs, four ad campaigns or five store layouts? Running a t-test on every pair raises the false-alarm rate (see Hypothesis Testing). Analysis of variance (ANOVA) answers one overall question instead: are all the group means equal?

The name sounds odd: you analyse variances to learn about means. The trick is to compare two kinds of variation. If group means truly differ, the variation between groups is large compared with the ordinary variation within groups. The ratio of the two is the F-ratio, and it follows an F distribution when the groups are the same. This lesson builds the idea by hand, shows the shortcut in SciPy, and extends it to two factors.

The F distribution and the F-test for variances

If you divide two independent sample variances, each from normal data, the ratio follows an F distribution with two degrees-of-freedom values: one for the numerator and one for the denominator. F is never negative, is right-skewed, and its typical value is near 1 when the two variances are equal. That is also the basis of the simple F-test for equal variances: a ratio far above (or far below) 1 suggests that the spread differs. Hypothesis Testing shows that test on two samples; it is sensitive to non-normal data, so check normality first or use Levene's test.

The same distribution drives ANOVA, but there the two variances are mean squares built from the data, described next.

One-way ANOVA by hand

Imagine three groups with sizes n_1 โ€ฆ n_k, total N, group means and one grand mean. ANOVA splits the total squared variation into two parts:

  • Between-group sum of squares (SSB): for every group, n_i ร— (group mean - grand mean)ยฒ, added up. It measures how far the group means are from each other.
  • Within-group sum of squares (SSW): the squared distance of every observation from its own group mean, added up. It measures ordinary noise.

Divide each by its degrees of freedom to get mean squares: MSB = SSB / (k - 1) and MSW = SSW / (N - k). The F-ratio is

F = MSB / MSW

Under the null hypothesis (all means equal), F follows an F distribution with k - 1 and N - k degrees of freedom. A large F means the group means are far apart compared with the noise, and the p-value is the upper tail probability.

Python One-way ANOVA by hand, then with SciPy

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Interpreting the F-ratio

  • F close to 1: the group means differ no more than noise would produce. Do not reject the null.
  • F clearly larger than the critical value (or p below your chosen threshold, often 0.05): at least one group mean differs. It does not tell you which one.
  • ANOVA assumes (1) independent observations, (2) roughly normal values within each group, and (3) similar variances across groups. It is fairly robust to mild violations when groups are about the same size. With clearly unequal variances use Welch's ANOVA, and with clearly non-normal data use Kruskal-Wallis (both mentioned in Hypothesis Testing).
  • A significant F is only the start. Report an effect size, such as eta squared SSB / (SSB + SSW), the share of variation explained by the factor.

Which groups differ? Post-hoc tests

After a significant ANOVA, compare the pairs with a method that keeps the overall error rate at 5%. Tukey's HSD is the standard choice for all pairs. Before ANOVA, check equal variances with Levene's test (stats.levene): a small p-value means the spread differs between groups.

Python Levene, eta squared and Tukey's HSD

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Two-way ANOVA: two factors at once

One-way ANOVA has one factor (the page design). Two-way ANOVA has two, for example design and device type. It answers three questions in one analysis:

  1. Main effect of factor A: do the average results differ across designs, ignoring device?
  2. Main effect of factor B: do they differ across devices, ignoring design?
  3. Interaction A ร— B: does the effect of the design depend on the device? A design might win on phones and lose on desktops. That is an interaction.

When every combination (cell) has the same number of observations, the sums of squares can be computed directly: SSA from the row means, SSB from the column means, SS(AB) from what is left in the cell means, and SSW from the spread inside cells. Each is divided by its degrees of freedom and compared with MSW. If the interaction is significant, interpret it first: the main effects can be misleading when the factors depend on each other.

Python Two-way ANOVA with interaction, by hand with NumPy

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Look at the cell means in the output: design A loses about 9 seconds on phones, design B only about 2. That is why the interaction row is significant. Plots of the cell means with one line per design make interactions visible: roughly parallel lines mean no interaction, and crossing or diverging lines mean interaction.

In practice you can also use statsmodels (ols plus anova_lm) to fit two-way models with unequal group sizes and more factors; the hand version above shows what it computes.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does a one-way ANOVA F-ratio compare?

    F = MSB / MSW: how different the group means are, relative to ordinary noise.
  2. 2. A one-way ANOVA is significant for three groups. What can you conclude?

    The test is an overall one. A post-hoc test such as Tukey's HSD shows which pairs differ.
  3. 3. With 3 groups and 18 observations in total, what are the degrees of freedom of the F-ratio?

    Between: k - 1 = 2. Within: N - k = 15.
  4. 4. What does a significant interaction in two-way ANOVA mean?

    An interaction means the factors do not act independently; interpret it before the main effects.

Summary

  • The F distribution is the ratio of two variances (mean squares); ANOVA uses it to test several means at once.
  • F = MSB / MSW with k - 1 and N - k degrees of freedom; large F means the means differ more than noise explains.
  • scipy.stats.f_oneway runs one-way ANOVA; check variances with Levene and follow a significant result with Tukey's HSD.
  • Report an effect size such as eta squared, not just the p-value.
  • Two-way ANOVA tests two main effects and their interaction; read the interaction first.

Next: Chi-Square Tests, the test for counts and categories.