Skip to content
Tutorials Center
My learning

Topics

Lesson 37 of 57 ยท Hypothesis Testing

Chi-Square Tests

Use chi-square tests for categorical data: goodness of fit and independence in contingency tables, expected counts, degrees of freedom, assumptions and Cramer's V, with Python.

The t-test and z-test compare numbers such as means. But much data is categorical: which brand a customer picked, whether a patient recovered, which day of the week a call came in. For counts in categories, the standard tool is the chi-square test (written with the Greek letter chi, ฯ‡ยฒ). It compares the counts you observed with the counts you would expect if some hypothesis were true.

The chi-square statistic

For each category, compute (Observed - Expected)^2 / Expected and add them up:

TEXT
chi^2 = sum of (O - E)^2 / E

If observed and expected counts match perfectly, chi^2 = 0. The more they disagree, the larger it gets. Under H0 (and with big enough counts), the statistic follows a chi-square distribution with the right degrees of freedom. It is right-skewed, only takes values at or above 0, and large values give small p-values, so the test is always right-tailed.

Goodness-of-fit test

Question: do the observed category counts fit a claimed distribution? H0 states the claimed probabilities. Expected counts are n x p_i, and df = (number of categories) - 1.

Example: is a die fair? We rolled it 120 times.

Python Is the die fair?

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The statistic (about 4.5) is below the critical value 11.07, so there is no evidence against fairness.

You can also test against uneven expectations, for example the share of customers per region, by supplying the claimed proportions times n.

Python Observed share versus a claimed split

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Test of independence

Do two categorical variables have an association? Arrange counts in a contingency table (rows = one variable, columns = another). H0: the variables are independent. Under independence the expected count in a cell is

TEXT
E = (row total x column total) / grand total
df = (rows - 1) x (columns - 1)
Python Is preference linked to age group?

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

A small p-value says age group and device preference are associated. Cramer's V (0 to 1) measures how strongly: roughly 0.1 small, 0.3 medium, 0.5 large.

To see which cells drive the result, look at the standardized residuals (O - E) / sqrt(E); values beyond about 2 in absolute size stand out.

Python Which cells differ most?

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

2 x 2 tables and the continuity correction

For a 2 x 2 table, chi2_contingency applies Yates' continuity correction by default (correction=True) to compensate for using a continuous curve with discrete counts. When expected counts are small (say below 5 in any cell), use Fisher's exact test instead.

Python Small 2 x 2 table: chi-square versus Fisher

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Assumptions

  • Counts (not percentages) of independent observations; each unit falls in exactly one cell.
  • Random sampling.
  • Expected counts large enough: the usual rule asks every expected count to be at least 5 (some say 80% of cells at 5 or more and none below 1). If not, merge categories or use an exact test.
flowchart TD
    A["Categorical data"] --> B{"One variable or two?"}
    B -- "One variable vs a claimed distribution" --> C["Goodness of fit"]
    B -- "Two variables in a table" --> D["Test of independence"]
    D --> E{"Expected counts all >= 5?"}
    E -- "No" --> F["Fisher's exact test"]
    E -- "Yes" --> G["Chi-square test"]

Which categorical test to use.

A significant chi-square test shows association, not causation, and it does not say what kind or how strong; always inspect the table and report an effect size.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. In a goodness-of-fit test with 5 categories, the degrees of freedom are:

    df = number of categories - 1 = 4.
  2. 2. In a 3 x 4 contingency table, the degrees of freedom are:

    df = (3 - 1)(4 - 1) = 6.
  3. 3. A row total of 80, column total of 50 and grand total of 200 give an expected count of:

    E = 80 x 50 / 200 = 20.
  4. 4. When expected counts are very small in a 2 x 2 table, use:

    Fisher's exact test does not rely on the large-sample chi-square approximation.

Summary

  • Chi-square tests compare observed and expected counts: sum (O - E)^2 / E, always right-tailed.
  • Goodness of fit: one variable against claimed probabilities, df = categories - 1.
  • Independence: two variables in a table, E = row x column / total, df = (r - 1)(c - 1).
  • Check that expected counts are at least about 5; otherwise use Fisher's exact test.
  • Report Cramer's V and residuals to show how strong and where.

Next: ANOVA