Skip to content
Tutorials Center
My learning

Topics

Lesson 31 of 39 ยท Statistics and Modeling

Hypothesis Tests in R: t.test, chisq.test, cor.test and More

Run and interpret hypothesis tests in R: t.test (one, two-sample, paired), chisq.test, prop.test, cor.test, shapiro.test, wilcox.test, aov, p.adjust and power calculations.

A hypothesis test asks whether an effect you see in a sample could plausibly be just random noise. You state a null hypothesis (no difference, no relationship), compute how surprising your data would be if that were true, and report the p-value. A small p-value (conventionally below 0.05) means the data would be unusual under the null hypothesis, so you reject it.

R has a function for every common test, and they all return the same kind of object, so once you can read one, you can read them all. This lesson shows the tests you will use most, and how to interpret their output. It builds on descriptive statistics and probability distributions; for the theory, see the Statistics and Probability tutorial.

R is not available on this site, so the code is read-only. The numbers in the comments were cross-checked with SciPy, and you can reproduce them in the Python block near the end.

How to read a test result

Every htest object prints a title, the data used, a test statistic, degrees of freedom where relevant, the p-value, a confidence interval and sample estimates. You can also pull the pieces out of the result:

R
res <- t.test(a, b)
res$statistic     # the t value
res$p.value       # the p-value
res$conf.int      # the confidence interval
res$estimate      # the group means

The p-value is not the probability that the null hypothesis is true, and not the probability the result was due to chance. It is: if there were truly no effect, how often would we see data at least this extreme? Always report the effect size (a difference, a correlation) with its confidence interval, not just "p < 0.05".

t-tests: comparing means

Two groups of measurements, for example plant heights under two fertilizers:

R
a <- c(5.1, 4.9, 6.2, 5.8, 6.0, 5.5, 5.3, 6.1)
b <- c(6.3, 6.8, 5.9, 7.1, 6.6, 7.0, 6.4, 6.9)

t.test(a, b)
# Welch Two Sample t-test
# t = -4.525, df = 13.579, p-value = 0.0005127
# 95 percent confidence interval: -1.4938 to -0.5312
# sample estimates: mean of x = 5.6125, mean of y = 6.625

The group means differ by about 1.01 and the interval does not include 0, so the difference is statistically significant (p = 0.0005). By default t.test() uses Welch's version, which does not assume equal variances, a safe default. Use var.equal = TRUE for the classic pooled test (here t = -4.525, df = 14, p = 0.0004759).

With a formula and a data frame, the group is a column: t.test(height ~ fertilizer, data = plants).

A one-sample test compares a mean with a fixed value: t.test(a, mu = 5). A paired test is for before-and-after measurements on the same subjects, where the pairs are not independent:

R
before <- c(120, 135, 128, 142, 150, 138)
after  <- c(115, 130, 127, 135, 141, 136)
t.test(before, after, paired = TRUE)
# t = 3.9537, df = 5, p-value = 0.01081
# 95 percent CI: 1.6909 to 7.9758;  mean difference = 4.833333

The paired test works on the differences (5, 5, 1, 7, 9, 2), which removes person-to-person variation. Use alternative = "greater" or "less" for one-sided tests, and only when you decided that before looking at the data.

Checking assumptions

The t-test assumes roughly normal data (or a reasonably large sample). Check with a histogram or Q-Q plot (qqnorm(a); qqline(a)) first, and a formal test if you wish:

R
shapiro.test(a)     # W = 0.936, p-value = 0.57  -> no evidence against normality

A large p-value here means "no evidence against normality", not proof of it. With small samples the test has little power. If the data are clearly skewed, use the rank-based alternative:

R
wilcox.test(a, b)                 # Wilcoxon rank-sum (Mann-Whitney) test
wilcox.test(before, after, paired = TRUE)

Categorical data: chi-squared and proportions

A chi-squared test of independence asks whether two categorical variables are related. Suppose 100 people tried two designs, and you counted who clicked:

R
tab <- matrix(c(30, 15, 20, 35), nrow = 2,
              dimnames = list(design = c("A", "B"), clicked = c("yes", "no")))
tab
chisq.test(tab)
# X-squared = 7.9192, df = 1, p-value = 0.004891
chisq.test(tab)$expected          # 22.5 27.5 in both rows

R applies Yates' continuity correction for 2 by 2 tables by default (use correct = FALSE to turn it off). The $expected counts are what you would see if design and clicking were independent; the test measures the gap. Rule of thumb: every expected count should be at least 5; otherwise use fisher.test(tab).

prop.test() compares proportions (for example 30 of 45 versus 20 of 55), and binom.test(14, 20) tests whether 14 successes out of 20 is consistent with a 50 percent chance (p = 0.1153: no evidence against it).

Correlation test

R
cor.test(mtcars$mpg, mtcars$wt)
# t = -9.559, df = 30, p-value = 1.294e-10
# sample estimates: cor = -0.8676594

The correlation of -0.87 is very unlikely to be zero in the population. For skewed data or ordinal ratings use method = "spearman".

Comparing three or more groups: ANOVA

Running many t-tests inflates the false-positive rate. Analysis of variance compares all the group means in one test:

R
fit <- aov(mpg ~ factor(cyl), data = mtcars)
summary(fit)
# F value 39.7 on 2 and 29 degrees of freedom, Pr(>F) = 4.98e-09
TukeyHSD(fit)        # which pairs of groups differ, adjusted for multiple comparisons

A small p-value says "at least one group mean differs"; TukeyHSD() tells you which. For non-normal data use kruskal.test(mpg ~ cyl, data = mtcars).

Multiple comparisons

If you run 20 independent tests at the 5 percent level, one false positive is expected by chance. Adjust the p-values with p.adjust():

R
p <- c(0.01, 0.02, 0.03)
p.adjust(p, method = "bonferroni")   # 0.03 0.06 0.09
p.adjust(p, method = "holm")         # 0.03 0.04 0.04
p.adjust(p, method = "BH")           # 0.03 0.03 0.03  (controls false discovery rate)

Bonferroni and Holm control the chance of any false positive; BH controls the expected share of false positives among the findings and is the usual choice when screening many candidates.

Planning a study: power

Power is the probability that a test detects an effect of a given size. Plan the sample size before collecting data:

R
power.t.test(n = 20, delta = 1, sd = 1)           # power is about 0.869
power.t.test(power = 0.8, delta = 0.5, sd = 1)    # n per group needed (about 64)

An underpowered study will often miss real effects, and the "significant" ones it finds tend to overstate the effect.

Reproduce the numbers in Python

This Python block runs the same tests with SciPy on the same data so you can check the figures quoted above.

Python Python comparison: the tests in this lesson

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Common mistakes

  • Treating "p > 0.05" as proof of no effect. It only means the data are not conclusive.
  • Reporting p-values without effect sizes or confidence intervals.
  • Using an unpaired test on paired data (or the reverse).
  • Testing many hypotheses and reporting only the significant ones; adjust or pre-register.
  • Using chisq.test() on tables with tiny expected counts, or on percentages instead of counts.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What is the default t.test(a, b) in R?

    R's t.test uses Welch's version unless you set var.equal = TRUE or paired = TRUE.
  2. 2. When should you use paired = TRUE?

    Paired data are not independent, so the test works on the within-pair differences.
  3. 3. A p-value of 0.004 means:

    The p-value is a statement about the data given the null hypothesis, not about the hypothesis itself or the size of the effect.
  4. 4. Which function compares the means of three or more groups?

    aov() performs the analysis of variance; follow it with TukeyHSD() to find which pairs differ.
  5. 5. Why use p.adjust()?

    Each extra test adds a chance of a false positive. Adjustments such as Holm or BH account for the number of tests.

Summary

  • Tests return the same structure: statistic, df, p-value, confidence interval, estimates.
  • t.test() (Welch by default; paired, mu, alternative), wilcox.test() for non-normal data.
  • chisq.test(), fisher.test(), prop.test() and binom.test() handle counts and proportions.
  • cor.test() tests correlations; aov() + TukeyHSD() compare several means.
  • Adjust for multiple tests with p.adjust(), plan with power.t.test(), and report effect sizes.

Next: Linear and Logistic Regression in R