Skip to content
Tutorials Center
My learning

Topics

Lesson 38 of 76 ยท Statistics for Data Science

Inferential and Parametric Methods Overview

Understand inferential statistics: estimation versus testing, sampling distributions and standard error, what parametric methods assume, and when to use nonparametric or bootstrap methods.

You almost never see the whole population. A survey reaches 1,200 voters out of millions, an A/B test shows a page to a few thousand visitors, a factory measures 50 parts out of a day's production. Inferential statistics is the set of methods for using a sample to say something careful about the larger group it came from, including how sure you can be.

This lesson is a map of that territory before you dive into the individual tools. You will meet the two jobs of inference (estimation and testing), the idea of a sampling distribution, what parametric methods assume, how nonparametric and bootstrap methods avoid those assumptions, and how to choose. It connects Probability and Distributions to Confidence Intervals and Hypothesis Testing.

Descriptive versus inferential

Descriptive statistics summarise the data you have: the mean of these 200 orders, the median of those response times. Inferential statistics make claims about data you do not have: the mean of all orders, whether a new design is better for future visitors. The jump from sample to population is where uncertainty enters, and inference is the discipline of measuring that uncertainty.

flowchart LR
    P["Population<br/>(unknown parameter)"] -->|"random sampling"| S["Sample<br/>(statistic you compute)"]
    S -->|"inference"| P
    S --> E["Estimation<br/>how big, with what margin?"]
    S --> T["Testing<br/>is there an effect at all?"]

A parameter describes the population; a statistic describes the sample. Inference goes from the second back to the first.

The two jobs: estimation and testing

Estimation asks "how large is it?" You give a point estimate (the sample mean) and an interval estimate (a confidence interval) that shows the plausible range for the true value.

Testing asks "is there evidence of an effect or difference?" You state a null hypothesis (no effect), compute how surprising your data would be if it were true, and report a p-value.

The two views are connected. A 95% confidence interval that excludes zero corresponds to a test of "no difference" that is significant at the 5% level. Estimates carry more information than p-values, because they show the size and the uncertainty. A tiny p-value with a trivial effect is not interesting; a wide interval around a large effect says "promising, but collect more data".

Python Estimation and testing on the same sample

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The last line shows the link: when the interval contains the target value, the two-sided test does not reject it at the same confidence level.

Sampling distributions and standard error

The mean of a sample is itself a random number: take another sample and you get a slightly different mean. The sampling distribution is the distribution of that statistic over all possible samples. Its spread is the standard error (SE). For a mean it is s / sqrt(n): more data and less noise give a smaller SE, but because of the square root, four times as much data only halves the error.

The central limit theorem says the sampling distribution of a mean is close to normal for moderately large samples, even when the data are not. This is why so many methods built on the normal and t distributions work in practice. You can see it by simulation.

Python The sampling distribution of a mean

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The observed spread of the sample means matches sigma / sqrt(n), and the skew of the means shrinks toward zero as n grows: the means become more normal even though the population is far from normal.

Parametric methods

A parametric method assumes the data come from a distribution described by a few parameters, usually the normal distribution with a mean and a standard deviation. The t-test, ANOVA, Pearson correlation and linear regression are parametric. If the assumptions hold, these methods are efficient: they get the most information out of a small sample, and they produce neat formulas for intervals and p-values.

Their usual assumptions are:

  1. Independence: observations do not influence each other (no repeated measurements of the same person treated as separate people).
  2. Normality: the data, or for means the sampling distribution, is roughly normal. Matters most for small samples.
  3. Equal variances: groups being compared have similar spread (Welch's versions relax this).
  4. Scale: the variable is numeric with meaningful distances.

Independence is the one you cannot rescue with a different test. Clustered or repeated data need methods that model the clustering. See Sampling and Bias for how sampling design affects it.

Nonparametric and resampling methods

Nonparametric methods make weaker assumptions. Many replace values with their ranks, which makes them robust to outliers and skew: Mann-Whitney U (instead of the two-sample t-test), Wilcoxon signed-rank (instead of the paired t-test), Kruskal-Wallis (instead of one-way ANOVA) and Spearman correlation (instead of Pearson). The price is a little less power when the parametric assumptions actually hold.

Resampling methods need no formula at all. The bootstrap estimates the sampling distribution by drawing many samples with replacement from your own sample and recomputing the statistic each time. The permutation test shuffles group labels to see how often chance alone produces a difference as large as the one observed.

Python Parametric versus bootstrap on skewed data

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice the t interval is symmetric around the mean while the bootstrap interval is not: it reflects the skew in the sample. Bootstrap intervals for skewed data and small samples are not perfect either, but they adapt to the data's shape instead of assuming it.

How good is the parametric interval? A coverage check

You can test a method by simulation, because you can choose the true value. A 95% interval should contain the true mean about 95% of the time. When the data are skewed and the sample is small, the t interval can fall short.

Python Coverage of the t interval on normal and skewed data

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

For normal data the coverage is close to the promised 0.95 at every sample size. For heavily skewed data it falls below that when the sample is small and approaches 0.95 only as n grows. This is the central limit theorem at work, and a practical reason to look at your data's shape before trusting a formula.

Choosing a method

flowchart TD
    A["What are you comparing?"] --> B{"Numeric outcome,<br/>enough data, roughly normal?"}
    B -->|"yes"| C["Parametric:<br/>t-test, ANOVA, Pearson"]
    B -->|"skewed, outliers, small n"| D{"Is a rank-based question OK?"}
    D -->|"yes"| E["Nonparametric:<br/>Mann-Whitney, Kruskal-Wallis, Spearman"]
    D -->|"need the mean or a custom statistic"| F["Bootstrap or permutation"]
    A --> G["Counts or categories"]
    G --> H["Chi-square, Fisher's exact,<br/>test of proportions"]

A practical decision path. Independence of the observations is required in every branch.

Some guidelines:

  • Large samples (hundreds or more per group): parametric methods are rarely a problem, because the sampling distribution of a mean is nearly normal.
  • Small, skewed or outlier-ridden samples: use rank-based tests, transform the data (for example with a logarithm), or bootstrap.
  • Unusual statistics (median, ratio, correlation difference): bootstrap or permutation.
  • Always report the estimate and an interval together with any p-value.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does the standard error of the mean measure?

    The standard error is the standard deviation of the sampling distribution of the mean, s divided by the square root of n.
  2. 2. Which assumption cannot be fixed just by switching to a nonparametric test?

    Rank-based tests relax normality, but they still assume observations are independent. Clustered or repeated data need different methods.
  3. 3. What does the bootstrap do?

    Resampling the sample many times approximates the sampling distribution without needing a formula.
  4. 4. To make a standard error half as large, roughly how much more data do you need?

    SE shrinks with the square root of n, so halving it needs four times the sample size.

Summary

  • Inference uses a sample to make careful claims about a population, through estimation (how big?) and testing (is there an effect?).
  • The sampling distribution describes how a statistic varies between samples; its spread is the standard error, s / sqrt(n) for a mean.
  • Parametric methods (t-test, ANOVA, Pearson, regression) assume independence and, for small samples, roughly normal data with similar variances.
  • Nonparametric tests use ranks; bootstrap and permutation tests use resampling and need no distribution formula.
  • Check shape and sample size before trusting a formula, and report estimates with intervals.

Next: Confidence Intervals.