Skip to content
Tutorials Center
My learning

Topics

Lesson 59 of 68 ยท Probability and Statistics for Engineers

Inferential Statistics for Engineers

Learn to draw conclusions about a population from a sample: standard error, the central limit theorem, t confidence intervals, hypothesis tests, p-values, error types and power, with Python.

Statistics and Curve Fitting described data: means, spreads, fits. Descriptive statistics summarize the numbers you have. Inferential statistics goes one step further: it uses a sample to say something about the whole population it came from, and it says how sure you can be. Is the new alloy stronger than the old one? Is the sensor biased? Did the redesign cut the defect rate? You cannot test every part ever made, so you test some and reason carefully.

Populations, samples and estimators

  • The population is everything you care about (all bolts from this factory); its fixed but unknown summaries are parameters: mean mu, standard deviation sigma.
  • A sample is a subset you actually measure, ideally chosen at random. Its summaries are statistics: xbar, s.
  • An estimator is a rule that turns a sample into a guess for a parameter. It is unbiased if its average over many samples equals the parameter. The sample mean is unbiased for mu, and the sample variance with divisor n - 1 is unbiased for sigma**2 (that is why np.var(x, ddof=1) uses n - 1).

A statistic changes from sample to sample, so it has its own distribution, the sampling distribution. For the sample mean:

  • its mean is mu,
  • its standard deviation is the standard error SE = sigma / sqrt(n), which shrinks as the sample grows,
  • by the central limit theorem, it is approximately normal for large n, whatever shape the population has.

Quadrupling the sample size only halves the standard error: precision is expensive.

Confidence intervals

A point estimate such as xbar = 52.3 is almost certainly a little wrong. A confidence interval adds the uncertainty. For the mean of roughly normal data with unknown sigma:

xbar +/- t* * s / sqrt(n)

where t* is the critical value of the Student's t distribution with n - 1 degrees of freedom (about 2.0 to 2.3 for 95% unless n is tiny).

What "95% confident" means: if you repeated the whole experiment many times and built an interval each time, about 95% of those intervals would contain the true mean. It does not mean there is a 95% chance that this interval contains it (the true value is fixed; the interval is the random part). The simulation below shows the long-run coverage directly.

Python Sampling distribution, standard error and interval coverage

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Hypothesis tests

A hypothesis test asks whether the data are consistent with a default claim.

  1. State the null hypothesis H0, the "nothing is happening" claim (for example, mu = 50), and the alternative H1 (mu != 50, or mu > 50).
  2. Choose a significance level alpha, commonly 0.05: the largest chance of a false alarm you will accept.
  3. Compute a test statistic, for one mean: t = (xbar - mu0) / (s / sqrt(n)).
  4. Find the p-value: the probability, assuming H0 is true, of getting a result at least as extreme as the one observed.
  5. If p <= alpha, reject H0; otherwise you fail to reject it. Failing to reject does not prove H0; it only means the data do not contradict it.

Common tests:

Question Test
Is one mean equal to a target? one-sample t-test
Do two independent groups have the same mean? two-sample (Welch) t-test
Did the same units change after treatment? paired t-test
Do observed counts match expected counts? chi-square goodness-of-fit
Are two categorical variables independent? chi-square test of independence

Use the version that matches your design. A paired design (before and after on the same parts) is usually far more powerful than treating the two sets as independent.

Python t-tests by hand and in SciPy

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Errors, power and sample size

Any decision can be wrong in two ways.

H0 is actually true H0 is actually false
Reject H0 Type I error (false alarm), probability alpha correct (probability = power)
Do not reject correct Type II error (miss), probability beta

alpha is chosen by you. The power 1 - beta is the chance of detecting a real effect, and it rises with a bigger sample, a larger true effect, a smaller spread, and a larger alpha. Planning a test means choosing n so that power is acceptable (often 80%) for the smallest effect that matters, before collecting data.

A small p-value does not mean a large or important effect; with enough data a trivial difference becomes "significant". Always report the size of the effect and its confidence interval along with the p-value. Also, testing many hypotheses and reporting only the significant ones inflates false alarms; correct for multiple comparisons or pre-register what you test.

Python Type I error rate and power by simulation

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. A sample of n = 25 has s = 10. What is the standard error of the sample mean?

    SE = s / sqrt(n) = 10 / 5 = 2.
  2. 2. What does a 95% confidence interval mean?

    The procedure captures the true value in 95% of repetitions. The true mean is fixed; the interval is the random part.
  3. 3. A test gives p = 0.03 with alpha = 0.05. What is the correct conclusion?

    p <= alpha, so reject H0. The p-value is the probability of data this extreme if H0 were true, not the probability that H0 is true, and it says nothing about effect size.
  4. 4. What is a Type II error?

    A Type II error is a miss: a real effect goes undetected. Its probability is beta, and power is 1 - beta.
  5. 5. Which change increases the power of a test?

    More data shrinks the standard error, making real effects easier to detect.

Summary

  • Inferential statistics uses a sample to draw conclusions, with stated uncertainty, about a population.
  • The sample mean has standard error sigma / sqrt(n) and is approximately normal for large n (central limit theorem).
  • A t confidence interval is xbar +/- t* s / sqrt(n); "95%" describes the long-run coverage of the method.
  • A hypothesis test compares a statistic with what H0 predicts; the p-value measures how surprising the data would be if H0 were true.
  • Type I error probability is alpha; power is 1 - beta and grows with sample size and effect size.
  • Report effect sizes and intervals, not just p-values.

Next: Practice and exercises.