Skip to content
Tutorials Center
My learning

Topics

Lesson 29 of 39 ยท Statistics and Modeling

Descriptive Statistics in R: mean, sd, quantile, summary, cor

Summarize data in R: mean, median, mode, variance, standard deviation, range, quantiles, IQR, summary(), table(), tapply(), aggregate(), correlation, z-scores and missing values.

R was designed by statisticians, so summarizing data is a one-liner. Descriptive statistics answer two simple questions about a variable: where is the center (a typical value), and how spread out are the values? Add a view of the shape of the distribution and of how variables move together, and you have the first pass of nearly every analysis.

R is not available on this site, so the R code is read-only. Printed output is shown only where it is certain; every number was cross-checked with an equivalent Python calculation, which you can run at the end of the lesson. For the theory behind these measures, see the Statistics and Probability tutorial.

We use a tiny vector so that you can verify every result by hand, then the built-in mtcars data:

R
x <- c(2, 4, 4, 4, 5, 5, 7, 9)

Center: mean, median, mode

R
mean(x)      # 5
median(x)    # 4.5

The mean adds everything up and divides by the count: 40 / 8 = 5. The median is the middle value of the sorted data; with eight values it is the average of the 4th and 5th, (4 + 5) / 2 = 4.5. The mean is pulled by extreme values; the median is not, which is why income and house prices are reported as medians.

R has no built-in function for the statistical mode (mode() reports the storage type). Use table():

R
tab <- table(x)
names(tab)[tab == max(tab)]      # "4"   (the most frequent value)

Other averages: weighted.mean(x, w), and a trimmed mean that ignores the extremes: mean(x, trim = 0.1) drops 10 percent from each end.

Spread: range, variance, standard deviation, IQR

R
range(x)     # 2 9
diff(range(x))   # 7
var(x)       # 4.571429
sd(x)        # 2.13809
IQR(x)       # 1.5

var() and sd() use the sample formula with n - 1 in the denominator. The squared deviations from the mean 5 are 9, 1, 1, 1, 0, 0, 4, 16, which sum to 32; 32 / 7 = 4.571429, and its square root is 2.13809. If you want the population version, multiply by (n - 1) / n. The IQR is the distance between the first and third quartile (here 5.5 - 4 = 1.5) and ignores outliers, like the median.

Quantiles and summary()

R
quantile(x)
#   0%  25%  50%  75% 100% 
#  2.0  4.0  4.5  5.5  9.0 

quantile(x, probs = c(0.1, 0.9))
quantile(x, 0.9, type = 6)         # pick another of the 9 definitions

summary(x)
#    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
#     2.0     4.0     4.5     5.0     5.5     9.0 

summary() is the fastest overview: the five-number summary plus the mean. On a data frame it summarizes every column, and on a factor or character column it gives counts. Quantiles use "type 7" by default, a linear interpolation between neighbors: for the 75th percentile of eight values R takes position 1 + 0.75 * 7 = 6.25, which lies a quarter of the way from the 6th value (5) to the 7th value (7), giving 5.5.

Missing values

One NA poisons most summaries. Remove it for the calculation with na.rm = TRUE:

R
y <- c(1, NA, 3)
mean(y)                # NA
mean(y, na.rm = TRUE)  # 2
sum(is.na(y))          # 1  (how many are missing)

Count the missing values first, and ask why they are missing. Silently dropping them can bias results.

Counts and proportions

R
table(mtcars$cyl)
#
#  4  6  8 
# 11  7 14 

prop.table(table(mtcars$cyl))
#
#       4       6       8 
# 0.34375 0.21875 0.43750 

round(100 * prop.table(table(mtcars$cyl)), 1)    # percentages: 34.4 21.9 43.8

table() works for any categorical variable; give it two vectors for a cross-tabulation: table(mtcars$cyl, mtcars$gear).

Summaries by group

Real questions are comparisons: "mileage by cylinder count".

R
tapply(mtcars$mpg, mtcars$cyl, mean)
#        4        6        8 
# 26.66364 19.74286 15.10000 

aggregate(mpg ~ cyl, data = mtcars, FUN = mean)
#   cyl      mpg
# 1   4 26.66364
# 2   6 19.74286
# 3   8 15.10000

tapply() returns a named vector; aggregate() returns a data frame, and its formula interface accepts several grouping variables (mpg ~ cyl + am). With dplyr the same job reads mtcars |> group_by(cyl) |> summarise(mean_mpg = mean(mpg), n = n()).

Column summaries across a data frame: colMeans(mtcars[, c("mpg", "wt")]) gives 20.09 and 3.22 after rounding, and sapply(mtcars, sd) gives every column's standard deviation.

Standardizing: z-scores

A z-score says how many standard deviations a value lies from the mean: (x - mean(x)) / sd(x). It puts variables on a common scale.

R
round((x - mean(x)) / sd(x), 2)
# -1.40 -0.47 -0.47 -0.47  0.00  0.00  0.94  1.87

scale(x) does the same and returns a matrix. Z-scores also flag outliers: values beyond roughly 3 are unusual. Another rule uses the IQR: anything below Q1 - 1.5 * IQR or above Q3 + 1.5 * IQR is an outlier, which is how box plots decide what to draw as dots.

R
q <- quantile(x, c(0.25, 0.75))
fence <- q[2] + 1.5 * IQR(x)       # 5.5 + 2.25 = 7.75
x[x > fence]                       # 9

Correlation

Correlation measures how strongly two numeric variables move together, from -1 (perfect opposite) through 0 (no linear relation) to +1 (perfect same direction).

R
height <- c(150, 160, 170, 180, 190)
weight <- c(52, 58, 69, 77, 88)
cor(height, weight)               # 0.995979

cor(mtcars$mpg, mtcars$wt)        # -0.8676594
cor(mtcars[, c("mpg", "wt", "hp")])   # correlation matrix
cor(mtcars$mpg, mtcars$wt, method = "spearman")   # rank-based, resists outliers

Heavier cars have lower mileage (r = -0.87). Correlation only captures linear association and says nothing about causation. A scatter plot (base graphics or ggplot2) should always accompany it.

Shape

R's base packages have no skewness function; the formula is short. A positive value means a long right tail (the mean is above the median):

R
skew <- function(v) mean((v - mean(v))^3) / mean((v - mean(v))^2)^1.5

Packages such as e1071 (skewness(), kurtosis()) and moments provide ready-made versions, with slightly different small-sample corrections, so mention which one you used.

The same numbers in Python

This is a comparison you can run: the Python versions of the calculations above, with the same inputs, so you can check the R results yourself.

Python Python comparison: the descriptive statistics above

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Common mistakes

  • Forgetting na.rm = TRUE and getting NA.
  • Reporting the mean of a skewed variable (incomes, durations) without the median.
  • Using mode(x) to get the most frequent value.
  • Mixing up sample and population standard deviation; sd() is the sample version.
  • Reading a correlation of 0 as "no relationship": a curved relation can have r near 0.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What does sd(x) divide by under the square root?

    sd() and var() use the sample formula with n - 1 (Bessel's correction).
  2. 2. Which measure of center is least affected by one extreme value?

    The median depends on the middle of the sorted data, so a single outlier barely moves it.
  3. 3. What does mean(c(4, NA, 8)) return?

    Missing values propagate. Use na.rm = TRUE to calculate with the non-missing values only.
  4. 4. Which call gives the mean of mpg for each value of cyl as a data frame?

    aggregate() with a formula returns a data frame with one row per group. tapply() returns a named vector.
  5. 5. A correlation of -0.87 between mpg and weight means:

    The sign gives direction and the size gives strength. Correlation alone says nothing about causation.

Summary

  • mean(), median(), var(), sd(), IQR(), quantile() and summary() cover center and spread; use na.rm = TRUE for missing values.
  • table() and prop.table() summarize categories; tapply() and aggregate() summarize by group.
  • scale() or (x - mean) / sd gives z-scores; the 1.5 x IQR rule flags outliers.
  • cor() measures linear association, never causation.

Next: Probability Distributions in R