Skip to content
Tutorials Center
My learning

Topics

Lesson 19 of 39 ยท Data Frames and Data Wrangling

dplyr Basics: filter, select, mutate, summarise

Wrangle data with dplyr. Learn filter(), select(), arrange(), mutate(), group_by(), summarise() and count(), chain them with the pipe, and compare them with base R.

You can do everything in the last two lessons with base R, but the code gets dense. dplyr, part of the tidyverse, gives data manipulation a small vocabulary of verbs. Each verb does one job, takes a data frame as its first argument and returns a data frame. Because the input and output are the same kind of thing, you can chain verbs with the pipe |> and read an analysis like a recipe.

R
library(dplyr)

Throughout this lesson we use a small sales table (a tibble, the tidyverse data frame from Data Frames):

R
sales <- tibble(
  region  = c("North", "South", "North", "East", "South", "North"),
  product = c("A", "A", "B", "B", "A", "A"),
  units   = c(10, 20, 15, 5, 25, 30),
  price   = c(2, 2, 3, 3, 2, 2)
)

The core verbs

Five verbs cover most work

filter Keep rows
select Keep columns
arrange Sort rows
mutate Add columns

Plus summarise() with group_by().

1 / 2

filter(): keep rows

R
filter(sales, units >= 15)
#> # A tibble: 4 ร— 4
#>   region product units price
#>   <chr>  <chr>   <dbl> <dbl>
#> 1 South  A          20     2
#> 2 North  B          15     3
#> 3 South  A          25     2
#> 4 North  A          30     2

Column names are written bare, with no sales$. Several conditions separated by commas mean AND: filter(sales, region == "North", units > 10). For OR use |, and for membership use %in%: filter(sales, region %in% c("North", "East")). Unlike base-R brackets, filter() silently drops rows where the condition is NA, so the trap from Subsetting Data Frames does not occur.

select(): keep columns

R
select(sales, region, units)       # by name
select(sales, region:units)        # a range
select(sales, -price)              # everything except price
select(sales, starts_with("p"))    # helpers: product, price
select(sales, where(is.numeric))   # by type: units, price

rename(sales, qty = units) changes names without dropping columns, and relocate() moves columns.

arrange(): sort rows

R
arrange(sales, units)              # ascending
arrange(sales, desc(units))        # descending
arrange(sales, region, desc(units))

mutate(): create or change columns

R
sales2 <- mutate(sales, revenue = units * price)
sales2$revenue
#> [1] 20 40 45 15 50 60

You may use a column you just created in the same call, and several columns at once: mutate(sales, revenue = units * price, big = revenue > 40). mutate() keeps all rows. Its sibling transmute() keeps only the new columns.

Conditional columns use the case_when() function from Conditionals:

R
mutate(sales, size = case_when(
  units >= 25 ~ "large",
  units >= 10 ~ "medium",
  TRUE        ~ "small"
))

summarise() and group_by()

summarise() collapses many rows into one:

R
summarise(sales, total_units = sum(units), avg_price = mean(price))

Combined with group_by() it produces one row per group:

R
sales |>
  group_by(region) |>
  summarise(total_units = sum(units), n = n())
#> # A tibble: 3 ร— 3
#>   region total_units     n
#>   <chr>        <dbl> <int>
#> 1 East             5     1
#> 2 North           55     3
#> 3 South           45     2

n() counts the rows in each group. Other common summaries: mean(), median(), min(), max(), sd(), n_distinct(). Pass na.rm = TRUE if the column has missing values.

For the very common "how many of each" question there is a shortcut:

R
count(sales, region, sort = TRUE)
#> # A tibble: 3 ร— 2
#>   region     n
#>   <chr>  <int>
#> 1 North      3
#> 2 South      2
#> 3 East       1

Chaining with the pipe

The pipe |> (see Writing Functions) feeds the result of one step into the next, so a multi-step analysis reads top to bottom:

R
sales |>
  mutate(revenue = units * price) |>
  group_by(region) |>
  summarise(revenue = sum(revenue)) |>
  arrange(desc(revenue))
#> # A tibble: 3 ร— 2
#>   region revenue
#>   <chr>    <dbl>
#> 1 North      125
#> 2 South       90
#> 3 East        15

Read it aloud: "Take sales, add a revenue column, group by region, add up the revenue, sort from largest." Check: North is 20 + 45 + 60 = 125.

flowchart LR
    A["sales"] --> B["mutate<br/>revenue"] --> C["group_by<br/>region"] --> D["summarise<br/>sum"] --> E["arrange<br/>desc"]

Figure: each verb takes a data frame and returns a data frame, so they chain.

Summarizing many columns: across()

R
sales |>
  summarise(across(c(units, price), mean))

across() applies the same function to several columns, which saves repeating yourself. Use it inside mutate() too: mutate(across(where(is.character), toupper)).

More handy verbs

Verb What it does
distinct(df, col) Unique rows or values
slice_head(df, n = 3) First rows (also slice_tail, slice_max, slice_min)
slice_max(df, units, n = 2) Rows with the largest values
pull(df, col) Extract a column as a vector
ungroup() Remove grouping after group_by()
glimpse(df) Compact overview, like str()

Grouping stays active until you remove it. If a later step behaves oddly, you may need ungroup().

dplyr versus base R

Task Base R dplyr
Keep rows df[df$x > 3, ] filter(df, x > 3)
Keep columns df[, c("a", "b")] select(df, a, b)
Sort df[order(df$x), ] arrange(df, x)
New column df$y <- df$x * 2 mutate(df, y = x * 2)
Group summary tapply() / aggregate() group_by() |> summarise()

Neither is "better". dplyr is easier to read and chain; base R needs no packages. You will see both in real code, so it pays to know both.

Common mistakes

  1. Using = instead of == in filter(). R reports an error asking whether you meant ==.
  2. Forgetting to save the result: filter(df, ...) does not change df. Assign it, df <- filter(df, ...), or keep piping.
  3. Writing column names in quotes inside filter() or mutate().
  4. Leaving data grouped by accident; add ungroup().
  5. Loading dplyr after another package and being surprised that filter() changed (name masking, see R Packages).

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Which dplyr verb keeps only some rows?

    filter() keeps rows by condition; select() keeps columns.
  2. 2. What does df |> group_by(g) |> summarise(m = mean(x)) return?

    summarise collapses each group to one row.
  3. 3. How do you add a column total equal to a + b in dplyr?

    mutate() creates or modifies columns and keeps all rows.
  4. 4. What is count(df, region, sort = TRUE) equivalent to?

    count() is a shortcut for group_by, summarise(n = n()) and arrange.
  5. 5. Why is filter(df, x > 3) alone not enough to change df?

    dplyr verbs never modify in place; they return a result.

Summary

  • dplyr verbs each do one job: filter(), select(), arrange(), mutate(), summarise().
  • group_by() plus summarise() produces per-group summaries; count() is a shortcut for counting.
  • Chain verbs with |> so a pipeline reads in order.
  • across() applies one function to many columns; ungroup() removes grouping.
  • dplyr and base R solve the same problems; learn both.

Next: Tidying, Joining and Missing Values