Skip to content
Tutorials Center
My learning

Topics

Lesson 36 of 39 ยท Advanced R

R Performance and Packages: Vectorization, Profiling, Your First Package

Speed up R code with vectorization, preallocation and profiling (system.time, bench, profvis), handle big data, run in parallel, then package your work with usethis, roxygen2 and testthat.

R has a reputation for being slow. Usually the code is slow, not the language: a loop that grows a vector, a data frame modified one row at a time, a calculation repeated a million times. This lesson shows how to measure first, fix the usual causes, and then how to package reusable code so that you, your team and CRAN can install it with one command.

R is not available on this site, so the code is read-only. We avoid quoting timings, because they depend on your computer. The Python block at the end measures a similar loop-versus-vectorized difference live so you can see the principle.

Rule one: measure before you optimize

Programmers are notoriously bad at guessing where time goes. Time the slow part:

R
system.time({
  total <- 0
  for (i in 1:1e6) total <- total + i^2
})
#  user  system elapsed  (numbers depend on your machine)

For small, repeatable comparisons use the bench or microbenchmark packages, which run each expression many times:

R
library(bench)
x <- runif(1e5)
mark(
  loop = { s <- 0; for (v in x) s <- s + v },
  vectorized = sum(x)
)

For a long script, a profiler shows which functions eat the time: profvis::profvis({ ... }) draws a flame graph of your code, and base R's Rprof() collects the raw data. Optimize the biggest block, measure again, and stop when it is fast enough. Correct and readable beats fast and clever.

Vectorize

Most R functions work on whole vectors in compiled code. A vectorized expression replaces a loop with a single call:

R
sum((1:1e6)^2)                   # 3.333338e+17, with no loop
x <- c(3, 8, -2, 5)
ifelse(x > 0, "positive", "non-positive")   # vectorized if/else
pmax(x, 0)                       # element-wise maximum with 0: 3 8 0 5
cumsum(x)                        # 3 11 9 14

Helpful vectorized tools: arithmetic and comparison operators, ifelse(), pmin()/pmax(), cumsum(), rowSums()/colMeans(), %in%, match(), which(), outer(), matrix algebra (%*%, crossprod()) and table(). The apply family hides a loop, but it is not faster than a loop by itself; it is tidier. Real speed comes from compiled vectorized functions.

Never grow an object in a loop

Appending with c() copies the whole vector at every step, so the time grows with the square of the length:

R
# slow: reallocates every iteration
res <- c()
for (i in 1:1e5) res <- c(res, i^2)

# fast: allocate once, fill in place
res <- numeric(1e5)
for (i in 1:1e5) res[i] <- i^2

# fastest: no loop
res <- (1:1e5)^2

The same applies to data frames: rbind() inside a loop is a classic cause of slow scripts. Collect the pieces in a list and combine once at the end: do.call(rbind, pieces) or dplyr::bind_rows(pieces).

Other common speedups

  • Work on columns, not rows. df$x[i] <- value in a loop copies repeatedly; modify the whole column.
  • Choose better data structures. Matrices are much faster than data frames for numeric work. Use a named vector or environment as a dictionary for fast lookups instead of searching a data frame.
  • Do less. Filter rows and drop columns before a heavy join or model.
  • Use faster packages for big tables. data.table (fread() for reading, in-place updates, fast grouping) and dplyr with arrow or duckdb handle millions of rows. Base read.csv() is slow on large files; readr::read_csv(), vroom and data.table::fread() are many times faster.
  • Avoid repeating work. Compute a value once and store it, or cache pure functions with memoise::memoise().
  • Mind memory. R holds objects in RAM. Remove what you do not need (rm(big); gc()), read only needed columns (col_select), and check sizes with object.size(x) or lobstr::obj_size(x).

Parallel processing and compiled code

If the work is a set of independent calculations (simulations, one model per group), run them on several cores:

R
library(parallel)
results <- mclapply(1:8, \(i) heavy(i), mc.cores = 4)     # macOS and Linux
cl <- makeCluster(4); results <- parLapply(cl, 1:8, heavy); stopCluster(cl)   # also Windows

The future and furrr packages give one interface for all platforms (furrr::future_map() replaces purrr::map()). Parallelism has overhead, so it only pays off when each task takes noticeably long.

When a loop truly cannot be vectorized (each step depends on the last), write the hot loop in C++ with Rcpp: Rcpp::cppFunction("double f(NumericVector x) { ... }") compiles it and exposes it as an R function. That is the last resort, after vectorizing and choosing better algorithms.

Packages: sharing your code

When you copy the same helper functions between scripts, it is time for a package. A package is a folder with a standard layout, which gives you documentation, tests, dependency management and easy installation:

TEXT
mytools/
  DESCRIPTION        name, version, license, dependencies (Imports:)
  NAMESPACE          exported functions (generated for you)
  R/                 your functions, one or more .R files
  man/               help pages (generated from comments)
  tests/testthat/    tests

The tooling does the boring parts. In a fresh R session:

R
usethis::create_package("~/mytools")     # creates the skeleton and opens the project
usethis::use_r("square")                 # creates R/square.R
usethis::use_testthat()                  # sets up testing
usethis::use_package("dplyr")            # adds a dependency to DESCRIPTION
devtools::load_all()                     # load your code without installing
devtools::document()                     # write man/ pages and NAMESPACE from roxygen comments
devtools::test()                         # run the tests
devtools::check()                        # the full CRAN-style check
devtools::install()                      # install the package locally

Documenting with roxygen2

Special comments above a function become its help page and its entry in NAMESPACE:

R
#' Square a number
#'
#' @param x A numeric vector.
#' @return The squares of `x`.
#' @examples
#' square(3)
#' @export
square <- function(x) {
  stopifnot(is.numeric(x))
  x^2
}

@export makes the function visible to users. Without it the function stays internal. Reach other packages' functions with pkg::fun() (and list the package in Imports:), never with library() inside package code.

Testing with testthat

Tests turn your examples into a safety net. Each test_that() block holds expectations:

R
# tests/testthat/test-square.R
test_that("square works on numbers", {
  expect_equal(square(3), 9)
  expect_equal(square(c(1, 2)), c(1, 4))
})

test_that("square rejects text", {
  expect_error(square("a"))
})

Run them with devtools::test(). Write a test whenever you fix a bug, so it cannot return unnoticed. Use expect_equal() for numbers (it allows tiny floating-point differences), expect_identical() for exact matches, expect_error(), expect_warning() and expect_snapshot() for printed output.

Reproducible projects with renv

Packages change over time. renv records the exact package versions of a project in a lockfile so that the analysis runs the same next year:

R
renv::init()        # create a private library and renv.lock
renv::snapshot()    # record the current versions
renv::restore()     # on another machine: install the recorded versions

Put renv.lock under version control with your code. Together with an RStudio Project (the folder is the working directory, so paths are relative) and a script that runs top to bottom, that is a reproducible analysis.

Where to find packages

  • CRAN: the main repository (install.packages("name")), checked and stable.
  • Bioconductor: genomics and bioinformatics (BiocManager::install("name")).
  • GitHub: development versions (pak::pak("user/repo") or remotes::install_github("user/repo")).

Before depending on a package, look at its documentation, last release date and number of users. Fewer dependencies make your own package easier to maintain.

See the principle in Python

Vectorization is not unique to R. This Python block times two ways of building a list: growing it by copying at every step (like c(res, x) in R) and preallocating it. The numbers will differ on each run, but the growing version is always far slower per element.

Python Python comparison: grow versus preallocate

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Common mistakes

  • Optimizing before measuring, or optimizing code that runs once.
  • Growing vectors and data frames inside loops.
  • Believing sapply() is faster than a loop.
  • Calling library() inside package functions instead of pkg::fun().
  • Skipping tests, then breaking old behavior when adding a feature.
  • Installing packages globally and losing track of versions; use renv.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What should you do before optimizing slow R code?

    Profile first. The slow part is rarely where you expect.
  2. 2. Why is res <- c(res, x) inside a loop slow?

    Preallocate with numeric(n) or vectorize the calculation instead.
  3. 3. Which function builds the help pages and NAMESPACE from roxygen comments?

    document() runs roxygen2, which writes man/*.Rd and NAMESPACE.
  4. 4. What does renv::snapshot() do?

    renv.lock lets you or a colleague restore the same package versions later.

Summary

  • Measure first with system.time(), bench::mark() and profvis; optimize the real bottleneck.
  • Vectorize, preallocate, never grow objects in loops, and use matrices or data.table/arrow for big data.
  • Parallelize independent work (parallel, future); reach for Rcpp only as a last resort.
  • A package gives your code structure: usethis and devtools for the workflow, roxygen2 for documentation, testthat for tests.
  • Use renv to pin package versions for reproducible projects.

Next: Project: Exploring the mtcars Data Set