Skip to content
Tutorials Center
My learning

Topics

Lesson 146 of 171 ยท Modern C++ and Beyond

Writing Fast C++

Learn to write fast C++ - measure with chrono, cache-friendly data layouts, vector versus list, avoiding copies and allocations, reserve, move, small string optimization and inlining.

C++ gives you control over memory layout, copies and allocations, so it can be extremely fast. It can also be surprisingly slow when that control is used carelessly: a loop that copies a vector on every iteration, a linked list where an array would do, or a million tiny heap allocations can make C++ code slower than the same idea in a "slow" language. Speed in C++ comes much less from clever tricks than from a handful of habits about data and memory.

This lesson starts with the rule that matters most, measure first, and a reliable way to time code. Then it covers the factors that dominate real-world performance: how the CPU cache rewards contiguous data, why std::vector beats std::list for almost everything, how to avoid needless copies and allocations, what the small string optimization does, and where inlining and the optimizer fit in. Every claim comes with a program you can run and measure yourself. It builds on std::vector in Depth, Move Semantics and Compilers, Flags and Optimization.

Measure first

Programmers are notoriously bad at guessing where time goes. The function you suspect is often irrelevant, and the real cost hides in an innocent-looking line. So the workflow is always: make it correct, measure, find the hot spot, improve that, measure again. Optimizing code that runs for 1% of the time can at best make the program 1% faster.

For quick measurements, std::chrono::steady_clock (a clock that never jumps backwards, unlike the system clock) is all you need (see Dates, Times and Durations with chrono). A small helper keeps the timing code out of the way:

C++ A reusable timing helper

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with std::cin or std::getline), it gets the next line from this box.

Three habits make measurements trustworthy:

  • Always measure optimized builds (-O2 or at least -O1, as our editor uses). Unoptimized timings are meaningless.
  • Use the result. If a computed value is never used, the optimizer may delete the computation, and you'll "measure" an impossibly fast nothing. Printing the result, as the helper does, prevents that.
  • Repeat and take the minimum or median. Single runs are noisy, especially on a shared server like ours.

For serious work, use a profiler to find hot spots in the whole program (perf on Linux, Visual Studio's profiler, Instruments on macOS, or Intel VTune) and a micro-benchmark library such as Google Benchmark, which handles warm-up, repetition and statistics for you.

The CPU cache: memory layout is speed

A modern CPU can do several arithmetic operations per nanosecond, but fetching a value from main memory takes on the order of 100 nanoseconds. Caches hide that gap: small, fast memories (L1, L2, L3) keep recently used data close to the processor. Memory moves between RAM and cache in cache lines of 64 bytes, and the hardware prefetcher notices sequential access patterns and loads the next lines before you ask for them.

So code that walks through memory sequentially is fast, because every cache line brought in is fully used and the next one is already on its way. Code that jumps around memory pays the full trip to RAM again and again. The clearest demonstration is the same sum done two ways over a 2D grid stored row by row:

C++ Row-by-row versus column-by-column

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with std::cin or std::getline), it gets the next line from this box.

Both loops add the same four million integers, yet on our server the column-by-column loop is typically 10 to 20 times slower. Each of its steps lands on a different cache line, and only 4 of every 64 bytes it loads are used before the line is evicted. Nothing about the arithmetic changed; only the order of memory accesses. The fix is usually just swapping the loops so the innermost one walks contiguous memory (see Multidimensional Arrays and Matrices).

std::vector versus std::list

Big-O analysis says a linked list has O(1) insertion in the middle while a vector has O(n), so lists should win for some jobs. In practice, on modern hardware, std::vector wins almost always: its elements are contiguous, so traversal streams through the cache, while every std::list node is a separate heap allocation with two pointers of overhead, and visiting the next node means chasing a pointer to an unpredictable address.

C++ Traversing a vector and a list

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with std::cin or std::getline), it gets the next line from this box.

The list traversal is slower by a factor of a hundred or more on our server for the same million values (the vector loop takes well under a millisecond), because almost every node visit is a cache miss. The list also uses about six times as much memory (each 4-byte int drags along two 8-byte pointers plus padding, before the allocator's own overhead). Even "insert in the middle" usually favors a vector for anything but very large elements, because finding the position in a list is a slow traversal, while a vector shifts its elements with a fast, cache-friendly memmove.

Use std::list only when you truly need its guarantees: iterators and references that stay valid while other elements are inserted and erased, or splice to move nodes between lists without copying (see std::deque, std::list and std::forward_list). The same logic favors sorted vectors or std::flat_map over std::map for read-mostly lookups, and arrays of plain values over arrays of pointers to heap objects.

Avoiding copies

Copying a std::string or std::vector means allocating memory and copying every element. A few habits remove almost all unintended copies:

  • Pass large objects by const& when the function only reads them, or by std::string_view/std::span for strings and contiguous ranges (see Parameters).
  • Loop with const auto&: for (auto s : names) copies every string; for (const auto& s : names) doesn't.
  • Return by value. Returning a local object uses copy elision or a move, never a deep copy (see Returning Values).
  • Move when you're done with something: list.push_back(std::move(line));, or take a "sink" parameter by value and move it into place.
  • Construct in place with emplace_back when building elements from arguments.

Copies are easy to miss because they're silent. One way to see them is to count heap allocations: replacing the global operator new (allowed by the standard, and exactly what memory profilers do) lets a program report every allocation it makes.

C++ Counting the allocations behind copies

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with std::cin or std::getline), it gets the next line from this box.

The numbers tell the story. Passing the vector by value and looping by value makes 2001 allocations (the vector's buffer, 1000 string copies for the parameter, and 1000 more for the loop variable); the const& version makes zero. Growing a vector without reserve reallocates every time capacity runs out (about 18 times for 100,000 ints with libstdc++'s doubling growth), while reserve makes it exactly one. And moving a string into a container transfers its buffer instead of copying it, saving one allocation per element.

Replacing the global operator new like this is a real debugging technique, but keep it out of production code. Two details make it reliable under optimization: the counter is a std::atomic, because GCC otherwise assumes an allocation can't change an ordinary global and may reuse a stale value of the counter, and the [[gnu::noinline]] on operator delete silences a false -Wmismatched-new-delete warning from GCC 15. In a real project, tools such as heaptrack, Valgrind's massif, or your platform's profiler report allocations without code changes.

Small string optimization

Look again at that example: the strings are 40 and 100 characters long, on purpose. Every major standard library implements the small string optimization (SSO): a std::string object contains a small internal buffer, and short strings are stored right inside the object with no heap allocation at all. Only strings that don't fit allocate.

C++ Where short strings live

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with std::cin or std::getline), it gets the next line from this box.

With libstdc++ (GCC), a std::string is 32 bytes and holds up to 15 characters inline; the 16th character forces a heap allocation. libc++ (Clang) stores up to 22 characters in a 24-byte object, and MSVC's library up to 15. The exact limit is an implementation detail, so never rely on it for correctness, but it explains why programs that juggle many short strings (names, keys, tokens) are faster than you'd expect, and why std::string_view matters most for long strings. A related trick, the small buffer optimization, is used by std::function and many third-party containers, for example small_vector types that keep the first few elements inline.

Inlining and abstraction cost

Modern C++ is built on small functions: getters, operators, lambdas, iterators and templates. In optimized builds these are inlined (their bodies are copied into the caller), so a call like v[i] or std::ranges::count_if(values, pred) compiles to the same machine code as a hand-written loop. That's what "zero-cost abstractions" means. A few practical consequences:

  • Don't avoid small functions or standard algorithms for speed. They're free when optimized, and they're often faster than hand-written loops because the library implementers tuned them.
  • Templates and lambdas inline well; calls through function pointers, std::function and virtual functions usually can't be inlined unless the compiler can see the exact target. In a hot loop over millions of elements, prefer a template parameter or a lambda passed to a template over std::function (see Function Objects and std::function).
  • Code in other .cpp files can't be inlined unless you enable link-time optimization (-flto). That's one reason small, hot functions live in headers.
  • The inline keyword is about linkage, not speed; the optimizer decides what to inline (see inline Functions).

A performance checklist

When a program is too slow, work through this list in order, measuring after each change:

  1. Is it built with optimization? Release builds with -O2 are often 5 to 10 times faster than debug builds.
  2. Is the algorithm right? An O(nยฒ) loop over a million items loses to any O(n log n) solution, no matter how well tuned. Use the right container: hash lookups (std::unordered_map) or sorted data with std::lower_bound instead of linear searches.
  3. Is the data contiguous? Prefer std::vector and arrays of values; avoid pointer-chasing structures and scattered heap objects in hot paths.
  4. Are there hidden copies or allocations? Pass by const&, loop by const auto&, reserve vectors, move instead of copy, reuse buffers across loop iterations instead of creating new strings and vectors inside the loop.
  5. Is the work done more than once? Hoist invariant computations out of loops, cache results, and avoid recomputing size() of things that are expensive to measure (like strlen).
  6. Only then consider lower-level work: -O3, -march, LTO and PGO, data layout changes like structure-of-arrays, SIMD, or multithreading with threads or parallel algorithms.

And keep the code clear. Most of a program's time is spent in a small part of it; the rest should be optimized for readability.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. A benchmark computes a large sum but never prints or otherwise uses it, and it reports 0 ms. What's the most likely explanation?

    Work whose result isn't observable can be deleted under the as-if rule. Always use the result of benchmarked code, for example by printing it.
  2. 2. A grid is stored row by row in one vector. Which loop order is faster for summing every element?

    Walking along a row visits neighboring addresses, so every cache line is fully used and the prefetcher can stream data. Walking down a column jumps a whole row each step.
  3. 3. Why is traversing a std::list usually much slower than traversing a std::vector of the same values?

    Vector elements are contiguous, which the cache and prefetcher love. List nodes may be anywhere, and each also carries two pointers of overhead.
  4. 4. for (auto s : names) loops over a vector of 1000 long strings. What does it cost compared with for (const auto& s : names)?

    auto deduces std::string, so each iteration copy-constructs a new string. For strings longer than the small-string buffer, every copy allocates.
  5. 5. With GCC's standard library, how many heap allocations does creating std::string s(10, 'x'); make?

    libstdc++ stores up to 15 characters inside the string object itself. Only longer strings allocate.

Summary

  • Measure first, always with optimized builds; time with steady_clock, use the results, repeat runs, and use a profiler for whole programs.
  • Memory access patterns dominate: sequential access over contiguous data is fast thanks to cache lines and prefetching; jumping around memory is slow.
  • Prefer std::vector (and sorted vectors) over std::list and std::map for most workloads; keep values contiguous instead of scattered behind pointers.
  • Avoid hidden copies and allocations: const& parameters, const auto& loops, reserve, std::move, emplace_back, reused buffers.
  • The small string optimization stores short strings (up to 15 characters in libstdc++) without allocating.
  • Small functions, templates and lambdas are inlined away in optimized builds; indirect calls through std::function or virtual functions may not be.

Next: Design Patterns in C++, where classic patterns like factory, observer, strategy and CRTP get modern C++ implementations.