Skip to content
Tutorials Center
My learning

Topics

Lesson 79 of 182 ยท Delegates, Events and LINQ

Deferred Execution and LINQ Pitfalls

Understand LINQ deferred execution in C# - lazy queries, multiple enumeration, ToList snapshots, captured variables, side effects, stable sorting and LINQ versus loops.

A LINQ query written as a variable is not a result. It is a description of work that runs later, every time you enumerate it. That single fact explains most "LINQ surprises": a list that seems to change by itself, a database called twice, a counter that increments at the wrong moment, an expensive method that runs ten times instead of once.

This lesson explains deferred (lazy) execution precisely, shows which operators are lazy and which are not, and walks through the pitfalls professionals learn the hard way: multiple enumeration, captured variables, side effects, modification during enumeration, and when LINQ costs more than a plain loop. If operators like Where and Select are new to you, start with LINQ Basics.

A query is a recipe, not a result

C# The query sees later changes

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

Where returned an object holding a reference to numbers and the lambda. Each time we enumerated big, it walked the list as it was at that moment. This is deferred execution: the query runs when you iterate, not when you define it.

Which operators are deferred?

  • Deferred (lazy): Where, Select, SelectMany, Take, Skip, TakeWhile, OrderBy, GroupBy, Join, Zip, Concat, Distinct, Union, Chunk, Reverse, OfType, Cast. They return IEnumerable<T> (or IOrderedEnumerable<T>).
  • Immediate: operators that return a single value or a collection: ToList, ToArray, ToDictionary, ToHashSet, ToLookup, Count, Sum, Min, Max, Average, Aggregate, First, Single, Last, Any, All, Contains, ElementAt.

A quick rule: if the return type is a sequence, it is probably deferred; if it is a number, bool, element or concrete collection, it runs now.

There are two flavors of deferral. Streaming operators (Where, Select, Take) produce one element at a time with almost no memory. Buffering operators (OrderBy, GroupBy, Reverse, Join, Distinct) are also deferred, but once you pull the first element they read the entire source before they return it, because they cannot know the smallest element without seeing all of them.

C# Streaming versus buffering

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

The streaming query stopped after finding 2; the buffering query had to read all five values. The yield return method is explained in Iterators and yield; for now it is just a way to make a source that reports each element it produces.

Pitfall 1: multiple enumeration

Every enumeration of a lazy query repeats all the work. That is harmless for a cheap in-memory filter and expensive for a query that reads a file, calls a database or runs a heavy computation.

C# Each foreach re-runs the query

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

The first block runs the selector 15 times (5 items in each of the three queries), and the snapshot runs it exactly 5 times. Materialize a query with ToList() or ToArray() when you will read it more than once, or when it is expensive.

Static analyzers know this problem: you may see warning CA1851 "Possible multiple enumerations of IEnumerable collection". A good design habit is to accept IEnumerable<T> parameters only if you enumerate them once, or take IReadOnlyCollection<T> / IReadOnlyList<T> when you need Count or indexing.

The opposite mistake: materializing too early

ToList() in the middle of a pipeline forces all the work and all the memory up front. Compare:

C# ToList too early wastes work

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

Both give 7, but the early ToList() tested all 1,000 numbers and the lazy version stopped after 7. Materialize at the end of the pipeline (or at a deliberate "snapshot" point), not in the middle.

Pitfall 2: the source changes underneath the query

You saw this in the first example. It cuts both ways: the lazy query is always current, but it can also give a different answer than you expected at definition time:

C# Query result depends on when you enumerate

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

Decide per case: do you want a live view or a frozen snapshot? If a method returns a lazy query over its own mutable field, callers will see future changes and may even trigger work on your object's internals. Return a snapshot (ToList(), ToArray()) or a read-only wrapper if you want to avoid that.

Modifying a collection while enumerating it

A foreach over a List<T> that adds or removes elements throws, because the list's enumerator checks a version number:

C# Collection was modified (fails on purpose)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

The run prints Unhandled exception. System.InvalidOperationException: Collection was modified; enumeration operation may not execute. Fix it by enumerating a snapshot (numbers.Where(...).ToList()) or, for removal, by using numbers.RemoveAll(n => n % 2 == 0).

Pitfall 3: captured variables

A lambda captures variables, not values. A lazy query reads those variables when it runs, which may be after they changed:

C# The captured variable changes before the query runs

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

The first query prints only 5, because threshold was already 4 by the time it ran; the second still prints 4, 5 because it reads the copy. Closures are explained in Lambda Expressions.

The loop variable of a for loop is the classic trap, because there is only one variable for all iterations. A foreach variable is a fresh variable per iteration (since C# 5), so foreach is safe:

C# for versus foreach captured variables

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

In the for version all three queries read the same i, which is 4 when the loop ends, so they all print 0, 4, 8. The foreach version captures a separate i per iteration and prints 0, 1, 2, then 0, 2, 4, then 0, 3, 6. The fix for for is int copy = i; inside the loop, or calling ToList() immediately.

Pitfall 4: side effects inside queries

Because of deferral, a lambda with side effects runs at an unpredictable time and possibly several times, or never:

C# Side effects in a lazy Select

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

You can see the log grow in steps: 0 after defining, 1 after First(), 4 after ToList() (it processed all three again) and 7 after Count() (all three once more). Every enumeration repeated the side effect. Runtimes may skip a selector when they can answer without it (newer .NET versions do that for some Select queries), so even the number of times a side-effecting lambda runs is not something to depend on.

The rule: keep query lambdas pure (no writes, no I/O). If you need a side effect per element, write a foreach loop. LINQ is for describing data, not for performing actions; there is even no ForEach on IEnumerable<T> for this reason (only List<T>.ForEach).

Pitfall 5: ordering, stability and ties

  • OrderBy and ThenBy are stable: equal keys keep their original order. List<T>.Sort and Array.Sort are not stable.
  • Sort keys are computed once per element, but the key selector may still run on every enumeration of the sorted query.
  • GroupBy, Distinct and Join keep first-seen order in practice, and the documentation promises it for Distinct, GroupBy and Join on LINQ to Objects.
  • HashSet<T> and Dictionary<K,V> give no ordering guarantee. A Select over a dictionary prints in whatever order the dictionary yields.
C# Stable sorting

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

Within each age, the first output keeps the original order (Ben before Dan; Ada, Cleo, Eve). Relying on stability is fine for LINQ, but say so in a comment or write the tie-breaker explicitly.

Exceptions are deferred too

Errors in the lambda or the source appear when the query is enumerated, not where it is defined. Argument checks that you write in a method returning a lazy sequence have the same problem:

C# The exception appears late

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

Put your try/catch around where the query is consumed, and validate method arguments eagerly in a non-iterator wrapper method (see Iterators and yield).

LINQ versus loops: counting the work

LINQ is not slow, but each operator adds a layer: an enumerator object per stage, a delegate call per element per stage, and sometimes an allocation. For most application code that cost is irrelevant next to I/O. In a hot loop over millions of items it can matter. Rather than guess with timings, count the delegate invocations:

C# Counting delegate calls in a pipeline

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

The pipeline makes 100 + 50 + 50 delegate calls for 100 numbers, while the loop makes none. Practical guidance:

  • Write LINQ first for clarity. Measure (BenchmarkDotNet, not a hand-rolled stopwatch) before rewriting.
  • In hot paths over Span<T>, arrays and primitive types, a loop avoids allocations and delegates. Rewrites are worthwhile when a profiler points at the query.
  • Avoid repeated Count() or ElementAt(i) inside a loop over a lazy sequence; each call re-walks it.
  • Contains on a List<T> inside a Where is a hidden O(n squared): put the lookup values in a HashSet<T> first.
  • Prefer Any() to Count() > 0, Count property over Count() on a list, and TryGetNonEnumeratedCount when you only want a free count.
  • Use ToList() once, at the end, and ToArray() when you know the result will not change size.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. var q = list.Where(x => x > 5); is followed by list.Add(100); and then foreach (var x in q) .... Is 100 included?

    Where stores a reference to the source and runs only during enumeration, so it sees the list as it is at that time, including 100.
  2. 2. Which of these runs immediately instead of being deferred?

    Count returns a number, so it must enumerate right away. Select, OrderBy and Take return sequences and are deferred.
  3. 3. Why does OrderBy(...).First() read the whole source while Where(...).First() may stop early?

    A sort needs every element before it can yield the smallest. Where streams, so it yields the first matching element as soon as it finds one.
  4. 4. A for (int i...) loop adds Select(x => x * i) queries to a list without materializing them. What do they all see when enumerated after the loop?

    A for loop has one variable i for all iterations, and lambdas capture the variable, not its value. Copy it into a local inside the loop or materialize the query.
  5. 5. When should you call ToList() on a query?

    Queries do not cache. Materialize deliberately when repeated enumeration would repeat expensive work or when the source may change; avoid doing it mid-pipeline.

Summary

  • A LINQ query returning a sequence is a recipe; it runs each time it is enumerated and sees the source as it is then.
  • Streaming operators (Where, Select, Take) yield one element at a time; buffering operators (OrderBy, GroupBy, Reverse, Join) read everything on the first pull. Operators returning values or collections (Count, ToList, First) run immediately.
  • Enumerating a query twice repeats the work; materialize with ToList() or ToArray() at the end of the pipeline or at a deliberate snapshot point, not in the middle.
  • Lambdas capture variables, not values: watch loop variables in for loops and values that change before the query runs.
  • Keep query lambdas free of side effects; errors and side effects happen at enumeration time.
  • OrderBy is stable. Measure before replacing LINQ with loops, and avoid hidden quadratic costs such as Contains on a list inside Where.

Next: LINQ Query Syntax