Skip to content
Tutorials Center
My learning

Topics

Lesson 104 of 182 ยท Async and Concurrency

Async Streams

Learn async streams in C# - IAsyncEnumerable of T, await foreach, async iterator methods with yield return, cancellation with EnumeratorCancellation, paging APIs, cleanup and async LINQ in .NET 10.

Sometimes a result arrives piece by piece, over time: rows streaming from a database, pages from a web API, lines from a log being written, messages from a socket. You have two bad options with what you already know. A method returning Task<List<T>> makes the caller wait for everything and hold it all in memory. A method returning IEnumerable<T> can produce items lazily, but it can only wait by blocking a thread.

C# 8 added the missing combination: async streams. A method returns IAsyncEnumerable<T>, produces items with yield return, can await between items, and the caller consumes them with await foreach. If you know iterators and yield and async/await, this lesson is the natural meeting of the two.

The idea in one picture

One item Many items
Synchronous T IEnumerable<T>
Asynchronous Task<T> IAsyncEnumerable<T>

An async stream is "an IEnumerable<T> where getting the next item may take time, and you await for it".

Producing a stream: async iterator methods

A method becomes an async iterator when it is declared async, returns IAsyncEnumerable<T>, and uses yield return. Inside it you can use await freely. The compiler turns it into a state machine like any other iterator, which pauses at each yield return until the consumer asks for the next item.

C# An async iterator and await foreach

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

Look at the order of the output. The producer's code runs only when the consumer asks for the next value: yield return hands the item over and the method sleeps until the next loop iteration. Nothing is computed ahead, and the producer and consumer take turns.

await foreach is the only syntax you need to read a stream. It awaits MoveNextAsync() before each iteration, and when the loop ends (normally, with break, or by an exception) it awaits DisposeAsync() on the enumerator so the producer can clean up.

Why not return a List?

Streams matter when data is big, slow or endless:

  • Memory: a stream holds one item at a time. A file with 10 million lines never needs 10 million strings in memory.
  • Latency: the first item reaches the consumer as soon as it is ready, not after the last one.
  • Early exit: if the consumer stops after 5 items, the producer does no more work.
  • Endless sources: a stream of events from a sensor never "finishes" yet is perfectly consumable.
C# Early exit stops the producer

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

Although the source could produce a million values, only six were ever created: five used, plus the sixth that triggered break. A Task<List<int>> version would have built all million first.

Cleanup with try/finally

Because await foreach disposes the enumerator, a try/finally (or using/await using) inside an async iterator runs even when the consumer breaks early or an exception is thrown. This makes streams a good fit for resources like database readers and files.

C# finally runs when the consumer stops early

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

"closing the source" appears before "consumer is done": breaking out of the loop disposed the enumerator, and disposal ran the finally. (For the full story on disposal, see IDisposable and using.) A yield return is not allowed inside a try block that has a catch clause, but try/finally is fine.

Cancellation: [EnumeratorCancellation]

Long streams need a way to be stopped from outside. The consumer passes a CancellationToken with WithCancellation(token), and the producer receives it in a parameter marked [EnumeratorCancellation]. The attribute tells the compiler "when the consumer supplies a token, merge it into this parameter".

C# WithCancellation and EnumeratorCancellation

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

The while (true) stream would run forever, but cancellation makes the awaited delay throw OperationCanceledException, which travels out through await foreach. If you also want to take a token as a normal argument (TicksAsync(token)), both ways work with the same attribute. Pass the token to every await inside the producer; a producer that never checks it cannot be cancelled. See Cancellation and Timeouts.

Real-world shape: paging an API

The most common reason to write a stream is a paged API: you fetch page 1, then page 2, and so on, until a page says there is no more. The caller does not want to know about pages, just "give me all the orders".

The example fakes the web service with a local method, which makes the output deterministic and runnable. The idea transfers directly to HttpClient calls.

C# Streaming a paged API

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

Reading all 23 orders takes three requests (10 + 10 + 3). Stopping after four orders takes just one, because the next page is only fetched when the consumer asks for item 11. A Task<List<string>> GetAllOrdersAsync() would always fetch everything.

Async LINQ in .NET 10

Until recently, filtering and projecting a stream needed an extra NuGet package. .NET 10 includes async LINQ in the base library (System.Linq.AsyncEnumerable): Where, Select, Take, ToListAsync, CountAsync, FirstAsync and many more work directly on IAsyncEnumerable<T>. They return new streams (still lazy) or ValueTask<T> results you await. If you must target an older .NET version, the same names come from the System.Linq.Async package.

C# LINQ over an async stream

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

Note the await before ToListAsync() and CountAsync(): these are the operators that actually run the stream. Where, Select and Take only describe the pipeline, just like in ordinary LINQ with deferred execution.

Streams and channels

A stream is pull-based (the consumer asks). A channel is push-based (producers write when they like). They connect easily: channel.Reader.ReadAllAsync() turns a channel into an IAsyncEnumerable<T>, so any code written for streams can consume a channel.

C# Turning a channel into a stream

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

Errors in a stream

An exception thrown inside the iterator surfaces at the await foreach line, on the iteration that needed the failing item. Items already yielded were delivered normally, and the finally blocks run. After the exception the stream is finished; you cannot resume it.

C# An exception inside a stream

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

Using the enumerator by hand

Occasionally you need more control than await foreach gives, for example to read the first item separately. GetAsyncEnumerator() returns an IAsyncEnumerator<T> that you drive with MoveNextAsync() and dispose with await using.

C#
await using IAsyncEnumerator<int> e = NumbersAsync().GetAsyncEnumerator();
if (await e.MoveNextAsync())
{
    Console.WriteLine($"first = {e.Current}");
}
while (await e.MoveNextAsync())
{
    Console.WriteLine($"next  = {e.Current}");
}

The await using is not optional: it is what runs the producer's finally blocks if you stop before the end.

Guidelines

  • Return IAsyncEnumerable<T> when items arrive over time or the full set is large; return Task<List<T>> (or an array) when the caller needs all items anyway and the set is small.
  • Always accept an [EnumeratorCancellation] CancellationToken in public streaming methods and pass it to every awaited call.
  • Name streaming methods with the Async suffix, as for other async methods.
  • A stream is a cold sequence: each await foreach starts the producer again from the beginning. If the work is costly, do not enumerate the same stream twice; materialize with ToListAsync() first.
  • Do not share one enumerator between threads. await foreach is sequential by design.
  • In ASP.NET Core, a controller or minimal API endpoint that returns IAsyncEnumerable<T> sends items to the client as they are produced (a JSON array streamed element by element).
  • To consume items concurrently, read the stream and push items into a channel with several workers.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What three things make a method an async iterator?

    An async iterator is declared async, returns IAsyncEnumerable<T> (or IAsyncEnumerator<T>) and contains yield return.
  2. 2. A consumer breaks out of await foreach after two items. What happens to a try/finally in the producer?

    await foreach awaits DisposeAsync when the loop ends for any reason, which runs the iterator's finally blocks.
  3. 3. What is the [EnumeratorCancellation] attribute for?

    Mark the CancellationToken parameter of the iterator so tokens given via WithCancellation(token) flow into it.
  4. 4. Why is a stream better than Task<List<T>> for a paged API when the caller wants only the first 10 items?

    An async stream is lazy: if the consumer stops early, no further pages are requested.
  5. 5. In .NET 10, which call turns IAsyncEnumerable<int> source into a list?

    ToListAsync is an async LINQ operator that runs the stream and returns a ValueTask<List<T>> you await.

Summary

  • IAsyncEnumerable<T> is a sequence whose items arrive asynchronously; produce it with an async method that uses yield return, consume it with await foreach.
  • Streams are lazy and pull-based: the producer runs only when the consumer asks, so early break saves work and memory.
  • try/finally in the producer runs on completion, early exit and errors because await foreach disposes the enumerator.
  • Support cancellation with [EnumeratorCancellation] CancellationToken and WithCancellation(token), and pass the token to every await.
  • Paged APIs are the classic use; .NET 10 adds async LINQ (Where, Select, Take, ToListAsync, CountAsync) in the base library.
  • channel.Reader.ReadAllAsync() turns a channel into a stream.

Next: Async Patterns and Pitfalls