Skip to content
Tutorials Center
My learning

Topics

Lesson 117 of 182 ยท Standard Library

Streams, Readers and Writers

Learn C# streams - Stream, MemoryStream, StreamReader and StreamWriter, BinaryReader and BinaryWriter, buffering, GZip and Brotli compression, and pipelines.

A stream is a sequence of bytes you read or write in order, without needing the whole thing in memory. A file on disk, a network connection, the bytes of an HTTP response, an in-memory buffer and a compressed archive are all streams, and in .NET they all share the abstract base class System.IO.Stream. Code written against Stream works with any of them.

This lesson builds from the base class up: reading and writing raw bytes, wrapping a stream in text readers and writers, binary formats, buffering, copying, compression, and a short look at pipelines for high-throughput code. We use MemoryStream for most examples because it behaves like a file but needs no disk, so the output is identical on every run.

The Stream class

Every stream has the same small set of members. Not every stream supports all of them, so check the Can... properties when you accept a stream from someone else:

Member Meaning
Read, ReadAsync Copy up to N bytes into your buffer; returns how many were copied (0 = end)
Write, WriteAsync Send bytes to the stream
Position, Seek Where the next read or write happens (only if CanSeek)
Length Total size (only if CanSeek)
Flush Push buffered bytes to the destination
Dispose / Close Release the underlying resource
CanRead, CanWrite, CanSeek Capabilities
C# MemoryStream, Position and Seek

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

The most common beginner bug is forgetting to rewind. After you write 5 bytes, Position is 5, so a Read immediately returns 0 bytes. Set Position = 0 (or Seek(0, SeekOrigin.Begin)) before reading what you just wrote.

Read may return fewer bytes than you asked for

Read(buffer, 0, 100) is allowed to return 7. On a network stream it returns whatever has arrived. Always loop until you have what you need or Read returns 0:

C# The read loop

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

ReadExactly (since .NET 7) is the safe way to read a fixed-size header. Old code that called Read once and ignored the return value is a source of rare, hard-to-reproduce bugs.

Text: StreamWriter and StreamReader

Bytes are not text. A StreamWriter turns strings into bytes with an encoding (UTF-8 by default) and a StreamReader turns them back:

C# StreamWriter and StreamReader

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

Two details deserve attention:

  • Disposal flushes. StreamWriter keeps text in an internal buffer. If you never dispose or flush it, the last bytes are lost. using is the fix.
  • Disposal closes the stream under it unless you pass leaveOpen: true. Without it, ms would be closed when the writer ends and ms.Position = 0 would throw ObjectDisposedException.

A StreamReader can also read in other ways: Read() returns one char, Read(char[], ...) fills a buffer, and ReadToEnd() returns everything left. ReadLine() returns null at the end, so test for null, not for an empty string (an empty line is a legitimate "").

Encodings and byte-order marks

C# Encodings change the byte count

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

ASCII cannot represent รผ or ไธ–, so those characters turn into ?. When you read a file, a StreamReader detects a byte-order mark (BOM) automatically; when no BOM is present it assumes UTF-8. More background is in Unicode and Encoding.

Binary data: BinaryWriter and BinaryReader

For compact, fixed-layout files (game saves, protocols, caches) use BinaryWriter and BinaryReader. They write primitives in little-endian binary form and strings with a length prefix:

C# BinaryWriter and BinaryReader

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

You must read values in exactly the order and types you wrote them; the format has no self-description. 1 byte of length prefix + 6 characters + 4 + 8 + 1 + 2 = 22 bytes. For anything that must survive across program versions, add a version number as the first field. If you need self-describing data, use JSON or XML instead.

For arrays of primitives and structs, System.Buffers.Binary.BinaryPrimitives reads and writes integers with an explicit byte order directly in a Span<byte>, which is how network protocols do it (big-endian):

C# BinaryPrimitives and endianness

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

Buffering

Every call to a file or network stream is a trip into the operating system. Writing one byte a million times is far slower than writing a megabyte once. FileStream, StreamReader and StreamWriter already buffer internally (4 KB to 16 KB by default). BufferedStream adds a buffer to a stream that has none:

C# BufferedStream

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

The little CountingStream is also a template for writing your own stream: derive from Stream, implement the abstract members, and throw NotSupportedException for what you cannot do. Stream.Null is a stream that throws everything away, useful for benchmarks and for measuring sizes.

Copying and chaining streams

CopyTo and CopyToAsync move all remaining bytes from one stream to another in chunks, without loading everything into memory:

C# CopyTo and chaining

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

Streams compose like pipes: a StreamReader wraps a GZipStream that wraps a FileStream. Each layer adds one behavior (decoding text, decompressing, reading from disk). That is the whole design idea, and it is why compression and encryption in .NET (CryptoStream) are just more streams.

Compression

System.IO.Compression provides GZipStream, DeflateStream, BrotliStream and ZLibStream. They wrap another stream; writing compresses and reading decompresses:

C# GZip and Brotli in memory

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

Crucial rule: dispose the compressor before you read the target. The final compressed block is only written on dispose or flush. If you call target.ToArray() inside the using, the result is truncated and decompressing it fails. Highly repetitive text shrinks dramatically, already-compressed data (JPEG, MP4, ZIP) does not shrink at all.

Choose by use case: GZip is universal (HTTP, .gz), Brotli compresses web content better, Deflate is the raw algorithm without GZip's header. For .zip archives with several entries use ZipArchive in the same namespace:

C# ZipArchive in memory

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

Async streams and cancellation

Use the Async versions on servers and UI threads. They take a CancellationToken so a request that the user abandoned stops reading:

C# Async reading with cancellation

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

Prefer the overloads taking Memory<byte>/ReadOnlyMemory<byte> over the older byte[], offset, count forms: they avoid allocations and are what the runtime optimizes. See async and await and Span, Memory and Slicing for more.

Pipelines: when streams are not enough

For servers that parse many network messages per second, streams make you manage buffers yourself: leftover bytes, partial messages, resizing. System.IO.Pipelines (used by Kestrel) separates a writer that fills buffers from a reader that consumes them, and it handles buffer pooling and back-pressure for you. The shared ASP.NET Core framework includes it, so we can run a small example:

C# A Pipe with a writer and a reader

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

The writer sent three pieces that split words and lines in the middle, yet the reader sees complete lines because unconsumed bytes stay in the pipe until more data arrives. That "keep what you could not parse yet" behavior is the main reason to use pipelines. You rarely need them in application code; knowing they exist helps when you read ASP.NET Core sources or write protocol parsers. PipeReader.Create(stream) and PipeWriter.Create(stream) connect the two worlds.

Good habits

  • Always dispose streams (using), and dispose compressors before reading their output.
  • Pass leaveOpen: true when a reader or writer should not close a stream you still need.
  • Rewind with Position = 0 before re-reading a MemoryStream.
  • Loop on Read or use ReadExactly; never assume one call fills the buffer.
  • Prefer Async methods with a CancellationToken in server code.
  • Do not use MemoryStream for huge data; it keeps everything in memory (and has a 2 GB limit).

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. You write 10 bytes to a MemoryStream and then call Read. It returns 0. What is the most likely cause?

    After writing, the position sits after the last byte. Reading from there reports the end of the stream.
  2. 2. What does Stream.Read return when the end of the stream has been reached?

    Read returns the number of bytes read, and 0 means the end. (ReadByte returns -1 instead.)
  3. 3. Why must a GZipStream used for compression be disposed before you use the output bytes?

    Until disposal or flush, part of the data is still inside the compressor, so the output would be incomplete.
  4. 4. What does new StreamWriter(stream, leaveOpen: true) change?

    By default a writer closes the stream it wraps. With leaveOpen: true the stream stays usable.
  5. 5. Which type is best for reading fixed-size primitives in the exact order they were written?

    BinaryReader has ReadInt32, ReadDouble, ReadString and so on; you must read in the same order the BinaryWriter wrote.

Summary

  • Stream is the common abstraction for bytes: files, memory, network, compression. Read returns 0 at the end and may return less than requested.
  • StreamReader/StreamWriter add text and encodings; BinaryReader/BinaryWriter add primitives; StringReader/StringWriter do the same over strings.
  • Dispose wrappers to flush them, and use leaveOpen to keep the inner stream alive.
  • BufferedStream and the built-in buffers reduce expensive system calls; CopyToAsync moves data in chunks.
  • GZipStream, BrotliStream and ZipArchive compress and extract data; dispose the compressor before reading its output.
  • Pipelines manage buffers and partial messages for high-performance parsers.

Next: JSON with System.Text.Json