Skip to content
Tutorials Center
My learning

Topics

Lesson 52 of 66 ยท I/O and Disk Scheduling

Buffering, Caching and Spooling

Learn the difference between buffering, caching and spooling, how single, double and circular buffers work, and how a print spooler lets slow devices share a fast CPU.

Devices and programs rarely run at the same speed. A program can produce a million lines per second, but a printer prints a few per second. A network delivers data in bursts while the application reads in steady sips. Three related techniques smooth out these mismatches: buffering, caching and spooling. They sound similar and are often confused, so this lesson pins down what each does, when each helps and how to build them.

A short way to remember them:

  • A buffer holds data in transit between two parties that work at different speeds or sizes.
  • A cache keeps a copy of data that already lives somewhere slower, so the next request is fast.
  • A spool holds whole jobs for a device that cannot be shared, until the device is ready.

Why buffer?

There are three classic reasons:

  1. Speed mismatch. A fast producer and a slow consumer (or the reverse) need a place to put data so neither waits on every byte.
  2. Size mismatch. A network delivers 1,500-byte packets, but the application reads lines; a disk reads 4 KB blocks, but a program reads 7 bytes. The buffer reassembles and slices.
  3. Copy semantics. When write(fd, buf, n) returns, the program may modify buf immediately. The kernel copies the data into its own buffer first, so what reaches the device is what the program had at the moment of the call.

Single, double and circular buffering

Single buffering. One buffer sits between the device and the process. The device fills it, then the process must empty it before the device can fill it again. The two work alternately, never at once.

Double buffering. Two buffers. While the device fills buffer A, the process consumes buffer B, then they swap. This overlaps producing and consuming. It is how graphics are drawn: the screen shows one frame while the next is built in the back buffer, which prevents flicker.

Circular (ring) buffering. For a steady stream with bursts, use several buffers in a ring with a head (where the producer writes) and a tail (where the consumer reads). If the ring is full the producer waits; if empty the consumer waits. It is the structure inside keyboard drivers, audio pipelines and network cards. This is the same producer and consumer problem from the synchronization chapter.

flowchart LR
    D["Device<br/>(fast bursts)"] --> B1["buffer 1"]
    B1 --> B2["buffer 2"]
    B2 --> B3["buffer 3"]
    B3 --> B4["buffer 4"]
    B4 --> B1
    B3 --> P["Process<br/>(steady reader)"]

A ring of buffers: the device writes at the head, the process reads at the tail.

Timing difference

Suppose the device needs 4 time units to fill a buffer and the process needs 3 to handle one.

Python Single vs double buffering, in time units

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

For long streams, double buffering costs max(fill, work) per item instead of fill + work, so the slower side sets the pace.

A ring buffer in Python

Python Circular buffer between a producer and a consumer

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The ring fills during the burst, then drains. It decouples the bursty producer from the steady consumer, as long as the average rates match.

Caching

A cache is a small, fast store holding copies of data that lives in a larger, slower store. The big idea is locality: data used recently (temporal locality) and data near it (spatial locality) is likely to be used again.

The OS keeps a page cache (also the buffer cache or file cache) in free RAM:

  • read: if the block is in the cache (a hit), return it with no disk access; otherwise (a miss) read from the disk, store a copy, and return it.
  • write: write into the cache and mark the page dirty; a background thread writes dirty pages out later (write-back). The alternative, write-through, writes to the disk immediately and is safer but slower.
  • read-ahead: when the pattern looks sequential, load the next blocks before they are asked for.
  • eviction: when the cache is full, choose a victim to discard, usually by an LRU-like policy (see page replacement).

A buffer is often the only copy of the data during a transfer; a cache is, by definition, a copy. On a real kernel the same memory serves both roles: the "buffer cache" holds file data that is both buffering writes and caching reads.

Python An LRU block cache with hit rate

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

A scan touches every block once, so a cache cannot help. Reuse is what it rewards.

Spooling

Spooling (Simultaneous Peripheral Operations On-Line) stores whole jobs in a queue, usually on disk, for a device that serves one user at a time. The classic example is a printer. If two programs wrote directly to the printer their pages would interleave. Instead, each program writes its output to a spool file, finishes immediately, and goes on. A spooler daemon takes jobs from the queue in order and feeds them to the printer.

Benefits:

  • Programs do not wait for the slow device.
  • The device is kept busy continuously.
  • Jobs are isolated, can be reordered, prioritized, paused or cancelled.
  • Several users can share a non-shareable device without conflict.

Spooling also covers input: card readers once spooled jobs onto disk before the CPU ran them, which is how batch systems kept the CPU busy.

Python A print spooler with a queue

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Spooling versus buffering

Buffering Spooling
Holds Pieces of one stream Whole jobs
Lives in Main memory Usually disk
Overlaps I/O of one job with its own CPU work I/O of many jobs with computation
Device Any Non-shareable (printer)
Typical size Bytes to megabytes Whole files

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which technique stores copies of frequently used data so repeat reads are fast?

    A cache keeps copies of data from a slower store. A buffer merely holds data in transit.
  2. 2. How does double buffering speed up processing?

    The two activities overlap, so each item costs about the longer of the two times rather than their sum.
  3. 3. Why does a print spooler let applications continue immediately?

    Writing to a queue on disk is quick. The spooler daemon handles the slow device separately.
  4. 4. What does write-back caching do?

    Dirty pages are flushed in the background. This is fast, but unflushed data can be lost in a crash.

Summary

  • Buffers hold data in transit to absorb differences in speed and size and to give copy semantics; caches hold copies of slower data to exploit locality; spools queue whole jobs for non-shareable devices.
  • Single buffering alternates device and process; double buffering overlaps them; circular buffering handles bursts with head and tail pointers.
  • The page cache uses write-back, read-ahead and LRU-style eviction.
  • Spooling lets many programs share a printer and keeps the device busy.

Next: Disk Structure and Access Time