Skip to content
Tutorials Center
My learning

Topics

Lesson 55 of 66 ยท I/O and Disk Scheduling

Disk Management, Swap Space and RAID

Learn how an OS prepares disks (formatting, partitions, boot blocks, bad blocks), manages swap space, and uses RAID 0, 1, 5, 6 and 10 for speed and fault tolerance, with XOR parity in Python.

A brand-new disk is just a pile of magnetic domains or flash cells. Before it can hold files, the OS must prepare it, divide it, start from it, work around its flaws and, if the data matters, spread it across several drives so that one failure does not destroy everything. This lesson covers that last stage of the storage story: disk management, swap space, and RAID.

Preparing a disk

Low-level and logical formatting

Low-level (physical) formatting divides a disk into sectors the controller can read and write, adding headers and error-correcting codes (ECC). Manufacturers do this at the factory. Logical formatting is what the OS does when you "format" a volume: it writes the initial data structures of a file system (superblock or boot record, empty directory, free-space map; see the Unix file system). Some programs, such as databases, skip the file system and use a disk as a raw array of blocks.

Partitions

A disk is divided into one or more partitions, each acting like a separate device and holding its own file system, swap area or raw data. The partition table lives in the first sectors: the old MBR scheme (4 primary partitions, 2 TB limit) or the modern GPT (128 partitions by default, enormous sizes, a backup copy at the end of the disk). Volume managers (LVM, Windows Storage Spaces) add another layer so several disks can be combined and resized.

The boot sequence

When the machine powers on, firmware (UEFI or the older BIOS) runs, finds a bootable disk and loads a small program:

  1. Firmware loads the boot loader from the EFI System Partition (or from the MBR boot sector).
  2. The boot loader (GRUB, Windows Boot Manager) finds the OS kernel in a partition, loads it into memory and passes control.
  3. The kernel initializes devices, mounts the root file system and starts the first process (init or systemd).

The partition holding the OS is the boot (system) partition and the disk is the boot disk. Secure boot (see cryptography in the OS) verifies signatures along this chain.

Bad blocks

Sectors can fail. Drives hold spare sectors and a list of defects; when the controller finds a bad sector it can remap (sector sparing) a logical block to a spare one. Older systems marked bad blocks in the file system's own table so they were never allocated. With sector slipping, all sectors after the bad one shift over by one to keep spares physically close. SSDs do similar remapping at the flash block level.

Swap space management

Swap space is disk (or SSD) area that extends virtual memory. When RAM is short the OS moves less-used pages out to swap and brings them back when needed.

  • Where: a dedicated swap partition (fast, no file system overhead) or a swap file (flexible, can be resized). Linux can use several, with priorities.
  • How much: it depends on the workload and on whether the machine hibernates (hibernation writes all of RAM to swap). A few gigabytes suffice for many desktops; servers tune it by measuring.
  • Allocation: swap uses simple, fast allocation, often contiguous or in large chunks, because speed matters more than flexibility; it is cleared at boot, so it needs no crash recovery.
  • Danger: if the working set exceeds RAM, the system spends its time moving pages back and forth, which is thrashing. Swapping on an SSD wears the drive, so many systems use compressed RAM (zram, macOS compression) first.

RAID: Redundant Array of Independent Disks

One disk is slow and will fail someday. RAID combines several disks into one logical device using three tools:

  • Striping: split data across disks in chunks (the stripe unit), so reads and writes proceed in parallel: more speed and capacity.
  • Mirroring: keep identical copies on two disks: more reliability and read speed.
  • Parity: store a checksum, computed by XOR, from which any single lost block can be rebuilt: reliability with less overhead than mirroring.

RAID can be done in hardware (a controller card) or software (Linux mdadm, ZFS, Windows).

Level Idea Min. disks Usable space (n disks) Survives
RAID 0 Striping, no redundancy 2 n Nothing (any failure loses all)
RAID 1 Mirroring 2 n / 2 All but one disk per mirror
RAID 5 Striping + distributed parity 3 n - 1 One disk
RAID 6 Striping + two parities 4 n - 2 Two disks
RAID 10 Striped mirrors 4 n / 2 One per mirror pair
flowchart LR
    subgraph R0["RAID 0: stripes"]
      A1["A1"] --- A2["A2"]
      A3["A3"] --- A4["A4"]
    end
    subgraph R1["RAID 1: mirror"]
      B1["B1"] --- B1c["B1 copy"]
      B2["B2"] --- B2c["B2 copy"]
    end
    subgraph R5["RAID 5: parity rotates"]
      C1["C1"] --- C2["C2"] --- P1["P(C1,C2)"]
    end

Striping spreads data; mirroring copies it; parity lets one lost block be recomputed.

Why XOR parity works

For data blocks D1 and D2, the parity is P = D1 xor D2. XOR is its own inverse, so if D1 is lost then D1 = D2 xor P. With more disks, P is the XOR of all data blocks in the stripe. RAID 5 rotates the parity block among the disks so no single disk becomes a bottleneck for parity writes.

Python RAID 5: lose a disk, rebuild it

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The rebuilt bytes match exactly. In a real array the controller does this for every stripe, which is why rebuilding takes hours on large disks, and a second failure during that window is fatal for RAID 5 (this is why RAID 6 exists).

The small-write problem

Updating one block in RAID 5 means reading the old data and old parity, computing the new parity (P_new = P_old xor D_old xor D_new) and writing both. That is 2 reads and 2 writes for one logical write.

Python Capacity, fault tolerance and write cost by RAID level

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Choosing a level

  • RAID 0: scratch space and video editing where speed matters and the data is replaceable. Never for data you care about.
  • RAID 1: boot drives and small, critical volumes. Simple and fast to recover.
  • RAID 5: read-heavy storage with good capacity efficiency, on small disks.
  • RAID 6: large arrays of big disks, where a second failure during a long rebuild is a real risk.
  • RAID 10: databases and write-heavy loads needing both speed and safety, at the price of 50% capacity.

RAID is not a backup. It protects against disk failure, not against deleting a file, ransomware, fire, or a controller bug that corrupts every copy at once. Keep separate, versioned, off-site backups.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What does logical formatting do?

    Physical (low-level) formatting creates sectors. Logical formatting creates the file system on top.
  2. 2. What is the usual order of the boot process?

    Firmware finds the boot loader, the boot loader loads the kernel, and the kernel takes over.
  3. 3. With RAID 5 on 5 disks of 2 TB each, how much space is usable?

    One disk's worth of capacity holds parity, so usable space is (5 - 1) x 2 = 8 TB.
  4. 4. Why can a lost RAID 5 block be rebuilt?

    XOR is its own inverse. Combining the surviving data blocks and the parity yields the lost block.
  5. 5. Why is RAID not a substitute for backups?

    RAID protects against disk failure only. Deletions and corruption are faithfully replicated to every disk.

Summary

  • Disks need low-level formatting (sectors), partitioning (MBR or GPT) and logical formatting (a file system); boot starts in firmware, then the boot loader, then the kernel.
  • Bad blocks are handled by sector sparing or slipping; SSDs remap flash blocks.
  • Swap space extends memory with a partition or file; too much swapping leads to thrashing.
  • RAID uses striping, mirroring and XOR parity. RAID 0 is fast but fragile, 1 mirrors, 5 survives one failure, 6 survives two, 10 stripes mirrors.
  • RAID improves availability but is never a backup.

Next: Protection Domains and Access Control