Skip to content
Tutorials Center
My learning

Topics

Lesson 51 of 67 ยท Input/Output

Buses and Interconnects

Learn how buses carry address, data and control signals, how arbiters share them, and how to calculate bandwidth for DDR memory, PCIe, USB and SATA.

Every part of a computer needs to talk to every other part. The CPU reads memory, a graphics card receives textures, a disk delivers files, a keyboard reports keys. The wires and rules that make this possible are called buses and interconnects. Their speed often decides how fast the whole machine feels, because the fastest processor is useless if it waits for data to arrive.

In this lesson you will learn what a bus is made of, how several devices share one without crashing into each other (arbitration), why modern systems replaced wide shared buses with fast point-to-point links, and how to calculate the bandwidth of memory, PCIe, USB and SATA with a few lines of Python. It continues from I/O organization.

What a bus is

A bus is a set of wires shared by several components, together with a protocol that says who may use them and when. Classic buses have three groups of lines:

  • The address bus carries the location being accessed. With n address lines you can name 2n locations.
  • The data bus carries the value. Its width (8, 16, 32, 64 bits) is how much moves per transfer.
  • The control bus carries signals such as read/write, clock, "ready", and interrupt or arbitration requests.

A bus transfer is a short conversation: a master (the CPU or a DMA controller) starts it, and a slave (memory or a device controller) answers. The slave may need extra time, so many protocols include a wait or ready signal.

Buses come in two timing styles:

  • Synchronous buses use a shared clock. Everything happens on clock edges, which is simple and fast, but every device must keep up with the clock.
  • Asynchronous buses use handshake signals (request, acknowledge) instead of a clock, so slow and fast devices can coexist, at the cost of handshake delays.

Another trick is multiplexing: sending addresses and data over the same wires at different moments. It saves pins and wires but needs extra cycles.

flowchart LR
  CPU["CPU<br/>(master)"] --- AB["Address bus"]
  CPU --- DB["Data bus"]
  CPU --- CB["Control bus"]
  AB --- MEM["Memory<br/>(slave)"]
  DB --- MEM
  CB --- MEM
  AB --- DEV["Device<br/>(slave)"]
  DB --- DEV
  CB --- DEV

One shared set of address, data and control lines connects a master to many slaves.

Sharing a bus: arbitration

If two masters (say the CPU and a DMA controller) both want the bus in the same cycle, something must choose. That job belongs to the bus arbiter. Common schemes:

  • Daisy chain: the grant signal passes from device to device along a chain. The device closest to the arbiter has the highest priority. Cheap, but far devices may starve.
  • Centralized parallel: each master has its own request and grant lines, and the arbiter picks one, by fixed priority or by a fairer rule.
  • Distributed: no central arbiter; devices compare priorities themselves (the old CAN bus in cars works like this).

Priorities raise the question of fairness. If the highest priority master requests constantly, the others may never get a turn. A round-robin arbiter fixes this by rotating who goes next. The simulation below gives three always-busy masters, A, B and C, twelve bus cycles and two different arbiters.

Python Fixed priority versus round robin

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Fixed priority gives A all twelve cycles, so B and C starve. Round robin hands each master four. Real arbiters often combine both ideas: high priority for time-critical devices (such as a display refresh) and round robin among the rest.

A wide shared bus has physical limits. Every device on the wires adds electrical load, signals from parallel wires arrive at slightly different times (skew), and as the clock rises the wires act like antennas. For that reason, modern systems moved to serial, point-to-point links: a few wires carrying very fast signals between exactly two chips, with a switch or root complex to connect more than two.

The result is a hierarchy of interconnects, each matched to what it connects:

flowchart TB
  CPU["CPU cores + caches"] ---|"memory channels"| RAM["DRAM (DDR)"]
  CPU ---|"PCIe lanes"| GPU["Graphics card"]
  CPU ---|"PCIe lanes"| NVME["NVMe SSD"]
  CPU ---|"chip-to-chip link"| PCH["Chipset / hub"]
  PCH --- USB["USB ports"]
  PCH --- SATA["SATA drives"]
  PCH --- LAN["Ethernet, audio, ..."]

A typical desktop: fast memory and graphics hang directly off the CPU; slower devices share the chipset.

Memory buses

The memory bus connects the CPU's memory controller to DRAM. Today it carries DDR (double data rate) traffic, where data moves on both the rising and falling clock edges. A DDR channel is 64 bits (8 bytes) wide (a DDR5 DIMM is split into two 32-bit subchannels, but the channel total is the same 64 bits of data). Memory is rated in MT/s (mega-transfers per second): DDR4-3200 does 3,200 million transfers per second, DDR5-6400 does 6,400 million. Peak bandwidth is simply transfers per second times bytes per transfer, times the number of channels.

PCI Express

PCIe is the standard high-speed link for graphics cards, SSDs and network cards. It is built from lanes, and each lane is a pair of one-way serial connections (one pair per direction). Devices use x1, x4, x8 or x16 lanes. Each generation roughly doubles the per-lane speed: 8 GT/s for PCIe 3.0, 16 for 4.0, 32 for 5.0 and 64 for 6.0. Generations 3.0 to 5.0 use 128b/130b encoding, so a little under 1.6 percent of the bits are overhead. Generation 6.0 changes the signaling and uses a different packet structure, so its overhead is estimated here with an approximate figure.

USB and SATA

USB is a host-controlled serial bus designed for easy, hot-pluggable peripherals. The host schedules every transfer, so devices cannot talk on their own. Speeds run from 480 Mb/s (USB 2.0 High-Speed) through 5 and 10 Gb/s (USB 3.2 Gen 1 and Gen 2) to 40 Gb/s (USB4). SATA III connects storage at 6 Gb/s with 8b/10b encoding, which sends 10 bits on the wire for every 8 bits of data, leaving 80 percent for payload.

Calculating bandwidth

Bandwidth calculations all follow the same pattern: raw rate, multiplied by the encoding efficiency, multiplied by the number of lanes or channels, then divided by 8 to convert bits to bytes. The calculator below produces the theoretical peak of several real interfaces. Remember that these are best-case numbers: protocol headers, arbitration and device speed lower what you actually see.

Python Peak bandwidth calculator

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Look at the PCIe lines: a PCIe 4.0 x4 link, which is what a typical NVMe SSD uses, tops out at about 7.9 GB/s per direction, far above SATA's 600 MB/s. That gap is why NVMe drives feel so much faster for large transfers. And a dual-channel DDR5-6400 memory system peaks at 102.4 GB/s, which is only about 1.6 times the 63 GB/s of one PCIe 5.0 x16 slot.

Latency matters too

Bandwidth says how much data flows per second once a transfer is going. Latency is the delay before the first byte arrives. A transfer of s bytes takes roughly:

TEXT
time = latency + size / bandwidth

For large transfers the second term dominates; for tiny ones, the latency does. That is why a thousand small requests are much slower than one large request, and why designs batch work and use bigger bursts. The next example moves different amounts of data over a SATA SSD and an NVMe SSD, with typical (round, illustrative) numbers for latency and bandwidth.

Python Latency or bandwidth: which one dominates?

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

For a single 4 KiB request almost all of the time is latency, so the NVMe drive is only four times faster. For a gigabyte the latency vanishes in the noise and bandwidth decides, giving a gap of about six times. Which number matters depends on your workload: databases doing many small random reads care about latency, video editors copying big files care about bandwidth.

A few more ideas

  • Bus width versus clock: you can raise throughput by widening the bus or by running it faster. Parallel buses hit skew limits at high clocks, which is why the industry switched to fast serial lanes.
  • Burst transfers send many consecutive words after a single address, amortizing the address phase. DRAM is built around bursts.
  • Split transactions let a master issue a request, release the bus, and receive the answer later, so a slow slave does not block the bus.
  • Cache-coherent interconnects (such as the links between cores, or between CPU sockets) also carry coherence traffic. See Cache Coherence and Memory Consistency.
  • Topology: a shared bus has one path; a crossbar connects every input to every output at once; a ring or mesh scales to many cores. Larger chips use these on-chip networks.

Centralized arbitration schemes and I/O processors

When several devices want the bus at the same time, something has to pick one. With centralized arbitration a single arbiter (often part of the CPU or the chipset) makes the decision. There are three classic wiring schemes; they differ in how many wires they need, how fast they decide and how fair they are.

Scheme How the winner is found Wires Speed Fairness
Daisy chain One grant line passes through the devices in order; the first device that wants the bus keeps the grant Few (request, grant, busy) Slow, the grant ripples through every device Fixed priority by position; far devices can starve
Polling The arbiter sends a device number on a few poll lines, counting up until a requesting device answers log2(N) poll lines plus request Medium Priorities are programmable (change the polling order)
Independent requests Each device has its own request and grant line pair 2N Fast, the arbiter sees all requests at once Any policy (fixed, rotating, fair)
flowchart LR
  subgraph Daisy["Daisy chain"]
    A1["Arbiter"] -->|"grant"| D1["Dev 1"] -->|"grant"| D2["Dev 2"] -->|"grant"| D3["Dev 3"]
  end
  subgraph Indep["Independent requests"]
    A2["Arbiter"] <-->|"REQ1/GNT1"| E1["Dev 1"]
    A2 <-->|"REQ2/GNT2"| E2["Dev 2"]
    A2 <-->|"REQ3/GNT3"| E3["Dev 3"]
  end

Daisy chain saves wires and loses speed and fairness; independent request lines are the opposite.

Python Three bus arbitration schemes compared

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Decentralized schemes also exist (devices compare priority codes on shared lines, as in some backplane buses), but the three above are what exams and many embedded designs use. See also the interrupt version of the daisy chain in I/O Interfaces and Handshaking.

The I/O processor (IOP)

A DMA controller moves one block at a time and then needs the CPU again. An I/O processor (also called an I/O channel on mainframes) goes further: it is a small processor of its own with its own instruction set, whose programs, called channel programs, are stored in main memory. The CPU only says "run the I/O program at this address". The IOP then fetches those instructions itself, performs a whole sequence of operations (seek the disk, read a block, check the status, repeat for the next block, handle errors) and interrupts the CPU once, at the end.

flowchart LR
  CPU["CPU"] -->|"start I/O program"| IOP["I/O processor"]
  IOP -->|"fetch channel program"| MEM["Main memory"]
  IOP <-->|"data"| MEM
  IOP --- D1["Disk"]
  IOP --- D2["Printer"]
  IOP --- D3["Terminal"]
  IOP -->|"interrupt when done"| CPU

The CPU starts the IOP and carries on; the IOP runs the I/O program on its own.

Programmed I/O Interrupt I/O DMA I/O processor
CPU involvement Every word Every word or byte Setup and completion Start and completion only
Intelligence in the device side None None Address and count registers Its own program, error handling
Typical use Tiny devices Keyboards, mice Disks, network cards Mainframes, today's smart disk and network controllers

Today the idea survives in intelligent controllers: an NVMe SSD, a network card with offload engines or a GPU all contain processors that run firmware and move data without the CPU. The operating system hands them command queues in memory, which is a channel program in modern clothing.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What does the address bus carry?

    The address bus names the memory cell or device register. The data bus carries the value, and the control bus carries signals like read, write and ready.
  2. 2. Three masters always request the bus and the arbiter uses fixed priority. What happens?

    With fixed priority the top requester always wins. Round robin rotates the grant to be fair.
  3. 3. Why did computers move from wide parallel buses to serial point-to-point links such as PCIe?

    At high clock rates, parallel wires arrive at slightly different times and shared wires add load. A few fast serial lanes between two chips scale better.
  4. 4. A DDR5-6400 channel is 8 bytes wide. What is its peak bandwidth?

    6,400 million transfers per second times 8 bytes per transfer is 51,200 MB/s, or 51.2 GB/s.
  5. 5. For a 4 KiB read from an SSD, which term of time = latency + size / bandwidth usually dominates?

    A small transfer finishes the data phase almost instantly, so the fixed delay before it starts is what you wait for.

Summary

  • A bus is shared wires plus a protocol: address, data and control lines, a master and slaves, synchronous or asynchronous timing.
  • Arbiters decide who gets the bus. Fixed priority can starve low-priority masters; round robin is fair.
  • Modern systems use fast serial point-to-point links (PCIe, USB, SATA, memory channels) joined by switches and chipsets.
  • Peak bandwidth is rate times encoding efficiency times lanes or channels, divided by 8 for bytes. Real throughput is lower.
  • Transfer time is latency plus size over bandwidth: latency dominates small transfers, bandwidth dominates large ones.

Continue with Parallel Processing and Flynn's Taxonomy.