Skip to content
Tutorials Center
My learning

Topics

Lesson 32 of 37 ยท Design Patterns

Pipelining for Speed

Split long logic paths with pipeline registers in VHDL to raise the clock frequency, understand latency versus throughput and build a pipelined multiply-add.

When the critical path limits your clock (timing lesson), the most powerful tool is pipelining: cut the long path into stages by inserting registers between them. Each stage does a fraction of the work in one clock cycle, and several data items are in flight at once, like cars on an assembly line. You pay with extra flip-flops and extra latency, and in return the clock can run much faster.

VHDL is shown read-only on this site; the C programs simulate pipelines clock edge by clock edge so you can see the data move.

The idea

Suppose y = (a * b) + (c * d) + e is computed combinationally. The path goes through two multipliers and two adders. Suppose it takes 12 ns, so the clock cannot run faster than about 83 MHz. Now insert registers:

flowchart LR
  I["a b c d e"] --> M["multiply"]
  M --> R1["registers"]
  R1 --> A1["add"]
  A1 --> R2["registers"]
  R2 --> A2["add e"]
  A2 --> R3["y register"]

Three stages, each short enough for a fast clock. The result appears three clocks after the inputs.

If each stage takes about 4 ns, the clock can run at 250 MHz. One new result is produced every clock cycle after the pipeline has filled.

Latency versus throughput

  • Latency: time from an input entering to its result appearing (here 3 cycles).
  • Throughput: results per second (here one per cycle).

Pipelining increases the clock frequency, and therefore throughput, but never reduces latency; it usually increases the latency in cycles. It is the right choice for streaming data (video, filters, packets) and the wrong one for a single result needed right now.

The VHDL

VHDL
library ieee;
use ieee.std_logic_1164.all;
use ieee.numeric_std.all;

entity mac3 is
  port (clk : in std_logic;
        a, b, c, d, e : in  signed(7 downto 0);
        y : out signed(17 downto 0));
end entity;

architecture rtl of mac3 is
  signal p1, p2 : signed(15 downto 0);   -- stage 1 results
  signal e1     : signed(7 downto 0);    -- e delayed to stay aligned
  signal s      : signed(16 downto 0);   -- stage 2 result
  signal e2     : signed(7 downto 0);
begin
  process (clk)
  begin
    if rising_edge(clk) then
      -- stage 1: multiply
      p1 <= a * b;
      p2 <= c * d;
      e1 <= e;
      -- stage 2: add the products
      s  <= resize(p1, 17) + resize(p2, 17);
      e2 <= e1;
      -- stage 3: add e
      y  <= resize(s, 18) + resize(e2, 18);
    end if;
  end process;
end architecture;

Two points matter. First, every signal that is needed later must be delayed through the same number of stages: e is registered twice (e1, e2) so it meets the sum at the right time. Forgetting to delay a side signal is the most common pipeline bug: the result mixes data from different items. Second, nothing in the pipeline is cleared by reset because data registers need none; only control signals (such as a "valid" bit) need to be reset.

Simulating the pipeline in C

The program keeps three stages as variables updated "at the edge" in reverse order so each stage reads its predecessor's old value (the equivalent of VHDL signal semantics). It feeds six inputs and prints which result appears when.

C C stand-in: a 3-stage multiply-add pipeline

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with scanf or fgets), it gets the next line from this box.

The y column shows the first result (19) on the third edge, which is the edge that has captured all three register stages, then one result per edge. At the start the pipeline is still filling (zeros), at the end it must be flushed with empty cycles.

Valid bits

A real pipeline needs to know which outputs are meaningful. Carry a valid flag alongside the data through every stage:

VHDL
signal v1, v2, v3 : std_logic;
...
v1 <= valid_in;  v2 <= v1;  v3 <= v2;
valid_out <= v3;

Reset the valid flags (so garbage in the pipeline is never reported as a result) and leave the data registers without reset.

Stalls and enables

If the consumer cannot always take a result, the pipeline must stall: every stage holds its value when the downstream is not ready. The simple way is a global enable en that gates all pipeline registers (if en = '1' then ...). More flexible designs pass ready/valid handshakes between stages (see FIFOs and Handshakes).

How many stages?

More stages mean a faster clock only until you reach the limits of the fabric: clock-to-Q plus setup plus routing is never zero. A rule of thumb for FPGAs: aim for about 3-4 LUT levels per stage at several hundred MHz. Stages cost flip-flops (plentiful in FPGAs), latency and complexity, and they make verification harder because the expected value appears several cycles after the stimulus.

C C stand-in: clock speed versus number of stages

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with scanf or fgets), it gets the next line from this box.

Notice the diminishing returns: going from 1 to 2 stages nearly doubles the speed, going from 5 to 6 barely helps, and the latency in nanoseconds grows with every stage.

Retiming

Synthesis tools can move registers across logic automatically (retiming) to balance stages. Writing a design with a generic number of register stages after the logic and enabling retiming in the tool lets it spread them. It is convenient, but know the manual technique first.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does pipelining mainly improve?

    Shorter stages allow a faster clock, so more results per second, at the price of extra latency.
  2. 2. A 4-stage pipeline accepts one input per clock. After the pipeline has filled, how often does a result appear?

    Throughput is one result per clock once the stages are full.
  3. 3. Why must a side input such as e be delayed through the pipeline?

    Each stage works on a different item; side signals must travel with their data.
  4. 4. Which signals usually need a reset in a pipeline?

    Valid flags tell the user which outputs are real; stale data in data registers is harmless.

Summary

  • Pipelining inserts registers into a long path to raise the clock frequency.
  • Latency in cycles goes up; throughput goes up with the clock.
  • Delay every side signal through the same number of stages and carry a valid bit.
  • Reset the control flags, not the data registers.
  • Returns diminish quickly; stop when other limits (routing, memory) dominate.

Next: Clock Enables and Clock-Domain Crossing