Skip to content
Tutorials Center
My learning

Topics

Lesson 36 of 67 ยท CPU Design

VLIW, EPIC and Classes of ILP Architectures

Compare superscalar, superpipelined and VLIW processors, see how a compiler packs instructions into bundles, and try a Python bundle scheduler with latencies.

A program contains instruction-level parallelism (ILP): many instructions that do not depend on each other and could run at the same time. Getting that parallelism out of the code is the job of the processor, the compiler, or both. Superscalar, Out-of-Order and Speculative Execution showed the hardware answer: the CPU examines a window of instructions at run time and issues independent ones together. This lesson looks at the other end of the design space: let the compiler find the parallelism, and keep the hardware simple.

That is the idea behind VLIW (very long instruction word) and its descendant EPIC. You will see what a bundle is, how a compiler fills one, and why this approach is rare in desktop CPUs but common in DSPs and GPUs.

Four ways to get more instructions per cycle

Approach Who finds the parallelism How throughput rises Examples
Pipelining Nobody, it is overlap of stages Up to 1 instruction per cycle Classic five-stage RISC
Superpipelining Nobody, deeper pipeline Shorter cycles, more stages Pentium 4 style designs
Superscalar The CPU, at run time Several instructions issued per cycle Modern x86, ARM, RISC-V cores
VLIW / EPIC The compiler, at build time One wide instruction holds several operations TI C6000 DSPs, Itanium

A superscalar CPU needs a lot of logic: decode several instructions, check dependences, rename registers and reorder results. That logic costs area and power. A VLIW design moves the work to the compiler, which has the whole program and unlimited time.

What is a VLIW instruction?

A VLIW instruction (often called a bundle) packs several independent operations into one long word, one per functional unit. Suppose a machine has two integer ALUs, one load/store unit and one floating-point unit. Each bundle then has four slots:

Text
| ALU 1      | ALU 2      | MEM          | FPU        |
| add r1,r2  | sub r3,r4  | load r5,[r6] | fmul f1,f2 |

The hardware issues the whole bundle in one cycle. It does no dependence checking among the slots: the compiler guarantees that the operations are independent. If it cannot find four useful operations, it fills the empty slots with no-ops, which waste code space.

The consequences are easy to state:

  • Simple, fast hardware. No issue logic, no reorder buffer, no register renaming.
  • Complex compiler. It must know the exact latencies and units of the machine and schedule every instruction.
  • Code is tied to the hardware. A binary built for a 4-slot machine does not run on a 6-slot one without recompiling. Superscalar CPUs keep binary compatibility because the hardware decides the schedule.
  • Cache misses and other unpredictable delays stall the whole bundle, because the compiler cannot know them in advance.

How the compiler fills bundles

The compiler builds a dependence graph of the instructions in a block and then places them into bundles, respecting two rules: an operation may only start after the results it needs are ready (taking each unit's latency into account), and a bundle holds at most one operation per slot type.

This is list scheduling, a greedy algorithm: at each cycle, take the ready operations whose slot is free.

Python A small VLIW bundle scheduler

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice the empty cycles. After the loads are issued, the adds must wait three cycles for the load results. A real compiler fills those cycles with work from other loop iterations; this is software pipelining and loop unrolling. Unrolling the loop copies its body several times, giving the scheduler more independent operations to place in each bundle.

Python How unrolling fills the bundles

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

You should see the cycles per iteration fall as the unroll factor grows. That is exactly the effect compilers for VLIW machines chase.

EPIC and the Itanium

EPIC (explicitly parallel instruction computing) is Intel and HP's refinement of VLIW, used in the Itanium processors. It kept the compiler-does-the-scheduling idea but fixed some of VLIW's weaknesses:

  • Instructions are grouped in 128-bit bundles of three operations plus a template that tells the hardware which units they use and where the stop bits (end of an independent group) fall. A later, wider implementation can issue several bundles at once, so the binary survives a change of width.
  • Predication: most instructions carry a predicate register. A short if is turned into predicated instructions that all issue, and those whose predicate is false have no effect. This removes many branches, and so many mispredictions.
  • Speculative loads and advanced loads let the compiler hoist a load above a branch or a store, with a check instruction to recover if the guess was wrong.
  • A large register file (128 general and 128 floating-point registers) and register rotation support software-pipelined loops.

Itanium never won the server market. The compilers could not find enough static parallelism in typical branchy integer code, and out-of-order x86 chips improved faster than expected. But the ideas live on: predication in GPUs and ARM, speculation hints, and VLIW itself in DSPs (such as Texas Instruments C6000 and Qualcomm Hexagon), where loops are regular and the compiler can schedule them well.

Choosing between them

Superscalar (out-of-order) VLIW / EPIC
Hardware complexity High Low
Compiler complexity Moderate Very high
Code size Compact Larger (no-ops fill slots)
Binary compatibility Good across generations Poor unless the ISA hides the width
Handles unpredictable latency (cache misses) Well Poorly
Best workloads General-purpose, branchy code Regular loops: signal processing, media, GPUs

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. In a VLIW processor, who guarantees that the operations in one bundle are independent?

    VLIW hardware does no dependence checking within a bundle. The compiler schedules the operations and fills unused slots with no-ops.
  2. 2. What is a main drawback of VLIW compared with superscalar designs?

    The compiler bakes the machine's unit counts and latencies into the schedule, so code generally needs recompiling for a different implementation.
  3. 3. What does predication do in an EPIC-style machine?

    Predicated instructions issue but only take effect when their predicate is true, turning control dependence into data dependence.
  4. 4. Why does loop unrolling help a VLIW compiler?

    Copies of the loop body from different iterations are independent, so empty slots and latency gaps can be filled with their operations.
  5. 5. Which workloads suit VLIW best?

    Regular, predictable loops let the compiler find static parallelism and schedule around fixed latencies.

Summary

  • ILP can be found by hardware at run time (superscalar, out-of-order) or by the compiler at build time (VLIW, EPIC).
  • A VLIW bundle holds one operation per functional unit; the hardware trusts the compiler and fills unused slots with no-ops.
  • Compilers fill bundles with list scheduling, loop unrolling and software pipelining, accounting for each unit's latency.
  • EPIC adds bundle templates and stop bits, predication, speculative loads and rotating registers.
  • VLIW trades simple hardware for a hard compiler problem and poor binary compatibility, so it thrives in DSPs and less in general-purpose CPUs.

Related: Branch Prediction and Dynamic Scheduling. Next: The Memory Hierarchy and Locality.