Skip to content
Tutorials Center
My learning

Topics

Lesson 4 of 67 ยท Big Picture

Measuring Performance

Learn the CPU execution time equation, clock rate, CPI, MIPS, benchmarks, speedup and Amdahl's law, with runnable calculators and plots you can change.

"This computer is faster" sounds simple, but it hides a question: faster at what? A phone may beat a laptop at opening an app and lose badly at video editing. To compare designs honestly, computer architects use a handful of precise definitions and one central equation. Once you know them, claims like "twice the clock speed, twice the performance" can be checked in seconds.

This lesson defines performance, builds the CPU execution time equation from clock cycles, shows why MIPS can mislead, and ends with speedup and Amdahl's law, which explains why adding more processors often helps less than you hope.

Performance is the inverse of time

For a given program on a given machine:

performance = 1 / execution time

If machine A runs a program in 10 seconds and machine B in 15 seconds, then A is 15 / 10 = 1.5 times as fast as B. Always state the program: performance only has meaning for a particular workload.

Two kinds of time exist. Response time (latency) is how long one task takes. Throughput is how many tasks finish per unit time. A second CPU core may raise throughput without making any single task finish sooner.

The CPU execution time equation

A CPU is driven by a clock, so time can be counted in cycles:

CPU time = clock cycles x clock cycle time = clock cycles / clock rate

The clock cycle time is the period (the inverse of the rate: a 2 GHz clock has a 0.5 ns period). Clock cycles themselves depend on the program:

clock cycles = instruction count x CPI

CPI means cycles per instruction: the average number of clock cycles one instruction needs. Putting it all together gives the most important formula in this track:

CPU time = instruction count x CPI x clock cycle time

flowchart LR
  IC["Instruction count<br/>program, compiler, ISA"] --> T["CPU time"]
  CPI["CPI<br/>organization, instruction mix"] --> T
  CT["Clock cycle time<br/>technology, organization"] --> T

Three factors multiply to give execution time; each is influenced by a different layer.

Each factor has a different owner:

Factor Who influences it
Instruction count Algorithm, programming language, compiler, instruction set
CPI CPU organization (pipeline, caches) and the instruction mix
Clock cycle time Hardware technology and organization

Improving one factor while worsening another may gain nothing. A faster clock with a deeper pipeline can raise CPI. A more powerful instruction can cut the instruction count but need more cycles.

Python CPU time calculator

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The machine with the highest clock rate is the slowest, because its CPI is also the highest. Clock rate alone is a poor guide. (CPI below 1 is possible on processors that finish several instructions per cycle, see Superscalar, Out-of-Order and Speculative Execution.)

Different instructions, different cycles

Not every instruction costs the same. If a program mixes instruction classes, the overall CPI is a weighted average:

CPI = sum over classes of (CPI of the class x fraction of instructions in the class)

Python Average CPI from an instruction mix

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The table shows where cycles are spent. Loads are only 20 percent of the instructions but a third of the cycles, so speeding them up pays off. This habit of finding the dominant cost is the core of architectural thinking.

MIPS and why it misleads

MIPS (million instructions per second) is instruction count / (execution time x 10^6). It sounds like a speed rating, but it has three flaws:

  1. It ignores what an instruction does. A machine with simple instructions can show a high MIPS while needing more instructions for the same job.
  2. It varies between programs on the same machine.
  3. It can go up when performance goes down. Adding a quick but useless instruction raises the MIPS number without finishing the work sooner.

A related unit, FLOPS (floating-point operations per second), is useful for scientific computing, but it too measures only one kind of work. The reliable measure is always the execution time of real programs.

Benchmarks

Because "the program" matters so much, the industry uses benchmarks: standard sets of programs that represent typical work. Examples include suites of compilers, compression and simulation tasks for general computing, and separate suites for graphics and machine learning. Good benchmarking follows a few rules:

  • Use real applications, not tiny kernels that a clever compiler can optimise away.
  • Run each test several times and report the spread, not just the best result.
  • Summarise ratios with the geometric mean, which gives consistent answers regardless of which machine is used as the reference.
  • Beware of results tuned specifically to a benchmark.
Python Arithmetic vs geometric mean of speedups

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The arithmetic mean says B is 1.5 times slower than A, but dividing the other way it says the two machines are exactly equal (1.000), a contradiction. The geometric mean gives matching answers in both directions (1.260 and 0.794 are reciprocals), which is why benchmark suites use it.

Speedup and Amdahl's law

Speedup compares two times:

speedup = time before / time after

Suppose you improve only part of a program. Let f be the fraction of the original execution time that benefits, and s the speedup of that part. The overall speedup is given by Amdahl's law:

overall speedup = 1 / ((1 - f) + f / s)

The part you did not improve, (1 - f), stays as it was and limits the gain. Even with an infinitely fast improved part (s going to infinity), the speedup cannot exceed 1 / (1 - f). If 90 percent of the work can be sped up, the best possible result is 10 times, no matter how many processors you add.

Python Amdahl's law calculator and plot

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Look at the curves: each flattens at a ceiling set by its serial fraction. With f = 0.95 you get about 9 times from 16 processors, but 1,024 processors only reach about 19.6 times, with a hard limit of 20. This is why reducing the serial part often beats buying more hardware. The lesson Parallel Processing and Flynn's Taxonomy revisits the law together with Gustafson's more optimistic view.

A worked comparison

Putting the pieces together, suppose a program runs in 100 seconds, of which 40 seconds are spent waiting on memory accesses. Engineers offer you two upgrades: a CPU that is 2 times faster, or a memory system that makes memory waiting 4 times faster.

Python Which upgrade is better?

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The faster CPU saves 30 seconds and the faster memory saves 30 seconds, a tie in absolute terms (speedup 1.43x each), and combining them gives 2.5x. If the memory share were bigger, memory would win. The method, not the particular numbers, is what to remember: compute the share of time each part takes, then apply Amdahl's law.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. A program executes 4 billion instructions with an average CPI of 1.5 on a 3 GHz CPU. What is its CPU time?

    Cycles = 4e9 x 1.5 = 6e9. Time = 6e9 / 3e9 = 2 seconds.
  2. 2. Why is MIPS a poor way to compare computers with different instruction sets?

    A simple instruction does less work than a complex one, so a machine can have a high MIPS rating and still take longer to finish a program.
  3. 3. 40% of a program's run time is made 10 times faster. What is the overall speedup?

    1 / (0.6 + 0.4 / 10) = 1 / 0.64 = 1.5625.
  4. 4. What is the maximum overall speedup if 80% of a program can be parallelised, however many processors you use?

    The serial 20% remains, so the limit is 1 / 0.2 = 5.
  5. 5. Which summary statistic do benchmark suites use for ratios of execution times?

    The geometric mean gives consistent results regardless of which machine is the reference.

Summary

  • Performance is the inverse of execution time for a specific program; latency and throughput are different goals.
  • CPU time = instruction count x CPI x clock cycle time; each factor is influenced by a different layer.
  • Average CPI is a weighted average over the instruction mix; look for the instructions that dominate cycles.
  • MIPS and clock rate alone are poor guides; use the run time of real programs and benchmark suites, summarised with the geometric mean.
  • Amdahl's law: speedup = 1 / ((1 - f) + f / s). The unimproved part caps the gain at 1 / (1 - f).

Continue with Number Systems and Conversions.