Skip to content
Tutorials Center
My learning

Topics

Lesson 6 of 37 ยท Caching Concepts

Hit Ratio and Average Access Time

Learn how to compute cache hit ratio, miss ratio and average access time, read the stats counters get_hits and get_misses, and set a realistic hit ratio target.

The single most useful number for any cache is its hit ratio: the fraction of lookups that found what they were looking for. It tells you whether the cache is earning its keep, and it is the first thing to check when a site gets slow.

This lesson defines the ratios precisely, shows where Memcached reports the raw counters, and explains why a "good" ratio depends on the cost of a miss.

Definitions

For some period of time, count the lookups:

  • hits: lookups that returned a value
  • misses: lookups that found nothing (never stored, expired, or evicted)

Then:

TEXT
hit ratio  = hits / (hits + misses)
miss ratio = misses / (hits + misses) = 1 - hit ratio

Memcached keeps these counters since the server started, in the stats command output as get_hits and get_misses (also cmd_get, the total number of get commands, which counts every key in a multi-get). Because they are cumulative, you compute a ratio over a window by taking two snapshots and subtracting.

TEXT
STAT cmd_get 1048210
STAT get_hits 986120
STAT get_misses 62090

Here the ratio is 986,120 divided by 1,048,210, about 94.1%.

Three kinds of miss

Not all misses are equal, and the fix differs:

Miss type Cause Fix
Cold (compulsory) First request ever for the key Warm the cache ahead of time
Expired TTL ran out Longer TTL, or refresh before expiry
Capacity (evicted) Memory was full, the item was pushed out Add memory, store less, shrink values

If you see many capacity misses together with a rising evictions counter, the cache is too small for the working set. If you see mostly cold misses, you may have just restarted.

Computing ratios from counters

Python Hit ratio from two stats snapshots

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The lifetime number moves slowly because millions of old lookups dominate it. Always graph the window ratio (per minute, say): it reacts within a minute when a deploy breaks the cache keys.

Average access time

The speed a user feels is the average of hits and misses weighted by how often each happens. This is the same idea computer architects call average memory access time (AMAT):

TEXT
average time = hit_time + miss_ratio * miss_penalty

Here hit_time is the cost of looking in the cache (paid on every request, hit or miss), and miss_penalty is the extra time to get the data from the source when the cache has nothing. The formula is equivalent to the weighted average from the previous lesson, just arranged so you can see that the lookup is paid every time.

Python Average access time for different ratios

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice how an expensive miss changes the target. If a miss costs 60 ms you need about 96% to average 3 ms. If a miss costs only 5 ms, even 50% hits give a decent average. The right target is not a universal number; it follows from the miss penalty and from how much load your backend can take.

Hit ratio is not the whole story

A high ratio can hide problems:

  • Hot keys. A 99% ratio made of one key read a million times says little about the other 100,000 keys.
  • Tiny, cheap items. If you cache lots of cheap lookups and miss the expensive ones, the ratio looks great while the slow queries still hit the database. Weight by cost where you can.
  • Misses that cost a lot. A single miss that takes 30 seconds because of a slow report can matter more than 1,000 cheap hits.

Track database load and p95/p99 latency next to the hit ratio. If latency is bad while the ratio is fine, look at what the misses cost.

Setting a target

Common starting points:

  • Page and fragment caches for read-heavy sites: 90% to 99%.
  • Per-user data: often 60% to 85%.
  • Rarely repeated query results: below 50% may still pay off when misses are very costly.

To decide, work backwards from the database: if it can handle C queries per second and you receive R requests per second, you need a miss ratio at most C / R. With 20,000 requests per second and a database limit of 1,000, you need at least a 95% hit ratio, with some headroom.

Python Minimum hit ratio the database can survive

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. A server reports get_hits 900 and get_misses 100. What is the hit ratio?

    900 divided by (900 + 100) is 0.9, so 90%.
  2. 2. Why graph the ratio for a recent window instead of since server start?

    get_hits and get_misses are cumulative. Subtracting two snapshots gives a ratio that reacts quickly.
  3. 3. Many misses together with a rising evictions counter most likely means:

    Items are being pushed out to make room, so they are missing when asked for again: capacity misses.
  4. 4. With a 60 ms miss penalty and 0.4 ms hit time, which hit ratio is roughly needed for a 3 ms average?

    3 = 0.4 + miss_ratio * 60 gives a miss ratio of about 4.3%, so a hit ratio near 96%.

Summary

  • Hit ratio is hits divided by hits plus misses; Memcached exposes the counters in stats.
  • Misses are cold, expired or capacity misses, and each has a different fix.
  • Average access time is the lookup cost plus the miss ratio times the miss penalty.
  • The target ratio depends on how costly a miss is and how much load the backend can handle.
  • Watch latency and backend load too; the ratio alone can mislead.

Next: Locality and Working Sets