Skip to content
Tutorials Center
My learning

Topics

Lesson 32 of 42 ยท Operations, Security and Performance

Performance and Latency

Understand Redis speed: the single-threaded command loop, I/O threads, slow commands, SLOWLOG and LATENCY, big and hot keys, compact encodings and redis-benchmark.

Redis is fast enough that most applications never think about its speed. A single instance routinely handles over a hundred thousand commands per second on one CPU core, with answers in well under a millisecond. When Redis does become slow, the cause is almost never "Redis is slow" in general. It is one specific thing: a command that takes too long, a key that is too big, a network that adds round trips, or the machine itself. This lesson gives you the mental model and the tools to tell those cases apart.

The key fact is in the first section: Redis executes commands one at a time. Everything else, from why KEYS is dangerous to why big keys hurt other clients, follows from it. You have already seen the single-threaded idea in What Is Redis?; here we use it to diagnose problems.

Redis cannot run on this page, so shell and redis-cli snippets are read-only. The two runnable TryIts are plain JavaScript simulations in virtual time: costs are made-up, round numbers chosen to show shapes, not measurements of real hardware. Measure your own server with the tools shown below.

One command at a time

One main thread Executes commands in order.
Atomic No locks needed between commands.
Slow command Everyone behind it waits.
1 / 2

The single-threaded model

One thread, the main thread, reads each request from the sockets, runs the command against the in-memory data and writes the reply. Because only one command runs at any moment, commands are atomic with no locking, which is a big part of why Redis is both simple and quick. The price is that a command that takes 200 ms blocks every other client for 200 ms: their requests wait in line.

Since Redis 6, I/O threads (io-threads 4, default 1) can take over reading from and writing to sockets, which matters when the network work (many clients, large replies) dominates. Command execution stays on the main thread, so I/O threads raise throughput but do not make a slow command faster or let two commands run at once. Other work already happens in the background: closing files, fsync of the AOF, lazy freeing of large values (UNLINK), and the child process that BGSAVE forks.

The practical rule: keep every command short. A command's cost depends on what it does with the data. GET, SET, HGET, ZADD and LPUSH are O(1) or O(log N). KEYS pattern, SMEMBERS on a huge set, HGETALL on a huge hash, LRANGE 0 -1 on a huge list, SORT, DEL of a big collection and FLUSHALL are O(N) in the size of the data. The documentation of every command states its complexity; get used to reading it.

The first simulation puts a web application doing small GETs next to an admin who needs to list keys. Costs are in virtual microseconds: a GET costs 5, KEYS costs 0.5 per key (here 1,000,000 keys), a SCAN call that visits 1,000 keys costs 500. The admin either runs one KEYS *, or walks the key space with 1,000 SCAN calls, one after another.

JS One slow command delays everyone

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Read the three rows. With no admin work every request takes 5 virtual microseconds. With KEYS *, every request that arrives during the 500,000-microsecond block piles up behind it, so the maximum latency jumps to half a virtual second and thousands of requests are late. With SCAN, the same total work is cut into small pieces and web requests slip in between them, so the worst case is the cost of one SCAN call plus a GET. That is why production code uses SCAN, SSCAN, HSCAN and ZSCAN (and UNLINK instead of DEL for large keys) and never KEYS. The Keys, Expiry and TTL lesson showed the commands; now you know the reason.

Finding slow commands: SLOWLOG

Redis can remember the commands that were slow. The slow log records any command whose execution time (not including network or queueing time) exceeds slowlog-log-slower-than microseconds, keeping the newest slowlog-max-len entries in memory. Defaults: 10,000 microseconds (10 ms) and 128 entries. A value of 0 logs everything, a negative value disables it.

Text
127.0.0.1:6379> CONFIG SET slowlog-log-slower-than 5000
OK
127.0.0.1:6379> SLOWLOG GET 2
1) 1) (integer) 14
   2) (integer) 1767225600
   3) (integer) 18432
   4) 1) "SMEMBERS"
      2) "followers:celebrity"
   5) "10.0.0.7:51234"
   6) "web-worker"
127.0.0.1:6379> SLOWLOG LEN
(integer) 14
127.0.0.1:6379> SLOWLOG RESET
OK

Each entry has an id, the Unix time it started, the duration in microseconds, the command with its arguments (long arguments are truncated), and the client address and name when set. Here a single SMEMBERS on a huge set took 18 ms, and that is the kind of line you are looking for. Because SLOWLOG shows only execution time, a high client-side latency with an empty slow log points elsewhere: the network, a client-side queue, or the fork and disk pauses below.

The second simulation applies a threshold and a maximum length to a stream of commands with fixed costs.

JS A slow log in miniature

Press Run (or Ctrl + Enter) to see the console output here.

No output.

The log keeps only the three newest slow commands, so an old KEYS that caused trouble an hour ago can already be gone. Raise slowlog-max-len when you investigate, and ship the entries to your monitoring if you need history. The fast GET, HGETALL and UNLINK never appear: they are well below the threshold, which is the whole point.

Latency beyond commands

If the slow log is quiet but clients still see spikes, look at the other sources:

  • Fork latency. BGSAVE and AOF rewrites fork the process; on a large dataset the fork itself can pause the main thread for tens or hundreds of milliseconds, more on virtual machines (see Persistence: RDB and AOF). INFO stats shows latest_fork_usec.
  • Disk. With appendfsync always every write waits for the disk; everysec can still stall if the disk is saturated.
  • Swap and memory. If the OS swaps Redis pages to disk, latency becomes terrible. Keep maxmemory below physical memory and monitor used_memory_rss (see Memory Limits and Eviction). Disable Transparent Huge Pages on the host.
  • Expiry storms. Many keys expiring at the same second cost the active expiry cycle time; add random jitter to TTLs.
  • The network. Each request is a round trip. A thousand sequential commands at 0.5 ms each take half a second no matter how fast Redis is; Pipelining and Pub/Sub fixes that.

The built-in tools:

Text
$ redis-cli --latency                 # continuous round-trip latency (min / max / avg)
$ redis-cli --latency-history -i 15   # the same, with a new sample line every 15 s
$ redis-cli --intrinsic-latency 5     # run on the SERVER host: the latency of the machine itself
127.0.0.1:6379> CONFIG SET latency-monitor-threshold 100
127.0.0.1:6379> LATENCY LATEST
127.0.0.1:6379> LATENCY DOCTOR        # a human-readable analysis with advice

latency-monitor-threshold is in milliseconds: events (fork, fsync, a slow command, expiry cycles) that take longer than this are recorded, and LATENCY LATEST and LATENCY HISTORY event show them.

Big keys and hot keys

A big key is a key whose value holds a huge amount of data: a list of ten million items, a hash with a million fields, a 200 MB string. Reading, deleting or even replicating it costs a lot and blocks the main thread, and in a cluster it makes one node heavier than the others. Find them with:

Text
$ redis-cli --bigkeys          # scans with SCAN and reports the biggest key per type
$ redis-cli --memkeys          # the same, by memory use
127.0.0.1:6379> MEMORY USAGE followers:celebrity SAMPLES 0

Fix them by splitting data (a hash into buckets, a long list into time-sliced lists), deleting with UNLINK, and iterating with SCAN variants.

A hot key is one key that receives a disproportionate share of the traffic, such as a global counter or a viral item. Redis can run it fast, but it concentrates load on one node and one slot. redis-cli --hotkeys finds them when maxmemory-policy is an LFU policy. Remedies: cache the value in the application process for a second or two, or replicate the key under several names (item:7:a ... item:7:d) and read a random one.

Compact encodings

Redis stores small collections in compact encodings: hashes, lists and sorted sets use a listpack (a flat, contiguous byte array) while they are small, and sets of integers use an intset. When they pass a threshold, Redis converts them to the general structures (hash table, quicklist, skip list). Thresholds are config values such as hash-max-listpack-entries 128 and hash-max-listpack-value 64. A listpack saves a lot of memory, and for a few dozen elements scanning it is fast. You can check what Redis chose:

Text
127.0.0.1:6379> HSET small a 1 b 2
(integer) 2
127.0.0.1:6379> OBJECT ENCODING small
"listpack"

This is why storing a million small objects as small hashes (a few fields each) can use far less memory than a million separate string keys: see Hashes.

Benchmarking

redis-benchmark ships with Redis and generates load against a server:

Text
$ redis-benchmark -h 127.0.0.1 -p 6379 -t set,get -n 100000 -c 50 -q
$ redis-benchmark -t set,get -n 1000000 -c 50 -P 16 -q   # pipelines 16 commands per request

-n is the number of requests, -c the parallel connections, -P the pipeline depth and -d the value size in bytes. Treat the numbers carefully: they depend on the machine, the network, the value size and persistence settings, and benchmarking on the same machine as the server measures something else than your application will see. Benchmark with your real command mix, value sizes and a client on another host, and compare p99 latency, not only the average.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Why can a single KEYS * on a big database hurt an entire application?

    Redis executes commands on one main thread. An O(N) command occupies it for the whole run, and the requests of all other clients queue behind it.
  2. 2. What does the slow log measure?

    SLOWLOG records the execution time. A slow client with an empty slow log points to the network, queueing, a fork or disk problem.
  3. 3. What do io-threads change?

    I/O threads help when network work dominates, but command execution remains single-threaded.
  4. 4. Which is the safe way to walk all keys in production?

    SCAN returns a few keys per call, so other commands run between calls. KEYS does the whole walk in a single blocking command.
  5. 5. Which is a hot key problem rather than a big key problem?

    A hot key is about traffic concentrated on one key, not its size. Cache it locally or spread it over several keys.

Summary

  • Redis runs commands one at a time on its main thread, so a slow command delays everyone; I/O threads only offload socket work.
  • Read each command's complexity: avoid KEYS, SMEMBERS/HGETALL/LRANGE 0 -1 on huge keys, and big DEL; use SCAN variants and UNLINK.
  • SLOWLOG GET shows commands whose execution time exceeded slowlog-log-slower-than; LATENCY events and redis-cli --latency find fork, disk and system stalls.
  • Find big keys with --bigkeys, --memkeys and MEMORY USAGE; handle hot keys by local caching or key replication.
  • Small collections use compact encodings (listpack, intset), which save memory.
  • redis-benchmark gives a baseline; measure with your real workload and look at p99.

Next, the numbers worth watching all the time: Monitoring and Troubleshooting.