Skip to content
Tutorials Center
My learning

Topics

Lesson 27 of 42 ยท Persistence and Durability

Persistence: RDB and AOF

Understand Redis persistence: RDB snapshots, BGSAVE and save points, the append-only file, fsync policies, AOF rewrite, multi-part AOF, hybrid mode and backups.

Redis keeps its working data in memory, so a restart would normally start you with an empty server. Persistence is the set of features that write data to disk so Redis can load it again after a restart or a crash. There are two mechanisms: RDB snapshots, which save the whole dataset at points in time, and the AOF (append-only file), which logs every write command so Redis can replay them.

Neither is free, and neither is all-or-nothing. Persistence is a dial between speed, disk usage and how much data you may lose when the machine dies. Choosing a setting starts with a question: if Redis vanished right now, what would I be willing to lose? For a pure cache, "everything" is fine and you can disable persistence. For sessions, queues or leaderboards, you want a small, known loss window.

Real Redis cannot run on this page, so configuration and redis-cli snippets are read-only. The runnable examples are JavaScript stand-ins: an append-only log that is written and replayed, a simulation of crash losses under each setting, and a save-point evaluator. They model the logic, not the real file formats or disk behaviour.

Two ways to save

RDB snapshot The whole dataset at one moment.
AOF Every write command, in order.
Both Snapshot base plus a log of changes.
1 / 2

RDB snapshots

An RDB file (by default dump.rdb in the dir directory) is a compact binary image of the whole dataset. SAVE writes it in the main thread and blocks every client until done, so it is rarely used in production. BGSAVE is the practical command: Redis calls fork(), and the child process writes the file while the parent keeps serving requests. The operating system shares memory pages between parent and child and copies a page only when one of them changes it (copy-on-write), so the snapshot is consistent as of the instant of the fork.

sequenceDiagram
    participant C as Clients
    participant P as Redis main process
    participant K as Child process
    participant D as Disk
    C->>P: writes and reads
    P->>K: BGSAVE (fork)
    K->>D: write temp-dump.rdb
    C->>P: more writes (pages copied on change)
    K->>D: rename to dump.rdb
    K-->>P: done

The child writes a temporary file and renames it at the end, so a crash mid-save never leaves a half-written dump.rdb.

Redis triggers snapshots by itself through save points: "after N seconds if at least M keys changed". Recent versions default to:

TEXT
save 3600 1 300 100 60 10000
dbfilename dump.rdb
dir /var/lib/redis
rdbcompression yes
stop-writes-on-bgsave-error yes

That means: save after 3600 s if at least 1 change, or after 300 s if at least 100 changes, or after 60 s if at least 10,000 changes. Whichever rule matches first starts a BGSAVE. LASTSAVE returns the time of the last successful save, and INFO persistence shows rdb_changes_since_last_save and the status of the last save. Redis also saves on a clean SHUTDOWN.

Strengths: one compact file, ideal for backups and disaster recovery, and fast restarts for big datasets. Weaknesses: you lose everything written since the last snapshot (minutes, with the defaults), and on a large dataset the fork itself can pause Redis for a noticeable moment while the kernel copies page tables.

JS When does a save point trigger?

Press Run (or Ctrl + Enter) to see the console output here.

No output.

A busy server snapshots every minute, a lightly used one every five minutes, and an almost idle one only once an hour. That is the loss window for RDB alone.

The append-only file

With appendonly yes, Redis appends every write command to a log in the same format clients use on the wire (RESP, see installing Redis and redis-cli). Reads are not logged. After a restart, Redis starts empty and replays the log from the start, rebuilding the dataset exactly. (Commands with relative times, such as EXPIRE, are logged as absolute times so a replay days later gives the same result.)

How safe the log is depends on appendfsync, which controls when Redis asks the operating system to force the data onto the physical disk (fsync):

Setting Behaviour Loss after a crash
always fsync before replying to the client Essentially none for acknowledged writes; slowest
everysec fsync once per second in the background (the default) About one second of writes
no Never calls fsync; the OS flushes when it chooses Whatever the OS had not flushed, often up to tens of seconds

everysec is the usual choice: close to the speed of no fsync, losing at most about a second.

Rewrite. An append-only log grows forever, and INCR counter a million times would log a million lines for one key. BGREWRITEAOF builds a new, minimal log from the current state (one SET counter 1000000) in a forked child, then swaps it in. Redis does this automatically when the file has grown by auto-aof-rewrite-percentage (default 100, meaning doubled) and is at least auto-aof-rewrite-min-size (default 64 MB).

The next TryIt writes a real RESP log from a tiny emulator, restarts by replaying it, rewrites it, and shows what happens when the file is cut in the middle of a command (as after a crash).

JS An AOF written, replayed, rewritten and truncated

Press Run (or Ctrl + Enter) to see the console output here.

No output.

The log has 11 entries and the rewritten version only 4, yet both rebuild identical state. In the last step the final command was cut off in the middle, so it is ignored (real Redis does this by default through aof-load-truncated yes and logs a warning; the redis-check-aof --fix tool can repair a damaged file). Anything that was damaged in the middle of the file stops the load and needs repair.

Multi-part AOF and the hybrid format

Since Redis 7.0 the AOF is not one file but a directory (appenddirname, default appendonlydir) with three kinds of files:

  • a base file: the snapshot produced by the last rewrite,
  • one or more incremental files: commands logged since then,
  • a manifest file listing which files make up the current AOF, and in what order.

A rewrite just creates a new base and starts a fresh incremental file, with no more buffering of changes in memory during the rewrite. With aof-use-rdb-preamble yes (the default) the base file uses the compact RDB format, so you get a fast load of the base followed by a replay of only recent commands. That is the hybrid mode, and why modern AOF files are small and quick to load.

Choosing a setup

Goal Setup
Pure cache, data is rebuildable No persistence (save "", appendonly no), or RDB only for faster warm starts
Reasonable safety with simple backups RDB with default save points
Lose at most about a second AOF with appendfsync everysec (often together with RDB for backups)
Almost no acknowledged loss AOF with appendfsync always, accepting lower throughput
Highest safety AOF plus replicas and WAIT (see replication)

If both RDB and AOF are enabled, Redis loads the AOF on restart because it is the more complete record. Turn AOF on in a running server with CONFIG SET appendonly yes; Redis starts a rewrite to produce the first base file.

The next simulation compares loss windows. A write arrives every 100 ms for ten minutes, and the process dies at 4 minutes 50.35 seconds. It counts how many acknowledged writes survive under each policy, using the models described above (the OS flush interval for no is a typical Linux figure, not a guarantee).

JS How many writes survive a crash?

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Under everysec only the last fraction of a second is at risk, no loses a few hundred writes in this model, RDB with a 60 second rule loses everything since the last minute mark, and with a 300 second rule the first snapshot has not even happened yet. Remember the always case still depends on the disk honouring fsync; consumer SSDs and virtual disks with write caches can lie about it.

Operating persistence safely

  • Backups: an RDB file is written atomically (temp file, then rename), so copying dump.rdb is safe. Copy it off the machine on a schedule, keep several generations, and test a restore; a backup you never restored is a hope. For a multi-part AOF, copy the whole directory (and avoid copying during a rewrite). redis-cli --rdb file.rdb can fetch a snapshot from a remote server.
  • Disk and memory headroom: the fork can need extra memory (copy-on-write) when many keys change during a save, and a rewrite needs disk space for the new base file. Monitor both; stop-writes-on-bgsave-error yes makes Redis refuse writes after a failed save so you notice instead of silently running unprotected.
  • Disk full or slow: a stalled fsync can delay writes under always and, rarely, even under everysec. Use fast local storage for the data directory.
  • Persistence is not replication: a persisted instance on one machine still dies with the machine. Add replicas, covered in replication, and remember that replication is asynchronous.
  • Verify files: redis-check-rdb and redis-check-aof inspect and (for AOF) fix damaged files.

Backup and restore, step by step

The previous section lists the rules; this section walks through the routines. Treat a backup plan as three questions: how much data can I afford to lose (the recovery point), how long can I be down (the recovery time), and where do the copies live if the whole machine or region is gone.

Taking a consistent backup

  1. Snapshot. Run BGSAVE (never SAVE on a busy server) and wait until INFO persistence shows rdb_bgsave_in_progress:0 and rdb_last_bgsave_status:ok. LASTSAVE should have moved forward.
  2. Copy. Copy dump.rdb from the dir directory. It is replaced atomically, so the copy is always a complete file. For the append-only file, copy the whole appendonlydir (base, incremental files and manifest belong together), and do not copy while a rewrite is running.
  3. Ship it off the machine with a timestamp in the name, keep several generations, and encrypt it if it holds personal data.
  4. Verify. Run redis-check-rdb dump.rdb or redis-check-aof appendonlydir/*.manifest, and restore into a scratch instance now and then.

Prefer to take the copy from a replica, so the fork and disk activity do not touch the primary.

Restoring

  1. Stop the server (or start a fresh one). Redis reads its files only at startup, and replacing dump.rdb under a running server does nothing, because the next save would overwrite it.
  2. Remove or move away any old dump.rdb and appendonlydir you do not want loaded. If AOF is on, Redis loads the AOF and ignores dump.rdb, so restoring an RDB into a server with appendonly yes quietly loads the empty log instead.
  3. Put the backup file in the dir directory under the name in dbfilename, with permissions the Redis user can read.
  4. Start Redis and watch the log for DB loaded from disk. Check DBSIZE and a few known keys.
  5. Point clients and replicas back. Replicas will do a full resync from the restored primary.

If you need AOF but have only an RDB backup, start the server without AOF, let it load the RDB, then enable it with CONFIG SET appendonly yes. Redis rewrites the dataset into a first AOF base file in the background, and once aof_rewrite_in_progress is 0 you add appendonly yes to redis.conf so it survives restarts.

Switching from RDB to AOF safely

The order matters. If you add appendonly yes to the config and restart, Redis starts with an empty AOF and ignores your RDB, so you lose everything. Instead:

Text
redis-cli BGSAVE                         # safety copy first
redis-cli CONFIG SET appendonly yes      # starts the first rewrite
redis-cli INFO persistence               # wait for aof_rewrite_in_progress:0
redis-cli CONFIG REWRITE                 # writes appendonly yes into redis.conf

Disaster scenarios

What fails Recovery
Process crash, machine healthy Restart; Redis loads its own files. Loss is bounded by your save rules or appendfsync.
Disk corrupted or deleted Restore the latest off-machine backup, or promote a replica.
Whole machine gone Promote a replica (see Sentinel), or build a new instance from the backup.
Whole region gone Restore from a backup stored in another region, or use a cross-region replica.
Bad deployment wiped keys (FLUSHALL, a buggy script) Replicas copied the damage. Only an older backup helps, so keep generations.
Truncated AOF after a crash redis-check-aof --fix removes the broken tail; Redis can also load a truncated AOF with aof-load-truncated yes.

The model below compares how much recent data each plan could lose in the worst case, given a snapshot interval or fsync policy and a write rate. It is a calculation, not a Redis server.

JS How much data can each plan lose?

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Notice that always shows zero risk only while the disk survives: when the machine is lost, the risk is whatever your off-site copy or replica holds. A plan protects only against the failures it was designed for. Fast restarts need a short load time, which is another reason to keep the dataset lean (see eviction and memory).

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Why does BGSAVE not block clients the way SAVE does?

    The child gets a copy-on-write view of memory as of the fork and writes it to disk, so the main process is free to handle commands.
  2. 2. With the default appendfsync everysec, how much data can you lose in a crash?

    Redis calls fsync once per second, so writes since the most recent fsync can be lost, which is roughly a second.
  3. 3. Why does Redis rewrite the AOF?

    A log keeps every change, so a counter incremented a million times has a million lines. A rewrite replaces them with one SET.
  4. 4. Both RDB and AOF are enabled and Redis restarts. Which one is loaded?

    When AOF is enabled Redis prefers it, since it normally contains writes newer than the last snapshot.
  5. 5. What does the hybrid format (aof-use-rdb-preamble yes) put in the AOF base file?

    The base file after a rewrite is an RDB-format snapshot for fast loading, and newer writes live in the incremental files listed in the manifest.

Summary

  • RDB saves point-in-time snapshots (BGSAVE forks a child; save points such as save 3600 1 300 100 60 10000 trigger it); it is compact and backup-friendly but can lose minutes of writes.
  • AOF logs every write command; appendfsync (always, everysec, no) sets the loss window, with everysec as the usual balance.
  • BGREWRITEAOF shrinks the log; Redis 7 stores the AOF as a base file, incremental files and a manifest, with an RDB-format base by default.
  • A truncated AOF tail is ignored at load; other damage needs redis-check-aof --fix.
  • With both enabled, restart loads the AOF. Back up RDB files off-machine, test restores, and combine persistence with replication.

Next, copying data to other servers: Replication.