Skip to content
Tutorials Center
My learning

Topics

Lesson 66 of 95 ยท Projects

Project: A Backup Script

Build a real backup tool in Bash: timestamped tar.gz archives, SHA-256 verification, rotation of old backups, dry-run mode, logging and a safe restore script.

In this project you build a small but honest backup tool, step by step. It takes a source directory, writes a compressed tar archive with a timestamp in its name, records a checksum next to it, deletes the oldest backups when there are too many, logs what it did, and has a matching restore script that refuses to restore a corrupted archive.

Along the way you practise almost everything from the Shell Scripting chapter: functions, option parsing, error handling and traps, archives and the design habits from Designing Maintainable Scripts. The script is deliberately deterministic: you can pass the "current time" as an environment variable, so every example here prints the same names on every run, and the same trick makes the script easy to test.

The plan

TEXT
  source dir  --tar+gzip-->  dest/backup-NAME-20260105T020000Z.tar.gz
                                      |
                                      +--sha256sum-->  ....tar.gz.sha256
                                      |
   rotation: keep the newest K, delete the rest (and their .sha256)
   log:      dest/backup.log, one line per event
   restore:  verify checksum -> extract into an empty directory

Design decisions, made up front:

  • Names sort chronologically. A timestamp like 20260105T020000Z (year first, UTC) means a plain alphabetical sort is also a time sort, so rotation needs no date arithmetic.
  • Time is an input. The script reads BACKUP_NOW (seconds since 1970) and falls back to $EPOCHSECONDS. Tests set it; production does not.
  • Write, verify, then publish. The archive is built under a temporary name and renamed only when complete, so a crash never leaves a half-written file that looks like a backup.

Step 1: a deterministic archive

First, the core: build a tar.gz of a directory. GNU tar has options that make the result reproducible: --sort=name (a fixed file order), --mtime= (one fixed time stamp for all entries), --owner=0 --group=0 --numeric-owner (no user names), and gzip -n (no name or time stored in the gzip header). With these, the same input gives a byte-identical archive, which means the same checksum, so you can even tell whether anything changed between two backups.

Bash Step 1: create and inspect a timestamped archive

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

-C . tells tar which directory to start from, and giving the relative name project means the archive stores project/... rather than absolute paths. Storing absolute paths is a classic mistake: restoring would try to write into the original location. GNU tar strips the leading / and warns, but a relative layout is cleaner.

Step 2: checksums and verification

An archive you have never verified is a hope, not a backup. sha256sum writes a line HASH FILENAME that sha256sum -c can check later. Create the checksum file in the same directory and run the check from there, so the relative file name inside it resolves.

Bash Step 2: checksum and detecting corruption (fails on purpose)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

sha256sum --check prints OK or FAILED per file, then a warning line, and returns non-zero when any check failed, so it works directly in if. The same hash lines appear each time you run this, because the archive is reproducible. A checksum protects against silent corruption (a failing disk, a truncated copy); it does not protect against tampering by someone who can also rewrite the .sha256 file.

Step 3: rotation

Backups pile up. Rotation keeps only the newest K archives. Because names sort chronologically, this is a glob, a count and a loop. The glob expands in sorted order, so the oldest names come first:

Bash Step 3: keep only the newest K backups

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Three details matter. nullglob makes an unmatched pattern expand to nothing instead of the literal text. The arithmetic ${#files[@]} - keep goes negative when there are fewer backups than keep, in which case the for loop simply does not run. And rm -f -- "$f" "$f.sha256" removes the checksum together with the archive; orphaned .sha256 files would otherwise accumulate forever.

Rotation is the dangerous part of any backup tool, because it deletes. Never rotate before the new backup has been written and verified. If the new archive fails, you still want the old ones.

Step 4: the complete tool

Now the full script. It uses a small run helper so that --dry-run shows what would happen without touching anything, writes a log, builds the archive under a temporary name guarded by an EXIT trap, verifies the archive right after creating it, and only then rotates. Read it from top to bottom; each function is one idea from the earlier steps.

Bash Step 4: backup.sh and restore.sh, tested with fixed times

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Walk through what happens on one real run:

  1. getopts parses -n and -k; everything left in $@ must be exactly two paths.
  2. The script builds the archive name from the timestamp and writes to a hidden .partial file. The EXIT trap removes that file if anything fails before the mv.
  3. mv publishes the finished archive, a checksum file is written, and --check --status verifies it silently (the status alone is the answer). A failure dies before rotation can delete anything.
  4. rotate removes the oldest archives beyond KEEP, and run makes those deletions respect --dry-run.
  5. Every step is logged with the (injected) time.

The restore script does the mirror image: check files exist, verify the checksum, refuse a non-empty target, and only then extract. Refusing to overwrite is a deliberate safety choice; a restore that silently merges into a live directory can do more harm than the problem it fixes.

Testing and extending

Because time is injected, tests are easy: run the script with fixed BACKUP_NOW values and assert on the file names, as the demo did. A good next step is to turn those checks into assertions with the approach from Designing Maintainable Scripts or the framework you will build in Project: A Mini Test Framework.

Ideas to extend it:

  • Exclude patterns with tar --exclude=PATTERN (for node_modules, caches, .git).
  • Incremental backups using tar --listed-incremental=snapshot.file, or rsync --link-dest for hard-link based snapshots that share unchanged files.
  • Remote copies: push the finished archive with scp or rsync (see SSH, scp and Working Remotely). Read-only here: our sandbox has no network.
  • Scheduling: run it from cron or a systemd timer (Packages, Services and systemd), with a lock so two runs never overlap.
  • Encryption: pipe the archive through gpg or age before it leaves the machine.
  • The 3-2-1 rule: three copies, on two kinds of media, one off-site. A backup in the same room as the original is not yet a disaster plan, and a backup you have never restored is not yet a backup. Test restores regularly.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Why does the script use timestamps like 20260105T020000Z in archive names?

    With the year first and a fixed-width format in UTC, sorting names sorts by time. The oldest archives come first in the glob, so rotation just removes the first few.
  2. 2. What is the purpose of writing to a .partial file and then running mv?

    The final name appears only when the archive is complete. If the script is interrupted, only the hidden partial file exists, and the EXIT trap removes it.
  3. 3. Why should rotation run only after the new backup has been verified?

    Deleting old backups before knowing that the new one is good can leave you with nothing. Verify first, rotate afterwards.
  4. 4. What does sha256sum --check --status file.sha256 return?

    --status suppresses all output; the exit code is the result. That fits if and || in scripts.
  5. 5. Why does the restore script refuse to extract into a non-empty directory?

    A restore that overwrites files in place can destroy newer data. Forcing an empty target makes the person (or script) choose where the restored copy goes.

Summary

  • Build archives with tar into gzip -n; --sort=name, --mtime, --owner=0 --group=0 --numeric-owner make them reproducible.
  • Put a UTC, year-first timestamp in the name so alphabetical order is time order.
  • Write to a temporary name, verify with sha256sum --check, then rename; clean up with an EXIT trap.
  • Rotate only after a verified new backup; delete the .sha256 file with its archive.
  • Inject the time (BACKUP_NOW) so runs and tests are deterministic, and wrap deletions in run for --dry-run.
  • The restore script verifies first, refuses non-empty targets, and a backup is only real once you have restored it successfully.

Next: Project: Log Analyzer