Skip to content
Tutorials Center
My learning

Topics

Lesson 14 of 95 ยท Working with Files

Archives and Compression

Learn tar, gzip, bzip2 and xz on Linux: create, list and extract archives, use -C and streaming pipes, compare compression ratios, extract safely and verify files with sha256sum.

Sooner or later you have to send a folder to someone, back it up, or download a program as a single file. Linux has two classic jobs for this, and they are done by two separate kinds of tool. Archiving bundles many files and directories (with names, permissions and timestamps) into one file. Compression makes a file smaller. The tool tar does the first, gzip, bzip2 and xz do the second, and the famous .tar.gz file ("tarball") is simply the two combined: first tar the files together, then compress the result.

Keeping the jobs apart is deliberate: it is the Unix philosophy of small tools that do one thing, connected by pipes. This lesson teaches tar and the compressors, compares their results with real numbers, shows how to extract archives safely, and finishes with checksums (sha256sum) for proving that a file has not changed. It follows File Permissions (tar stores them) and prepares you for the backup project.

tar: the archiver

tar ("tape archive", a name from the days of magnetic tape) has one mode option and a few modifiers. You must pick exactly one mode:

Option Mode
-c create an archive
-x extract
-t list the contents
-r append files to an archive

Common modifiers: -f FILE (the archive file name; it must be followed directly by the name), -v (verbose, list files as they are processed), -C DIR (change into DIR first), and a compression flag: -z gzip, -j bzip2, -J xz. A newer GNU tar picks the compression from the file name if you add -a (--auto-compress) when creating, and detects the format by itself when you extract or list.

The classic way to remember the creating command is the phrase "create ze file": tar -czf archive.tar.gz folder/. Old-style tar lets you leave out the dash (tar czf ...), and you will see both in the wild.

Bash Create, list and extract a tar archive

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

Three things to notice. tar stores the paths as you typed them: because we ran tar -cvf project.tar project, every member starts with project/, and extraction recreates that folder. -C DIR goes before the names it applies to when creating (tar -czf out.tgz -C /var/www site) and works as "cd to DIR first" when extracting. And -t is the safe first look at any archive you did not make yourself.

The trailing-path habit

How you name the input decides what is inside the archive. Compare:

Bash What goes inside the archive depends on how you name it

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

tar -cf one.tar site produces a tarball that unpacks into a single site/ folder: polite behaviour, and the convention for downloads. Archiving . with -C site makes the opposite, a "tar bomb" that scatters files into whatever directory you extract in. Always make archives that contain one top-level directory, and be wary of archives that do not. About absolute paths: GNU tar removes the leading / from member names (and says so on stderr: tar: Removing leading /' from member names; the listing shows the remaining relative path), so that extracting never overwrites system files such as /etc/passwd` by accident.

Compression: gzip, bzip2 and xz

The three standard compressors work on one file at a time (a stream of bytes). By default they replace the input file with a compressed version (file becomes file.gz), which surprises people because the original disappears.

Tool Extension Speed Compression Notes
gzip .gz fast good Universal. Decompression is very fast.
bzip2 .bz2 slow better Rarely chosen today.
xz .xz slowest to compress best Used for software releases; decompression is moderately fast.

Each has a matching tool to look inside (zcat, bzcat, xzcat), and -d to decompress (gunzip is gzip -d). Useful flags: -k keep the original, -1 to -9 set the level (default 6), -c write to standard output.

Let's compare them on data that compresses well, and print real sizes. We use gzip -n so no file name or time is stored, which keeps sizes the same on every run.

Bash Compare gzip, bzip2 and xz

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

(The numbers depend on the data: log files are extremely repetitive, so the ratios here are far better than you will get on photos or ZIP files.) The point is the pattern: all three shrink this text enormously, and here bzip2 and xz beat gzip by a factor of about ten, because the repetition spans long stretches that their larger windows can exploit. On ordinary text the gap is usually much smaller (xz is typically 20 to 40 percent smaller than gzip).

Already-compressed data does not compress again. JPEG, PNG, MP4, ZIP and .gz files are already squeezed; compressing them again wastes CPU and sometimes makes them slightly larger. Random data cannot be compressed at all, which is a fundamental limit, not a tool weakness.

The destructive default, and -k

gzip file deletes file. In scripts that is usually what you want (saves space); when experimenting, add -k. To compress without touching the original, use a redirect: gzip -c file > file.gz.

Bash gzip replaces the file unless you ask otherwise (fails on purpose)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

The last command shows the real message, gzip: b.txt already exists; followed by not overwritten, and an exit status of 2 (a warning). In a terminal gzip would ask whether to overwrite; with no one to answer (standard input is empty here) it refuses. It never overwrites silently, unlike cp.

tar + compression: the .tar.gz family

Add one letter to tar and it pipes through a compressor for you:

TEXT
tar -czf backup.tar.gz  dir/     gzip      (.tar.gz or .tgz)
tar -cjf backup.tar.bz2 dir/     bzip2     (.tar.bz2)
tar -cJf backup.tar.xz  dir/     xz        (.tar.xz)
tar -xf  anything.tar.*          extract: tar detects the compression itself
tar -tzf backup.tar.gz           list a gzip tarball (z optional when extracting/listing)
Bash Compressed tarballs and their sizes

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

A plain .tar is noticeably bigger than its contents (624,640 bytes against 436,800 here) because each member gets a 512-byte header and its data is padded to a multiple of 512 bytes. That is why many small files take more space in a tar than their raw bytes. Compression wins all of that back and much more: the .tar.gz is about a tenth of the tar. Compression recovers all of that and much more.

Streaming: tar and pipes

Because tar can read and write standard input and output (-f -, or simply no -f in some configurations), archives can flow through pipes without ever touching the disk. Combined with -C, this is the classic way to copy a directory tree, preserving permissions:

TEXT
tar -cf - -C /src .  |  tar -xf - -C /dst          local copy of a whole tree
tar -czf - dir       |  ssh host 'tar -xzf - -C /backups'   over the network
ssh host 'tar -czf - /etc' > etc-backup.tgz        pull a remote tree into a local file

The second and third lines use ssh, which the sandbox cannot do, so they are read-only here. The first works anywhere:

Bash Copy a tree through a tar pipe

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

The tar pipe keeps the 600 mode of one.txt. When tar extracts as a normal user it keeps permissions but cannot restore other owners (only root can); as root it also restores owners unless you add --no-same-owner.

Extracting safely

An archive is untrusted input, so before extracting an archive from somewhere else:

  1. List it first: tar -tf file.tar.gz | head. Look for a single top-level directory.
  2. Extract into a fresh empty directory (mkdir work && tar -xf file.tar.gz -C work), never straight into a directory full of important files.
  3. Beware absolute and .. paths. GNU tar strips a leading / and refuses members with .. by default, but old or foreign tools may not. tar -xf x.tar --one-top-level (GNU) puts everything in a folder named after the archive.
  4. Beware overwrites. Add -k (--keep-old-files) to refuse to replace existing files, or --skip-old-files to silently skip them.
  5. Check ownership and setuid bits. Extracting as root restores setuid and ownership from the archive, which is dangerous with an untrusted archive: use --no-same-owner --no-same-permissions.
Bash tar refuses to overwrite with -k

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

ZIP files

The ZIP format is common on Windows and macOS. On Linux you need the zip and unzip tools, which are not part of every base install and are not available in our sandbox, so the following is read-only:

CONSOLE
$ zip -r project.zip project/        create (recursive)
$ unzip -l project.zip               list
$ unzip project.zip -d output/       extract into a directory

ZIP differs from tar.gz in that it compresses each file separately inside the archive, which gives random access to single files but a worse ratio than compressing the whole tarball in one stream. For Linux-to-Linux transfers, .tar.gz or .tar.xz is preferred because tar also keeps Unix permissions, symlinks and owners properly.

Checksums: proving a file is intact

How do you know the 2 GB file you downloaded is identical to the original? You compare a checksum (hash): a short fingerprint computed from every byte. If one bit changes, the fingerprint changes completely. The tools are sha256sum (recommended), sha512sum, sha1sum and md5sum (fast, but broken for security: fine for spotting accidental corruption, never trust it against an attacker).

The standard workflow writes the checksums into a file and later checks them with -c:

Bash sha256sum: compute and verify (fails on purpose)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with read or cat), it gets the next line from this box.

The output format is HASH FILENAME (two spaces). sha256sum -c prints OK or FAILED per file, plus a warning and a non-zero exit status when something does not match, which is exactly what you want in scripts. Note that a checksum proves integrity only if you got the expected value from a trustworthy place (for example the vendor's HTTPS page), not from the same place as the file; for real authenticity people also sign the checksum list with GPG.

Archives made by tar contain timestamps and owner names, so tarballs of identical files made at different times have different checksums. For reproducible archives, GNU tar supports --sort=name --mtime='2026-01-01 UTC' --owner=0 --group=0 --numeric-owner, and gzip -n avoids storing a name and time. Build systems use this to get bit-identical packages.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What is the difference between tar and gzip?

    tar archives (keeps names, permissions and timestamps of many files in one file); gzip, bzip2 and xz compress one stream. A .tar.gz is both together.
  2. 2. Which command lists the contents of backup.tar.gz without extracting it?

    -t lists. -x extracts and -c creates (and would overwrite the archive). gzip -l shows compression statistics, not the member names of the tar inside.
  3. 3. What does gzip report.txt do by default?

    gzip removes the original after compressing. Use -k to keep it, or gzip -c report.txt > report.txt.gz.
  4. 4. Why should archives you create contain one top-level directory?

    An archive of '.' unpacks its files directly into the extraction directory (a tar bomb). Archiving a named directory makes extraction predictable.
  5. 5. How do you check a file against a list of saved SHA-256 checksums in SHA256SUMS?

    -c reads the HASH FILENAME lines, recomputes each hash and reports OK or FAILED, with a non-zero exit status if anything differs.

Summary

  • tar bundles files into one archive: -c create, -t list, -x extract, -f file name, -C change directory, -v verbose.
  • -z (gzip), -j (bzip2), -J (xz) add compression; on extract and list tar detects it for you.
  • gzip is fast and universal, xz compresses best; they replace the original unless you use -k or -c.
  • Archive one named directory, list before extracting, extract into an empty directory, and use -k against overwrites.
  • Tar pipes (tar -cf - ... | tar -xf - ...) copy trees without temporary files.
  • sha256sum creates a fingerprint and sha256sum -c verifies it; cmp compares two local files byte by byte.
  • zip/unzip exist for compatibility but compress file by file and lose Unix details.

Next: Disk Usage and File Information