Skip to content
Tutorials Center
My learning

Topics

Lesson 92 of 147 ยท Standard Library

archive/zip, tar and compress/gzip

Create and read zip and tar archives in Go, compress data with gzip and zlib, stream files safely, and avoid zip-slip and decompression-bomb problems, with runnable examples.

Archives and compression are two different ideas that are often combined. An archive bundles many files into one (with their names, sizes and modes). Compression makes a stream of bytes smaller. A .zip file is an archive where each file is compressed on its own. A .tar.gz file is a tar archive (no compression) that is then compressed as a whole with gzip.

Go's standard library covers the common formats: archive/zip, archive/tar, compress/gzip, compress/zlib, compress/flate and compress/bzip2 (read-only). Everything is built on io.Reader and io.Writer, so the pieces snap together like pipes (see Reading and Writing with os, io and bufio). To keep the examples runnable here, they use in-memory buffers rather than files; replace bytes.Buffer with an *os.File to work on disk.

flowchart LR
    A["Files"] --> B["tar.Writer<br/>(archive)"]
    B --> C["gzip.Writer<br/>(compress)"]
    C --> D["archive.tar.gz"]
    D --> E["gzip.Reader<br/>(decompress)"]
    E --> F["tar.Reader<br/>(extract)"]
    F --> G["Files"]

A .tar.gz is two layers: tar bundles the files, gzip squeezes the whole stream. Reading peels the layers in reverse.

Compressing a stream with gzip

gzip.NewWriter wraps any io.Writer. Everything you write to it is compressed and passed on. You must call Close to flush the final bytes and the checksum; forgetting it produces a truncated file.

Go Compress and decompress with gzip

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

Repetitive text compresses extremely well; random data and already-compressed files (JPEG, MP4, zip) barely shrink. gzip.NewWriterLevel lets you pick gzip.BestSpeed, gzip.BestCompression or gzip.DefaultCompression. The header can carry a file name and modification time (zw.Name, zw.ModTime) if you set them before the first Write.

The sibling packages work the same way. compress/zlib uses a different, lighter wrapper (it is what PNG and many network protocols use), and compress/flate is the raw algorithm underneath both. compress/bzip2 can only decompress. compress/gzip is the format you meet in HTTP (Content-Encoding: gzip) and in log files.

Streaming without holding everything in memory

Because gzip is a reader/writer wrapper, you can connect it directly to files or network connections with io.Copy. This compresses a huge file using a small, constant amount of memory:

Go
func gzipFile(dst, src string) error {
	in, err := os.Open(src)
	if err != nil {
		return err
	}
	defer in.Close()

	out, err := os.Create(dst)
	if err != nil {
		return err
	}
	defer out.Close()

	zw := gzip.NewWriter(out)
	if _, err := io.Copy(zw, in); err != nil {
		return err
	}
	if err := zw.Close(); err != nil {
		return err
	}
	return out.Close()
}

Creating and reading zip archives

A zip file stores a list of entries and a central directory at the end, so a reader can jump to any file without scanning the whole archive. That is why zip.NewReader needs an io.ReaderAt and the total size, rather than a plain io.Reader.

To write, create a zip.Writer, call Create(name) for each file (it returns an io.Writer), write the content, and finish with Close:

Go Create a zip in memory and list its entries

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

A zip.Reader also implements fs.FS through zr.Open, so zip contents work with fs.WalkDir, fs.ReadFile and http.FS (see Embedding Files with go:embed for the fs package in action). On disk, use zip.OpenReader(path), which returns a reader that you must close.

Use zw.CreateHeader(&zip.FileHeader{Name: ..., Method: zip.Deflate}) when you need control over modification time, permissions or the compression method (zip.Store keeps the data uncompressed).

Creating and reading tar archives

Tar is the standard archive on Unix. It is a simple sequence of header + data blocks, with no index, so you read it from start to finish. A header (tar.Header) holds the name, size, mode, modification time and the type of the entry.

When writing, you must give the exact Size in the header before you write the content. Tar writes are checked: writing more bytes than the header says returns tar.ErrWriteTooLong.

Go Build a .tar.gz and read it back

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

tr.Next() moves to the next entry, and tr itself then reads just that entry's data, ending with io.EOF at the entry's end. When Next itself returns io.EOF, there are no more entries.

Extracting archives safely

Archives come from the outside world, so treat them as untrusted input. Two attacks matter most:

  1. Zip slip / path traversal. An entry can be named ../../etc/cron.d/evil. If you join that name to your target directory and write the file, you escape the directory. Always check that the cleaned destination stays inside the target.
  2. Decompression bombs. A tiny file can expand to many gigabytes. Limit how much you read with io.LimitReader, and check sizes against a budget before writing.

Here is a safe pattern for validating names. It uses filepath.IsLocal (Go 1.20 and later), which rejects absolute paths and anything that climbs out with ..:

Go Rejecting dangerous archive entry names

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with bufio.Scanner or fmt.Scan), it gets the next line from this box.

For tar, also look at hdr.Typeflag: skip or handle symbolic links and device files explicitly instead of blindly writing them. Since Go 1.24, os.Root can also confine file access to one directory, but explicit validation stays the most readable defense. See Files and Paths for the path helpers.

Which format should you use?

Need Choose
One file or network stream, smaller gzip (or zlib)
Many files, random access to one of them, works everywhere (Windows, Office files) zip
Many files in a pipeline on Unix, streaming, keeping permissions tar, usually with gzip
HTTP responses gzip, via Content-Encoding

Always check the errors from Close: for writers, Close is where the last bytes and the checksums are written, so an ignored error can hide a corrupt archive.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What is the difference between an archive and compression?

    tar and zip combine many files (zip also compresses each one); gzip compresses a single stream. A .tar.gz uses both.
  2. 2. What happens if you forget to call Close on a gzip.Writer?

    Close flushes buffered data and writes the gzip trailer. Without it the stream is incomplete.
  3. 3. Why does zip.NewReader need an io.ReaderAt and a size?

    A zip's index lives at the end, so the reader seeks there first and then jumps to individual entries.
  4. 4. What must you set in a tar.Header before writing an entry?

    Tar headers come before the data, so the size must be known in advance. Writing more than Size returns tar.ErrWriteTooLong.
  5. 5. How do you protect against a "zip slip" attack when extracting?

    Names like ../../x can escape the destination. Reject non-local paths (for example with filepath.IsLocal) before creating files.

Summary

  • Archives (archive/zip, archive/tar) bundle files; compressors (compress/gzip, zlib, flate) shrink streams. .tar.gz combines both.
  • gzip.NewWriter and gzip.NewReader wrap any writer or reader; always Close writers and check the error.
  • Zip has a central directory: write with zip.NewWriter and Create, read with zip.NewReader (needs io.ReaderAt and size) or zip.OpenReader.
  • Tar is sequential: write a tar.Header with the exact Size, then the data; read with Next() and the reader.
  • Stream with io.Copy to keep memory use small.
  • Treat archives as untrusted: validate entry names (zip slip), limit decompressed size (bombs) and handle special file types deliberately.

Next: math/big: Big Integers, Rationals and Floats covers arbitrary-precision arithmetic.