Skip to content
Tutorials Center
My learning

Topics

Lesson 120 of 201 ยท Input and Output

Compressing Data with GZIP and ZIP

Compress data in Java with GZIPOutputStream and GZIPInputStream, build and read ZIP archives, verify data with CRC32, and avoid zip slip and zip bombs.

Text, logs, JSON and CSV are full of repetition, and repetition compresses well. Compressing a 10 MB log can shrink it to 1 MB, which means less disk space, faster downloads and cheaper storage. Java has compression built in: the java.util.zip package reads and writes the GZIP and ZIP formats with no extra libraries.

Both formats use the same algorithm underneath, DEFLATE, and both are lossless: decompressing gives back exactly the original bytes. The difference is the container. A GZIP file holds one compressed stream (think access.log.gz), while a ZIP file is an archive of many named entries, like a folder in a single file. JAR files are ZIP files too.

The classes are ordinary byte streams: you wrap them around another stream, write or read bytes, and close them. All the examples in this lesson work in memory with ByteArrayOutputStream, so they run anywhere.

GZIP: compressing a single stream

To compress, wrap any OutputStream in a GZIPOutputStream and write to it. To decompress, wrap an InputStream in a GZIPInputStream and read.

Java GZIP round trip in memory

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

Five hundred identical log lines shrink to almost nothing, because DEFLATE replaces repeats with short back-references. Real logs vary more and usually compress to 5โ€“15% of their size.

The compressed data is not complete until the stream is closed (or finish() is called). Closing writes the remaining compressed bytes and an 8-byte trailer with a checksum and the original length. Calling buffer.toByteArray() inside the try block gives a truncated stream, and reading it later fails with EOFException: Unexpected end of ZLIB input stream. The try-with-resources pattern above avoids that.

Every GZIP stream starts with the bytes 1f 8b. That magic number lets tools detect GZIP data, and you can use it too: peek at the first two bytes before deciding whether to wrap a stream in GZIPInputStream.

Files and real streams

With files, the pattern is identical. You just put a buffered file stream at the bottom of the chain and let transferTo copy the bytes:

Java
// Compress app.log -> app.log.gz
try (InputStream in = Files.newInputStream(Path.of("app.log"));
     OutputStream out = new GZIPOutputStream(
             new BufferedOutputStream(Files.newOutputStream(Path.of("app.log.gz"))))) {
    in.transferTo(out);
}

// Read a .gz file line by line, without unpacking it to disk first
try (var reader = new BufferedReader(new InputStreamReader(
        new GZIPInputStream(Files.newInputStream(Path.of("app.log.gz"))),
        StandardCharsets.UTF_8))) {
    reader.lines().filter(line -> line.contains("ERROR")).forEach(System.out::println);
}

The second snippet shows a powerful idea: streams compose. Bytes flow from the file, through decompression, through UTF-8 decoding, into lines, and at no point is the whole file in memory. The same chain works for an HTTP response body or a socket.

What compresses well, and compression levels

DEFLATE finds repeated byte sequences. Text and structured data have lots of them. Data that is already compressed (JPEG, PNG, MP4, ZIP, encrypted bytes) or truly random has none, and compressing it again makes it slightly bigger.

You can trade speed for size with a compression level from 0 (store only) to 9 (smallest). GZIPOutputStream always uses the default level (6), so to choose one you use the lower-level Deflater class directly:

Java What compresses, and how levels compare

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

The text shrinks a lot even at level 1, while the random bytes never get smaller than the input. Level 0 only adds framing. Level 9 usually saves less than 10% over the default, at a noticeably higher CPU cost on large inputs. Rules of thumb:

  • Use the default level for general work, level 1 (Deflater.BEST_SPEED) when compressing on the fly under load.
  • Don't compress JPEGs, videos or existing archives again.
  • Inflater is the matching decompressor. Deflater and Inflater hold native memory, so call end() when you use them directly. The stream classes do that for you on close().

Creating a ZIP archive

A ZIP archive contains entries, each with a name (which may include / for folders), compressed data and metadata. With ZipOutputStream you call putNextEntry, write that entry's bytes, call closeEntry, and repeat:

Java Writing and reading a ZIP archive

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

The 3.9 KB CSV, with its repeated rows, shrinks to a few dozen bytes. The two tiny text files actually get a couple of bytes bigger: below a few dozen bytes there's nothing to gain, and DEFLATE's framing costs more than it saves. A few details that matter:

  • ZipInputStream.readAllBytes() reads to the end of the current entry, not the whole archive. getNextEntry() returns null when there are no more entries.
  • When you read a ZIP as a stream, sizes and CRCs may only be known after the entry's data has been read, because ZipOutputStream writes them after the data. That's why the example prints getCompressedSize() after readAllBytes(). Before that it can be -1.
  • Entry names always use /, even on Windows. A name ending in / is a directory entry.
  • ZIP archives start with PK, the initials of Phil Katz, who created the format.
  • Entry names are encoded as UTF-8 by default. Pass a Charset to the constructors only for old archives made by tools that used another encoding.

Random access: ZipFile and the zip file system

ZipInputStream reads front to back, which is perfect for streams from the network. When the archive is a file on disk, ZipFile is better: it reads the central directory at the end of the file, so it can list all entries instantly and open any one of them directly.

Even more convenient is the zip file system: Java can mount a ZIP file as a FileSystem, and then the familiar Path and Files methods from Reading and Writing Files work inside the archive.

Java ZipFile and the zip FileSystem

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

The zip file system is the easiest way to add or replace a single entry in an existing archive: open it, Files.writeString the entry, close it. With the stream classes you would have to copy every entry into a new archive.

Checksums: CRC32 and friends

ZIP and GZIP both store a CRC-32 checksum of the original data, and verify it when you decompress; a mismatch throws ZipException. You can use the same checksum classes on their own to detect accidental corruption of files, messages or cache entries:

Java Detecting corruption with CRC32

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

CRC32, CRC32C (Java 9+, often faster on modern CPUs) and Adler32 all implement the Checksum interface. They detect accidental damage only. Anyone can change data and recompute a CRC, so for protection against tampering use a cryptographic hash such as SHA-256 (UUIDs, Hashing and Base64).

Unzipping safely: zip slip and zip bombs

Extracting an archive you didn't create is a classic security hole. Two attacks to guard against:

  • Zip slip. An entry named ../../.bashrc or /etc/cron.d/job would be written outside the folder you meant to extract into. Always resolve the entry name against the target folder, normalize() it, and check that the result still starts with the target folder.
  • Zip bombs. A tiny archive can expand to gigabytes (a file of zeros compresses about 1000 to 1). Never trust the sizes stored in the archive; count the bytes you actually extract and stop at a limit.
Java A safe unzip that rejects zip slip

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

The first archive extracts normally. The second one contains a path that climbs out of the target folder, so unzip stops with an error. Notice that ok.txt was already extracted before the bad entry was reached; production code often extracts into a temporary folder and moves it into place only when everything succeeded.

Which class to use

Task Use
Compress one file or stream (.gz) GZIPOutputStream / GZIPInputStream
Choose a compression level, raw DEFLATE Deflater / Inflater (call end())
Create a ZIP from a stream of data ZipOutputStream
Read a ZIP arriving as a stream ZipInputStream
Read a ZIP file on disk, random access ZipFile
Treat a ZIP like a folder, add or replace entries FileSystems.newFileSystem(path)
Detect accidental corruption CRC32, CRC32C, CheckedInputStream

Other formats, such as .tar, .7z, .xz, Zstandard or Brotli, aren't in the JDK; libraries like Apache Commons Compress add them.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. You write data into a GZIPOutputStream wrapped around a ByteArrayOutputStream and call toByteArray() before closing the GZIP stream. What is the likely result?

    The final compressed block and the trailer (CRC and length) are only written by finish() or close(). Read the buffer after the try-with-resources block ends.
  2. 2. Why does compressing an MP4 video with GZIP usually make it slightly larger?

    Already-compressed or random data has no redundancy for DEFLATE to remove, so the output is the input plus a little framing overhead.
  3. 3. Inside a while ((entry = zip.getNextEntry()) != null) loop over a ZipInputStream, what does zip.readAllBytes() return?

    A ZipInputStream behaves like the current entry's decompressed data and reports end of stream at the end of that entry. getNextEntry moves to the next one.
  4. 4. An archive contains an entry named "../config/settings.xml". What should a safe extractor do?

    This is the zip slip attack. Normalizing and checking startsWith(root) catches entries that would land outside the extraction folder.
  5. 5. What does a CRC32 checksum protect against?

    CRCs are fast error-detection codes. An attacker can modify data and recompute the CRC, so use SHA-256 or a MAC when tampering is a concern.

Summary

  • java.util.zip supports GZIP (one compressed stream) and ZIP (an archive of named entries); both use lossless DEFLATE.
  • Wrap streams: GZIPOutputStream/GZIPInputStream, ZipOutputStream/ZipInputStream. Close (or finish) the compressing stream before using its output.
  • Text compresses very well; already-compressed and random data don't. Use Deflater to pick a level and call end() on it.
  • For ZIP files on disk, ZipFile gives random access and the zip FileSystem lets you use Path and Files inside an archive.
  • CRC32/CRC32C detect accidental corruption, not tampering.
  • When extracting untrusted archives, block zip slip with normalize() + startsWith, and cap the bytes you extract.

Next: Threads, where your programs start doing several things at once.