Skip to content
Tutorials Center
My learning

Topics

Lesson 91 of 134 ยท Standard Library

Zlib, Open3 and Net::HTTP

Compress and decompress data with Zlib and gzip, run external commands and capture output with Open3, and make HTTP requests with Net::HTTP. Runnable loopback examples.

Three standard-library tools solve three very common jobs in scripts and services. Zlib compresses data so it takes less space or bandwidth. Open3 runs another program and gives you its output, errors and exit status separately and safely. Net::HTTP talks to web servers. They look unrelated, but they share a theme: each moves bytes across a boundary (memory to a smaller form, your process to a child process, your machine to another), and each is easiest to use correctly if you wrap the boundary in a small method with a clear failure story.

The runner on this site has no internet access and cannot install gems, so the examples compress in memory, run Ruby itself as the child process, and start a tiny HTTP server on the loopback address (127.0.0.1) so Net::HTTP can talk to it for real. The same code works against real servers.

Compressing with Zlib

Zlib.deflate compresses a string, Zlib.inflate restores it. Text with repetition compresses well; random data hardly at all.

Ruby deflate, inflate and checksums

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

The result of deflate is binary data (ASCII-8BIT), not text. Store or send it as bytes, never as a UTF-8 string.

gzip: the format files and HTTP use

deflate produces the raw zlib format. gzip wraps the same compression with a header (and optionally a file name and timestamp), and it is what .gz files and the Content-Encoding: gzip HTTP header use. In Ruby that is Zlib::GzipWriter and Zlib::GzipReader, and you can point them at any IO, including a StringIO.

Ruby Gzip in memory

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

Zlib.gzip and Zlib.gunzip are the one-line versions for strings in memory. For files, Zlib::GzipWriter.open("file.gz") { |gz| gz.write(...) } and Zlib::GzipReader.open("file.gz") { |gz| ... } follow the same block pattern as File.open (see Files and I/O). Always close a GzipWriter, or the trailer is never written and the file is truncated.

Never inflate untrusted input without a size limit. A tiny compressed payload can expand to gigabytes (a "zip bomb"). Read in chunks with GzipReader#read(length) and stop when a limit is exceeded.

Running programs with Open3

Ruby has several ways to start another program: backticks, system, exec, spawn and IO.popen (see Processes). Open3 is the friendliest when you need the output, because it keeps standard output and standard error apart and gives you the exit status.

Method Returns
Open3.capture2(cmd, stdin_data:) stdout, status
Open3.capture2e(cmd) stdout and stderr merged, status
Open3.capture3(cmd) stdout, stderr, status
Open3.popen3(cmd) { ... } stdin, stdout, stderr streams and a wait thread, for interaction

To keep the examples portable, the child process below is Ruby itself, found with RbConfig.ruby.

Ruby capture3 with output, errors and status

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

Pass the command as separate arguments (RUBY, "-e", script), not as one string. A single string goes through a shell, which splits on spaces and interprets characters such as ; and $; with separate arguments there is no shell, so a user-supplied filename can never turn into extra commands. That is the single most important rule when running programs.

Never build a command by interpolating untrusted text into a string: capture2("grep #{input} file") is a command-injection hole. Use the multi-argument form.

Failing safely

A missing program raises Errno::ENOENT. A program that starts and fails does not raise at all; you must check status.success? yourself. A small wrapper turns both cases into one clear error.

Ruby A safe wrapper around Open3

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

For long-running or interactive children, Open3.popen3 yields the three streams so you can write input while reading output. Child programs buffer their output when it goes to a pipe, so output from stdout and stderr can arrive out of order unless the child flushes (the merged example sets $stdout.sync = true for that reason). Remember to close stdin when you are done, or the child waits forever. If both stdout and stderr produce a lot of output, read them in separate threads (see Threads), otherwise a full pipe buffer can block the child.

Making HTTP requests with Net::HTTP

Net::HTTP is Ruby's built-in HTTP client. It is low-level but complete: GET, POST, headers, redirects (which you handle yourself), timeouts, TLS, streaming. Application code often wraps it or uses a gem, but you need to know it, because it is always there.

sequenceDiagram
  participant C as Net::HTTP client
  participant S as Server
  C->>S: GET /hello (headers)
  S-->>C: 200 OK + headers
  S-->>C: body
  C->>C: parse response, close connection

A request is a method, a path and headers; the response is a status code, headers and a body.

To run this lesson's demo, the program starts a minimal web server on a random free port of 127.0.0.1. Do not write servers like this for real work (use a proper server such as Puma, see Sinatra and Rack); it is here so the client has something real to talk to.

Ruby GET and POST against a local server

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

Key facts you just saw:

  • Net::HTTP.get_response(uri) returns a response object. .code is a string ("200"), .body is the body, response["header-name"] reads a header.
  • Net::HTTP.post(uri, body, headers) sends a POST. For forms there is Net::HTTP.post_form(uri, "key" => "value").
  • Response classes follow the status: Net::HTTPOK, Net::HTTPNotFound, and groups such as Net::HTTPSuccess and Net::HTTPRedirection. A 404 or 500 is not an exception. If you want an exception, call response.value, which raises for any non-2xx status.

A client with timeouts and error handling

Production clients need explicit timeouts and a plan for network errors. Use the instance form Net::HTTP.start, which keeps one connection open for several requests.

Ruby Start, timeouts and failures

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

Notes for real use:

  • Set open_timeout (connecting) and read_timeout (waiting for data). Defaults are 60 seconds, far too long for a web request.
  • For HTTPS pass use_ssl: true to Net::HTTP.start (or use an https:// URI with get_response).
  • Net::HTTP does not follow redirects. Check response.is_a?(Net::HTTPRedirection), read response["location"] and request again, with a limit on hops.
  • Compressed responses: send Accept-Encoding: gzip yourself and decode the body with Zlib.gunzip; Net::HTTP already does this for you if you do not set that header yourself.
  • For parsing the URL pieces and building query strings see URI and Networking; for JSON bodies see JSON.

Putting them together

A script that downloads a log, checks it, and compresses it is a few lines with these tools. The shape is always: fetch bytes (Net::HTTP), transform them (Zlib), or hand them to another program (Open3), with each boundary wrapped in a method that raises a clear error when something goes wrong.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Why must you close a Zlib::GzipWriter?

    The gzip trailer (checksum and size) is written on close. Without it readers see a truncated file.
  2. 2. What is the safest way to run a program with user-supplied arguments using Open3?

    With separate arguments Ruby starts the program directly. A single string goes through a shell, which allows injection.
  3. 3. What does Open3.capture3 return?

    capture3 returns the standard output string, the standard error string, and the status object that has exitstatus and success?.
  4. 4. A server answers a Net::HTTP request with 404. What happens?

    HTTP error statuses are normal responses. Net::HTTP raises only for network problems such as timeouts or refused connections.
  5. 5. Which two timeouts should every Net::HTTP client set?

    open_timeout limits connecting and read_timeout limits waiting for data. The 60 second defaults are too long for most callers.

Summary

  • Zlib.deflate/inflate and Zlib.gzip/gunzip compress strings; GzipWriter and GzipReader stream to IO objects, and the writer must be closed.
  • Compressed output is binary; never inflate untrusted data without a size limit.
  • Open3.capture3 returns stdout, stderr and status; pass command and arguments separately to avoid shell injection, and check status.success?.
  • Net::HTTP returns response objects for any HTTP status; set open_timeout and read_timeout, handle redirects yourself and rescue network errors.
  • A tiny TCPServer on 127.0.0.1 lets you test client code without internet access.

Next: BigDecimal, Rational and Complex