Skip to content
Tutorials Center
My learning

Topics

Lesson 84 of 134 ยท Standard Library

URI and Networking Basics

Learn Ruby's URI library (parse, join, build, query strings, encoding) and networking basics - Net::HTTP requests, a local TCPServer demo, timeouts and the exceptions that network code raises.

Almost every program eventually talks to something else: it reads a web address, builds a link with a search term in it, downloads JSON from an API, or listens on a port. Ruby ships with the tools for all of this. URI takes web addresses apart and puts them together safely, Net::HTTP makes HTTP requests, and Socket, TCPServer and TCPSocket give you raw network connections underneath.

This lesson teaches URI fully, because you can run it anywhere, and then shows networking in the only way our runner allows: no outside network is available here, so we start a tiny server on 127.0.0.1 (your own machine) inside the same script and talk to it. Everything you learn transfers directly to real servers. For the data formats that travel over these connections see JSON, and for string building see String Methods.

Parsing a URI

A URI (Uniform Resource Identifier, a URL being the common kind) has a standard shape: scheme://userinfo@host:port/path?query#fragment. URI.parse (or the shorter URI(...)) splits one into named parts. The library comes with Ruby; load it with require 'uri'.

Ruby The parts of a URI

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

URI("https://...") returns a URI::HTTPS, and http gives URI::HTTP; both are subclasses of URI::Generic, which is also what you get for a relative reference such as /just/a/path. Missing pieces are nil (no userinfo in http://example.com) except where the scheme has a default: the port comes from the scheme (80 for HTTP, 443 for HTTPS) and the path of an empty address is "".

Bad input raises InvalidURIError

A space or another illegal character is not allowed in a URI, and the parser refuses it instead of guessing.

Ruby URI::InvalidURIError (fails on purpose)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

The run ends with the error bad URI (is not URI?): "http://example.com/my file.txt" (URI::InvalidURIError). The first backtrace lines point into the library files (uri/rfc3986_parser.rb); your own code is the line from main.rb:3:in '<main>'. Notice that the message only says the text is not a URI; it does not say which character is wrong. The fix is to encode the unsafe characters (see below) before you parse. When the URI comes from users, rescue it:

Ruby Rescuing bad addresses

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

Parsing succeeding is not the same as having a usable web address. URI.parse("ftp://example.com/file") works but is not HTTP, and a relative path parses fine but has no host. Check the class and the host as well, as the helper does.

Building and changing URIs

You can combine addresses, change one part, or build one from pieces.

Ruby join, setters and build

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

URI.join(base, relative) resolves a link the way a browser does, including ... Mind the trailing slash: https://example.com/docs/guide (a "file" called guide) joined with intro.html gives https://example.com/docs/intro.html, while .../guide/ (a "directory") gives .../guide/intro.html. Setters such as query=, path= and port= change the object in place. Note that changing the scheme does not change the port or the class: a URI::HTTPS object with its scheme set to http is still a URI::HTTPS object with port 443. Build a new URI instead.

normalize lowercases the scheme and host, so two spellings of the same address compare equal.

Query strings and encoding

The query string (?q=ruby+lang&page=2) is a list of key=value pairs where special characters must be written as %XX escapes. Never glue values in with string interpolation; let the library do the escaping.

Ruby encode_www_form and decode_www_form

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

encode_www_form takes a hash (or an array of pairs for repeated keys), turns spaces into +, and escapes &, = and non-ASCII characters as UTF-8 percent sequences. decode_www_form returns an array of pairs, because a key can repeat; converting to a hash with to_h keeps only the last one. encode_www_form_component and encode_uri_component (Ruby 3.2+) are for single values: the first writes a space as +, the second as %20, which is what you want inside a path. The hand-built version in the last lines breaks the query: the & in Tom & Jerry would start a new parameter.

Do not use URI.encode, URI.escape or URI.unescape. They were removed years ago (Ruby 3.0), and the fix is always one of the encode_* functions above.

HTTP with Net::HTTP

Net::HTTP is the HTTP client in Ruby's standard library. Real-world use looks like this. It is read-only here because the runner has no internet access:

Ruby
# read-only example: needs a network connection
require 'net/http'
require 'json'

uri = URI("https://api.example.com/v1/users?limit=5")

response = Net::HTTP.get_response(uri)
case response
when Net::HTTPSuccess
  users = JSON.parse(response.body, symbolize_names: true)
  puts users.size
when Net::HTTPRedirection
  puts "moved to #{response['location']}"
else
  warn "request failed: #{response.code} #{response.message}"
end

Net::HTTP.get_response(uri) returns a response object whose class tells you the outcome family: Net::HTTPSuccess for 2xx, Net::HTTPRedirection for 3xx, Net::HTTPClientError for 4xx, and so on, so case with when classes (see case and when) reads well. Use Net::HTTP.get(uri) when you only want the body string, and note that 404 is not an exception: it is a normal response that you must check.

To see all of this work without internet, the next example starts a minimal HTTP server on a free port of this machine, in a thread, and talks to it with the same Net::HTTP code you would use for a real server. The server is hand-written to keep the lesson dependency-free; in real projects use a proper server (see Sinatra and Rack).

Ruby Net::HTTP against a local server

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

What the server saw is the proof: GET /greet?name=ada HTTP/1.1 is the request_uri from the previous section on the request line, and the POST carried the X-Token header and the JSON body. Net::HTTP.start(host, port) { |http| ... } opens one connection, runs your block and closes the connection afterwards, which is the right shape when you send several requests to one host. Set http.use_ssl = true for HTTPS (the https URI helpers do this for you with Net::HTTP.get_response), and set timeouts: by default, opening a connection waits up to 60 seconds and so does reading.

Forms and defaults

Ruby Building requests without sending them

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

Request objects are plain data until you hand them to http.request, which makes them easy to inspect and test. Even a bare GET carries default headers (accept, user-agent, accept-encoding). set_form_data encodes the hash as application/x-www-form-urlencoded and fills in the content type.

Network errors

Network code fails in ways ordinary code does not, and each failure is a specific exception. Our sandbox is a good place to see two of them.

Ruby Connection refused and no network

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

The usual suspects: Errno::ECONNREFUSED (nothing listens there), Errno::ECONNRESET (the other side hung up), Socket::ResolutionError (a subclass of SocketError, raised when the host name cannot be resolved), Net::OpenTimeout and Net::ReadTimeout (both Timeout::Errors), and OpenSSL::SSL::SSLError for certificate problems. A robust client rescues these specific classes, retries transient ones a limited number of times (see Exceptions for retry), and gives up with a clear message. Never write rescue Exception.

A raw TCP conversation

HTTP is just text over TCP. Below Net::HTTP sits TCPServer, which listens, and TCPSocket, which connects. This echo server upper-cases whatever it receives, line by line.

Ruby TCPServer and TCPSocket

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

accept blocks until a client connects and returns a socket for that one conversation; gets returns nil when the peer closes, which ends the loop. A server that handles several clients accepts in a loop and starts a thread for each connection (see Threads). Always close sockets, ideally with the block forms (TCPSocket.open(...) { |s| ... }) or ensure.

When to use what

  • Use URI for every address you build or parse. Never concatenate query strings.
  • Use Net::HTTP for simple requests in scripts and tools. For larger clients, gems such as Faraday or HTTP.rb (read-only in this tutorial, not available here) add retries, middleware and nicer APIs, but they are built on the same ideas.
  • Use plain sockets only for custom protocols or for learning; reach for a web server (Rack-based, see Sinatra and Rack) before writing HTTP by hand.
  • Always set timeouts, handle non-2xx responses, and treat everything coming from the network as untrusted input.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What does URI.parse("http://example.com/my file.txt") do?

    The parser refuses illegal characters such as a space. Encode the unsafe characters first (or reject the input).
  2. 2. Which call builds a correct query string for a value that may contain &, spaces or accents?

    encode_www_form escapes every special character. Interpolating a raw value lets an & or # change the meaning of the address.
  3. 3. What does Net::HTTP.get_response do when the server answers with status 404?

    HTTP error statuses are normal responses. Only connection problems and timeouts are exceptions, so check the class of the response, for example with Net::HTTPSuccess.
  4. 4. Why does URI.join("https://example.com/docs/guide", "intro.html") give /docs/intro.html?

    Relative references are resolved against the directory of the base, like in a browser. Without a trailing slash the last segment is replaced.
  5. 5. Which exception tells you that nothing is listening on the port you connected to?

    ECONNREFUSED is a SystemCallError raised when the connection attempt is rejected, usually because no server is listening.

Summary

  • URI.parse or URI(...) splits an address into scheme, host, port, path, query and fragment; bad input raises URI::InvalidURIError, and a successful parse does not guarantee an HTTP address.
  • URI.join resolves relative links; pay attention to the trailing slash. Setting the scheme does not change the class or port.
  • Build and read query strings with URI.encode_www_form and URI.decode_www_form, never by hand; URI.encode and URI.escape no longer exist.
  • Net::HTTP.get_response, Net::HTTP::Post and Net::HTTP.start make requests; non-2xx statuses are responses, not exceptions, and you should set timeouts.
  • Network failures are specific exceptions: Errno::ECONNREFUSED, Socket::ResolutionError, Net::OpenTimeout, Net::ReadTimeout.
  • TCPServer and TCPSocket are the raw layer; 127.0.0.1 with port 0 is the safe way to experiment.

Next: OptionParser and Logger