Skip to content
Tutorials Center
My learning

Topics

Lesson 105 of 134 ยท Testing and Tooling

RubyGems and Bundler

Understand RubyGems and Bundler - what a gem is, version numbers and the ~> operator, Gemfile and Gemfile.lock, bundle install and exec, and how to structure your own gem.

Almost every real Ruby program uses code written by somebody else: a JSON parser, a web framework, a test library, a date helper. In Ruby, that code is packaged as a gem. A gem is a folder of Ruby files plus a small description (its name, its version, its dependencies) that has been zipped up and published, usually on rubygems.org. The tool that installs and loads gems is RubyGems, and it has shipped with Ruby for many years. On top of it sits Bundler, which solves a different problem: making sure that every machine running your project uses exactly the same gem versions.

This lesson explains both tools from the ground up. One practical note about this page: our runner has no network, so gem install and bundle install cannot run here. Anything that needs the internet is shown as a read-only example and labelled as such. The good news is that RubyGems itself, and the Gem::Version and Gem::Requirement classes that do all the version arithmetic, are part of Ruby and run fine, so you can experiment with the rules that matter most.

What a gem is

Think of a gem as a library with a name tag and a version. When you write require 'json', Ruby searches its load path for a file called json.rb. RubyGems adds the directories of installed gems to that load path on demand, so the same require line works whether the code came with Ruby or from a gem you installed.

Ruby 4.0 ships with a long list of gems already installed. Some of them (like json, date, set and minitest) you use all the time. Let us look at what this very runner has.

Ruby Gems that come with this Ruby

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

Gem::Specification is the registry of installed gems. Each entry (a gemspec) knows its name, version, summary, dependencies, and the files it contains. Asking for a gem that is not there raises Gem::MissingSpecError, which is a subclass of Gem::LoadError. Remember that error name; you will meet its cousin LoadError whenever you forget a dependency.

Since Ruby 3.4 and 4.0, several libraries that used to be part of the standard library moved into bundled gems: csv, base64, bigdecimal, observer, ostruct, drb, mutex_m and a few more. They are installed with Ruby, so a plain script can still require 'csv' (as the examples on this site do). But once your project uses a Gemfile, Bundler only lets you load gems that are listed in it, so a real project must add, for example, gem 'csv' to its Gemfile. If you ever see a warning that a library "will no longer be part of the default gems", that is what it means.

Version numbers

Gem versions look like 3.2.1: segments separated by dots. RubyGems recommends semantic versioning: MAJOR.MINOR.PATCH. A patch release fixes bugs, a minor release adds features without breaking anything, and a major release may break compatibility. Not every gem follows this perfectly, but it is the convention that the ~> operator below relies on.

Versions are not strings. "3.10" sorts before "3.2" as text, but version 3.10 is newer than 3.2. Gem::Version compares segment by segment, as numbers.

Ruby Comparing versions correctly

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

A version with a letter in it, like 2.0.0.rc1, is a prerelease and sorts before the release it is heading towards. That is why 2.0.0.rc1 came first in the sorted list. Bundler never picks a prerelease unless you ask for it.

Requirements and the pessimistic operator

When your project depends on a gem, you do not usually say "version 5.2.1 exactly". You say which range of versions you accept. This is a requirement, and it is built from an operator and a version: ">= 1.0", "< 3", "= 2.4.0", "!= 1.5". The most important operator is ~>, called the pessimistic or "twiddle-wakka" operator.

~> 3.2 means "at least 3.2, but below 4.0". ~> 3.2.1 means "at least 3.2.1, but below 3.3". In both cases it allows the last segment you wrote to grow and keeps everything before it fixed. This is the operator that expresses the semantic-versioning promise: ~> 3.2 trusts that no 3.x release will break you.

Ruby What does ~> allow?

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

Notice that ">= 3.2" accepts 4.0.0, which may break your code. Open-ended >= requirements are a trap for libraries; prefer ~>.

Using gems from a plain script

Without Bundler, you can install a gem from the command line and require it. This is a read-only example because it needs the network:

Terminal
# read-only example: needs the internet, not run on this page
gem install colorize         # download and install the newest version
gem install colorize -v 0.8.1
gem list                     # what is installed
gem uninstall colorize

In the script you just require 'colorize'. You can also pin a version with gem 'colorize', '~> 0.8' before the require. Calling gem activates a specific version, and raises Gem::MissingSpecVersionError if none matches.

What happens when the gem is missing altogether? You get a LoadError. This failing example is exactly the message you will see the first time you forget to install something:

Ruby Requiring a gem that is not installed (fails on purpose)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

The real message is cannot load such file -- colorize_not_installed (LoadError). The line above it printed because the first require succeeded. The fix is to install the gem, or, in a project, to add it to the Gemfile.

Why Bundler exists

Imagine your app works on your laptop with gem A version 2.3 and gem B version 1.8. A teammate runs gem install a month later and gets A 2.4, which changed a method you use. Or B needs A below 2.4, and now the two cannot be installed together. Dependencies of dependencies make this worse: your project might use 80 gems that require each other in complicated ways.

Bundler solves this with two files:

  • Gemfile: you write it. It lists the gems your project needs, with loose requirements.
  • Gemfile.lock: Bundler writes it. It records the exact version of every gem, including indirect ones, that satisfied the Gemfile.

Here is a typical Gemfile. It is a normal Ruby file, evaluated by a small DSL (a domain-specific language: see DSLs):

Ruby
# read-only example: a Gemfile in your project root
source "https://rubygems.org"

ruby "~> 4.0"

gem "sinatra", "~> 4.0"
gem "csv"                      # a bundled gem since Ruby 3.4: list it explicitly
gem "puma", ">= 6.0"

group :development, :test do
  gem "minitest", "~> 5.25"
  gem "rake"
end

group :development do
  gem "rubocop", require: false  # installed, but not auto-required
end

And the lockfile that bundle install generates from it (shortened):

TEXT
GEM
  remote: https://rubygems.org/
  specs:
    mustermann (3.0.3)
      ruby2_keywords (~> 0.0.1)
    puma (6.6.0)
      nio4r (~> 2.0)
    sinatra (4.1.1)
      mustermann (~> 3.0)
      rack (>= 3.0.0, < 4)
    ...

PLATFORMS
  x86_64-linux

DEPENDENCIES
  puma (>= 6.0)
  sinatra (~> 4.0)

BUNDLED WITH
   2.6.3

You commit both files to version control. The Gemfile says what you want; the lockfile says what you got. When a teammate or a server runs bundle install, Bundler reads the lockfile and installs exactly those versions. Everyone runs the same code.

The Bundler commands you need

Terminal
# read-only example: shell commands, not run here
bundle install            # resolve the Gemfile, write Gemfile.lock, install the gems
bundle exec rake test     # run a command with only the locked gems available
bundle update             # re-resolve to newer versions allowed by the Gemfile
bundle update puma        # update just one gem (and what it needs)
bundle outdated           # list gems that have newer versions
bundle add rack           # add a gem line to the Gemfile and install it
bundle init               # create a starter Gemfile

bundle exec is the one beginners forget. It starts your command in an environment where only the locked gems exist, so you never accidentally run a different version that happens to be installed globally. In your own code you can get the same effect by putting require 'bundler/setup' at the top of your entry script, or Bundler.require to load every gem in the Gemfile at once (Rails does this).

bundle install only changes the lockfile when the Gemfile can no longer be satisfied by it. Day to day it just installs what the lockfile already says. Reach for bundle update deliberately, in its own commit, so a failing test points at one upgrade and not at everything at once.

How a resolver thinks (a stand-in you can run)

The heart of Bundler is a dependency resolver: given your requirements and a list of available versions for each gem, find one set of versions that satisfies everyone. The real one is sophisticated, but the central idea fits in a few lines. This stand-in uses a made-up gem index and picks, for each gem, the highest version that matches every requirement placed on it.

Ruby A tiny dependency resolver

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

The first two runs show that choosing a different version of web changes which router is selected. The third run shows a conflict: web 2 needs router 2.x and logger_plus 1.0 needs router below 2. Real Bundler backtracks and tries older versions before giving up, and when it cannot find a solution it prints a message that traces the conflict, "Bundler could not find compatible versions for gem ...". The cure is the same as here: loosen a requirement, update a gem, or drop one.

Gem anatomy and building your own

A gem has a conventional layout:

TEXT
my_gem/
  my_gem.gemspec       # name, version, authors, dependencies
  lib/
    my_gem.rb          # the file `require "my_gem"` loads
    my_gem/version.rb  # VERSION = "0.1.0"
  test/ or spec/
  Rakefile
  README.md

bundle gem my_gem generates this skeleton. The .gemspec is plain Ruby that builds a Gem::Specification; you can create one in memory to see what it holds:

Ruby A gemspec built in memory

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

Publishing is two commands: gem build my_gem.gemspec (makes my_gem-0.1.0.gem) and gem push my_gem-0.1.0.gem (uploads it, and needs a rubygems.org account). Two good habits: add runtime dependencies with add_dependency and tools only you need with add_development_dependency, and bump the version in lib/my_gem/version.rb following semantic versioning.

Pitfalls professionals know

  • Forgetting to commit Gemfile.lock for applications. Apps commit it; libraries traditionally do not, because a library must work with whatever versions its users have.
  • Using sudo gem install. Installing into the system Ruby needs administrator rights and pollutes it. Use a version manager (see Ruby versions and how Ruby runs) so gems install into your home directory.
  • Running rake instead of bundle exec rake. It may load a different version of a gem than the one locked.
  • Unbounded requirements such as gem "foo", ">= 1". When foo 3 arrives and breaks you, nobody will have warned you. Prefer ~>.
  • Trusting any gem blindly. A gem runs with all your program's powers. Check downloads, maintenance, and the source before adding a dependency, and keep the list short.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Which version is the newest?

    Versions compare segment by segment as numbers, so 2.10.0 beats 2.9.9. As plain strings, "2.9.9" would wrongly sort higher.
  2. 2. What does the requirement ~> 3.2.1 accept?

    The pessimistic operator lets the last written segment grow. With three segments, that is the patch level, so the upper bound is 3.3.
  3. 3. Which file records the exact versions of every gem, including indirect dependencies?

    Bundler writes Gemfile.lock when it resolves the Gemfile. You commit it so that every machine installs the same versions.
  4. 4. Why should you usually run bundle exec rake test rather than rake test?

    bundle exec builds an environment from the lockfile, so a different globally installed version cannot be loaded by accident.
  5. 5. What error does require 'not_a_real_gem' raise?

    Ruby raises LoadError with the message "cannot load such file -- not_a_real_gem" when no file by that name is on the load path.

Summary

  • A gem is a packaged library with a name and a version; RubyGems installs and loads gems, and require finds them through the load path.
  • Gem::Version compares versions as numbers, and prerelease versions like 2.0.0.rc1 sort before their release.
  • Requirements combine an operator with a version; ~> allows the last written segment to grow and is the safe default.
  • A Gemfile states what you want, Gemfile.lock records what you got, and bundle exec runs commands against the locked set.
  • Some libraries (csv, base64, bigdecimal, ostruct, ...) are bundled gems and need a Gemfile entry in a real project.
  • A gemspec is plain Ruby that builds a Gem::Specification; bundle gem creates the skeleton of a new gem.

Next: Rake and Automation