Skip to content
Tutorials Center
My learning

Topics

Lesson 126 of 134 ยท Projects

Project: A Threaded Job Runner

Build a threaded job runner in Ruby with a Queue, worker threads, per-job errors, retries and timeouts, then extend it into a small concurrent crawler.

Many programs have a long list of independent tasks: download pages, resize images, call an API for each customer. Doing them one after another wastes time whenever a task mostly waits (for the network, a disk, a sleep). Ruby threads are a good fit for exactly this kind of work. In this project we build a reusable job runner: a pool of worker threads that take jobs from a queue, collect results, survive errors, retry flaky jobs and give up on slow ones. Then we reuse the same ideas for a small crawler that discovers new jobs while it runs.

You should know the basics from Threads, Mutex, Queue and ConditionVariable and Thread Pools and Parallel Work. The sandbox has no network, so our "websites" are in-memory hashes and slow work is simulated with sleep. A sleep behaves like real input/output waiting: while one thread sleeps, others run.

Ruby's global VM lock (GVL) lets only one thread run Ruby code at a time. Threads therefore speed up programs that wait on input/output, but not programs that only calculate. For calculation-heavy parallelism see Parallelism and Ractors.

Step 1: why a pool helps

Start with eight jobs that each wait 50 milliseconds. One thread needs about 0.4 seconds in total; four workers sharing the queue need about a quarter of that. We print only whether the speedup happened, never the raw times, since times differ on every run.

Ruby Step 1: serial versus four workers

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

The queue is filled first and then closed, so each worker's while (n = queue.pop) loop ends by itself when the queue runs dry (pop returns nil on a closed, empty queue). Results arrive in whatever order the workers finish, so we sort before comparing. That rule applies throughout the project: never let the output depend on thread timing.

Step 2: a Result type and a reusable runner

Our first draft ties the pool to squaring numbers. A reusable runner takes a list of jobs and a block that does the work, and returns one result per job. Each result must say whether the job worked or failed, so we model it with Data.

Ruby Step 2: a JobRunner class

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

Notice the division of labour. The queue is thread-safe by itself. The results array is not, so every append happens inside lock.synchronize. Finally sort_by on the job id gives a stable order. The block passed to run is the actual work; work.call(job) runs it inside the worker thread.

Step 3: what happens when a job fails

So far we assumed every job succeeds. Unfortunately, an exception inside a thread does not crash the program immediately. It ends that thread, and the exception is stored. It comes back to life when somebody calls join or value on the thread.

Ruby An exception travels through join (fails on purpose)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

The thread died quietly, so alive? is false and status is nil (a thread that ends by an exception reports nil; a normal finish reports false). The error appears only at worker.join, where the program stops with main.rb:4:in 'block in <main>': bad input (ArgumentError), followed by a copy of the offending source line (Ruby 4 adds such excerpts to some error messages). In our runner this would be a disaster: one bad job would kill a worker, its remaining jobs would never be processed (or worse, the main thread would crash during join). By default Ruby also prints a report when a thread dies; we switched it off here with Thread.report_on_exception = false to keep the output short.

The fix is simple: each job runs inside its own begin/rescue, and a failure becomes a result like any other. The worker thread itself never raises.

Step 4: errors, retries and timeouts

Real jobs fail in different ways, and a runner should treat them differently:

  • A permanent failure (bad input) should be reported at once.
  • A flaky failure (network hiccup) is worth a retry.
  • A job that is too slow should be abandoned with Timeout.timeout, which raises Timeout::Error inside the job after the given number of seconds.
Ruby Step 4: rescue, retry and timeout

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

The result list tells the story of each job. beta raised ArgumentError, a permanent problem, so it was reported after one attempt. gamma failed twice with IOError and succeeded on the third attempt, because retry jumps back to the start of the begin block and the retry limit was 2. delta failed every time and was reported after three attempts. epsilon slept for a second but the limit was 0.2 seconds, so Timeout.timeout interrupted it.

Two design decisions deserve a comment. First, the rescue clauses name specific classes. We deliberately do not catch Exception or even every StandardError: an unexpected NoMethodError is a bug in the program, and hiding it in a result would make it hard to find. A real runner would add a final rescue StandardError => e clause that records the error class and message, but it would also log it. Second, the tries counter is a local variable inside attempt, so each job has its own and the threads share nothing.

Timeout.timeout works by raising an exception in the thread from the outside, at an arbitrary point. That is acceptable for pure waiting or calculation, but it can interrupt code in the middle of cleanup. Prefer timeouts that the library itself supports (socket and HTTP read timeouts, Queue#pop(timeout:)) when they exist.

Step 5: a crawler that finds new jobs

A fixed list of jobs is easy because the queue can be closed at the start. A crawler is harder: fetching one page reveals more pages to fetch, so the queue grows while the workers run. Two new problems appear.

  1. Duplicates. Pages link to each other in cycles, so we need a shared "already seen" set, protected by a mutex.
  2. When are we done? An empty queue does not mean the end: another worker may be fetching a page that is about to add new links. The right question is "how many jobs are queued or running?". We keep a pending counter: add one for every job queued, subtract one when a job is finished, and close the queue when the counter reaches zero.

Our "website" is a hash from path to the list of links on that page. It raises errors for missing pages, like a real server would.

Ruby The complete concurrent crawler

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

The numbers add up: eight distinct addresses were seen, six pages were fetched successfully and two ended in errors (/missing is a 404 and /broken a 500), so every address was fetched exactly once even though pages link to each other in circles. Run it as often as you like with any number of workers: the report is always the same, because it is built from sorted hashes and not from arrival order.

Study the ordering carefully, because this is where concurrency bugs hide. The pending counter goes up before the new links are pushed and down in ensure after the page is processed. If we pushed first and counted afterwards, a fast worker could finish the new job and bring the counter to zero while the first worker was still adding to it. The ensure clause guarantees that a job is counted off even when fetch raises; forget it and one failing page would make the crawler wait forever, because the counter would never reach zero.

Choosing the number of workers

More workers help only up to a point. For waiting-dominated work (network calls) the limit is usually the remote server, not your CPU: fifty threads hammering one host may get you blocked. For calculation work on CRuby, extra threads add nothing because of the GVL. Start with a small number such as 4 to 8, make it a parameter, and measure.

Ideas to extend it

  • Add a progress counter shared through a Mutex, or print results as they arrive from a result queue consumed by the main thread.
  • Limit the crawl depth, or stay inside one host with URI.parse.
  • Replace Timeout.timeout by per-fetch time limits inside Site#fetch.
  • Use SizedQueue to limit memory when jobs are large (see Mutex, Queue and ConditionVariable).
  • Add exponential back-off between retries: sleep(0.1 * 2**tries).
  • Compare with fibers and the scheduler for the same task.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What happens to an exception raised inside a thread that nobody joins?

    A thread that raises ends; the exception is stored and re-raised in the thread that calls join or value. That is why workers rescue errors per job.
  2. 2. Why does the job runner sort the results before returning them?

    Completion order depends on scheduling and timing. Sorting by a stable key (the job id) makes the output reproducible.
  3. 3. In the crawler, why is the pending decrement placed in an ensure clause?

    The crawler is finished when the counter hits zero. A job that fails still has to be counted as done.
  4. 4. Which kind of work benefits from threads on CRuby (with the GVL)?

    A thread that waits releases the GVL, so others can run. Pure calculation takes turns instead of running in parallel.
  5. 5. What does retry do inside a rescue clause?

    retry re-runs the begin block. Always bound it with a counter, as the runner does with tries <= @retries.

Summary

  • A pool of worker threads reading from a closed Queue is the simplest way to run many independent jobs; the workers stop when pop returns nil.
  • Share nothing mutable without a Mutex. Give each worker local state and collect results under a lock.
  • An exception ends the thread silently and re-appears at join or value; rescue errors per job and turn them into result values.
  • Use specific rescue clauses, retry with a counter for flaky failures, and Timeout.timeout for jobs that may hang (preferring library-level timeouts where available).
  • Sort or key results so output never depends on thread timing.
  • For work that creates more work, track a pending counter (up before pushing, down in ensure) and close the queue when it reaches zero.

Next: Project: A JSON Parser