Skip to content
Tutorials Center
My learning

Topics

Lesson 36 of 37 ยท Projects and Practice

Project: A Worker Pool with Channels in Crystal

Build a concurrent worker pool in Crystal with fibers and channels: jobs, results, graceful shutdown, errors, timeouts and a statistics owner fiber, with Ruby models.

A worker pool is the standard shape of concurrent programs: a stream of jobs goes in, a fixed number of workers process them in parallel, and results come out. It limits how much work runs at once, so ten thousand URLs do not mean ten thousand simultaneous connections. In this project you assemble the ideas from Channels and select, Mutex and Parallelism into one reusable piece.

The Crystal code is read-only here. The Ruby versions use threads and queues to model the same structure, and print only values that do not depend on thread timing (results are sorted before printing).

The design

flowchart LR
  G["Producer"] --> J(("jobs"))
  J --> W1["Worker 1"]
  J --> W2["Worker 2"]
  J --> W3["Worker 3"]
  W1 --> R(("results"))
  W2 --> R
  W3 --> R
  R --> C["Collector (main)"]

Jobs fan out to workers; results fan in to the collector.

Rules for the design:

  1. One fiber sends jobs and then closes the jobs channel.
  2. Each worker loops on jobs.receive? and ends when the channel closes.
  3. Something must know when all workers are done so it can close the results channel; a WaitGroup or a counter does that.
  4. The collector reads results until it closes.

Step 1: a basic pool

CRYSTAL
record Job, id : Int32, n : Int32
record Result, job_id : Int32, value : Int64, worker : Int32

def fib(n : Int32) : Int64
  n < 2 ? n.to_i64 : fib(n - 1) + fib(n - 2)
end

def run_pool(inputs : Array(Int32), workers : Int32) : Array(Result)
  jobs    = Channel(Job).new(inputs.size)
  results = Channel(Result).new(inputs.size)
  done    = Channel(Nil).new

  workers.times do |worker_id|
    spawn do
      while job = jobs.receive?
        results.send(Result.new(job.id, fib(job.n), worker_id))
      end
      done.send(nil)                 # I am finished
    end
  end

  inputs.each_with_index { |n, i| jobs.send(Job.new(i, n)) }
  jobs.close                         # no more jobs: workers will stop

  workers.times { done.receive }     # wait for every worker
  results.close

  all = [] of Result
  while result = results.receive?
    all << result
  end
  all.sort_by(&.job_id)
end

run_pool([30, 10, 25, 5, 20], 3).each do |r|
  puts "job #{r.job_id}: fib = #{r.value} (worker #{r.worker})"
end

The channels are buffered to the number of inputs so neither the producer nor the workers block on a full channel. The done channel is a simple way to wait: each worker sends one value when its loop ends, and the main fiber receives exactly workers values. Sorting by job_id makes the output independent of which worker finished first. Which worker handled which job can change between runs, so never test for it.

Ruby Ruby stand-in: the same pool with threads

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

Step 2: errors must not kill workers

If a job raises, an unhandled exception ends that fiber, and the pool silently loses a worker. Catch errors inside the loop and report them as results:

CRYSTAL
record Outcome, job_id : Int32, value : Int64?, error : String?

spawn do
  while job = jobs.receive?
    begin
      value = risky(job.n)
      results.send(Outcome.new(job.id, value, nil))
    rescue ex
      results.send(Outcome.new(job.id, nil, ex.message))
    end
  end
  done.send(nil)
end

Now one bad input shows up as a failed outcome while the other jobs continue. The collector prints failures at the end or retries them. Retrying belongs in a bounded loop (for example, three attempts with a growing sleep), never an endless one.

Ruby Ruby stand-in: errors become results

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

Step 3: timeouts per job

A hung job must not block a worker forever. Run the job in its own fiber and use select with a timeout (see select, Mutex and Parallelism):

CRYSTAL
def with_timeout(limit : Time::Span, &block : -> Int64) : Int64?
  reply = Channel(Int64).new(1)
  spawn { reply.send(block.call) }
  select
  when value = reply.receive
    value
  when timeout(limit)
    nil           # the job fiber keeps running; its result is dropped
  end
end

The abandoned fiber is not killed (Crystal has no forced cancellation), so the job itself should still be written to finish in bounded time, for example by using read timeouts on sockets (socket.read_timeout = 5.seconds).

Step 4: one fiber owns the statistics

Counting finished and failed jobs from many workers invites races. Let the collector own the counters, which makes locks unnecessary:

CRYSTAL
processed = 0
failed = 0
total = 0_i64

while outcome = results.receive?
  if outcome.error
    failed += 1
  else
    processed += 1
    total += outcome.value.not_nil!
  end
end

puts "ok=#{processed} failed=#{failed} sum=#{total}"

Only the main fiber touches these variables, so everything is safe even with -Dpreview_mt.

Ruby Ruby stand-in: statistics owned by the collector

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

Step 5: graceful shutdown

A long-running service should stop cleanly on Ctrl+C: finish the jobs in progress, accept no new ones, and then exit.

CRYSTAL
stop = Channel(Nil).new

Signal::INT.trap do
  stop.close              # closing wakes every receiver at once
end

spawn do
  loop do
    select
    when stop.receive?
      break
    when job = jobs.receive?
      process(job) if job
    end
  end
end

Closing a channel is a broadcast: all fibers waiting on stop.receive? wake up together, which is more convenient than sending one value per worker.

Tuning the pool

  • Size. For I/O work (HTTP calls) use many workers (dozens); for CPU work you need -Dpreview_mt and about one worker per core.
  • Buffers. A small buffer gives back-pressure; a big one smooths bursts.
  • Ordering. Results arrive in completion order; carry an id (as Job#id here) to restore the original order.
  • Measuring. Time the whole run with Time.measure { ... } and compare worker counts before guessing.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Who should close the jobs channel?

    Only the producer knows there are no more jobs; closing makes each worker's receive? loop end.
  2. 2. Why must errors be caught inside the worker loop?

    Catch the error, send a failed outcome and continue with the next job.
  3. 3. Why does the example sort results by job_id?

    Workers finish in unpredictable order, so an id restores the original order.
  4. 4. What is the benefit of letting only the collector update counters?

    Ownership by one fiber removes the race.

Summary

  • A pool is jobs in, workers in the middle, results out, connected by channels.
  • The producer closes jobs; workers end when receive? returns nil; something closes results after all workers finish.
  • Catch exceptions per job; add timeouts with select.
  • Let one fiber own shared statistics.
  • Closing a channel is a broadcast, useful for shutdown.

Next: Practice: Crystal Exercises with Solutions