Skip to content
Tutorials Center
My learning

Topics

Lesson 23 of 37 ยท Collections and Blocks

Enumerable, Iterators and Lazy Pipelines in Crystal

Learn Crystal Enumerable and Iterator: map, select, reduce, group_by, each_slice, zip, lazy iterator chains, and how to make your own class enumerable.

Almost every collection in Crystal (arrays, hashes, ranges, sets, files, even strings through each_char) includes the Enumerable module. Implement one method called each in your own class, include Enumerable(T), and you immediately get dozens of methods: map, select, reduce, min_by, group_by and many more. This lesson tours the ones you will use daily and then shows the lazy Iterator, which processes values one at a time.

All Crystal code is read-only; the Ruby stand-ins run on the site and use the same names.

The everyday transformations

CRYSTAL
nums = [1, 2, 3, 4, 5, 6]

nums.map { |n| n * n }                 # [1, 4, 9, 16, 25, 36]
nums.select(&.even?)                   # [2, 4, 6]
nums.reject(&.even?)                   # [1, 3, 5]
nums.reduce(0) { |sum, n| sum + n }    # 21
nums.sum                               # 21
nums.count(&.odd?)                     # 3
nums.find { |n| n > 3 }                # 4, type Int32?
nums.any? { |n| n > 5 }                # true
nums.all?(&.positive?)                 # true
nums.partition(&.even?)                # {[2, 4, 6], [1, 3, 5]}
nums.min_by { |n| (n - 4).abs }        # 4
nums.sort_by { |n| -n }                # [6, 5, 4, 3, 2, 1]

find returns Int32?, so the compiler makes you handle the nil case. partition returns a tuple with two arrays, which you can destructure: evens, odds = nums.partition(&.even?). When a block is a single method call, the short form &.even? keeps it readable (see Blocks, Procs and yield).

Ruby Ruby stand-in: map, select, reduce

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

Grouping, slicing and combining

CRYSTAL
words = %w(apple avocado banana blueberry cherry)

words.group_by(&.[0])                # {'a' => [...], 'b' => [...], 'c' => [...]}
words.tally_by(&.size)               # {5 => 1, 7 => 1, 6 => 2, 9 => 1}  (counts)
words.each_slice(2).to_a             # [["apple", "avocado"], ...]
words.each_cons(2).to_a              # overlapping pairs
words.zip(1..5)                      # [{"apple", 1}, {"avocado", 2}, ...]
words.each_with_index { |w, i| puts "#{i}: #{w}" }
words.each_with_object({} of Char => Int32) { |w, h| h[w[0]] = (h[w[0]]? || 0) + 1 }
words.flat_map(&.chars.first(2))
[[1, 2], [3]].flatten                # [1, 2, 3]
words.max_by(&.size)

Notice zip returns tuples {String, Int32}, and each_slice returns an iterator (more below), which is why we add .to_a to look at the result. Use each_with_object when you need to build a collection while walking; the "memo" object is returned at the end.

Ruby Ruby stand-in: group_by, each_slice, zip

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

Iterators: laziness on demand

Methods such as each_slice(2) or each_char called without a block return an Iterator. An iterator does not compute everything up front; it produces one value each time you ask with next. That makes it possible to describe a pipeline over even an infinite sequence and pull out only what you need.

CRYSTAL
squares = (1..Int32::MAX).each.map { |n| n * n }   # nothing computed yet
squares.first(3)          # [1, 4, 9]

evens = 1.step.select(&.even?)     # 1.step is an infinite iterator
evens.first(4)                     # [2, 4, 6, 8]

iter = [10, 20].each
iter.next      # 10
iter.next      # 20
iter.next      # Iterator::Stop  (a special end marker)

# use select / map / take_while / skip / with_index / zip lazily
(1..20).each.select(&.even?).map { |n| n * 3 }.first(3)

Plain Enumerable methods like Array#map are eager: they build a full array. The iterator version, array.each.map { ... }, is lazy and returns another iterator. In Ruby this is called an Enumerator (and lazy); in Crystal it is just Iterator.

Ruby Ruby stand-in: a lazy pipeline over an endless range

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

Make your own class enumerable

Write each with yield, include Enumerable(T), and everything above becomes available. The type parameter says what each yields.

CRYSTAL
class Playlist
  include Enumerable(String)

  def initialize(@songs : Array(String))
  end

  def each(& : String ->)
    @songs.each { |song| yield song }
  end
end

list = Playlist.new(["Blue", "Red", "Green"])
list.map(&.upcase)            # ["BLUE", "RED", "GREEN"]
list.sort                     # ["Blue", "Green", "Red"]
list.includes?("Red")         # true
list.first                    # "Blue"
list.size                     # 3

To get lazy iteration as well, include Iterable(T) and write each without a block to return an iterator. For most classes plain Enumerable is enough.

Ruby Ruby stand-in: include Enumerable

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

Picking the right method

flowchart LR
  A["Each item changes"] --> M["map"]
  B["Keep some items"] --> S["select / reject"]
  C["Many to one"] --> R["reduce / sum / min / max"]
  D["Bucket by key"] --> G["group_by / tally"]
  E["Need only the first few"] --> I["iterator + first(n)"]

Choose the method by what you want out of the collection.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What do you need to write to make your class work with map, select and sort?

    Enumerable builds all its methods on top of each, which yields the elements.
  2. 2. What does [1, 2, 3].find { |n| n > 5 } return in Crystal?

    find returns the first match or nil, which is why its type is Int32?.
  3. 3. What is different about array.each.map { } compared with array.map { }?

    each without a block returns an Iterator; chaining on it does not build intermediate arrays until you ask for values.
  4. 4. Which method splits a collection into two groups by a condition?

    partition returns a tuple: the items for which the block is true, and the rest.

Summary

  • Enumerable gives map, select, reduce, group_by, partition and much more to any class with each.
  • Methods like find return a nilable value; handle the nil.
  • A method called without a block often returns an Iterator, which works lazily.
  • each_slice, each_cons, zip and each_with_object handle most "walk through in pieces" tasks.

Next: Macro Basics