Skip to content
Tutorials Center
My learning

Topics

Lesson 27 of 37 ยท Functional Programming

Scala For-Comprehensions Explained

Understand Scala for-comprehensions: loops, yield, guards, nested generators, how they desugar into map, flatMap and withFilter, and use with Option, Either and Future.

Chaining several flatMap calls works, but the code soon drifts to the right and gets hard to read. Scala has a notation for exactly this situation: the for-comprehension. Despite its name it is not just a loop. It is syntax for sequencing operations on any type that has map, flatMap and (optionally) withFilter: lists, options, eithers, futures and many more.

Basic forms

Without yield, a for is a loop that performs side effects:

SCALA
for i <- 1 to 3 do println(i)       // prints 1, 2, 3 on separate lines

With yield, it builds a new collection from the results:

SCALA
val squares = for i <- 1 to 5 yield i * i
println(squares)   // Vector(1, 4, 9, 16, 25)

val names = List("ann", "bo", "cy")
val shouted = for n <- names yield n.toUpperCase
println(shouted)   // List(ANN, BO, CY)

The result has the same kind of collection as the first generator: a Range yields a Vector, a List yields a List.

Guards and several generators

A line starting with if is a guard. Several generators (<-) nest, like nested loops:

SCALA
val evens = for n <- 1 to 10 if n % 2 == 0 yield n
println(evens)   // Vector(2, 4, 6, 8, 10)

val pairs =
  for
    x <- 1 to 3
    y <- 1 to 3
    if x < y
  yield (x, y)

println(pairs)   // Vector((1,2), (1,3), (2,3))

You can use braces or the indented form shown above; with several lines the indentation style is the clearest in Scala 3. You can also define intermediate values with =:

SCALA
val result =
  for
    word <- List("apple", "fig", "banana")
    len = word.length
    if len > 3
  yield s"$word has $len letters"
// List(apple has 5 letters, banana has 6 letters)

Parts of a for-comprehension

Generator x <- source pulls each value.
Guard if cond skips values.
Definition y = expr names an intermediate value.
yield Builds the final result.
1 / 2

What the compiler does: desugaring

A for-comprehension is rewritten by the compiler into method calls:

SCALA
// This for...
for
  x <- xs
  y <- ys
yield x + y

// ...is the same as:
xs.flatMap(x => ys.map(y => x + y))

The rule is simple: every generator except the last becomes a flatMap; the last becomes map (with yield) or foreach (without it); a guard becomes withFilter. Because the rewrite only needs those method names, any type that defines them can be used in a for-comprehension, including your own.

For-comprehensions with Option

Remember the manager lookup from Option, Either and Try written with flatMap? Here it is again, readable:

SCALA
case class User(id: Int, name: String, managerId: Option[Int])
val db = Map(
  1 -> User(1, "Ann", Some(2)),
  2 -> User(2, "Bo", Some(3)),
  3 -> User(3, "Cy", None))

def bossOfBoss(id: Int): Option[String] =
  for
    user <- db.get(id)
    boss <- user.managerId.flatMap(db.get)
    bigBoss <- boss.managerId.flatMap(db.get)
  yield bigBoss.name

println(bossOfBoss(1))   // Some(Cy)
println(bossOfBoss(2))   // None
println(bossOfBoss(99))  // None

Each <- extracts the value from an Option; if any line is None, the whole comprehension is None and the following lines never run. Think of it as straight-line code with a built-in early exit.

With Either: fail-fast validation

SCALA
def parseInt(s: String): Either[String, Int] =
  s.toIntOption.toRight(s"'$s' is not a number")

def positive(n: Int): Either[String, Int] =
  Either.cond(n > 0, n, s"$n is not positive")

def area(w: String, h: String): Either[String, Int] =
  for
    width <- parseInt(w)
    height <- parseInt(h)
    _ <- positive(width)
    _ <- positive(height)
  yield width * height

println(area("4", "5"))    // Right(20)
println(area("4", "x"))    // Left('x' is not a number)
println(area("-2", "5"))   // Left(-2 is not positive)

The first Left stops the process and becomes the result. The _ <- form runs a check and ignores its value.

With Futures

The same syntax sequences asynchronous work. Each line waits for the previous result without blocking a thread:

SCALA
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global

def fetchUser(id: Int): Future[String] = Future(s"user$id")
def fetchOrders(user: String): Future[List[Int]] = Future(List(1, 2, 3))

val total: Future[Int] =
  for
    user <- fetchUser(7)
    orders <- fetchOrders(user)
  yield orders.sum

Futures are covered in Futures and Promises. One caution: two independent futures created inside the comprehension run one after the other; start them above it to run in parallel.

For-comprehensions on your own types

If you define map and flatMap, your type works in a for. A tiny "writer" that records log lines:

SCALA
case class Logged[A](value: A, log: List[String]):
  def map[B](f: A => B): Logged[B] = Logged(f(value), log)
  def flatMap[B](f: A => Logged[B]): Logged[B] =
    val next = f(value)
    Logged(next.value, log ++ next.log)

def addOne(n: Int): Logged[Int] = Logged(n + 1, List(s"added one to $n"))
def double(n: Int): Logged[Int] = Logged(n * 2, List(s"doubled $n"))

val out =
  for
    a <- addOne(5)
    b <- double(a)
  yield b

println(out)   // Logged(12,List(added one to 5, doubled 6))

This is the idea behind monads, a pattern for sequencing computations with context (absence, errors, logging, asynchrony). You do not need the theory to use it: if a type has well-behaved map and flatMap, a for-comprehension chains it.

Seeing it on the JVM

The Java and Kotlin examples below are not Scala. Java has no for-comprehension, so you see the equivalent chain of flatMap calls on streams and Optional, which is what Scala code desugars to.

Java Java: the flatMap chain behind a for-comprehension (Java, not Scala)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

Kotlin Kotlin: nested iteration with sequences (Kotlin, not Scala)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with readln()), it gets the next line from this box.

Common mistakes

  • Mixing collection types. for x <- List(1); y <- Option(2) yield ... combines a List and an Option in awkward ways. Keep the same container in one comprehension.
  • Forgetting yield. Without it the comprehension returns Unit and the values are discarded.
  • Starting futures inside the comprehension and losing parallelism.
  • Expecting an Either to collect all errors. It stops at the first Left.
  • Using println for debugging inside a long comprehension and being surprised by repeated output for nested generators.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which method does for x <- xs yield f(x) desugar to?

    A single generator with yield becomes map. With more generators, the earlier ones become flatMap.
  2. 2. What does for a <- Some(1); b <- None: Option[Int] yield a + b return?

    As soon as one step is None, the whole for-comprehension on Option is None.
  3. 3. Which is required for a custom type to be used in a for-comprehension with yield?

    The compiler only rewrites the syntax into map/flatMap (and withFilter for guards) calls.
  4. 4. How do you introduce a guard in a for-comprehension?

    A line beginning with if after a generator is a guard and desugars to withFilter.

Summary

  • A for-comprehension sequences operations on any type with map/flatMap; yield builds the result.
  • Generators use <-, guards use if, intermediate values use =.
  • The compiler rewrites it into flatMap, map, withFilter and foreach.
  • It makes chains over Option, Either, Try and Future read like straight-line code with automatic short-circuiting.

Next: Type Classes and Givens