Skip to content
Tutorials Center
My learning

Topics

Lesson 47 of 149 ยท Functions

Iterators and the Iteration Protocol

See how Python for loops really work - iterables vs iterators, iter() and next(), writing classes with __iter__ and __next__, and a first look at itertools.

A for loop works on lists, strings, dictionaries, files, ranges, generators and countless other objects. How can one loop handle so many different types? The answer is a small agreement called the iteration protocol. Any object that follows it can be looped over, unpacked, passed to sum() or list(), and used in a comprehension.

This lesson opens up that machinery. You'll learn the difference between an iterable and an iterator, drive iteration by hand with iter() and next(), write your own iterator classes with __iter__ and __next__, and get a preview of itertools, the standard library's toolbox for iterators. It builds directly on Generators.

Iterables and iterators

Two words sound alike but mean different things:

  • An iterable is anything you can loop over: a list, a string, a dict, a range, a file. It can hand you a fresh iterator whenever you ask for one, via iter(obj).
  • An iterator is the object that actually walks through the values. It remembers its position and gives you the next value each time you call next() on it. When there's nothing left, it raises StopIteration.

Think of a book (iterable) and a bookmark (iterator). The book can have many bookmarks at different pages; each bookmark only moves forward.

Python iter() and next()

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

A list isn't an iterator: calling next(colors) directly raises TypeError. You always go through iter() first.

What a for loop really does

A for loop is a convenient wrapper around exactly those calls. This loop:

Python
for color in colors:
    print(color)

behaves like this code:

Python A for loop by hand

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The same protocol runs behind list(x), sum(x), x in y for many types, tuple unpacking (a, b = pair), "".join(x), comprehensions and yield from. Learn it once and you understand all of them.

Iterators are iterable too

Every iterator also has an __iter__ method that returns itself. That's why you can put an iterator in a for loop. But because it returns itself and not a fresh copy, the position is shared: a second loop continues where the first stopped.

Python Shared position

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

This also explains why generators, map objects, filter objects, zip and enumerate results are single-use: they're iterators, so they can only move forward once. A list, a range or a dict is an iterable that creates a fresh iterator each time, so you can loop over it as often as you like.

Object Iterable? Iterator? Reusable?
list, tuple, str, dict, set Yes No Yes
range(10) Yes No Yes
generator, map, filter, zip, enumerate Yes Yes No
open file Yes Yes No (without seeking back)

Writing an iterator class

To make your own iterator, define a class with two special methods:

  • __iter__(self) returns the iterator, which is self.
  • __next__(self) returns the next value, or raises StopIteration when finished.

(If classes are new to you, Classes and Objects explains self and __init__. You can still follow the idea here.)

Python An iterator class

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

That works, but notice the last-but-one line: a Countdown object is single-use, just like a generator.

Reusable iterables

Usually you want the container to be reusable, like a list. The trick is to make the class an iterable whose __iter__ returns a new iterator each time. The easiest way is to write __iter__ as a generator method, so Python builds the iterator for you:

Python A reusable iterable

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

This is the pattern to reach for in real code. You get all the benefits of the iteration protocol with a few lines, and you never have to raise StopIteration yourself, because the generator does it when the method ends.

Watch out: Inside a generator, don't raise StopIteration to stop early. Since Python 3.7 that's converted into a RuntimeError. Use return instead.

The older sequence protocol

You may also see classes that support iteration by defining only __getitem__ with integer indexes starting at 0. Python falls back to calling it with 0, 1, 2โ€ฆ until an IndexError. It still works, but __iter__ is clearer and is what modern code should use.

Checking whether something is iterable

The reliable test is to try iter(obj) and catch TypeError. The collections.abc.Iterable check works for most cases, but misses classes that rely on the old __getitem__ protocol:

Python
def is_iterable(obj) -> bool:
    try:
        iter(obj)
    except TypeError:
        return False
    return True

print(is_iterable([1, 2]), is_iterable("hi"), is_iterable(42))  # True True False

Remember that strings are iterable (character by character). Functions that accept "a list of names" often need to guard against a single string being passed by mistake, or they'll process "Ana" as "A", "n", "a".

A first look at itertools

The itertools module is a set of fast building blocks that take iterators and return iterators. Because they're lazy, they combine nicely with generators and work on endless streams. Here are a few highlights; the itertools and functools lesson covers the module properly.

Python itertools preview

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

pairwise arrived in Python 3.10 and batched in 3.12. Both are available here, since this runner uses Python 3.14, but check your version if you run the code elsewhere.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Which statement about lists is correct?

    Lists are iterables that create a new iterator each time you call iter() (which a for loop does for you).
  2. 2. What signals that an iterator has no more values?

    __next__ raises StopIteration when finished; for loops catch it and end quietly.
  3. 3. Which two methods must an iterator class define?

    __iter__ returns the iterator itself and __next__ produces each value.
  4. 4. Why can you loop over a Playlist (with a generator __iter__) many times, but a Countdown iterator only once?

    An iterable that builds a fresh iterator in __iter__ is reusable; an iterator that returns self keeps a single, forward-only position.
  5. 5. What does next(iter([]), "empty") return?

    The second argument to next() is a default returned instead of raising StopIteration.

Summary

  • An iterable can produce an iterator with iter(); an iterator produces values with next() and raises StopIteration when it's done.
  • A for loop calls iter() once and next() repeatedly, stopping at StopIteration. The same protocol powers list(), sum(), unpacking and comprehensions.
  • Iterators return themselves from __iter__, so they're single-use and share their position. Lists, ranges and dicts create a fresh iterator each time.
  • An iterator class defines __iter__ (return self) and __next__ (return a value or raise StopIteration).
  • For reusable containers, write __iter__ as a generator method; inside generators, stop with return, never raise StopIteration.
  • itertools provides lazy building blocks like count, cycle, islice, chain, pairwise and batched.

Next: Modules and Imports, where you'll organize code into files and use Python's standard library.