Skip to content
Tutorials Center
My learning

Topics

Lesson 53 of 149 ยท Modules and the Standard Library

collections

Learn Python's collections module. Count things with Counter, group with defaultdict, build fast queues with deque, and use namedtuple and ChainMap well.

Lists, tuples, sets and dictionaries handle most jobs, but some patterns come up so often that the standard library includes specialized containers for them. Counting things, grouping items under keys, keeping the last few values, giving tuple fields names: the collections module has a ready-made, well-tested tool for each.

Every container in this lesson builds on a type you already know, so it behaves the way you'd expect, with extra powers on top. Brush up on dictionaries and tuples first if they feel shaky.

Counter: counting made easy

Counting how often things appear is one of the most common tasks in programming. With a plain dictionary, you'd check whether each key exists before adding one. A Counter does the bookkeeping for you:

Python Counting with Counter

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

A Counter is a subclass of dict, so tally.items(), tally.keys() and for item in tally all work as usual. Its extras make it shine:

  • most_common(n) returns the n most frequent items, highest first.
  • Missing keys return 0 instead of raising KeyError.
  • update() adds more counts; subtract() removes them.
  • total() (Python 3.10+) adds up all the counts.

Counter arithmetic

Counters can be combined like multisets, which is handy for inventories and comparisons:

Python Combining counters

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

defaultdict: dictionaries that fill themselves

A defaultdict takes a factory: a function it calls to create a value whenever you access a missing key. It's perfect for grouping:

Python Grouping with defaultdict

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Common factories are list (group items), set (group unique items), int (count or sum) and dict (nest). You can pass any zero-argument function, including a lambda: defaultdict(lambda: "unknown").

Even reading a missing key creates it. by_genre["horror"] adds an empty list under "horror". To check without side effects, use "horror" in by_genre or by_genre.get("horror"), which don't call the factory.

How does this compare to the plain-dict options? dict.setdefault(key, []).append(x) does the same job without an import, and dict.get(key, 0) + 1 works for counting. defaultdict is cleaner when you do it many times; Counter is better still for pure counting.

deque: a fast double-ended queue

A list is great at adding and removing at its end, but list.insert(0, x) and list.pop(0) must shift every other element, which gets slow for big lists. A deque (pronounced "deck") is fast at both ends:

Python deque basics

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Use a deque whenever you need a queue (first in, first out), such as a task list or a breadth-first search. For a stack (last in, first out), a plain list with append() and pop() is perfect.

Keeping only the last N items

Give a deque a maxlen and it becomes a sliding window: once full, adding to one end silently drops an item from the other. That's ideal for "recent history" features and moving averages:

Python Sliding window with maxlen

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

namedtuple: tuples with field names

A tuple like (52.52, 13.40) is compact, but which number is the latitude? namedtuple creates a small class whose instances are still tuples, but whose fields also have names:

Python namedtuple

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Like all tuples, named tuples are immutable and light on memory. _replace() returns a changed copy and _asdict() converts to a dictionary; the leading underscore just avoids clashes with your field names.

The typing module has a class-based version, typing.NamedTuple, that adds type hints and default values. If you need mutable fields, methods or more control, reach for data classes instead.

ChainMap: layered lookups

A ChainMap groups several dictionaries into one view. Lookups search them in order and return the first match. That fits configuration perfectly: command-line options override user settings, which override defaults:

Python Layered settings with ChainMap

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

No data is copied: if you change defaults later, settings sees it. ChainMap is less common than the others, but when you need layers, it's exactly the right tool.

OrderedDict, briefly

Before Python 3.7, regular dictionaries didn't keep insertion order, so OrderedDict was essential. Today plain dicts keep order, and you'll mostly see OrderedDict in older code. It still has two niche extras: move_to_end(key) and equality checks that care about order. Those make it a neat base for a tiny least-recently-used cache, though functools.lru_cache usually does that job better.

Converting back to plain types

Because these containers are subclasses of (or behave like) the built-in types, converting them back is easy when you need to: dict(counter), dict(default_dict), list(a_deque) and named._asdict() all return ordinary objects. That's useful before printing for users or saving data as JSON, which only knows about plain dictionaries and lists.

There's one subtle difference to watch for when you pass a defaultdict to other code: any function that reads a missing key creates it as a side effect. If that matters, convert it with dict() first, so the receiving code gets a normal KeyError instead of silently growing your data.

Performance-wise, all of these are implemented in C and are as fast as or faster than equivalent hand-written code. Choosing them is never a trade-off between clarity and speed.

Choosing the right container

You need to... Use
Count occurrences Counter
Group items under keys defaultdict(list)
Add and remove at both ends, or keep the last N items deque (with maxlen)
Give tuple positions readable names namedtuple or typing.NamedTuple
Look up keys across layered dictionaries ChainMap

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What does Counter("banana")["z"] return?

    A Counter returns 0 for missing items instead of raising KeyError.
  2. 2. Which code collects student names under their class, creating the list automatically?

    defaultdict(list) calls list() for a missing key, so .append() always has a list to work on. A plain dict raises KeyError.
  3. 3. You create deque(maxlen=3) and append 1, 2, 3, 4. What does it hold?

    When a bounded deque is full, appending to the right drops the oldest item from the left.
  4. 4. Why use a deque instead of a list for a queue that removes items from the front?

    deque.popleft() takes constant time; list.pop(0) gets slower as the list grows.
  5. 5. With settings = ChainMap(user, defaults), where does settings["color"] = "red" store the value?

    Lookups search every mapping, but writes and deletions only affect the first one.

Summary

  • Counter counts items, returns 0 for missing keys, offers most_common() and supports +, -, &, |.
  • defaultdict(factory) creates missing values automatically; list, set and int are the usual factories.
  • deque is fast at both ends; maxlen turns it into a sliding window of recent items.
  • namedtuple gives tuple fields names while staying immutable and lightweight.
  • ChainMap searches several dictionaries in order, ideal for layered settings.

Next: itertools and functools, two modules full of building blocks for working with iterables and functions.