Skip to content
Tutorials Center
My learning

Topics

Lesson 40 of 43 ยท Projects and Practice

Project: A Hopfield Network That Remembers

Build a Hopfield network in Python that stores patterns in connection weights and recalls them from noisy cues, then test its capacity and discuss what it says about memory.

Try to recall a song from one fragment of its melody, or a friend's face from a blurry photo. Human memory is content-addressable: part of a memory is enough to retrieve the rest. A computer's usual memory works differently: you give it an address, and it returns whatever is stored there.

In 1982 physicist John Hopfield showed that a simple network of binary units, with symmetric connections and a Hebbian learning rule, behaves like a content-addressable memory. The network settles from a noisy starting state into the nearest stored pattern, much like a ball rolling downhill into a valley. This project builds one, tests it, and asks what it does and does not tell us about human memory.

Four steps: store, corrupt, settle, recall.
How a Hopfield memory recalls a pattern from a corrupted cue.

What you will learn

  • How Hebbian learning stores patterns in weights.
  • How asynchronous updates make the network settle.
  • Why the network has limited capacity, and roughly how large it is.
  • What the model captures about memory, and what it leaves out.

Key terms

Hopfield network: a fully connected network of binary units with symmetric weights.

Hebbian learning: "cells that fire together wire together": the weight between two units grows when they are active together.

Attractor: a stable state that nearby states are drawn into.

Energy function: a number that never increases as the network updates, so the network must settle.

Spurious state: a stable state that is not one of the stored patterns.

The model

Each unit is +1 or -1. Patterns are lists of such values. To store patterns, set the weight between units i and j to

w[i][j] = (1 / N) * sum over patterns of (pattern[i] * pattern[j]), with w[i][i] = 0.

To recall, start from a cue and repeatedly update units one at a time in random order: unit i becomes +1 if the weighted sum of the other units' states is positive and -1 otherwise. Hopfield proved that with symmetric weights each update can only lower (or keep) an energy E = -1/2 * sum of w[i][j] * s[i] * s[j], so the process must stop at a local minimum, an attractor.

Step 1: Store and recall letters

Python Recall letters from corrupted cues

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice what the energies show: the recalled state sits at a lower energy than the corrupted cue. The memory is not retrieved by looking it up. It emerges from the dynamics.

Step 2: How many memories can it hold?

Each added pattern makes the others harder to recall, because the weights are a sum of all patterns. Amit, Gutfreund and Sompolinsky (1985) used statistical physics to show that a network of N units can reliably store about 0.14 N random patterns. Beyond that, recall collapses. This experiment measures it.

Python Memory capacity experiment

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Step 3: Interpret the results

What does the model teach us?

  • Content-addressable retrieval emerges from simple local rules, with no explicit address or search.
  • Pattern completion and noise tolerance match how people recall from partial cues.
  • Capacity is limited, and failing memories can blend or converge on spurious states, loosely reminiscent of false memories.
  • Distributed storage: no weight holds a pattern, and damage to a few weights causes gradual decline.

What does it leave out? Human memory is not static. It reconstructs, forgets over time, depends on emotion and context and involves distinct systems such as the hippocampus and cortex. The Hopfield network treats all patterns as equal and independent. Modern research extends the idea in several ways. For example, "modern Hopfield networks" have much higher capacity and are related to the attention mechanism of transformers.

Common misconceptions

"The Hopfield network proves how the hippocampus works." It is a simple, influential model of one aspect of memory, not a full description of any brain region.

"The network stores the pattern in one unit." The pattern is spread over all the weights.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What does it mean for a memory to be content-addressable?

    A partial or noisy cue is enough to retrieve the full pattern.
  2. 2. Which rule sets Hopfield weights?

    Weights are sums of products of the stored patterns' unit values.
  3. 3. Why does a Hopfield network settle?

    With symmetric weights and asynchronous updates, energy never rises, so the state reaches a local minimum.
  4. 4. Roughly how many random patterns can a network of N units store reliably?

    Amit, Gutfreund and Sompolinsky derived a capacity of about 0.14 N.
  5. 5. A spurious state is:

    Overloaded networks can settle into blends or other unintended attractors.

Summary

  • A Hopfield network stores patterns in symmetric Hebbian weights and recalls them by settling from a cue.
  • An energy function guarantees settling into an attractor.
  • Capacity is limited to roughly 0.14 N random patterns.
  • The model captures pattern completion and distributed storage but omits much of real memory.

Next: Glossary