Skip to content
Tutorials Center
My learning

Topics

Lesson 33 of 43 ยท Artificial Intelligence and Cognition

Learning Machines and Brains

Compare deep networks with the visual cortex, reinforcement learning with dopamine reward prediction errors, and see what language models do and do not show about minds.

For decades, AI and brain science borrowed ideas from each other more quietly than people realize. A mathematical idea from neuroscience, the weighted unit, became the artificial neuron. A design idea from the visual cortex, building complex features from simple ones, became the convolutional network. A learning rule from animal psychology, reward-driven trial and error, became reinforcement learning, which in turn helped neuroscientists understand dopamine.

This lesson looks at three cases of the exchange and ends with the most debated one: large language models, which have prompted new arguments about what it takes to have a mind.

Four stages shared in spirit by deep networks and the ventral stream.
Both a deep network and the visual system build complexity in stages.

What you will learn

  • Why convolutional networks echo the ventral visual stream, and where the analogy ends.
  • How temporal-difference learning explains dopamine responses.
  • What language models do, and what remains open about what they show.
  • Some clear differences between machine and human learning.

Key terms

Convolutional neural network (CNN): a network whose early layers apply the same small filter at many image positions.

Reinforcement learning (RL): learning actions from rewards by trial and error.

Reward prediction error: the difference between the reward you got and the reward you expected.

Language model: a system trained to predict the next piece of text, called a token.

Transformer: the network architecture introduced in 2017 that uses attention to relate all parts of the input.

Deep networks and the visual system

Hubel and Wiesel's edge detectors (chapter 6) inspired Kunihiko Fukushima's neocognitron in 1980 and then Yann LeCun's convolutional networks, which read handwritten digits in the late 1980s and 1990s. In 2012 a CNN called AlexNet, from Alex Krizhevsky, Ilya Sutskever and Geoffrey Hinton, won the ImageNet recognition competition by a wide margin, and deep learning took off.

The surprising part came later. In 2014 Daniel Yamins, James DiCarlo and colleagues found that CNNs optimized to recognize objects developed internal representations that predicted neural responses in monkey inferior temporal cortex, at the top of the ventral stream, better than earlier models did. Later, benchmarks such as Brain-Score compared many models with brain recordings. Roughly, early network layers behave like early visual areas, and later layers like later ones.

But the similarity is partial.

  • Networks are often fooled by adversarial examples: tiny, carefully designed changes to an image that people cannot see yet flip the network's answer.
  • Many networks rely more on texture than on shape, unlike people.
  • Brains have strong feedback connections and process continuous time, while standard CNNs are feedforward.
  • Brains learn from few examples and few labels, while networks typically need large labeled sets.

Good fit to brain data tells you a model captures some of what the brain computes, not that it works the same way.

The reinforcement learning loop.
Observe, act, receive reward and update predictions from the error.

Reinforcement learning and dopamine

An RL agent faces a world where good and bad outcomes arrive late. How does it credit earlier actions? Richard Sutton's temporal-difference (TD) learning updates a value estimate using the difference between successive predictions:

error = reward + value(next state) - value(current state)

Then value(current state) += learning_rate * error.

In the 1990s Wolfram Schultz recorded dopamine neurons in monkeys. They fired to unexpected juice rewards. After the monkey learned that a light predicted the juice, the neurons stopped firing to the juice and fired to the light instead. If the expected reward failed to come, firing dipped below baseline. In 1997 Schultz, Peter Dayan and Read Montague showed that these responses match a TD error. Dopamine looks like a teaching signal for reward prediction error, one of neuroscience's clearest successes in using an AI algorithm to explain data. Later work shows dopamine also codes more than simple reward, so the story is richer than it first appeared.

Large language models

A language model assigns probabilities to the next token. Modern large language models, built on the transformer architecture of Ashish Vaswani and colleagues (2017) and trained on huge text collections, produce fluent essays, code and explanations. What does this tell us about minds? Researchers disagree, and the field moves fast, so treat the following as a map of the questions rather than a verdict.

  • A case for taking them seriously as models. Next-word prediction is, as psycholinguists have long argued, something humans do too: we anticipate upcoming words. Language-model surprise predicts human reading times and some brain responses. Models trained only on text acquire surprising syntactic knowledge.
  • A case for caution. Emily Bender, Timnit Gebru and colleagues argued in 2021 that models can produce fluent text without grounding it in the world, a "stochastic parrot" worry that echoes Searle and the symbol grounding problem. Models can fail on simple reasoning tasks, make up facts and are sensitive to wording. Results on theory-of-mind-style tests are mixed and depend on how the test is posed.
  • A data gap. Language models see far more text than any child, by several orders of magnitude, yet children acquire language while also interacting with people and the world. That difference shapes debates about innate structure and about whether models are good models of acquisition.
  • Other open points. Networks suffer catastrophic forgetting when trained on new tasks (McCloskey and Cohen, 1989) unless special methods are used, while human memory is more graceful.

None of this settles whether such systems understand, but it moves the debate from armchair thought experiments to experiments that can be run.

Common misconceptions

"Deep networks work just like the brain." They share some principles, but differ in learning methods, data needs, feedback and robustness.

"Language models are only autocomplete, so they tell us nothing." Prediction is a powerful training objective that yields rich internal structure, which is a real scientific puzzle even if you doubt they understand.

Try it

Python Dopamine as a prediction error

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Python From scores to probabilities: choosing the next token

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What did Yamins, DiCarlo and colleagues (2014) report?

    Networks trained for recognition developed internal representations that matched monkey inferior temporal cortex responses well.
  2. 2. What does a temporal-difference error compare?

    It is reward plus next-state value minus current-state value.
  3. 3. After learning that a light predicts juice, dopamine neurons in Schultz's experiments:

    The prediction error moves from the reward to the earlier predictive cue.
  4. 4. An adversarial example is:

    Such changes are often invisible to people yet fool the network.
  5. 5. Which statement about language models is the most defensible?

    They are scientifically interesting, and whether they understand is contested.

Summary

  • Deep CNNs echo the staged organization of the ventral stream, with real but limited similarities.
  • Temporal-difference learning gives a precise account of dopamine as a reward prediction error.
  • Language models are a new puzzle: powerful predictors whose relation to understanding is debated.
  • Machine and human learning differ in data needs, feedback, robustness and grounding.

Next: Chapter review