Skip to content
Tutorials Center
My learning

Topics

Lesson 10 of 50 ยท Methods and Cognitive Architecture

Computational Models and Cognitive Architectures

Learn what a computational model of cognition is, compare symbolic, connectionist and Bayesian approaches, and run tiny working examples of each in your browser.

Verbal theories are slippery. "Attention acts as a filter" sounds clear, but what exactly passes the filter, how strongly and when? A computational model forces precision: the theory is written as a program whose behavior can be run and compared with data. If the model's predictions are wrong, the theory is in trouble. If it makes a surprising prediction that turns out to be right, the theory earns credibility. A model can also reveal that two theories said to be different make the same predictions, or that an idea that sounded plausible cannot actually work.

Symbolic, connectionist and Bayesian model families
Three families of computational models, each making a different bet about what cognition is made of.

What you will learn

  • What a computational model is and what it is for.
  • The main ideas of symbolic, connectionist and Bayesian models.
  • What a cognitive architecture is.
  • How to judge a model: fit, prediction, generalization and simplicity.

Key terms

Computational model: a precise, runnable version of a theory that produces predictions.

Production rule: an if-then rule: if the current situation matches a condition, then perform an action.

Connectionist (neural network) model: a model made of simple connected units whose connection strengths (weights) are learned.

Bayesian model: a model in which beliefs are probabilities updated by evidence using Bayes' rule.

Cognitive architecture: a general model of the structure of the mind that is meant to explain many tasks with the same fixed components.

Overfitting: matching noise in the data, so the model fits the old data well but predicts new data poorly.

Why build models?

  1. Precision. Writing the theory as code exposes hidden assumptions.
  2. Prediction. You can run the model on new conditions before running people.
  3. Exploration. Models show how a simple mechanism can produce a complex pattern.
  4. Comparison. Two models can be tested head to head on the same data.

A model is not the theory's truth. It is a tool, and a model that is flexible enough to fit any possible data is not testable, so it is not informative.

Family 1: symbolic models and cognitive architectures

Symbolic models assume that the mind manipulates explicit symbols by rules, like a program. The best-known example is a production system: a store of facts (working memory) and a set of if-then rules. Rules match the facts, one fires, the facts change and the cycle repeats.

  • Soar, developed by Allen Newell and colleagues, grew out of the work with Herbert Simon described in the history lesson. It aims to be a unified theory of cognition.
  • ACT-R (Adaptive Control of Thought-Rational), developed by John R. Anderson and colleagues, combines production rules with a long-term store of "chunks" whose accessibility depends on how recently and often they were used. It includes modules for vision, motor control, goals and memory, and it makes quantitative predictions about response times and errors. ACT-R models have been used to explain performance in tasks such as arithmetic, learning to program and driving.

A cognitive architecture like these is built from a fixed set of mechanisms. The aim is that explaining a new task means supplying only the knowledge (the rules and chunks) for the task, not changing the machinery. This is demanding: the same architecture must explain many tasks.

Family 2: connectionist models

Connectionist models, also called parallel distributed processing (PDP) models after the 1986 volumes of David Rumelhart, James McClelland and colleagues, are inspired by (but are far simpler than) networks of neurons. A network has:

  • Units that each have an activation level;
  • Connections with weights between units;
  • A learning rule that changes the weights from experience. The 1986 paper of Rumelhart, Geoffrey Hinton and Ronald Williams popularized backpropagation, which adjusts weights to reduce the error between the network's output and a target.

Key properties:

  • Knowledge is distributed: a concept is a pattern across many units, not a single symbol.
  • Learning is gradual and shows graded behavior. Damaging some units degrades performance gradually, similar to how brain damage can be partial.
  • The network learns from examples and can generalize to new inputs.

A well-known and still-debated example is the model of learning English past tenses by Rumelhart and McClelland (1986), which showed U-shaped development like children (they first say "went", then "goed", then "went" again) without any explicit rules. Steven Pinker and Alan Prince criticized the model in 1988, arguing that rules are needed for regular forms. Later models addressed some of the points. The debate shows how a model can sharpen a theoretical dispute.

Modern deep learning systems are large descendants of these networks. They can perform impressive tasks, but good performance does not by itself make them good models of human cognition: a theory of the mind must also explain human errors, learning speed and limits.

Family 3: Bayesian models

Bayesian models ask: given what the evidence is, and given what the person already believes, what is the most reasonable conclusion? Bayes' rule updates a belief:

posterior โˆ prior ร— likelihood

If you see a blurry animal at dusk, your prior (dogs are more common in your street than wolves) and the likelihood (how well the shape fits a dog or a wolf) combine to a posterior belief. Bayesian models have explained how people perceive motion and depth, judge how likely events are, learn word meanings from few examples and predict what happens next. They work at the computational level of Marr: they say what an ideal reasoner would do, not necessarily how the brain computes it. A key question is how close people come to the ideal, and where they systematically depart from it.

How to judge a model

Criterion Question
Fit Does it reproduce the data?
Prediction Does it predict new data that it was not tuned on?
Generalization Does it work across tasks and people?
Parsimony Does it use as few free parameters as possible?
Falsifiability Is there a result that would count against it?
Plausibility Is it consistent with brain and behavioral evidence?

A model with many free parameters can fit anything, including noise. This is overfitting. A fair comparison therefore penalizes complexity or tests the model on held-out data.

Common misconceptions

"If a model fits the data, the theory is true." Many different models can fit the same data. The test is whether the model predicts data it was not fitted to.

"Neural networks are accurate models of the brain." Most connectionist models use idealized units and learning rules. They capture some general principles, not neural detail.

"Bayesian models show people are optimal." They specify what an ideal observer would do. Whether people match it is an empirical question, and often they do not.

Try it

JS A tiny production system (symbolic)

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Python A tiny connectionist model: learning by error (delta rule)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Python Bayes' rule: prior x likelihood = posterior

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. A production rule has the form

    Production rules fire an action when their condition matches the current contents of working memory.
  2. 2. In connectionist models, knowledge is mainly stored in

    Learning changes weights, and a concept is a distributed pattern of activation.
  3. 3. Which description fits a Bayesian model?

    Bayes' rule multiplies prior belief by the likelihood of the data to get a posterior belief.
  4. 4. Overfitting means

    A flexible model can match random fluctuations and so fails on new data.
  5. 5. Why is a model that can fit any pattern of data not useful as a theory?

    A theory must be falsifiable: it should rule out some possible data.

Summary

  • Computational models make theories precise enough to predict and to fail.
  • Symbolic models and architectures such as Soar and ACT-R use rules and structured knowledge.
  • Connectionist models learn weights across distributed units and generalize from examples.
  • Bayesian models describe how evidence should update beliefs.
  • Judge models by fit, prediction on new data, parsimony and falsifiability; beware overfitting.

Next: Chapter 2 Review