Skip to content
Tutorials Center
My learning

Topics

Lesson 31 of 50 ยท Knowledge and Language

Understanding Language

Follow speech from sound to meaning: categorical perception, the McGurk effect, lexical access, ambiguity, garden path sentences, N400 and situation models.

When someone speaks to you, a stream of air pressure changes becomes a thought in your head in a fraction of a second. You do it without effort, in noisy rooms, across accents, while the speaker talks at three words a second. Yet the signal does not come with spaces between words, and many words and sentences are ambiguous. Explaining how listeners cope is one of the central achievements, and puzzles, of cognitive psychology.

Four cards joined by arrows: sounds, words, sentences, meaning
Four levels of comprehension. They overlap in time: later levels send feedback to earlier ones.

What you will learn

  • Why speech perception is hard and what categorical perception and the McGurk effect show.
  • How listeners recognize words and resolve ambiguous words.
  • What garden path sentences reveal about parsing.
  • How brain responses such as the N400 and situation models figure in comprehension.

Key terms

Phoneme: the smallest sound unit that can distinguish words in a language, such as /b/ versus /p/.

Lexicon: the mental store of words and their properties.

Parsing: assigning a grammatical structure to a sentence.

Garden path sentence: a sentence that leads the reader to a wrong parse that must be revised.

Situation model: a mental representation of the situation a text describes.

Level 1: from sound to phoneme

Speech has no reliable gaps between words, and the same phoneme sounds different depending on its neighbors and on who is speaking (the lack of invariance problem). Listeners nevertheless hear clear categories. Alvin Liberman and colleagues (1957) varied a continuous acoustic cue, voice onset time (VOT): the delay between releasing the lips and the start of vocal-cord vibration. A short delay gives /ba/ and a longer one gives /pa/. Listeners did not hear a gradual change; they heard /ba/ up to a boundary and /pa/ beyond it, and tended to discriminate pairs better when the two sounds fell on opposite sides of the boundary. This is categorical perception. It is not unique to speech or to humans, and modern work finds that within-category differences are still detected, but the effect shows how perception is tuned to the contrasts a language uses.

Vision joins in. In the McGurk effect (McGurk and MacDonald, 1976), a video of a person saying "ga" paired with the sound "ba" is often heard as "da". Hearing is influenced by watching lips, even though the listener knows the sound is "ba". Speech perception combines sources of information, and context fills in gaps: in the phoneme restoration effect (Warren, 1970), listeners hear a sentence in which one sound is replaced by a cough and report hearing the missing sound as if it had been present.

Level 2: recognizing words

Skilled listeners know tens of thousands of words and recognize each in a few hundred milliseconds, often before the word is complete. William Marslen-Wilson's cohort model (1987) proposes that the first sounds activate all words that begin that way (the cohort), and as more sounds arrive the candidates are eliminated until a single word remains: the uniqueness point. Words are recognized, on average, shortly after that point. Frequent words are recognized faster (the word frequency effect), and context speeds recognition. In reading, eye-tracking shows that readers fixate longer on rare words.

Ambiguity. Many words have several meanings ("bank", "bug", "bat"). In a cross-modal priming study, David Swinney (1979) had people listen to sentences containing an ambiguous word, such as "bug", in a context favoring one meaning (spies searching a room) and then presented a related word on a screen. Immediately after the ambiguous word, related words for both meanings (ant and spy) were primed, but a short time later only the meaning that fit the context remained active. This supports the view that access to word meaning is initially exhaustive, and context quickly selects the right meaning, although other studies show that a strong context and a dominant meaning can change the picture.

Level 3: sentences and parsing

To understand "The dog chased the cat", the listener must find who did what to whom. Parsing is the process of assigning structure, and it happens incrementally, word by word, without waiting for the sentence to finish. The evidence is the garden path sentence. Compare:

  • "The horse raced past the barn fell." (Bever, 1970)
  • "The old man the boats."
  • "The professor told the student she was confident would pass."

Readers commit to the most likely structure, find it fails at the final word, and must go back. In "The horse raced past the barn fell", "raced" is read as the main verb, and "fell" then has nowhere to go, although the sentence is grammatical ("the horse that was raced past the barn fell").

Two families of theory explain how readers choose. Syntax-first accounts, such as Lyn Frazier's garden path model, propose that the parser builds the simplest structure ("minimal attachment") using only word categories at first, and uses meaning later to repair it. Constraint-based accounts (MacDonald, Pearlmutter and Seidenberg, 1994) say that many sources of information, including word frequency, meaning, context and visual scene, compete from the start. Eye-tracking in scenes supports the second view: Tanenhaus and colleagues (1995) found that listeners' eye movements to objects reflect how a sentence is interpreted as it unfolds, and that the visual context can prevent a garden path. Most researchers now accept that multiple cues act early, though they disagree about how.

Level 4: meaning and situation models

Understanding a text goes beyond parsing. Readers make inferences to connect sentences ("She dropped the vase. It shattered." requires a link between the vase and the shattering), and they build a situation model, a mental representation of the events, people, goals and places described, not of the exact words (Walter Kintsch, Rolf Zwaan). In a classic demonstration by Bransford, Barclay and Franks (1972), people who read sentences about a situation later falsely "recognized" sentences that were consistent with the described situation but never presented. They had stored the meaning, not the wording. Readers also track the protagonist's location, goals and time, and are slowed by shifts.

Pragmatics is the last layer: "Can you pass the salt?" is a request, not a question about ability. Listeners infer the speaker's intention from context and from norms of cooperation (H. Paul Grice), which is why sarcasm and indirect speech are normally understood so quickly.

Brain signals

The brain tracks these processes within milliseconds. Marta Kutas and Steven Hillyard (1980) recorded EEG while people read sentences ending in a surprising word ("He spread the warm bread with socks"). The surprising final word produced a large negative wave peaking about 400 ms after the word, the N400, which indexes difficulty integrating a word's meaning with its context. Grammatical violations tend to produce a different later positive wave, the P600. The N400 is now one of the main tools for studying meaning in real time.

Common misconceptions

"We hear words as separate units." Speech is continuous, and words are recovered by the listener.

"Understanding a sentence is decoding the grammar." Meaning, context and world knowledge act from the start.

"People remember the exact words they read." After a short time most readers remember the gist, not the wording.

Try it

Python Categorical perception of voice onset time (a model)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

JS The cohort narrows as the word unfolds

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Categorical perception means that

    The /ba/-/pa/ continuum is perceived as two categories separated by a boundary.
  2. 2. In the McGurk effect, an auditory 'ba' with a visual 'ga' is often heard as

    Vision and hearing combine in speech perception, and the percept can differ from both.
  3. 3. In the cohort model, the uniqueness point is where

    At that point the cohort is reduced to a single candidate.
  4. 4. A garden path sentence is one in which

    Incremental parsing commits to a likely structure, which later words may contradict.
  5. 5. The N400 is larger for

    Kutas and Hillyard found a larger N400 for the anomalous ending 'socks' than for sensible endings.

Summary

  • Speech perception turns a continuous signal into categories, combining hearing, vision and context.
  • Word recognition begins before a word ends; ambiguous words briefly activate several meanings.
  • Parsing is incremental, and garden path sentences show that readers commit to likely structures; many cues act early.
  • Comprehension builds situation models of what is described, and inference and pragmatics fill in the rest.
  • Brain signals such as the N400 track meaning in real time.

Next: Producing Language and Thinking in It