Skip to content
Tutorials Center
My learning

Topics

Lesson 20 of 50 ยท Perception and Pattern Recognition

Object and Face Recognition

Compare template, feature and structural theories of object recognition, learn about geons and viewpoint problems, and see why faces are processed holistically.

You recognize a chair whether it is wooden or plastic, seen from above or from the side, half hidden behind a table, or drawn as a cartoon. A computer program that handled one photograph of a chair could easily fail on the next. Humans do this with astonishing speed: studies show that people can identify the category of a picture seen for only a fraction of a second. Pattern recognition is the problem of matching variable sensory input to stored knowledge. This lesson compares the main theories and then looks at the most important special case: faces.

Four cards: templates, features, geons and holistic faces
Four theories of recognition, each solving a problem left by the previous one.

What you will learn

  • Template, feature and structural theories of object recognition, and the problems each solves.
  • What geons are and what Biederman's theory predicts.
  • The viewpoint problem and the view-based alternatives.
  • Key effects in face recognition and the debate over whether faces are special.
  • What visual agnosia and prosopagnosia reveal.

Key terms

Template: a stored exact pattern that is compared with the input.

Feature: a simple component such as a line, curve or angle.

Geon: one of a small set of basic 3D shapes (cylinder, cone, wedge and so on) proposed as the building blocks of objects.

Viewpoint invariance: recognizing an object regardless of the angle from which it is seen.

Holistic processing: processing a face as an integrated whole rather than as separate parts.

Theory 1: template matching

The simplest idea is that we store a copy (template) of every pattern we know and compare the input with each one. Barcode and some letter-reading machines work like this. As a theory of human recognition it fails: a letter A can appear in countless sizes, positions, fonts and orientations, and we would need an enormous number of templates, plus a way to normalize each input first. People also recognize new instances they have never seen. Templates still have a role in recognition at a more flexible level, but not as exact copies.

Theory 2: feature analysis

Feature theories say patterns are made up of simpler features, and recognition means detecting the features and matching their combination to stored descriptions. Oliver Selfridge's Pandemonium model (1959) pictured layers of "demons": image demons, feature demons (each shouting more when its feature is found), cognitive demons (one per letter, shouting in proportion to matching features) and a decision demon choosing the loudest. Biological support came from David Hubel and Torsten Wiesel, who recorded from cat and monkey visual cortex in the 1950s and 1960s and found cells tuned to oriented edges and bars, with more complex cells combining them. Experiments on confusions between letters also support features: letters sharing features (such as E and F) are confused more than dissimilar letters.

Feature theories solve the problem of size and position changes. The weakness: a list of features is not enough, since the arrangement matters (the features of L and T partly overlap, and the same parts can make different objects).

Theory 3: structural descriptions and geons

Irving Biederman (1987) proposed recognition by components (RBC). The key ideas:

  1. Objects are described as a set of simple 3D volumes called geons ("geometric ions"), about 36 in total according to the theory, such as cylinders, cones, bricks and curved versions. Different arrangements of a few geons make a huge number of objects: a cup is a cylinder with a curved handle attached to the side, a bucket the same cylinder with the handle on top.
  2. Geons are identified by non-accidental properties of the 2D image, such as straight versus curved edges, parallel lines and symmetry, which are the same from nearly any viewpoint. A straight edge in the image is almost always a straight edge in the world.
  3. Therefore recognition is largely viewpoint-invariant as long as the geons can be seen.

One test removed parts of line drawings. Deleting the midsegments of lines (leaving the vertices, so you can still tell the shapes) did little harm, but deleting the vertices (leaving midsegments) made recognition much worse, because vertices carry information about geons. RBC has a clear mechanism and explains recognition of familiar categories quickly. It is less good at fine-grained recognition (telling one kind of cat from another, or one face from another), and is not the whole story.

The viewpoint problem and view-based theories

Is recognition really viewpoint invariant? Evidence suggests it is partly. People recognize objects faster and more accurately from canonical views, typical angles that show the most information (Palmer, Rosch and Chase, 1981): a horse seen from three-quarter front is easier than from straight above. Studies by Heinrich Bรผlthoff and Shimon Edelman (1992) and Michael Tarr found that recognition of novel objects decreases as the view rotates away from those learned, suggesting that we store a set of views and interpolate between them. A reasonable summary is that humans use both structure and views, depending on the task and the level of detail needed.

The brain: the ventral stream and agnosias

Object recognition depends heavily on the ventral visual stream, ending in inferotemporal (IT) cortex. Neurons there respond to complex shapes, with similar responses across changes in size and position (but not completely across viewpoint).

Brain damage can separate stages of recognition:

  • Apperceptive agnosia: patients see (can detect light, color, edges) but cannot organize the input into shapes. They cannot copy drawings or match shapes. This is a failure of the early, perceptual stages.
  • Associative agnosia: patients can copy and match pictures accurately but cannot say what the object is or what it is for. Perception is intact, but the link to meaning is broken.

The pair of conditions supports the idea that recognition includes stages, which Marr and others described. Pure cases are rare, and the categories are descriptive.

Faces: a special case?

Faces are all alike in basic layout (two eyes above a nose above a mouth), yet we distinguish thousands of individuals from small differences in spacing and shape. Several findings suggest that faces are processed differently from most objects.

  • Inversion effect. Robert Yin (1969) found that turning a face upside down hurts recognition far more than turning other objects, such as houses, upside down.
  • Thatcher illusion. Peter Thompson (1980) turned the eyes and mouth upside down in a face photo. The result looks grotesque when the face is upright, but is hard to spot when the whole picture is upside down. We are tuned to the typical upright arrangement.
  • Part-whole effect. James Tanaka and Martha Farah (1993) found that a person's nose is recognized better in the context of the whole face than in isolation, an effect not seen with houses or scrambled faces.
  • Composite effect. Combining the top half of one person's face with the bottom half of another makes it hard to identify the top half alone, unless the halves are misaligned (Young, Hellawell and Hay, 1987). The two halves fuse into a new face.

These support holistic processing: faces are encoded as integrated patterns, including the configuration (spacing between features), not as lists of parts.

The fusiform face area and the expertise debate

Nancy Kanwisher, Josh McDermott and Marvin Chun (1997) used fMRI to show a region in the fusiform gyrus of the ventral temporal lobe, the fusiform face area (FFA), that responds more strongly to faces than to other objects. Other face-selective regions exist, forming a network.

Is the FFA specialized for faces as such, or for fine-level discrimination of things we are expert in? Isabel Gauthier and colleagues proposed the expertise hypothesis, with evidence that bird and car experts show greater FFA activation for their specialty, and that training people with artificial "greebles" leads to face-like effects. Kanwisher and others argue that face selectivity is not explained by expertise alone. The debate is not settled, and the answer may include both innate organization and experience.

When face recognition fails

Prosopagnosia (face blindness) is a severe difficulty recognizing faces, despite normal vision and intelligence. It may follow brain injury (acquired) or be present from early life without known damage (developmental). Estimates suggest that around 2 percent of people have developmental prosopagnosia, with a range depending on criteria. Many people with it rely on voice, hair, clothes and gait.

Everyone also shows limits. Recognizing unfamiliar faces from photos is much less accurate than most people expect (matching a passport photograph to a live person produces many errors), while recognizing familiar faces is very robust. People are better at recognizing faces of their own ethnic group (other-race effect), reflecting experience. These findings have consequences for eyewitness identification and security checks.

Common misconceptions

"We recognize objects by matching photograph-like images." The evidence points to flexible descriptions and multiple views, not exact copies.

"The FFA is the brain's face module and nothing else." The FFA is part of a network, and debate continues about how specialized it is.

"Prosopagnosia means you cannot see faces." People with it see faces normally, but cannot use them to tell who someone is.

"Face recognition is easy for humans." It is excellent for familiar faces and surprisingly error-prone for unfamiliar ones.

Try it

Python Why exact templates fail: shift a letter by one pixel

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The Thatcher illusion with a cartoon face (a demonstration)

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Why is exact template matching a poor theory of human recognition?

    Real input varies endlessly, and people recognize novel instances.
  2. 2. Biederman's geons are identified through

    Properties that survive viewpoint change identify the volumes.
  3. 3. Patients who can copy a drawing accurately but cannot say what it is show

    Perception is intact, but the link to meaning is lost.
  4. 4. The face inversion effect is that

    Faces rely on holistic, upright configuration information.
  5. 5. The fusiform face area is best described as

    It is part of a network, and its face-specific nature versus expertise remains debated.

Summary

  • Template theories fail on variability; feature theories handle it but need arrangement; structural theories such as RBC describe objects as arrangements of geons.
  • Recognition is partly viewpoint dependent: canonical views and view-based storage matter.
  • Agnosias show separable stages of recognition, from perception to meaning.
  • Faces are processed holistically (inversion, Thatcher, part-whole and composite effects) in a network that includes the FFA.
  • Familiar faces are recognized well, unfamiliar ones poorly; prosopagnosia affects a small percentage of people.

Next: Chapter 4 Review