Skip to content
Tutorials Center
My learning

Topics

Lesson 36 of 63 ยท Sequences and Structure

Coreference Resolution and Word Sense Disambiguation

Learn how NLP resolves what words refer to and which meaning they carry: a simplified Lesk algorithm and rule-based pronoun linking written from scratch.

"Maria called Tom because she lost the report." You know at once that she is Maria. In "He sat on the bank and watched the water" you know that bank is a riverside, not a building with a vault. People do both without noticing, but a program that only sees tokens has no idea. These are two classic problems of semantic interpretation: coreference resolution (which expressions point to the same thing?) and word sense disambiguation, or WSD (which meaning of this word is intended?).

Both matter in practice. A summarizer that cannot tell who they are produces nonsense. A search engine that cannot separate bat the animal from bat the club returns the wrong pages. In this lesson you will learn how both tasks are framed, build a simplified Lesk algorithm for word senses, and build a small rule-based linker for pronouns. Neither is a state-of-the-art system, but each shows exactly what such a system must decide, and why the real ones need so much data.

Two problems, one theme

Coreference resolution

Mentions Names, nouns and pronouns that refer to something.
Clusters Maria, she and her all point to one person.
Constraints Gender, number and type must agree.

Answers the question "who is this?"

1 / 2

Both tasks resolve ambiguity by looking at context. They differ in what they choose between. WSD chooses from a fixed list of meanings that a dictionary gives you in advance. Coreference chooses among the earlier mentions in the text, so the candidates are different for every document.

Word senses and WordNet

Most common words have more than one meaning. Bank can be a financial institution, the land beside a river, or a row of machines. Each meaning is a sense. A sense inventory lists the senses of every word, with a short definition called a gloss. The best-known inventory for English is WordNet, a hand-built lexical database. It groups words into sets of synonyms (synsets) such as bank.n.01, where n means noun, and links them: a dog is a kind of canine (a hypernym relation), a wheel is part of a car (a meronym relation).

In a real project you would use NLTK's WordNet interface, which needs a data download that does not run in the browser, so here is how it looks as a read-only block:

Python
from nltk.corpus import wordnet as wn   # needs nltk.download("wordnet")

for synset in wn.synsets("bank", pos=wn.NOUN)[:3]:
    print(synset.name(), "-", synset.definition())
# bank.n.01 - sloping land (especially the slope beside a body of water)
# depository_financial_institution.n.01 - a financial institution that accepts deposits ...
# ...

Because the download is not available, the runnable examples below use a hand-made sense inventory with just a few words. The idea is identical to WordNet's, only smaller, and the algorithms below do not care how big the dictionary is.

The Lesk algorithm

In 1986, Michael Lesk proposed a beautifully simple idea: the right sense is the one whose definition shares the most words with the surrounding sentence. If the sentence mentions river and water, the gloss "land beside a river or stream where the water meets the shore" overlaps with it, while the gloss about deposits and lending does not.

The simplified Lesk version used below works like this:

  1. Collect the content words of the sentence (drop stopwords such as the and of; see stopwords), and remove the target word itself.
  2. For each sense, collect the content words of its gloss.
  3. Score each sense by the size of the overlap.
  4. Return the sense with the highest score. If nothing overlaps, fall back to the first sense, which in a real inventory is the most frequent one.
Python Simplified Lesk on a tiny sense inventory

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

All six sentences get the right sense. The first one wins with a single shared word (money), which is a thin margin. The riverside sentence overlaps with river and water, and the verbose listing shows the other two senses scoring zero. Notice two weaknesses. First, the method needs the exact words: deposited does not match deposits because we only strip a trailing s, so a real system lemmatizes (see stemming and lemmatization). Second, some of the matches are accidents: in the plant factory sentence, a common word like use counts just as much as machines, and a sentence that happened to mention both water and machines would produce a tie. Lesk is a good baseline, not a dependable solution.

Glosses are short, so overlaps are small or empty. The extended Lesk idea (Banerjee and Pedersen) enlarges each sense's signature with the glosses of related senses, such as its hypernyms and example sentences. We imitate that here by adding a short list of related words to each sense, then compare three systems on eight test sentences: gloss only, gloss plus related words, and a baseline that always chooses the first sense.

Python Gloss only vs gloss plus related words

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

With glosses alone, four of the eight sentences are wrong, and every error is a fallback: none of their words appears in the gloss, so the algorithm shrugs and returns the first sense. Adding related words fixes all four, giving 8 of 8. Do not take that number seriously. We wrote the related words after looking at the test sentences, and eight sentences prove nothing. The honest lesson is the direction: richer signatures mean fewer empty overlaps. On real benchmarks, Lesk-style methods reach roughly 50 to 60% accuracy, the most-frequent-sense baseline is a surprisingly hard 60 to 65% to beat, and modern systems that use contextual embeddings from pretrained models reach the low 80s.

Contextual embeddings from models such as BERT change the picture: the vector for bank already differs between sentences, so you can pick the sense whose gloss embedding is closest, or train a classifier on top. Compare this with the single vector per word problem described in distributional semantics.

Coreference: mentions, clusters and constraints

A mention is any span that refers to something: a name (Maria), a noun phrase (the report) or a pronoun (she). A coreference cluster is the set of mentions that refer to the same entity. In "Maria called Tom because she lost the report", the clusters are {Maria, she}, {Tom} and {the report}. The earlier mention a pronoun points back to is its antecedent. The task has two steps: find the mentions, then link each one to an earlier mention or to nothing.

Hard rules already remove many wrong candidates. A pronoun must agree with its antecedent:

  • Number: they needs a plural antecedent, it a singular one.
  • Gender: she needs a female candidate, he a male one.
  • Animacy or type: it rarely refers to a person, and he rarely refers to an organization.

Among the candidates that survive, a simple heuristic picks the nearest one, because pronouns usually refer to something mentioned recently. Better rules prefer subjects over objects, and the best learned systems score every candidate pair with a neural network over contextual embeddings. The program below implements the rules: a lexicon with gender, number and type, an agreement test, and a nearest-first scan.

Python Rule-based pronoun linking and where it breaks

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The first sentence works through agreement alone: she cannot be Tom, so Maria is chosen even though Tom is closer. The third works through number: Acme is singular, so they goes to engineers. The second sentence shows the limit of the rules. Both Tom and Alan are male singular people, so agreement does not help, and the nearest-first heuristic answers Alan. That may be right, but nothing in the code knows: a reader would say it depends on who did the fixing. The last sentence is a famous test case. Because it was too big, it is the trophy. Change the last word to small and it becomes the suitcase. Our rules answer suitcase both times, since they only see recency and agreement, never meaning. Examples of this kind are called Winograd schemas, and they are used to test whether a system uses real-world knowledge.

Real coreference systems

Modern systems learn to do what our rules cannot. A typical neural coreference model enumerates candidate spans, builds a vector for each span from a pretrained encoder, and for every span scores each earlier span as its possible antecedent, including a "no antecedent" option for first mentions. Training data comes from corpora such as OntoNotes, where annotators have marked the clusters. Evaluation uses cluster-based metrics (MUC, B-cubed, CEAF), usually averaged into one score called CoNLL F1.

Be aware of three practical problems. Coreference models are domain sensitive and degrade on medical, legal or conversational text. They can be biased: a system may link doctor with he more readily than with she because the training data did, a known issue measured with the Winogender test set (see evaluation, bias and responsible NLP). And errors propagate: one wrong link can merge two entities into a single cluster. Large language models can resolve many references from the prompt alone, but for pipelines that need clusters and character offsets, a dedicated model is still more predictable.

Where do you use all this? Summarization (replace she by Maria so a sentence stands alone), information extraction (the tenth fact about "the company" belongs to Acme), question answering ("Who lost the report?") and parsing-based relation extraction all need it.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What does the simplified Lesk algorithm use to choose a word sense?

    Lesk scores each sense by the overlap between its gloss and the words around the target word, and picks the highest score.
  2. 2. Why did the "gloss only" variant get four of eight test sentences wrong?

    Glosses are short, so overlap is often empty. With zero overlap the algorithm returns the default first sense, which is wrong half the time here.
  3. 3. In coreference resolution, what is an antecedent?

    In "Maria lost her key", Maria is the antecedent of her.
  4. 4. Why does a nearest-compatible-mention rule fail on "The trophy did not fit in the suitcase because it was too big"?

    Both nouns agree with it in number and type. Only knowing that a too-big thing cannot fit inside another tells you it is the trophy.
  5. 5. Why is the 8 of 8 result for "gloss plus related words" not strong evidence that the method is good?

    Tuning on the test data and using very few examples makes the number optimistic. A fair evaluation uses a separate benchmark of many sentences.

Summary

  • Word sense disambiguation picks one meaning of a word from a sense inventory such as WordNet; coreference resolution links mentions (names, nouns, pronouns) that refer to the same entity.
  • Simplified Lesk scores each sense by the overlap between its gloss and the context; extended Lesk enlarges the gloss with related words to avoid empty overlaps.
  • The first-sense (most frequent sense) baseline is hard to beat, so always compare against it, and never trust a tiny, tuned test set.
  • Coreference rules use agreement (number, gender, type) to filter candidates and recency to choose among them; they fail when only world knowledge can decide, as in Winograd schemas.
  • Modern systems score spans with contextual embeddings and are evaluated with cluster metrics such as CoNLL F1; they inherit domain sensitivity and bias from their training data.

Next, go deeper into sentence structure with Dependency Parsing and Semantic Role Labeling.