Skip to content
Tutorials Center
My learning

Topics

Lesson 34 of 63 ยท Sequences and Structure

Named Entity Recognition

Learn named entity recognition: entity types, a gazetteer and regex baseline, BIO tagging, and entity-level precision, recall and F1, with runnable Python.

Read this sentence: "In June 1943, Alan Turing left Bletchley Park for London." A person instantly sees one person, one organization, one place and one date. Named entity recognition (NER) teaches a program to do the same: find the spans of text that name real-world things and label each with a type. It is the step that turns text into structured data, such as a table of companies mentioned in news, a list of drug names in clinical notes, or the city and date in a travel request.

NER is a sequence labeling task like part-of-speech tagging, with one important difference: the things we want are spans of several words (Bletchley Park), so we need a way to mark where a span begins and ends. In this lesson we build a rule-based recognizer, learn the standard BIO tagging scheme, and, most importantly, learn to evaluate the result properly.

What counts as an entity

Entity types depend on the project. The most common general-purpose set is:

The classic entity types

PER People, such as Alan Turing.
ORG Companies, agencies, teams.
LOC Cities, countries, rivers.
DATE June 1943, next Friday.

Spans are labeled by what they mean in this sentence.

1 / 2

The label depends on context, which is what makes NER harder than a dictionary lookup. Washington may be a person, a city or a state, and Apple a fruit or a company. Entities can also be nested ("Bank of England" contains "England"), though most systems label only the longest span.

A rule-based baseline

Before reaching for a model, build the simplest thing that could work: a gazetteer (a list of known names for each type) plus regular expressions for things with a regular shape, such as years and prices. See Rule-Based Information Extraction for more patterns. There is one design decision worth noting: when two matches overlap, prefer the one that starts first and, on a tie, the longer one, so New York is not cut into smaller pieces.

Python Gazetteer and regex entity finder

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The finder reports seven entities, with exact character offsets. It also silently misses the second mention, the lone word Turing in the last sentence, because only the full name is in the gazetteer. That is the characteristic weakness of rules: they are precise on what they know (they never invented an entity here) and blind to everything else. Adding Turing to the list would then make it fire on Turing machine, and so on. Models trained on labeled text handle this through context instead of lists.

BIO tags

Statistical taggers cannot output spans directly, so we turn spans into one tag per token. In the BIO scheme:

  • B-LABEL marks the beginning of an entity,
  • I-LABEL marks a token inside the same entity,
  • O marks a token outside any entity.

The B- tag matters because two entities may stand next to each other (New York Boston): without it, you could not see where the first ends. Some corpora use richer variants such as BIOES (adding End and Single), but BIO is the one you will meet most.

Converting both ways is a small exercise. Our gold data is written in a bracket markup, [PER Ada] ..., which the code parses into plain text plus character spans.

Python Spans to BIO tags and back

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The table shows Alan as B-PER and Turing as I-PER, and bio_to_spans recovers the three spans as token ranges: tokens 0 to 2, 3 to 5 and 6 to 7. The last two lines show how decoders cope with imperfect tag sequences from a model. A run that starts with I-PER (no B-) is usually treated as the start of an entity, and B-LOC followed by B-LOC correctly gives two separate entities. A decoder that is not defensive here produces silent errors.

Evaluating NER correctly

How good is the rule baseline? The obvious number, token accuracy (what fraction of tokens got the right BIO tag), is a trap. Most tokens in most text are O, so a system that never finds any entity at all scores well. The right unit is the entity: a predicted entity counts as correct only if both its boundaries and its type exactly match a gold entity. From the counts of true positives (tp), false positives (fp) and false negatives (fn):

TEXT
precision = tp / (tp + fp)       how many reported entities are right
recall    = tp / (tp + fn)       how many real entities were found
F1        = 2 * precision * recall / (precision + recall)

If you need a refresher, see Evaluating Classifiers. Entity-level F1 is the standard score in NER papers and shared tasks.

Python Entity-level precision, recall and F1

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Of the 15 gold entities, the rules find 12 exactly (precision 0.92, recall 0.80, F1 0.86). The error report is instructive. Charles Babbage and the lone Turing are missed because they are not in the list. Cambridge University is the sharper case: the gazetteer knows Cambridge as a place, so it reports a LOC with the wrong boundaries and the wrong type, which costs both a false positive and a missed entity. A partial match is not partial credit under strict scoring.

Now compare the two token accuracies at the bottom. The rules reach 93.9%, which sounds excellent, yet a program that outputs O for every token and finds nothing at all still scores 69.5%. The token metric is dominated by the easy O tokens; F1 focuses on the entities, which are the point of the task. On real news text, where entities are rarer than in our compact examples, the all-O baseline scores even higher.

Using trained models

For real work you rarely write the recognizer yourself. A neural pipeline is trained on tens of thousands of labeled sentences and uses the surrounding words, capitalization and word shape to decide. The code below is read-only because the model download is large.

Python
import spacy

nlp = spacy.load("en_core_web_sm")
doc = nlp("In June 1943, Alan Turing left Bletchley Park for London.")
for ent in doc.ents:
    print(ent.text, ent.label_, ent.start_char, ent.end_char)

# spaCy's labels include PERSON, ORG, GPE (countries, cities), DATE and MONEY.
# Token-level BIO tags are available as token.ent_iob_ and token.ent_type_.

Pre-trained general models cover the classic types. For your own domain (contracts, chemistry, product names) you can fine-tune with a few hundred annotated examples, often starting from a pre-trained transformer, see pre-trained language models. A practical hybrid is common: a statistical model for the broad coverage and a gazetteer for the names it must never get wrong.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Why does the BIO scheme have a separate B- tag instead of only I- tags?

    Without B-, two neighboring entities of the same type would merge into a single span. B- marks every new entity start.
  2. 2. What is the main weakness of a gazetteer-based recognizer?

    A list is precise for what it contains but blind to unlisted names, and the same string can be different types in different contexts.
  3. 3. Under entity-level evaluation, when is a predicted entity counted as correct?

    Strict entity-level scoring needs an exact span and the right label. A partly overlapping span counts as a false positive and also a missed entity.
  4. 4. Why is token accuracy a poor measure for NER?

    In running text the O tag dominates. Entity-level precision, recall and F1 measure what matters: whether entities were found with the right boundaries and types.
  5. 5. A model reports 10 entities, 8 of which are correct, and the gold data has 16 entities. What are precision and recall?

    Precision is 8 correct out of 10 reported (0.80); recall is 8 found out of 16 real entities (0.50).

Summary

  • NER finds spans of text that name entities (people, organizations, places, dates, money, or domain terms) and labels each with a type.
  • A gazetteer plus regular expressions makes a precise but low-recall baseline; overlapping matches need a rule (earliest start, then longest).
  • BIO tagging turns spans into one tag per token (B-, I-, O); decode defensively when a model emits an I- without a B-.
  • Evaluate at the entity level: exact boundary and type match, then precision, recall and F1. Token accuracy is inflated by the many O tokens.
  • Trained models such as spaCy's use context and generalize to unseen names; hybrids with gazetteers are common in production.

Next, look at how words combine into phrases and sentences in Syntactic Parsing.