Skip to content
Tutorials Center
My learning

Topics

Lesson 35 of 39 ยท Text and Web Mining

Topic Modeling and Sentiment Analysis

Discover hidden topics with NMF and LDA, then measure opinion with lexicon-based and machine-learning sentiment analysis, handle negation, and extract entities with regular expressions.

Two of the most common text mining jobs are answering "what are these documents about?" and "how do the authors feel?". The first is topic modeling, an unsupervised method that discovers themes. The second is sentiment analysis (opinion mining), which classifies text as positive, negative or neutral. Both build on the vectors from Text Mining Basics.

Topic modeling

A topic is a group of words that tend to appear together, such as {goal, match, team, coach}. A topic model describes every document as a mixture of topics and every topic as a distribution over words. Two popular methods:

  • LDA (Latent Dirichlet Allocation) is a probabilistic model working on word counts. Each document picks topic proportions; each word picks a topic, then a word from that topic.
  • NMF (non-negative matrix factorization) factorizes the TF-IDF matrix V into two non-negative matrices, V โ‰ˆ W H: W says how much of each topic is in each document, H says which words define each topic. It is deterministic given a seed and often gives crisp topics on short texts.

You choose the number of topics. The topics are just word lists: a human must read them and give them names.

Python Topics with NMF

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The three themes (sport, finance, gadgets) emerge without any labels, and the dominant topic per document groups the sentences accordingly.

LDA with counts

LDA expects raw counts, not TF-IDF. The same corpus with scikit-learn:

Python Topics with LDA

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

LDA is stochastic, so topics can differ a little between seeds and on tiny corpora; real use needs hundreds of documents. Judge topic quality by reading the top words, and with measures such as coherence (do the top words really co-occur?).

Sentiment analysis

Sentiment analysis predicts the polarity of a text. There are two main approaches.

Lexicon-based

Keep a dictionary of words with scores (great = +2, awful = -3) and sum them. It needs no training data, and is explainable, but it must handle context: negation ("not good"), intensifiers ("very good"), and domain words ("cold" is bad for coffee, good for beer).

Python A lexicon scorer with negation

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Machine learning approach

Collect labeled reviews, vectorize them (TF-IDF with unigrams and bigrams) and train a classifier, as in Nearest Neighbors and Naive Bayes. Bigrams help because "not good" becomes its own feature.

Python Sentiment classifier with bigrams

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Unigrams alone cannot tell "good" from "not good" (accuracy near chance), while bigrams learn the flip almost perfectly. Real reviews are far messier than these templates. Pitfalls in real projects: sarcasm ("great, another delay"), mixed opinions in one review, domain shift (a model trained on movie reviews performs worse on product reviews) and class imbalance.

Aspect-based sentiment and information extraction

Overall polarity hides detail: "great screen, terrible battery" is positive for one aspect and negative for another. Aspect-based sentiment analysis finds aspects and scores each. Related is information extraction: pulling structured facts out of text, such as dates, prices, emails or named entities. For simple formats, regular expressions are enough.

Python Extracting facts with regular expressions

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

For people and organizations, which have no fixed pattern, use trained named entity recognition models (spaCy, transformers).

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does a topic model produce?

    Topics are distributions over words; each document is a mixture of topics.
  2. 2. Which method works on raw word counts?

    LDA is a probabilistic count model; NMF is usually applied to TF-IDF.
  3. 3. Why do bigrams help sentiment classification?

    A unigram model sees 'not' and 'good' separately; the bigram 'not good' carries the negative meaning.
  4. 4. What is aspect-based sentiment analysis?

    It separates opinions per aspect instead of giving one overall score.

Summary

  • Topic models (LDA, NMF) find themes without labels; you choose the number of topics and name them by reading top words.
  • Sentiment analysis is lexicon-based (rules, explainable) or machine-learning based (needs labeled data); negation and domain matter.
  • Aspect-based sentiment and information extraction turn text into structured, actionable data.

Next: Project: Customer Segmentation