Skip to content
Tutorials Center
My learning

Topics

Lesson 38 of 40 ยท Projects and Practice

Project: Sentiment Classifier

Build a movie-review style sentiment classifier: text preprocessing, bag-of-words baseline, Keras embedding and LSTM models, metrics beyond accuracy and error analysis.

Sentiment analysis, deciding whether a piece of text is positive or negative, is the standard first text project. It touches every skill from the sequence chapter: vectorization, embeddings, recurrent layers and honest evaluation. As in the image project, the runnable code uses a tiny dataset and NumPy; the Keras version for a real corpus (such as the IMDB reviews) is read-only.

TensorFlow is not available in the browser runner. keras code is read-only; NumPy blocks run here.

Step 1: data and cleaning

A real dataset has tens of thousands of reviews. Here is a small hand-written one, enough to build the whole pipeline.

Python Clean, tokenize and split

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Cleaning choices matter: lowercasing is usually safe, removing stop words (the, a, was) often hurts sentiment ("not" is one!), and stemming rarely helps neural models. Keep the text close to raw and let the model learn.

Step 2: a bag-of-words baseline

A bag-of-words logistic regression is a strong, fast baseline for sentiment. It treats each word as a feature with a learned weight.

Python Bag of words + logistic regression

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Training accuracy is perfect but test accuracy is lower, and with only six test reviews it is also very noisy. That gap is the usual sign of too little data; real corpora have thousands of reviews. The weights are readable: positive words get positive weights. Unknown words in new reviews are simply ignored, and the model cannot see negation ("not good" counts "not" and "good" separately). Those are the limits sequence models try to fix.

Step 3: metrics beyond accuracy

On balanced data accuracy is fine. With 95 percent positive reviews, a model that always says "positive" is 95 percent accurate and useless. Use precision (of predicted positives, how many are right), recall (of real positives, how many were found) and F1 (their harmonic mean).

Python Precision, recall and F1

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Step 4: the Keras models

First the bag-of-words baseline in Keras, using TextVectorization in multi_hot mode:

Python
import keras
from keras import layers

(train_texts, train_labels), (test_texts, test_labels) = load_imdb_as_text()  # your loader

vectorize = layers.TextVectorization(max_tokens=20000, output_mode="multi_hot")
vectorize.adapt(train_texts)

baseline = keras.Sequential([
    keras.Input(shape=(1,), dtype="string"),
    vectorize,
    layers.Dense(16, activation="relu"),
    layers.Dropout(0.5),
    layers.Dense(1, activation="sigmoid"),
])
baseline.compile("adam", "binary_crossentropy", metrics=["accuracy"])
baseline.fit(train_texts, train_labels, validation_split=0.2, epochs=5, batch_size=64)

Then a sequence model that sees word order:

Python
vectorize_seq = layers.TextVectorization(max_tokens=20000, output_sequence_length=250)
vectorize_seq.adapt(train_texts)

model = keras.Sequential([
    keras.Input(shape=(1,), dtype="string"),
    vectorize_seq,
    layers.Embedding(20000, 64, mask_zero=True),
    layers.Bidirectional(layers.LSTM(32)),
    layers.Dropout(0.4),
    layers.Dense(1, activation="sigmoid"),
])
model.compile("adam", "binary_crossentropy", metrics=["accuracy"])
model.fit(train_texts, train_labels, validation_split=0.2, epochs=4, batch_size=64,
          callbacks=[keras.callbacks.EarlyStopping(patience=1, restore_best_weights=True)])

On a real corpus a bag-of-words model is typically within a few points of the LSTM, and both usually sit in the high 80s in percent accuracy on IMDB. A pretrained Transformer is better still but much heavier. Choose the simplest model that meets your target, and keep the preprocessing inside the model so the exported version accepts raw strings.

Step 5: error analysis

Look at the mistakes. Typical groups:

  • Negation and sarcasm: "not bad" or "great, another sequel".
  • Mixed reviews: praise and criticism in one text.
  • Domain words: a "sick" game is good.
  • Label noise: a wrongly labeled example.
Python Finding failure cases

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The most confident mistakes tell you what to fix first: add data for negation, relabel noisy examples, or switch to a model that reads word order.

Step 6: threshold and deployment

The sigmoid output is a probability. The 0.5 cut-off is a choice, not a law: if missing a complaint is costly, lower the threshold to raise recall and accept more false alarms. Export the model with its TextVectorization layer inside (see Deploying Models), and test it with a few raw strings before release.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why is accuracy a poor metric on a dataset with 95% positive examples?

    Accuracy rewards predicting the majority class; precision, recall and F1 expose the problem.
  2. 2. What does recall measure?

    Recall = true positives / (true positives + false negatives).
  3. 3. Which limitation does a bag-of-words model have that an LSTM can address?

    Counts discard order, so 'not good' and 'good, not bad' look similar.
  4. 4. Why put TextVectorization inside the exported model?

    Bundled preprocessing avoids training/serving skew.

Summary

  • Start with a bag-of-words baseline; it is fast, strong and readable.
  • Use precision, recall and F1 when classes are imbalanced.
  • Embedding plus Bidirectional LSTM models word order; compare it against the baseline honestly.
  • Study confident mistakes: negation, sarcasm, domain words and label noise.
  • Tune the decision threshold to the cost of each kind of error and export the model with its preprocessing.

Next: Project: Time Series Forecasting