Skip to content
Tutorials Center
My learning

Topics

Lesson 37 of 40 ยท Projects and Practice

Project: Image Classifier

Build an image classifier end to end: data split, baseline, CNN with augmentation, evaluation with a confusion matrix, error analysis and export, with a runnable NumPy mini version.

This project follows the workflow you would use on a real problem: define the task, split the data, build a baseline, train a CNN, evaluate honestly, look at the mistakes and export the result. The runnable parts use a tiny synthetic dataset and NumPy, so you can execute the whole pipeline in the browser. The Keras code (read-only) is what you run on a real dataset such as flower photos or CIFAR-10 on your own machine.

TensorFlow is not available in the in-browser runner. Blocks that import keras are read-only; blocks that import only numpy run here.

The plan

Image classifier workflow

Data Folders and a split
Baseline A simple model first
CNN Augment and train
Evaluate Confusion matrix

Compare every model to the baseline.

1 / 2

Step 1: a dataset we can run here

Real images need files. For a runnable stand-in we generate 8 by 8 images of three classes: a horizontal bar, a vertical bar and a diagonal, each at a random position with noise. The classes differ in shape, not location, which is precisely what convolutions are good at.

Python Generate and split the data

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Always hold out a test set that you touch only at the very end. The validation set is for choosing settings; if you tune against the test set, its score becomes optimistic.

Step 2: a baseline

Before a deep model, fit the simplest reasonable one. If a CNN barely beats it, the problem or data needs attention rather than more layers. Here is the baseline: softmax regression on raw pixels.

Python Baseline: softmax regression on pixels

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Pixel-position models struggle because a bar at a new position looks completely different in raw pixels. That is the weakness a convolution removes.

Step 3: a convolutional feature

A real CNN learns its filters. To keep the demo short we use three hand-set kernels (horizontal, vertical, diagonal detectors), apply them, max-pool each map globally, and fit a softmax on those 3 numbers. Global max pooling makes the features position-independent, the same effect a trained CNN achieves.

Python Convolution features + global max pooling

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The confusion matrix shows which classes get mistaken for each other; the diagonal holds the correct predictions. This is the evaluation tool to use when classes are more than two: accuracy hides which class fails.

Step 4: the Keras model for real photos

Python
import keras
from keras import layers

train_ds = keras.utils.image_dataset_from_directory(
    "flowers", validation_split=0.2, subset="training",
    seed=1, image_size=(128, 128), batch_size=32)
val_ds = keras.utils.image_dataset_from_directory(
    "flowers", validation_split=0.2, subset="validation",
    seed=1, image_size=(128, 128), batch_size=32)
class_names = train_ds.class_names

AUTOTUNE = tf.data.AUTOTUNE
train_ds = train_ds.cache().shuffle(1000).prefetch(AUTOTUNE)
val_ds = val_ds.cache().prefetch(AUTOTUNE)

augment = keras.Sequential([
    layers.RandomFlip("horizontal"),
    layers.RandomRotation(0.1),
    layers.RandomZoom(0.1),
])

model = keras.Sequential([
    keras.Input(shape=(128, 128, 3)),
    augment,
    layers.Rescaling(1 / 255),
    layers.Conv2D(32, 3, padding="same", activation="relu"),
    layers.MaxPooling2D(),
    layers.Conv2D(64, 3, padding="same", activation="relu"),
    layers.MaxPooling2D(),
    layers.Conv2D(128, 3, padding="same", activation="relu"),
    layers.GlobalAveragePooling2D(),
    layers.Dropout(0.3),
    layers.Dense(len(class_names)),          # logits
])
model.compile(
    optimizer="adam",
    loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
    metrics=["accuracy"],
)

callbacks = [
    keras.callbacks.EarlyStopping(patience=5, restore_best_weights=True),
    keras.callbacks.ModelCheckpoint("best.keras", save_best_only=True),
]
history = model.fit(train_ds, validation_data=val_ds, epochs=40, callbacks=callbacks)

Notes: the model outputs logits and the loss has from_logits=True (numerically more stable than a softmax layer); data loading comes from Loading Images and Text; augmentation and the option of a pretrained base are from Augmentation and Transfer Learning. If accuracy plateaus low, swap the hand-built stack for a frozen MobileNetV2 base.

Step 5: error analysis

Aggregate accuracy does not tell you what to fix. Collect the validation images the model got wrong, sorted by confidence, and look at them:

Python
import numpy as np
images, labels = next(iter(val_ds.unbatch().batch(1000)))
probs = keras.ops.softmax(model.predict(images)).numpy()
wrong = np.where(probs.argmax(1) != labels.numpy())[0]
worst = wrong[np.argsort(-probs[wrong].max(1))][:16]   # confident mistakes

Typical findings: mislabeled files, near-duplicate classes, tiny objects, odd backgrounds. Fixing data often beats changing the architecture.

Step 6: save and test the export

Python
model.save("flower_classifier.keras")
model.export("export/flowers/1")
# then verify the exported model reproduces the Keras predictions on a sample

See Saving and Loading Models and Deploying Models.

Ideas to extend

  • Fine-tune a pretrained base and compare against the baseline.
  • Plot the accuracy and loss curves; find the epoch where validation loss turns.
  • Try mixed precision and measure the time per epoch.
  • Add a small web page that sends an image to the exported model.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why build a simple baseline before a CNN?

    A baseline shows what 'good enough' looks like and exposes data problems early.
  2. 2. What does a confusion matrix show?

    Rows are true classes and columns predictions; off-diagonal cells are the mistakes.
  3. 3. Why use from_logits=True with a final Dense layer without softmax?

    The loss combines softmax and cross-entropy in a stable way.
  4. 4. When should the test set be used?

    Repeated use for decisions leaks information and makes the final score optimistic.

Summary

  • Split the data first, keep a test set untouched, and compare every model with a baseline.
  • Convolutions with pooling give position-independent features, which is why CNNs beat pixel models on images.
  • Use augmentation, dropout and early stopping with checkpoints to control overfitting.
  • Evaluate with a confusion matrix and inspect confident mistakes.
  • Save the model, export it, and verify the export before shipping.

Next: Project: Sentiment Classifier