Skip to content
Tutorials Center
My learning

Topics

Lesson 24 of 46 ยท Classical Machine Learning for Images

Classifying Handwritten Digits

Train logistic regression and SVM classifiers on the 8x8 digits dataset, compare raw pixels with HOG features, and read a confusion matrix to see the errors.

Handwritten digit recognition is the "hello world" of image classification, and for good reason: the images are small, the classes are clear, and a first model takes five lines. scikit-learn ships a compact version of the problem, 1,797 grayscale images of the digits 0 to 9, each only 8 by 8 pixels. In this lesson you will train two classifiers on it, compare raw pixels with the HOG features from Hand-Crafted Features and HOG, and learn to read a confusion matrix to see which mistakes a model makes.

The goal is not to chase the last tenth of a percent. It is to learn the workflow that every image classifier follows: turn images into feature vectors, split the data honestly, fit a model, measure it on data it has never seen, and study the errors.

The dataset

load_digits() returns a bundle. digits.images holds the pictures as an array of shape (1797, 8, 8) with pixel values from 0 (white paper) to 16 (full ink). digits.data is the same data flattened to (1797, 64), one row per image, and digits.target holds the labels. Dividing by 16 rescales the pixels to the range 0 to 1, which helps most models.

flowchart LR
    A["Images<br/>(1797, 8, 8)"] --> B["Features<br/>pixels or HOG"]
    B --> C["Split<br/>train 75%, test 25%"]
    C --> D["Fit model<br/>on the training set"]
    D --> E["Predict on<br/>the test set"]
    E --> F["Accuracy,<br/>confusion matrix"]

The same five-stage workflow applies to every image classifier in this course.

A first classifier

Two details deserve attention in the code below.

  • train_test_split divides the data into a training set the model learns from and a test set it never sees until evaluation. Without this split, a model that simply memorizes the training pictures would look perfect. random_state=0 makes the split reproducible.
  • stratify=y keeps the class proportions the same in both parts, so that each digit appears about equally often in the test set.

We compare logistic regression, a linear model that draws straight-line boundaries between classes in the 64-dimensional pixel space, with an SVM (support vector machine) using the RBF kernel, which can draw curved boundaries.

Python Pixels into two classifiers

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The classes are well balanced (174 to 183 images each), so accuracy, the fraction of correct predictions, is a fair measure here. Logistic regression reaches 0.983 on the training images and 0.969 on the test images. The SVM reaches 1.000 on the training set, a sign that it has fit every training image, and 0.991 on the test set. Always report the test number: it estimates how the model behaves on new pictures. The gap between training and test accuracy is the first thing to look at when judging overfitting.

Note: With only one split, the exact accuracy depends a little on which images ended up in the test set. For decisions between close models, use cross-validation (cross_val_score), which repeats the split several times and reports the average and spread. The Project: Handwritten Digit Recognizer does exactly that.

Pixels versus HOG

On this dataset raw pixels are already strong, because the digits are small, centered, and drawn with a consistent stroke. That is unusual. Real photos are rarely so tidy, which is why we should check what the features add. The next example computes a HOG descriptor for each 8 by 8 image, using cells of only 2 by 2 pixels (the images are tiny), which gives 4 by 4 cells, 3 by 3 blocks and 324 numbers per image. It then trains both models on both kinds of features.

The HOG function here processes a whole stack of images at once (it is the same recipe as in the previous lesson, vectorized so that it runs quickly). The second half of the example asks a tougher question: what happens if the digits in the test set are slightly shifted, a situation the training data never contained? Training stays the same; only the test images move.

Python Pixels, HOG and shifted digits

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Reading the results:

Features Logistic regression SVM
Raw pixels (64) 0.969 0.991
HOG (324) 0.982 0.991

HOG helps the weaker, linear model (0.969 to 0.982) because orientation features make the classes easier to separate with straight boundaries, while the SVM was already near its ceiling. So on tidy data the extra features are a small gain.

The shifted test tells a more important story. When every test digit moves one pixel to the right, the pixel-based SVM drops from 0.991 to 0.529, and with a one-pixel diagonal shift to 0.160, close to guessing (chance is 0.10 for ten classes). The HOG-based SVM falls to 0.769 and 0.349: clearly more robust, but still far from its 0.991. Two lessons follow. First, good features buy tolerance to small changes, not immunity. Second, the model was trained only on centered digits, so it never learned that shifted digits are still the same digit. Two remedies appear later in the course: data augmentation (add shifted copies to the training set) and convolutional networks, whose design builds in tolerance to position. See Training a Small Network on Images.

The confusion matrix

Accuracy is a single number. It hides which digits get confused. A confusion matrix counts, for each true class (rows), how often each class was predicted (columns). A perfect model would put all its counts on the diagonal.

To see more than a handful of errors we use the weaker model, logistic regression on pixels. From the matrix we can also compute two per-class measures:

  • Recall for class k: of all images that truly are k, the share the model found. It is the diagonal entry divided by the row sum.
  • Precision for class k: of all images the model called k, the share that really are k. It is the diagonal entry divided by the column sum.
Python Reading a confusion matrix

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The model makes 14 mistakes out of 450 test images. The matrix shows where they are: the row for digit 8 has the lowest recall, 0.88, with three 8s taken for 1s, one for 5 and one for 7. The column for digit 1 has the lowest precision, 0.87, because the model drops uncertain images into class 1 more often than into any other class: 2s, 5s, 6s (twice) and 8s (three times) were called 1. Notice also that precision and recall of the same digit can differ: a class that the model uses as a "default" has good recall but poor precision.

The printed pictures are the most valuable part. Looking at a few misclassified images tells you whether the errors are understandable (a squashed 4 that looks like an 8 at 8 by 8 resolution) or point to a bug. This habit, called error analysis, is often more productive than trying another model.

flowchart TB
    A["Confusion matrix"] --> B["Diagonal<br/>correct predictions"]
    A --> C["Off-diagonal<br/>mistakes"]
    C --> D["Row sum<br/>true count of a class"]
    C --> E["Column sum<br/>predicted count of a class"]
    D --> F["Recall = diagonal / row sum"]
    E --> G["Precision = diagonal / column sum"]

Every standard classification metric comes from counts in this one table.

Practical advice

  • Scale your inputs. Pixels in the range 0 to 1 are a good default for linear models and SVMs, because features on very different scales can slow or distort training.
  • Never tune on the test set. If you try many settings and keep the best test score, the test set is no longer unseen. Use cross-validation on the training data for choices, and look at the test set once at the end.
  • Compare against a baseline. Predicting the most common class gets about 10 percent here. Every model should clearly beat it.
  • Look at the errors, not just the score.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Why do we evaluate a classifier on a test set that was not used for training?

    Accuracy on training data can be perfect for a model that merely memorized the images. The test set measures how well the model generalizes to new pictures.
  2. 2. A confusion matrix has the true class on the rows. How do you compute the recall of class 8?

    Recall asks: of all images that truly are 8, how many were found? That is the diagonal entry divided by the total of the row. Dividing by the column sum gives precision.
  3. 3. The pixel-based SVM scored 0.991 on the test set but fell to 0.529 when digits were shifted by one pixel. What does this show?

    With raw pixels a one-pixel shift changes many features at once, and the training data contained no shifted digits. Better features, augmentation or convolutional networks address this.
  4. 4. What does stratify=y do in train_test_split?

    Stratified splitting makes sure each digit appears in both sets in roughly the same proportion as in the full dataset.
  5. 5. A model has training accuracy 1.000 and test accuracy 0.991. How should you interpret the small gap?

    A small drop from training to test accuracy is normal. A large gap (for example 1.00 versus 0.70) would signal overfitting.

Summary

  • The scikit-learn digits dataset has 1,797 images of 8 by 8 pixels; flatten them to 64 features, scale to 0 to 1, and always split into train and test sets (use stratify for balanced classes).
  • Logistic regression (linear) and an RBF SVM (curved boundaries) reach 0.969 and 0.991 test accuracy on raw pixels; HOG lifts logistic regression to 0.982.
  • Tidy data hides the weakness of pixel features. Shifted digits drop the pixel SVM to 0.529 and the HOG SVM to 0.769, which motivates augmentation and convolutional networks.
  • A confusion matrix has the true class on rows and the predicted class on columns. Recall is diagonal over row sum; precision is diagonal over column sum.
  • Study the misclassified images, not only the score, and tune on training data only.

Next: PCA, Eigenfaces and Image Compression.