Skip to content
Tutorials Center
My learning

Topics

Lesson 59 of 73 ยท Projects

Project: Shape Recognition With a CNN

Generate images of circles, squares and triangles, train a small convolutional network from scratch, use augmentation to generalize, and inspect learned filters.

Convolutional networks are built on one idea: look for the same small pattern everywhere in an image. In this project you put that idea to work on a task small enough to understand completely. You will draw your own dataset of circles, squares and triangles, train a CNN built from scratch in NumPy, see why data augmentation makes a network cope with shapes in places it never saw, and open the network up to look at the filters it learned.

Making the data yourself has a real advantage: you control every nuisance factor. Position, size, brightness and noise are all parameters, so you can ask precise questions such as "what happens if training shapes are always in the middle but test shapes are not?" Real datasets rarely let you run that experiment so cleanly.

The plan

Draw Generate 10 by 10 noisy images of three shapes.
Build Conv, ReLU, max pool, flatten, dense.
Train Adam, mini-batches, a held-out test set.
Inspect Look at filters and feature maps.
1 / 2

Drawing the dataset

Each image is 10 by 10 pixels. A circle is every pixel within a radius of its centre. A square is every pixel within a half-width. A triangle points up and widens by one pixel per two rows. After drawing, we multiply the shape by a random brightness (0.7 to 1.0), shift it by a random offset from the middle, and add Gaussian noise. The result is a task where the answer is obvious to a human but noisy enough that a network must learn the pattern rather than look up a fixed picture.

The generator takes a spread argument, the largest offset in pixels from the centre. We will use it later to build training and test sets that deliberately differ.

Python Generate and look at the shapes

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The classes are roughly balanced (140, 141 and 169 images, because labels are drawn at random), so plain accuracy is a fair metric, and a model that always guesses the most common class would score only about 38 percent. Notice how little information there is: a circle and a square at this resolution differ mostly in four corner pixels. That is what the network has to find.

A small CNN

The architecture is the one from Training a CNN From Scratch, resized for 10 by 10 images:

TEXT
input (1, 10, 10) -> conv 3x3, 8 filters, padding 1 -> ReLU -> max pool 2
                  -> flatten (8 x 5 x 5 = 200) -> dense 3 -> softmax cross-entropy

It has only 8 * 9 + 8 + 200 * 3 + 3 = 683 trainable numbers. The code reuses the im2col trick for the convolution; since the first layer needs no gradient with respect to the image, Conv.backward computes only the weight gradients. We train with Adam in mini-batches of 32 and measure accuracy on a test set of 300 fresh images.

After training, the example prints a confusion matrix and draws two diagnostic figures. The first shows the 8 learned 3 by 3 filters. The second shows, for one image of each shape, the eight feature maps that come out of the convolution and ReLU, so you can see what each filter responds to.

Python Train the shape classifier and inspect it

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

After the first epoch the network is at 42 percent test accuracy, and after 20 epochs (about 300 optimizer steps) it classifies 92.7 percent of the test shapes correctly. The confusion matrix shows which shapes get mixed up: every triangle is right, and the 21 mistakes are all between circle and square, the shapes that differ only in their corners.

The filter picture is a reminder that nobody designed these filters. Some look like oriented edge detectors, others like blob or corner detectors. In the feature maps, a bright pixel means "this filter's pattern is here". Different filters light up in different places for different shapes, and the dense layer learns which combination of lit-up places means which class. Do not expect textbook-perfect filters from 450 noisy images and a few hundred optimization steps. They are tiny and a little messy, but they work.

Generalizing to new positions

The test set above had shapes in roughly the same place as the training set. Real inputs are not that polite. Here is a harsher experiment: we train on only 45 images with perfectly centred shapes (spread=0) and test on 300 images where the shape can sit up to 2 pixels off centre. The dense layer at the end of our network ties each output to specific pixel positions, so it has never seen what a triangle looks like one pixel to the left.

Data augmentation is the standard remedy. Every time a batch is drawn, we move each image by a random shift of up to 2 pixels. The label is unchanged, because a circle is still a circle, so the network is taught that position is something it should ignore. We compare plain training, random shifts, and random shifts plus horizontal flips. Horizontal flips are legal for all three shapes, because a circle, a square and an upward triangle all look the same in a mirror.

Python Augmentation versus no augmentation

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Without augmentation the network memorizes the 45 centred images (its training loss goes to nearly zero) yet gets less than half of the off-centre test shapes right. With random shifts, the same architecture and the same 45 images reach about 81 percent, because the network now practises on every position it will meet, and adding flips gives about 85 percent. Notice that augmentation did not change the model or the data on disk, only what each epoch sees.

Choose augmentations that match the real world. Flipping a "6" turns it into something that is not a 6, and rotating a triangle by 180 degrees changes its label in a task that cares about "pointing up". Augmentation should change the picture without changing the answer.

A second route to the same goal is architectural: global pooling or deeper stacks of convolution and pooling layers reduce the dependence on exact position, which is why real image models are deep. See Classic CNN Architectures.

Where to go from here

  • Add a fourth shape, such as a plus sign or a diamond, and look at the new confusion matrix.
  • Raise the noise to 0.3 and see how accuracy degrades, then try more filters.
  • Add a second conv layer (Conv(8, 16)) after the pooling step. You will need to implement the input gradient, which is the col2im step you saw in Training a CNN From Scratch.
  • Draw the training and validation curves and check whether the network overfits when the training set is tiny.
  • Port the project to a framework: the read-only code in PyTorch Essentials maps line by line onto this one.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Why was a flatten-plus-dense head a problem when training on centred shapes only?

    The convolution is shift-equivariant, but the dense layer after flattening ties every weight to a location. Training on one position does not teach it about others.
  2. 2. What does data augmentation do?

    Augmentation produces new training views of the same labelled examples. The network must give the same answer for all of them.
  3. 3. Which augmentation would be a bad choice for a task that distinguishes upward-pointing triangles from downward-pointing ones?

    A vertical flip turns an upward triangle into a downward one, so it changes the correct label. Augmentations must preserve the label.
  4. 4. In the feature-map figure, what does a bright pixel in map 3 mean?

    Each feature map is the output of one filter slid over the image (after ReLU). A large value marks a place where the filter's pattern matches.
  5. 5. How many trainable parameters does the 8-filter network have?

    Conv: 833 weights + 8 biases = 80. Dense: 200*3 weights + 3 biases = 603. The total is 683.

Summary

  • Generating your own images lets you control position, size and noise, and run clean experiments about generalization.
  • A small CNN (conv, ReLU, max pool, flatten, dense) with 683 parameters separates three noisy shapes at above 90 percent accuracy.
  • A confusion matrix shows which classes the network mixes up.
  • Learned filters and feature maps can be inspected directly; they are small, imperfect pattern detectors.
  • A dense head after flattening does not generalize to unseen positions; random-shift augmentation fixes much of that without more data.
  • Augmentations must preserve the label.

Next project: Sentiment Classifier.