Skip to content
Tutorials Center
My learning

Topics

Lesson 24 of 40 ยท Convolutional Networks

Augmentation and Transfer Learning

Reduce CNN overfitting with data augmentation layers, reuse pretrained models with transfer learning, and fine-tune safely in Keras. Includes NumPy augmentation demos.

Small image datasets make CNNs overfit: the network memorizes the few thousand training pictures instead of learning what a cat looks like. Two techniques fix most of this. Data augmentation shows the network randomly altered copies of each image so it cannot memorize them. Transfer learning starts from a network that was already trained on millions of images and adapts it to your task, so you need far less data and far less training time.

TensorFlow is not available in the browser runner. Keras code is read-only; the runnable blocks demonstrate the ideas with NumPy.

Data augmentation

If a photo of a dog is flipped left to right, it is still a dog. Cropping slightly, rotating a few degrees or changing brightness also keep the label. Augmentation applies such random changes during training, so each epoch the network sees slightly different images.

Python Augmenting an image with NumPy

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The same original image gives three different training samples. In Keras you build this as preprocessing layers at the front of the model. They are active only while training and do nothing at prediction time.

Python
import keras
from keras import layers

augment = keras.Sequential([
    layers.RandomFlip("horizontal"),
    layers.RandomRotation(0.05),      # +/- 5% of a full turn
    layers.RandomZoom(0.1),
    layers.RandomContrast(0.2),
])

inputs = keras.Input(shape=(160, 160, 3))
x = augment(inputs)
x = layers.Rescaling(1 / 255)(x)
# ... convolution blocks ...

Choose augmentations that keep the label true. Horizontal flips are fine for animals but not for text or digits (a flipped 2 is not a 2). Vertical flips suit satellite photos but not street scenes.

Why augmentation works: a count

How much does it help? With 100 images and 6 random choices per image, the network effectively sees thousands of distinct inputs. Here is a rough count for a flip, every 4x4 crop of a 6x6 image and five brightness levels.

Python How many variants from one image?

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Transfer learning

A CNN trained on ImageNet (1.2 million images, 1000 classes) has learned general-purpose visual features: edges, textures, shapes, parts of objects. Those features are useful for almost any image task. The recipe:

  1. Load a pretrained base model without its original classifier (include_top=False).
  2. Freeze it (trainable = False) so its weights do not change.
  3. Add a new small head for your classes and train only the head.
  4. Optionally fine-tune: unfreeze the top layers of the base and train a little more with a tiny learning rate.
flowchart LR
    A["Your images"] --> B["Pretrained base<br/>(frozen)"]
    B --> C["GlobalAveragePooling2D"]
    C --> D["Dropout"]
    D --> E["New Dense head<br/>(trained)"]
    E --> F["Class probabilities"]

Only the right-hand part learns at first; the base acts as a fixed feature extractor.

Python
base = keras.applications.MobileNetV2(
    input_shape=(160, 160, 3),
    include_top=False,
    weights="imagenet",
)
base.trainable = False

inputs = keras.Input(shape=(160, 160, 3))
x = augment(inputs)
x = keras.applications.mobilenet_v2.preprocess_input(x)
x = base(x, training=False)          # keep BatchNorm in inference mode
x = layers.GlobalAveragePooling2D()(x)
x = layers.Dropout(0.2)(x)
outputs = layers.Dense(5, activation="softmax")(x)

model = keras.Model(inputs, outputs)
model.compile(optimizer=keras.optimizers.Adam(1e-3),
              loss="sparse_categorical_crossentropy",
              metrics=["accuracy"])
model.fit(train_ds, validation_data=val_ds, epochs=5)

Two details matter. Each pretrained model expects its own input scaling, so use that model's preprocess_input (MobileNetV2 wants values in the range -1 to 1). And call the base with training=False so that its BatchNormalization layers keep their learned statistics.

Fine-tuning

After the new head has converged, unfreeze the last part of the base and continue with a learning rate about 10 to 100 times smaller, so you nudge the pretrained weights instead of destroying them.

Python
base.trainable = True
for layer in base.layers[:-30]:      # keep early layers frozen
    layer.trainable = False

model.compile(optimizer=keras.optimizers.Adam(1e-5),
              loss="sparse_categorical_crossentropy",
              metrics=["accuracy"])      # recompile after changing trainable
model.fit(train_ds, validation_data=val_ds, epochs=5)

You must recompile after changing trainable. Early layers hold generic features and are rarely worth unfreezing; later layers are more task-specific.

Why freezing is cheap: a stand-in

When the base is frozen you can compute its features once and train only a small classifier on them. This NumPy demo mimics that: a fixed random "feature extractor" and a trained logistic head on two classes.

Python Frozen features + trained head

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Only 17 numbers were learned, while the frozen extractor holds 128. Real transfer learning has the same shape at a larger scale, for example 2.2 million frozen weights and a head of a few thousand.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. When are Keras random augmentation layers (RandomFlip, RandomRotation) active?

    They randomize inputs while fitting and pass images through unchanged when predicting.
  2. 2. Which augmentation would be a bad idea for a digit classifier?

    A flipped digit can look like a different symbol, so the label would no longer be correct.
  3. 3. What does include_top=False do when loading a pretrained model?

    You keep the convolutional base and replace the final classification layers.
  4. 4. What must you do after changing layer.trainable and before calling fit again?

    compile() captures which weights are trainable, so changes only take effect after recompiling.
  5. 5. Why use a smaller learning rate when fine-tuning?

    Large updates would quickly destroy the useful features the base already learned.

Summary

  • Augmentation creates varied training images that keep their label; pick transforms that make sense for your data.
  • Keras preprocessing layers only act in training mode.
  • Transfer learning reuses a pretrained base: freeze it, train a new head, optionally fine-tune the top with a tiny learning rate.
  • Use the base model's preprocess_input and call it with training=False.
  • Recompile after changing trainable.

Next: Text Vectorization and Embeddings