Skip to content
Tutorials Center
My learning

Topics

Lesson 22 of 40 ยท Convolutional Networks

Padding, Strides and Pooling

Control CNN output sizes with padding and strides, use max and average pooling, and compute shapes with the output-size formula. NumPy demos and Keras Conv2D code.

How convolution works showed that a plain convolution shrinks the image and treats every position the same. Three settings let you shape the result: padding decides how the border is handled, strides decide how far the kernel jumps, and pooling layers shrink feature maps on purpose. Together they determine every tensor shape in a CNN, and shape mismatches are the most common CNN error you will see.

Padding

Without padding, border pixels are covered by fewer windows than central pixels, and each layer loses k - 1 rows and columns. Padding adds a frame of zeros around the input first. Keras offers two choices through the padding argument:

  • "valid" means no padding. The output shrinks.
  • "same" pads so that, at stride 1, the output has the same height and width as the input.
Python Zero padding with np.pad

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Look at the corners of the "same" output: they are smaller than the real average because zeros were averaged in. That border effect is the price of keeping the size.

Strides

The stride is how many pixels the window moves each step. Stride 1 visits every position. Stride 2 skips every other position and roughly halves the output. A strided convolution is a learnable way to downsample.

Python Stride 1 versus stride 2

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The output-size formula

For one axis, with input size n, kernel size k, padding p on each side and stride s:

output = floor((n + 2p - k) / s) + 1

For Keras, "valid" means p = 0. With "same" the output is ceil(n / s), so only stride matters. For example n = 28, k = 3:

Setting Output
valid, stride 1 26
same, stride 1 28
valid, stride 2 13
same, stride 2 14
flowchart LR
    A["28x28x1 input"] --> B["Conv2D 8 filters, 3x3, same"]
    B --> C["28x28x8"]
    C --> D["MaxPooling2D 2x2"]
    D --> E["14x14x8"]
    E --> F["Conv2D 16 filters, 3x3, valid"]
    F --> G["12x12x16"]

Shapes after each layer for a batch element. Channels equal the filter count of the last convolution.

Pooling

A pooling layer has no weights. It slides a window over each channel separately and summarizes it with one number.

  • Max pooling keeps the largest value. It says "was the feature present anywhere in this window?" and gives a small tolerance to shifts.
  • Average pooling keeps the mean and gives a smoother summary.

The common setting is a 2 by 2 window with stride 2, which halves height and width and removes 75 percent of the activations.

Python Max and average pooling

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

A shifted input demonstrates why max pooling gives tolerance: move the 9 one cell and the pooled output barely changes.

Python Pooling tolerates small shifts

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The reshape trick splits each axis into blocks and takes the maximum inside each block. This is also how fast pooling is implemented in practice.

Global pooling

GlobalAveragePooling2D averages each whole feature map down to one number, turning a tensor of shape (batch, h, w, channels) into (batch, channels). It replaces the huge Flatten plus Dense step at the end of modern CNNs and removes thousands of parameters.

In Keras

Python
from keras import layers

layers.Conv2D(32, 3, padding="same", strides=1, activation="relu")
layers.Conv2D(64, 3, padding="valid", strides=2)
layers.MaxPooling2D(pool_size=2)           # stride defaults to pool_size
layers.AveragePooling2D(pool_size=2)
layers.GlobalAveragePooling2D()
layers.Flatten()

Common mistakes: using Flatten on a large feature map (millions of parameters), forgetting that odd sizes lose a row when pooled by 2, and mixing channel orders ("channels_last" is the Keras default).

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. With padding="same" and stride 1, a 3x3 convolution on a 32x32 image gives:

    'same' pads the border so the spatial size is unchanged at stride 1.
  2. 2. A 2x2 max pooling layer with stride 2 is applied to a 12x12x16 tensor. What is the output shape?

    Pooling halves height and width and works per channel, so channels stay at 16.
  3. 3. How many trainable parameters does a pooling layer have?

    Pooling only applies a fixed max or mean, so there are no weights to learn.
  4. 4. What does GlobalAveragePooling2D output for an input of shape (batch, 7, 7, 64)?

    Each 7x7 map is averaged to a single value, leaving one number per channel.

Summary

  • "valid" shrinks the output; "same" keeps the size at stride 1 by padding with zeros.
  • Stride is the step of the window; larger strides downsample.
  • Output size per axis is floor((n + 2p - k) / s) + 1.
  • Max pooling keeps the strongest response and tolerates small shifts; average pooling smooths.
  • Pooling has no parameters; global average pooling replaces Flatten at the end of many CNNs.

Next: Building a CNN with Keras