Skip to content
Tutorials Center
My learning

Topics

Lesson 14 of 54 ยท Geometric Transformations

Image Pyramids: pyrDown, pyrUp and Laplacian Pyramids

Build Gaussian and Laplacian image pyramids with pyrDown and pyrUp, see why blurring comes before shrinking, and use pyramids for multi-scale search and seamless blending.

How big is the thing you are looking for? A face might fill the whole picture or only a few dozen pixels. A fixed-size detector or template cannot handle both. One of the oldest tricks in computer vision is to look at the picture at many sizes at once: a stack of copies of the same image, each half the width and height of the one before. That stack is called an image pyramid, because drawn from the largest at the bottom to the smallest at the top it has the shape of one.

Pyramids are used to find objects at any scale (the sliding-window idea behind Face Detection with Haar Cascades), to speed up algorithms such as optical flow and registration by working coarse-to-fine, and to blend two images without visible seams. In this lesson you will learn the two OpenCV functions pyrDown and pyrUp, why a blur is built into them, how a Laplacian pyramid stores the detail that was lost, and how to use it for blending. Resizing in general is covered in Resizing, Cropping and Flipping and the filters in Blurring and Noise Reduction.

Gaussian pyramids: pyrDown and pyrUp

A Gaussian pyramid is built by repeating two steps: blur the image with a small Gaussian kernel, then throw away every second row and every second column. The result has a quarter of the pixels. cv2.pyrDown does both steps in one call.

Python
import cv2

img = cv2.imread("photo.jpg")           # for example 800 x 600
lower = cv2.pyrDown(img)                # 400 x 300
lowest = cv2.pyrDown(lower)             # 200 x 150
print(img.shape, lower.shape, lowest.shape)

cv2.pyrUp goes the other way: it doubles the size by inserting zero rows and columns and then blurs, filling in the gaps. By default the output size is exactly twice the input. If the original had an odd size, pass a dstsize to get it back.

Python
up = cv2.pyrUp(lower)                   # back to 800 x 600

Going down and then up does not restore the original. pyrDown throws away information (every other pixel and the fine detail), and pyrUp can only guess. The round trip gives a blurry copy. That loss is not a defect; it is the key to the next idea.

Why blur before shrinking?

If you simply keep every second pixel, thin features and fine textures shrink into false patterns, the same effect as a wagon wheel that appears to spin backwards on film. This is aliasing, and it also makes edges look jagged. Blurring first removes the fine detail that the smaller image cannot represent. This is why pyrDown blurs, and why cv2.resize with INTER_AREA (which averages) is the recommended choice for shrinking; see Interpolation and Image Quality.

Building a whole pyramid

A loop collects the levels:

Python
def gaussian_pyramid(img, levels):
    pyramid = [img]
    for _ in range(levels):
        img = cv2.pyrDown(img)
        pyramid.append(img)
    return pyramid

levels = gaussian_pyramid(img, 4)
for i, level in enumerate(levels):
    print(i, level.shape)

Level 0 is the original. Each further level halves the width and height, so after four steps the image is only 1/16 of the width. The total memory of all levels is only about one third more than the original, because 1 + 1/4 + 1/16 + ... converges to 4/3.

What pyramids are good for

  • Multi-scale search. Run a detector or template match on every level, so an object of any size matches at one of them.
  • Coarse-to-fine algorithms. Optical flow and image registration find a rough answer on the tiny top level, where large motions look small, then refine it on each larger level. This is both faster and more robust.
  • Speed. An expensive filter on a quarter-size image costs a quarter as much; many pipelines process a downscaled copy and map the results back.
  • Blending and compression. The Laplacian pyramid below.

Laplacian pyramids

Each Gaussian level has lost detail compared with the level below it. The Laplacian pyramid keeps exactly that lost detail. For level i, you take the Gaussian image G_i, enlarge the next smaller level G_{i+1} with pyrUp, and subtract:

L_i = G_i - pyrUp(G_{i+1})

Each L_i is mostly zero with edges and texture at that scale, like a band of the picture's detail. The last level is kept as it is (the smallest Gaussian image).

Python
def laplacian_pyramid(img, levels):
    gauss = gaussian_pyramid(img, levels)
    lap = []
    for i in range(levels):
        size = (gauss[i].shape[1], gauss[i].shape[0])
        expanded = cv2.pyrUp(gauss[i + 1], dstsize=size)
        lap.append(cv2.subtract(gauss[i], expanded))      # saturated subtract
    lap.append(gauss[-1])
    return lap

The pyramid is invertible: start with the smallest level, pyrUp, and add the detail layer, repeating until you reach the top. With floating-point arithmetic (and no saturation) you get the original back exactly, which is why this representation was used in early image compression: the detail layers are mostly small numbers that compress well.

Python
def reconstruct(lap):
    img = lap[-1]
    for detail in reversed(lap[:-1]):
        size = (detail.shape[1], detail.shape[0])
        img = cv2.add(cv2.pyrUp(img, dstsize=size), detail)
    return img

With uint8 images, saturated subtraction clips negative detail to zero, so use np.float32 or cv2.CV_16S for exact work.

Blending with pyramids

A plain cross-fade (cv2.addWeighted) of two images with a sharp seam shows a visible line, and a wide feathered mask makes ghosts. Pyramid blending gets a clean result by blending each frequency band separately:

  1. Build Laplacian pyramids LA and LB of the two images.
  2. Build a Gaussian pyramid GM of the mask (white where image A should show).
  3. At every level, combine: L = GM * LA + (1 - GM) * LB. Fine details switch over at a sharp seam while the coarse tones, which live on the upper levels, are mixed over a wide transition.
  4. Reconstruct from the blended pyramid.

This is the method behind the famous "apple and orange" demo and is used by panorama software to hide seams; compare with the approach in Project: Panorama Stitching. OpenCV also offers cv2.detail_MultiBandBlender for stitching, and cv2.seamlessClone for Poisson blending when you paste an object into a scene.

Try it: one pyramid level in NumPy

The stand-in below does what pyrDown does: blur with the 5-tap binomial kernel [1 4 6 4 1] / 16 (OpenCV's Gaussian approximation), then keep every second sample. A 1-D signal with a fine alternating pattern shows why the blur matters.

Python Blur then subsample, and why

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The naive version keeps only the zeros of the alternating pattern and reports "nothing there". The blurred version keeps the average brightness. The round trip is close but not equal: the fine alternating detail is gone.

Try it: a Laplacian pyramid that reconstructs exactly

This second model builds a two-level Laplacian pyramid on a 1-D signal using floats, then reconstructs it. Because the detail layer stores exactly what pyrUp could not guess, the reconstruction error is zero.

Python Laplacian pyramid and exact reconstruction

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

What is real: pyrDown and pyrUp are a Gaussian blur plus a 2x resample, the Laplacian pyramid is the difference of levels and is invertible, and the blending recipe is the standard multi-band algorithm. What is the stand-in: the NumPy code works on 1-D signals with the same 5-tap kernel; real OpenCV works on 2-D images and handles borders and odd sizes with dstsize.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What does cv2.pyrDown do?

    pyrDown smooths with a Gaussian kernel and then downsamples by 2 in both directions, giving a quarter of the pixels.
  2. 2. Why is pyrUp(pyrDown(img)) not equal to the original?

    Information is lost when every other pixel is dropped. pyrUp can interpolate but not recover the detail.
  3. 3. What does a Laplacian pyramid level store?

    L_i = G_i - pyrUp(G_{i+1}). Together with the smallest Gaussian level it lets you rebuild the image exactly.
  4. 4. Why does shrinking without blurring cause aliasing?

    Dropping samples without removing the fine detail folds it into lower frequencies, producing jagged edges and moire.
  5. 5. How is pyramid blending of two images done?

    Each frequency band is blended with its own mask level, so fine detail changes over at a sharp seam while coarse tones mix smoothly.

Summary

  • An image pyramid is a stack of the same image at decreasing sizes. cv2.pyrDown blurs and halves; cv2.pyrUp doubles and smooths.
  • The blur before shrinking prevents aliasing. A pyrDown/pyrUp round trip is lossy and returns a blurry image.
  • A Laplacian pyramid stores the detail lost between levels, L_i = G_i - pyrUp(G_{i+1}), and can rebuild the image exactly when no clipping happens.
  • Use pyramids for scale-independent search, coarse-to-fine alignment and optical flow, faster processing on smaller copies, and seamless multi-band blending.

Next: turn grayscale images into black and white regions with Thresholding.