Skip to content
Tutorials Center
My learning

Topics

Lesson 14 of 46 ยท Features and Matching

Image Pyramids and Scale Space

Build Gaussian and Laplacian pyramids from scratch, rebuild an image exactly, and find blobs of any size with scale-normalized Laplacian of Gaussian.

The same object can fill your whole photo or only a handful of pixels, depending on how far away the camera was. A face detector, a feature matcher or a template search that only works at one size is not much use. Vision systems therefore look at an image at several scales at once: they make smaller and smaller copies, search each copy with the same fixed-size tool, and remember which scale gave the best answer.

This lesson builds that machinery from the pieces you already know. You will construct a Gaussian pyramid (a stack of blurred, shrinking images), turn it into a Laplacian pyramid that stores detail per scale and can rebuild the original exactly, and then use the Laplacian of Gaussian to detect blobs and measure their size. It builds on blurring, convolution and downsampling.

Why one scale is not enough

Imagine a detector that looks at a 9 by 9 window and asks, "is there a blob here?" It will notice a blob that is about 9 pixels wide. A blob 60 pixels wide looks like a flat patch inside that window, and a blob 2 pixels wide looks like noise. You have two ways to fix this:

  1. Keep the image fixed and make the window larger and larger (costly, and every window size needs its own tuning).
  2. Keep the window fixed and make the image smaller and smaller.

The second option is the classic answer. Shrinking the image by a factor of two turns a 60-pixel blob into a 30-pixel one, then a 15-pixel one, then a 7-pixel one, and at that point the 9-pixel window sees it clearly. The stack of shrinking copies is called an image pyramid.

flowchart TB
    L0["Level 0: 64 x 64<br/>full resolution"] --> L1["Level 1: 32 x 32"]
    L1 --> L2["Level 2: 16 x 16"]
    L2 --> L3["Level 3: 8 x 8<br/>smallest"]

Each level is a blurred copy of the previous one with every second row and column removed.

The Gaussian pyramid

Shrinking is not just throwing away rows. If you keep every second pixel of a sharp image, fine patterns can turn into false coarse ones (this is aliasing, see Resolution, Sampling and Noise). The cure is to blur first, so that detail finer than the new pixel size is removed before it can be misread. One pyramid step is therefore:

  1. Blur with a Gaussian (a sigma of about 1 works well for halving).
  2. Keep every second row and every second column: img[::2, ::2].

Repeat on the result to get level 2, then level 3, and so on. The next demo builds four levels of a small scene, prints the shapes, and compares the blurred route with plain subsampling.

Python A Gaussian pyramid

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice what the numbers say. The shapes halve each time, so the whole pyramid holds only about one third more pixels than level 0 (1 + 1/4 + 1/16 + ...). The mean brightness stays almost the same, because blurring averages instead of discarding. The maximum drops at the smallest level: the tiny disc has been smeared into its neighbours, which is exactly what "this detail does not exist at this scale" looks like.

Real libraries use a slightly different blur (a 5-tap binomial kernel) and a matching upsampling filter. The principle is identical: low-pass first, then discard.

The Laplacian pyramid

A Gaussian pyramid keeps the coarse picture at each scale. What each step throws away is the fine detail. The Laplacian pyramid stores exactly that: each band is a Gaussian level minus an enlarged copy of the next, coarser level.

flowchart LR
    G0["G0"] --> S["subtract"]
    G1["G1 (blurred, half size)"] --> U["upsample to G0 size"] --> S
    S --> B0["band L0 = detail lost between G0 and G1"]

A Laplacian band is what is left when you remove the blurry, enlarged version from the sharper image.

The last entry of the pyramid is simply the smallest Gaussian level, so no information is missing. To rebuild the image, start from the coarsest level, enlarge it, add the next band, enlarge again, add the next band, and so on. Because each band was defined as a difference with the same enlargement function, the sums cancel and the reconstruction is exact up to floating-point rounding.

Python Laplacian pyramid and exact reconstruction

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The bands are mostly zero with bright and dark lines where edges are, because a flat area looks the same at both scales and cancels out. Band 0 holds the finest edges, band 1 slightly coarser ones, and so on. This representation powers image compression (most band values are tiny and cheap to store), seamless image blending (blend each band separately with a smoother mask at coarser scales) and denoising.

Scale invariance and the Laplacian of Gaussian

Pyramids reduce the size of the problem; scale space goes one step further and treats the amount of blur, sigma, as a continuous third axis. Stack the image blurred with many sigmas and you get a 3D array: row, column and scale. A feature that is stable across scales can be found at the place where some filter response peaks along that third axis.

The filter for blobs is the Laplacian of Gaussian (LoG): blur with a Gaussian, then take the Laplacian (the sum of the second derivatives in y and x, the same operator you meet in edge detection). Applied to a bright blob it gives a strong negative response in the centre, because the centre is brighter than the average of its surroundings.

There is one catch. A bigger sigma smooths more, so raw LoG responses shrink as sigma grows and scales cannot be compared. Multiplying by sigma squared fixes this and gives the scale-normalized LoG, sigma**2 * LoG. For a flat bright disc of radius r, the normalized response at the centre is strongest when

TEXT
sigma = r / sqrt(2)

so finding the best sigma tells you the radius: r = sigma * sqrt(2). In scipy, ndi.gaussian_laplace(img, sigma) computes the LoG directly. The demo checks the formula on two discs and then finds several blobs of different sizes in one image by looking for maxima in the three-dimensional (scale, row, column) array.

Python Blob detection across scales

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The true radii were 4, 8, 14 and 6, and the estimates land close to them. Small errors come from the pixel grid and from the sigma step of 0.5. The size=(3, 7, 7) window in maximum_filter means a point is a peak only if it beats its neighbours in space and in scale; that is what makes the detection scale-aware. You will see the same idea again in the next lesson on corners, and it is the core of the SIFT detector, which uses a difference of Gaussians as a fast approximation to the LoG.

The pyramid also gives you a simple recipe for searching at every size:

  1. Build the pyramid.
  2. Run the same fixed-size detector (a template, a classifier window, a corner detector) at every level.
  3. Map each hit back to full-resolution coordinates: a hit at level k at position (y, x) is at (y * 2**k, x * 2**k) in the original, with a size multiplied by 2**k.
  4. Merge overlapping hits, usually with non-maximum suppression (you will meet it in object detection).

Levels that differ by a factor of two are often too coarse for good matching. Practical systems insert extra levels per octave, for example a ratio of about 1.2 between neighbours, trading speed for finer scale coverage.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why do we blur before keeping every second row and column in a pyramid step?

    Plain subsampling can fold fine detail into false coarse patterns. A Gaussian low-pass filter removes that detail first.
  2. 2. What does a Laplacian pyramid band contain?

    Each band is a Gaussian level minus an enlarged version of the next level, so it holds the detail at that scale. Together with the coarsest level it rebuilds the image.
  3. 3. A bright disc has radius 12 pixels. At roughly which sigma is the scale-normalized LoG response at its centre strongest?

    For a flat disc the peak lies at sigma = r / sqrt(2). Inverting that gives the radius from the best sigma: r = sigma * sqrt(2).
  4. 4. Why is the LoG multiplied by sigma squared before comparing scales?

    More smoothing flattens the second derivative, so un-normalized responses shrink with sigma. The sigma squared factor cancels that trend.

Summary

  • An image pyramid is a stack of progressively smaller, blurred copies; one step is Gaussian blur followed by img[::2, ::2].
  • Blurring first prevents aliasing; the whole pyramid costs only about one third more memory than the image.
  • A Laplacian pyramid stores the detail lost at each step; adding the bands back from the coarsest level rebuilds the original exactly.
  • Scale space treats sigma as a third axis; the scale-normalized LoG sigma**2 * LoG peaks at sigma = r / sqrt(2) for a disc of radius r.
  • Looking for maxima over (scale, row, column) finds blobs and estimates their size; multi-scale search reuses one fixed-size detector on every level.

Next, find the points that stay recognizable across scales and views in Corner Detection.