Skip to content
Tutorials Center
My learning

Topics

Lesson 28 of 38 ยท Sparse, Spatial and Image Data

Image Processing with scipy.ndimage

Smooth, sharpen and find edges with scipy.ndimage, label and measure objects, clean masks with morphology, and rotate or zoom images, all on a synthetic image.

A grayscale image is just a 2D array of numbers, so every array tool you know already applies to it. scipy.ndimage adds the operations that are special to images and other gridded data: neighborhood filters, edge detection, labeling of connected regions, morphology and geometric transforms. The name says "n-dimensional", and the functions work equally on 1D signals, 2D photos and 3D medical volumes.

In this lesson we avoid external image files and build a synthetic image (a few bright discs on a dark background plus noise). That keeps every example self-contained and lets us check results against the truth we put in. If you want to load real pictures from disk, see Image Processing with Pillow; once an image is a NumPy array, everything below applies.

flowchart LR
    A["Noisy image<br/>2D array"] --> B["gaussian_filter<br/>remove noise"]
    B --> C["threshold<br/>img > t"]
    C --> D["binary_opening<br/>clean the mask"]
    D --> E["label<br/>number each blob"]
    E --> F["measure<br/>sum, center_of_mass"]

A typical pipeline: smooth, threshold, clean, label, measure.

Building a test image

We draw filled discs by comparing each pixel's squared distance from a center with the squared radius. np.mgrid (or np.ogrid) produces the coordinate grids that make this easy.

Python A synthetic noisy image

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Smoothing and filters

The workhorse is gaussian_filter(img, sigma), which replaces every pixel by a weighted average of its neighbors, with weights shaped like a bell curve. A larger sigma blurs more. Because noise is random from pixel to pixel while objects are smooth, averaging suppresses noise much more than it destroys objects.

Other filters in the same family:

  • uniform_filter averages over a square window ("box blur").
  • median_filter takes the median of a window; it removes isolated bad pixels ("salt and pepper") while keeping edges sharp.
  • maximum_filter and minimum_filter take the largest or smallest value in the window.
  • generic_filter applies any Python function to each window (flexible, but slow).
Python Comparing smoothing filters

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The numbers show the trade-off: all three smoothers reduce the error, and the median filter wins when the damage is a few extreme pixels rather than gentle noise. The mode argument controls what happens at the border ("reflect" by default mirrors the image), which matters for pixels near the edge.

Finding edges

An edge is a place where brightness changes quickly, so edge detection means estimating the gradient. sobel(img, axis=...) computes the derivative along one axis with a built-in bit of smoothing. Combine the two directions with np.hypot to get the gradient magnitude, which is large at edges regardless of their orientation. Differentiating amplifies noise, so smooth first (gaussian_gradient_magnitude does both in one step).

Python Sobel edges

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The strong responses sit at distances close to 25 from the center, exactly where the disc's boundary is. Smoothing first gives a clean ring instead of speckle. Other useful tools here are laplace (second derivative, finds edges as zero crossings) and prewitt, a cousin of sobel.

Labeling and measuring objects

To count or measure objects you first need a binary mask (True for object pixels), usually produced by a threshold. ndi.label(mask) then numbers each connected group of True pixels with 1, 2, 3, ... and returns (labels, count). The default connectivity treats only up/down/left/right neighbors as connected; pass structure=np.ones((3, 3)) to include diagonals.

Measurement functions take the original image, the label array and a list of labels: ndi.sum_labels (total intensity), ndi.center_of_mass, ndi.mean, ndi.maximum, plus ndi.find_objects for bounding slices.

Python Count and measure the discs

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

We planted four discs of radii 10, 8, 6 and 12 at known centers, and the pipeline recovers them: the labels are numbered in scan order (top to bottom), and the equivalent radii and centers come out close to the truth. Small deviations are expected because smoothing and thresholding trim the edges slightly.

Morphology: cleaning masks

A threshold of a noisy image leaves stray pixels and small holes. Morphological operations use a small shape, the structuring element, to repair them.

  • binary_erosion shrinks objects (a pixel survives only if the whole element fits).
  • binary_dilation grows them.
  • binary_opening is erosion followed by dilation: it deletes specks smaller than the element and keeps larger shapes about the same size.
  • binary_closing is dilation then erosion: it fills small holes.
  • binary_fill_holes fills every hole completely enclosed by an object.
Python Opening, closing and filling

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Opening and closing are the pair to remember: opening removes small bright specks, closing removes small dark holes. The distance transform distance_transform_edt gives each object pixel its distance to the nearest background pixel, a building block for splitting touching objects.

Rotating, zooming and shifting

Geometric transforms resample the image onto a new grid, which requires interpolation between pixels. The order argument picks the interpolating spline: 0 is nearest neighbor (no new values, good for label images), 1 bilinear, 3 cubic (the default, smooth, and may overshoot slightly past the original value range).

  • rotate(img, angle, reshape=True) rotates by degrees; with reshape=True the output grows so nothing is cut off.
  • zoom(img, factor) scales by a factor and returns an array of the new shape.
  • shift(img, (dy, dx)) translates by possibly fractional pixels.
  • map_coordinates samples at arbitrary positions: the general tool behind all the others. For the maths of interpolation itself see Multidimensional and Scattered-Data Interpolation.
Python Rotate, zoom and shift

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

When you transform a label image, always use order=0. Interpolating between label 1 and label 3 would invent a label 2 along the boundary, which is not meaningful.

Custom warps with geometric_transform

rotate, zoom and shift are ready-made. For any other warp, ndi.geometric_transform(image, mapping, order=1) lets you write the geometry yourself. The twist is that you give the mapping backwards: for every pixel (row, col) of the output, mapping returns the coordinates in the input image where the value should be read from, and ndimage interpolates there. (This avoids holes in the output.) The mapping receives a tuple of output coordinates and must return a tuple of input coordinates. map_coordinates does the same with whole coordinate arrays and is faster when you can build the arrays in NumPy.

Cropping, flipping and sharpening

Many image tasks need no special function at all, because an image is simply an array: cropping is slicing, flipping is a reversed slice (np.flipud, np.fliplr), and writing raw pixels to disk is arr.tofile(...) with np.fromfile(...) to read them back (you must remember the shape and dtype yourself, as raw files have no header). SciPy adds the filters. A classic way to sharpen is the unsharp mask: blur the image, subtract the blur from the original to get the "detail" layer, and add a multiple of that detail back.

Python Crop, flip, a custom warp, unsharp mask and raw files

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The crop is a view of the original (no copy of the pixels), the wave mapping reads each output pixel from a sine-shifted column of the input, and sharpening raises the edge contrast measured by the Sobel filter. The raw file holds exactly one byte per pixel, which is why you must save the shape separately. Image files with headers (PNG, JPEG) are read and written with matplotlib.pyplot.imread/imsave, imageio or Pillow rather than by SciPy.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Which filter is best at removing isolated extreme pixels (salt-and-pepper noise) while keeping edges sharp?

    The median of a window ignores a few extreme values. Averaging filters like gaussian_filter and uniform_filter spread them into their neighbors instead.
  2. 2. Why smooth an image before computing Sobel edges?

    Gradients respond strongly to random pixel-to-pixel variation. A light Gaussian blur suppresses that noise, so only real edges stand out.
  3. 3. What does labels, count = ndi.label(mask) return?

    label numbers each connected component 1..count (background stays 0), which you then pass to sum_labels, center_of_mass and similar measurement functions.
  4. 4. Which operation removes small bright specks from a binary mask while keeping large shapes roughly the same size?

    Opening is erosion followed by dilation. The erosion deletes specks smaller than the structuring element and the dilation restores the surviving shapes.
  5. 5. You rotate an image of integer labels. Which interpolation order should you pass?

    Nearest-neighbor interpolation (order=0) only copies existing values, so no invented in-between labels appear.

Summary

  • Images are plain arrays; scipy.ndimage provides the neighborhood operations: gaussian_filter, median_filter, uniform_filter, maximum_filter and friends.
  • Edge detection means estimating gradients: sobel per axis, combined with np.hypot, ideally after smoothing (or with gaussian_gradient_magnitude).
  • The classic measurement pipeline is smooth, threshold, clean with morphology, label, then sum_labels, center_of_mass, find_objects.
  • Opening removes small bright specks, closing removes small dark holes, binary_fill_holes fills enclosed holes.
  • rotate, zoom and shift resample with spline interpolation; use order=0 for labels and masks.

Next up: the special functions behind statistics and numerical code, plus k-means clustering, in Special Functions and Clustering.