Skip to content
Tutorials Center
My learning

Topics

Lesson 4 of 46 ยท Images as Data

Resolution, Sampling and Noise

See how sampling creates pixels, why downsampling causes aliasing, compare nearest and bilinear interpolation, and generate Gaussian and salt-and-pepper noise.

The real world is continuous; an image is not. A camera measures light at a finite grid of points, and each measurement is rounded to one of 256 levels. Two things follow. First, detail finer than the grid is lost or, worse, disguised as something else. Second, every measurement carries a little random error. Understanding sampling and noise tells you what a pixel can and cannot tell you, and why almost every vision pipeline starts by resizing or smoothing.

This lesson covers resolution, downsampling and the aliasing it can cause, the two simplest interpolation methods, the common noise models, and the signal-to-noise ratio. It builds on Images as Arrays.

Sampling and resolution

Resolution is the number of pixels, for example 1920 by 1080. More pixels capture finer detail, but also cost memory and time: doubling width and height quadruples the pixel count. A vision system should use the smallest resolution that still shows what it needs to see. Detecting a face may need 64 by 64 pixels; reading small print may need thousands.

The Nyquist idea says how fine a grid must be: to capture a repeating pattern you need at least two samples per cycle. With fewer, the pattern cannot be represented, and it gets misread as a different, slower pattern. This false pattern is called aliasing. You have seen it as moire stripes on a striped shirt on TV, or a wheel that seems to spin backward.

flowchart LR
    A["Fine scene<br/>stripes every 2 px"] -->|"sample every 2nd pixel"| B["Aliased result<br/>all stripes look the same"]
    A -->|"blur first, then sample"| C["Smooth gray<br/>no false pattern"]

Sampling a pattern that is too fine invents a false one; blurring before sampling avoids that.

The simplest way to shrink an image is to keep every n-th pixel with a slice. It is fast, and it demonstrates aliasing perfectly.

Python Aliasing from naive downsampling

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The original alternates 0, 1, 0, 1. Keeping every second column gives all zeros: the image turns black, a completely wrong summary. Averaging each pair first gives a steady 0.5, which is the honest answer at that resolution. Production resizers do this kind of low-pass filtering for you (area averaging, or Gaussian blur before subsampling). We study blurring in Blurring and Denoising.

Interpolation: filling in between pixels

When you enlarge an image, or rotate or warp it (see Geometric Transforms), you need values at positions that fall between the original pixels. Interpolation estimates them.

  • Nearest neighbor copies the closest pixel. It is fast and never creates new values, but produces blocky edges.
  • Bilinear blends the four surrounding pixels with weights based on distance. The result is smooth, but edges blur slightly.
  • Bicubic uses a 4 by 4 neighborhood for even smoother results (mentioned here for completeness).

For a point at fractional position (y, x), let y0, x0 be the integer parts and dy, dx the fractions. Bilinear interpolation is

v = (1-dy)(1-dx)ยทa + (1-dy)ยทdxยทb + dyยท(1-dx)ยทc + dyยทdxยทd

where a, b, c, d are the top-left, top-right, bottom-left and bottom-right neighbors.

Python Nearest versus bilinear upscaling

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Nearest neighbor repeats existing values (only 0, 100 and 200 appear). Bilinear creates in-between values such as 25, 75, 125 and 175 along the first row, producing a smooth ramp. For label images such as segmentation masks, always use nearest: an in-between class number like 1.5 means nothing.

Noise

No sensor is perfect. Light arrives as discrete photons (shot noise), electronics add their own wobble (read noise), and dim scenes force the camera to amplify the signal, amplifying the noise too. Two simple models cover most of what you need:

  • Gaussian noise: every pixel gets a random offset drawn from a bell curve with standard deviation sigma. It is a good stand-in for sensor noise. Add it in float, then clip to the valid range.
  • Salt-and-pepper noise: a small fraction of pixels is replaced by pure black or pure white, as from dead pixels or transmission errors.
Python Generating Gaussian and salt-and-pepper noise

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Always use a seeded generator (np.random.default_rng(seed)) in experiments, so results can be reproduced. The two noise types call for different cures: averaging helps with Gaussian noise, while a median filter is better against salt and pepper (see Blurring and Denoising).

Signal-to-noise ratio

How noisy is "too noisy"? The signal-to-noise ratio (SNR) compares the strength of the useful signal with the noise. A simple version is

SNR = mean of the signal / standard deviation of the noise

or in decibels 20 ยท log10(SNR). Another common measure is PSNR (peak signal-to-noise ratio) for comparing a processed image with a reference: 10 ยท log10(255ยฒ / MSE), where MSE is the mean squared error between the two images.

Python Measuring noise with SNR and PSNR

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The last two lines show why astronomers and phone cameras combine many exposures: averaging 16 frames cuts the noise to a quarter (the square root of 16), at no loss of detail if the scene is static.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What is aliasing?

    When the sampling grid is too coarse for the detail in the scene (fewer than two samples per cycle), fine patterns are misrepresented as coarser ones.
  2. 2. Which interpolation method should be used when resizing a segmentation mask whose pixel values are class numbers?

    Nearest neighbor never invents new values. Blending class IDs (for example 1 and 3 into 2) would produce meaningless labels.
  3. 3. A median filter is usually the best fit for which noise?

    Salt-and-pepper pixels are extreme outliers, which a median ignores. Averaging filters would smear them into neighbors.
  4. 4. You average 16 independent noisy frames of a static scene. By roughly what factor does the noise standard deviation drop?

    Independent noise averages down by the square root of the number of frames: sqrt(16) = 4.
  5. 5. Why should you clip after adding Gaussian noise to a uint8-range image?

    Noise can push values below 0 or above 255. Clipping keeps the image valid and avoids wrap-around if you cast back to uint8.

Summary

  • Sampling turns a continuous scene into a pixel grid; detail finer than two samples per cycle is lost or aliased into false patterns.
  • Reduce resolution by averaging (low-pass first) rather than just dropping pixels.
  • Nearest-neighbor interpolation keeps original values (use it for label masks); bilinear blends four neighbors for smooth results.
  • Gaussian noise models sensor wobble; salt-and-pepper noise models stuck pixels. Add noise in float, clip, and seed your random generator.
  • SNR and PSNR quantify noise, and averaging N frames reduces noise by the square root of N.

Next, start changing pixel values on purpose: Brightness, Contrast and Gamma.