Skip to content
Tutorials Center
My learning

Topics

Lesson 19 of 54 ยท Image Processing

Edge-Preserving Denoising, Non-Local Means and Inpainting

Remove noise without blurring edges using bilateral filtering and non-local means, then repair scratches and unwanted objects with cv2.inpaint, with runnable NumPy models.

A Gaussian blur removes noise, but it also removes edges, because it averages every neighbor equally. This lesson covers two better tools: the bilateral filter and non-local means, which smooth flat areas while keeping edges sharp. It then moves to a related repair job, inpainting: filling in a missing or damaged region (a scratch, a date stamp, a power line) from its surroundings.

All the OpenCV calls are shown read-only, and NumPy stand-ins let you run the core ideas. Read Blurring and Noise Reduction and Arithmetic, Bitwise Ops and Masking first, since inpainting depends on a mask.

The problem with plain blurring

Take a pixel on the boundary between a dark wall and a bright window. A Gaussian blur averages it with pixels from both sides, so the boundary turns into a gray gradient. What you want is to average the pixel only with neighbors that look like it. Both filters below do exactly that, in different ways.

The bilateral filter

The bilateral filter weights each neighbor by two things at once:

  • Spatial closeness: pixels near the center count more (like a Gaussian blur).
  • Similarity of value: pixels whose brightness (or color) is close to the center pixel's count more; very different pixels count almost nothing.

A neighbor on the other side of an edge is spatially close but very different in value, so it is ignored, and the edge survives. In flat regions every neighbor is similar, so the filter behaves like a normal blur.

Python
import cv2

img = cv2.imread("portrait.jpg")
smooth = cv2.bilateralFilter(img, d=9, sigmaColor=75, sigmaSpace=75)
Parameter Meaning
d Neighborhood diameter in pixels. 5 is fast, 9 is typical, large values are slow. If d is 0 it is computed from sigmaSpace.
sigmaColor How different two colors may be and still be mixed. A larger value mixes more, and edges soften.
sigmaSpace How far away pixels may be and still count. Larger values give a smoother, wider effect.

Rules of thumb: use sigmaColor and sigmaSpace near 10 for a subtle effect and near 150 for a cartoon-like effect. Advantages: edge-preserving, no halos at strong edges, simple to use. Limitations: it is slow compared with a Gaussian (the weights change at every pixel), it can create a "plastic skin" look, and with large parameters it flattens texture you wanted to keep. Applying it several times with small parameters often looks better than once with large ones.

Non-local means denoising

The bilateral filter compares single pixels. Non-local means (NLM) compares small patches. To denoise a pixel, it looks at patches in a wide search window, finds ones that look like the patch around the pixel (even if they are far away), and averages their center pixels. Repeating textures, such as brick or fabric, give many matching patches, so noise cancels while the texture stays intact. It is usually the best classic denoiser available in OpenCV, at the price of speed.

Python
# grayscale
den = cv2.fastNlMeansDenoising(gray, None, h=10, templateWindowSize=7, searchWindowSize=21)

# color: converts to a luma/chroma space internally
den_color = cv2.fastNlMeansDenoisingColored(
    img, None, h=10, hColor=10, templateWindowSize=7, searchWindowSize=21
)
Parameter Meaning
h Filter strength for luminance. Larger removes more noise and more detail.
hColor The same for color components (colored version only). Usually equal to h, and 10 is a good start.
templateWindowSize Side of the patch that is compared. Use an odd number, 7 is typical.
searchWindowSize Side of the area searched for similar patches. Use an odd number, 21 is typical. Cost grows with its square.

A related function, cv2.fastNlMeansDenoisingMulti, uses several consecutive video frames, and cv2.fastNlMeansDenoisingColoredMulti does the same in color, taking advantage of the fact that noise changes from frame to frame but the scene does not.

Which denoiser?

Noise or goal Tool
Salt-and-pepper specks cv2.medianBlur
Mild, fast, any noise cv2.GaussianBlur
Smooth skin and keep edges cv2.bilateralFilter
Best quality for photos, textured areas cv2.fastNlMeansDenoising / Colored
Noisy video fastNlMeansDenoisingMulti

Whatever you choose, denoise before edge detection or thresholding, and compare results with your eyes: all of these trade noise against detail.

Inpainting: filling in missing regions

Inpainting reconstructs pixels you mark as bad using the pixels around them. It was invented to restore damaged photos and film, and today it removes watermarks, date stamps, scratches and small objects. OpenCV's cv2.inpaint needs three things: the image, an 8-bit mask where non-zero pixels mark the area to repair, and a radius.

Python
import cv2

img = cv2.imread("damaged.jpg")
mask = cv2.imread("scratch_mask.png", cv2.IMREAD_GRAYSCALE)   # white = repair here

_, mask = cv2.threshold(mask, 10, 255, cv2.THRESH_BINARY)
mask = cv2.dilate(mask, None, iterations=2)                    # cover the edges of the damage

restored = cv2.inpaint(img, mask, inpaintRadius=3, flags=cv2.INPAINT_TELEA)

There are two algorithms:

  • cv2.INPAINT_TELEA (Telea's fast marching method) fills the region pixel by pixel from its boundary inward, each new pixel being a weighted average of known pixels nearby, with more weight to those close to the boundary and in the direction of the edges.
  • cv2.INPAINT_NS (Navier-Stokes) treats image structure like a fluid: it propagates the direction of brightness lines (isophotes) into the hole, aiming to continue edges.

Both give a similar result on thin damage; try both and keep the better one. inpaintRadius is the neighborhood around each point that is used, so a larger radius gives a smoother fill but can blur.

How to make a good mask: draw it with the mouse (Mouse Events and Trackbars), with a color threshold (for a colored scratch) or from a detector. Dilate it a little, because the damaged region usually extends a few pixels beyond what you marked, and leftover edge pixels poison the fill.

Limits: inpainting copies no texture, it smoothly interpolates. Small, thin damage looks perfect; a large hole becomes a visible smudge. For big objects, modern deep-learning inpainting models do much better, and can be run with the dnn module.

Filters need pixels beyond the image edge. cv2.copyMakeBorder builds them explicitly; see Resizing, Cropping and Flipping.

Try it: a bilateral filter in NumPy

This version works on a 1-D signal with a step edge and noise. It applies a Gaussian blur and a bilateral filter, and prints how much each smears the edge. The weights are the same two Gaussians described above.

Python Bilateral versus Gaussian on a noisy edge

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The Gaussian turns the step into a ramp: the pixels beside the edge land between 40 and 200. The bilateral keeps them near their true side, while both filters reduce the noise on the flat parts.

Try it: simple inpainting by diffusion

The second model repairs a damaged gradient by repeatedly replacing each masked pixel with the average of its four neighbors, which is the simplest diffusion fill. Real cv2.inpaint is smarter about edge direction, but the idea of "fill from the boundary inward" is the same.

Python Fill a hole from its surroundings

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The diffusion recovers the ramp inside the hole almost exactly, because a smooth gradient is easy to interpolate. Try a textured pattern and the error grows, which is the limit of inpainting.

What is real: the bilateral weighting, non-local means parameters, the Telea and Navier-Stokes inpainting flags and the advice about masks are how OpenCV works. What is the stand-in: the NumPy models are 1-D and a simple diffusion; they show the idea, not OpenCV's exact algorithms or speed.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Why does a bilateral filter keep edges sharp?

    Weights depend on both distance and similarity of value. A pixel on the other side of an edge has a very different value, so its weight is tiny.
  2. 2. What does non-local means compare when it denoises a pixel?

    NLM averages the centers of patches that resemble the current patch, even if they are far away, which suits repeating textures.
  3. 3. What does the h parameter of fastNlMeansDenoising control?

    A higher h smooths more aggressively, which also erases fine detail.
  4. 4. What must you pass to cv2.inpaint besides the image?

    inpaint(src, inpaintMask, inpaintRadius, flags) fills the masked pixels from their surroundings.
  5. 5. Why is it a good idea to dilate an inpainting mask slightly?

    Cover the full damaged area including soft edges so no bad pixels remain to be used as source data.

Summary

  • Gaussian blur removes noise and edges together. The bilateral filter (cv2.bilateralFilter(img, d, sigmaColor, sigmaSpace)) weights neighbors by distance and by similarity, preserving edges, but is slow and can look plastic.
  • Non-local means (fastNlMeansDenoising, ...Colored, ...Multi for video) averages similar patches from a wide window; h sets the strength, and it is the best classic denoiser for photographs.
  • Inpainting (cv2.inpaint with INPAINT_TELEA or INPAINT_NS) fills masked pixels from their surroundings. Build the mask carefully and dilate it; it works for thin damage, not for large missing objects.
  • Match the tool to the noise: median for salt and pepper, Gaussian for speed, bilateral for edges, NLM for quality.

Next: detect and separate the edges of the cleaned image in Gradients, Sobel and Laplacian.