Skip to content
Tutorials Center
My learning

Topics

Lesson 30 of 54 ยท Features and Matching

Template Matching

Find a small image inside a bigger one with cv2.matchTemplate. Compare the six methods, read minMaxLoc, find several matches, and see the limits with scale.

Sometimes you know exactly what you are looking for: a button icon in a screenshot, a logo on a scanned page, a coin of one fixed size on a table. In that case you do not need machine learning. You can cut the object out once as a small template image, slide it over the big image, and measure at every position how well the two agree. The place with the best agreement is where the object is. That is template matching, and cv2.matchTemplate does it in a single call.

It is simple, fast to try and surprisingly useful, but it has sharp limits: it only works when the object looks almost exactly like the template, at the same size and the same orientation. This lesson shows how to use it, how to choose among its six comparison methods, and why it breaks when the scale or rotation changes. A brute-force NumPy version makes every step visible.

cv2.matchTemplate and cv2.minMaxLoc appear in read-only blocks because opencv-python does not run in the browser. The runnable stand-ins implement the same comparisons with NumPy's sliding_window_view, on small arrays of numbers. OpenCV computes the same scores (it uses FFT-based convolution for speed on large templates), so the idea and the shape of the result are identical.

The basic call

Python
import cv2

img = cv2.imread("screenshot.png", cv2.IMREAD_GRAYSCALE)
tmpl = cv2.imread("button.png", cv2.IMREAD_GRAYSCALE)
h, w = tmpl.shape                      # shape is (rows, cols) = (height, width)

result = cv2.matchTemplate(img, tmpl, cv2.TM_CCOEFF_NORMED)
min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(result)

top_left = max_loc                     # (x, y) of the best match
bottom_right = (top_left[0] + w, top_left[1] + h)
cv2.rectangle(img, top_left, bottom_right, 255, 2)
print("score:", max_val)

The template is slid over every position where it fits completely inside the image. For an image of size W x H and a template of size w x h, that gives (W - w + 1) by (H - h + 1) positions, so the result is a 2D map of scores that is a little smaller than the image, as a float32 array. Each score belongs to the position of the template's top-left corner. cv2.minMaxLoc finds the extreme values and their locations, returned as (x, y) points (column first), just like contour points in Finding and Drawing Contours.

Template matching in four steps

Cut a template A small picture of the object.
Slide it Over every position.
Score it One number per position.
Find the peak With minMaxLoc.
1 / 2

The six methods

matchTemplate has three scores, each in a plain and a normalized form:

  • cv2.TM_SQDIFF (and TM_SQDIFF_NORMED): sum of squared differences. A perfect match scores 0, so you look for the minimum.
  • cv2.TM_CCORR (and TM_CCORR_NORMED): cross-correlation, the sum of products. A better match scores higher, so look for the maximum.
  • cv2.TM_CCOEFF (and TM_CCOEFF_NORMED): correlation of the values after subtracting each patch's mean. Look for the maximum; the normalized form lies between -1 and 1, and 1 is a perfect match.

The location to use depends on the method: for the two SQDIFF methods it is min_loc, for all others max_loc. Forgetting this is the most common bug, and it produces a rectangle on the wrong spot with no error message.

Plain TM_CCORR has a serious weakness: bright regions score high regardless of what is in them, because multiplying large numbers gives large products. An empty white patch beats a faint but correct match. TM_CCOEFF_NORMED removes both the brightness offset (by subtracting the mean) and the contrast (by dividing by the standard deviations), so it is the best default. TM_SQDIFF is intuitive and exact but sensitive to lighting changes.

The first stand-in implements three of the methods by brute force and runs them on an image that contains the template's true location and a very bright decoy patch:

Python Three scores on the same image

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The image is 24 rows by 32 columns and the template 5 by 6, so the result map is 20 by 27, matching the formula. The squared difference finds the true position (14, 6) with a score of exactly 0, and the normalized correlation coefficient finds it with a score of 1.000. Plain cross-correlation, however, is fooled: it picks the bright decoy at row 2, column 20, because a patch of constant 250s produces larger products than the true match. Notice the argument order: NumPy gives (row, col), while OpenCV's minMaxLoc would report this point as (x, y) = (6, 14).

Limits: scale, rotation and clutter

Template matching compares pixels, so the object must look nearly identical to the template. Three things break it:

  • Scale. If the object appears larger or smaller than the template, the pixels no longer line up. Workarounds: loop over several resized versions of the image (or template) and keep the best score, which is called multi-scale matching, or use features that survive scaling (see Corners and Keypoints).
  • Rotation. Likewise, a turned object does not match. You could try several rotated templates, which gets expensive quickly.
  • Lighting and background. TM_CCOEFF_NORMED tolerates global brightness and contrast changes, but not shadows, reflections or a changed background around the object.

The second stand-in measures each effect with normalized correlation:

Python What NCC tolerates and what breaks it

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The exact copy and the darker, flatter version both score 1.0 at the true position (10, 12): a linear brightness and contrast change does not affect the normalized score. Added noise lowers the score to about 0.93 but the location survives. Doubling the image size or rotating it by 90 degrees drops the best score to about 0.33 and 0.38, and the "best" location is meaningless, which is the practical signal that the object is not there in the template's form. In real code, always check the score against a threshold instead of trusting the location of the maximum.

Multi-scale matching

When the object's size is unknown, run the match at several scales of the image and keep the best score. Shrink the image (not the template) step by step, remember the scale, and map the best location back to the original coordinates.

Python
best = None                                    # (score, location, scale)
for scale in np.linspace(1.0, 0.3, 15):
    resized = cv2.resize(gray, None, fx=scale, fy=scale, interpolation=cv2.INTER_AREA)
    if resized.shape[0] < th or resized.shape[1] < tw:
        break                                  # the image became smaller than the template
    result = cv2.matchTemplate(resized, template, cv2.TM_CCOEFF_NORMED)
    _, score, _, loc = cv2.minMaxLoc(result)
    if best is None or score > best[0]:
        best = (score, loc, scale)

score, (x, y), scale = best
x0, y0 = int(x / scale), int(y / scale)       # back to original coordinates
x1, y1 = int((x + tw) / scale), int((y + th) / scale)
cv2.rectangle(img, (x0, y0), (x1, y1), (0, 255, 0), 2)

The cost is one full match per scale, so use a coarse list of scales first, or an image pyramid, and a threshold on score to reject frames where the object is simply absent. Multi-scale matching still does not handle rotation or perspective; for that use feature matching. A related application is detecting printed form fields: match a template of a field label in the scanned page and read the area next to it.

Finding several matches

minMaxLoc only gives the single best spot. To find every occurrence (all the coins of one kind, all the icons), threshold the score map:

Python
import numpy as np

result = cv2.matchTemplate(img, tmpl, cv2.TM_CCOEFF_NORMED)
ys, xs = np.where(result >= 0.8)
for x, y in zip(xs, ys):
    cv2.rectangle(color, (x, y), (x + w, y + h), (0, 255, 0), 1)

np.where returns (rows, cols), so the loop reads xs and ys in the order that OpenCV wants. One real object usually produces a small cluster of neighboring pixels above the threshold, so the same object gets drawn several times. Remove the duplicates by keeping only local maxima, or by merging boxes that overlap heavily (non-maximum suppression, which you will meet again in the dnn lesson).

Practical tips

  • Work in grayscale unless color is the discriminating feature; it is three times faster.
  • Use TM_CCOEFF_NORMED first, and set your threshold from a few real examples (0.8 is a typical starting point, not a rule).
  • Keep the template small and distinctive: it should contain texture, not a flat color.
  • To match a non-rectangular template, pass the optional mask argument (supported by some methods, including TM_SQDIFF and TM_CCORR_NORMED), so background pixels around the object are ignored.
  • Template matching is great for screenshots, UI testing, industrial parts at fixed distance and PCB inspection. For photos of 3D objects taken from varying distances, use features and matching from the next lessons.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. A 100 by 60 image is matched with a 20 by 10 template (width by height). What are the dimensions of the result map?

    The template fits W - w + 1 = 81 times horizontally and H - h + 1 = 51 times vertically, so the result is 81 wide and 51 high.
  2. 2. For which method is the best match the MINIMUM of the result?

    SQDIFF measures squared differences, so 0 means a perfect match. For the other methods larger scores are better, so you use the maximum location.
  3. 3. Why is plain TM_CCORR often a poor choice?

    Cross-correlation multiplies pixel values, so large values give large scores. The normalized coefficient method removes the brightness offset and the contrast.
  4. 4. The object in the scene is twice as large as in your template. What happens with matchTemplate?

    matchTemplate compares pixels at one fixed size. Multi-scale search means resizing the image or template in a loop and keeping the best score.
  5. 5. How do you find all locations whose score is at least 0.8?

    minMaxLoc only gives one best position. np.where on the thresholded score map returns every position at or above the threshold, as (rows, cols).

Summary

  • cv2.matchTemplate(img, tmpl, method) returns a score map of size (H-h+1, W-w+1); cv2.minMaxLoc finds the best position as (x, y).
  • Use the minimum for the two TM_SQDIFF methods and the maximum for all the others.
  • TM_CCOEFF_NORMED is the best default: it ignores brightness offset and contrast scale.
  • Template matching is not invariant to scale or rotation; check the score against a threshold and try several scales if needed.
  • Threshold the map with np.where for several matches, and merge duplicates.

Next lesson: Corners and Keypoints: Harris, Shi-Tomasi, ORB.