Skip to content
Tutorials Center
My learning

Topics

Lesson 46 of 54 ยท Projects

Project: Document Scanner

Build a document scanner step by step: find the page, order its four corners, warp it flat with a homography and clean it with a threshold, with a live NumPy demo.

Phone apps that turn a photo of a receipt into a flat, clean PDF page are a classic computer vision project, and they are built from steps you have already met: edges, contours, a perspective warp and a threshold. In this project you will put those steps together into one pipeline.

The real pipeline uses cv2, which cannot run on this site, so the OpenCV code appears in read-only blocks. Alongside it you get a live stand-in built from NumPy only: we draw a synthetic tilted page, locate its corners, solve the homography ourselves and warp the page flat. What is real and what is the stand-in is spelled out at every step, so you can run the ideas here and run the actual cv2 code on your own machine.

The scanner pipeline

Find the page Edges, then the biggest four-sided contour.
Order corners Top-left, top-right, bottom-right, bottom-left.
Warp flat A homography straightens the page.
Clean up A threshold gives crisp black on white.
1 / 2

The real pipeline (read-only)

Here is the whole scanner in OpenCV. It assumes the page is the largest four-cornered shape in the photo, on a darker background.

Python
import cv2
import numpy as np

def order_points(pts):
    pts = pts.reshape(4, 2).astype("float32")
    s = pts.sum(axis=1)             # x + y
    d = np.diff(pts, axis=1)[:, 0]  # y - x
    return np.array([
        pts[np.argmin(s)],          # top-left: smallest x + y
        pts[np.argmin(d)],          # top-right: smallest y - x
        pts[np.argmax(s)],          # bottom-right: largest x + y
        pts[np.argmax(d)],          # bottom-left: largest y - x
    ], dtype="float32")

def find_page(image):
    ratio = image.shape[0] / 500.0
    small = cv2.resize(image, (int(image.shape[1] / ratio), 500))
    gray = cv2.cvtColor(small, cv2.COLOR_BGR2GRAY)
    gray = cv2.GaussianBlur(gray, (5, 5), 0)
    edges = cv2.Canny(gray, 75, 200)

    contours, _ = cv2.findContours(edges, cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE)
    contours = sorted(contours, key=cv2.contourArea, reverse=True)[:5]
    for contour in contours:
        perimeter = cv2.arcLength(contour, True)
        approx = cv2.approxPolyDP(contour, 0.02 * perimeter, True)
        if len(approx) == 4:
            return approx.reshape(4, 2) * ratio     # back to full-size coordinates
    return None

def scan(image):
    corners = find_page(image)
    if corners is None:
        raise ValueError("no four-cornered page found")
    tl, tr, br, bl = order_points(corners)

    width = int(max(np.linalg.norm(br - bl), np.linalg.norm(tr - tl)))
    height = int(max(np.linalg.norm(tr - br), np.linalg.norm(tl - bl)))
    target = np.array([[0, 0], [width - 1, 0],
                       [width - 1, height - 1], [0, height - 1]], dtype="float32")

    matrix = cv2.getPerspectiveTransform(np.array([tl, tr, br, bl], dtype="float32"), target)
    flat = cv2.warpPerspective(image, matrix, (width, height))

    gray = cv2.cvtColor(flat, cv2.COLOR_BGR2GRAY)
    return cv2.adaptiveThreshold(gray, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C,
                                 cv2.THRESH_BINARY, 21, 10)

# result = scan(cv2.imread("receipt.jpg")); cv2.imwrite("scan.png", result)

Read it as four small jobs. We detect on a shrunk copy (500 pixels tall) because edge detection on a 12-megapixel photo is slow and the extra detail does not help. We multiply the corners by ratio afterwards so the warp uses the full-resolution image. The ideas behind these calls are in Canny Edge Detection, Finding and Drawing Contours, Detecting Lines, Circles and Shapes (for approxPolyDP) and Perspective Transforms and Document Scanning.

Real vs stand-in: the block above is genuine OpenCV code, but it cannot run here. Below, the live demos replace edges and contours with a simple brightness mask, and replace getPerspectiveTransform and warpPerspective with NumPy functions we write ourselves.

Step 1: order the four corners

A contour gives you four points in an arbitrary order, but a homography needs each source corner paired with the right target corner. The trick in order_points uses sums and differences of coordinates. For a page on a dark background:

  • the top-left corner has the smallest x + y,
  • the bottom-right corner has the largest x + y,
  • the top-right corner has the smallest y - x,
  • the bottom-left corner has the largest y - x.

This works for any page that is not rotated close to 45 degrees, which covers almost every handheld photo. Coordinates here are (x, y), where x is the column and y the row, the order OpenCV uses for points (arrays are indexed [row, column], so be careful when you mix them).

The live demo builds a synthetic scene: a flat 40 by 60 page with dark bars standing in for text, warped onto a 60 by 90 canvas at a tilt. We then find the corners from the bright pixels and recover the homography.

Python Find and order the corners, solve the homography

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The four recovered points sit on or within a pixel of the corners we used to draw the page (the scene is tiny and edge pixels are quantized). The last loop is a useful habit: after you compute a transform, push the corners through it and check that they land where you expect.

Step 2: warp the page flat

cv2.getPerspectiveTransform takes four source points and four target points and returns the 3x3 homography matrix. cv2.warpPerspective then builds the output by inverse mapping: for every pixel in the output, it asks "which source pixel feeds you?" and copies it. The warp function in the demo does exactly that, with the nearest-neighbour rule. The real function offers INTER_LINEAR (the default) and INTER_CUBIC for smoother results, as covered in Interpolation and Image Quality.

The output size matters. If you pick a target that does not match the page's aspect ratio, the text looks stretched. That is why the real code measures the page's side lengths and keeps the longer of each pair.

Step 3: clean up with a threshold

A scan should look like black ink on white paper, no matter if the photo was taken under a lamp or beside a window. A single global threshold fails when the light is uneven: the shadowed corner turns all black. Adaptive thresholding decides per pixel by comparing it to the average of its neighbourhood, so a shadow that darkens everything locally cancels out. See Thresholding for the full story.

The demo below compares both on a page with a shadow ramp running across it and prints them as text, where # is ink and . is paper.

Python Global vs adaptive threshold under a shadow

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The global result drowns the right-hand side in ink, because the whole region is darker than 128. The adaptive result keeps three clean lines across the page. This is why scanners use ADAPTIVE_THRESH_GAUSSIAN_C (a Gaussian-weighted local mean) with a block size around 21 and a constant C around 10.

Step 4: put it all together

Now the full live pipeline: the synthetic photo goes in, a flat page comes out. We reuse the corner trick, the homography solver and the warp, then threshold.

Python The whole scanner on a synthetic photo

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

In the photo, the page leans and the text rows slope. In the scan, they are horizontal again and the dark background is gone. Pixel-level fuzz at the edges comes from nearest-neighbour sampling on a very small image; with real photos and bilinear interpolation the result is smooth.

Making it robust

The textbook pipeline works on clean photos. For real use, plan for these failures:

  1. No page found. If no contour has four corners, say so (None) instead of crashing. Try a larger blur, different Canny thresholds, or fall back to cv2.minAreaRect on the largest contour.
  2. Page on a similar-colored table. Edge contrast is weak. Closing small gaps in the edge map with a morphological close often reconnects a broken outline.
  3. Wrong corner order. Always test order_points on rotated pages, and add a unit test as described in Testing Computer Vision Code.
  4. Slow on big photos. Detect on a downscaled copy, then scale the corners (as the code above does).
  5. Faded ink. Increase C in the adaptive threshold or try CLAHE first (Histograms and Equalization).

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. In order_points, which corner has the smallest x + y?

    The top-left corner is closest to the origin of the image, so both its x and y are small and their sum is the smallest.
  2. 2. Why does the real pipeline detect the page on a shrunk copy and then multiply the corners by a ratio?

    Edge detection and contour search on a small image are much faster and the page outline survives shrinking. Scaling the corners back lets the warp use every pixel of the original.
  3. 3. Why is adaptive thresholding preferred over a single global threshold for scanned pages?

    A global threshold uses one value for the whole image, which fails when part of the page is in shadow. A local mean adapts to the local brightness.
  4. 4. warpPerspective builds its output by inverse mapping. What does that mean?

    Looping over output pixels and looking up where each comes from guarantees every output pixel gets a value, with no gaps.

Summary

  • A scanner is four steps: find the page (edges, contours, approxPolyDP), order the four corners, warp with a homography, then clean up with an adaptive threshold.
  • Order corners with sums and differences of coordinates: smallest x + y is top-left, largest is bottom-right, and the y - x extremes give the other two.
  • getPerspectiveTransform solves a 3x3 homography from four point pairs; warpPerspective applies it by inverse mapping.
  • Pick the output size from the page's side lengths so text is not stretched.
  • Adaptive thresholding handles shadows that defeat a global threshold.
  • The live demos use NumPy stand-ins for the same math; the cv2 code is read-only here.

Next project: Project: Color Object Detector.