Skip to content
Tutorials Center
My learning

Topics

Lesson 39 of 46 ยท Projects

Project: Document Scanner

Build a document scanner step by step: threshold the page, find its four corners, warp it flat with a perspective transform and clean up the ink.

A phone scanner app takes a crooked photo of a sheet of paper and hands back a flat, evenly lit, black-on-white page. In this project you will build the same pipeline yourself. It uses ideas from earlier lessons in one place: thresholding, morphology and connected components, geometric transforms and edge and boundary analysis.

The input is a synthetic photo, so we know the true answer and can score the result. The page is a made-up sheet with a title bar and rows of "words" (dark bars). We photograph it with a tilt (a perspective view), uneven lighting, a table behind it and sensor noise. The scanner must undo all of that.

flowchart LR
    A["photo"] --> B["blur + Otsu<br/>page mask"]
    B --> C["largest component<br/>+ fill holes"]
    C --> D["boundary points<br/>four edge lines"]
    D --> E["four corners"]
    E --> F["homography<br/>perspective warp"]
    F --> G["shading correction<br/>threshold, despeckle"]
    G --> H["clean scan"]

Seven small steps, each one a technique you already know.

Step 1: A synthetic photo

Real photos need files, and a real page has no ground truth. We generate both sides. make_page draws the flat page (140 rows by 100 columns, paper at 235, ink at 40). make_scene places it into a 180 by 240 photo: for each photo pixel it finds the matching page position with a homography (explained in step 3), reads the paper there with bilinear interpolation, and otherwise paints a table. Then it multiplies by a left-to-right light ramp and adds Gaussian noise.

Python Generating the page and the photo

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice that the right side of the paper is far brighter than the left, so a single global cutoff for the ink will struggle later. That is by design.

Step 2: Find the page and its corners

The page is the largest bright region. A Gaussian blur first (sigma 2) erases the text strokes so that the page becomes one solid shape, then Otsu picks the cutoff between table and paper. Binary opening removes small bright specks, binary_fill_holes closes any gap, and ndimage.label keeps the largest component only.

Now the corners. A quick method takes the extreme points: the page's top-left corner is the boundary point with the smallest x + y, bottom-right has the largest x + y, top-right has the largest x - y and bottom-left the smallest. This is fast but quantized to whole pixels and slightly rounded by the blur. The refinement step splits the boundary pixels into four sides, fits a straight line to each (total least squares through the SVD), and intersects neighbouring lines, which gives sub-pixel corners.

Python Mask, boundary and refined corners

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

All four corners land within a couple of pixels of the truth. The remaining error comes from the blur and the noise. On a real photo you would also try a Canny edge map plus a Hough transform for the four edge lines, or a contour approximation; these are the standard alternatives. We use the line-intersection idea because it needs only NumPy.

Step 3: A perspective warp from four points

A tilted rectangle is not related to the flat one by a rotation or scale; it needs a homography, a 3 by 3 matrix H that maps (x, y) to (u, v) by

TEXT
[u', v', w'] = H @ [x, y, 1]        u = u'/w'     v = v'/w'

The division by w' is what makes distant parts of the page shrink. H has eight free numbers (its bottom-right entry is fixed at 1), and each point correspondence gives two equations, so four corners determine it exactly. homography(src, dst) builds that 8 by 8 linear system and solves it. For the warp we use inverse mapping, as in Geometric Transforms: for each pixel of the output page we compute where it comes from in the photo, then interpolate there.

Python Solving for the homography

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

If you only know the corners and not the page size, the usual trick is to choose an output size from the longest opposite edges, or to assume a standard paper ratio. Here we simply use the known 140 by 100.

Warning: Order the corners consistently (top-left, top-right, bottom-right, bottom-left) for both point sets. Swap two of them and the page comes out mirrored or twisted, with no error message.

Step 4: The complete scanner

Now we connect everything. rectify warps the photo into a flat page. clean has three jobs. First it removes the lighting: a large median filter followed by a Gaussian blur estimates the background brightness, and dividing the page by it flattens the light ramp (the ink survives because the median ignores thin strokes). Next it thresholds the result and applies a 2 by 2 opening to drop stray pixels. Finally it labels the ink and deletes any component under 12 pixels.

We run the pipeline on three photos: a mild tilt, a strong tilt with one corner closer to the camera, and a dim, noisy shot. To score each scan we compare its ink mask with the true ink of the flat page through the intersection over union (IoU). The "raw" column is the plain alternative: rectify, then one global Otsu threshold with no lighting correction or cleanup.

Python The full pipeline on three photos

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The scan beats the raw threshold on every photo, and the gap is largest when the photo is dim and noisy: the lighting correction and speckle removal matter most there. Scores below 1.0 are expected, because a one-pixel shift of a 4-pixel-high bar already costs a lot of overlap, and the corner estimates are off by a pixel or two.

Where to go next

  • Corner search in real photos. Add a Canny edge map and a Hough transform for lines, then intersect the four strongest. Check that the four corners form a convex quadrilateral and fall back to the full image otherwise.
  • Better output size. Estimate the page aspect ratio from the homography instead of assuming it.
  • Real photos. With OpenCV the same pipeline is findContours, approxPolyDP, getPerspectiveTransform, warpPerspective and adaptiveThreshold, and the code is shorter; the version above shows what those functions do.
  • Quality checks. Reject scans when the page area is a tiny fraction of the photo or when the line fits have large residuals.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why blur the photo before running Otsu to find the page?

    Dark text strokes would otherwise split the paper into many pieces and distort the histogram. A blur removes them so the page mask is one connected shape.
  2. 2. How many point correspondences are needed to determine a homography, and why?

    With the bottom-right entry fixed at 1 there are eight unknowns, and every point pair gives an x equation and a y equation, so four pairs solve it.
  3. 3. Why is inverse mapping used to build the rectified page?

    Looping over the output pixels and asking where each comes from guarantees a value for every pixel, while forward mapping can leave holes. Interpolation is still needed at the fractional source positions.
  4. 4. What does dividing the page by a heavily smoothed copy of itself achieve in the clean-up step?

    The smoothed copy estimates the slowly changing paper brightness. The ratio is close to the same value everywhere on blank paper, and ink stays clearly darker.

Summary

  • A scanner is a pipeline: page mask, corners, perspective warp, lighting correction, threshold and despeckle.
  • Blur plus Otsu plus the largest connected component finds the page. Fitting lines to the four boundary sides and intersecting them gives sub-pixel corners.
  • A homography (3 by 3, eight unknowns) is solved from four point pairs. Use inverse mapping with interpolation to build the flat page, and keep corner order consistent.
  • Dividing by a smoothed background removes uneven lighting; opening and size filtering remove speckle.
  • A synthetic scene with known truth lets you score each step with IoU instead of judging by eye.

Next project: Counting Objects.