Skip to content
Tutorials Center
My learning

Topics

Lesson 12 of 54 ยท Geometric Transformations

Perspective Transforms and Document Scanning

Straighten skewed photos with getPerspectiveTransform and warpPerspective, order four corner points, understand the 3x3 homography, and solve it yourself with NumPy.

Take a photo of a sheet of paper on a desk from a slight angle. The page is a perfect rectangle in reality, but in the picture it is a trapezoid: the far edge looks shorter than the near edge. An affine transform cannot fix that, because it keeps parallel lines parallel, and here they are not. What you need is a perspective transform, which maps any four points to any four other points.

This is the heart of every document-scanning app: find the four corners of the page, say "these should be the corners of an upright rectangle", and let OpenCV warp the picture accordingly. In this lesson you will learn cv2.getPerspectiveTransform and cv2.warpPerspective, the 3 by 3 homography matrix behind them, the trick for ordering four corner points, and how to solve the homography yourself with NumPy. The cv2 calls are read-only here; the NumPy stand-ins run live. The previous lesson, Translation, Rotation and Affine Transforms, covers the simpler 2 by 3 case.

Four points in, one matrix out

Python
import cv2
import numpy as np

img = cv2.imread("page.jpg")

src = np.float32([[120, 80], [540, 60], [600, 420], [90, 450]])   # corners in the photo
dst = np.float32([[0, 0], [400, 0], [400, 500], [0, 500]])         # an upright 400x500 page

M = cv2.getPerspectiveTransform(src, dst)       # 3x3 matrix
flat = cv2.warpPerspective(img, M, (400, 500))  # dsize = (width, height)

getPerspectiveTransform takes exactly four point pairs, as float32 arrays of shape (4, 2), and returns a 3 by 3 matrix. warpPerspective applies it to the image, just as warpAffine does for affine matrices (with the same flags, borderMode and borderValue arguments, and the same inverse-mapping idea). When you have more than four matches, possibly with mistakes among them, use cv2.findHomography instead; you will meet that in Feature Matching and Homography.

Scanning a page in four steps

Find corners Edges, contour, four-point polygon.
Order them Top-left, top-right, bottom-right, bottom-left.
Get the matrix getPerspectiveTransform on four pairs.
Warp warpPerspective to a flat rectangle.

Clean up with a threshold for the scanned look.

1 / 2

The 3 by 3 homography

A homography works on homogeneous coordinates. Write a point as (x, y, 1), multiply by the 3 by 3 matrix H, and you get (u, v, w). The actual new point is found by dividing by the last value:

TEXT
[u]       [x]
[v] = H * [y]        x' = u / w,   y' = v / w
[w]       [1]

That division by w is what an affine transform does not have, and it is what makes far-away things look smaller. The matrix has nine entries, but scaling all of them by the same factor gives the same transform, so only eight numbers are free. Four point pairs give eight equations, exactly enough. OpenCV normalizes the matrix so that the bottom-right entry H[2, 2] is 1.

Python Apply a homography to points

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

With the last row [0.002, 0, 1], the divisor is 1 + 0.002 * x, so points with larger x are divided by a larger number and pulled toward the origin: parallel horizontal lines would now converge to the right.

Ordering the four corners

getPerspectiveTransform pairs the i-th source point with the i-th destination point, so the order matters. If your source corners come in an arbitrary order (as they do from a contour finder), the page comes out rotated, mirrored or twisted into a bow-tie. The fix is to sort the corners into a fixed order, usually top-left, top-right, bottom-right, bottom-left.

A reliable trick uses two sums:

  • The top-left corner has the smallest x + y, the bottom-right has the largest.
  • The top-right corner has the smallest y - x, the bottom-left has the largest.

This works for any reasonably upright quadrilateral, although a page rotated close to 45 degrees makes the two sums ambiguous.

Python Order four corner points

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The output size is the longer of each pair of opposite edges, so the flattened page does not lose resolution. Use the first output size as the dsize for warpPerspective and [[0, 0], [w, 0], [w, h], [0, h]] (with w and h reduced by one if you want exact pixel indices) as the destination points.

Solving the homography with NumPy

How does OpenCV get H from four pairs? Each pair (x, y) -> (x', y') gives two linear equations in the eight unknowns, once you fix H[2, 2] = 1. Multiply out x' = (h0 x + h1 y + h2) / (h6 x + h7 y + 1) and rearrange:

TEXT
h0*x + h1*y + h2 - h6*x*x' - h7*y*x' = x'
h3*x + h4*y + h5 - h6*x*y' - h7*y*y' = y'

Stack the eight equations into a matrix, solve with np.linalg.solve, and reshape the answer into 3 by 3. This is a stand-in for the OpenCV function, which also works for four points in general position and may use a different numerical method.

Python Solve the homography

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

If you run it, the four source corners land on the four destination corners, and the inverse matrix takes them back. One caveat: if three of your four points lie on one line, the equations are degenerate (np.linalg.solve raises LinAlgError) because such a quadrilateral cannot be mapped to a proper rectangle.

Warping a tiny image

The same inverse-mapping idea as the affine lesson applies: for every pixel of the flat output, apply the inverse homography to find where in the photo to read from. Below, a flat page is first rendered into a skewed photo, and then recovered from the photo using only its four corners.

Python Flatten a trapezoid

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

In the photo the page is a trapezoid: the frame leans inward at the top and the text lines look shorter there. After the warp the frame is a rectangle again and both text lines are horizontal and the same length. The result is ragged (some lines are a pixel thicker than others) because the demo image is tiny and uses nearest-neighbor sampling; real scanners use bilinear or cubic sampling (see Interpolation and Image Quality).

A document scanner in outline

Putting the steps together gives the classic scanner pipeline. These are read-only here; you will build the full version in Project: Document Scanner.

Python
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
blur = cv2.GaussianBlur(gray, (5, 5), 0)
edges = cv2.Canny(blur, 75, 200)

contours, _ = cv2.findContours(edges, cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE)
page = None
for c in sorted(contours, key=cv2.contourArea, reverse=True)[:5]:
    approx = cv2.approxPolyDP(c, 0.02 * cv2.arcLength(c, True), True)
    if len(approx) == 4:
        page = approx.reshape(4, 2)
        break

ordered = order_points(page).astype(np.float32)
# ... compute (w, h) from the edges, build dst, then:
M = cv2.getPerspectiveTransform(ordered, dst)
scan = cv2.warpPerspective(img, M, (w, h))

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. How many point pairs does cv2.getPerspectiveTransform need?

    A homography has eight free numbers and each point pair gives two equations, so four pairs determine it exactly. For more points (with possible errors) use findHomography.
  2. 2. Why do we divide by the third component w after multiplying by the homography?

    The result (u, v, w) is in homogeneous coordinates; the real point is (u/w, v/w). Because w depends on x and y, distant points can be scaled differently, which is the perspective effect.
  3. 3. In the sum-and-difference trick for ordering corners, which corner has the smallest x + y?

    The top-left corner is closest to the origin, so x + y is smallest there. The bottom-right has the largest sum.
  4. 4. Your scanned page comes out mirrored. What is the most likely cause?

    Points are paired by position. If the source corners and destination corners are in different orders, the warp maps the page onto the rectangle flipped or rotated. Order both lists the same way.

Summary

  • A perspective transform (homography) maps any four points to any four points; it is a 3 by 3 matrix with eight free numbers.
  • cv2.getPerspectiveTransform(src, dst) needs four float32 pairs; cv2.warpPerspective(img, M, (width, height)) applies it with inverse mapping.
  • Points are divided by the last homogeneous component, which is what makes distant things look smaller.
  • Sort the corners (top-left, top-right, bottom-right, bottom-left) before pairing them, using x + y and y - x, and take the output size from the longer opposite edges.
  • You can solve the homography with np.linalg.solve from eight linear equations; degenerate point sets (three points on a line) cannot be solved.

Next up: learn how pixel values are estimated during warps in Interpolation and Image Quality.