Skip to content
Tutorials Center
My learning

Topics

Lesson 41 of 54 ยท Camera Geometry

Camera Calibration and Undistortion

Learn the pinhole camera model, intrinsics and distortion coefficients, then see calibrateCamera and undistort used with chessboard images, plus projection in NumPy.

A camera turns a 3D world into a flat grid of pixels. Two things get in the way of measuring that world accurately. First, you do not know the camera's internal numbers, such as its focal length in pixels. Second, real lenses bend straight lines, especially wide-angle lenses, so a door frame can look like a bow. Camera calibration finds both sets of numbers, and undistortion uses them to straighten the picture.

You need calibration whenever pixels have to become real measurements: ArUco pose, stereo depth, robot navigation, augmented reality. For casual photo work you can skip it.

What is real and what is a stand-in. cv2.calibrateCamera and cv2.undistort need opencv-python and photographs of a printed chessboard, which this page cannot run. The OpenCV code below is read-only. The live examples implement the underlying math (projection, distortion and its inverse, reprojection error) with NumPy in real Pyodide.

What calibration finds

Intrinsics Focal lengths and the image center.
Distortion How the lens bends straight lines.
Extrinsics Where the camera was for each photo.

Intrinsics and distortion are properties of the camera. Extrinsics change per shot.

1 / 2

The pinhole model

Imagine a tiny hole through which light from the scene passes onto a sensor. A 3D point (X, Y, Z) in camera coordinates (Z pointing forward, away from the camera) lands on the image at

Text
u = fx * (X / Z) + cx
v = fy * (Y / Z) + cy

The dividing by Z is why far things look small. The four numbers fx, fy, cx, cy are the intrinsics. fx and fy are focal lengths measured in pixels (usually almost equal), and (cx, cy) is where the optical axis hits the sensor, normally near the image center. They are packed into the camera matrix:

Text
K = [[fx,  0, cx],
     [ 0, fy, cy],
     [ 0,  0,  1]]
Python Project 3D points with K

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The straight-ahead point lands exactly on the center (320, 240). The 0.5 m offset appears 200 pixels from center at 2 m, but only 100 pixels at 4 m: the same object looks half as big when it is twice as far away.

Lens distortion

Real lenses add two effects on top of the pinhole model. Radial distortion bends lines more the further they are from the center: barrel distortion (lines bulge outward, typical of wide lenses) or pincushion (lines pinch inward). Tangential distortion is a small shift that appears when the lens is not perfectly parallel to the sensor.

OpenCV models both with five coefficients, stored in this order: k1, k2, p1, p2, k3. Work in normalized coordinates x = X/Z, y = Y/Z, and let rยฒ = xยฒ + yยฒ. The radial part is

Text
x_distorted = x * (1 + k1*rยฒ + k2*rโด + k3*rโถ)
y_distorted = y * (1 + k1*rยฒ + k2*rโด + k3*rโถ)

and p1, p2 add the tangential terms. A negative k1 gives barrel distortion. Undistortion is the inverse mapping, and since there is no closed form, it is solved by iteration. Here is the radial part both ways.

Python Distort and undistort

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The distorted line has a different y at each point: it has become a curve, with the ends pulled in more than the middle. Undistorting by fixed-point iteration brings it back to straight. OpenCV does this per pixel (it precomputes a lookup table of where each output pixel comes from) and then samples the source image with interpolation.

Calibrating with a chessboard

A chessboard is perfect for calibration because its inner corners are easy to detect precisely and their real-world spacing is known. The recipe: photograph the board from many angles, find the inner corners in every image, then let OpenCV solve for the parameters that best explain all those observations.

Python
import cv2
import glob
import numpy as np

pattern = (9, 6)            # inner corners per row and column, NOT squares
square = 0.025              # square size in meters

# 3D positions of the corners in the board's own coordinates (z = 0)
objp = np.zeros((pattern[0] * pattern[1], 3), np.float32)
objp[:, :2] = np.mgrid[0:pattern[0], 0:pattern[1]].T.reshape(-1, 2) * square

object_points, image_points = [], []
criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 30, 0.001)

for path in glob.glob("calib/*.jpg"):
    gray = cv2.imread(path, cv2.IMREAD_GRAYSCALE)
    found, corners = cv2.findChessboardCorners(gray, pattern)
    if not found:
        continue
    corners = cv2.cornerSubPix(gray, corners, (11, 11), (-1, -1), criteria)
    object_points.append(objp)
    image_points.append(corners)

h, w = gray.shape
rms, K, dist, rvecs, tvecs = cv2.calibrateCamera(
    object_points, image_points, (w, h), None, None)

print("RMS reprojection error (pixels):", rms)
print("camera matrix:\n", K)
print("distortion (k1, k2, p1, p2, k3):", dist.ravel())

Things to get right: pattern counts inner corners (a board of 10 by 7 squares has 9 by 6 inner corners), the image size is (width, height), and the calibration images must all come from the same camera at the same resolution and zoom. Calibrate once, save K and dist with np.save, and reuse them. Refocusing or zooming changes the intrinsics.

Judging the result

rms is the root-mean-square reprojection error: project the known board corners through the solved model and measure how far they land from the corners you actually detected. Below about 0.5 pixels is good. A large error usually means a bent board, blurry photos, or too few viewing angles.

Python Reprojection error

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The numbers vary with the noise, so the example prints a yes/no check about the RMS rather than the value itself. The idea is the same one calibration uses: a model is good when its predictions land close to the measurements.

Applying the result

Once you have K and dist, straightening images is two calls.

Python
img = cv2.imread("photo.jpg")
h, w = img.shape[:2]

new_K, roi = cv2.getOptimalNewCameraMatrix(K, dist, (w, h), alpha=1)
undistorted = cv2.undistort(img, K, dist, None, new_K)

x, y, rw, rh = roi
cropped = undistorted[y:y + rh, x:x + rw]

alpha=0 crops to only valid pixels (no black borders but you lose edge content); alpha=1 keeps every source pixel and leaves black corners, and roi tells you the valid rectangle. For video, build the lookup table once with cv2.initUndistortRectifyMap and apply it each frame with cv2.remap; that is much faster than calling undistort repeatedly.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which of these are intrinsic parameters?

    Intrinsics are the camera's internal numbers: focal lengths in pixels and the principal point. Position is an extrinsic.
  2. 2. A 9 by 6 pattern passed to findChessboardCorners means:

    The pattern size counts inner corners, which is one fewer than the number of squares in each direction.
  3. 3. You zoom the lens after calibrating. What should you do?

    Focal length and distortion depend on the lens setting, so a zoom or refocus changes them.
  4. 4. What does a large reprojection error most likely indicate?

    Reprojection error measures how far the model's predicted corners are from detected ones; large values point to bad input data.

Summary

  • The pinhole model maps 3D points to pixels with u = fx*X/Z + cx, v = fy*Y/Z + cy.
  • Intrinsics (K) and distortion coefficients belong to the camera; rotation and translation belong to each shot.
  • Calibrate with 10 to 20 photos of a flat chessboard from varied angles, using findChessboardCorners and calibrateCamera.
  • Reprojection error tells you how good the calibration is.
  • undistort straightens an image; use initUndistortRectifyMap plus remap for video.

Next, use two calibrated views to recover depth in Stereo Vision and Depth.