Skip to content
Tutorials Center
My learning

Topics

Lesson 8 of 46 ยท Pixel Operations

Geometric Transforms

Move, rotate, scale and shear images with affine matrices and homogeneous coordinates, and learn inverse mapping and interpolation with scipy.ndimage.

The operations in the previous lessons changed what is stored in each pixel. A geometric transform does something different: it changes where the pixels are. Rotating a scanned page, shrinking a photo to a thumbnail, straightening a skewed document or shifting an image by a few pixels are all geometric transforms, and they all follow the same recipe: a small matrix describes the motion, and a resampling step fills in the new pixel grid.

This lesson builds that recipe in layers. First you will describe motions with matrices on plain coordinates. Then you will see why practical code runs the matrix backwards (inverse mapping), write a small warp function from scratch, compare interpolation methods, and finally hand the same job to scipy.ndimage.affine_transform. You should be comfortable with matrix multiplication from NumPy matrices and with array coordinates from Images as Arrays.

Points, matrices and homogeneous coordinates

A point in an image has coordinates (x, y), where x is the column and y is the row. Throughout this lesson, matrices use this (x, y) order, which matches the maths you may know from school. Note that NumPy array indexing uses the opposite order, img[y, x]; keep an eye on that swap, because it causes most transform bugs.

Rotation, scaling and shear can each be written as a 2x2 matrix that multiplies the point. Translation (moving by a fixed offset) cannot: it needs an addition. The standard trick is homogeneous coordinates: append a 1 to every point, so (x, y) becomes (x, y, 1), and use 3x3 matrices. Now translation is a multiplication too, and a chain of transforms collapses into one matrix by multiplying the matrices together.

TEXT
translate by (tx, ty)    scale by (sx, sy)      rotate by angle a        shear in x by k
| 1  0  tx |             | sx  0  0 |           | cos a  -sin a  0 |     | 1  k  0 |
| 0  1  ty |             | 0  sy  0 |           | sin a   cos a  0 |     | 0  1  0 |
| 0  0   1 |             | 0   0  1 |           |   0       0    1 |     | 0  0  1 |

Any matrix of this form (last row 0 0 1) is an affine transform: it keeps straight lines straight and parallel lines parallel. It can move, rotate, scale, shear and flip, but it cannot make a rectangle into a general trapezoid; that needs a perspective transform, which you will meet in the document-scanner project.

Python Transforming the corners of a rectangle

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The last two lines show why order matters. With M = B @ A, the point is hit by A first and B second, so reading a product from right to left gives the order of events. Rotating (4, 0) by 90 degrees puts it at (0, 4), and then the translation moves it to (10, 4). Doing it the other way round moves it to (14, 0) first and then rotates it to (0, 14).

Note: On screen the y axis points down (row 0 is at the top). A positive rotation angle in the formula above therefore turns things clockwise on screen, the opposite of the maths textbook picture.

Rotating about the centre

The rotation matrix turns points around the origin, which in an image is the top-left corner. To rotate around the image centre, move the centre to the origin, rotate, and move back. In code that is a three-matrix product:

Python
center = 5.5
M = translate(center, center) @ rotate(30) @ translate(-center, -center)

The same sandwich (move, act, move back) works for scaling or shearing about any chosen point.

Inverse mapping

Suppose you want to scale an image up by 2. The obvious way is forward mapping: for every source pixel, compute where it lands and write it there. The problem is that source pixels land two columns and two rows apart, so three quarters of the output pixels never get written. The loop below counts the holes:

Python Forward mapping leaves holes

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Inverse mapping turns the question around. Loop over the output pixels, use the inverse matrix to find which source position each one comes from, and read the value there. Every output pixel is written exactly once, so there are no holes and no overlaps. This is how every serious image library implements transforms.

That also tells you what scipy.ndimage.affine_transform expects. If your forward matrix is M (source to output), you give the library inv(M), because it walks over output coordinates and wants the source position for each.

flowchart LR
    A["output pixel<br/>(x, y)"] --> B["multiply by inv(M)"]
    B --> C["source position<br/>(sx, sy), maybe fractional"]
    C --> D["interpolate the<br/>source around it"]
    D --> E["output value"]

Inverse mapping visits each output pixel once and asks the source image what colour it should have.

Interpolation: reading between pixels

The source position (sx, sy) is almost never a whole number. After a 30 degree rotation it might be (3.62, 7.18), which falls between four real pixels. Interpolation decides what value to read there:

  • Nearest neighbour (order 0) rounds to the closest pixel. It is fast and never invents new values, so it is the right choice for label images and masks. It also gives blocky, jagged edges.
  • Bilinear (order 1) blends the four surrounding pixels, weighting each by how close it is. Edges look smoother but slightly blurred.
  • Bicubic and higher (order 3) fit curves through a 4x4 neighbourhood. They look sharpest but can overshoot, producing values above the maximum or below the minimum of the input.

The next TryIt writes both a nearest and a bilinear warp function with plain NumPy, rotates a small arrow by 30 degrees about its centre, and then checks the result against SciPy. The helper to_ndi converts an (x, y) matrix into the (row, column) form that SciPy expects: it inverts the matrix and swaps the two axes with the permutation matrix P.

Python A warp function from scratch

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The hand-written functions agree with SciPy: the nearest-neighbour result is identical, and the bilinear results match to rounding error. The nearest-neighbour arrow has hard stair-steps and keeps only the values 0 and 1, while the bilinear arrow shows soft grey edges. The pixel sums are almost unchanged, because a rotation moves ink around without adding or removing any.

Tip: Mind the border. Output pixels whose source position falls outside the image have nothing to read. The default is to fill with zeros (black). scipy.ndimage calls this the mode argument and also offers 'nearest', 'reflect' and 'wrap' for other behaviours.

Interpolation orders compared

scipy.ndimage.rotate is a convenient wrapper for rotations, and order= selects the interpolation. Comparing the orders on the same arrow shows the trade-off in numbers:

Python Nearest, bilinear and cubic

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Order 0 only ever produces the two input values. Order 1 stays inside the range 0 to 1 but creates many in-between greys. Order 3 produces a minimum of about -0.13 and a maximum of about 1.31: it overshoots. For a normal photo you would clip the result back into range; for a label mask you would never use it. Rule of thumb: order 0 for masks and labels, order 1 as a safe default for photos, order 3 when quality matters and you clip afterwards.

Scaling, shear and using affine_transform

Once you have to_ndi, any combination of motions becomes a single call. Shear is a good example. A horizontal shear shifts each row sideways by an amount proportional to its height, turning a rectangle into a parallelogram. The text-like block below is sheared about its centre row with a bilinear affine transform. A second call scales an image up with ndi.zoom.

Python Shear and zoom with SciPy

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

In the sheared output the bottom of the bar sits to the right of the top: with the forward formula x' = x + k * (y - centre_row), rows below the centre row move right and rows above it move left, and the centre row stays put. If a transform ever goes the wrong way, you have probably passed the forward matrix where the inverse was expected, or the reverse.

Two scaling details are worth knowing. ndi.zoom with a factor of 2 on a 4x4 image returns 8x8, and its corners line up exactly with the corners of the input (the top-left output is 1.0 and the bottom-right is 16.0). Downscaling by a large factor also needs a blur first, to avoid the aliasing from Resolution, Sampling and Noise; a plain bilinear shrink by 8x simply skips most source pixels.

Practical advice

  • Keep one matrix. Compose every motion (centre, rotate, scale, move back) into a single M and resample once. Resampling twice blurs the image twice.
  • Decide on the output size deliberately. A rotated image no longer fits its old frame; either crop it (reshape=False) or enlarge the canvas so the corners survive.
  • Separate forward and inverse in your head and in variable names (M_fwd, M_inv). Most mistakes are one of them mixed up.
  • Use nearest neighbour for masks and label images, so labels stay valid integers.
  • Rotating by a multiple of 90 degrees or flipping needs no interpolation: np.rot90, img[::-1] and img[:, ::-1] are exact and instant.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Why do image transforms use homogeneous coordinates (x, y, 1)?

    Translation is an addition in 2D. The extra coordinate turns it into a multiplication, so rotation, scaling, shear and translation can be multiplied together.
  2. 2. You compute M = translate(10, 0) @ rotate(90). Which transform is applied to a point first?

    In a matrix product the rightmost matrix touches the point first, so the point is rotated and then translated.
  3. 3. What problem does inverse mapping solve compared with forward mapping?

    Forward mapping pushes source pixels to new places and can miss output pixels when scaling up. Inverse mapping visits every output pixel and looks up its source.
  4. 4. Which interpolation order is the safest choice when transforming a label mask whose values are class numbers?

    Blending the labels 1 and 3 would produce 2, a class that is not there. Nearest neighbour only copies existing values.
  5. 5. ndi.affine_transform is called with a matrix. Which matrix does it expect?

    SciPy walks over output pixels and needs the source position for each one, so it wants the inverse transform in the array's (row, column) order.

Summary

  • A geometric transform moves pixels. Rotation, scale, shear and translation are 3x3 matrices in homogeneous coordinates, and they compose by matrix multiplication (the rightmost matrix acts first).
  • To rotate or scale about a point other than the origin, translate it to the origin, apply the transform and translate back.
  • Practical code uses inverse mapping: for each output pixel compute the source position with inv(M) and read the image there, which avoids holes.
  • Interpolation fills in fractional positions: nearest for masks, bilinear for general use, cubic for quality (with clipping).
  • scipy.ndimage.affine_transform wants the output-to-input matrix in (row, column) order; a small to_ndi helper converts an (x, y) forward matrix.
  • Compose all motions into one matrix and resample once. Exact flips and 90 degree turns are plain slicing and np.rot90.

Next up: slide a small window over the image to blur, sharpen and find edges in Convolution and Correlation.