Skip to content
Tutorials Center
My learning

Topics

Lesson 25 of 31 ยท Projects

Project: Image Blur and Convolution

Build a CUDA image blur step by step: a 2D grid, a constant-memory filter and halo tiles in shared memory, checked with a runnable thread-grid simulator.

Image filtering is the classic first "real" GPU project. Every output pixel is computed from a small neighbourhood of input pixels, all pixels are independent of each other, and the data is already a 2D grid. That is almost exactly the shape a CUDA launch has. A blur, a sharpen filter and an edge detector are all the same program with different numbers in the filter, which is called a convolution.

In this project you build the blur in three stages: a naive kernel with a 2D grid, a version that keeps the filter weights in constant memory, and a version that loads each block's pixels into shared memory together with a border of extra pixels called the halo. You will use ideas from Grids, Blocks and Threads, The Memory Hierarchy and Shared Memory and Tiling.

The CUDA C++ code in this lesson is read-only: the site has no GPU. The Python examples are stand-ins. A small thread-grid simulator calls a kernel function once per thread, so the index math, bounds checks and results are real. Counts of loads come from the models, not from hardware, and no timings are claimed.

The problem

A convolution with a k x k filter (k odd, radius r = k / 2) computes each output pixel as a weighted sum of the k x k input pixels around it:

out[y][x] = sum over fy, fx of filter[fy][fx] * in[y + fy - r][x + fx - r]

The pixels at the image border have neighbours that do not exist. We need a rule. This project uses clamping: a coordinate outside the image is replaced by the nearest valid one, so the border pixels are repeated outward. (Other rules are zero padding and mirroring.) For a blur we use a Gaussian-like 3 x 3 filter whose weights add up to 1, so the overall brightness does not change:

Text
1 2 1
2 4 2   divided by 16
1 2 1

Stage 1: a 2D grid and a naive kernel

Use one thread per output pixel, arranged in 2D blocks (16 x 16 is a common choice, 256 threads). The grid must cover the whole image, so round the number of blocks up and add a bounds check, as in Grids, Blocks and Threads:

C++
__global__ void blurNaive(const float* src, float* dst, int w, int h,
                          const float* filter, int r) {
    int x = blockIdx.x * blockDim.x + threadIdx.x;
    int y = blockIdx.y * blockDim.y + threadIdx.y;
    if (x >= w || y >= h) return;                 // threads past the edge do nothing

    float acc = 0.0f;
    for (int fy = -r; fy <= r; ++fy) {
        for (int fx = -r; fx <= r; ++fx) {
            int sx = min(max(x + fx, 0), w - 1);  // clamp to the image
            int sy = min(max(y + fy, 0), h - 1);
            acc += src[sy * w + sx] * filter[(fy + r) * (2 * r + 1) + (fx + r)];
        }
    }
    dst[y * w + x] = acc;
}
// dim3 block(16, 16);
// dim3 grid((w + 15) / 16, (h + 15) / 16);
// blurNaive<<<grid, block>>>(d_src, d_dst, w, h, d_filter, 1);

Threads with neighbouring x read neighbouring pixels, so the loads from src are coalesced (see Global Memory and Coalescing). The weak spot is that each pixel is read by nine different threads, and every thread also reads the nine filter weights from global memory.

The first example runs this exact kernel logic. launch loops over blocks and threads and calls the kernel function with the same indices CUDA would provide. A small image with a bright square lets you see the result as text.

Python Naive blur kernel in the thread-grid simulator

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The 16 x 12 image needs a 2 x 2 grid of 8 x 8 blocks, which launches 256 threads for 192 pixels. The 64 extra threads are the ones the bounds check turns away. The blurred picture shows soft edges around the square and a light halo on the rows next to the top line. The last check compares the simulated kernel with a NumPy reference built from shifted copies of an edge-padded image, which is a good habit: always keep a simple CPU version to test a kernel against.

Stage 2: the filter in constant memory

All threads read the same filter weights, and they read the same weight at the same moment (fy, fx are identical across the warp). That is the perfect use of constant memory: a small, read-only region (64 KB) with a cache that can broadcast one value to a whole warp in a single step.

C++
#define R 1
#define K (2 * R + 1)

__constant__ float c_filter[K * K];          // lives in constant memory

// host side, once, before launching:
// float h_filter[K * K] = {1,2,1, 2,4,2, 1,2,1};   // then divide each by 16
// cudaMemcpyToSymbol(c_filter, h_filter, sizeof(h_filter));

The kernel now reads c_filter[(fy + R) * K + (fx + R)] instead of a global pointer. That removes the filter traffic from global memory and frees you from passing a pointer. Remember the limits: it is only fast when the threads of a warp read the same address. If each thread indexed it differently, the reads would be serialized.

Stage 3: halo tiles in shared memory

Now the main optimization. Neighbouring output pixels share almost all of their inputs. A block of B x B outputs with radius r needs an input region of (B + 2r) x (B + 2r) pixels: the B x B core plus a halo r pixels wide on every side.

flowchart TB
    G["Global memory<br/>source image"] -->|"each thread loads<br/>one or more pixels"| T["Shared tile<br/>(B+2r) x (B+2r)<br/>core + halo"]
    T -->|"__syncthreads()"| C["Each thread reads its<br/>k x k neighbourhood<br/>from shared memory"]
    C --> D["Global memory<br/>output image"]

One block loads its core and halo once, then every thread filters from fast shared memory.

There are more tile pixels than threads, so the threads loop: each takes its index and then steps by the block size until the whole tile is loaded. Clamping happens while loading, so the compute loop needs no border logic at all:

C++
#define BX 16
#define BY 16

__global__ void blurTiled(const float* src, float* dst, int w, int h) {
    __shared__ float tile[BY + 2 * R][BX + 2 * R];

    int x0 = blockIdx.x * BX - R;             // top-left of the tile in image coordinates
    int y0 = blockIdx.y * BY - R;

    for (int ty = threadIdx.y; ty < BY + 2 * R; ty += BY) {
        for (int tx = threadIdx.x; tx < BX + 2 * R; tx += BX) {
            int sx = min(max(x0 + tx, 0), w - 1);
            int sy = min(max(y0 + ty, 0), h - 1);
            tile[ty][tx] = src[sy * w + sx];
        }
    }
    __syncthreads();                           // the whole tile is ready

    int x = blockIdx.x * BX + threadIdx.x;
    int y = blockIdx.y * BY + threadIdx.y;
    if (x < w && y < h) {                      // check AFTER the barrier
        float acc = 0.0f;
        for (int fy = 0; fy < K; ++fy)
            for (int fx = 0; fx < K; ++fx)
                acc += tile[threadIdx.y + fy][threadIdx.x + fx] * c_filter[fy * K + fx];
        dst[y * w + x] = acc;
    }
}

Two details matter. The bounds check wraps only the final computation and store: every thread must take part in the loads and reach __syncthreads(), even if its own pixel is outside the image. And the tile index threadIdx.y + fy already includes the halo offset, because tile row 0 is image row y0 = blockIdx.y * BY - R.

How much does this save? The next example does the real thing on a small image: each simulated block fills a halo tile (counting every pixel it fetches), then computes its outputs only from that tile. It uses a box filter for simplicity and compares sizes k = 3, 5, 7.

Python Halo tiles: results and global loads

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

For the 3 x 3 filter the four blocks each load a 10 x 10 tile, so 4 x 100 = 400 loads replace the naive 192 x 9 = 1,728, about 4.3 times fewer. At 7 x 7 the saving grows to 12 times, because the work per pixel grows with k^2 while the halo grows only with k. In the model, edge blocks pay for a full tile too (loading clamped duplicates), which is how the real kernel behaves. Real GPUs have an L1 cache that already absorbs some of the naive version's repeated loads, so measured gains are smaller. Profile before you believe any number.

Stage 4: separable filters

Many useful filters, including the Gaussian, are separable: the 2D weights equal the outer product of a 1D row and a 1D column. Then you can filter every row, and afterwards every column, with a 1D kernel. The work per pixel drops from k^2 to 2k multiply-adds, and each pass has a simpler halo (only to the left and right, or above and below).

Python A separable filter gives the same result with less work

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The two methods agree to rounding error. The price of the separable version is a second kernel launch and an intermediate image in global memory, so for small filters like 3 x 3 the single tiled kernel is often just as good. For large radii the separable version wins clearly.

Where to go next

  • Process colour images by storing three channels (planar RRRโ€ฆGGGโ€ฆBBBโ€ฆ coalesces better than interleaved RGBRGBโ€ฆ, see Global Memory and Coalescing).
  • Try a Sobel edge detector: only the filter weights change, the tiling stays.
  • Compare your kernel with a library routine such as NPP or cuDNN convolution (see The CUDA Library Ecosystem) and keep whichever is faster for your sizes.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. A 3 x 3 convolution with 16 x 16 blocks uses a shared tile of what size?

    The radius is 1, so the tile is (16 + 2) x (16 + 2) = 18 x 18: the core plus a one-pixel halo on every side.
  2. 2. Why is a small, read-only filter a good fit for constant memory?

    Constant memory is cached and broadcasts a single address to a whole warp in one step. Divergent addresses would be serialized.
  3. 3. In the tiled kernel, why must the bounds check come after the loading loops and __syncthreads()?

    Returning early would skip loads other threads depend on and could leave some threads out of the barrier, which hangs or corrupts the block.
  4. 4. What does a separable filter change?

    When the 2D filter is the outer product of two 1D filters, filtering rows then columns gives the same output with far fewer multiply-adds.

Summary

  • A convolution is an independent computation per pixel, so use a 2D grid with one thread per output pixel and a bounds check for the rounded-up grid.
  • Put small read-only filters in __constant__ memory, where a warp reading one address is served by a broadcast.
  • Halo tiles load (B + 2r)^2 pixels per block once, handle border clamping while loading, and cut global loads roughly by the filter area.
  • All threads must reach __syncthreads(); do the bounds check only around the final compute and store.
  • Separable filters trade one 2D pass for two cheap 1D passes. Keep a simple CPU reference to test every version.

Continue with Project: Fast Matrix Multiplication.