Skip to content
Tutorials Center
My learning

Topics

Lesson 28 of 31 ยท Projects

Project: Monte Carlo Pi and Option Pricing

Use cuRAND with one random state per thread, reduce results across blocks and judge statistical accuracy in a Monte Carlo pi and option pricing project.

Monte Carlo methods answer a question by random sampling: throw many random darts, count how many land where you care, and read the answer off the fraction. They are a natural fit for GPUs because every sample is independent. A million samples are simply a million threads (or fewer threads with a loop each) that never need to talk to each other until the final sum. Monte Carlo is used for physics, finance, graphics and risk analysis.

This project has three parts: estimating pi (the "hello world" of Monte Carlo), a closer look at how random numbers must be handled per thread, and pricing a financial option, where you also learn to say how accurate your answer is. The CUDA code uses the cuRAND device API and is read-only. The runnable examples are vectorized NumPy stand-ins in which a 2D array plays the role of the thread grid: one row per thread.

The site has no GPU, so the CUDA is read-only. The Python examples are real programs, and the statistics they print are real statistics. They use NumPy's generators, not cuRAND, so the exact random numbers differ from what a GPU would draw. The principles (independent streams, reduction, error shrinking as 1/sqrt(N)) are the same.

Estimating pi

Draw points uniformly in the unit square. The quarter circle x^2 + y^2 <= 1 has area pi/4, so the fraction of points inside it approaches pi/4. Therefore pi ~ 4 * hits / samples.

Each GPU thread needs its own random number generator, and this is the part beginners get wrong. A generator keeps state (an internal number that changes with every draw). If threads share one generator they would fight over it. So each thread keeps its own state in a global-memory array, created once by an initialization kernel:

C++
#include <curand_kernel.h>

__global__ void initStates(curandState* states, unsigned long long seed, int n) {
    int id = blockIdx.x * blockDim.x + threadIdx.x;
    if (id < n) {
        // seed, subsequence number, offset inside the subsequence, state to fill
        curand_init(seed, id, 0, &states[id]);
    }
}

The same seed is used by all threads, but the subsequence argument (id) is different, which jumps each thread to a far-away, non-overlapping part of one huge random sequence. That is what makes the streams independent. Now the sampling kernel. Each thread draws perThread points and counts its hits, the block adds up its threads' counts in shared memory (a tree reduction as in Parallel Reduction), and thread 0 of each block adds the block total to a global counter with one atomic:

C++
__global__ void piKernel(curandState* states, unsigned long long* total,
                         int perThread, int n) {
    __shared__ unsigned int blockHits[256];                // launch with 256 threads
    int id = blockIdx.x * blockDim.x + threadIdx.x;

    unsigned int hits = 0;
    if (id < n) {
        curandState local = states[id];                    // copy the state into registers
        for (int s = 0; s < perThread; ++s) {
            float x = curand_uniform(&local);              // in (0, 1]
            float y = curand_uniform(&local);
            if (x * x + y * y <= 1.0f) ++hits;
        }
        states[id] = local;                                // save for later launches
    }
    blockHits[threadIdx.x] = hits;
    __syncthreads();

    for (int stride = blockDim.x / 2; stride > 0; stride >>= 1) {
        if (threadIdx.x < stride) blockHits[threadIdx.x] += blockHits[threadIdx.x + stride];
        __syncthreads();
    }
    if (threadIdx.x == 0) atomicAdd(total, (unsigned long long)blockHits[0]);
}
// pi = 4.0 * total / ((double)n * perThread)

Three details are worth remembering. Working on a local copy of the state keeps it in a register for the loop instead of reading and writing global memory on every draw. Writing the state back lets a later launch continue the sequence without repeating numbers. And the final accumulation uses a 64-bit counter, because 32 bits overflow after about four billion hits, and a serious run draws more than that.

What goes wrong without independent streams

What if every thread is seeded identically? Then every thread produces exactly the same numbers and the "million samples" are the same few samples repeated. The simulation below shows the effect with NumPy generators. "Same seed" gives each of 64 simulated threads default_rng(1234), while "spawned" derives 64 independent streams from one seed with SeedSequence.spawn, which is NumPy's counterpart to cuRAND's subsequence idea.

Python Per-thread random streams: same seed versus independent seeds

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

All three threads with the same seed start with the identical number, and the 64-thread estimate equals the single-thread estimate bit for bit: 128,000 samples were drawn, but only 2,000 distinct ones. The independent streams use all 128,000 and land much closer to pi. The bug is silent: the program runs, prints a number that looks plausible, and is simply less accurate than it claims.

Reduction and the cost of accuracy

The kernel above does three levels of summation: a register counter in each thread, a shared-memory tree per block, and an atomic across blocks. The next stand-in has the same shape, with rows as threads. It also measures how the error shrinks, which is the central fact about Monte Carlo: the error falls as 1/sqrt(N). To gain one more digit of accuracy you need 100 times more samples. Theory says the standard error of the pi estimate is 4 * sqrt(p(1-p)/N) with p = pi/4.

Python Thread, block and global reduction, and the 1/sqrt(N) law

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The first table shows single runs, so their errors scatter around the expected standard error: usually below it, but the first row is above it. The repeated experiments tell the clean story: the measured root-mean-square error matches the theoretical value and shrinks by about 3 (the square root of ten is 3.16) for every tenfold increase in N. On a real GPU you reach for billions of samples, so the cost of that last digit is exactly why variance-reduction tricks (antithetic variates, importance sampling, quasi-random sequences) matter.

Note one thing that is not the same on a GPU: floating-point sums depend on the order of the additions, and the order of block completion is not fixed. A sum of float values can therefore differ in the last bits from run to run when it uses atomics. Counting integers (as with hits) is exact and repeatable, which is another reason to prefer integer counters when you can. For the effect of summation order see Parallel Reduction.

Pricing a European call option

The second application is finance. A European call option pays max(S_T - K, 0) at time T, where S_T is the stock price at expiry and K the agreed strike price. In the standard model the price follows geometric Brownian motion, and the end price can be sampled directly with a single normal random number z:

S_T = S0 * exp((r - sigma^2/2) * T + sigma * sqrt(T) * z)

The option's fair price today is the average discounted payoff, exp(-r*T) * mean(max(S_T - K, 0)), over many samples. Every sample is independent, so the kernel is as parallel as the pi kernel. It uses curand_normal and sums payoffs in double to limit rounding errors; the payoff and its square are accumulated so that the host can also compute the standard error:

C++
__global__ void callKernel(curandStatePhilox4_32_10_t* states,
                           double* sum, double* sumSq,
                           int perThread, int n,
                           float S0, float K, float r, float sigma, float T) {
    int id = blockIdx.x * blockDim.x + threadIdx.x;
    double s = 0.0, s2 = 0.0;
    if (id < n) {
        curandStatePhilox4_32_10_t local = states[id];
        float drift = (r - 0.5f * sigma * sigma) * T;
        float vol = sigma * sqrtf(T);
        for (int i = 0; i < perThread; ++i) {
            float z = curand_normal(&local);
            float ST = S0 * expf(drift + vol * z);
            double payoff = fmax(ST - K, 0.0f);
            s += payoff;  s2 += payoff * payoff;
        }
        states[id] = local;
    }
    // reduce s and s2 across the block (shared memory), then:
    // if (threadIdx.x == 0) { atomicAdd(sum, blockSum); atomicAdd(sumSq, blockSumSq); }
}
// host: price = exp(-r*T) * sum / N;  variance = sumSq/N - (sum/N)^2

The Philox generator is a popular choice on GPUs because its state is small (fewer registers and less memory traffic than the default generator) and it supports jumping to any position cheaply. atomicAdd on double requires compute capability 6.0 or newer.

How can we know the answer is right? For this simple option there is a closed-form price, the Black-Scholes formula, which gives us something to check against. And for any Monte Carlo result, the sample standard deviation gives a confidence interval: mean +/- 1.96 * stddev / sqrt(N) contains the true value in about 95 percent of experiments.

Python Option price with a confidence interval, checked against Black-Scholes

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Every Monte Carlo result must come with its error bar. Here the standard error shrinks from about 0.39 at 1,000 samples to about 0.0125 at a million, matching the 1/sqrt(N) law, and the intervals narrow around the exact price of 7.1281. One of the single runs (N = 10,000) misses the exact price: that is not a bug but statistics, since about one interval in twenty is expected to. The last line checks this claim directly: over 200 repeated runs, roughly 95 percent of the intervals contain the exact value.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Why does each GPU thread need its own cuRAND state?

    A generator's state changes on every draw, so each thread keeps a private state in registers or global memory and draws from its own stream.
  2. 2. What does the subsequence argument of curand_init(seed, subsequence, offset, &state) achieve?

    Using the thread id as the subsequence gives every thread its own far-apart slice of one long sequence.
  3. 3. To make a Monte Carlo estimate ten times more accurate you need about how many times more samples?

    The error falls as 1/sqrt(N), so a tenfold gain in accuracy needs a hundredfold increase in samples.
  4. 4. Why accumulate the hit count in a 64-bit integer?

    Large simulations draw more than 2^32 samples, so a 32-bit total would wrap around and silently give a wrong answer.
  5. 5. What is a good way to check a Monte Carlo option price?

    A closed-form value (Black-Scholes) is an independent check, and the standard error tells you how much of the difference is expected.

Summary

  • Monte Carlo is embarrassingly parallel: each thread draws samples with its own cuRAND state, seeded with the same seed but a different subsequence.
  • Copy the state into a local variable for the sampling loop and write it back, so later launches continue the stream.
  • Reduce in three levels (thread registers, block shared memory, one atomic per block) and use 64-bit integers for large counts.
  • The statistical error shrinks as 1/sqrt(N); report a confidence interval, and validate against an analytic result whenever you can.
  • Identical streams in all threads are a silent bug: the answer looks fine but contains far fewer independent samples than you paid for.

Continue with Practice Exercises with Solutions.