Skip to content
Tutorials Center
My learning

Topics

Lesson 29 of 40 ยท Data and Science

Random Numbers and Statistics

Generate random numbers with rand and randn, set seeds for repeatable results, summarize data with Statistics, and use Distributions.jl, with NumPy stand-ins.

Simulations, sampling, shuffling a deck, testing an algorithm on made-up data: all of them need random numbers. And once you have data, real or simulated, you want to describe it with a mean, a spread and a few quantiles. Julia ships both abilities. rand and randn are in Base, the Random and Statistics standard libraries add the rest, and the Distributions.jl package covers probability distributions.

In this lesson you will draw numbers from ranges, collections and the normal distribution, make a computation repeatable with a seed, summarize data with the Statistics functions, and work with distribution objects. Julia is not installed on this site, so the Julia code is read-only. The runnable examples use NumPy, which has equivalents for almost every call. I deliberately do not print the numbers that real Julia would give you, because random values depend on the generator and the Julia version.

rand and randn

rand() returns a random Float64 in the half-open interval from 0 up to, but not including, 1. Give it dimensions to get arrays, a range to get integers, or a collection to pick from it:

JULIA
rand()                   # one Float64 in [0, 1)
rand(3)                  # a Vector of 3 values
rand(2, 4)               # a 2ร—4 Matrix
rand(1:6)                # one die roll
rand(1:6, 10)            # ten die rolls
rand(['a', 'b', 'c'])    # a random element of a collection
rand(Bool)               # true or false

randn() works the same way but draws from the standard normal distribution, the bell curve with mean 0 and standard deviation 1. Values are mostly between -3 and 3 and can be any real number:

JULIA
randn()                  # one normal value
randn(1000)              # 1000 of them

To get normal values with a different center and spread, scale and shift: 10 .+ 2 .* randn(1000) has mean about 10 and standard deviation about 2 (the dots are covered in Broadcasting and the Dot Syntax).

Shuffling and permutations

The Random standard library has helpers for ordering. It has to be loaded with using:

JULIA
using Random

shuffle([1, 2, 3, 4, 5])     # a shuffled copy
shuffle!(v)                  # shuffles v in place
randperm(5)                  # a random ordering of 1:5
randstring(8)                # a random string of letters and digits

randperm is handy for splitting data into training and test sets: take the first 80% of a random ordering of the row numbers.

Seeds: making randomness repeatable

A computer's "random" numbers come from a deterministic algorithm that starts from a seed. Starting from the same seed gives the same sequence, which makes an experiment reproducible and a bug easy to chase:

JULIA
using Random

Random.seed!(42)
a = rand(3)

Random.seed!(42)
b = rand(3)

a == b      # true

Random.seed! resets the global generator. A cleaner, safer habit for real programs is to create your own generator object and pass it explicitly. The default family, called Xoshiro, is fast and has a good statistical quality:

JULIA
rng = Xoshiro(2024)

rand(rng, 5)
randn(rng, 3)
shuffle(rng, 1:10)

Because each piece of code carries its own rng, the code does not disturb other parts of the program, and you can give each thread or task its own generator. Keep in mind that Julia does not promise the same stream across versions, so a seed makes a run reproducible on one version, not forever.

Where numbers come from

rand Uniform, integers and picks.
randn Normal, the bell curve.
shuffle, randperm Random orderings.
Seed or rng Make a run repeatable.
1 / 2

Summarizing data with Statistics

The Statistics standard library supplies the descriptive functions. They work on vectors, ranges and matrices:

JULIA
using Statistics

data = [4.0, 8.0, 15.0, 16.0, 23.0, 42.0]

mean(data)               # 18.0
median(data)             # 15.5
var(data)                # the sample variance
std(data)                # the sample standard deviation
extrema(data)            # (4.0, 42.0), from Base
quantile(data, 0.9)      # the 90th percentile
quantile(data, [0.25, 0.5, 0.75])    # the quartiles

The variance and standard deviation use the sample formula by default, dividing by n - 1. That matches R and pandas but not NumPy, whose np.std divides by n unless you pass ddof=1. If you ever compare numbers between tools, this is the first thing to check. In Julia you can ask for the population version with std(data, corrected = false).

On a matrix, pass dims to summarize by column or by row:

JULIA
A = randn(100, 3)

mean(A, dims = 1)        # a 1ร—3 matrix: the mean of each column
std(A, dims = 1)
mean(A, dims = 2)        # 100ร—1: the mean of each row

For two variables, cor(x, y) gives the correlation and cov(x, y) the covariance. Given a matrix, cor(A) returns the correlation matrix of its columns. Functions for weighted means, histograms, ranks and sampling without replacement live in the StatsBase package.

Distributions.jl: objects for probability distributions

rand and randn cover simple cases. For anything else, Distributions.jl represents a distribution as an object, and a small set of functions works on all of them. Install with ] add Distributions.

JULIA
using Distributions

d = Normal(100, 15)          # mean 100, standard deviation 15

rand(d)                      # one draw
rand(d, 1000)                # 1000 draws
mean(d), std(d)              # (100.0, 15.0)
pdf(d, 100)                  # height of the bell curve at 100
cdf(d, 115)                  # P(X โ‰ค 115), about 0.84
quantile(d, 0.975)           # the value below which 97.5% of the mass lies

pdf is the probability density, cdf the cumulative probability, and quantile is the inverse of cdf. Because the same functions work for all distributions, you can swap one for another by changing a single line:

JULIA
Binomial(10, 0.5)            # number of heads in 10 fair coin flips
Poisson(3.0)                 # events per interval when the average is 3
Uniform(0, 1)                # what rand() draws from
Exponential(2.0)             # waiting times with mean 2
Bernoulli(0.3)               # one yes/no event

You can also fit a distribution to data. fit(Normal, data) estimates the mean and standard deviation, which makes a quick model of a measured variable, and loglikelihood(d, data) tells you how well a distribution explains it.

Simulation: estimating a probability

The simplest use of random numbers is the Monte Carlo method: estimate a probability by repeating an experiment many times and counting. What is the chance that two dice add up to 7? The exact answer is 6/36, or about 0.1667:

JULIA
using Random

rng = Xoshiro(1)
rolls = 100_000

hits = count(_ -> rand(rng, 1:6) + rand(rng, 1:6) == 7, 1:rolls)

hits / rolls        # close to 1/6

The error shrinks as the number of trials grows, in proportion to one over the square root of the count, so a hundred times more trials gives about ten times more accuracy. The project on Monte Carlo estimation applies the idea to estimating pi and pricing options.

Running the ideas with NumPy

The calls below mirror the Julia ones. NumPy uses a generator object (np.random.default_rng), which is like passing your own rng in Julia.

Python Seeds, shuffling and summary statistics

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Python A normal distribution and a Monte Carlo estimate

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The shapes of the results (the same values for the same seed, a sample that agrees with the formula, an estimate close to 1/6) are what real Julia gives. The exact digits of the random draws are not the same as Julia's, because the generators differ.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What does rand() with no arguments return?

    rand() draws uniformly from the interval [0, 1). randn() is the normal one, and rand(Bool) gives true or false.
  2. 2. How can you make a random computation give the same numbers on every run?

    A seed fixes the starting state of the deterministic generator, so the sequence repeats.
  3. 3. Which is true of Julia's std(x) by default?

    Julia and pandas use the corrected sample formula by default. NumPy needs ddof=1 to match, and Julia needs corrected=false to match NumPy.
  4. 4. With d = Normal(0, 1), what does cdf(d, 0) return?

    The cdf is the probability of a value at or below the argument. Half of a symmetric distribution lies below its center. 0.3989 is the height of the density, pdf(d, 0).
  5. 5. In mean(A, dims = 1) for a 100ร—3 matrix, what shape is the result?

    dims = 1 collapses the rows, leaving one mean per column, so a 1ร—3 matrix.

Summary

  • rand draws uniform values, integers from ranges and elements of collections; randn draws from the standard normal.
  • Random adds shuffle, randperm and Random.seed!; pass your own rng = Xoshiro(seed) for reproducible, well-separated randomness.
  • Statistics provides mean, median, std, var, quantile, cor and cov; std and var use n - 1 by default, and dims summarizes columns or rows.
  • Distributions.jl represents distributions as objects that work with rand, pdf, cdf, quantile, mean and fit.
  • Monte Carlo estimates probabilities by repeating experiments, and their error shrinks with the square root of the trial count.

Next: check that your code does what you claim with Testing with Test.jl.