Skip to content
Tutorials Center
My learning

Topics

Lesson 32 of 37 ยท Experimentation Platforms

How Experiment Assignment Works: Hashing and Bucketing

Build deterministic user assignment for experiments: hash-based bucketing, per-experiment salts, ramping traffic without reshuffling users, and chi-square checks, with runnable Python code.

Every experimentation platform must answer one question millions of times per second: which variant does this user get? The answer has to be random-looking, sticky (the same user always gets the same variant), fast, and independent across experiments. The standard solution is hash-based bucketing, and it needs no database lookup at all.

The recipe

  1. Concatenate the experiment's salt (usually its unique key) and the user's id.
  2. Hash the string with a stable hash function and turn the result into an integer.
  3. Take the integer modulo a fixed number of buckets (commonly 10,000).
  4. Map bucket ranges to variants, for example 0-4999 to control and 5000-9999 to treatment.

Because hashing is deterministic, the same user and experiment always give the same bucket. Because good hashes behave like random numbers, users spread uniformly.

Hash-based assignment

salt + user id Experiment key and user
Hash, modulo 10,000 A stable bucket number
Bucket ranges Ranges map to variants
Sticky and fast No database needed

Same user, same experiment, same variant

1 / 2
Python Deterministic assignment

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The first and last users are the same person and get the same bucket, with no storage involved.

Why the salt matters: independence across experiments

If every experiment hashed the user id alone, a user in the control of experiment 1 would also be in the control of experiment 2, and the experiments would be correlated: effects from one would leak into the other. Using a different salt per experiment makes assignments independent.

Python Independence between two experiments

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

With separate salts every combination holds about a quarter of the users. With a shared salt only (C, C) and (T, T) appear: you could not separate the effects of the two experiments.

Checking the split is healthy

Run a chi-square goodness-of-fit test on the counts per arm. A tiny p-value signals a broken assignment; it is the same idea as a sample ratio mismatch check.

Python Is the split balanced?

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

A large p-value (above 0.001, the typical alarm threshold) means the split looks as intended.

Ramping traffic without reshuffling users

You often start an experiment at 5% of traffic and ramp to 50%. If you assigned users by a rule such as "bucket < 500 means treatment", a ramp must add buckets and never move users between arms. Arrange allocation as nested ranges:

  • Treatment: buckets 0-499 at 5%, 0-2499 at 25%, 0-4999 at 50%.
  • Control: the mirror ranges from the top end, or a separate carve-out.

Users already in treatment stay there; newly included users come from untouched buckets.

Python Nested ramp keeps earlier users stable

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Do not change the salt or the hash function in the middle of an experiment. Every user would be reshuffled and the earlier data becomes unusable. Also avoid the language's built-in hash() for assignment: Python randomizes it per process, so assignments would not be stable.

Practical details

  • Choose the right unit. User id for logged-in features, device or cookie id for anonymous visitors, account or company id for B2B products. See randomization and assignment units.
  • Hash function: MD5, SHA-1, SHA-256 or Murmur3 all work; speed matters at scale. Cryptographic strength is not required, only uniformity and stability.
  • Mutual exclusion and layers: to keep two experiments from ever touching the same user, put them in one layer: users are hashed into buckets once, and each experiment owns a disjoint bucket range.
  • Holdouts: reserve a few percent of buckets that are never exposed to any experiment to measure the cumulative effect of all launches.
  • Logging: log the assignment at the moment the user is exposed to the variant, not when the hash is computed, and include the experiment key and variant.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why is hash-based assignment sticky?

    Hashing is a deterministic function: identical input gives identical output.
  2. 2. What goes wrong if all experiments share one salt?

    The same user would land in the same bucket for every experiment, mixing effects.
  3. 3. To ramp traffic from 5% to 25% safely, you should...

    Nested ranges mean users already in treatment stay in treatment.
  4. 4. Why not use Python's built-in hash() for assignment?

    String hashing in Python is salted randomly each run, so values change between processes.

Summary

  • Assignment = hash(salt + user id) mod buckets, mapped to variant ranges. It is stable, fast and storage-free.
  • Use a per-experiment salt to keep experiments independent; use layers for mutual exclusion.
  • Verify the split with a chi-square test and never change salt or hash mid-test.
  • Ramp by expanding bucket ranges so users never switch arms.

Next: Feature Flags and Gradual Rollouts