Skip to content
Tutorials Center
My learning

Topics

Lesson 23 of 37 ยท Data Modeling Patterns

Hot Partitions and Write Sharding

Understand why uneven DynamoDB keys cause throttling, how partition limits work, and how to fix hot keys with write sharding and scatter-gather reads.

DynamoDB scales by splitting a table into partitions. Each item lives in the partition chosen by hashing its partition key. A table with millions of requests per second is only possible because those requests spread across many partitions. When they do not spread, when most traffic hits one key, you get a hot partition: that partition is throttled while the rest of the table sits idle.

A single partition can serve roughly 3,000 read capacity units and 1,000 write capacity units per second (the item-collection and per-partition limits are a documented design constraint). Adaptive capacity and burst capacity soften short spikes and can isolate a hot item, but they cannot make one key unlimited. The fix is in the data model, and this lesson shows the patterns.

What makes a key hot

  • Low cardinality. A partition key like STATUS#ACTIVE or a date 2026-03-09 for all of today's events puts all writes on one key.
  • Skewed popularity. A celebrity account, a viral product, a global counter, a "settings" item read by every request.
  • Time-ordered writes to one key. Writing every event to PK = LOGS because you want them sorted by time.
  • Sequential keys with a low-cardinality prefix. Many writers appending to the same logical collection.

The symptom is ProvisionedThroughputExceededException (or ThrottlingException) even though CloudWatch shows the table well under its total capacity. Contributor Insights can show you the most accessed keys.

Spread the load

Hot key One PK gets most traffic
Shard suffix Add a number 0 to N-1
Scatter-gather Query every shard, merge

More keys, same data.

1 / 2

Choosing a better partition key

The best fix is a key with many distinct values and even traffic: user ID, device ID, order ID. If your pattern is "all events of today", do not make the date the partition key. Make the device the partition key, and let a GSI or another view answer the date question.

When the busy entity is inherent (all events for a global leaderboard, one huge tenant), use sharding.

Write sharding

Write sharding appends a small number to the partition key: EVENTS#2026-03-09#7. Each write picks a shard, either randomly or by hashing something stable (like the event ID), so the load spreads over N keys. With N=10 you can absorb roughly ten times the write rate.

Choose how to pick the shard suffix:

  • Random (random.randint(0, N-1)): the best spread, but you cannot know which shard holds a specific item without checking all shards.
  • Calculated from the item (crc32(event_id) % N): reads by ID can go straight to the right shard, which keeps GetItem possible.
Python See the skew, then shard it

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The busiest shard now carries about a tenth of the writes. Because the shard is computed from the event ID, a GetItem for a single event still needs only one key.

Scatter-gather reads

Sharding moves the cost to reads: "all events for the day" now means querying all N shards and merging. This is called scatter-gather. You can run the N queries in parallel (they are independent), then sort the combined results in the application. If the sort key is a timestamp, a k-way merge of already sorted lists works well.

Python
# Real-service sketch with boto3 (read-only here).
import boto3
from concurrent.futures import ThreadPoolExecutor

table = boto3.resource("dynamodb").Table("Events")

def read_shard(day, shard):
    resp = table.query(
        KeyConditionExpression="PK = :pk",
        ExpressionAttributeValues={":pk": f"EVENTS#{day}#{shard}"},
    )
    return resp["Items"]

with ThreadPoolExecutor(max_workers=10) as pool:
    parts = list(pool.map(lambda s: read_shard("2026-03-09", s), range(10)))
items = sorted((i for part in parts for i in part), key=lambda i: i["SK"])

Pick N from your measured need, not from fear. Ten shards turn one read into ten, so over-sharding multiplies read cost.

Python Scatter-gather with a merge

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Other hot-key remedies

  • Cache hot reads with DynamoDB Accelerator (DAX) or an application cache. Most skewed read traffic repeats the same items.
  • Aggregate before writing. A global counter updated 5,000 times a second can be aggregated in memory per worker and written once a second, or split into several counter items and summed on read.
  • Use on-demand capacity to remove the need to guess a rate, but note that a single partition still has its own ceiling.
  • Randomize time buckets (EVENTS#2026-03-09T14#3) to keep each bucket's partition cool and each query bounded.
  • Move heavy items out of the table (S3 for large payloads) to lower write units per request.

Do not shard by default. Sharding makes every read more expensive and more complicated. Use it only for partition keys that measurement shows will be hot, and start with the smallest N that gives headroom.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why can throttling happen when table-level capacity looks underused?

    Throughput is served per partition. A hot key exhausts its partition even if other partitions are idle.
  2. 2. What is write sharding?

    A shard suffix (for example 0 to 9) multiplies the number of distinct partition keys that receive the writes.
  3. 3. What is the cost of sharding?

    Writes get cheaper to spread but a read across the logical key now needs one query per shard.
  4. 4. Which partition key is least likely to be hot?

    High-cardinality keys with even traffic spread across partitions. Constants, statuses and dates concentrate load.

Summary

  • Each partition has a throughput ceiling; uneven keys cause throttling on a hot partition.
  • Prefer high-cardinality partition keys with even traffic.
  • Write sharding adds a suffix (random or calculated) to spread writes over N keys.
  • Reads across shards use scatter-gather: query every shard, then merge.
  • Cache, aggregate and use on-demand capacity before sharding.

Next: Capacity Modes: On-Demand and Provisioned