Skip to content
Tutorials Center
My learning

Topics

Lesson 30 of 70 ยท Databases

ElastiCache, DocumentDB, Neptune and Redshift

Learn caching strategies (lazy loading, write-through, TTL), Redis vs Memcached, and when to pick DocumentDB, Neptune or Redshift, with a runnable cache emulator.

So far you have met two AWS databases: RDS and Aurora for relational data, and DynamoDB for key-value access at any scale. AWS also sells purpose-built databases for narrower jobs. The most common one in everyday applications is not a primary database at all, but a cache: a fast in-memory copy of data that lives somewhere slower.

This lesson explains caching strategies with a runnable emulator, compares Redis and Memcached on ElastiCache, and then gives a short tour of DocumentDB, Neptune and Redshift, with a simple way to choose between all of them.

None of these services can run on this site, and there is no AWS account here. The CLI commands are read-only. The runnable examples are plain Python stand-ins: a dictionary plays the cache, another plays the database, and a fake clock replaces real time so the output is the same every run.

Why cache?

A database read may take milliseconds and costs capacity; an in-memory read takes well under a millisecond and costs almost nothing. If the same data is read again and again (a product page, a user profile, a leaderboard), putting a cache in front of the database removes most of that load. The price is that you now have two copies, and they can disagree. Every caching strategy is a different answer to "when do I fill the cache, and when do I fix stale data?"

flowchart LR
    App["Application"] -->|"1. read"| Cache["Cache<br/>(ElastiCache)"]
    Cache -->|"hit: return value"| App
    App -->|"2. miss: read"| DB["Database<br/>(RDS, DynamoDB)"]
    DB -->|"value"| App
    App -->|"3. store with TTL"| Cache

Lazy loading (cache-aside): the application checks the cache first and fills it on a miss.

Lazy loading (cache-aside)

With lazy loading, the application owns the logic:

  1. Look in the cache. If the item is there (a hit), use it.
  2. If not (a miss), read the database, put the result in the cache with a TTL (time to live), and return it.

Strengths: only data that is actually requested gets cached, and a cache failure is survivable, because you can fall back to the database. Weaknesses: the first request for each item is slow, and data can be stale until its TTL runs out or you delete the key.

Python Cache-aside with a TTL on a fake clock

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Seven reads caused only two database reads: the first miss at t=0, and a second at t=61 when the 60-second TTL had run out. After the update, the read at t=70 still returned the old name, and only the read at t=130 (the entry from t=61 had expired at t=121) saw the new one. That delay is exactly the staleness trade-off. A short TTL means fresher data and more database reads; a long TTL does the opposite.

Write-through

With write-through, every write goes to the database and to the cache in the same code path. Reads are almost always hits, and data is never stale in the cache because it is updated with the database. The costs: you cache data that may never be read (wasting memory), and a write now has two steps that can fail halfway. Many systems combine both ideas: write-through (or delete the key on write) plus lazy loading plus a TTL as a safety net.

A third option is write-behind (or write-back), where the application writes to the cache and a background process saves to the database later. It is fast, but risky: a cache node failure can lose data. ElastiCache does not do this for you; you would build it yourself.

Python Lazy loading versus write-through after an update

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

With lazy loading and no invalidation, the reader after the update still sees the old price 2. With write-through, it sees 3 without touching the database. In real code the common fix for lazy loading is to delete the cache key on every write, which keeps the cache simple and still correct.

Amazon ElastiCache: Redis and Memcached

Amazon ElastiCache runs managed in-memory caches inside your VPC. It handles node provisioning, patching, failover and monitoring. You choose an engine:

Redis (and Valkey) Memcached
Data types Strings, hashes, lists, sets, sorted sets, streams Strings only
Persistence and backups Yes (snapshots) No
Replication and automatic failover Yes (replicas across AZs) No
Multi-threaded Mostly single-threaded per shard, scale by sharding Yes, uses multiple cores per node
Typical use Sessions, leaderboards, queues, rate limits, caches you cannot lose Simple, disposable, very fast object cache

Valkey is an open-source fork of Redis that ElastiCache also offers, with the same client protocol. If you are unsure, choose Redis or Valkey: it covers everything Memcached does, plus features you will probably want later. ElastiCache also has a serverless option that scales capacity for you.

A cluster is reachable only from inside your network, so you control access with security groups. Creating one from the CLI looks like this (read-only):

Terminal
aws elasticache create-replication-group \
  --replication-group-id shop-cache \
  --replication-group-description "Shop cache" \
  --engine redis \
  --cache-node-type cache.t4g.small \
  --num-cache-clusters 2 \
  --automatic-failover-enabled \
  --cache-subnet-group-name private-subnets \
  --security-group-ids sg-0123456789abcdef0

DynamoDB DAX

DynamoDB has its own dedicated cache called DAX (DynamoDB Accelerator). It sits in front of a table and speaks the DynamoDB API, so reads that hit the cache return in microseconds with little code change. Use DAX when your bottleneck is repeated reads of the same DynamoDB items. For anything more general, ElastiCache gives you more control.

DocumentDB, Neptune and Redshift

Three more services are worth knowing by name:

  • Amazon DocumentDB stores JSON-like documents and offers a MongoDB-compatible API. It is a good fit when you already have MongoDB drivers and code and want a managed service. It does not support every MongoDB feature, so check compatibility before migrating.
  • Amazon Neptune is a graph database. You use it when relationships are the data: social networks, fraud rings, recommendations, knowledge graphs. Queries use graph languages such as Gremlin, openCypher or SPARQL, and walking many hops is fast where SQL joins would be painful.
  • Amazon Redshift is a data warehouse. It stores data in columns and runs SQL over large amounts of data (gigabytes to petabytes) for reports and analytics. It is built for a few heavy queries, not for thousands of tiny transactions per second.

There are more (Timestream for time series, Keyspaces for Cassandra, MemoryDB for a durable Redis-compatible database), but these cover most conversations.

Choosing the right database

Start from the shape of the data and the access pattern, not the product name.

Python A small database chooser

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Real choices are rarely this clean, since one application often uses several databases together: Aurora as the system of record, ElastiCache in front for hot reads, Redshift for reporting. The chooser just shows the questions to ask first.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. In the lazy loading (cache-aside) strategy, what happens on a cache miss?

    The application owns the logic. It checks the cache first, and on a miss it loads from the database and fills the cache, usually with a TTL.
  2. 2. Which problem is typical for lazy loading without any invalidation?

    Writes go only to the database, so the cache keeps the old value. Use a TTL, delete the key on write, or use write-through.
  3. 3. When is Redis the better ElastiCache engine than Memcached?

    Redis (and Valkey) offers rich data types, replication, failover and persistence. Memcached is simpler and multi-threaded but has none of those features.
  4. 4. Which service fits a social network where you need friends of friends of friends quickly?

    Neptune is a graph database made for traversing relationships over many hops.
  5. 5. Write-through caching means...

    Write-through updates both stores on each write. Reads are fast and fresh, but you may cache data nobody reads.

Summary

  • A cache trades freshness for speed; every strategy decides when to fill it and how to handle stale data.
  • Lazy loading fills the cache on a miss and uses TTLs; write-through updates the cache on every write. Deleting the key on write is the simple fix for stale lazy data.
  • ElastiCache offers Redis/Valkey (rich types, replicas, snapshots) and Memcached (simple, multi-threaded); DAX is the cache made for DynamoDB.
  • DocumentDB is MongoDB-compatible documents, Neptune is a graph database, Redshift is a columnar data warehouse for analytics.
  • Choose by data shape and access pattern, and expect to combine several databases in one system.

Next: Lambda Functions