Skip to content
Tutorials Center
My learning

Topics

Lesson 31 of 40 ยท Architecture and Operations

Architecture, Scaling and Peak Traffic

How to structure an online shop and keep it fast under load: monolith vs headless vs composable, caching, queues, read replicas, search offload, flash sales, graceful degradation and observability.

A shop that works for ten visitors an hour can fall over at ten thousand. The reason is rarely the code you wrote for the happy path. It is the shape of the system: where pages are built, what is cached, which work happens while the customer waits, and what happens when one part gets slow. This lesson is about those shapes, and about the habits that keep a shop standing during a sale.

We stay platform-neutral. Whether you run a hosted platform, an open-source one or your own code, the same forces apply: product pages are read far more than they are written, checkout is small but precious, and a few popular products can attract most of the traffic.

Three ways to put a shop together

Most shops fall into one of three architectures. None is "best"; each trades simplicity for flexibility.

Monolith. One application renders pages, runs the cart and checkout, talks to the database and hosts the admin. Deployment is a single unit, a code change can touch any layer, and transactions across catalog, stock and orders are easy because they share one database. Most platforms start here, and most small and mid-size shops should stay here.

Headless. The commerce backend exposes an API (REST or GraphQL), and a separate front end, often a JavaScript application or a static site generator, renders the pages. You gain freedom over the user experience and can serve web, mobile apps and kiosks from one backend. You pay with two codebases, two deployments, and the work of rebuilding things the monolith gave you for free: previews, SEO plumbing, the checkout flow.

Composable. The shop is assembled from specialist services: one for the catalog, one for search, one for carts and checkout, one for payments, one for content. Each can be replaced independently. The price is integration work, many contracts to keep stable, and distributed failures. The integrations lesson covers how those pieces are kept in sync.

flowchart LR
  subgraph M["Monolith"]
    M1["App: pages + cart<br/>+ checkout + admin"] --> M2[("One database")]
  end
  subgraph H["Headless"]
    H1["Front end"] --> H2["Commerce API"] --> H3[("Database")]
  end
  subgraph C["Composable"]
    C1["Front end"] --> C2["Catalog"]
    C1 --> C3["Search"]
    C1 --> C4["Cart and checkout"]
    C4 --> C5["Payments"]
  end

Three architectures: more boxes mean more freedom, but also more places to fail.

A good rule: add a boundary only when it buys something you can name, such as an independent team, a different scaling profile or a replaceable vendor. "It is modern" is not a reason.

Cache the read path

On a typical shop, 95 percent or more of requests are anonymous reads: home page, category pages, product pages. They look the same for everyone, which makes them ideal for caching. Layers, from the customer inward:

  1. Browser cache for images, scripts and styles with long, fingerprinted file names.
  2. CDN or edge cache for HTML and API responses that are identical for all visitors. A CDN absorbs most of the traffic before it touches your servers.
  3. Application cache (for example Redis) for expensive computations: a menu tree, a category listing, a computed price table.
  4. Database query cache or read replica as the last line before the primary.

The hard part is not caching; it is invalidation and personalization. When a price or stock level changes, which cached pages are now wrong? Two common answers: give cached responses a short time to live (TTL), or tag them by product and purge the tags when the product changes. Keep per-user pieces (cart count, greeting, prices from a customer group) out of the cached HTML and load them with a small separate request, so the page itself stays shareable.

Two refinements matter at scale. Stale-while-revalidate serves the old copy instantly while one background request refreshes it. Request coalescing (sometimes called a cache lock) makes sure that when a popular entry expires, one request rebuilds it instead of a thousand at once; the pile-up is called a cache stampede.

JS Cache with TTL, stale-while-revalidate and a stampede guard

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Notice that the shop rebuilt the page three times for seven requests, and no visitor between t=12 and t=14 waited for a rebuild. Real systems add details (locks across servers, jitter on TTLs so that many keys do not expire together), but the idea is the same.

Move slow work off the request

Anything the customer does not need to wait for should leave the request. After payment succeeds, the customer needs to see "Thank you". They do not need to wait while you send the email, update the ERP, create the shipping label and refresh the search index. Put those into a queue, processed by background workers.

Queues give you three things at once: faster responses, smoothing of spikes (work waits in line instead of overloading the database), and retries when an external service is down. They also create obligations. Jobs may run twice, so they must be idempotent (see idempotency); they may run out of order; and you need to watch queue depth, because a growing queue is an early warning of trouble.

Read replicas and search offload

A single primary database handles writes (orders, stock changes) and, by default, reads. Browsing is read-heavy, so you can add read replicas and send catalog queries to them. The catch is replication lag: a replica may be a second behind. That is fine for a product description and dangerous for "is this item in stock right now?" or "show me the order I just placed". Rule of thumb: reads that follow your own write, and any decision that reserves stock or money, go to the primary.

Product listing with filters and free-text queries is also a bad fit for a relational database at scale. Counting facets across a hundred thousand products on every page view is expensive. A dedicated search engine holds a denormalized copy of the catalog and answers those queries quickly; the database stays focused on transactions. See search and filters for how indexing and facets work.

flowchart TB
  U["Visitors"] --> CDN["CDN / edge cache"]
  CDN --> APP["Application servers"]
  APP --> RC[("Redis cache")]
  APP --> SE["Search engine"]
  APP --> RR[("Read replica")]
  APP --> PR[("Primary database")]
  PR -. replication .-> RR
  APP --> Q["Job queue"]
  Q --> W["Workers: email, ERP, indexing"]

Reads fan out to caches, search and replicas; writes and reservations stay on the primary; slow work goes to the queue.

Flash sales and peak traffic

A flash sale is the worst case: huge traffic arrives at one instant, everybody wants the same few products, and stock is limited. Preparation looks like this:

  • Load test with a realistic shape. Many people browsing, fewer adding to cart, a few checking out. Test checkout, not just the home page.
  • Make the hot product page static. Pre-render it, cache it at the edge, and fetch only the stock status dynamically.
  • Protect the scarce resource. Stock decrements must be atomic and contention on a single row is the bottleneck; see reservations and overselling.
  • Use a waiting room. A queue page that lets customers into checkout at a rate your backend can handle is kinder than random errors.
  • Cap concurrency to expensive paths. Rate-limit by IP and account, and cap how many checkouts run at once. Bots will join the party.
  • Warm things up. Fill caches and open database connections before the doors open.

Degrade gracefully

Something will fail: the recommendation service, the tax API, the review widget. The question is whether the whole shop fails with it. Decide in advance what is essential (browse, cart, pay) and what is optional (recommendations, reviews, recently viewed). Optional features get short timeouts and a fallback, and a circuit breaker stops calling a dependency that is clearly down, so slow failures do not pile up and exhaust your workers.

JS A circuit breaker around an optional service

Press Run (or Ctrl + Enter) to see the console output here.

No output.

The page always renders something. After the third failure at t=2 the breaker opens, so the calls at t=3 and t=4 do not even try. At t=7 the cooldown has passed, one trial call fails and the breaker opens again; at t=13 the service is back and the trial succeeds.

Observe the system

You cannot defend what you cannot see. A shop needs three kinds of signals:

  • Metrics are numbers over time: requests per second, error rate, p95 and p99 latency, queue depth, cache hit rate, database connections. Add business metrics too: orders per minute, payment success rate, add-to-cart rate. A technical dashboard that is green while orders drop to zero is a lie.
  • Logs record events. Make them structured (JSON), and include a request ID and an order or cart ID so you can follow one customer's journey across services.
  • Traces show where time goes inside one request across services.

Alert on symptoms customers feel (checkout errors, slow pages, failed payments) rather than on every internal blip, and write down what each alert means and what to do about it. After an incident, record what happened and what will change; the same failure should not surprise you twice.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which request is the best candidate for an edge (CDN) cache?

    Anonymous reads that are identical for every visitor are safe and valuable to cache. Personal data and writes must not be served from a shared cache.
  2. 2. What problem does a stampede guard (request coalescing) solve?

    When a popular entry expires, every concurrent request would rebuild it. A guard lets one request rebuild while others wait or get the stale copy.
  3. 3. Why should a stock reservation read from the primary database instead of a read replica?

    Replication lag means a replica can be slightly behind. Decisions that spend a scarce resource need the authoritative, current value.
  4. 4. What does a circuit breaker do for an optional recommendation service that is down?

    After repeated failures the breaker skips the dependency for a cooldown period, keeping the essential parts of the shop fast.

Summary

  • Monolith, headless and composable architectures trade simplicity against flexibility; add a boundary only for a reason you can name.
  • Most traffic is anonymous reads: cache at the browser, CDN, application and database layers, and keep personal data out of shared HTML.
  • Use TTLs, stale-while-revalidate and stampede guards to keep caches fast and safe.
  • Push slow work (email, ERP sync, indexing) to idempotent queue jobs.
  • Send catalog reads to replicas and search engines, but keep reservations and "just ordered" reads on the primary.
  • For flash sales: load test checkout, protect the scarce row, use a waiting room and rate limits.
  • Degrade gracefully with timeouts, fallbacks and circuit breakers; watch business metrics, structured logs and traces.

Next, see how a shop exchanges data with the systems around it in Integrations: ERP, PIM, OMS and Marketplaces.