Skip to content
Tutorials Center
My learning

Topics

Lesson 4 of 65 ยท Thinking in Systems

Scalability - Vertical and Horizontal Scaling

Compare scaling up and scaling out, learn when each wins, see how auto scaling reacts to load, and simulate both with runnable Python code.

A system is scalable if it can handle more load by adding resources, without a redesign and without the cost or latency growing out of proportion. "Load" can mean users, requests per second, data volume or all three. Your capacity estimate says how much you need today; scalability is about how easily you can reach the next number.

There are two ways to add capacity. You can make a machine bigger (vertical scaling) or add more machines (horizontal scaling). Almost every large architecture uses both, at different layers.

Vertical scaling: scale up

Vertical scaling means giving one server more CPU cores, more RAM, faster disks or a faster network card. You move from a 4-core machine to a 64-core machine.

Advantages

  • Simplest option: no code changes, no distributed-systems problems.
  • Everything stays in one process or one machine, so there is no network between components.
  • Transactions and joins on one database server are easy and fast.

Limits

  • There is a hardware ceiling; the biggest machine you can buy is only so big.
  • Price grows faster than capacity at the high end (a machine twice as large can cost far more than twice as much).
  • It is a single point of failure: if that one machine goes down, everything goes down.
  • Upgrades often need downtime to restart on the new hardware.

Horizontal scaling: scale out

Horizontal scaling means running many ordinary machines and spreading the work across them. Doubling capacity means doubling the number of machines.

Advantages

  • Almost no ceiling: add machines as long as the money lasts.
  • Commodity hardware is cheap and the cost grows roughly linearly.
  • Built-in redundancy: losing one of 20 servers costs 5% of capacity, not 100%.
  • You can add or remove machines without downtime.

Costs

  • You need something to spread the work: a load balancer.
  • Servers must be stateless, or share state through an external store.
  • Data must be split (sharding) or copied (replication), which brings consistency questions.
  • More machines mean more to monitor, deploy and secure.

Scale up or scale out

Vertical One bigger machine. Simple, but capped.
Horizontal Many small machines. Flexible, more complex.
Use both Scale up first, scale out when it hurts.

Databases often scale up longer than web servers.

1 / 2
flowchart TB
  subgraph V["Vertical: one bigger box"]
    A["1 server<br/>64 cores, 512 GB"]
  end
  subgraph H["Horizontal: many boxes"]
    LB["Load balancer"] --> S1["Server"]
    LB --> S2["Server"]
    LB --> S3["Server"]
    LB --> S4["Server"]
  end

Figure: scaling up replaces the machine; scaling out multiplies it behind a balancer.

Which one to choose?

A practical rule: scale up while it is cheap and simple; design to scale out before you run out of room.

  • Stateless web and API tiers scale out easily; start there.
  • Caches scale out by splitting keys across nodes (consistent hashing).
  • Relational databases scale up first (bigger machine, more RAM, faster disks), then add read replicas, and shard only when you must, because sharding complicates queries and transactions.
  • Batch and analytics jobs scale out naturally by splitting data into chunks.

Cost and capacity compared

The next simulation compares upgrading one machine against adding more machines. The vertical price curve is made up but follows the usual shape: the larger the box, the more each extra core costs.

Python Vertical vs horizontal: capacity per dollar

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The assumptions are invented, so do not quote the dollar amounts. The shape is the point: at small sizes scaling up is competitive and much simpler, while at large sizes the horizontal cost line stays straight and the vertical one bends upward. Real price lists show the same bend.

Auto scaling

Traffic is not constant. A shop is busy at lunchtime and quiet at 3 a.m. Auto scaling adds servers when load rises and removes them when it falls, so you pay for what you use. A typical rule watches a metric, such as average CPU, and adjusts the server count:

  • If CPU stays above 70% for 3 minutes, add servers.
  • If CPU stays below 30% for 10 minutes, remove servers.
  • Never go below a minimum or above a maximum.

New servers take time to start (boot, pull images, warm caches), so scale-out should begin before the machines are saturated, and scale-in should be slower than scale-out to avoid flapping.

Python A simple auto scaler

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Because the scaler can only react after it sees the load, the rapid ramp from 300 to 1,200 requests per second leaves the system at 86 to 100% utilization for three minutes, with no headroom (a slightly bigger spike would have overloaded it), and then it keeps too many servers while load falls because scale-in is deliberately slow. Fixes include scheduled scaling before known peaks, lower thresholds, faster startup, and keeping a buffer of spare capacity.

Scalability is more than adding machines

Adding servers helps only if the bottleneck is the part you added. Typical limits:

  • A shared database. Ten web servers all hitting one database just move the queue. See bottlenecks.
  • A global lock or counter that every request must touch.
  • Serial work. If 10% of the work cannot be parallelized, 100 machines give at most a 10x speedup (Amdahl's law).
  • Slow dependencies, such as a third-party API.

Design to remove these: cache, partition data, avoid global locks, make work independent, and use queues to smooth bursts.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which is a typical drawback of vertical scaling?

    A single bigger machine is simple, but it has a maximum size and can fail as a whole.
  2. 2. What must usually be true of web servers before you can scale them horizontally?

    Any server must be able to answer any request, so session and data state cannot live only in one server's memory.
  3. 3. Why should scale-in be slower than scale-out?

    Removing capacity too eagerly causes the next small spike to overload the system, so you add again, and the count oscillates.
  4. 4. A database is the bottleneck. What does adding more web servers do?

    More web servers send more concurrent queries to the same database, so the true limit is unchanged.

Summary

  • Vertical scaling makes one machine bigger: simple, capped, single point of failure.
  • Horizontal scaling adds machines: near-unlimited, redundant, but needs balancing, stateless design and data partitioning.
  • Use both: scale up while cheap, design to scale out; stateless tiers scale out first, databases last.
  • Auto scaling follows load, but reaction time means you need buffers or scheduled scaling.
  • Scalability is limited by the slowest shared part, not by how many servers you add.

Next: Latency, Throughput and Bottlenecks