Skip to content
Tutorials Center
My learning

Topics

Lesson 13 of 57 ยท Compute

Scaling and Autoscale

Learn vertical vs horizontal scaling in Azure, VM scale sets, autoscale rules, cooldowns, schedule vs metric scaling and how to avoid flapping, with a simulator.

Every application has a busy hour and a quiet hour. If you buy enough capacity for the busy hour, you pay for idle machines the rest of the day. If you buy for the quiet hour, users see errors at the peak. Scaling is how you change capacity, and autoscale is how Azure does it for you while the load moves.

This lesson covers the two directions of scaling, virtual machine scale sets (VMSS), how autoscale rules and cooldowns work, and why badly chosen thresholds make a system "flap". The real commands are read-only because they need a subscription. The runnable examples are simulators that apply the same logic to a made-up load curve, so you can feel how rules behave.

Vertical and horizontal scaling

There are two ways to add capacity:

  • Scale up (vertical) means moving to a bigger machine: more vCPUs and memory, for example from Standard_B2s to Standard_D4s_v5. It is simple, because the application does not change. But there is a ceiling (the biggest size), it usually needs a restart, and one big machine is still one point of failure.
  • Scale out (horizontal) means adding more copies of the same machine and spreading requests across them with a load balancer (see Load Balancer, Application Gateway and Front Door). It has almost no ceiling and survives the loss of one instance. The price is that your application must cope with running as several copies: no state stored only in one machine's memory or disk.

Scaling in and scaling down are the reverse moves. Most cloud designs prefer horizontal scaling, which is why stateless web tiers are so common. Vertical scaling is still the right answer for things that are hard to spread, such as a single database server.

flowchart LR
    A["1 instance<br/>2 vCPU"] -->|"scale up"| B["1 instance<br/>8 vCPU"]
    A -->|"scale out"| C["3 instances<br/>2 vCPU each"]

Scale up makes one machine bigger; scale out adds machines.

Virtual machine scale sets

A single VM from Virtual Machines does not scale by itself. A virtual machine scale set is a group of VMs created from one model (image, size, network settings, extensions) that Azure manages as one unit. You say "I want 4 instances" and it creates, deletes and replaces machines until that is true. A scale set can also spread instances across availability zones and fault domains, and it plugs into a load balancer.

Creating one with the CLI looks like this (shown read-only):

Terminal
az vmss create \
  --resource-group rg-demo \
  --name vmss-web \
  --image Ubuntu2204 \
  --vm-sku Standard_B2s \
  --instance-count 2 \
  --admin-username azureuser \
  --generate-ssh-keys

az vmss scale --resource-group rg-demo --name vmss-web --new-capacity 5
az vmss list-instances --resource-group rg-demo --name vmss-web --output table

az vmss scale is a manual change. Scale sets have two orchestration modes: Flexible (the current default, which gives you more control over individual VMs and works well with mixed sizes) and Uniform (identical instances managed purely by the model). Because instances can be deleted at any time, treat them as disposable: configure each machine at boot from the image or an extension, and keep data outside the instance.

App Service plans, Container Apps and Functions have their own scaling features that follow the same ideas. The autoscale service described next supports several of these resource types.

How autoscale works

Autoscale is a feature of Azure Monitor. You attach an autoscale setting to a resource such as a scale set or an App Service plan. The setting has:

  • Limits: a minimum, a maximum and a default instance count. The minimum protects availability and the maximum protects your budget.
  • Rules: each rule has a metric (for example average CPU), a condition over a time window (for example "greater than 70 percent over 10 minutes") and an action (add or remove instances, or set an exact count).
  • Cooldown: the time Azure waits after one scale action before it acts again. The default is 5 minutes. It exists because a new instance takes time to start and warm up, so the metric needs time to reflect the change.
Terminal
az monitor autoscale create \
  --resource-group rg-demo \
  --resource vmss-web \
  --resource-type Microsoft.Compute/virtualMachineScaleSets \
  --name autoscale-web \
  --min-count 2 --max-count 10 --count 2

az monitor autoscale rule create \
  --resource-group rg-demo --autoscale-name autoscale-web \
  --condition "Percentage CPU > 70 avg 10m" \
  --scale out 1 --cooldown 5

az monitor autoscale rule create \
  --resource-group rg-demo --autoscale-name autoscale-web \
  --condition "Percentage CPU < 30 avg 10m" \
  --scale in 1 --cooldown 5

Notice that you write two rules: one to scale out and one to scale in. If you only write the first, the system grows and never shrinks. Rules also evaluate averages over a window, not single spikes, so a few busy seconds do not add a machine.

Metric-based and schedule-based scaling

Metric-based scaling reacts to what is happening now: CPU, memory, queue length, requests per second. It handles surprises, but it is always a little late, because it needs the load to arrive, the average to cross the line, and the new instance to boot.

Schedule-based scaling uses profiles: "from 08:00 to 18:00 on weekdays, minimum 6 instances". It is perfect for predictable patterns (office hours, a nightly batch, a known sale), because capacity is ready before the users arrive. A setting can have a default profile plus scheduled profiles, and each profile can contain its own metric rules. A common mix is a higher minimum during business hours with metric rules still allowed to add more.

For a queue worker, scaling on queue length is often better than CPU, because it measures the actual backlog (see Azure Files, Queues and Tables).

An autoscale simulator

The program below is not Azure. It is a stand-in that models one rule set: each instance handles 100 requests per second, CPU is load divided by capacity, the rule averages CPU over 5 minutes, and a 3-minute cooldown applies. The load ramps up at minute 10, stays high, then falls at minute 35. Watch how the average lags the load and how scale-in is slower than scale-out because the average must fall below 30 first.

JS Autoscale simulator with a load curve

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Two things stand out. First, CPU sits at 100 percent for several minutes while the setting adds one instance at a time: the rule protects the cluster from overreacting, but it also reacts slowly. If you know a surge is coming, a schedule profile or a larger step (for example --scale out 2) helps. Second, the autoscaled run uses far fewer instance-minutes than a fixed 8-instance fleet, which is the whole point.

Flapping

Flapping means the system scales out, then immediately scales in, then out again, forever. It happens when the scale-in threshold is too close to the scale-out threshold. Imagine four instances at 82 percent CPU: the rule adds one (five instances, 66 percent). If your scale-in rule says "below 70", it removes one straight away, and the loop repeats. Every cycle costs boot time and can drop connections.

Azure helps with a built-in check: before it scales in, it estimates the metric after the change, and if that estimate would immediately trigger a scale-out, it skips the action. You should still leave a wide gap between the two thresholds (the usual advice is out above 70, in below 30 to 40) and use sensible cooldowns. The next stand-in runs the same constant load of 330 requests per second against three rule choices.

JS Why thresholds need a gap

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Option A flaps 15 times in 30 minutes. Option B, with a wide gap, settles after two actions. Option C mimics what Azure's projection check does and also settles. The honest conclusion: the guard is a safety net, but choosing good thresholds is your job.

Practical advice

  • Set a minimum of at least 2 for anything that must stay available, so one failure does not take you down. Spread across zones where you can.
  • Set a maximum you can afford, and add a budget alert (see Pricing, Cost Management and Budgets). An attack or a bug can otherwise scale you into a big bill.
  • Scale on the metric that actually limits you: CPU for compute-heavy work, memory for caches, queue length for workers, requests per instance for web apps.
  • Keep instances stateless and fast to start; slow boots make autoscale late.
  • Test with load before launch. An untested autoscale setting is a guess.

Autoscale versus a load balancer

These two are often confused because they work together, but they solve different problems.

Autoscale Load balancer
Question it answers "How many instances should exist?" "Which instance gets this request?"
Acts on The number of instances (scale out and in) Individual connections or requests
Driven by Metrics (CPU, queue length) or a schedule Health probes and a distribution rule
Advantages Matches cost to demand; handles spikes without manual work Spreads load evenly; removes unhealthy instances from rotation, so one failure is invisible to users
Limits Takes minutes to add capacity; badly tuned rules cause flapping or overspend Only spreads load over what exists; cannot add capacity by itself

A resilient web tier uses both: the load balancer (see Load Balancing) sends traffic to every healthy instance, and autoscale adds or removes instances behind it as demand changes.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What is the main difference between scaling up and scaling out?

    Vertical scaling (up) changes the size of one instance. Horizontal scaling (out) changes the number of instances behind a load balancer.
  2. 2. Why do you normally create both a scale-out rule and a scale-in rule?

    Autoscale only does what its rules say. A scale-out rule alone never removes instances, so the cost stays high after the peak.
  3. 3. What is the purpose of a cooldown period?

    After a scale action the metric needs time to reflect the change. The cooldown avoids reacting again to numbers that are out of date.
  4. 4. Which situation is best handled by schedule-based scaling?

    Schedules prepare capacity before predictable load arrives. Metric rules are for load you cannot predict, and they are always a little late.
  5. 5. What usually causes flapping?

    When removing one instance pushes the metric right back over the scale-out threshold, the system loops. A wide gap between the thresholds prevents it.

Summary

  • Scale up changes the size of a machine; scale out changes the number of machines. Prefer out for stateless tiers.
  • A virtual machine scale set manages a group of identical VMs from one model, and az vmss scale changes the count by hand.
  • Azure Monitor autoscale uses minimum, maximum and default counts, rules on metric averages over a window, and a cooldown (5 minutes by default).
  • Write a scale-out rule and a scale-in rule, keep a wide gap between their thresholds, and set a maximum you can afford.
  • Use schedule profiles for predictable load and metric rules for the unpredictable part.

Next: Choosing a Compute Service.