Skip to content
Tutorials Center
My learning

Topics

Lesson 43 of 78 ยท Scheduling and Scaling

Horizontal Pod Autoscaler

How the Horizontal Pod Autoscaler works: metrics-server, the scaling formula, tolerance, stabilization windows, custom metrics and KEDA, with a runnable simulation.

Traffic is rarely constant. A shop is busy at lunchtime, a report service is hammered on Monday mornings, a batch consumer falls behind when a queue fills up. You could provision for the peak and pay for idle pods the rest of the day, or you could let Kubernetes add and remove replicas as demand changes. The Horizontal Pod Autoscaler (HPA) does the second: it watches a metric, compares it to a target, and changes the replicas field of a Deployment (or StatefulSet) up or down.

"Horizontal" means more or fewer copies of the pod. The alternative, giving each pod more CPU and memory, is vertical scaling, covered in the next lesson.

How the HPA loop works

Measure Read pod metrics every 15 seconds.
Compute Apply the scaling formula.
Stabilize Smooth out noisy recommendations.
Scale Patch replicas on the target.
1 / 2

The prerequisites

The HPA is a built-in controller, but it needs data. Resource metrics (CPU and memory) come from metrics-server, a small add-on that collects usage from each kubelet and serves it through the metrics.k8s.io API. Without it the HPA reports <unknown> for the current value. You can check with:

CONSOLE
$ kubectl top pods
NAME                   CPU(cores)   MEMORY(bytes)
web-6c9f7d-4pqkz       143m         92Mi
web-6c9f7d-9tz2w       151m         90Mi

The second prerequisite is easy to forget: CPU utilization is a percentage of the pod's CPU request. If a container has no requests.cpu, the HPA cannot compute a percentage and stays idle. Set requests first (see Resource Requests and Limits).

A first HPA

The autoscaling/v2 API is the current one. This HPA keeps average CPU utilization around 50% and stays between 2 and 10 replicas:

YAML
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web
  minReplicas: 2
  maxReplicas: 10
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 50

The quick imperative form is kubectl autoscale deployment web --cpu=50% --min=2 --max=10. Inspect it with kubectl get hpa and kubectl describe hpa web; the Events and Conditions explain why it did or did not scale.

Once an HPA owns the replica count, remove replicas from the Deployment manifest you keep in Git. Otherwise every kubectl apply resets the count to the manifest's value and fights the autoscaler.

The scaling formula

Every 15 seconds (the controller's default sync period) the HPA computes:

TEXT
desiredReplicas = ceil( currentReplicas * currentMetricValue / targetMetricValue )

With 4 pods averaging 90% CPU against a target of 50%, the ratio is 1.8, so the desired count is ceil(4 * 1.8) = 8. With the same pods at 20%, the ratio is 0.4 and the desired count is ceil(1.6) = 2. The result is clamped to minReplicas and maxReplicas.

To avoid constant tiny adjustments, there is a tolerance: if the ratio is within 10% of 1.0 (the default of --horizontal-pod-autoscaler-tolerance, a cluster-wide setting), the HPA does nothing. Pods that are not yet ready, or that have no metrics yet, are handled carefully so a rollout does not trigger a runaway scale-up: they are left out or counted conservatively.

Try it: the formula and the tolerance band

Python Desired replicas for different loads

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Look at the second and third rows of the table: 54% against a target of 50% (ratio 1.08) is inside the band, so nothing happens, but 56% (ratio 1.12) adds one pod. The next-to-last row shows the clamp: a ratio of 4 on 8 pods would ask for 32, and maxReplicas limits it to 10. The last row shows that an idle service shrinks to minReplicas, here 1 because the helper's lo is 1 (use 2 or more in production for availability).

Stabilization and scaling behavior

Raw metrics are noisy. If the HPA followed every recommendation, a short spike followed by a dip would add pods and then immediately remove them, a pattern called flapping. Pods take time to start, so flapping wastes work and can hurt users.

The behavior section of autoscaling/v2 controls this separately for scaling up and down:

YAML
spec:
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
        - type: Percent
          value: 50
          periodSeconds: 60
    scaleUp:
      stabilizationWindowSeconds: 0
      policies:
        - type: Pods
          value: 4
          periodSeconds: 15
        - type: Percent
          value: 100
          periodSeconds: 15
      selectPolicy: Max
  • The stabilization window looks back over the recommendations of the last N seconds. When scaling down, the HPA uses the highest recommendation in the window, so it only shrinks after the load has stayed low for the whole window. The default is 300 seconds for scale-down and 0 for scale-up, which makes the HPA quick to grow and slow to shrink.
  • Policies limit how fast it moves: "at most 4 pods or 100% per 15 seconds" for scale-up, "at most 50% per minute" for scale-down. selectPolicy chooses which policy applies when several do (Max is the default, Min and Disabled also exist). The values shown for scaleUp are the defaults.

Try it: with and without a stabilization window

This simulation uses a 15-second step like the real controller. To keep the output short, the window is 4 steps (60 seconds) instead of the default 20 (300 seconds). Demand is the total CPU use of the service in millicores; each pod requests 200m, and the target is 50% utilization.

Python Flapping versus a stabilization window

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

In the first table the service drops from 10 pods to 4 at 90 s, then 3, then jumps back to 8 at 135 s when a short burst arrives, then falls to 3 again: six changes of the replica count in under three minutes, with a burst right after the first drop. In the second table, the scale-down at 90 s is held back because the 60-second window still contains a recommendation of 10, so the burst at 135 s is absorbed by pods that are still there, and the count only falls (to 8, then to 3) once the load has been low for the whole window. The price of stabilization is that you pay for a few spare pods for a while, which is almost always the right trade.

Beyond CPU: memory, custom and external metrics

type: Resource supports CPU and memory (and, with ContainerResource, a single container's usage, useful when a sidecar skews pod averages). CPU is the usual choice; memory is a poor autoscaling signal for most apps because many runtimes do not release memory when load falls.

Other metric types need an extra adapter that implements a metrics API:

  • Pods: a per-pod custom metric, for example requests per second per pod. Served through custom.metrics.k8s.io, commonly by the Prometheus Adapter.
  • Object: a metric describing another object, such as requests per second on an Ingress.
  • External: a metric from outside the cluster, such as the length of a cloud queue, through external.metrics.k8s.io.
YAML
  metrics:
    - type: Pods
      pods:
        metric:
          name: http_requests_per_second
        target:
          type: AverageValue
          averageValue: "100"

If you list several metrics, the HPA computes a desired count for each and picks the largest. That is why adding a second metric can only make the service scale out more, never less.

KEDA: event-driven autoscaling

KEDA (Kubernetes Event-Driven Autoscaling) is a CNCF project that sits on top of the HPA. You describe a ScaledObject with one or more scalers (a Kafka topic lag, an SQS queue depth, a cron schedule, a Prometheus query ...), and KEDA creates and manages an HPA for you, feeding it the external metric. Its main extra is scaling to zero: KEDA can take a Deployment to zero pods when there is no work and bring it back when an event arrives, which the plain HPA does not do by default. It is a good fit for queue consumers and scheduled jobs.

Limits of the HPA

  • It is reactive. The HPA reacts to load that already exists. Metrics are scraped about every 15 seconds and the controller checks about every 15 seconds, then new pods need time to schedule, pull an image and become ready. A sudden spike hurts before extra pods arrive. Pre-scaling on a schedule (KEDA's cron scaler) or keeping a higher minReplicas before known peaks helps.
  • It has no memory of cost. It scales on a metric, not on price, and it does not know about spot-instance interruptions or price changes in your cloud.
  • Scaling steps are coarse. The granularity is a whole pod. A pod that is large and sized for peak load can make steps bigger than needed, so right-size requests first (Resource Requests and Limits).
  • Only pods that can be copied scale this way. Stateful workloads, or programs that keep session state in memory, need design work before more replicas help.
  • Fixed intervals and stabilization (see above) slow scale-down on purpose, to avoid flapping, so costs fall later than load does.
  • Pods alone are not enough. New pods that find no room stay Pending. Add the Cluster Autoscaler or Karpenter (Vertical Pod Autoscaler and Cluster Autoscaling).

Common mistakes

  • No requests.cpu on the container, so the HPA shows <unknown> and never scales.
  • minReplicas: 1 for a service that needs high availability.
  • Setting replicas in the Deployment manifest while an HPA is active.
  • Using memory as the only metric for a garbage-collected runtime.
  • A target utilization that is too high (90%) leaves no room for the time pods take to start. 50-70% is a common range.
  • Combining an HPA and a VPA that both act on CPU or memory (see the next lesson).
  • Scaling out pods when the cluster has no room: the new pods go Pending until a node appears. That needs the Cluster Autoscaler.

What is real and what is the stand-in: the manifests and kubectl output above are real but read-only on this site. The Python programs reproduce the formula, tolerance, clamp and the scale-down stabilization idea; the real controller adds readiness handling, per-metric calculation and the scaling policies.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Four pods average 80% CPU utilization and the target is 40%. What is the desired replica count (ignoring limits)?

    desired = ceil(4 * 80 / 40) = 8. The ratio of 2.0 doubles the replica count.
  2. 2. A Deployment's containers have no CPU request. What does a CPU-utilization HPA do?

    Utilization is a percentage of the request. With no request there is no denominator, so the HPA reports unknown metrics.
  3. 3. What does the default 300-second scale-down stabilization window do?

    The HPA only shrinks after the load has stayed low for the whole window, which prevents flapping.
  4. 4. What does KEDA add compared with the plain HPA?

    KEDA creates and drives an HPA from event sources such as queues and can scale workloads down to zero pods.

Summary

  • The HPA changes a workload's replicas to keep a metric near its target, checking about every 15 seconds.
  • desired = ceil(current * metric / target), clamped to min/max, with a 10% tolerance band around the target.
  • CPU utilization needs metrics-server and container CPU requests.
  • behavior controls stabilization windows and scaling policies; fast up, slow down is the default.
  • With several metrics the HPA takes the largest result; custom and external metrics need adapters.
  • KEDA builds on the HPA for event-driven scaling and scale-to-zero.

Next: grow pods vertically and add nodes in Vertical Autoscaling and Cluster Autoscaler.