Skip to content
Tutorials Center
My learning

Topics

Lesson 44 of 78 ยท Scheduling and Scaling

Vertical Autoscaling and Cluster Autoscaler

Right-size pods with the Vertical Pod Autoscaler, add and remove nodes with Cluster Autoscaler or Karpenter, and see bin packing decide the node count.

The Horizontal Pod Autoscaler answers "how many copies of this pod?". Two other questions remain. First: "how big should each pod be?" Developers guess CPU and memory requests when they write a manifest, and the guess is usually wrong, often by a factor of five. Second: "how many nodes does the cluster need?" More pods are useless if no machine has room for them, and idle machines cost money.

Kubernetes has a separate tool for each: the Vertical Pod Autoscaler (VPA) adjusts pod requests, and a node autoscaler (Cluster Autoscaler or Karpenter) adds and removes nodes. The three work at different layers and are designed to be combined.

Three layers of scaling

HPA How many pods.
VPA How big each pod is.
Node autoscaler How many machines.
1 / 2

Why requests matter

Every scaling decision here is based on requests, not on actual usage. The scheduler places pods by summing requests; the HPA's utilization is relative to requests; node autoscalers add machines when requests do not fit. Oversized requests waste money (the node looks full while it is mostly idle), undersized requests cause throttling and out-of-memory kills. See Resource Requests and Limits for the basics. Right-sizing requests is the first and cheapest optimization of a cluster.

The Vertical Pod Autoscaler

VPA is not part of core Kubernetes: it is a project in the autoscaler repository that you install into the cluster, together with metrics-server. It consists of three parts:

  • the recommender watches usage history (and out-of-memory events) and computes recommended requests,
  • the updater evicts pods whose requests differ too much from the recommendation (if the mode allows it),
  • the admission controller is a webhook that rewrites the requests of newly created pods.

You declare it as a custom resource:

YAML
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: api
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: api
  updatePolicy:
    updateMode: "Off"
  resourcePolicy:
    containerPolicies:
      - containerName: "*"
        minAllowed:
          cpu: 50m
          memory: 64Mi
        maxAllowed:
          cpu: "2"
          memory: 2Gi
CONSOLE
$ kubectl describe vpa api
  Recommendation:
    Container Recommendations:
      Container Name:  api
      Lower Bound:     Cpu: 118m   Memory: 210Mi
      Target:          Cpu: 138m   Memory: 262Mi
      Upper Bound:     Cpu: 259m   Memory: 620Mi

The recommendation has three numbers. Target is what the VPA would set. Lower bound and upper bound form a band: if the current request is outside the band, the VPA considers it wrong enough to act. (The numbers above are an illustration, not the output of a real cluster.)

Update modes

Mode What happens
Off Recommendations only. Nothing is changed. The safest way to start.
Initial The target is applied when pods are created, but running pods are left alone.
Recreate The updater evicts pods that are outside the band; replacements get the new requests.
Auto Currently behaves like Recreate; the name is kept so VPA can later choose better mechanisms.

Newer VPA releases add a mode that tries to resize a running pod in place and falls back to eviction. This builds on Kubernetes' own in-place pod resize feature: since Kubernetes 1.33 (beta) you can change a running container's CPU and memory through the pod's resize subresource, with resizePolicy per resource saying whether the container needs a restart (NotRequired or RestartContainer). Check what your cluster version and VPA version support before relying on it.

Rules of thumb for VPA

  • Start with Off, read the recommendations for a week or two, and copy sensible values into your manifests. Many teams never go beyond this and that is fine.
  • VPA needs usage history. A pod that has run for ten minutes gets a poor recommendation.
  • Do not let an HPA and a VPA both act on the same resource (CPU or memory) of the same workload. They would chase each other: the HPA adds pods when utilization is high, the VPA enlarges requests, which lowers utilization, and so on. Scale on CPU with the HPA and use the VPA for memory only, or scale the HPA on a custom metric.
  • In Recreate or Auto mode, pods restart. Use a PodDisruptionBudget (next lesson) and run at least two replicas.
  • A single-replica workload that restarts to resize means downtime.

Try it: how a recommendation is derived

The VPA uses decaying histograms of usage, a percentile and a safety margin. This stand-in keeps only the idea: take a high percentile of observed usage and add a margin, so that a rare spike does not set the request but normal peaks are covered.

Python A percentile-based recommendation

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The 400 m spike in the data (the maximum) does not drive the result: the target comes from the 90th percentile (120 m) plus 15%. The original request of 1000 m is about 7 times larger than needed, so the VPA would flag it as far above the upper bound and, in Recreate mode, replace the pod. The real recommender weights recent samples more heavily and treats memory differently (an out-of-memory kill pushes the recommendation up), so treat this as an illustration of the idea only.

Adding nodes: the Cluster Autoscaler

Pods and requests are only half of capacity planning. When the HPA scales out and the scheduler finds no node with enough free requests, the new pods stay Pending (see How the Scheduler Works). The Cluster Autoscaler (CA) reacts to exactly that situation:

  1. It finds Pending pods that failed to schedule for lack of resources.
  2. It simulates, for each node group (an AWS Auto Scaling group, a GKE node pool, an Azure scale set), whether adding a node of that type would let the pod run.
  3. It asks the cloud API to grow the group. The new machine boots, joins, and the scheduler places the pods.

For scale-down, the CA finds nodes that are under-used (by default, below 50% of requested resources) for a while (10 minutes by default), checks that all their pods can be moved elsewhere, and then drains and removes the node. It will not remove a node if a pod there cannot be rescheduled elsewhere: pods with local storage, pods not managed by a controller, pods with a restrictive PodDisruptionBudget or the annotation cluster-autoscaler.kubernetes.io/safe-to-evict: "false".

The CA works with predefined node groups of a fixed machine type per group. It is mature and runs on most clouds, but it reacts in minutes, and a node group with large machines can waste a lot when a small pod needs one more node.

Karpenter and node provisioning by need

Karpenter (originally from AWS, now also available for other clouds through providers) takes a different approach. It has no fixed node groups. You declare constraints in a NodePool (allowed instance families, zones, capacity types such as spot or on-demand, limits), and Karpenter looks at the pending pods, chooses the best-fitting instance type itself, and launches that machine directly. It also consolidates continuously: it replaces several half-empty nodes with fewer or cheaper ones and removes empty nodes quickly.

YAML
apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: general
spec:
  template:
    spec:
      requirements:
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["spot", "on-demand"]
        - key: kubernetes.io/arch
          operator: In
          values: ["amd64"]
  limits:
    cpu: "200"
  disruption:
    consolidationPolicy: WhenEmptyOrUnderutilized

Both tools depend on honest requests: they add nodes because requests do not fit, not because usage is high. Both also need pods to be movable. That is where PodDisruptionBudgets, topology spread constraints and taints enter the picture, and why a good autoscaling setup always includes them.

Bin packing: how many nodes?

Choosing the number of nodes is a bin-packing problem: pods are items with sizes (CPU and memory requests), nodes are bins with a capacity (their allocatable, which is smaller than the machine's capacity because the kubelet and system daemons reserve some). Finding the optimal packing is NP-hard, so schedulers and autoscalers use heuristics. A classic one is first-fit decreasing: sort the items from largest to smallest and put each into the first bin where it fits.

Python How many nodes do nine pods need?

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The nine pods need 8,250 m of CPU, and each node offers 3,600 m after the reserve, so the lower bound is three nodes, and the heuristic achieves it. Two nodes end up with only 100 m of CPU free, while the third has 2,350 m left over: that is the room for the next pods. Change the pod list, add a few more pods, and compare the node count with the lower bound printed on the last line. Whenever the heuristic needs more nodes than the bound, the gap is fragmentation: free space that is split into pieces too small for the next pod. It is also why many small pods pack better than a few huge ones.

Putting it together

A typical production setup: requests are tuned using VPA recommendations (mode Off), an HPA scales the Deployment on CPU or a request-rate metric, a node autoscaler adds machines when pods are pending, and PodDisruptionBudgets protect availability while nodes are removed. Add a small amount of headroom, such as low-priority "placeholder" pods that get evicted when real pods need space (see PodDisruptionBudgets, Priority and Eviction), so that a scale-up does not wait for a new machine to boot.

What is real and what is the stand-in: the manifests, the VPA output (illustrative numbers) and the commands are read-only here. The Python programs show the arithmetic ideas: a percentile with a margin, and first-fit-decreasing packing. Real autoscalers use richer inputs (histograms, scheduling simulation, cloud pricing).

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which VPA update mode only publishes recommendations and never changes any pod?

    Off is recommendation-only. Initial applies values to new pods, and Recreate/Auto evict running pods to apply them.
  2. 2. Why should an HPA and a VPA not both act on CPU for the same workload?

    HPA utilization is measured against requests, which the VPA changes. The two feedback loops interfere. Split the resources between them or use a custom metric for the HPA.
  3. 3. What triggers the Cluster Autoscaler to add a node?

    The CA looks for unschedulable pods, simulates whether a new node of a configured group would fit them, and grows that group.
  4. 4. How does Karpenter differ from the Cluster Autoscaler?

    Karpenter evaluates pending pods against NodePool constraints and provisions suitable machines directly, and it consolidates nodes continuously.

Summary

  • HPA scales the number of pods, VPA the size of each pod, and a node autoscaler the number of machines.
  • VPA is an add-on with a recommender, updater and admission controller; start in Off mode and avoid combining it with an HPA on the same resource.
  • Kubernetes can resize running containers in place (beta since 1.33), which newer VPA releases can use.
  • The Cluster Autoscaler grows node groups for Pending pods and removes under-used nodes whose pods can move; Karpenter provisions fitting nodes directly and consolidates.
  • Node counts are a bin-packing problem driven by requests, not by actual usage.
  • PodDisruptionBudgets and honest requests make scaling down safe.

Next: PodDisruptionBudgets, Priority and Eviction.