Skip to content
Tutorials Center
My learning

Topics

Lesson 65 of 78 ยท Operations and Observability

Cost, Reliability and Multi-Cluster

Cut Kubernetes cost with right-sizing and spot nodes, compare multi-tenancy models and vcluster, plan multi-cluster setups and use a high availability checklist.

Kubernetes makes it easy to start things and easy to forget about them. A cluster that is well-run but never examined often wastes half of the money it costs: pods that ask for far more CPU than they use, nodes that are mostly empty, test environments that run all weekend. At the same time, saving money must not make the system fragile. This lesson ties together cost, reliability and the question of how many clusters you should have.

The manifests and commands are read-only, since this site has no cluster. The Python stand-ins let you see the arithmetic: how requests turn into node counts and a bill, and how to pick a better request from measured usage. The prices are made-up round numbers for teaching. Real prices differ by cloud, region and year.

Where the money goes

You pay for nodes, not for pods. A node costs the same whether it runs one pod or fifty. The scheduler places pods by their requests (requests and limits), so what you request decides how many nodes the cluster needs, regardless of what the pods actually use. Most waste lives in this gap.

Where waste hides

Oversized requests Pods reserve more than they use.
Empty space Nodes that are barely filled.
Forgotten workloads Old namespaces and idle test stacks.
Orphaned disks Volumes of deleted apps.

Requests drive the bill, not usage.

1 / 2

From requests to a bill

The stand-in below packs a set of Deployments onto identical nodes using first-fit decreasing (a simple bin-packing heuristic), then calculates the monthly bill. The cluster autoscaler does something similar, just continuously (vertical and cluster autoscaling).

Python Estimate nodes and cost from requests

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Fifteen pods need three nodes, at $438 a month. CPU is the scarce resource here: the nodes are 88% full by CPU but have plenty of memory left, so the scarcer resource sets the node count. Real clusters have DaemonSet pods, system reserves and failure headroom too, so keep some slack on purpose.

Right-sizing

A request is a promise to the scheduler. If a container requests 1,000m CPU but its 95th percentile of use is 220m, you pay for 780m that nobody uses. Right-sizing means setting requests near measured use plus a safety margin. The Vertical Pod Autoscaler in recommendation mode, Prometheus queries (metrics and monitoring) and tools like Goldilocks or OpenCost help you find the numbers. A common recipe:

  • CPU request: around the p90 to p95 of usage, because CPU can be throttled in a pinch.
  • Memory request and limit: close to the observed maximum plus 15 to 25% headroom, because exceeding memory means the container is killed.
Python A right-sizing recommendation

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The web container asked for 500m but peaks around 180m, so it gets a much smaller request. The worker container is already using almost all of its 1,000m request, so the recommendation goes up and costs a little more: right-sizing is not "make everything smaller", it makes requests honest. It also shows the multiplication effect: each saving applies per replica, so a six-replica Deployment multiplies it by six.

Do not cut requests blindly. Measure over a full business cycle that includes peaks (a week or a month), keep headroom for memory, and change one workload at a time. Setting CPU limits too low causes throttling and hurts latency even when the node has idle CPU.

Spot nodes and other levers

Spot (or preemptible) instances are spare cloud capacity sold at a deep discount, but the provider can reclaim them with a short warning, often two minutes. They are a good fit for stateless, replicated or batch workloads and a poor fit for a single-replica database. A safe pattern is a small on-demand node pool for critical pods and a large spot pool for the rest, separated by labels and taints:

YAML
spec:
  nodeSelector:
    node-type: spot
  tolerations:
    - key: spot
      operator: Equal
      value: "true"
      effect: NoSchedule

Combine spot with PodDisruptionBudgets, several replicas spread over zones, and graceful shutdown handling. Other levers: committed-use discounts for the steady base load, ARM instances for supported workloads, scheduled scale-down of non-production namespaces, short retention on logs and metrics, and cleaning up unused load balancers and volumes.

Multi-tenancy models

When several teams or customers share infrastructure, you are doing multi-tenancy. The question is how hard the walls are.

Three tenancy models

Namespaces One cluster, soft walls. RBAC, quotas, policies.
Virtual clusters Own API server per tenant, shared nodes.
Cluster per tenant Hard isolation, highest cost.

Pick by how much you trust the tenants.

1 / 2

Namespace-based tenancy (namespaces and quotas) is enough for trusted teams in one company: give each team a namespace, RBAC, a ResourceQuota, a LimitRange and default-deny network policies. The weak spots are shared cluster-scoped resources (CRDs, admission webhooks), a shared kernel and noisy neighbors.

Virtual clusters, for example vcluster, run a lightweight Kubernetes control plane inside a namespace of a host cluster. Each tenant gets its own API server and can install CRDs and cluster-wide objects freely, while pods still run on the shared host nodes. This is popular for developer environments and CI, where every pull request gets its own "cluster" in seconds.

Cluster per tenant gives the hardest isolation and is common for strict compliance, untrusted customers or very different needs, but costs more and multiplies the operational work, including every upgrade in the previous lesson.

Multi-cluster patterns

Even with one team, you often end up with several clusters:

  • Per environment: separate dev, staging and production clusters, so that a mistake in dev cannot hurt production.
  • Per region: clusters near users for low latency, and for regional failure tolerance.
  • Active-active or active-passive: the same app in two regions, with a global load balancer or DNS in front; state needs replication that Kubernetes does not give you.
  • Fleet management: tools such as Argo CD ApplicationSets or Flux (GitOps), Cluster API and cloud fleet services keep many clusters consistent from one repository.

Do not start with multi-cluster out of fashion: each cluster is a fixed cost and a thing to patch. Two or three well-managed clusters beat ten neglected ones.

A high availability checklist

Reliability is a set of habits, and most of them are visible in the manifests. Review each production workload against a list like this:

  • At least two or three replicas, spread across nodes and zones with topology spread constraints (affinity).
  • Readiness and liveness probes that match the real health of the app (probes).
  • Requests set and sensible limits; the right QoS class.
  • A PodDisruptionBudget so drains and upgrades never take every replica.
  • Graceful shutdown: handle SIGTERM and set a suitable terminationGracePeriodSeconds.
  • A multi-zone control plane (managed services do this by default) and nodes in several zones.
  • Rolling update settings with maxUnavailable configured and automatic rollback on failure (rolling updates).
  • Backups of etcd and persistent data, with a tested restore.
  • Alerts on the golden signals and a runbook for each page.

The last stand-in scores a Deployment against the first part of this checklist.

Python Score a workload for availability

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Two items fail: no PodDisruptionBudget and no SIGTERM handling. Both are cheap to fix and both show up exactly during node drains and upgrades.

What is real and what is a stand-in

The packing code is a first-fit heuristic, not the real scheduler or autoscaler, and the prices are invented. The right-sizing recipe (percentile plus margin) is a common rule of thumb, not an official formula. The manifests and the checklist describe real practice, but none of them can run on this site.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why do oversized resource requests cost money even if pods use little?

    The scheduler reserves requested capacity on nodes, so large requests force more nodes. You pay for nodes whether or not the pods use what they asked for.
  2. 2. Which workload is the best fit for spot nodes?

    Spot nodes can disappear on short notice. Replicated, stateless workloads tolerate that; single stateful instances do not.
  3. 3. What does a virtual cluster (such as vcluster) give each tenant?

    A virtual cluster runs a lightweight control plane in a namespace, so tenants can use CRDs and cluster-scoped objects, while pods still run on shared nodes.
  4. 4. Which item on the availability checklist protects replicas from being evicted all at once during a node drain?

    A PodDisruptionBudget limits voluntary evictions so that enough replicas stay up during drains and upgrades.

Summary

  • You pay for nodes, and requests decide how many you need, so right-sizing requests is the biggest saving.
  • Measure usage over a full cycle; use p95 plus margin for CPU, and peak plus headroom for memory.
  • Spot nodes cut cost for replicated stateless work; keep critical pods on on-demand nodes and use PDBs and zone spread.
  • Tenancy ranges from namespaces (cheap, soft walls) through virtual clusters to a cluster per tenant (hard walls, more work).
  • Add clusters for real reasons such as environments, regions and compliance, and manage them with GitOps and fleet tools.
  • Use a high availability checklist: replicas, probes, requests, PDBs, zone spread, graceful shutdown, tested backups, alerts.

Next: Project: Deploy a Web App with a Database, where you put the pieces together.