Skip to content
Tutorials Center
My learning

Topics

Lesson 73 of 78 ยท Projects

Project: Build a Mini Cluster Simulator

Build a small Python model of a cluster with a scheduler, a deployment controller, a service load balancer and an autoscaler, and watch them react to load and node failure.

The best way to understand Kubernetes is to realize that it is not magic: it is a handful of small programs, each watching some state and nudging it toward a goal. In this last project you build a miniature cluster in Python in which four of them work together: a scheduler that places pods on nodes, a deployment controller that keeps the right number of pods alive, a service that spreads traffic over ready pods and an autoscaler that changes the replica count when load changes.

Everything in this project runs in your browser. It is a teaching model, not Kubernetes: there is no API server, no networking, no rolling updates and no real containers, and the numbers are chosen to make the behavior easy to see. Each component is a simplified version of the real one, covered in How the Scheduler Works, Deployments, Services and Horizontal Pod Autoscaler.

Four small programs

Scheduler Pending pod gets a node.
Controller Keeps N pods alive.
Service Splits traffic over ready pods.
Autoscaler Changes N when load changes.

None of them talks to another one directly.

1 / 2

That second slide is the key design idea. The components never call each other. The autoscaler changes a number, the controller notices that the number and the pods disagree and creates pods, the scheduler notices pods without a node and places them. This is how the real control plane works, and it is why each part can fail and restart on its own (Cluster Architecture).

Part 1: scheduler and controller

Start with the world. We have three nodes with 1000 millicores of CPU each, and every pod requests 300m, so a node holds three pods. A pod is just a name mapped to a node, where None means "Pending".

The controller compares the wanted count with the real count; the scheduler finds nodes that fit (the filter step) and picks the one with the most free CPU (a simple score), breaking ties by name. Run it and follow the story: five pods are created and spread out, a node fails, its pods are replaced elsewhere, a scale-up no longer fits (some pods stay Pending) and then the node comes back.

Python Part 1: a scheduler and a deployment controller

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Look at what the second step did: nobody wrote "restart the pods from n3 elsewhere". The pods were simply gone, the count was too low, and the same two functions as before repaired the situation. Real Kubernetes has a few more details (the node controller waits several minutes before it evicts pods from an unreachable node, and pods owned by a Deployment get new names), but the shape is exactly this.

Part 2: the complete cluster

Now add the other two components and a clock. Each tick is one round of the world:

  1. Events happen (a node fails or recovers; the load changes).
  2. The autoscaler reads the load and changes the wanted replica count.
  3. The controller fixes the pod count, the scheduler places new pods.
  4. Pods that were placed become ready after two ticks (a stand-in for container start and readiness probes).
  5. The service spreads the current load over the ready pods only.

The autoscaler uses the real formula, ceil(readyPods * utilization / target), with a 10 percent tolerance and a three-tick window that slows down scale-down. One pod handles 100 requests per second at 100 percent CPU, and the target is 50 percent. The whole program is below; read it once before you run it.

Python Part 2: scheduler, controller, service and autoscaler together

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Trace the story slowly. At the start there are no ready pods, so the service has nothing to send traffic to. When the load jumps, the first scale-up is limited by the number of ready pods (the formula uses what is actually serving), and new pods need two ticks to become ready, so the service is overloaded for a moment. Then the node fails: pods vanish, the controller replaces them, and because only two nodes are left some stay Pending while the autoscaler asks for more. When the node returns, the scheduler places the waiting pods. Notice that the autoscaler does not shrink the deployment while the load is still falling: only at tick 9, when the last three recommendations are all low, does it drop to the minimum in one step.

Do not skip the exercise of reading why a line printed. For every change, ask which component acted, what state it read and what state it wrote. That chain is how you debug a real cluster.

Experiments to try

The program is small enough to bend. Change one thing at a time and predict the result before you run it.

  • Set START_DELAY to 0 and to 5. How do the overload moments change? This is the effect of slow startup and why readiness and startup probes matter (Liveness, Readiness and Startup Probes).
  • Change POD_CPU to 500. Pods per node drop to two, so the cluster can no longer hold MAX_R pods and some stay Pending even with all nodes up. In a real cluster the Cluster Autoscaler would add nodes (Vertical Autoscaling and Cluster Autoscaler).
  • Set WINDOW to 1 and watch the replica count react faster, and flap more, on the way down.
  • Make the scheduler choose the node with the least free CPU (max of -free). This is packing instead of spreading; think about which is better for resilience and which for cost.
  • Add a pod priority and let the scheduler evict a lower-priority pod when nothing fits (preemption, see PodDisruptionBudgets, Priority and Eviction).
  • Give nodes zones and make the scheduler prefer a zone with fewer pods of the same app (Node Selectors, Affinity and Anti-Affinity).

What this model leaves out

Honesty about the simplifications helps you read the real documentation. The real scheduler runs many filter and score plugins, scores in parallel and binds through the API server. Real controllers use informers and work queues rather than loops over everything (Controllers and the Reconcile Loop). A Deployment owns ReplicaSets and performs rolling updates (Rolling Updates and Rollbacks), and Services are implemented by kube-proxy rules, not by a function that splits a number (How Services Really Work: kube-proxy). The real autoscaler reads metrics from the metrics server and has separate behavior for scale-up and scale-down. What the model gets right is the architecture: independent, level-triggered loops that share state through one store and never call each other.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. In the simulator, how does the autoscaler cause new pods to be created?

    Components only share state. The autoscaler writes the replica count, the controller sees the gap and creates pods, and the scheduler later places them.
  2. 2. Why does a pod with no node stay Pending instead of failing?

    The scheduler skips a pod when no node fits and looks again on the next pass. The pod waits until a node recovers or capacity appears.
  3. 3. Why does the service send traffic only to ready pods?

    Readiness means the pod can handle requests. Excluding pods that are still starting avoids errors during scale-ups and rollouts.
  4. 4. What is the purpose of the three-tick window in the autoscaler?

    When scaling down, the autoscaler uses the highest recent recommendation, so a short dip in load does not remove pods that will be needed again soon.

Summary

  • A cluster is a set of small, level-triggered loops that share state and never call each other.
  • The controller closes the gap between wanted and existing pods, and the scheduler places pods that have no node.
  • The service routes only to ready pods; the autoscaler turns load into a replica count with tolerance and a stabilization window.
  • Failures need no special code: lost pods are a gap between wanted and actual, and the same loops fix it.
  • The simulator is a model for learning; real components are more capable, but follow the same architecture.

Finish the course with practice: Exercises: Pods, Deployments and Services.