Skip to content
Tutorials Center
My learning

Topics

Lesson 68 of 78 ยท Projects

Project: Microservices with Probes and Autoscaling

Deploy three cooperating services on Kubernetes with probes, service discovery, an autoscaler, a disruption budget and network policies, plus runnable simulations.

In the first project you deployed one web app with a database. Real systems usually consist of several small services that call each other, and each one needs the same care: it must report when it is ready, scale when load rises, survive node maintenance and talk only to the services it should. In this project you assemble all of those pieces for a tiny shop made of three services.

The manifests below are read-only, because no cluster runs on this site. Everything that would normally happen inside the cluster (readiness gating, autoscaling, network policy decisions) is shown with Python stand-ins that you can run. The stand-ins model the rules faithfully but are not the real Kubernetes code. Where a lesson already covered a topic, you get a link instead of a repeat.

The architecture

The shop has three services in one namespace called shop:

  • gateway receives traffic from outside (through an Ingress, see Ingress and Ingress Controllers) and calls the other two.
  • orders stores and lists orders. It calls inventory to reserve stock.
  • inventory answers stock questions and calls nobody.

Three services, one namespace

gateway Public entry point.
orders Creates orders.
inventory Knows the stock.

Each one gets its own Deployment and Service.

1 / 2
flowchart LR
  U["Users"] --> G["gateway"]
  G --> O["orders"]
  O --> I["inventory"]
  G --> I

Allowed calls. Anything not on an arrow should be blocked by a network policy.

Inside the cluster, services find each other by DNS name. From a pod in the shop namespace, http://orders reaches the orders Service, and the full name is orders.shop.svc.cluster.local (see DNS and Service Discovery). Your code needs no IP addresses, only a URL taken from configuration.

The namespace and one complete service

Start with a namespace so that quotas, policies and permissions can be applied to the whole shop at once.

YAML
apiVersion: v1
kind: Namespace
metadata:
  name: shop
  labels:
    app.kubernetes.io/part-of: shop

The orders service shows the full pattern. The other two look almost identical, with a different name, image and port.

YAML
apiVersion: apps/v1
kind: Deployment
metadata:
  name: orders
  namespace: shop
  labels:
    app.kubernetes.io/name: orders
    app.kubernetes.io/part-of: shop
spec:
  replicas: 2
  selector:
    matchLabels:
      app.kubernetes.io/name: orders
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  template:
    metadata:
      labels:
        app.kubernetes.io/name: orders
        app.kubernetes.io/part-of: shop
    spec:
      containers:
        - name: orders
          image: registry.example.com/shop/orders:1.4.0
          ports:
            - name: http
              containerPort: 8080
          env:
            - name: INVENTORY_URL
              value: "http://inventory"
          resources:
            requests:
              cpu: 100m
              memory: 128Mi
            limits:
              memory: 256Mi
          startupProbe:
            httpGet:
              path: /healthz
              port: http
            periodSeconds: 2
            failureThreshold: 30
          readinessProbe:
            httpGet:
              path: /ready
              port: http
            periodSeconds: 5
            failureThreshold: 2
          livenessProbe:
            httpGet:
              path: /healthz
              port: http
            periodSeconds: 10
            failureThreshold: 3
---
apiVersion: v1
kind: Service
metadata:
  name: orders
  namespace: shop
spec:
  selector:
    app.kubernetes.io/name: orders
  ports:
    - name: http
      port: 80
      targetPort: http

Several choices here are worth explaining.

  • Requests matter for autoscaling. The HPA computes utilization as a percentage of the CPU request. Without requests.cpu a CPU-based autoscaler cannot work. The lesson Resource Requests and Limits explains why this manifest sets a memory limit but no CPU limit.
  • Three probes with three jobs. The startup probe gives a slow application up to 60 seconds (2 s times 30 attempts) before liveness takes over. The readiness probe controls whether the pod receives Service traffic. The liveness probe restarts a stuck process. Point liveness at a cheap /healthz that does not check the database; otherwise one slow database restarts every pod at once. Details are in Liveness, Readiness and Startup Probes.
  • maxUnavailable: 0 means a rolling update never removes a ready pod before its replacement is ready (Rolling Updates and Rollbacks). With two replicas that costs one extra pod during updates, which is the point.
  • The Service targets a port name, http, instead of a number. The container can change its port without touching the Service.

Readiness and traffic

A Service sends traffic only to pods whose readiness probe succeeds. This is what makes rollouts and scale-ups safe: a brand-new pod that is still warming up is not in the rotation. The first stand-in models a Service with round-robin selection over ready pods, where each new pod needs a few ticks before it reports ready.

Python Service traffic with readiness gating

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Look at the tick when orders-c is created: it exists, but it is missing from the endpoint list until its readiness time arrives. Real Kubernetes tracks this in EndpointSlices, which the Service controller keeps up to date (Services).

The autoscaler

The HorizontalPodAutoscaler watches CPU use and changes the replica count. It is attached to the Deployment, never to individual pods.

YAML
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: orders
  namespace: shop
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: orders
  minReplicas: 2
  maxReplicas: 8
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 60
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300

The formula is desired = ceil(current * currentUtilization / targetUtilization), bounded by the minimum and maximum, with a 10 percent tolerance in which nothing changes (Horizontal Pod Autoscaler has the full story). The scale-down window of 300 seconds is the default; it stops the replica count from flapping when load wobbles. The HPA also needs the metrics server to be installed in the cluster. The second stand-in replays a load curve against this rule, using a window that remembers the highest recommendation of recent ticks.

Python Autoscaling the orders service under a load curve

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice that the replica count climbs quickly when the load jumps but falls more slowly afterwards. That asymmetry is deliberate: adding pods late hurts users, while removing them late only costs a little money.

Surviving maintenance: the PodDisruptionBudget

When an administrator drains a node for an upgrade, Kubernetes evicts its pods. A PodDisruptionBudget (PDB) tells it how many pods of a service may be missing at the same time.

YAML
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: orders
  namespace: shop
spec:
  minAvailable: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: orders

With two replicas and minAvailable: 1, a drain may evict one orders pod, wait for it to be rescheduled and ready, and only then evict the other. A PDB protects against voluntary disruptions such as drains; it cannot help when a node crashes (PodDisruptionBudgets, Priority and Eviction). One trap: with replicas: 1 and minAvailable: 1, no eviction is ever allowed and node upgrades hang. Use at least two replicas for anything with a PDB, or use maxUnavailable: 1.

Locking it down with network policies

By default every pod can talk to every other pod. Start from "deny all" and then open exactly the arrows from the diagram. Policies need a network plugin that enforces them (Network Policies).

YAML
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-ingress
  namespace: shop
spec:
  podSelector: {}
  policyTypes:
    - Ingress
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: inventory-from-orders-and-gateway
  namespace: shop
spec:
  podSelector:
    matchLabels:
      app.kubernetes.io/name: inventory
  policyTypes:
    - Ingress
  ingress:
    - from:
        - podSelector:
            matchLabels:
              app.kubernetes.io/name: orders
        - podSelector:
            matchLabels:
              app.kubernetes.io/name: gateway
      ports:
        - port: 8080

Similar policies would let only gateway reach orders, and let the ingress controller's namespace reach gateway. The empty podSelector: {} in the first policy selects every pod in the namespace, and listing Ingress without any rules means nothing is allowed in. The final stand-in checks a list of calls against such rules.

Python Which calls does the policy allow?

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Two things stand out. The reverse call from inventory to orders is denied, which would catch a bug or an intruder. And ingress rules say who may call a pod, so the decision always belongs to the destination. If you also deny egress, remember to allow DNS (port 53 to the cluster DNS pods), or service names stop resolving.

Checklist and next steps

Before you call a service production-ready, check it against this list: probes tell the truth, requests are set, replicas are at least two, a PDB exists, the HPA has sensible bounds, network policies match the call graph, the image tag is pinned and the pod runs as a non-root user (Pod Security and Security Contexts). To go further, add a ServiceAccount per service with no extra permissions, a ResourceQuota for the namespace and dashboards for the four golden signals (Metrics and Monitoring with Prometheus).

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why must the orders Deployment set a CPU request for the HPA to work?

    Utilization is calculated relative to the requested CPU. Without a request there is no baseline, so a CPU-based autoscaler cannot compute a percentage.
  2. 2. What does a failing readiness probe do?

    Readiness controls traffic. The pod keeps running, but the Service stops sending requests to it. Restarts are the job of the liveness probe.
  3. 3. A Deployment has replicas: 1 and a PDB with minAvailable: 1. What happens when you drain its node?

    Evicting the only pod would break the budget, so the eviction API refuses it and the drain blocks. Use at least two replicas or a looser budget.
  4. 4. After applying a default-deny ingress policy to a namespace, what is true?

    An empty pod selector with policyTypes Ingress selects all pods and lists no allowed sources. Egress is only affected if Egress is listed in policyTypes.

Summary

  • Each service gets a Deployment, a Service, probes, resource requests and labels; services find each other by DNS name.
  • Readiness decides who receives traffic, liveness decides who restarts, and startup protects slow starters.
  • The HPA scales on a percentage of the CPU request and scales down more cautiously than up.
  • A PodDisruptionBudget limits voluntary evictions; it needs at least two replicas to be useful.
  • Default-deny network policies plus explicit allow rules turn the call graph into enforced rules.
  • The Python models here illustrate the rules; the real controllers live inside the cluster.

Next, package the same ideas into reusable templates: Project: Package an App as a Helm Chart.