Skip to content
Tutorials Center
My learning

Topics

Lesson 18 of 78 ยท Workloads

Deployment Strategies

Compare Recreate, rolling, blue-green and canary releases on Kubernetes, build them with labels and Services, and see where Argo Rollouts and Flagger fit.

A rolling update is the right default, but it is not the only way to release software. Sometimes you must stop the old version completely before the new one starts. Sometimes you want to test the new version with five percent of real traffic first. Sometimes you want a full second environment ready, so that switching back is instant. These are deployment strategies, and Kubernetes supports several of them with nothing more than Deployments, labels and Services.

This lesson walks through the four classic strategies, shows the manifests that implement each one (read-only, since the site has no cluster), and simulates traffic splitting in Python. It ends with a short tour of the tools that automate progressive delivery. You should know Deployments and rolling updates first.

Four ways to release

Recreate Stop old, then start new.
Rolling Swap pods a few at a time.
Blue-green Two full copies, flip the switch.
Canary Send a small share to the new one.
1 / 2

Recreate

The Recreate strategy scales the old ReplicaSet to zero, waits until its pods are gone, and only then starts the new ones:

YAML
spec:
  strategy:
    type: Recreate

There is a gap with no running pods, so users see errors or a maintenance page. That is the price for a guarantee: the two versions never run at the same time. Use it when versions cannot coexist, for example a database schema change that the old code cannot read, an app that holds an exclusive lock on a resource, or a single-instance workload with a ReadWriteOnce volume that cannot be attached to two nodes. For development environments it is also the fastest way to get a clean state.

Rolling update

The default. Old and new versions run side by side for a while, with the pace set by maxSurge and maxUnavailable. It needs no extra tooling and no downtime, but it requires your app to tolerate two versions at once: a request may hit v1, the next v2. APIs and database schemas must stay backward compatible between adjacent versions (add columns before using them, remove them a release later).

Blue-green

In blue-green, you run two complete Deployments, one per version, and a Service decides which one receives traffic. Blue is live; you deploy green next to it, test it through a separate address, and then flip the Service's selector. Rolling back is flipping it back.

YAML
# Two Deployments, same app, different 'track' label
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-blue
spec:
  replicas: 3
  selector:
    matchLabels: { app: web, track: blue }
  template:
    metadata:
      labels: { app: web, track: blue }
    spec:
      containers:
        - name: web
          image: registry.example.com/web:1.0
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-green
spec:
  replicas: 3
  selector:
    matchLabels: { app: web, track: green }
  template:
    metadata:
      labels: { app: web, track: green }
    spec:
      containers:
        - name: web
          image: registry.example.com/web:2.0
---
apiVersion: v1
kind: Service
metadata:
  name: web
spec:
  selector:
    app: web
    track: blue          # change to green to switch
  ports:
    - port: 80
      targetPort: 8080

The switch itself is one command:

Terminal
kubectl patch service web -p '{"spec":{"selector":{"app":"web","track":"green"}}}'

Pros: instant cutover, instant rollback, full pre-production testing on the real cluster. Cons: you pay for two full sets of pods during the release, and in-flight connections are cut over abruptly. State is the hard part: both colors must be able to share the same database, so schema changes still need care.

Canary

A canary release sends a small share of traffic to the new version, watches the metrics, and increases the share only while things look healthy. The name comes from the canary in the coal mine: if the new version is going to hurt, it hurts a few users, not all of them.

With plain Kubernetes you can approximate it using a single Service that selects both Deployments by a shared label. A Service load-balances across all ready endpoints, so the traffic share follows the replica ratio:

YAML
# Both Deployments carry app: web; only the canary has track: canary
# Service selects only app: web, so it spreads traffic over both
apiVersion: v1
kind: Service
metadata:
  name: web
spec:
  selector:
    app: web
  ports:
    - port: 80
      targetPort: 8080

With 9 stable pods and 1 canary pod, roughly 10% of requests reach the canary. The simulation below shows that, and also blue-green's all-or-nothing selector switch. Randomness is seeded, so the run is repeatable, but individual shares will wobble around the ideal value just like real traffic.

Python Replica-ratio canary and blue-green switch

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Replica-ratio canaries are crude. To send 1% of traffic you need 100 pods; the share cannot be set independently of the number of pods; and you cannot route by header or cookie ("only internal testers"). Finer control comes from layers that understand HTTP: an Ingress controller with canary annotations (for example nginx.ingress.kubernetes.io/canary-weight in ingress-nginx), the Gateway API with weighted backends, or a service mesh.

YAML
# Gateway API: split by weight, independent of pod counts
rules:
  - backendRefs:
      - name: web-stable
        port: 80
        weight: 95
      - name: web-canary
        port: 80
        weight: 5

Progressive delivery tools

Doing canaries by hand means editing weights, watching dashboards and deciding when to proceed. Progressive delivery tools automate that loop:

  • Argo Rollouts replaces the Deployment with a Rollout resource (same pod template) that has canary and blueGreen strategies built in, with steps like "set weight 10, pause 5 minutes, set weight 50". It can query Prometheus and other providers through AnalysisTemplate objects and abort automatically.
  • Flagger keeps your normal Deployment and drives the traffic shifting through a mesh or ingress (Istio, Linkerd, NGINX, Gateway API), creating a -primary copy and a -canary copy and promoting based on metric thresholds.
  • Flux and Argo CD deploy what is in Git; they pair with the tools above for the gradual part (see GitOps).

These are read-only topics here: they are installed as extra controllers and CRDs (see custom resources and operators). The loop they run is easy to model, so the second stand-in below shows an automated canary: it increases the weight in steps, measures the canary's error rate, and aborts and rolls back when the threshold is crossed. This is the idea, not the real API of either tool.

Python Automated canary with an abort gate

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The broken release is caught at the first step, when only about 5% of requests (roughly 90 out of 2000) reached it. That is the whole value of a canary: the blast radius of a bad release is small and the rollback is automatic.

A/B testing and shadow (mirrored) deployments

Two more patterns look like canaries but answer different questions.

A/B testing compares two versions for a business outcome (does the new checkout convert better?). A canary asks "is the new version safe?" and sends a small share of random traffic. An A/B test splits users by a rule and keeps each user on one variant for the whole experiment:

  • by a header (X-Variant: b), which is how testers and QA choose a side;
  • by a cookie set on the first visit, so the user stays on the same side;
  • by a stable hash of a user id, so 20% of users always see B;
  • by geography, device or account type, when the experiment calls for it.

Plain Services cannot do this, because they pick endpoints without looking at the request. You need an HTTP-aware layer. With ingress-nginx, a second Ingress marked as a canary can route by header or cookie:

YAML
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: web-b
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-by-header: "X-Variant"
    nginx.ingress.kubernetes.io/canary-by-header-value: "b"
    # nginx.ingress.kubernetes.io/canary-by-cookie: "variant"
spec:
  ingressClassName: nginx
  rules:
    - host: shop.example.test
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: web-b
                port:
                  number: 80

With the Gateway API, header matching is part of the standard HTTPRoute:

YAML
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: shop
spec:
  parentRefs:
    - name: public-gateway
  hostnames: ["shop.example.test"]
  rules:
    - matches:
        - headers:
            - name: X-Variant
              value: b
      backendRefs:
        - name: web-b
          port: 80
    - backendRefs:                 # everyone else
        - name: web-a
          port: 80

Service meshes (Istio, Linkerd) offer the same through their routing resources (Tracing and Service Mesh). Statistical analysis (sample size, significance) happens outside Kubernetes, in your analytics tool. What Kubernetes gives you is stable, rule-based routing and metrics per variant.

Shadow deployment (also called traffic mirroring or dark launch) sends a copy of real production requests to the new version while users still get their answers from the old version. The mirror's responses are discarded, so users cannot be hurt by it. It answers questions such as "does the new version survive real traffic, and how do its latency and error rate compare?" before any user sees it. A Gateway API route mirrors with a filter:

YAML
rules:
  - backendRefs:
      - name: web-v1
        port: 80
    filters:
      - type: RequestMirror
        requestMirror:
          backendRef:
            name: web-v2-shadow
            port: 80

Meshes have equivalents (for example the mirror field in an Istio VirtualService). Mirroring is easy to get wrong, so keep these rules in mind:

  • Side effects are duplicated. If a mirrored request creates an order, sends an email or charges a card, the shadow version does it a second time. Mirror only safe (read-only) traffic, or give the shadow its own copy of the database and stub out every outbound call.
  • The shadow carries real load. Size it for the mirrored share, and mirror a fraction if you cannot afford 100%.
  • Data is real. Mirrored requests contain real user data, so the shadow namespace needs the same access controls and logging rules as production.
  • Compare, do not just look. Record latency and error rates for both versions, and diff the responses offline if you need to prove equivalence.

The model below routes requests by header, cookie and stable hash, and marks each request as mirrored. Check that the same user always lands on the same variant, and that about 20% of users see B:

Python A/B routing with sticky buckets, and a shadow mirror

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Pattern Who gets the new version Purpose
Canary A small random share Is it safe?
A/B test Users selected by a rule Which version is better for the business?
Shadow Nobody (a copy of the traffic) Does it hold up under real traffic?

Choosing a strategy

Strategy Downtime Extra resources Versions at once Rollback speed
Recreate Yes None One Slow (redeploy)
Rolling No Little Two, briefly Medium (reverse rollout)
Blue-green No Double Two, both full Instant
Canary No Little to moderate Two, for a long time Instant (set weight 0)

Start with rolling updates and good readiness probes. Add canaries when you have the metrics to judge them and the traffic to make small percentages meaningful. Use blue-green when you need instant rollback and can afford the capacity. Use Recreate only when the two versions truly cannot overlap. Feature flags are a complementary tool: deploy the code dark, then turn the feature on for a few users without any new pods.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. When is the Recreate strategy the right choice?

    Recreate stops all old pods before starting new ones, which guarantees no overlap at the cost of downtime.
  2. 2. In a blue-green release using two Deployments and one Service, how do you switch traffic?

    The Service chooses its endpoints from the pods matching its selector, so changing the selector moves all traffic at once.
  3. 3. A Service selects app: web, and there are 9 stable pods and 1 canary pod with that label. Roughly what share of requests reaches the canary?

    A Service spreads requests across ready endpoints, so the share follows the pod ratio: 1 in 10.
  4. 4. Why use the Gateway API or a mesh for canaries instead of replica ratios?

    Weighted routing at the HTTP layer allows a 1% canary with two pods and header-based routing, which replica ratios cannot do.
  5. 5. What does a tool such as Argo Rollouts or Flagger add on top of a plain Deployment?

    They automate the canary loop: shift traffic in steps, analyze metrics, and promote or roll back.

Summary

  • Recreate stops everything first (downtime, no overlap); rolling replaces pods gradually (default, two versions overlap).
  • Blue-green runs two full copies and flips a Service selector, giving instant rollback at double the cost.
  • A basic canary uses a shared label so the traffic share follows the replica ratio; weighted routing (Gateway API, ingress, mesh) decouples share from pod count.
  • Argo Rollouts and Flagger automate progressive delivery with metric analysis and automatic abort.
  • Whatever you pick, keep versions backward compatible and gate on readiness probes and real metrics.

Next: StatefulSets.