Skip to content
Tutorials Center
My learning

Topics

Lesson 35 of 37 ยท Projects and Practice

Project: Multi-environment Promotion with Argo CD

Build a promotion workflow: repository layout, one ApplicationSet for dev, staging and prod, image tag promotion by pull request, guardrails and rollbacks.

In this project you design the GitOps setup for a small product, Shop, that runs in three environments: dev, staging and prod. You will decide the repository layout, generate the Applications with one ApplicationSet, promote a new version through the environments with pull requests, and add guardrails. There is no live cluster on this site, so each step shows the files and commands; a runnable stand-in models the promotion rules.

Goals

  • One definition of the application, with small per-environment differences.
  • Every change to any environment is a reviewed Git commit.
  • A version reaches prod only after it has been in staging.
  • Rolling back is a revert.

Step 1: Repository layout

Keep application source code and deployment configuration in different repositories. CI builds images from the code repo; Argo CD watches the config repo.

TEXT
shop-gitops/
  apps/
    shop/
      base/
        deployment.yaml
        service.yaml
        kustomization.yaml
      overlays/
        dev/
          kustomization.yaml      # images: newTag: 2.9.0
        staging/
          kustomization.yaml      # images: newTag: 2.8.1
        prod/
          kustomization.yaml      # images: newTag: 2.8.0
  platform/
    applicationset.yaml
    projects.yaml

Each overlay contains only what differs: replica counts, resource limits, hostnames and the image tag. The tag is the version pinned for that environment, so "what is in prod" is answered by reading one file.

Step 2: Projects

Separate non-production from production with two AppProjects:

YAML
apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
  name: shop-nonprod
  namespace: argocd
spec:
  sourceRepos: ["https://git.example.com/shop/shop-gitops.git"]
  destinations:
    - name: nonprod
      namespace: "shop-*"
---
apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
  name: shop-prod
  namespace: argocd
spec:
  sourceRepos: ["https://git.example.com/shop/shop-gitops.git"]
  destinations:
    - name: prod
      namespace: shop-prod
  syncWindows:
    - kind: deny
      schedule: "0 22 * * 5"     # Friday 22:00
      duration: 62h               # until Monday 12:00
      applications: ["*"]

The prod project can only target the prod cluster and has a sync window that denies automatic syncs over the weekend. See RBAC Policies for who may sync what.

Step 3: One ApplicationSet for all environments

YAML
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: shop
  namespace: argocd
spec:
  goTemplate: true
  goTemplateOptions: ["missingkey=error"]
  generators:
    - list:
        elements:
          - env: dev
            cluster: nonprod
            project: shop-nonprod
            auto: "true"
          - env: staging
            cluster: nonprod
            project: shop-nonprod
            auto: "true"
          - env: prod
            cluster: prod
            project: shop-prod
            auto: "false"
  template:
    metadata:
      name: "shop-{{ .env }}"
    spec:
      project: "{{ .project }}"
      source:
        repoURL: https://git.example.com/shop/shop-gitops.git
        targetRevision: main
        path: "apps/shop/overlays/{{ .env }}"
      destination:
        name: "{{ .cluster }}"
        namespace: "shop-{{ .env }}"
      syncPolicy:
        syncOptions: [CreateNamespace=true]
  templatePatch: |
    {{- if eq .auto "true" }}
    spec:
      syncPolicy:
        automated:
          prune: true
          selfHeal: true
    {{- end }}

Dev and staging sync automatically. Prod does not: someone clicks Sync (or CI calls the API with a narrow token) after the change merges. Many teams turn auto-sync on for prod as well, once they trust their checks; the sync window and project limits remain as the safety net.

Step 4: Promotion by pull request

flowchart LR
  CI["CI builds image 2.9.0"] --> PR1["PR: dev tag to 2.9.0"]
  PR1 --> Dev["Dev syncs"]
  Dev --> PR2["PR: staging tag to 2.9.0"]
  PR2 --> Stg["Staging syncs, tests pass"]
  Stg --> PR3["PR: prod tag to 2.9.0, approvals"]
  PR3 --> Prod["Prod sync"]

Each hop is a one-line commit. The history of the overlay files is your release log.

A promotion is the same edit in the next overlay:

YAML
# apps/shop/overlays/staging/kustomization.yaml
images:
  - name: shop/web
    newTag: 2.9.0     # was 2.8.1

Automate the pull requests: CI opens the dev PR (or commits directly to dev), and a promotion job opens the staging and prod PRs after the previous environment is healthy. Require reviewers (code owners) on the prod overlay path.

An alternative is Argo CD Image Updater, which watches a registry and writes new tags back to Git; the principle is the same, with the tag change recorded in Git.

Step 5: Guardrails

  • Branch protection on main; code owners for overlays/prod/.
  • CI checks on every pull request: kustomize build for each overlay, schema validation with a tool like kubeconform, and a rule that prod's tag already exists in staging.
  • Health gates: the promotion job reads argocd app get shop-staging and refuses to proceed unless the app is Synced and Healthy.
  • Rollback: git revert the prod commit; Argo CD syncs the previous tag.
  • Notifications to a channel for failed syncs and degraded health (Notifications and Webhooks).

Step 6: Promotion rule checker (stand-in)

The stand-in encodes the central rule, "a version may only be promoted to an environment if it is currently deployed in the one before it", plus a health requirement. Use it as a model for the CI check described above.

Python Stand-in: promotion rules

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Stretch goals

  • Add a preview environment per pull request with the pull request generator.
  • Add a second prod cluster in another region and generate it with a cluster generator.
  • Gate prod promotion on Argo Rollouts analysis (Progressive Delivery with Argo Rollouts).

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why keep deployment configuration in a separate repository from application code?

    A config repo gives a clean audit trail and separate access control.
  2. 2. In this layout, how do you find which version runs in prod?

    The overlay pins the version, and Argo CD makes the cluster match it.
  3. 3. What is the rollback procedure for a bad prod release?

    Rollback is a Git operation, so history stays truthful.
  4. 4. What does the weekend sync window in the prod project do?

    A deny window prevents syncs in the matching time range, reducing weekend risk.

Summary

  • One base plus an overlay per environment, with the image tag as the pinned version.
  • One ApplicationSet generates all environments; a template patch controls auto-sync per environment.
  • Promotion is a pull request that edits the next overlay; CI checks and code owners add guardrails.
  • Rollback is a revert; projects and sync windows protect production.

Next: Project: Bootstrap a Platform