Skip to content
Tutorials Center
My learning

Topics

Lesson 35 of 36 ยท Projects and Practice

Project: Reviewing a Deployment Design

Practice reviewing a GitLab deployment design: promotion across environments, approvals, secrets, rollback and audit, using a scoring checklist and a promotion simulator you run.

Writing a pipeline is one skill. Reviewing someone else's pipeline, and finding the missing guard before it hurts, is another. In this project you act as the reviewer for a team that wants to deploy a payments service. You will read a design, list the risks, run a promotion simulator, and write down the changes you would request.

GitLab cannot run on this site, so the design is described in text and YAML (read-only) and the examples are JavaScript stand-ins: a scoring checklist and a simulator of promotion through environments with gates, approvals and a rollback. The ideas come from Environments and Deployments, Deployment Strategies and Rollbacks, CI Variables and Secrets Security and Compliance, Audit Events and Approvals.

The design under review

The team of the payments service proposes this:

  • Branch model: feature branches, merge requests into main. Developers can push to main directly "in emergencies".
  • Environments: staging and production on the same Kubernetes cluster, different namespaces.
  • Pipeline: build, test, push image payments:latest, deploy to staging automatically, deploy to production automatically 30 minutes later by a scheduled job.
  • Secrets: one KUBECONFIG file variable defined at group level, available to all jobs; the database password is committed in deploy/values.yaml "because the repository is private".
  • Rollback: "we redeploy from main".
  • Security: scans exist in a separate pipeline "that someone can start by hand".
  • Access: all 14 developers are Maintainers.

Here is the central deploy job:

YAML
deploy_production:
  stage: deploy
  script:
    - kubectl --kubeconfig "$KUBECONFIG" -n payments-prod set image deploy/payments app=registry.example.com/payments:latest
  environment:
    name: production
  only:
    - schedules

Review it step by step

Read the design with these questions. For each, decide risk level and requested change.

  1. Who can change what runs in production? Direct pushes to main and 14 Maintainers means a single person can ship unreviewed code and edit protected settings. Request: protected main with push "No one", merge for Developers and Maintainers, approvals, CODEOWNERS on .gitlab-ci.yml and deploy/, and only 2-3 Maintainers (see Roles, Permissions and Protected Branches).
  2. What exactly is deployed? The tag latest moves; you cannot say what runs or roll back to a known state. Request: deploy $CI_COMMIT_SHA or a release tag, record it in the environment.
  3. Where is the human decision? A timer decides when production changes. Request: a manual job (when: manual) behind a protected environment with approvals, and a deploy freeze check.
  4. Secrets. A group-wide KUBECONFIG is visible to every project and every branch, and a committed database password is a leak waiting to happen (private repositories become public by mistake, get forked and cloned). Request: a protected, environment-scoped file variable for each environment, a secret manager or sealed secrets for the database password, the committed password rotated, and secret detection in the pipeline (see Security Scanning).
  5. Blast radius. Staging and production share a cluster. A bad manifest or resource leak in staging can affect production. Request: separate clusters or at least separate node pools and quotas, and credentials for staging that cannot touch the production namespace.
  6. Rollback. "Redeploy from main" assumes main is good. Request: a documented rollback using the previous deployment job in the environment, backward compatible database migrations, and a timed rollback drill.
  7. Security scans. A manual optional pipeline is not a control. Request: scanners in the merge request pipeline, a gate on new High or Critical findings, and a policy that forces scanners to run.
  8. Evidence. Nobody can show who approved a release. Request: deployment approvals, audit event streaming, and signed commits or verified tags for releases.
  9. The syntax itself. only: schedules is the older keyword; use rules. Add resource_group: production so two deploys cannot overlap and interruptible: false.

A repaired job could look like this:

YAML
deploy_production:
  stage: deploy
  script:
    - kubectl --kubeconfig "$KUBECONFIG" -n payments-prod set image deploy/payments app="$CI_REGISTRY_IMAGE:$CI_COMMIT_SHA"
    - kubectl --kubeconfig "$KUBECONFIG" -n payments-prod rollout status deploy/payments --timeout=180s
  environment:
    name: production
    url: https://payments.example.com
  resource_group: production
  interruptible: false
  rules:
    - if: $CI_DEPLOY_FREEZE != null
      when: never
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
      when: manual

The rollout status line makes the job fail when the new pods do not become ready, so a bad release shows up as a failed deployment instead of a green job with a broken service.

flowchart LR
  B["build :SHA"] --> T["tests and scans"]
  T --> S["staging (auto)"]
  S --> A{"approvals + not frozen"}
  A -->|"approved"| P["production (manual, protected)"]
  P --> V{"rollout healthy?"}
  V -->|"yes"| OK["recorded in history"]
  V -->|"no"| RB["rollback to previous deployment"]

The reviewed design: every step up in trust adds a check.

Stand-in: a design scorer

The scorer turns the review into numbers, so you can compare designs and show progress. Each check is a yes or no question with a weight; the output lists what is missing.

JS Deployment design scorecard

Press Run (or Ctrl + Enter) to see the console output here.

No output.

The second example simulates a promotion through staging and production, with a freeze, approvals and a failed health check that triggers a rollback.

JS Promotion with gates and rollback

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Your turn

Write a one-page review for the team: a short verdict, the three changes you consider mandatory before go-live, three you consider important, and one you would postpone. Justify each with the harm it prevents. A good review is specific (name the setting), proportionate (do not demand a service mesh for a small team) and kind.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why is deploying payments:latest a problem?

    latest moves with every build. A SHA or release tag identifies exactly one image.
  2. 2. What is the main flaw of a group-wide, unprotected KUBECONFIG variable?

    Protection and environment scope limit it to protected refs and the production environment.
  3. 3. Which change makes production deploys a deliberate decision?

    A manual job needs a person, and the protected environment limits which people and requires approvals.
  4. 4. What does kubectl rollout status add to the deploy job?

    Without a readiness check the job can succeed while the service is broken.

Summary

  • A good review asks who can change production, what exactly is deployed, who decides, where secrets live, how far a failure spreads, how to roll back and what evidence exists.
  • Immutable tags, protected branches and environments, scoped secrets, approvals and health checks answer most of those questions.
  • Use a scorecard to make progress visible, and justify every requested change by the harm it prevents.

Next: Practice: GitLab CI/CD Questions