Skip to content
Tutorials Center
My learning

Topics

Lesson 29 of 37 ยท Security and RBAC

Secrets in GitOps: Sealed Secrets, ESO, SOPS and Vault

Keep secrets out of Git plaintext: compare Sealed Secrets, External Secrets Operator, SOPS and Vault plugin approaches for Argo CD, with a runnable scanner.

GitOps says: everything is in Git. Secrets are the exception that makes people uneasy. A Kubernetes Secret is only base64, not encrypted, so committing one is the same as committing the password. Yet the cluster still needs these values. This lesson covers the safe patterns that fit Argo CD.

Why base64 is not protection

Terminal
echo -n 'S3cr3t!' | base64      # UzNjcjN0IQ==
echo 'UzNjcjN0IQ==' | base64 -d # S3cr3t!

Anyone with read access to the repository can reverse this in one command, and Git history keeps it forever, even after deletion. Treat a leaked secret as compromised and rotate it.

Two families of solutions

  1. Encrypt in Git, decrypt in the cluster. The repo holds ciphertext. Examples: Sealed Secrets, SOPS.
  2. Keep values outside Git, reference them. The repo holds only a pointer; a controller fetches the real value from a secret manager. Example: External Secrets Operator (ESO), Secrets Store CSI driver.

A third, less preferred family injects values at render time inside Argo CD (the Argo CD Vault Plugin and helm-secrets). It works, but secrets then flow through the repo-server and end up in Argo CD's cache and in rendered manifests.

flowchart LR
  subgraph Git
    S1["SealedSecret (ciphertext)"]
    S2["ExternalSecret (pointer)"]
  end
  S1 --> C1["Sealed Secrets controller"] --> K["Kubernetes Secret"]
  S2 --> C2["External Secrets Operator"] --> K
  V["Secret manager"] --> C2

Both routes end with an ordinary Secret created inside the cluster, not by Argo CD.

Sealed Secrets

The controller holds a private key in the cluster. Developers encrypt with the public certificate using the kubeseal CLI, and commit the resulting SealedSecret:

YAML
apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
  name: db-credentials
  namespace: shop
spec:
  encryptedData:
    password: AgBy3i4OJSWK+...
  template:
    metadata:
      name: db-credentials

Argo CD syncs the SealedSecret; the controller decrypts it into a Secret. Strengths: simple, no external service. Weaknesses: secrets are bound to one cluster key (back it up!), and rotation means re-sealing.

External Secrets Operator

You describe where a secret lives; the operator reads it from AWS Secrets Manager, Google Secret Manager, Azure Key Vault, HashiCorp Vault and others:

YAML
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: db-credentials
  namespace: shop
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: aws-store
    kind: ClusterSecretStore
  target:
    name: db-credentials
  data:
    - secretKey: password
      remoteRef:
        key: prod/shop/db
        property: password

Git contains no sensitive material at all. Rotation happens in the secret manager and propagates on the refresh interval. This is the pattern most platform teams choose for production.

SOPS

SOPS encrypts only the values of a YAML file with a key from age, KMS or PGP, keeping the structure readable in pull requests. Argo CD can decrypt through a plugin (a CMP sidecar with KSOPS or the helm-secrets plugin), which means the decryption key must be available to the repo server. Pick this when you want reviewable diffs and already use SOPS.

Argo CD Vault Plugin

Manifests contain placeholders such as <path:kv/data/shop/db#password> that a plugin replaces during rendering. It is flexible but puts plaintext secrets in rendered manifests and Argo CD's cache, and is generally discouraged for new setups in favor of operators.

Comparing the approaches

Approach Secret in Git Where decryption happens Rotation Extra moving parts
Sealed Secrets Ciphertext In cluster Re-seal and commit One controller
ESO Pointer only In cluster (operator) In the secret manager Operator and a secret manager
SOPS plus plugin Ciphertext In the Argo CD repo server Re-encrypt and commit Plugin sidecar and keys
Vault plugin Placeholder In the Argo CD repo server In Vault Plugin and Vault

Things to remember about Argo CD

  • Diffing: operator-created Secrets are not in Git. Argo CD does not manage them (they are owned by the operator), so no drift appears. Do not enable prune on resources you do not track.
  • Ordering: a Deployment that mounts a Secret needs it to exist. Put the ExternalSecret in an earlier sync wave, or accept a brief restart loop that heals itself.
  • Argo CD's own secrets: repository credentials and cluster Secrets are Kubernetes Secrets too. Manage them with the same tools.
  • Prevent mistakes: add a pre-commit or CI scan that rejects plaintext Secret manifests.

Stand-in: a plaintext secret scanner

A scanner like this belongs in CI. This version reads manifests as dictionaries; real scanners (for example gitleaks or kubeconform plus custom rules) parse files.

Python Stand-in: reject plaintext Secrets

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why is committing a Kubernetes Secret manifest unsafe?

    Anyone with repo access can decode it, and old commits keep the value.
  2. 2. What does the repository contain with External Secrets Operator?

    ESO resources only point at the remote secret; the operator creates the Secret in the cluster.
  3. 3. Which approach puts decrypted secrets through the Argo CD repo server?

    Plugin-based methods decrypt while rendering, so values exist in the repo server and its cache.
  4. 4. A Deployment fails at first because its Secret is not there yet. A fix?

    Sync waves order resources so dependencies exist first.

Summary

  • Never commit plaintext Secrets; base64 is not encryption.
  • Either encrypt in Git (Sealed Secrets, SOPS) or reference an external store (ESO).
  • Operator-based approaches keep secrets out of the Argo CD repo server.
  • Scan for plaintext Secrets in CI and order dependencies with sync waves.

Next: Notifications and Webhooks