Skip to content
Tutorials Center
My learning

Topics

Lesson 32 of 78 ยท Configuration and Secrets

External Secrets, Sealed Secrets and Config Patterns

Keep secrets out of Git: compare External Secrets Operator, Sealed Secrets, SOPS and the Secrets Store CSI driver, plus twelve-factor configuration patterns.

A Kubernetes Secret is just base64 text. If you commit its YAML to Git, you have committed your password. But modern delivery is built on Git as the source of truth (see GitOps), which creates a puzzle: how do you keep everything in Git except the secrets, without clicking secrets into the cluster by hand?

There are four mainstream answers. Two keep the secret outside the cluster and copy it in (External Secrets Operator and the Secrets Store CSI driver). Two keep an encrypted version in Git that only the cluster or your team can open (Sealed Secrets and SOPS). This lesson compares them and then covers the configuration habits that make all of it easier. Every manifest here is read-only, since none of these tools can run in a browser; the live examples are small stand-ins of the core ideas.

These tools are separate projects that you install into a cluster, and their APIs evolve. The manifests show the typical shape; always check the project's documentation for the version you install.

The four approaches

Where does the secret live?

External store Vault or a cloud secret manager holds the truth.
External Secrets An operator copies it into a Secret.
CSI driver Mounts it straight into the Pod.

Git holds only a reference, never the value.

1 / 2

External Secrets Operator

The External Secrets Operator (ESO) is a controller that reads values from an external system (AWS Secrets Manager, Google Secret Manager, Azure Key Vault, HashiCorp Vault and many others) and writes them into ordinary Kubernetes Secrets, keeping them in sync.

You define where to read from with a SecretStore (namespaced) or ClusterSecretStore (cluster-wide):

YAML
apiVersion: external-secrets.io/v1
kind: ClusterSecretStore
metadata:
  name: cloud-secrets
spec:
  provider:
    aws:
      service: SecretsManager
      region: eu-central-1
      auth:
        jwt:
          serviceAccountRef:
            name: eso-reader
            namespace: external-secrets

Then an ExternalSecret says which values to fetch and what Secret to create:

YAML
apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
  name: db-credentials
  namespace: shop
spec:
  refreshInterval: 1h
  secretStoreRef:
    kind: ClusterSecretStore
    name: cloud-secrets
  target:
    name: db-credentials
    creationPolicy: Owner
  data:
    - secretKey: password
      remoteRef:
        key: prod/shop/db
        property: password

Only the ExternalSecret goes into Git; it contains a name, not a value. The operator authenticates using a workload identity (see Service Accounts and Workload Identity) so there is no long-lived credential to leak, then creates a normal Secret named db-credentials. Your Deployments reference it as usual. When the value changes in the store, the next refresh (here every hour) updates the Secret; Pods that read it through a volume see the change, while environment variables need a restart (see ConfigMaps for update propagation). The pull-based design also centralizes rotation and auditing in one place.

The cost: an extra component to run and secure, a dependency on the external store being reachable (existing Secrets keep working if it is down), and the fact that the values still end up as Kubernetes Secrets, so RBAC and encryption at rest still matter.

Sealed Secrets

Sealed Secrets (from Bitnami) uses asymmetric encryption. A controller in the cluster owns a private key. Anyone can encrypt a Secret with the matching public certificate, using the kubeseal command, producing a SealedSecret that is safe to commit. Only the controller can decrypt it and create the real Secret.

Terminal
kubectl create secret generic db-credentials \
  --from-literal=password='s3cr3t-Passw0rd' \
  --dry-run=client -o yaml \
  | kubeseal --format yaml > db-credentials-sealed.yaml

The result looks like this, with the ciphertext shortened:

YAML
apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
  name: db-credentials
  namespace: shop
spec:
  encryptedData:
    password: AgBy3i4OJSWK+PiTySYZZA9rO43cGDEq...
  template:
    metadata:
      name: db-credentials
      namespace: shop

By default a sealed value is bound to the Secret's name and namespace (strict scope), so copying the ciphertext into another namespace does not work. Wider scopes exist for cases where names change. The tradeoffs: the private key must be backed up (lose it and you cannot decrypt your Git history), the key is per-cluster so each cluster needs its own sealed copy, and rotating the underlying password means re-sealing.

SOPS

SOPS encrypts the values inside a YAML or JSON file while leaving the keys readable, so diffs and code review still make sense. It uses keys from age, PGP or a cloud KMS, which means you can decide who may decrypt through key access policies.

YAML
apiVersion: v1
kind: Secret
metadata:
  name: db-credentials
type: Opaque
stringData:
  password: ENC[AES256_GCM,data:Zy8f3k...,iv:...,tag:...,type:str]
sops:
  age:
    - recipient: age1examplepublickey...

SOPS is not Kubernetes-specific. GitOps tools integrate it: Flux can decrypt SOPS files natively during reconciliation, and Argo CD uses plugins such as KSOPS or Helm integrations. Unlike Sealed Secrets, the decryption key lives wherever you choose (a cloud KMS, an age key stored in the cluster), and the same file format works for Terraform variables or application config.

Secrets Store CSI driver

The Secrets Store CSI driver takes a different path: it mounts secrets from an external store directly into the Pod as files, through a CSI volume (see Volumes), without creating a Kubernetes Secret at all unless you ask it to.

YAML
apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: shop-secrets
spec:
  provider: aws
  parameters:
    objects: |
      - objectName: "prod/shop/db"
        objectType: "secretsmanager"
---
apiVersion: v1
kind: Pod
metadata:
  name: shop-api
spec:
  containers:
    - name: api
      image: example/shop-api:2.1
      volumeMounts:
        - name: secrets
          mountPath: /mnt/secrets
          readOnly: true
  volumes:
    - name: secrets
      csi:
        driver: secrets-store.csi.k8s.io
        readOnly: true
        volumeAttributes:
          secretProviderClass: shop-secrets

The secret exists only in the Pod's memory-backed mount, which reduces exposure in etcd. The Pod cannot start until the secret is fetched, so a store outage can block new Pods. An optional sync can mirror values to a Kubernetes Secret when you need environment variables.

Choosing between them

Need Good fit
Central store, audit and rotation already in a cloud or Vault External Secrets Operator or the CSI driver
Small team, one cluster, everything in Git, no external store Sealed Secrets or SOPS
Reviewable diffs of encrypted config, multiple tools SOPS
Keep secrets out of etcd entirely CSI driver (file mounts)

These are not exclusive: a common setup is ESO for runtime credentials and SOPS for bootstrap values.

Twelve-factor configuration

All these tools serve a simple principle from the "twelve-factor app" methodology: strict separation of configuration from code. Whatever varies between deployments (hosts, credentials, feature flags) lives outside the image. In Kubernetes terms:

  • Build one image and promote it unchanged from staging to production.
  • Put non-secret settings in ConfigMaps and secret ones in Secrets (or an external store), both referenced by the manifest.
  • Let overlays or Helm values (Kustomize, Helm) pick environment-specific values; never branch the application code per environment.
  • Have the app fail fast and clearly when required configuration is missing.
  • Never log secrets, and name configuration after what it means, not what it is for one environment.

Two small stand-ins

First, a toy version of SOPS-style partial encryption: keys stay readable, values become ciphertext. The cipher below is a toy built from a hash (it is not secure and exists only to show the file shape and the key-based round trip). Never use it for real data; real tools use AES-GCM and proper key management.

Python Encrypt values, keep keys readable (toy)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Second, an External Secrets style sync loop: a store whose values change, a controller that copies them into a Secret at each refresh, and a Pod that either read the value at start (environment variable) or reads the mounted file again.

Python A refresh loop like External Secrets

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

You can see the rotation reach the Secret only at the next refresh tick, the mounted file follow immediately after, and the environment variable stay on the old value until the Pod restarts. The real operator and kubelet have their own timings, but the order of events is the same.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What is stored in Git when you use the External Secrets Operator?

    ESO keeps the secret in an external store. Git only has the ExternalSecret, which contains references like a store name and a remote key.
  2. 2. Why can you safely commit a SealedSecret to a public repository?

    Sealed Secrets uses asymmetric encryption. The ciphertext is useless without the private key held by the in-cluster controller.
  3. 3. What is a notable feature of SOPS compared with encrypting the whole file?

    SOPS encrypts values in place, so the structure of the file is visible while the sensitive data is protected.
  4. 4. A Pod gets its password through an ExternalSecret-created Secret as an environment variable. The password is rotated in the external store. What must happen for the container to use the new value?

    Even after the Secret is refreshed, a running container keeps the environment it started with, so a restart or rollout is needed.
  5. 5. Which twelve-factor habit does this lesson recommend?

    Separating config from code means one image can be promoted unchanged, with environment-specific values supplied at deploy time.

Summary

  • Plain Secret manifests do not belong in Git; pick a tool that keeps the value out of Git or encrypted.
  • External Secrets Operator syncs values from a cloud store or Vault into Secrets; the CSI driver mounts them straight into Pods.
  • Sealed Secrets encrypts with a public key so only the in-cluster controller can decrypt; SOPS encrypts values in place with age or a KMS and integrates with GitOps tools.
  • Rotation reaches mounted files on its own, but environment variables need a restart.
  • Follow twelve-factor: one image, configuration in ConfigMaps, Secrets or an external store, selected per environment by overlays or values.

Next, see how containers get their environment, commands and Pod metadata in Environment Variables and the Downward API.