Skip to content
Tutorials Center
My learning

Topics

Lesson 23 of 38 ยท Dependencies and Hooks

Helm Hooks: Running Jobs at Release Events

Run database migrations and other tasks at the right moment in a Helm release using hook annotations, all hook types, execution order and a lifecycle stand-in.

A release is more than a bag of manifests. Sometimes something has to happen before the new pods start (run a database migration), or after everything is up (send a notification), or before deleting (take a backup). Helm hooks let a chart attach Kubernetes objects, usually Jobs, to specific points in the release lifecycle.

A hook is a normal manifest with an annotation

Put the manifest in templates/ like any other and add the helm.sh/hook annotation:

YAML
apiVersion: batch/v1
kind: Job
metadata:
  name: {{ include "shop.fullname" . }}-migrate
  annotations:
    "helm.sh/hook": pre-upgrade,pre-install
    "helm.sh/hook-weight": "0"
    "helm.sh/hook-delete-policy": before-hook-creation,hook-succeeded
spec:
  backoffLimit: 0
  template:
    spec:
      restartPolicy: Never
      containers:
        - name: migrate
          image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
          command: ["./migrate", "--up"]

Because of the annotation, Helm does not treat this Job as a regular part of the release. It creates it at the named moment, waits for it to finish, and only then continues. If the hook Job fails, the install or upgrade fails.

The hook types

Hook When it runs
pre-install After templates render, before any resource is created
post-install After all resources are loaded into Kubernetes
pre-delete Before any resource is deleted on helm uninstall
post-delete After all release resources are deleted
pre-upgrade After render, before any resource is updated
post-upgrade After all resources have been upgraded
pre-rollback After render, before any resource is rolled back
post-rollback After all resources have been modified by a rollback
test Only when helm test is run (see tests)

You can combine several with commas, as in the example. Note that pre-install runs before the Service and Deployment exist, so a pre-install Job cannot call the app: it can only prepare things that already exist (a managed database, a ConfigMap declared as a hook too).

Hooks are not part of the release

This surprises beginners. Helm tracks hook objects only until they finish; helm uninstall will not delete leftover hook Jobs unless a delete policy cleans them up (next lesson). helm get manifest shows hooks separately from the main manifest, and helm get hooks <release> lists them.

Because Helm waits for a hook to complete, a Job that runs forever blocks the release until the --timeout expires (default 5 minutes). Keep hooks short and idempotent: an upgrade may be retried, so running your migration twice must be safe.

The lifecycle

sequenceDiagram
    participant U as helm upgrade
    participant K as Kubernetes
    U->>U: render templates
    U->>K: create pre-upgrade hooks
    K-->>U: hook Jobs complete
    U->>K: update chart resources
    U->>K: create post-upgrade hooks
    K-->>U: hook Jobs complete
    U->>U: mark release deployed

Hooks wrap the main resource update. A failing hook stops the sequence and the release is marked failed.

Hook execution order

Within one hook type, Helm sorts hooks by helm.sh/hook-weight (ascending, as numbers written as strings), then by resource kind, then by name. If three pre-upgrade hooks have weights -5, 0 and 5, they run in that order and each must finish before the next starts. This is how you run "create schema" before "load seed data". The next lesson digs into weights and cleanup.

The stand-in below simulates a release lifecycle with hooks and prints the order in which things happen. It shows the idea, not the Go implementation.

Python Stand-in: hook lifecycle during an upgrade

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice that backup does not run on install: it only declares pre-upgrade.

Typical uses

  • Database migrations in pre-upgrade,pre-install.
  • Secret or certificate creation that must exist before the Deployment starts.
  • Smoke tests or cache warm-up in post-upgrade.
  • Backups in pre-delete.
  • Registration with an external system in post-install, deregistration in post-delete.

Mistakes to avoid

  1. A migration that is not idempotent. Retries and rollbacks run hooks again.
  2. Hooks that need resources created later. A pre-install Job that mounts a ConfigMap from the same chart fails, because the ConfigMap is not there yet. Make the ConfigMap a hook with a lower weight, or use post-install.
  3. No backoffLimit or activeDeadlineSeconds. A looping Job hangs the release.
  4. Leftover Jobs. Without a delete policy, the second upgrade fails with "already exists".

When a hook fails, run kubectl logs job/<name> before retrying. Helm only reports "failed pre-upgrade hooks"; the real reason lives in the Pod.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which annotation turns a manifest into a hook?

    The helm.sh/hook annotation names the lifecycle event(s) at which Helm creates the resource.
  2. 2. In which order do hooks of the same type run?

    Helm sorts by hook-weight and waits for each to finish.
  3. 3. A pre-install hook Job fails. What happens?

    Helm waits for hooks and treats a failed hook as a failed operation.
  4. 4. Why should hooks be idempotent?

    The same hook can execute on every upgrade or rollback, so repeated runs must be safe.

Summary

  • Hooks are manifests annotated with helm.sh/hook, created at release events.
  • Helm waits for them and fails the operation if they fail.
  • Order comes from hook-weight; hooks are not removed with the release unless a delete policy says so.
  • Keep hooks short and idempotent.

Next: Hook Weights, Delete Policies and Crds