Skip to content
Tutorials Center
My learning

Topics

Lesson 102 of 119 ยท Architecture and Deployment

Deploying Java to Kubernetes

Run Spring Boot on Kubernetes properly - liveness, readiness and startup probes, graceful shutdown, memory and CPU requests and limits, JVM sizing, ConfigMaps, Secrets and troubleshooting.

Deploying Java Applications gave you the first tour of Kubernetes: pods, deployments, services. Getting a Spring Boot service to run on a cluster is easy. Getting it to run well (no restarts at deploy time, no mysterious OOMKilled, no traffic sent to a pod that isn't ready, configuration that can change without rebuilding the image) takes understanding the places where the JVM and Kubernetes interact. Those places are the subject of this lesson: probes, graceful shutdown, resource requests and limits, and ConfigMaps and Secrets.

Kubernetes commands and manifests are shown read-only here, since the course's code runner has no cluster. The runnable examples simulate the decisions Kubernetes makes (health-check timelines, quality-of-service classes, memory budgets, configuration trees) so you can experiment with the numbers. If the container side is still new to you, read Docker for Java Developers first, especially the sections on the JVM inside a container.

What Kubernetes does with your pod

stateDiagram-v2
  [*] --> Pending : scheduled onto a node
  Pending --> Starting : image pulled, container created
  Starting --> Running : startup probe succeeds
  Running --> Ready : readiness probe succeeds
  Ready --> Running : readiness fails (removed from Service)
  Running --> Restarting : liveness fails repeatedly
  Ready --> Restarting : liveness fails repeatedly
  Restarting --> Starting : container restarted
  Ready --> Terminating : delete, rollout, scale down
  Terminating --> [*] : SIGTERM, drain, exit

Probes drive the pod's life: startup decides when to begin checking, readiness decides whether it gets traffic, liveness decides whether it should be killed.

Probes: liveness, readiness and startup

Kubernetes cannot see inside your process. It asks, over HTTP (or TCP, gRPC or a command), three different questions:

Probe The question If it fails
Startup "Has the application finished starting?" Keep waiting; after failureThreshold failures, restart. Other probes don't run until it succeeds.
Liveness "Is the process stuck beyond repair?" The container is restarted.
Readiness "Can this pod handle requests right now?" The pod is removed from the Service's endpoints (no traffic) but keeps running.

Spring Boot Actuator has direct support. On Kubernetes, Boot detects the environment (through the KUBERNETES_SERVICE_HOST variable) and turns on the two availability health groups automatically; elsewhere, set management.endpoint.health.probes.enabled=true:

YAML
management:
  endpoint:
    health:
      probes:
        enabled: true
      probes.add-additional-paths: true     # also /livez and /readyz on the main port
  endpoints:
    web:
      exposure:
        include: health
server:
  shutdown: graceful
spring:
  lifecycle:
    timeout-per-shutdown-phase: 20s

This exposes /actuator/health/liveness and /actuator/health/readiness. The liveness state reflects whether the application itself is in a broken state (CORRECT or BROKEN); the readiness state is ACCEPTING_TRAFFIC once the context has started and REFUSING_TRAFFIC during start-up and shutdown. Your code can also change them, using AvailabilityChangeEvent.publish(...), for instance to take a pod out of rotation while it rebuilds a cache.

The matching container configuration:

YAML
startupProbe:
  httpGet: { path: /actuator/health/liveness, port: 8080 }
  periodSeconds: 5
  failureThreshold: 30            # up to 30 x 5 s = 150 s to start before Kubernetes gives up
livenessProbe:
  httpGet: { path: /actuator/health/liveness, port: 8080 }
  periodSeconds: 10
  timeoutSeconds: 2
  failureThreshold: 3             # three misses in a row (30 s) -> restart
readinessProbe:
  httpGet: { path: /actuator/health/readiness, port: 8080 }
  periodSeconds: 5
  timeoutSeconds: 2
  failureThreshold: 2             # two misses (10 s) -> no more traffic

The startup probe exists for slow JVM starts. Without it you would have to give the liveness probe a huge initialDelaySeconds, which also delays detecting a genuinely dead container later. With it, a fast start goes live in seconds and a slow start (loading a cache, running migrations) still gets its 150 seconds.

The big mistake: dependencies in the liveness probe

Liveness means "restarting this container will help". If the database is down, restarting your service does not help; it just adds restart storms on top of an outage, and every instance restarts at once because they all depend on the same database. The rule:

  • Liveness checks only the application itself (deadlocks, corrupted internal state). Keep the default group.
  • Readiness may include dependencies without which the pod can serve no request. Even then, think twice: if the shared database is down, every pod becomes unready at the same moment and the Service has no endpoints at all, instead of a degraded service. A common middle path keeps readiness to the application's own state and uses circuit breakers and fallbacks for dependencies (Circuit Breakers and Resilience).

To put a check into the readiness group anyway:

YAML
management.endpoint.health.group.readiness.include: readinessState,db

The simulation below runs two designs through the same incident: three seconds to start, then the database goes down for three ticks. A "tick" is one second. Pods restart after two consecutive failed liveness probes, and a restart means warming up again.

Java Probe timelines: liveness with and without the database

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

In design A the pod simply stops receiving traffic for the three seconds of the outage and resumes at once. In design B the liveness probe restarts the healthy application during the outage; it then needs another warm-up, and the pod is out of service for longer than the outage itself (eight seconds against six, and the gap grows with realistic start-up times of tens of seconds). With dozens of replicas, the restarts all hit the recovering database together.

Graceful shutdown during rollouts

Every deployment replaces pods, and each old pod must leave without dropping requests. The sequence is a race between two independent systems, and understanding it explains the odd-looking preStop hook you will see in most production manifests:

sequenceDiagram
  participant K as Kubernetes API
  participant E as Endpoints / load balancer
  participant L as kubelet
  participant A as Spring Boot app
  K->>E: remove pod from Service endpoints
  K->>L: terminate pod
  L->>A: preStop hook: sleep 10
  Note over E,A: proxies and ingress controllers<br/>learn about the change over a few seconds
  L->>A: SIGTERM
  A->>A: readiness = REFUSING_TRAFFIC, stop accepting, finish in-flight requests
  A-->>L: exit 0
  L->>A: SIGKILL if terminationGracePeriodSeconds has passed

Endpoint removal and the SIGTERM are sent in parallel. Without a short delay, some requests still arrive after the app has begun shutting down.

A pod that begins shutdown immediately may still receive new connections for a second or two, because the update to the Service endpoints propagates asynchronously. The standard remedy is a preStop hook that waits briefly before the SIGTERM:

YAML
spec:
  terminationGracePeriodSeconds: 45        # hard limit: preStop + shutdown must fit in here
  containers:
    - name: library-service
      lifecycle:
        preStop:
          exec:
            command: ["sleep", "10"]       # let the endpoint removal propagate (needs a shell image with sleep)

Then the application's own graceful shutdown (server.shutdown=graceful, timeout-per-shutdown-phase=20s) finishes in-flight work. Budget the numbers so that preStop (10) + shutdown timeout (20) < terminationGracePeriodSeconds (45). If the process is still running when the grace period ends, it gets SIGKILL, which is the failure mode you want to avoid. As the Docker lesson explained, the JVM must be PID 1 (exec form ENTRYPOINT) to receive SIGTERM at all.

Newer Kubernetes releases offer a built-in sleep action for preStop (lifecycle.preStop.sleep.seconds), which also works in distroless images that contain no sleep binary.

Resources: requests, limits and what the JVM does with them

Each container declares two numbers per resource:

  • Request: what the scheduler reserves. A pod is placed on a node with that much free capacity.
  • Limit: the maximum the container may use. Exceed the memory limit and the kernel kills the process (OOMKilled, exit code 137). Exceed the CPU limit and the container is throttled: it is simply not scheduled for part of every 100 ms period, which shows up as latency spikes rather than errors.

The ratio of requests to limits decides the pod's Quality of Service class, which matters when a node runs out of memory and Kubernetes must evict pods:

  • Guaranteed: every container has CPU and memory limits, and requests equal to them. Last to be evicted.
  • Burstable: some requests or limits are set, but not equal. Evicted after BestEffort.
  • BestEffort: nothing is set. First to be evicted. Never use it for services you care about.

The calculator below classifies pods, and then checks whether a JVM configuration fits into its memory limit. The memory figures for non-heap areas are rough, illustrative estimates; measure your own service with Actuator metrics (jvm.memory.used) or Native Memory Tracking (-XX:NativeMemoryTracking=summary).

Java QoS classes and a JVM memory budget

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

Use these rules of thumb for Java services:

  • Memory: request equals limit. The JVM sizes its heap at start-up from the limit, and memory (unlike CPU) cannot be squeezed back out of a process. Equal values avoid surprises and give you the Guaranteed class for the memory dimension.
  • Heap as a percentage of the limit: -XX:MaxRAMPercentage=70 to 75 for typical services; lower (50 to 60) for many threads, large direct buffers (Netty, NIO) or native libraries. Never set -Xmx equal to the limit.
  • Add -XX:+ExitOnOutOfMemoryError so a heap OutOfMemoryError ends the process and Kubernetes restarts it cleanly instead of leaving a half-dead JVM that still answers liveness probes.
  • CPU requests reflect typical usage and drive scheduling and HPA percentages. A CPU limit is debatable: it caps bursts but causes throttling, and JVM start-up and JIT compilation are CPU-hungry bursts. Many teams set a request and either a generous limit or none. Measure latency before choosing.
  • The JVM's idea of CPU count comes from the CPU limit. With only a request, recent JDKs ignore it and see all the node's cores, so they create too many GC and compiler threads for a small container. Set -XX:ActiveProcessorCount if you run without CPU limits.
  • A tiny CPU limit (below roughly 1 core) makes the JVM choose the Serial collector and slows start-up dramatically. Give Spring Boot services at least one full core, ideally two.
YAML
resources:
  requests:
    cpu: "500m"
    memory: "1Gi"
  limits:
    memory: "1Gi"               # CPU limit deliberately omitted; see the discussion above
env:
  - name: JAVA_TOOL_OPTIONS
    value: "-XX:MaxRAMPercentage=70 -XX:+ExitOnOutOfMemoryError"

Autoscaling. A HorizontalPodAutoscaler adds replicas when average CPU use, measured as a percentage of the request, exceeds a target such as 70%. Because the percentage is relative to the request, a request that is too low makes the HPA scale out constantly; too high and it never scales. A JVM that is busy compiling during start-up also looks "hot" for a minute, so use behavior.scaleUp stabilization or scale on application metrics (requests per second, queue depth) for better results.

Configuration: ConfigMaps and Secrets

The image stays identical in every environment; the ConfigMap (plain configuration) and Secret (sensitive values) differ. There are three ways to hand them to a Spring Boot pod.

1. As environment variables. Spring's relaxed binding turns LIBRARY_LOANS_MAXPERMEMBER into library.loans.max-per-member (Configuration, Properties and Profiles):

YAML
apiVersion: v1
kind: ConfigMap
metadata:
  name: library-config
data:
  LIBRARY_LOANS_MAXPERMEMBER: "5"
  SPRING_PROFILES_ACTIVE: "prod"
---
apiVersion: v1
kind: Secret
metadata:
  name: library-db
type: Opaque
stringData:                              # Kubernetes stores it base64-encoded
  SPRING_DATASOURCE_PASSWORD: change-me
---
# in the Deployment's container spec:
envFrom:
  - configMapRef: { name: library-config }
  - secretRef: { name: library-db }

2. As files with configtree. Mount the ConfigMap or Secret as a directory: every key becomes a file, and the file's content is the value. Spring Boot can import such a directory directly:

YAML
spring:
  config:
    import: "optional:configtree:/etc/config/"
YAML
volumeMounts:
  - { name: config, mountPath: /etc/config, readOnly: true }
  - { name: secrets, mountPath: /etc/secrets, readOnly: true }
volumes:
  - { name: config, configMap: { name: library-config-files } }
  - { name: secrets, secret: { secretName: library-db-files } }

(Add optional:configtree:/etc/secrets/ as a second import for the secret mount.) Files are preferred for secrets: environment variables leak easily (crash dumps, /proc, a debug endpoint printing env, child processes), while a mounted file can be given restrictive permissions. With a tree, a file called spring.datasource.password holds the password, and a nested folder structure also works, with directory names joined by dots.

3. A whole application.yaml from a ConfigMap, mounted into /config (a directory Spring Boot reads automatically relative to the working directory) or imported explicitly with spring.config.import=file:/etc/app/application.yaml.

This is what the configtree import does, in about twenty lines of plain Java:

Java How configtree turns files into properties

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

Three behaviors to remember about changing configuration later:

  • Environment variables from a ConfigMap are read when the container starts. Changing the ConfigMap does nothing until the pods restart (kubectl rollout restart deployment/library-service).
  • Mounted files are updated by the kubelet after a delay of up to a minute or so, but Spring Boot reads them at start-up and does not notice. Live reload needs extra machinery (Spring Cloud Kubernetes, or @RefreshScope as in Config Server). Restarting on change is simpler and safer.
  • A Secret is not encrypted by default; it is only base64. Restrict who can read it with RBAC, turn on encryption at rest in the cluster, and for serious setups use an external secret manager (External Secrets Operator, Vault, or your cloud's secret store) that syncs values into Kubernetes Secrets.

The complete Deployment

Putting the pieces together for the library service. Names and numbers are examples to adapt:

YAML
apiVersion: apps/v1
kind: Deployment
metadata:
  name: library-service
spec:
  replicas: 3
  strategy:
    rollingUpdate:
      maxSurge: 1                      # one extra pod during rollout
      maxUnavailable: 0                # never drop below the desired count
  selector:
    matchLabels: { app: library-service }
  template:
    metadata:
      labels: { app: library-service }
    spec:
      terminationGracePeriodSeconds: 45
      containers:
        - name: library-service
          image: ghcr.io/acme/library-service:1.4.2     # an immutable version tag or digest
          ports:
            - containerPort: 8080
          envFrom:
            - configMapRef: { name: library-config }
            - secretRef: { name: library-db }
          env:
            - name: JAVA_TOOL_OPTIONS
              value: "-XX:MaxRAMPercentage=70 -XX:+ExitOnOutOfMemoryError"
          resources:
            requests: { cpu: "500m", memory: "1Gi" }
            limits: { memory: "1Gi" }
          startupProbe:
            httpGet: { path: /actuator/health/liveness, port: 8080 }
            periodSeconds: 5
            failureThreshold: 30
          livenessProbe:
            httpGet: { path: /actuator/health/liveness, port: 8080 }
            periodSeconds: 10
            failureThreshold: 3
          readinessProbe:
            httpGet: { path: /actuator/health/readiness, port: 8080 }
            periodSeconds: 5
            failureThreshold: 2
          lifecycle:
            preStop:
              sleep: { seconds: 10 }
      securityContext:
        runAsNonRoot: true
---
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: library-service
spec:
  minAvailable: 2                      # node drains may never take more than one of three pods
  selector:
    matchLabels: { app: library-service }

The PodDisruptionBudget protects availability during voluntary disruptions such as node upgrades. Spread replicas over nodes or zones with topologySpreadConstraints, so a single node failure does not take all three pods down. Many teams manage these manifests with Kustomize overlays or a Helm chart, with one set of values per environment.

When something goes wrong

Start with the pod's events and previous logs; most Java problems announce themselves there:

Terminal
kubectl get pods -l app=library-service
kubectl describe pod library-service-7d9f8c6b5-x2x4k      # events, last state, probe failures, exit codes
kubectl logs library-service-7d9f8c6b5-x2x4k --previous     # logs of the crashed container
kubectl top pod -l app=library-service                      # CPU and memory (needs metrics-server)
kubectl rollout status deployment/library-service
kubectl rollout undo deployment/library-service             # back to the previous revision
kubectl port-forward deploy/library-service 8080:8080       # call it from your laptop
Symptom Usual cause for a Java service
OOMKilled, exit code 137 Heap plus non-heap exceeded the limit: lower MaxRAMPercentage, raise the limit, check thread counts and direct buffers
CrashLoopBackOff The app exits at start-up: read logs --previous (missing config, database unreachable, port clash)
Restarts right after start, Liveness probe failed Start-up slower than the probe allows: add a startupProbe, give the pod more CPU
0/1 Ready for a long time Readiness failing: call /actuator/health/readiness through port-forward and read the reason
Pending No node has the requested CPU or memory free, or a PVC or node selector can't be satisfied
ImagePullBackOff Wrong tag, missing registry credentials (imagePullSecrets)
Errors during every rollout Missing preStop delay or too short a grace period; readiness reports ready too early
Latency spikes under load, no errors CPU throttling from a low CPU limit

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What should happen when a pod's readiness probe fails?

    Readiness controls traffic only. A failing liveness probe is what restarts a container.
  2. 2. Why is it a bad idea to include the database check in the liveness probe?

    Liveness answers "will a restart help?". When a dependency is down, restarting the application only adds more load and downtime.
  3. 3. A pod has no resource requests or limits at all. Which QoS class does it get, and what does that mean?

    With nothing set the class is BestEffort. Services you care about should at least declare requests, and preferably equal memory requests and limits.
  4. 4. Which relationship between the numbers is correct for a graceful rollout?

    If the grace period ends first, the process is killed with SIGKILL while requests may still be running.
  5. 5. A Secret is mounted as files and imported with spring.config.import=optional:configtree:/etc/secrets/. The Secret in the cluster is edited. What happens to the running application?

    Configuration is bound at start-up. Roll the pods (kubectl rollout restart) or add a refresh mechanism to pick up changes.

Summary

  • Use three probes for their purposes: startup (be patient while booting), liveness (restart only a broken process), readiness (control traffic). Spring Boot's /actuator/health/liveness and /readiness map to the availability states.
  • Never put external dependencies in the liveness probe, and be careful with them in readiness.
  • For clean rollouts combine server.shutdown=graceful, a short preStop delay and a terminationGracePeriodSeconds larger than their sum; the JVM must be PID 1.
  • Size containers deliberately: memory request = limit, heap through MaxRAMPercentage (about 70 to 75), room for non-heap memory, ExitOnOutOfMemoryError; be aware of CPU throttling and how the JVM counts CPUs.
  • Deliver configuration through ConfigMaps and Secrets as environment variables or mounted files (configtree); changes need a rollout, and Secrets need real protection.
  • Add PodDisruptionBudgets, spread replicas, and learn the describe, logs --previous and rollout undo commands.

Next: Service Discovery shows how services find each other in a Spring Cloud setup, and how Kubernetes' own DNS-based discovery compares.