Skip to content
Tutorials Center
My learning

Topics

Lesson 31 of 36 ยท Testing and Deployment

Health Checks, Metrics and Tracing

Make a Quarkus service observable: SmallRye Health liveness and readiness probes, Micrometer metrics for Prometheus, OpenTelemetry traces and structured logs.

Once a service is deployed, nobody watches its console. Other systems have to ask it questions: "are you alive?", "can you take traffic?", "how many requests failed in the last minute?", "where did this slow request spend its time?". These are the three pillars of operations, and Quarkus supports them with small extensions that need almost no code: health checks, metrics and tracing (plus logging, the fourth friend).

This lesson shows each one, with the endpoints it exposes and how they connect to Kubernetes probes (containers and Kubernetes). Quarkus cannot run on this site, so the real code is read-only; the TryIts implement the core behavior (aggregation of checks, a metrics registry with Prometheus text, trace spans) in plain Java.

Health checks with SmallRye Health

Add quarkus-smallrye-health and you immediately get endpoints under /q/health:

Endpoint Question Kubernetes use
/q/health/live Is the process healthy, or should it be restarted? livenessProbe
/q/health/ready Can it take traffic right now? readinessProbe
/q/health/started Has it finished starting? startupProbe
/q/health All of the above together for people

Many extensions add their own checks automatically: a datasource adds a database readiness check, the Kafka connector adds channel checks. You add custom ones with annotations:

Java
import org.eclipse.microprofile.health.*;

@Readiness
@ApplicationScoped
public class PaymentProviderCheck implements HealthCheck {

    @Inject PaymentClient client;

    @Override
    public HealthCheckResponse call() {
        try {
            client.ping();
            return HealthCheckResponse.up("payment-provider");
        } catch (Exception e) {
            return HealthCheckResponse.named("payment-provider")
                    .down().withData("reason", e.getMessage()).build();
        }
    }
}

The response is JSON, and the HTTP status is 200 when everything is UP and 503 when any check is DOWN:

JSON
{
  "status": "DOWN",
  "checks": [
    { "name": "Database connections health check", "status": "UP" },
    { "name": "payment-provider", "status": "DOWN", "data": { "reason": "timeout" } }
  ]
}

Choose liveness and readiness carefully. A failing liveness probe makes Kubernetes kill and restart the pod. If the payment provider is down and you put that in liveness, every pod restarts in a loop while the provider is still down, which makes things worse. Dependencies belong in readiness (stop sending traffic, do not restart). Liveness should only detect "this process is stuck", for example a deadlock.

Java Stand-in: aggregating checks into UP or DOWN

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

Three signals

Health Alive? Ready? Started?
Metrics Numbers over time.
Traces One request across services.
Logs Events with context.

Each answers a different question.

1 / 2

Metrics with Micrometer

Add quarkus-micrometer-registry-prometheus. The endpoint /q/metrics now serves text in the Prometheus format, and Quarkus fills it with useful metrics automatically: HTTP server request counts and durations (by method, path template and status), JVM memory, garbage collection, thread counts and connection pools.

Your own metrics use the Micrometer API:

Java
import io.micrometer.core.instrument.MeterRegistry;
import io.micrometer.core.annotation.Counted;
import io.micrometer.core.annotation.Timed;

@ApplicationScoped
public class OrderService {

    private final MeterRegistry registry;

    OrderService(MeterRegistry registry) { this.registry = registry; }

    @Counted(value = "orders.placed", description = "Orders accepted")
    @Timed(value = "orders.processing", percentiles = {0.5, 0.95})
    public Order place(Order order) {
        registry.counter("orders.by.country", "country", order.country()).increment();
        return doPlace(order);
    }
}

Meter types you will use:

  • Counter: only goes up (requests served, errors).
  • Gauge: goes up and down (queue length, open sessions).
  • Timer and DistributionSummary: record durations or sizes, giving count, total and percentiles.

Tags (labels) split a metric into series: orders.by.country{country="DE"}. Keep tag values to a small, fixed set. A tag such as userId creates a new series per user and can overwhelm Prometheus; this is called a cardinality explosion.

Java Stand-in: a mini registry that prints Prometheus text

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

Tracing with OpenTelemetry

Metrics tell you that something is slow; a trace tells you where. With quarkus-opentelemetry every incoming HTTP request becomes a trace made of spans: one for the endpoint, nested ones for the JDBC call, a REST client call, a Kafka send. Each span has a start, a duration and attributes. Trace context travels between services in the traceparent HTTP header, so one trace follows a request across all your microservices.

PROPERTIES
quarkus.application.name=shop-orders
quarkus.otel.exporter.otlp.endpoint=http://otel-collector:4317
# sample everything in dev, a fraction in production
%dev.quarkus.otel.traces.sampler=always_on
%prod.quarkus.otel.traces.sampler=parentbased_traceidratio
%prod.quarkus.otel.traces.sampler.arg=0.1

Add your own spans where the automatic ones are too coarse:

Java
@WithSpan("price-calculation")
public int price(@SpanAttribute("book.id") long id) { ... }

Traces are sent to an OpenTelemetry Collector, which forwards them to Jaeger, Grafana Tempo, Zipkin or a commercial tool. In dev mode, Dev Services can start a Grafana LGTM stack so you see traces without setup.

Java Stand-in: spans with parent and child timing

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

Logging that supports operations

  • Use JSON logs in production (quarkus-logging-json, quarkus.log.console.json=true) so log tools can search fields.
  • With OpenTelemetry present, the trace id and span id are added to the log context, so a log line can be matched to its trace.
  • Log levels per category: quarkus.log.category."org.acme".level=DEBUG.
  • Never log secrets or whole request bodies (see secrets).

Putting it together on Kubernetes

YAML
livenessProbe:
  httpGet: { path: /q/health/live, port: 8080 }
readinessProbe:
  httpGet: { path: /q/health/ready, port: 8080 }
startupProbe:
  httpGet: { path: /q/health/started, port: 8080 }

Prometheus scrapes /q/metrics, the collector receives traces over OTLP, and your log collector reads JSON from standard output. If you generate manifests with the Kubernetes extension, the probes above are added for you.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which probe should include a check for an external payment provider?

    A failing dependency should stop traffic, not trigger restart loops. Liveness is for a stuck process.
  2. 2. Which HTTP status does /q/health/ready return when one check is DOWN?

    The overall response is 503 if any check is DOWN, which is what Kubernetes probes act on.
  3. 3. Why avoid a tag such as userId on a metric?

    High-cardinality tags multiply series; use small fixed sets such as country or status.
  4. 4. What carries trace context between two services over HTTP?

    OpenTelemetry uses the W3C traceparent header, so spans from several services join into one trace.

Summary

  • /q/health/live, /ready and /started map to Kubernetes probes; add custom checks with @Liveness, @Readiness, @Startup.
  • Micrometer exposes metrics at /q/metrics; use counters, gauges and timers, and keep tag cardinality low.
  • OpenTelemetry traces show where time goes across services; sample in production.
  • Use JSON logs with trace ids to connect logs, traces and metrics.

Next: Security with JWT and OpenID Connect