Skip to content
Tutorials Center
My learning

Topics

Lesson 33 of 38 ยท Deployment and Coroutines

Production Readiness for Ktor Servers

Prepare a Ktor server for real traffic: health and readiness endpoints, metrics, rate limiting, proxy headers, call IDs, request limits and graceful shutdown that drains in-flight requests.

A server that passes its tests can still fail in production. Traffic arrives in bursts, proxies hide the client's address, deployments kill processes mid-request, and nobody notices a problem until a user complains. This lesson is a checklist of the features that turn a working Ktor app into one you can operate: health checks, metrics, rate limiting, proxy awareness, correlation IDs, limits, and a shutdown that does not drop requests.

About the examples. The Ktor snippets are real and read-only (the site cannot run Ktor). Runnable blocks are labelled stand-ins: plain Kotlin that models the behavior (a token bucket, a health aggregator, a drain on shutdown) so you can see the logic work.

Health and readiness endpoints

Orchestrators and load balancers need two different answers:

  • Liveness: "is the process stuck?" If this fails, restart the container. Keep it trivial: return 200 if the server can answer at all.
  • Readiness: "can this instance take traffic right now?" This checks dependencies such as the database. If it fails, the load balancer stops sending requests, but the process is not killed.
Kotlin
routing {
    get("/health/live") { call.respondText("ok") }

    get("/health/ready") {
        val checks = mapOf(
            "database" to runCatching { dbQuery { exec("SELECT 1") } }.isSuccess,
            "cache" to cache.ping(),
        )
        val status = if (checks.values.all { it }) HttpStatusCode.OK else HttpStatusCode.ServiceUnavailable
        call.respond(status, checks)
    }
}

Do not put liveness checks behind authentication, and do not make liveness depend on the database: a database outage would then restart every healthy instance and make things worse.

Kotlin Stand-in: a readiness aggregator with timeouts

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with readln()), it gets the next line from this box.

A hanging dependency must not hang your health endpoint, so each check has a timeout and the checks run in parallel.

Metrics

You cannot fix what you cannot see. Ktor has a MicrometerMetrics plugin that records request counts, timings and JVM statistics and exposes them to a monitoring system such as Prometheus:

Kotlin
val registry = PrometheusMeterRegistry(PrometheusConfig.DEFAULT)

install(MicrometerMetrics) {
    this.registry = registry
    meterBinders = listOf(JvmMemoryMetrics(), JvmGcMetrics(), ProcessorMetrics())
}

routing {
    get("/metrics") { call.respond(registry.scrape()) }
}

Watch the RED signals for each route: Rate (requests per second), Errors (5xx share) and Duration (latency percentiles such as p95 and p99). Protect /metrics so it is reachable by your monitoring network only, not by the public internet.

Rate limiting

Without limits, one buggy client or one attacker can use all your capacity. Ktor 3 includes a RateLimit plugin:

Kotlin
install(RateLimit) {
    register(RateLimitName("api")) {
        rateLimiter(limit = 60, refillPeriod = 60.seconds)
        requestKey { call -> call.request.origin.remoteHost }
    }
    register(RateLimitName("login")) {
        rateLimiter(limit = 5, refillPeriod = 1.minutes)
        requestKey { call -> call.request.origin.remoteHost }
    }
}

routing {
    rateLimit(RateLimitName("api")) {
        get("/tasks") { /* ... */ }
    }
    rateLimit(RateLimitName("login")) {
        post("/login") { /* ... */ }
    }
}

When the limit is exceeded the client gets 429 Too Many Requests and rate-limit headers that tell it when to retry. Give sensitive endpoints such as login a much lower limit than ordinary reads. The classic algorithm behind this is the token bucket: a bucket holds up to N tokens, each request takes one, and tokens refill at a steady rate. Bursts are allowed until the bucket empties.

Kotlin Stand-in: a token bucket with a fake clock

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with readln()), it gets the next line from this box.

An in-memory limiter counts per instance. If you run several instances, each has its own counters; for a strict global limit, enforce it at the proxy or in a shared store such as Redis.

Know the real client: forwarded headers

Behind a load balancer, call.request.origin.remoteHost is the proxy's address, and the scheme looks like http even when the user connected over HTTPS. Install the forwarded-headers plugin that matches your proxy:

Kotlin
install(XForwardedHeaders)   // reads X-Forwarded-For, -Proto, -Host (for common proxies)
// or: install(ForwardedHeaders) for the standard Forwarded header

Only enable this when a trusted proxy sets the headers and strips them from outside requests. If the server is exposed directly to the internet, any client could forge X-Forwarded-For to dodge your rate limiter. Rate-limit keys, audit logs and redirect URLs all depend on this being right.

Correlation IDs

When one user action touches three services, you need a single identifier to follow it through the logs. CallId generates or reads one, and CallLogging can include it in every line:

Kotlin
install(CallId) {
    header(HttpHeaders.XRequestId)
    generate { UUID.randomUUID().toString() }
    verify { it.isNotEmpty() }
    replyToHeader(HttpHeaders.XRequestId)
}
install(CallLogging) {
    callIdMdc("call-id")
}

Add %X{call-id} to your Logback pattern (see Logging, CORS and Compression) and pass the ID on in calls to other services.

Limit what clients can send

  • Body size. Reject huge uploads early. Check Content-Length and read with a cap, or limit it at the proxy (client_max_body_size in nginx).
  • Timeouts. Set connection and idle timeouts in ktor.deployment or the engine configuration so slow clients (slowloris) cannot hold connections forever.
  • Input validation. Use request validation so bad data fails with 400 before it reaches your logic.
  • Security headers. The DefaultHeaders and a few explicit headers (X-Content-Type-Options: nosniff, Strict-Transport-Security on HTTPS) are cheap hardening.

Graceful shutdown

On every deployment the old instance receives a termination signal (SIGTERM). A hard stop would cut requests in flight and users would see errors during each release. Ktor can stop accepting new connections and give running requests time to finish:

Kotlin
embeddedServer(Netty, port = 8080, module = Application::module).start(wait = true)
YAML
ktor:
  deployment:
    shutdownGracePeriod: 2000   # ms: quiet period before stopping
    shutdownTimeout: 10000      # ms: hard limit

The grace period is how long the engine waits for no new activity; the timeout is the maximum time before it forces everything closed. Set the orchestrator's termination wait (for example Kubernetes terminationGracePeriodSeconds) longer than your timeout, or it will kill the process first. Also release resources in an ApplicationStopping hook: close the connection pool and cancel your background scope (see Coroutines in Ktor).

Kotlin Stand-in: draining in-flight requests on shutdown

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with readln()), it gets the next line from this box.

Production checklist

  • Liveness and readiness endpoints, with the right dependencies in each.
  • Metrics and structured logs with a call ID, shipped somewhere you can search.
  • Rate limits, with tighter ones on login and expensive endpoints.
  • Forwarded headers only behind a trusted proxy; HTTPS terminated at the proxy or in Ktor.
  • Request size and timeout limits; validated input.
  • Graceful shutdown tuned together with the orchestrator.
  • Secrets from the environment and a database pool sized for your instance count.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What is the difference between liveness and readiness?

    A failed liveness check restarts the process; a failed readiness check only removes it from the load balancer.
  2. 2. Why is it risky to install XForwardedHeaders on a server exposed directly to the internet?

    Only a trusted proxy should set these headers; otherwise rate limits and logs can be spoofed.
  3. 3. In a token bucket limiter with capacity 3, what happens to a burst of 5 requests at once?

    Each request spends a token; when the bucket is empty, requests get 429.
  4. 4. Why set the orchestrator's termination wait longer than shutdownTimeout?

    If the platform kills the process first, the graceful drain never completes.

Summary

  • Expose separate liveness and readiness endpoints; keep liveness simple and give checks timeouts.
  • Collect metrics (rate, errors, duration) and log a call ID with every line.
  • Rate-limit with RateLimit; use a token bucket idea and stricter limits for login.
  • Trust forwarded headers only from a known proxy.
  • Configure graceful shutdown so releases do not drop in-flight requests.

Next: The Ktor HTTP Client