Skip to content
Tutorials Center
My learning

Topics

Lesson 23 of 38 ยท Async, Tokio and Tower

Graceful Shutdown and Timeouts

Shut an Axum server down cleanly with with_graceful_shutdown, handle Ctrl+C and SIGTERM, drain in-flight requests, and add per-request timeouts, with runnable stand-ins for the shutdown and deadline logic.

Servers get restarted all the time: deploys, autoscaling, a crashed neighbour on the same host. If your process simply dies, every in-flight request is cut off mid-response and users see errors. A graceful shutdown stops accepting new connections, lets running requests finish, and only then exits. Pair it with timeouts so a single stuck request can never hold the server hostage.

Axum gives you the hook, with_graceful_shutdown, and Tokio gives you the signal handling. This lesson shows the real code (read-only) and models the logic with standard-library stand-ins you can run.

The real thing

Rust
use axum::{routing::get, Router};
use tokio::{net::TcpListener, signal};

#[tokio::main]
async fn main() {
    let app = Router::new().route("/", get(|| async { "hello" }));
    let listener = TcpListener::bind("0.0.0.0:3000").await.unwrap();

    axum::serve(listener, app)
        .with_graceful_shutdown(shutdown_signal())
        .await
        .unwrap();

    println!("server stopped, cleaning up");
}

async fn shutdown_signal() {
    let ctrl_c = async {
        signal::ctrl_c().await.expect("install Ctrl+C handler");
    };

    #[cfg(unix)]
    let terminate = async {
        signal::unix::signal(signal::unix::SignalKind::terminate())
            .expect("install SIGTERM handler")
            .recv()
            .await;
    };
    #[cfg(not(unix))]
    let terminate = std::future::pending::<()>();

    tokio::select! {
        _ = ctrl_c => {},
        _ = terminate => {},
    }
}

with_graceful_shutdown takes any future. When that future completes, the server stops accepting connections and serve(...).await returns once all open connections have finished. ctrl_c covers a developer pressing Ctrl+C; SIGTERM is what Docker, Kubernetes and systemd send on a stop, so a production server must handle it too. tokio::select! waits for whichever fires first.

After serve returns you have a natural place for clean-up: close the database pool, flush logs and metrics, finish background tasks.

Model: a flag and a drain

At its heart, graceful shutdown is a shared "please stop" flag plus a counter of work in flight. The stand-in below runs workers that check the flag between jobs and finish the job they are on.

Rust Stop accepting, finish in-flight work (stand-in)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with io::stdin().read_line()), it gets the next line from this box.

Requests 1 to 3 complete; later ones are refused. Real Axum does this for you at the TCP level: new connections are no longer accepted, and keep-alive connections are closed after their current response.

A graceful shutdown that waits forever is a hang. Orchestrators send SIGKILL after a grace period (Kubernetes defaults to 30 seconds). Pair the drain with a timeout so the process exits on its own terms.

Request timeouts

Three kinds of timeout matter in a web service:

  1. Per-request: the whole handler must finish within N seconds. Use tower_http::timeout::TimeoutLayer, which answers 408 Request Timeout.
  2. Per-operation: one database query or upstream call must not exceed N seconds. Use tokio::time::timeout(dur, fut).
  3. Idle connection: handled by the HTTP server (keep-alive timers); in front of a proxy, set it there.
Rust
// Real code: read-only.
use std::time::Duration;
use tokio::time::timeout;

async fn report(State(db): State<PgPool>) -> Result<Json<Report>, AppError> {
    let rows = timeout(Duration::from_secs(2), build_report(&db))
        .await
        .map_err(|_| AppError::Timeout)??; // Elapsed error, then the query's own error
    Ok(Json(rows))
}

timeout works by dropping the inner future when the deadline passes. Dropping cancels it at its next .await, which is safe in Rust but means any work between awaits is simply lost. Make operations idempotent or wrap them in a database transaction.

Model: a deadline with recv_timeout

std::sync::mpsc::Receiver::recv_timeout plays the role of tokio::time::timeout for a stand-in:

Rust Giving up after a deadline (stand-in)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with io::stdin().read_line()), it gets the next line from this box.

The slow worker thread still finishes in the background; with Tokio the future would actually be dropped and stop. That is one difference between the stand-in and the real timeout.

Putting it together

A production main usually does these steps in order: load config, set up tracing, create the pool, run migrations, build the router with TimeoutLayer and TraceLayer, bind, serve with graceful shutdown, then close the pool. We return to this in the deployment lessons.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does with_graceful_shutdown do when its future completes?

    serve(...).await resolves after the shutdown future fires and every open connection has completed.
  2. 2. Which signal do Docker and Kubernetes send first when stopping a container?

    SIGTERM asks the process to exit. SIGKILL follows only if it has not stopped after the grace period.
  3. 3. How does tokio::time::timeout stop a slow operation?

    Dropping the future cancels it at its next await point. The timeout returns Err(Elapsed).
  4. 4. Which response status does a request-timeout layer typically produce?

    tower-http's TimeoutLayer replies with 408 Request Timeout when the handler exceeds the limit.

Summary

  • Graceful shutdown stops accepting new connections and drains existing ones.
  • Handle both Ctrl+C and SIGTERM; combine them with tokio::select!.
  • Always bound the drain time; orchestrators will send SIGKILL eventually.
  • Use TimeoutLayer for whole requests and tokio::time::timeout for single operations.
  • Cancellation by timeout drops the future at an await point.

Next: Setting Up SQLx and Connection Pools