Skip to content
Tutorials Center
My learning

Topics

Lesson 57 of 78 ยท Extending Kubernetes

Serverless and Event-Driven Scaling: Knative and KEDA

Scale Kubernetes workloads to zero and on events: KEDA ScaledObjects for queues and schedules, Knative Services for request-driven apps, and how both differ from the HPA.

The Horizontal Pod Autoscaler (HPA) adds and removes Pods based on CPU or memory. That fits a web server under steady load. It fits badly two other common cases: a queue worker that should run only when messages are waiting, and an occasional HTTP service that is idle most of the day. For these you want scale to zero: no Pods, and therefore no cost, until work arrives.

Two CNCF projects add that on top of Kubernetes. KEDA scales workloads on events (queue depth, a schedule, a database query). Knative Serving scales HTTP services on requests and gives you revisions and traffic splitting. This lesson explains how each works, what their manifests look like, and how to choose. Neither can run on this site, so every manifest and command is read-only; the runnable examples are Python models of the scaling rules, not the real controllers.

Why not just the HPA?

Need Plain HPA KEDA Knative Serving
Scale on CPU / memory Yes Yes (it drives an HPA) Optional
Scale on queue length, Kafka lag, cron, SQL, Prometheus query Only with a custom metrics adapter you write Built-in "scalers" for 60+ sources Not its job
Scale to zero Not by default Yes Yes
Buffer requests while scaling up from zero No No (it is not in the request path) Yes (the activator)
Revisions and percentage traffic splitting No No Yes
Typical workload Web, API Queue consumers, batch, cron-like jobs HTTP APIs, functions, web apps

Scale to zero has a price: the first request or event after idle time waits for a Pod to be scheduled, pulled and started (a cold start). Choose it for work that tolerates a few seconds of delay.

KEDA: scaling on events

KEDA runs as a small operator in your cluster. You install it with Helm (Helm Basics):

Terminal
helm repo add kedacore https://kedacore.github.io/charts
helm install keda kedacore/keda --namespace keda --create-namespace

It then adds custom resources (Custom Resources and Operators). The main one is the ScaledObject, which points at a workload and lists triggers:

YAML
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: worker-scaler
spec:
  scaleTargetRef:
    name: worker               # a Deployment (or StatefulSet or any scalable resource)
  minReplicaCount: 0           # allow scale to zero
  maxReplicaCount: 10
  pollingInterval: 30          # check the event source every 30 s
  cooldownPeriod: 120          # wait this long after the last activity before going to zero
  triggers:
    - type: rabbitmq
      metadata:
        protocol: amqp
        queueName: tasks
        mode: QueueLength
        value: "20"            # aim for 20 waiting messages per replica
        hostFromEnv: AMQP_URL  # read the connection string from the target's environment

How it works:

  1. KEDA polls the event source (here the RabbitMQ queue length) every pollingInterval.
  2. If there is activity and the workload is at zero, KEDA sets the target to one replica itself ("activation").
  3. Between 1 and maxReplicaCount, KEDA creates and feeds a regular HPA with an external metric: desired replicas is roughly ceil(queueLength / value).
  4. When the metric is idle, KEDA waits for cooldownPeriod and then scales the target to minReplicaCount (0).

The model below plays that out for a burst of messages. Change COOLDOWN and PER_REPLICA to see the effect:

Python KEDA-style scaling of a queue worker

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Other triggers and jobs

The type selects a scaler. A few common ones:

Trigger Scales on
rabbitmq, kafka, aws-sqs-queue, azure-servicebus, redis (lists, streams) Backlog in a queue or topic
cron Time windows: run 5 replicas from 08:00 to 18:00 on weekdays, otherwise 0
prometheus The result of any PromQL query (Metrics and Monitoring)
postgresql, mysql A SQL query that returns a number
cpu, memory Resource metrics (these do not scale to zero)

You can list several triggers; KEDA scales to the highest demand. Secrets for brokers go in a TriggerAuthentication resource that references a Kubernetes Secret, so connection strings stay out of the ScaledObject (Secrets).

For batch work where each message should be handled by its own run, use a ScaledJob. It creates a Kubernetes Job per unit of work (up to a limit) instead of scaling a long-lived Deployment (Jobs and CronJobs).

Things to watch:

  • Consumers must be idempotent and shut down cleanly. When KEDA scales down, Pods get SIGTERM; a worker must finish the current message (Pod Lifecycle, and the worker in the RabbitMQ project shows the pattern).
  • Do not let two autoscalers fight. A target managed by a ScaledObject must not also have its own HPA.
  • Polling delay. With pollingInterval: 30 a burst can wait up to 30 seconds before the first replica starts.

Knative Serving: request-driven services

Knative Serving turns an HTTP container into a "serverless" service. You install it and a networking layer (Kourier, Istio or Contour), then declare a Knative Service:

YAML
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: hello
spec:
  template:
    metadata:
      annotations:
        autoscaling.knative.dev/min-scale: "0"      # scale to zero when idle
        autoscaling.knative.dev/max-scale: "6"
        autoscaling.knative.dev/target: "10"        # aim for 10 concurrent requests per pod
    spec:
      containers:
        - image: registry.example.com/team/hello:1.0.0
          ports:
            - containerPort: 8080
          env:
            - name: GREETING
              value: "Hello"

From this one object Knative creates a Route (a URL such as hello.default.example.com), a Configuration, and an immutable Revision for every change to the template. Under the hood each Revision is a Deployment plus a Service, but you do not manage those yourself.

What makes it different from a Deployment plus HPA:

  • Request-based scaling. Every Pod runs a small proxy (the queue-proxy) that counts in-flight requests. The Knative autoscaler (KPA) scales to keep the concurrency near target. Pods = ceil(in-flight / target). You can scale on requests per second instead.
  • The activator. When there are zero Pods, traffic goes to a shared component called the activator, which holds the request, asks the autoscaler for a Pod, and forwards the request once it is ready. That is how the first request after idle still succeeds, with cold-start latency.
  • Revisions and traffic splitting. Each deployment creates a new Revision, and you can split traffic by percentage, which is canary release built in (Deployment Strategies):
YAML
spec:
  traffic:
    - revisionName: hello-00001
      percent: 90
    - revisionName: hello-00002
      percent: 10
  • Scale-down delay. Knative waits about a minute after the last request before removing the final Pod, so short gaps do not cause constant cold starts. You can tune it, or keep one warm Pod with min-scale: "1".

A model of the concurrency rule:

Python Knative-style concurrency autoscaling

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Knative also has Eventing (brokers, triggers and sources that deliver CloudEvents to services), which is a separate part you can adopt later. kn, the Knative CLI, wraps common commands:

Terminal
kn service create hello --image registry.example.com/team/hello:1.0.0
kn service list
kn revision list

Choosing between them

  • A queue consumer or batch/cron workload: KEDA. It works with any Deployment and does not touch the request path.
  • A plain HTTP service that is idle for long stretches, where you want revisions and canary splits built in: Knative Serving.
  • Steady traffic, no scale to zero needed: the plain HPA (maybe with a custom metric) is simpler and has no extra components (HPA, VPA and Cluster Autoscaling).
  • Combine them: KEDA can scale the Pods, and the Cluster Autoscaler can scale the nodes, so an empty queue also frees machines.

Remember that scale-to-zero removes Pods, not nodes: costs only drop when the cluster autoscaler also removes the idle machines. And every extra controller is something you must upgrade and monitor.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What can KEDA do that the plain HPA cannot do by default?

    KEDA's scalers read external event sources and its activation step takes workloads from 0 to 1 replica; between 1 and the maximum it drives a normal HPA.
  2. 2. In a ScaledObject, what does cooldownPeriod control?

    pollingInterval is the polling frequency; cooldownPeriod is the quiet time before scaling down to minReplicaCount.
  3. 3. What is the role of the Knative activator?

    When no Pods exist, requests are routed to the activator, which waits for the autoscaler to start a Pod and then forwards the request.
  4. 4. Which workload fits KEDA best?

    KEDA scales on backlog, which maps directly onto queue consumers.
  5. 5. What is the main cost of scale to zero?

    Starting a Pod (schedule, pull, start, ready) takes time, so the first work after idle is delayed.

Summary

  • The HPA scales on resource metrics; KEDA adds event-driven scaling (queues, cron, Prometheus, SQL) and scale to zero; Knative Serving adds request-driven scale to zero with revisions and traffic splitting.
  • A KEDA ScaledObject names a target, min/max replicas, pollingInterval, cooldownPeriod and triggers; between 1 and max it manages an HPA. ScaledJobs create one Job per unit of work.
  • A Knative Service scales on concurrency per Pod, uses the activator to hold requests during a cold start, and splits traffic across Revisions.
  • Scale to zero needs idempotent consumers, clean SIGTERM handling and tolerance for cold starts; saving money also needs node autoscaling.
  • Pick KEDA for queue and batch workloads, Knative for idle-heavy HTTP services, and the HPA for steady traffic.

Next, see how to extend the API itself in Custom Resources and Operators.