Skip to content
Tutorials Center
My learning

Topics

Lesson 43 of 70 ยท Load balancing, scaling and delivery

Amazon EKS in Depth

Understand Amazon EKS: control plane vs nodes, managed node groups, Fargate profiles, kubectl and eksctl, IAM roles for service accounts, pod networking, pricing and EKS vs ECS.

ECS is AWS's own container orchestrator. Kubernetes is the open-source one that most of the industry knows, with a huge ecosystem of tools, charts and operators. Amazon EKS (Elastic Kubernetes Service) is AWS running Kubernetes for you: it operates the control plane, patches it, spreads it across Availability Zones and connects it to IAM, VPC and load balancers.

This lesson explains what EKS manages and what you still own, the three ways to run workloads, how to connect with kubectl, how a pod gets AWS permissions, and what it costs. It assumes you know what a pod, a deployment and a service are. If not, learn those first from the Docker tutorial and any Kubernetes introduction.

EKS cannot run on this site. The commands and manifests are real and read-only. The Python programs are stand-ins that compute things EKS decides for you: how many pods fit on a node, and whether a pod's token is allowed to assume an IAM role.

Control plane and data plane

A Kubernetes cluster has two halves:

  • The control plane is the brain: the API server, the etcd database, the scheduler and the controllers. EKS runs it in an AWS-managed account, across at least two Availability Zones, and replaces unhealthy components automatically. You never see those servers. You get an API endpoint to talk to.
  • The data plane is where your pods run: worker nodes. This part runs in your account and VPC, and you choose how to provide it.
flowchart TB
    subgraph AWS["AWS-managed account"]
        CP["EKS control plane<br/>API server, etcd, scheduler"]
    end
    subgraph YOU["Your account and VPC"]
        N1["Managed node group<br/>EC2 instances"]
        N2["Fargate profile<br/>one micro-VM per pod"]
        N3["Self-managed nodes<br/>your own AMI"]
    end
    CP --- N1
    CP --- N2
    CP --- N3

EKS owns the left side. You choose one or more kinds of nodes for the right side.

Three ways to run pods

Managed node groups are EC2 instances in an Auto Scaling group that EKS creates and updates for you. EKS provisions nodes with an EKS-optimized AMI, drains them gracefully during upgrades and lets you set min, max and desired sizes. This is the default choice for most teams. You can mix On-Demand and Spot in separate groups.

Fargate profiles run each pod in its own isolated micro-VM, with no nodes to patch. A profile says "pods in the namespace batch with the label tier=small run on Fargate". You pay for the CPU and memory each pod requests. Fargate does not support some features, such as DaemonSets and privileged containers, so it fits stateless services.

Self-managed nodes are EC2 instances you build and join to the cluster yourself. You do this when you need a custom AMI, a special operating system or hardware that managed groups do not support, and you accept the patching work.

EKS Auto Mode goes further: AWS also manages node provisioning, scaling and core add-ons, so you only deploy workloads. It suits teams that want the least operations.

Creating a cluster and connecting

The quickest way to create a cluster from a terminal is eksctl, a command-line tool that writes the CloudFormation for you (read-only):

Terminal
eksctl create cluster \
  --name demo \
  --region eu-west-1 \
  --version 1.31 \
  --nodegroup-name workers \
  --node-type t3.medium \
  --nodes 2 --nodes-min 2 --nodes-max 4

This takes around 15 minutes. For repeatable infrastructure, teams use Terraform, CDK or CloudFormation instead.

To talk to the cluster, you need a kubeconfig entry that tells kubectl where the API endpoint is and how to authenticate. The AWS CLI writes it:

Terminal
aws eks update-kubeconfig --name demo --region eu-west-1
kubectl get nodes
kubectl apply -f deployment.yaml

Authentication uses your IAM identity: kubectl asks the AWS CLI for a short-lived token. Which IAM principals are allowed to do what inside Kubernetes is configured with access entries (the modern way, replacing the older aws-auth ConfigMap), which map an IAM role or user to Kubernetes permissions such as cluster admin or view-only.

Pod networking

EKS uses the Amazon VPC CNI plugin. Each pod gets a real IP address from your VPC subnet, so pods can be reached by other VPC resources and covered by VPC routing and flow logs, with no overlay network. The consequences are:

  • Pods consume subnet IP addresses. A small subnet can run out of addresses before it runs out of CPU.
  • The number of pods per node is limited by how many network interfaces (ENIs) and addresses per interface the instance type supports. The default formula is ENIs * (IPs per ENI - 1) + 2.

The first stand-in applies that formula.

Python How many pods fit on a node?

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The numbers show why small instance types can be a poor fit for pod-heavy clusters, and why "prefix delegation" (giving each ENI blocks of addresses) is a common setting when many pods are needed per node.

Giving pods AWS permissions: IRSA

A pod that uploads to S3 needs credentials. Do not give the node's instance role to every pod on it, because then every pod can do everything the node can. EKS offers per-pod roles.

IAM Roles for Service Accounts (IRSA) works like this:

  1. The cluster has an OIDC identity provider, registered in IAM, that issues signed tokens for Kubernetes service accounts.
  2. You create an IAM role whose trust policy allows sts:AssumeRoleWithWebIdentity from that provider, with a condition that the token's subject (sub) is exactly system:serviceaccount:<namespace>:<name>.
  3. You annotate the service account with the role's ARN. The AWS SDK inside the pod finds the projected token file and exchanges it for temporary credentials.

The trust policy looks like this (read-only):

JSON
{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": {
      "Federated": "arn:aws:iam::123456789012:oidc-provider/oidc.eks.eu-west-1.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE"
    },
    "Action": "sts:AssumeRoleWithWebIdentity",
    "Condition": {
      "StringEquals": {
        "oidc.eks.eu-west-1.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE:sub": "system:serviceaccount:shop:orders-api",
        "oidc.eks.eu-west-1.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE:aud": "sts.amazonaws.com"
      }
    }
  }]
}

EKS Pod Identity is a newer alternative that avoids the OIDC setup: an add-on agent hands credentials to pods, and the association between a service account and a role is created with the EKS API. The idea is the same: one role per workload.

The second stand-in checks the subject condition, which is the part people get wrong.

Python Does this service account match the trust policy?

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Other pieces you will meet

  • Add-ons: CoreDNS, kube-proxy and the VPC CNI are managed as EKS add-ons that you upgrade through the EKS API.
  • AWS Load Balancer Controller: turns a Kubernetes Ingress into an Application Load Balancer and a Service of type LoadBalancer into a Network Load Balancer.
  • Autoscaling: the Horizontal Pod Autoscaler adds pods, while Cluster Autoscaler or Karpenter adds nodes to fit pending pods. See Auto Scaling for the general ideas.
  • Upgrades: Kubernetes releases a new minor version about three times a year, and each version is supported for a limited time (standard support is about 14 months, then extended support at higher cost). Plan an upgrade routine: control plane first, then add-ons, then nodes.

What EKS costs

You pay for three things:

  1. The cluster: a fixed hourly charge per cluster (about $0.10 per hour at the time of writing, a higher rate if you stay on a version in extended support). Check the current price on the EKS pricing page.
  2. The compute: EC2 instances for node groups, or per-pod vCPU and memory for Fargate.
  3. Everything else: load balancers, NAT gateways, EBS volumes, data transfer and logs.

Because the cluster fee is per cluster, running many tiny clusters is wasteful. Namespaces, quotas and network policies let several teams share one.

EKS or ECS?

ECS EKS
API AWS-specific Standard Kubernetes
Learning curve Small Larger
Ecosystem AWS integrations Huge: Helm charts, operators, service meshes
Portability Tied to AWS Same manifests work on other clouds and on premises
Control plane cost None Per-cluster fee
Best when You want the simplest path on AWS You already use Kubernetes, need its ecosystem, or want portability

Neither is "better". Many teams do well with ECS on Fargate, and pick EKS when Kubernetes skills or tooling matter.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Who operates the Kubernetes control plane in EKS?

    EKS manages the API server and etcd for you. Your side is the data plane, the nodes that run your pods.
  2. 2. Which option runs each pod in its own isolated micro-VM, with no nodes to patch?

    A Fargate profile selects pods by namespace and labels and runs each on Fargate, billed for the CPU and memory the pod requests.
  3. 3. Why is IRSA (or Pod Identity) better than using the node's instance role for pods?

    With the node role, every pod on the node shares the same permissions. A per-service-account role follows least privilege.
  4. 4. What limits how many pods a node can run with the default VPC CNI?

    Each pod needs its own VPC IP address, so the number of ENIs and the addresses per ENI set the upper limit unless prefix delegation is used.
  5. 5. Which command writes the kubeconfig entry so kubectl can reach an EKS cluster?

    update-kubeconfig adds the cluster endpoint and an authentication command that uses your IAM identity.

Summary

  • EKS is managed Kubernetes: AWS runs the control plane across zones, you run or delegate the data plane.
  • Run pods on managed node groups (the usual choice), Fargate profiles (no nodes), self-managed nodes (full control) or Auto Mode (least operations).
  • eksctl create cluster creates a cluster quickly, and aws eks update-kubeconfig connects kubectl using your IAM identity.
  • Pods get VPC IP addresses through the VPC CNI, so subnets and instance types limit pod counts.
  • Use IRSA or Pod Identity so each workload has its own IAM role.
  • You pay a per-cluster fee plus compute and the surrounding services. Choose EKS for Kubernetes skills and ecosystem, ECS for the simplest AWS-native path.

Next: Elastic Beanstalk, App Runner and Amplify