Skip to content
Tutorials Center
My learning

Topics

Lesson 58 of 79 ยท Modern Networks

Cloud and Data Center Networking

Understand how cloud networks are built: VPCs, public and private subnets, route tables, security groups versus network ACLs, leaf-spine data centers, ECMP and east-west traffic.

When you launch a virtual server in a cloud, you do not plug in cables. You still need an address, a subnet, a route to the Internet, a firewall and a path to the database, and you configure them with API calls instead of screwdrivers. Behind those calls there is a physical data center network with tens of thousands of servers and a very specific design, which is built for a kind of traffic that office networks never saw.

This lesson looks at both sides. First, the logical network you control as a cloud customer: the virtual private cloud, its subnets, route tables and the two layers of filtering. Then the physical network the provider builds underneath: the leaf-spine fabric, equal-cost multipath routing and the concept of east-west traffic. Both build on ideas from earlier lessons: subnetting, NAT, firewalls and overlays.

The virtual private cloud

A virtual private cloud (VPC) (Azure calls it a virtual network) is your private, isolated network inside the provider's cloud. You choose its address range, a private CIDR block such as 10.0.0.0/16, and it is invisible to other customers, even if they chose the same addresses. Underneath, the provider implements this with an overlay, such as the VXLAN-style encapsulation you saw in the previous lesson, so one physical network carries millions of VPCs.

A VPC lives in one region (a geographic area) and spans several availability zones, separate data centers in the same region, so that a failure in one building does not take down your application.

Subnets and route tables

You divide the VPC into subnets, each tied to one availability zone. A subnet is public or private only because of its route table, the list of rules that decides where each destination goes:

Subnet Route to the Internet Typical contents
Public 0.0.0.0/0 goes to an internet gateway, which gives instances with public addresses a two-way path to the Internet Load balancers, bastion hosts
Private 0.0.0.0/0 goes to a NAT gateway in a public subnet, so instances can start connections outwards (updates, APIs) but nothing from the Internet can start a connection to them Application servers, databases

Both tables also have the local route for the VPC's own range, so all subnets of a VPC can talk to each other by default. Longest-prefix matching, as in routing basics, picks between the entries. A typical three-tier design puts the load balancer in public subnets, the application servers in private subnets and the databases in separate private subnets with no route to the Internet at all.

Cloud providers keep a few addresses in every subnet for themselves (AWS reserves five: the network address, the router, DNS, a spare, and the broadcast address), so a /24 has 251 usable hosts, not 254. VPCs can be joined with peering or a transit gateway, and connected to your office with a VPN or a dedicated circuit.

Two layers of filtering

Cloud networks have two firewalls with different behavior:

Security group Network ACL
Attached to An instance's network interface A whole subnet
Rules Allow rules only. Everything not allowed is denied Numbered allow and deny rules, evaluated in order, first match wins
State Stateful: replies to allowed requests are allowed automatically Stateless: the request and the reply are judged separately
Typical use The main control: "the web tier accepts 443 from the Internet, the database accepts 5432 only from the app tier" A coarse extra layer, blocking an address range

The stateless nature of the network ACL causes the most classic cloud problem: a rule that allows inbound HTTPS on port 443 is not enough, because the reply leaves the server from port 443 towards a high ephemeral port on the client (usually 1024 to 65535), and the outbound direction needs its own allow rule. A security group, being stateful, remembers the connection and needs no such rule. Security groups can also refer to other security groups as sources ("allow traffic from the app tier") instead of address ranges, which keeps rules valid when instances come and go.

Python Subnets, route tables, security groups and NACLs

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The first part carves four /24 subnets from the VPC. The route tables show that the only difference between public and private subnets is where the default route points. The last block demonstrates the stateless trap: with a NACL that allows inbound 443 and outbound only 443, the reply to the client's port 51000 is blocked, so the connection fails even though the security group allows it. With an outbound rule for the ephemeral range, it works. Port 22 from the Internet is denied by both layers.

Inside the data center: east-west traffic

Classic enterprise networks were built for north-south traffic: clients outside talk to a server inside, and the data crosses the network edge. A modern data center looks different. A single user request to a web service triggers dozens of calls between services, databases, caches and storage nodes, and distributed jobs such as big-data processing and machine learning shuffle huge amounts of data between servers. This east-west traffic, between servers in the same data center, often exceeds the north-south traffic by a factor of ten or more.

The older design, a three-tier tree (access, aggregation, core), is poor at this. Traffic between racks has to climb to the top of the tree, and the links near the root become bottlenecks. Their capacity is limited by the biggest and most expensive switches.

The leaf-spine fabric

Modern data centers use a leaf-spine (also called Clos) topology:

  • Leaf switches sit at the top of each rack, and every server connects to a leaf.
  • Spine switches form the second layer. Every leaf connects to every spine, and leaves never connect to each other, nor do spines.
flowchart TB
    SP1["Spine 1"] --- L1["Leaf 1"]
    SP1 --- L2["Leaf 2"]
    SP1 --- L3["Leaf 3"]
    SP2["Spine 2"] --- L1
    SP2 --- L2
    SP2 --- L3
    L1 --- S1["Servers rack 1"]
    L2 --- S2["Servers rack 2"]
    L3 --- S3["Servers rack 3"]

Any two servers on different racks are exactly three switches apart: leaf, spine, leaf. There is one path per spine.

This design has useful properties. Every server pair on different leaves has the same number of hops (leaf, spine, leaf) and as many equal-cost paths as there are spines, so latency is predictable. To add capacity you add spines, and to add servers you add leaves, with no redesign. A failed spine removes only a fraction of the capacity. The cost is a large number of cables, which is the reason fabrics are built from many identical, cheap switches.

The key design number is the oversubscription ratio: the total bandwidth of the server-facing ports of a leaf divided by its uplink bandwidth. A leaf with 32 servers at 25 Gbit/s has 800 Gbit/s of downlink; with four 100 Gbit/s uplinks it has 400 Gbit/s upward, a ratio of 2:1. At 1:1 the fabric is non-blocking, so any server can send at full speed to any other, at a higher cost. Many designs accept 2:1 or 3:1 for general workloads and use 1:1 for storage and machine-learning clusters.

Equal-cost multipath

With several equal paths, the fabric must spread the traffic over them. Equal-cost multipath (ECMP) does this: each switch hashes a flow's five-tuple (addresses, ports, protocol) and uses the hash to pick one of the equal-cost next hops. All packets of one flow take the same path, so TCP is not confused by reordering, and different flows spread over all paths. The routing underneath is often plain BGP between the switches, an application of the protocols you studied in a very regular topology.

Python Leaf-spine capacity and ECMP hashing

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The first block compares fabrics of 512 servers: with 2 spines the oversubscription is 4:1, with 4 spines 2:1 and with 8 spines the fabric is non-blocking, while the number of paths between two racks equals the number of spines. The hash spreads 4,000 flows almost evenly (roughly a thousand per spine) and re-spreads them over the three remaining spines when one fails. The last block shows ECMP's limit: it balances flows, not bytes. Four huge flows landing by chance on the same spine overload it, which carries almost twice the average load. Real fabrics use larger hash inputs, flowlet switching, or congestion-aware load balancing to deal with elephants.

Other cloud networking building blocks

  • Load balancers distribute requests over many instances and often terminate TLS, as in proxies and load balancers. Cloud load balancers are managed services that scale automatically.
  • Private endpoints let you reach a managed service (storage, a database) with a private address inside your VPC, so traffic never crosses the public Internet.
  • DNS inside a VPC resolves instance names, and private zones let you give internal services stable names, as shown in DNS in practice.
  • Content delivery puts copies of static content close to the users with a CDN.
  • Egress costs. Providers usually charge for data leaving a region or the cloud, so network layout also affects the bill. Keeping chatty services in the same availability zone is cheaper and faster.
  • Shared responsibility. The provider secures the physical fabric and the hypervisors, while you configure the route tables, groups and rules. Most cloud security incidents come from a misconfigured rule, such as a security group open to 0.0.0.0/0.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What makes a subnet 'public' in a VPC?

    Public and private are defined by routing. A default route to an internet gateway makes the subnet public; a default route to a NAT gateway (or none) makes it private.
  2. 2. Why does a stateless network ACL that allows inbound port 443 still break an HTTPS connection when the outbound rules only allow port 443?

    A stateless filter judges each direction separately. The server's reply uses source port 443 and a high destination port, which needs an outbound allow rule.
  3. 3. What is true of a security group?

    Security groups are stateful allow lists attached to network interfaces. Return traffic of an allowed connection is automatically permitted.
  4. 4. In a leaf-spine fabric with 6 spines, how many equal-cost paths exist between servers on two different leaves?

    Every leaf connects to every spine, so there is exactly one path through each spine.
  5. 5. A leaf has 24 servers at 10 Gbit/s and four 40 Gbit/s uplinks. What is its oversubscription ratio?

    Downlink is 24 x 10 = 240 Gbit/s and uplink is 4 x 40 = 160 Gbit/s, so the ratio is 240 / 160 = 1.5.

Summary

  • A VPC is a private network in the cloud with its own CIDR range. It is divided into subnets in availability zones, and a subnet is public or private only because of its route table.
  • Security groups are stateful allow lists on instances. Network ACLs are stateless, ordered allow-and-deny lists on subnets, and need explicit rules for reply traffic on ephemeral ports.
  • Data center traffic is mostly east-west, between servers. Three-tier trees handle it badly.
  • A leaf-spine fabric connects every leaf to every spine, so any two racks are three hops apart with one equal-cost path per spine. Capacity grows by adding spines and leaves.
  • The oversubscription ratio is server downlink bandwidth divided by uplink bandwidth. ECMP hashes flows over the equal paths, which balances flows but not necessarily bytes.

Next, see how audio and video travel over these networks in VoIP, RTP and Streaming Media.