Skip to content
Tutorials Center
My learning

Topics

Lesson 17 of 79 ยท Data Link Layer

Link Aggregation and LACP

Learn how link aggregation bundles several Ethernet links into one logical link, how LACP negotiates it, how traffic is hashed across members, and how to configure it.

A single Ethernet cable has two weaknesses: it has a fixed speed, and it can fail. When a server or an uplink between two switches needs more than one link's worth of bandwidth, you could buy faster hardware. Often a cheaper answer is to plug in more cables of the same kind and let the devices treat them as one. That is link aggregation: several physical links between the same two devices are bundled into one logical link, with the combined capacity and with survival if one cable is cut.

The bundle has many names. Linux calls it bonding or teaming, Windows calls it NIC teaming, Cisco calls it an EtherChannel or port channel, and the IEEE standard (802.1AX, formerly 802.3ad) calls it a link aggregation group (LAG). This lesson explains what a LAG is and is not, how the Link Aggregation Control Protocol (LACP) builds one safely, and why a bundle of four 1 Gbit/s links does not give any single download 4 Gbit/s.

Why not just plug in two cables?

Plug two cables between the same pair of switches and you create a loop at layer 2. Spanning Tree Protocol notices and blocks one of the links, so you wasted a cable (it only works as a standby). A LAG tells both ends: "treat these ports as one port". The switch learns MAC addresses against the logical port, STP sees one link, and no loop exists.

Why aggregate links?

More bandwidth Four 1G links carry up to 4G in total
Redundancy A cut cable only shrinks the bundle
No STP block One logical port, so no loop

Cheaper than buying a faster NIC and switch port

1 / 2

The most important rule is this: a LAG spreads flows, not packets. Frames of one conversation should arrive in order, and if the switch sent consecutive frames of a TCP connection down different cables, small timing differences would reorder them and TCP would mistake that for loss. So the sender computes a hash of some header fields, and the hash value picks the member link. All frames with the same field values take the same cable.

Typical hash inputs, from the simplest to the most spread-out:

Policy Fields hashed Good when
Layer 2 Source and destination MAC Many different hosts talk through the bundle
Layer 2+3 MACs and IP addresses Traffic to many different IP addresses
Layer 3+4 IP addresses and TCP/UDP ports Few hosts, but many connections between them

The consequences are worth remembering:

  • A single TCP download over a 4 x 1 Gbit/s LAG is limited to 1 Gbit/s, because all of its packets hash to one link.
  • With a poor policy, several busy flows can land on the same member while others sit idle.
  • A router sitting behind the bundle has one MAC address. With a Layer 2 policy, all traffic from it to one server has one MAC pair and uses one link only.

The program below hashes 40 connections from one client to one server (same MACs and IPs, different source ports) with each policy and counts how many flows land on each of four links. The hash is CRC32, a stand-in: real switches use vendor-specific hash functions, so your actual distribution will differ, but the pattern of "which fields vary" is the same.

Python Which link does each flow take?

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

With layer 2 and layer 2+3, every one of the 40 connections has identical hash inputs, so they all use one link and three links sit idle. With layer 3+4, the changing source port spreads them out. When a member fails, the rest are re-hashed over the remaining links and existing flows may move; traffic keeps flowing after a short glitch.

Static bundles versus LACP

There are two ways to create a LAG:

  • Static (often written mode on): the administrator tells both ends "these ports are a bundle" and the devices trust it. If one end is wrongly cabled or configured, nothing detects the mistake. A cable plugged into the wrong switch can create a loop or black-hole traffic.
  • Dynamic with LACP: the two ends exchange LACP data units (LACPDUs) and bundle only the ports that really agree. This also detects a link that is "up" electrically but whose far end has stopped participating.

LACP is a layer 2 protocol. Its frames go to the slow-protocols multicast address 01:80:C2:00:00:02 and are never forwarded by other switches. Each LACPDU carries:

  • the sender's system priority and MAC address (its identity),
  • a port key (ports with the same key may be bundled together), a port priority and a port number,
  • state flags: active/passive, short/long timeout, aggregatable, synchronized, collecting and distributing.

Active and passive modes

Mode Behavior
Active Sends LACPDUs on its own, every 1 second (fast) or 30 seconds (slow).
Passive Sends LACPDUs only after it hears one from the partner.

At least one side must be active. Active with active works, and so does active with passive, but passive with passive never starts, because nobody speaks first. The program below models the main decisions a pair of switches make: modes, whether all member links lead to the same partner, and whether the speeds match.

Python Negotiating a bundle

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice how LACP quietly protects you: the miscabled port is kept out of the bundle instead of creating a loop. In a real network you would see the same result in the switch's show etherchannel summary or in /proc/net/bonding/bond0 on Linux.

flowchart LR
  subgraph A["Switch A"]
    a1["port 1"]
    a2["port 2"]
    a3["port 3"]
  end
  subgraph B["Switch B"]
    b1["port 1"]
    b2["port 2"]
    b3["port 3"]
  end
  a1 --- b1
  a2 --- b2
  a3 --- b3
  A -. "one logical link (LAG)" .- B

Three physical cables behave as one logical port on each switch.

Configuration examples

The snippets below are read-only illustrations; port names will differ on your equipment.

A Cisco-style switch puts ports into a channel group using LACP active mode:

TEXT
interface range GigabitEthernet1/0/1 - 3
 channel-group 1 mode active
!
interface Port-channel1
 switchport mode trunk

On Linux you can create an LACP bond with NetworkManager. The hash policy decides which fields spread the traffic (layer3+4 is a common choice, though it is not strictly standard-compliant for fragmented traffic):

Terminal
nmcli con add type bond ifname bond0 bond.options "mode=802.3ad,miimon=100,xmit_hash_policy=layer3+4"
nmcli con add type ethernet ifname eth1 master bond0
nmcli con add type ethernet ifname eth2 master bond0
cat /proc/net/bonding/bond0

Linux has other bonding modes that need no support from the switch, for example active-backup (one link carries traffic and the other waits) and balance-alb (adaptive load balancing). They give redundancy without a LAG on the switch side, at the price of less predictable balancing.

Limits and good practice

  • All members must match: the same speed, duplex and VLAN configuration. A mismatch keeps the port out of the bundle.
  • Each end hashes on its own: the switch and the server choose links for their own outgoing frames independently, so the two directions of one flow can use different member links.
  • Multi-chassis options: a LAG normally ends on one switch. Vendors offer MLAG, vPC or stacking so that the two ends of a server's bond can go to two different switches that appear as one to the server, which removes the switch as a single point of failure.
  • Not a speed upgrade: a LAG is not the same as a 4 Gbit/s link. Use 10 or 25 Gbit/s ports when one flow must go faster.
  • Monitor it. A bundle that has silently lost two of four members still works, but at half capacity.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why does a switch normally keep all frames of one TCP connection on the same member link?

    Different cables have slightly different delays. Spreading one flow's packets over them could deliver them out of order, which TCP treats as a sign of loss.
  2. 2. You bundle four 1 Gbit/s links. What is the best throughput a single TCP download can reach?

    A flow hashes to one member link, so one connection is limited to that link's speed. The bundle helps when there are many flows.
  3. 3. Which pair of LACP modes never forms a bundle?

    A passive port only answers LACPDUs. If both ends are passive, nobody sends the first one.
  4. 4. What advantage does LACP have over a static bundle?

    LACPDUs let both ends verify that the member links really lead to the same partner with compatible settings. A static bundle trusts the configuration blindly.

Summary

  • Link aggregation bundles several parallel links into one logical link (LAG, port channel, bond, team) for more capacity and redundancy, without creating a layer 2 loop.
  • Traffic is balanced per flow with a hash of MAC, IP or port fields, so one connection never exceeds one member's speed.
  • LACP (IEEE 802.1AX) negotiates the bundle with LACPDUs; at least one side must be active, and ports that disagree on partner or speed are excluded.
  • Choose a hash policy that matches your traffic, and use MLAG-style designs when you want redundancy across two switches.

Next: carry VLANs across the bundle in VLANs and ARP.