Skip to content
Tutorials Center
My learning

Topics

Lesson 19 of 70 ยท Networking with VPC

NACLs, Flow Logs and Network Security

Compare stateless network ACLs with stateful security groups, learn rule ordering and ephemeral ports, read VPC flow logs and troubleshoot reachability step by step.

A VPC has two built-in filters for traffic: security groups, which belong to network interfaces, and network ACLs (NACLs), which belong to subnets. They look similar and behave very differently, and confusing them is the root of many "it works one way but not the other" bugs. When something does not connect, VPC Flow Logs record what was accepted and rejected, which turns guessing into reading.

This lesson compares the two filters, explains rule ordering and the ephemeral port trap, shows the flow log format and gives a troubleshooting routine. The AWS commands are read-only. The runnable examples are stand-ins: a small NACL evaluator and a flow log parser, written in Python, that follow the real rules closely enough to explain behavior but are not the real network.

Security groups versus NACLs

You met security groups in Connecting to Instances and Security Groups. Here is the full comparison.

Security group Network ACL
Attached to A network interface (instance, load balancer, endpoint) A subnet
State Stateful: replies to allowed traffic are allowed automatically Stateless: request and reply are checked separately
Rule types Allow rules only Allow and deny rules
Evaluation All rules are checked together Rules are checked in number order, first match wins
Default Deny all inbound, allow all outbound Default NACL allows everything; a custom NACL denies everything until you add rules
Scope of a change Applies to the interfaces using that group Applies to every resource in the subnet

Because security groups are stateful and flexible, they are the main tool: reference other security groups instead of IP ranges, and keep rules narrow. NACLs are a coarse second layer, useful for things security groups cannot do: explicitly blocking an address range, or enforcing a rule that no one can bypass by changing a security group on an instance.

NACL rules and their order

Each NACL has an inbound and an outbound list. A rule has a number, a protocol, a port range, a source or destination CIDR, and an allow or deny action. AWS evaluates from the lowest number upward and stops at the first rule that matches. The last rule, shown as *, denies everything that nothing else matched, and you cannot delete it.

Leave gaps in rule numbers (100, 200, 300) so you can insert rules later. A deny rule must have a lower number than the allow rule it should override, otherwise the allow matches first:

Rule Port Source Action
90 22 203.0.113.0/24 Deny
100 22 0.0.0.0/0 Allow
* all 0.0.0.0/0 Deny

Here SSH from 203.0.113.0/24 is blocked even though rule 100 allows everyone else.

The stateless trap: ephemeral ports

Because NACLs are stateless, allowing a request in does not allow the reply out. When a client connects to your web server on port 443, the reply leaves the server from port 443 and goes to a random high port on the client, called an ephemeral port. The NACL on the server's subnet must allow outbound traffic to that port range. The exact ephemeral range depends on the client's operating system: Linux commonly uses 32768 to 60999, Windows 49152 to 65535, and many guides recommend allowing 1024 to 65535 to be safe.

The reverse is also true: when your instance calls out to an external API, the inbound NACL must allow the reply coming back to your instance's ephemeral port. Forget either side and you see connections that hang until they time out.

flowchart LR
    C["Client<br/>port 51234"] -->|"request to port 443"| IN["Subnet NACL inbound<br/>allow 443"]
    IN --> S["Web server"]
    S --> OUT["Subnet NACL outbound<br/>allow 1024-65535"]
    OUT -->|"reply to port 51234"| C

Each direction passes a NACL check of its own; the reply needs a rule for the client's ephemeral port.

The evaluator below applies these rules exactly as described: lowest rule number first, first match wins, implicit final deny, and each direction checked separately. Run it, then change the outbound rule to see the reply get dropped.

Python A stateless NACL evaluator

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Flow logs: a record of what was seen

VPC Flow Logs capture metadata about IP traffic going to and from network interfaces: not the packet contents, only who talked to whom. You can create a flow log for a VPC, a subnet or a single network interface, and send it to CloudWatch Logs, S3 or Data Firehose. Logs are delivered with a delay, typically in aggregation windows of one or ten minutes, so they are for diagnosis and auditing, not for live alerts.

Terminal
# read-only: send VPC flow logs to an S3 bucket
aws ec2 create-flow-logs \
  --resource-type VPC --resource-ids vpc-0aaa \
  --traffic-type ALL \
  --log-destination-type s3 \
  --log-destination arn:aws:s3:::example-flow-logs

The default format is a single line of space-separated fields:

TEXT
2 123456789010 eni-0a1b2c3d 198.51.100.20 10.0.1.25 51234 443 6 10 840 1620000000 1620000060 ACCEPT OK

The fields in order are: version, account-id, interface-id, srcaddr, dstaddr, srcport, dstport, protocol, packets, bytes, start, end, action and log-status. The protocol is the IANA protocol number: 6 is TCP, 17 is UDP and 1 is ICMP. action is ACCEPT or REJECT, and a REJECT means a security group or NACL dropped the traffic. You can also choose a custom format that adds fields such as the VPC, subnet and instance IDs, and the AWS service involved.

Two limits matter when you read logs. They do not show traffic to the instance metadata address, the Amazon DNS server, or DHCP. And because they come from the interface, a security group REJECT and a NACL REJECT look alike, so you need the rules to tell them apart.

Python A flow log parser

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The two rejected port 22 flows from the same address and the single rejected port 3389 attempt look like the background scanning that every public IP address receives. Repeated rejects against a port you never opened are usually harmless noise; accepted traffic you did not expect is the thing to investigate.

Troubleshooting reachability

When "it cannot connect", walk the path in order and stop at the first problem:

  1. Is the target running and listening? Check the instance state and that the service is bound to the right port and address, not only to 127.0.0.1.
  2. Route table. Does the source subnet have a route to the destination, such as an internet gateway route for public traffic?
  3. Public address. For internet access, does the instance have a public IP or Elastic IP?
  4. Security groups. Does the target's group allow the source, port and protocol? Remember it is stateful, so no outbound rule is needed for replies.
  5. NACLs. Check both subnets, both directions, with ephemeral ports in mind.
  6. Operating system firewall. The instance's own firewall can still block traffic.
  7. Flow logs. ACCEPT on both legs of a flow with no reply points to the application; a REJECT means a filter dropped it. The Reachability Analyzer tool can also trace a path through your configuration and name the component that blocks it.

A useful pattern in flow logs: you see the request ACCEPT on the destination's interface but no matching outbound reply. That points to a NACL (the reply was dropped) or a program that is not answering, not to a security group, because security groups allow replies to accepted requests automatically.

Defense in depth

Use the layers together rather than choosing one. Keep security groups as the primary control with narrow, referenced rules. Add NACLs only for coarse subnet-level policy or explicit blocks. Turn on flow logs at least for the VPC, send them to S3 for cheap long-term storage and query them when something goes wrong. Common Mistakes and Troubleshooting collects more patterns like this.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Which statement about network ACLs is true?

    NACLs apply to a whole subnet, treat each direction separately and can both allow and deny, evaluated in rule number order.
  2. 2. A NACL has rule 100 allowing SSH from everywhere and rule 200 denying SSH from 203.0.113.0/24. What happens to SSH from 203.0.113.9?

    NACLs stop at the first matching rule in number order, so the lower-numbered allow decides. The deny would need a lower number such as 90.
  3. 3. Why does a web server behind a custom NACL need an outbound rule for ports 1024-65535?

    NACLs are stateless, so replies to the client's ephemeral port must be allowed explicitly by an outbound rule.
  4. 4. A flow log line ends with REJECT. What does it tell you?

    REJECT means the traffic was not permitted by a security group or NACL. The log has no packet contents, and it does not say which of the two filters dropped it.
  5. 5. How long should you expect to wait before flow log records appear?

    Flow logs aggregate traffic over a window before delivery, so they suit diagnosis and auditing rather than instant alerting.

Summary

  • Security groups are stateful allow-lists on network interfaces; NACLs are stateless allow-and-deny lists on subnets.
  • NACL rules run in number order and the first match wins, with an implicit deny at the end; leave gaps in numbering.
  • Stateless filtering means replies need their own rules, including the client's ephemeral port range.
  • Flow logs record accepted and rejected connections as metadata in a fixed field order, with a delay.
  • Troubleshoot in path order: listener, route, public IP, security group, NACLs, operating system firewall, then flow logs or Reachability Analyzer.

Next: Peering, Transit Gateway, VPN and Direct Connect.