Security Groups, Network ACLs, and Traffic Inspection

Security groups, network ACLs, and traffic inspection are the packet-control tools most AWS engineers use inside an Amazon VPC. By the end of this lesson you should be able to look at a workload path, decide which layer should allow, deny, log, or inspect the flow, and explain why a packet is accepted or dropped.

This topic follows VPC routing in the AWS Cloud course because routing and filtering are separate decisions. A route table chooses the next hop: local VPC target, internet gateway, NAT gateway, transit gateway, endpoint, or appliance. Security groups, NACLs, and inspection policies decide whether the routed packet may continue. A correct route is necessary, but it does not override a closed port, a subnet deny, or a firewall policy.

Purpose and Outcome

A security group is a stateful virtual firewall attached to an elastic network interface. You may configure it from an EC2, load balancer, RDS, Lambda VPC, or interface endpoint page, but enforcement happens on the ENI used by that resource. Security groups express workload intent: a load balancer can reach a web tier on TCP 443, the web tier can reach an app tier on TCP 8080, and the app tier can reach a database on TCP 5432.

A network ACL is a stateless ordered rule list associated with a subnet. It evaluates traffic entering and leaving that subnet boundary. NACLs are best for coarse guardrails: an emergency deny for a hostile CIDR, a subnet-wide restriction, or a separation between subnet ownership and instance ownership. Traffic inspection adds a routed hop through AWS Network Firewall, a Gateway Load Balancer appliance, or a proxy when simple address, protocol, and port rules are not enough.

How Enforcement Works

Security groups have allow rules only. When a packet arrives at an ENI, AWS evaluates the union of all security groups attached to that interface. If any inbound rule allows the packet, it is accepted. If no inbound rule allows it, it is dropped. Outbound rules work the same way. Because the group is stateful, the response to an allowed request is automatically permitted, even when the response uses an ephemeral port that is not separately opened in the reverse direction.

Security group references are more durable than subnet CIDRs for tier-to-tier access. A rule can allow traffic from another security group, meaning the source ENI must have that group attached. It does not route traffic through the referenced group, and it does not mean every instance in the subnet is trusted. Use references when policy should follow the workload rather than its current private IP address.

NACLs work differently. Rules are evaluated by ascending rule number, and the first match wins. Each rule has direction, protocol, port range, CIDR, and an allow or deny action. Every custom NACL has an implicit final deny. Since NACLs are stateless, requests and responses must both be allowed explicitly. For TCP client traffic, that often means outbound service ports and inbound ephemeral return ports on the client subnet, plus inbound service ports and outbound ephemeral return ports on the server subnet.

The evaluation order matters during troubleshooting. For traffic between two instances in the same subnet, the subnet NACL is not used because the packet does not cross the subnet boundary, but both instances’ security groups still matter. For traffic entering or leaving a subnet, the NACL and security group can both block it. Flow Logs record accepted and rejected flows at the ENI, subnet, or VPC level, but they do not name the exact rule. You infer the rule by comparing direction, address, port, route target, and recent changes.

Inspection controls only the traffic routed to them. A firewall endpoint cannot inspect packets that bypass it. Centralized egress inspection normally uses protected subnet route tables that send default traffic to firewall or appliance endpoints, then inspection subnet routes that send allowed traffic toward NAT, internet, transit gateway, or another VPC. Symmetric routing matters because many inspection systems track connection state. If the request crosses the firewall but the response takes a different path, the flow may fail or logs may show only half the truth.

Rule Anatomy

A security group rule contains direction, protocol, port range, source or destination, and an optional description. Sources and destinations can be CIDRs, prefix lists, or security groups where supported. Descriptions are operational data, not decoration: they explain why TCP 443 is public or why one tier may call another.

A NACL rule adds a rule number and an explicit action. Leave gaps, such as increments of 10 or 100, so a more specific deny can be inserted ahead of a broader allow. The default NACL allows all inbound and outbound traffic. A new custom NACL denies traffic until you add rules. For ICMP, UDP, and TCP, make sure protocol numbers and port ranges match the actual traffic, not just the application name.

An inspection design has three required pieces: routes that force the path through the inspection point, a policy that allows, drops, alerts, or forwards traffic, and logs sent to CloudWatch Logs, S3, Kinesis Data Firehose, or a security analytics system. In multi-account networks, these pieces are often owned by different teams, so route-table evidence is as important as firewall policy evidence.

Worked Example: Public HTTPS

set -euo pipefail

VPC_ID="vpc-0123456789abcdef0"
WEB_SG_ID=$(aws ec2 create-security-group \
  --group-name course-web-sg \
  --description "Allow HTTPS from the internet for the course web tier" \
  --vpc-id "$VPC_ID" \
  --query 'GroupId' \
  --output text)

aws ec2 authorize-security-group-ingress \
  --group-id "$WEB_SG_ID" \
  --ip-permissions 'IpProtocol=tcp,FromPort=443,ToPort=443,IpRanges=[{CidrIp=0.0.0.0/0,Description="Public HTTPS"}]'

aws ec2 describe-security-groups \
  --group-ids "$WEB_SG_ID" \
  --query 'SecurityGroups[0].IpPermissions'

This creates a web security group and allows inbound TCP 443 from any IPv4 address. Expected behavior is narrow: new inbound HTTPS connections can reach ENIs using this group, while inbound TCP 22, TCP 80, and UDP remain denied unless another attached group allows them. Return packets for accepted HTTPS sessions are allowed by state tracking. If this group is attached to a load balancer instead of an instance, the instance still needs its own rule allowing traffic from the load balancer’s security group.

Worked Example: Tier-to-Tier Access

set -euo pipefail

VPC_ID="vpc-0123456789abcdef0"
WEB_SG_ID="sg-0web123456789abcd"
APP_SG_ID=$(aws ec2 create-security-group \
  --group-name course-app-sg \
  --description "Allow app traffic only from the course web tier" \
  --vpc-id "$VPC_ID" \
  --query 'GroupId' \
  --output text)

aws ec2 authorize-security-group-ingress \
  --group-id "$APP_SG_ID" \
  --protocol tcp \
  --port 8080 \
  --source-group "$WEB_SG_ID"

aws ec2 describe-security-groups \
  --group-ids "$APP_SG_ID" \
  --query 'SecurityGroups[0].IpPermissions[0].UserIdGroupPairs[0].GroupId' \
  --output text

The final query should print sg-0web123456789abcd. An ENI with course-web-sg can initiate TCP 8080 to an ENI with course-app-sg. Another instance in the same subnet but without the web group is denied, even if its private IP is inside the same subnet CIDR. This is the main benefit of group references: policy follows role, not address allocation. It also reduces accidental trust when development, batch, and web instances share one subnet.

Worked Example: Subnet Deny with a NACL

set -euo pipefail

NACL_ID="acl-0123456789abcdef0"
BAD_CIDR="203.0.113.0/24"

aws ec2 create-network-acl-entry \
  --network-acl-id "$NACL_ID" \
  --ingress \
  --rule-number 90 \
  --protocol 6 \
  --port-range From=22,To=22 \
  --cidr-block "$BAD_CIDR" \
  --rule-action deny

aws ec2 create-network-acl-entry \
  --network-acl-id "$NACL_ID" \
  --ingress \
  --rule-number 100 \
  --protocol 6 \
  --port-range From=1024,To=65535 \
  --cidr-block 0.0.0.0/0 \
  --rule-action allow

The lower-numbered rule denies SSH from 203.0.113.0/24 before any broader allow. The second rule allows inbound ephemeral TCP responses for clients in the subnet that initiated outbound connections. Expected behavior: SSH from the denied CIDR fails at the subnet boundary before a security group is evaluated, while normal outbound HTTPS can still work if outbound NACL rules, routes, and security groups allow it. A real production deny should use the actual hostile range, not a documentation CIDR.

Worked Example: Inspection Route

{
  "ProtectedSubnetRouteTable": {
    "DestinationCidrBlock": "0.0.0.0/0",
    "Target": "firewall-or-appliance-endpoint"
  },
  "InspectionSubnetRouteTable": {
    "DestinationCidrBlock": "0.0.0.0/0",
    "Target": "nat-gateway-or-internet-egress"
  }
}

This fragment shows the inspection shape rather than a complete template. Instances in protected subnets send default traffic to a firewall or appliance endpoint instead of directly to NAT. If policy allows the connection, packets continue to egress. If policy denies or drops it, the client sees a timeout or connection failure, and firewall logs should show the rule or default action that made the decision. For east-west inspection between VPCs, apply the same idea to transit gateway or peering routes.

Design Choices and Trade-Offs

Use security groups for application reachability between AWS resources. They are close to the workload, stateful, and resilient to IP changes when you use group references. Their limitation is the lack of explicit deny. If an interface has two groups and either one allows SSH, SSH is allowed. This additive model is simple, but it means old broad rules can silently defeat a newer restrictive group.

Use NACLs for subnet-level decisions that should apply before traffic reaches any ENI. They can deny traffic and are useful for emergency blocks, but ordered stateless rules are easy to misconfigure. Large NACLs are hard to audit, and ephemeral ports make narrow return-path rules brittle. A subnet containing mixed workloads is a poor place for detailed NACL policy because one rule set applies to every ENI in that subnet.

Use inspection when you need controls beyond five-tuple filtering: domain filtering, threat signatures, centralized egress logging, protocol awareness, or third-party appliance features. The trade-offs are cost, latency, route-table complexity, and availability design. Place inspection endpoints in each Availability Zone used by protected workloads so traffic does not depend on a single zone or incur avoidable cross-zone data charges.

Failure Modes and Troubleshooting

Symptom: HTTPS works from one source but not another. Cause: the security group source is narrower than expected, the client is reaching a different ENI, or another attached group changed the effective rule set. Diagnosis: inspect the destination ENI groups, confirm the resolved destination IP, and check VPC Flow Logs for ACCEPT or REJECT. Correction: update the intended source CIDR or group reference and remove obsolete broad rules.

Symptom: package downloads from a private subnet hang. Cause: a custom NACL allows outbound TCP 443 but blocks inbound ephemeral return traffic, or an inspection route lacks a valid return path. Diagnosis: review both NACL directions, compare source and inspection route tables, and look for one-way flow log records. Correction: allow the needed ephemeral range, fix symmetric routing, or restore the managed NAT route if inspection is not required.

Symptom: firewall logs are empty while applications make requests. Cause: traffic bypasses the inspection endpoint. Diagnosis: trace the route from the protected subnet to the destination and back, then verify that the firewall endpoint is the next hop for the relevant CIDR. Correction: update protected and return route tables so the firewall sees the complete flow.

Security, Performance, and Reliability

Review security groups for public administrative ports. Public HTTPS can be intentional; public SSH, RDP, or database ports usually are not. Use tags, descriptions, and prefix lists to reduce repeated CIDRs and make ownership clear. Prefer least privilege at the security group layer before adding broader subnet controls.

Keep NACLs short and reserved for decisions that truly belong at subnet scope. Inspection provides richer protection, but it becomes a dependency for every routed flow. Monitor firewall endpoint health, dropped packets, capacity, latency, and log delivery. A bad inspection route can affect an entire protected tier, so test route changes in one subnet before expanding them.

Hands-On Lab

Prerequisites: an AWS account, AWS CLI credentials that can read and modify EC2 networking resources, one test VPC, two test security groups, and a nonproduction subnet. Record original rule IDs, NACL entries, and route table targets before changing anything.

  1. Create or identify a web security group and an app security group in the same VPC.
  2. Add an inbound app rule allowing TCP 8080 from the web security group only.
  3. Launch or reuse two test instances: one with the web group and one without it. Attempt TCP 8080 to the app target from both sources.
  4. Enable VPC Flow Logs for the test subnet or ENI if they are not already enabled.
  5. Add a custom NACL deny for 203.0.113.0/24 on TCP 22 before any broader allow.
  6. Verify normal test traffic still works and the deny rule has the lower rule number.
  7. If a sandbox inspection endpoint exists, change only the test subnet default route to the inspection endpoint and confirm firewall logs receive records.

Verification: the app rule should show a source group reference, not a subnet CIDR. The source with the web group should connect to TCP 8080; the unrelated source should fail. Flow logs should match the expected decision. The NACL deny should appear before the allow. For the inspection step, verify both application behavior and a firewall log record; one without the other is incomplete evidence.

Cleanup: remove temporary ingress rules, delete unused test groups, restore original NACL entries, disable temporary flow logs if created only for this lab, and restore the original route table target if you tested inspection. Confirm cleanup by describing the security groups, NACL, and route table again.

Assessment Exercises

  1. A database subnet uses a custom NACL. Clients connect, but queries hang. Which inbound and outbound NACL directions would you inspect first, and why?
  2. An instance has two security groups. One allows SSH from 10.0.0.0/8; the other has no SSH rule. Is SSH allowed from 10.1.2.3? Explain the combined evaluation model.
  3. You need to block one malicious external CIDR immediately across every instance in a public subnet. Would you choose a security group change, a NACL rule, or an inspection policy? Name one risk.
  4. A team says traffic is protected by Network Firewall, but there are no firewall logs. What route-table evidence would you gather before editing policy?
  5. Design a three-tier VPC access model using security group references. Identify where, if anywhere, a NACL deny adds value.

Summary

Security groups protect ENIs with stateful allow rules. NACLs protect subnets with ordered stateless allow and deny rules. Traffic inspection protects routed paths by forcing packets through a firewall or appliance that can evaluate more than ports and CIDRs. Strong AWS network designs use each layer for the decision it is best suited to make, then verify behavior with routes, rule descriptions, reachability tests, flow logs, and firewall logs.