Elastic Load Balancing and Auto Scaling

Elastic Load Balancing and Auto Scaling are the AWS mechanisms that let a web tier keep one stable entry point while the fleet behind it changes. The load balancer receives client connections, selects healthy registered targets, and forwards each request. Auto Scaling watches capacity signals and changes the number of instances in an Auto Scaling group. Together they solve a common cloud engineering problem: traffic should keep flowing when an instance fails, and capacity should rise or fall without an operator manually creating servers.

In this compute and scaling section, the important outcome is not just knowing two service names. You should be able to design an Application Load Balancer in front of an Auto Scaling group, explain why an instance can be running but unavailable to users, choose a scaling signal that represents demand, and troubleshoot when traffic or capacity behaves unexpectedly.

How the Mechanism Works

Elastic Load Balancing is a regional service with load balancer nodes placed in enabled Availability Zones. For HTTP and HTTPS applications, an Application Load Balancer, or ALB, evaluates a listener rule, chooses a target group, then chooses a target inside that group. The listener defines the frontend protocol and port, such as HTTPS on 443. Rules can match host headers, paths, methods, headers, or query strings. The target group defines the backend protocol, port, target type, health check, and stickiness behavior.

A target can be an EC2 instance, IP address, Lambda function, or another supported target type depending on load balancer family. In the classic web fleet pattern, an Auto Scaling group launches EC2 instances across at least two subnets, registers them with an ALB target group, and deregisters them when they terminate. Health is evaluated at more than one layer: EC2 status checks tell whether the virtual machine is alive, Auto Scaling health tells whether the group should replace an instance, and ALB target health tells whether the application endpoint is responding correctly.

Auto Scaling uses a launch template or launch configuration to know what to create. The group holds minimum, maximum, and desired capacity. Desired capacity is the current target number of instances. Scaling policies adjust desired capacity. Target tracking tries to keep a metric near a value, step scaling changes capacity by configured increments when alarms breach thresholds, and scheduled scaling changes capacity at known times. Auto Scaling then launches or terminates instances while honoring subnet placement, cooldown or warmup behavior, health check grace periods, and termination policies.

Configuration Anatomy

A working design has four linked parts. The ALB security group permits client traffic, usually HTTP or HTTPS. The instance security group permits traffic from the ALB security group, not from the public internet. The target group health check points to a lightweight application path that returns success only when the process can serve real traffic. The Auto Scaling group references the target group so new instances are automatically registered after launch.

{
  "Type": "AWS::ElasticLoadBalancingV2::TargetGroup",
  "Properties": {
    "Protocol": "HTTP",
    "Port": 80,
    "TargetType": "instance",
    "HealthCheckPath": "/health",
    "HealthyThresholdCount": 2,
    "UnhealthyThresholdCount": 2,
    "Matcher": { "HttpCode": "200" }
  }
}

This target group fragment says that targets receive HTTP on port 80 and are considered healthy only after two successful checks to /health. A deterministic expected behavior is that an instance returning HTTP 200 twice moves toward healthy; an instance returning HTTP 500 twice moves toward unhealthy. The matcher should be narrow enough to reject broken application states but not so narrow that a harmless redirect or authentication challenge marks every instance bad.

Progressive Example 1: One Listener, One Target Group

Start with a simple public ALB listening on HTTP 80 and forwarding all traffic to one target group. A request arrives at the ALB DNS name, the listener default action forwards it, and the target group chooses a healthy instance. If two targets are healthy, requests are distributed across them according to the load balancer algorithm and connection behavior; you should not assume perfect alternating order for every request.

The expected user behavior is stable: clients use the ALB DNS name and do not know which instance answered. The expected operator behavior is also stable: stopping one instance should eventually remove it from target rotation after failed health checks, while requests continue to the remaining healthy instance. This example teaches the core separation between frontend endpoint and backend fleet membership.

Progressive Example 2: Add Auto Scaling

Next attach the target group to an Auto Scaling group with minimum capacity 2, desired capacity 2, and maximum capacity 4. The launch template installs the application and starts the web process. When an instance fails an ELB health check after the grace period, Auto Scaling can terminate and replace it. The ALB stops sending traffic to the unhealthy target, and the replacement instance registers when it launches.

{
  "TargetValue": 55.0,
  "PredefinedMetricSpecification": {
    "PredefinedMetricType": "ASGAverageCPUUtilization"
  },
  "DisableScaleIn": false
}

This target tracking policy asks Auto Scaling to keep average CPU utilization near 55 percent. If sustained CPU rises above the target, desired capacity increases within the group’s maximum. If CPU stays below the target and scale-in is enabled, desired capacity can decrease within the group’s minimum. The deterministic part is the boundary: capacity will not go below minimum or above maximum. The exact timing depends on CloudWatch metric periods, instance warmup, current alarms, and AWS control plane decisions.

Progressive Example 3: Observe Target Health During Change

The third example focuses on verification. During a deployment or scaling event, an instance may be initial, healthy, unhealthy, draining, or unused. These states explain whether the load balancer can send traffic to the target.

aws elbv2 describe-target-health \
  --target-group-arn "$TARGET_GROUP_ARN" \
  --query 'TargetHealthDescriptions[].{Instance:Target.Id,State:TargetHealth.State,Reason:TargetHealth.Reason}' \
  --output table

Expected output is a table with one row per target, including the instance id, state, and reason. A healthy target shows healthy. A new target often starts as initial. A target being removed after scale-in shows draining while existing requests finish during the deregistration delay. This view is more useful than only checking whether EC2 instances are running because it reports the load balancer’s application-level decision.

Design Choices and Trade-offs

Choose ALB for HTTP and HTTPS routing, host-based or path-based rules, WebSocket support, and integration with AWS WAF. Choose Network Load Balancer when you need TCP, UDP, TLS pass-through, very low latency, static IP addresses, or client IP preservation at layer 4. Use Gateway Load Balancer for inline appliances such as firewalls. The load balancer family changes what health checks mean, what routing features exist, and how client identity reaches the application.

For Auto Scaling signals, CPU is easy but not always correct. A request-heavy API might saturate on latency, queue depth, database connections, or memory before CPU looks high. Target tracking on Application Load Balancer request count per target is often better for stateless HTTP services because it describes work per instance. Scheduled scaling works well for predictable business peaks, but it should be paired with dynamic scaling for unexpected demand.

Scale-in is the risky direction. Terminating an instance with in-flight requests can create user errors unless the target group deregistration delay and application shutdown behavior are coordinated. Stateless instances are easiest to scale because any target can answer any request. Sticky sessions can reduce application changes, but they can also create uneven target load and make failure more visible to a subset of users.

Failure Modes and Troubleshooting

Symptom: every target is unhealthy. Common causes are a wrong health check path, an application bound to localhost only, a security group that blocks ALB-to-instance traffic, or a startup script that did not finish. Diagnose by checking target health reason codes, instance system logs, application logs, and security group rules. Correct the path, open the instance port only from the ALB security group, or fix bootstrapping so the service listens on the expected port.

Symptom: the ALB returns 503. For an ALB, this often means no healthy targets are available for the chosen rule. Confirm which listener rule matched, inspect the target group, and compare the health check protocol, port, and matcher with the application. Correction is to restore at least one healthy target or route the rule to a valid target group.

Symptom: capacity does not scale out during load. Causes include maximum capacity already reached, a metric that does not represent bottleneck demand, missing CloudWatch data, instance warmup delaying further action, or an alarm not breaching long enough. Diagnose desired, min, and max capacity; review scaling activities; inspect the policy metric; and compare application latency with the scaling metric. Correct by raising maximum capacity, changing the metric, reducing unrealistic warmup, or adding scheduled capacity before known peaks.

Symptom: capacity scales in and then errors spike. The group may be terminating too aggressively, sticky sessions may be concentrating users, or the deregistration delay may be shorter than request duration. Check scaling activity history, target draining states, request latency percentiles, and application shutdown logs. Correct by increasing minimum capacity, tuning scale-in cooldown or warmup, lengthening deregistration delay, and making the application stop accepting new work before process exit.

Security, Performance, and Reliability

Security starts with network direction. Clients reach the ALB; the ALB reaches instances; instances do not need public inbound access. Use HTTPS listeners with managed certificates for external traffic, restrict administrative access through separate channels, and log ALB access records when request investigation is required. If using host or path routing, remember that listener rules are routing controls, not authorization controls; the application still enforces user permissions.

Performance depends on healthy distribution and realistic limits. Cross-zone load balancing can improve balance when Availability Zones have uneven capacity, but it may affect data transfer cost depending on service details. Health checks should be frequent enough to remove failed targets promptly, but not so aggressive that a short dependency pause ejects the whole fleet. Reliability improves when the Auto Scaling group spans multiple Availability Zones and minimum capacity leaves enough targets after one zone or instance failure.

Hands-on Lab

Prerequisites: an AWS account, AWS CLI configured for a sandbox account, a VPC with two public subnets in different Availability Zones, permission to create EC2, Elastic Load Balancing, Auto Scaling, IAM instance profiles if needed, and a recent Linux AMI that can run a basic web server. Use a lab naming prefix so cleanup is unambiguous.

set -euo pipefail

export AWS_REGION="us-east-1"
export NAME="course-elb-asg-lab"
export VPC_ID="vpc-xxxxxxxx"
export SUBNET_A="subnet-aaaaaaaa"
export SUBNET_B="subnet-bbbbbbbb"
export AMI_ID="ami-xxxxxxxxxxxxxxxxx"
export KEY_NAME="optional-existing-key"

aws ec2 create-security-group \
  --group-name "$NAME-alb" \
  --description "ALB HTTP access for lab" \
  --vpc-id "$VPC_ID"

aws ec2 authorize-security-group-ingress \
  --group-name "$NAME-alb" \
  --protocol tcp \
  --port 80 \
  --cidr 0.0.0.0/0

aws autoscaling describe-auto-scaling-groups \
  --auto-scaling-group-names "$NAME-asg" \
  --query 'AutoScalingGroups[0].{Desired:DesiredCapacity,Min:MinSize,Max:MaxSize,Instances:length(Instances)}' \
  --output json

Steps: create separate security groups for the ALB and instances; create a launch template that installs a web server and exposes /health; create a target group with HTTP health checks; create an ALB in two subnets; create a listener forwarding to the target group; create an Auto Scaling group using the launch template and target group; add a target tracking policy; then open the ALB DNS name in a browser.

Verification: run describe-target-health until both targets are healthy, request the ALB DNS name several times, terminate one instance from the Auto Scaling group, and confirm that desired capacity returns to two and the ALB continues answering. Also review Auto Scaling activity history to see the replacement launch.

Cleanup: delete the Auto Scaling group after setting desired and minimum capacity to zero, delete the scaling policy, listener, load balancer, target group, launch template, and security groups. Verify no lab EC2 instances remain. Roll back by returning DNS records to the previous endpoint before deleting a load balancer that receives real traffic.

Assessment Exercises

  1. Your ALB shows 503 responses while two EC2 instances are running. List the checks that distinguish compute health from target health, and explain the most likely correction.
  2. A service scales out on CPU but latency climbs while CPU remains low. Propose a better scaling metric and explain what evidence would prove it represents demand.
  3. Design an ALB and Auto Scaling group for a stateless API across two Availability Zones. Specify minimum, desired, and maximum capacity and justify your choices for one Availability Zone failure.
  4. During scale-in, users with long requests receive errors. Identify the load balancer and application settings you would inspect before changing capacity limits.
  5. Compare path-based routing on one ALB with separate ALBs per service. Explain one cost advantage and one isolation disadvantage.

Summary

Elastic Load Balancing provides a durable frontend and target health decision. Auto Scaling provides controlled fleet size and replacement behavior. The useful design skill is connecting them: health checks must describe real readiness, scaling policies must follow genuine demand, and termination must respect in-flight work. When those pieces align, the compute tier can absorb routine failure and changing traffic without exposing individual instance lifecycles to users.