EC2 Instances, Images, Storage, and User Data

Amazon EC2 gives you virtual machines that you can size, boot, configure, stop, replace, and connect to other AWS services. In this lesson, the useful outcome is concrete: you should be able to launch an instance from an Amazon Machine Image, attach the right storage, bootstrap it with user data, verify what happened during boot, and diagnose the common reasons a new server does not behave as expected.

This topic sits at the compute layer of the AWS Cloud Engineering course. Earlier AWS decisions about Regions, Availability Zones, IAM, networking, and security groups all meet here. An EC2 instance is not just a server. It is the result of several launch-time choices: an image, an instance type, a subnet, security controls, storage devices, metadata settings, and optional startup instructions.

What EC2 Builds at Launch

When you launch an EC2 instance, AWS allocates compute capacity on physical hosts inside one Availability Zone and presents it as a virtual machine. The instance type defines the shape of that machine: vCPU count, memory, network performance, storage support, and processor family. The subnet places the instance in one VPC network boundary. The security group controls allowed traffic at the elastic network interface. IAM instance profiles let code on the instance call AWS APIs without storing long-lived access keys.

The boot disk usually comes from an Amazon Machine Image, or AMI. An AMI is a template containing a root volume snapshot, boot metadata, virtualization settings, architecture, and block device mappings. Public AMIs are convenient, but production teams commonly use golden AMIs or a repeatable build process so installed packages, agents, hardening, and baseline configuration are reviewed before launch.

Storage is separate from compute. Most EC2 root volumes use Amazon EBS, a network-attached block storage service scoped to one Availability Zone. EBS volumes persist independently from a running instance unless configured for deletion on termination. Some instance families also include instance store volumes. Instance store is physically attached to the host and can be fast, but it is temporary. Data on instance store disappears when the underlying instance is stopped, terminated, or moved.

User data is launch-time input made available to the guest operating system through the instance metadata service. On many Linux AMIs, cloud-init reads user data during first boot and runs shell commands or cloud-config directives. User data is best for small, idempotent bootstrap tasks: installing a package, writing a config file, registering a service, or fetching application artifacts. It is not a secret store and it should not be the only place important configuration knowledge exists.

AMI, Storage, and User Data Anatomy

The most important AMI fields are the AMI ID, owner, architecture, root device type, virtualization type, and block device mappings. The AMI ID is Region-specific. An AMI that exists in us-east-1 does not automatically exist in us-west-2 unless it has been copied there. The owner matters because a misleading AMI name can point to an untrusted publisher. The architecture must match the instance type, such as x86_64 or arm64.

Block device mappings describe which device names appear at launch and what backs them. For EBS, the mapping includes volume size, volume type, encryption, IOPS or throughput settings where relevant, and whether to delete the volume when the instance terminates. For a root volume, delete-on-termination is usually true for replaceable servers. For data volumes that hold durable state, it may be false, but then backup, ownership, cleanup, and recovery procedures must be explicit.

User data has size limits and is passed as opaque text. The AWS CLI can read it from a file, base64-encode it as needed for the API, and attach it during launch. In the instance, cloud-init logs are normally the first place to inspect whether user data ran. On Amazon Linux and many Ubuntu images, useful files include /var/log/cloud-init.log and /var/log/cloud-init-output.log.

Example 1: A Minimal Web Bootstrap

This first example uses user data to install a web server and create a fixed response page. The expected behavior is deterministic once the instance has booted and package installation succeeds: an HTTP request to the instance should return the text ec2 bootstrap complete.

#!/usr/bin/env bash
set -euo pipefail

if command -v dnf >/dev/null 2>&1; then
  dnf install -y nginx
elif command -v yum >/dev/null 2>&1; then
  yum install -y nginx
else
  apt-get update
  apt-get install -y nginx
fi

printf 'ec2 bootstrap complete\n' > /usr/share/nginx/html/index.html
systemctl enable nginx
systemctl restart nginx

The script detects a common package manager, installs nginx, writes the page, and starts the service. The set -euo pipefail line makes many script errors fail visibly instead of continuing silently. For repeatability, the script overwrites the page every time it runs. That matters because bootstrapping code should tolerate retry, rebuild, and replacement.

Example 2: Launch with an Explicit Root Volume

The second example shows a launch command that ties the main pieces together. It assumes you already selected a subnet, security group, key pair, AMI, and IAM instance profile. The storage choice is visible: a 20 GiB encrypted gp3 root volume that is deleted with the instance. The expected output is an instance ID from AWS when the request is accepted.

set -euo pipefail

aws ec2 run-instances \
  --image-id "$AMI_ID" \
  --instance-type t3.micro \
  --subnet-id "$SUBNET_ID" \
  --security-group-ids "$SECURITY_GROUP_ID" \
  --key-name "$KEY_NAME" \
  --iam-instance-profile Name="$INSTANCE_PROFILE_NAME" \
  --metadata-options HttpTokens=required,HttpEndpoint=enabled \
  --user-data file://user-data.sh \
  --block-device-mappings '[{"DeviceName":"/dev/xvda","Ebs":{"VolumeSize":20,"VolumeType":"gp3","Encrypted":true,"DeleteOnTermination":true}}]' \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=course-ec2-user-data},{Key=Course,Value=aws-cloud}]' \
  --query 'Instances[0].InstanceId' \
  --output text

This command also requires IMDSv2 by setting HttpTokens=required. That means software on the instance must first request a metadata token before reading instance metadata. The design reduces exposure from classes of server-side request forgery because a simple unauthenticated HTTP GET is not enough to retrieve metadata credentials.

Example 3: Inspect Metadata and User Data from the Instance

After connecting to the instance, you can query the Instance Metadata Service. The first command gets a token. The second reads the instance ID. The final command prints the exact user data text that was provided at launch, which is useful when you suspect the wrong script or an outdated file was attached.

set -euo pipefail

TOKEN=$(curl -sS -X PUT 'http://169.254.169.254/latest/api/token' \
  -H 'X-aws-ec2-metadata-token-ttl-seconds: 21600')

curl -sS -H "X-aws-ec2-metadata-token: $TOKEN" \
  'http://169.254.169.254/latest/meta-data/instance-id'

curl -sS -H "X-aws-ec2-metadata-token: $TOKEN" \
  'http://169.254.169.254/latest/user-data'

If the instance was launched with the first script, the user data output should include the nginx installation commands and the line that writes ec2 bootstrap complete. Metadata is link-local to the instance; it is not reached through the public internet, but software running on the instance can access it unless local firewalling or metadata settings prevent that.

Design Choices and Trade-Offs

Choosing an instance type is a workload decision. General purpose families are a reasonable default for small services, but CPU-bound jobs, memory-heavy caches, network appliances, and GPU workloads need different shapes. Under-sizing causes latency and throttling. Over-sizing wastes spend. Burstable instances can be cost-effective for low average CPU, but sustained CPU usage can exhaust credits or bill differently depending on configuration.

Choosing an AMI is a supply chain decision. A public AMI can speed up experimentation, but the owner, patch level, installed software, and hardening baseline must be trusted. A golden AMI reduces launch-time work and makes boot faster, but it introduces image maintenance. Heavy user data keeps images generic, but it can make launches slower and more dependent on package repositories during outages.

Choosing storage is a durability and performance decision. EBS is persistent and snapshot-friendly, so it is the normal choice for root volumes and durable data. Instance store can be useful for caches, temporary build output, and scratch space where loss is acceptable. Encryption should be enabled for EBS volumes unless there is a specific exception. For databases or stateful systems, volume performance settings, filesystem layout, backup policy, and restore testing matter more than raw volume size.

Failure Modes and Troubleshooting

Instance launches but the website is unreachable. The symptom is a running instance that does not respond on HTTP. Common causes are a security group missing inbound TCP port 80, no public route for a public test instance, nginx not running, or user data failure. Diagnose in layers: confirm instance status checks, inspect the security group, verify the subnet route table, connect with Session Manager or SSH, then run systemctl status nginx and inspect /var/log/cloud-init-output.log. Correct the network rule or script, then replace the instance or rerun the fixed configuration deliberately.

User data appears ignored. The symptom is that the instance boots but none of the bootstrap changes exist. Causes include launching with the wrong user data file, using an AMI without cloud-init support, a script syntax error, or expecting user data to rerun automatically after every reboot. Check the launch parameters, query /latest/user-data from metadata, and read cloud-init logs. Correct the script and launch a new instance. If rerun behavior is required, configure cloud-init for that intentionally or move the task into a systemd unit or configuration management tool.

Root volume fills unexpectedly. The symptom is application errors, failed package installs, or logs showing no space left on device. Causes include a too-small root volume, chatty logs, container layers, or temporary files placed on root. Diagnose with df -h, lsblk, and directory usage checks. Correct by expanding the EBS volume and filesystem, moving data to a separate volume, applying log rotation, or replacing the instance with a larger declared root volume.

Application cannot call AWS APIs. The symptom is access denied or missing credentials from code running on the instance. Causes include no IAM instance profile, an overly narrow role, blocked metadata access, or software that does not support IMDSv2. Diagnose with metadata token requests, aws sts get-caller-identity on the instance, and IAM policy simulation where appropriate. Correct the role or application configuration rather than placing access keys on disk.

Hands-On Lab

Prerequisites: an AWS account, AWS CLI configured for a sandbox account, permission to create EC2 instances, a VPC subnet, a security group allowing inbound HTTP from your own IP, an EC2 key pair or Session Manager access, and a recent Amazon Linux or Ubuntu AMI ID in your chosen Region.

Create a local user-data.sh file using the script from Example 1.
Set shell variables for AMI_ID, SUBNET_ID, SECURITY_GROUP_ID, KEY_NAME, and INSTANCE_PROFILE_NAME.
Run the launch command from Example 2 and save the returned instance ID.
Wait for both EC2 status checks to pass.
Find the public IPv4 address or DNS name if you launched in a public subnet.
Open http://PUBLIC_ADDRESS/ or run curl http://PUBLIC_ADDRESS/.
Verify that the response body is exactly ec2 bootstrap complete.
Connect to the instance and inspect metadata and user data with Example 3.
Review /var/log/cloud-init-output.log to confirm the package installation and service start path.

For cleanup, terminate the instance and confirm that the root EBS volume was deleted. If you created a temporary security group, IAM role, instance profile, or key pair for the lab, remove those resources after verifying that no other instance uses them.

set -euo pipefail

aws ec2 terminate-instances \
  --instance-ids "$INSTANCE_ID" \
  --query 'TerminatingInstances[0].CurrentState.Name' \
  --output text

aws ec2 wait instance-terminated --instance-ids "$INSTANCE_ID"

Assessment Exercises

You need ten identical web servers that start quickly during a traffic spike. Which work should move into an AMI, and which work should remain in user data? Explain the launch-time failure risks of your split.
An EC2 instance has a 200 GiB data volume with DeleteOnTermination=false. What operational problem does that solve, and what cleanup or ownership problem does it create?
A team can connect to an instance, but the application cannot call S3. List the EC2, metadata, and IAM checks you would perform before changing any policy.
Your user data installs packages from internet repositories on every launch. Describe two ways this can fail during scaling and one way to reduce the dependency.
Compare EBS and instance store for a build cache. What data loss behavior is acceptable, and what verification would prove the cache can be rebuilt?

Summary

EC2 launches are assembled from compute capacity, an AMI, network placement, permissions, storage mappings, metadata settings, and optional user data. The practical skill is understanding which choice owns which behavior. Use trusted AMIs for known boot baselines, EBS for durable block storage, instance store only for disposable data, IMDSv2 for metadata access, and user data for small, repeatable first-boot configuration. When an instance does not behave correctly, troubleshoot in order: launch parameters, network reachability, system status, cloud-init logs, storage state, and IAM identity.