Debugging a Failing Container

A failing container is usually not a mysterious Docker problem; it is a normal process that exited, cannot reach a file, has the wrong runtime settings, or is running but not reachable the way you expect. Docker gives you several views of that failure: the container list, captured logs, structured metadata, port mappings, mounts, and an optional shell inside containers that are still running. Learning the order to use those tools saves time and prevents random rebuilds.

Overview: How Container Debugging Works

A Docker image is a read-only template made from stacked filesystem layers and configuration metadata. A container is an instance of that image with a thin writable layer, runtime configuration, mounts, network attachments, and one main process. When that main process exits, the container stops. Docker does not keep a container alive just because the image exists, a port is exposed, or a background service was expected to start.

That process model is the center of container debugging. If a container starts and immediately disappears from docker ps, it may still exist in the stopped container list shown by docker ps -a. Its captured stdout and stderr are usually available through docker logs. Its exit code, start time, finish time, restart count, mount list, command, environment, and port bindings are available through docker inspect. If the container is still running but the application misbehaves, docker exec can run a diagnostic command inside its namespaces and filesystem.

The Docker CLI is the client. It sends requests to the Docker daemon, which manages containers, images, volumes, networks, and logs. On Linux, the daemon starts containers using kernel features such as namespaces, cgroups, and a union filesystem. On Docker Desktop for macOS and Windows, Linux containers run inside Docker’s managed Linux VM, but the debugging commands from your host terminal work the same way.

Good debugging separates image problems from container runtime problems. A broken image may have a bad default command, missing executable, wrong file permissions, or a dependency that was never copied into the image. A broken runtime may pass the wrong environment variable, forget a volume, publish no port, run as a user that cannot read a mounted file, or override the image command incorrectly. Rebuilding the image only helps the first category. Inspecting the container helps you see the second.

Syntax

docker ps -a --filter "name=CONTAINER"
docker logs [OPTIONS] CONTAINER
docker inspect CONTAINER
docker inspect --format '{{.State.Status}} {{.State.ExitCode}}' CONTAINER
docker exec [OPTIONS] CONTAINER COMMAND
Command Use it for
docker ps -a Finding containers that exited and no longer appear in the default running-only list.
docker logs CONTAINER Reading output captured from the container’s main process on stdout and stderr.
docker logs --tail N Showing only recent lines when a container produced a lot of output.
docker inspect CONTAINER Viewing structured metadata: state, exit code, command, environment, mounts, networks, ports, and restart count.
docker inspect --format TEMPLATE Printing one or two exact fields from inspect output. Quote the template so the shell passes it unchanged.
docker exec CONTAINER COMMAND Running a diagnostic command inside a container that is currently running.
docker port CONTAINER Listing published host ports for a container. An image’s EXPOSE metadata is not the same as a published port.

A practical first pass is: find the container, read its logs, inspect its state, compare the command and environment with what you intended, then check ports and mounts. Use docker exec only after you know the container is actually running.

Examples

Example 1: Debug a Container That Exits Immediately

docker rm -f crash-demo 2>/dev/null || true
docker run --name crash-demo alpine:3.20 sh -c "echo booting; echo missing config >&2; exit 42"
docker ps -a --filter "name=crash-demo"
docker logs crash-demo
docker inspect --format 'status={{.State.Status}} exit={{.State.ExitCode}} error={{.State.Error}}' crash-demo

Output:

booting
missing config
CONTAINER ID   IMAGE         COMMAND                  CREATED         STATUS                     PORTS     NAMES
7f1b2d8c4c11   alpine:3.20   "sh -c 'echo boot...'"   3 seconds ago   Exited (42) 2 seconds ago             crash-demo
booting
missing config
status=exited exit=42 error=

The container is not running because the command inside it exited with status code 42. The first two output lines appear because this example runs in the foreground; docker logs then shows the same captured lines after the container has stopped. docker inspect confirms the lifecycle state and exit code. In a real application, the log message might say a config file is missing, a database is unreachable, or a required environment variable was empty.

Example 2: Debug a Running Container That Is Not Reachable

docker rm -f debug-nginx 2>/dev/null || true
docker run -d --name debug-nginx nginx:1.27-alpine
docker ps --filter "name=debug-nginx"
docker port debug-nginx
docker inspect --format '{{json .NetworkSettings.Ports}}' debug-nginx

Output:

8e7a216f4bbad6f7a6e7c1b3c0a8d2e9d0f6a111222333444555666777888999
CONTAINER ID   IMAGE               COMMAND                  STATUS         PORTS     NAMES
8e7a216f4bba   nginx:1.27-alpine   "/docker-entrypoint...."   Up 4 seconds   80/tcp    debug-nginx
{"80/tcp":null}

Nginx is running, but no host port is published. The PORTS column shows 80/tcp without a host mapping, and inspect shows null for the mapping. This is a common source of confusion: EXPOSE in an image is documentation and metadata only. It does not bind a host port. Recreate the container with -p when you need host access:

docker rm -f debug-nginx
docker run -d --name debug-nginx -p 8080:80 nginx:1.27-alpine
docker port debug-nginx 80

Output:

debug-nginx
4d2a3f1c9e8b1234567890abcdef1234567890abcdef1234567890abcdef1234
0.0.0.0:8080

The replacement container has a real host-to-container port binding. Visiting http://localhost:8080 now reaches container port 80. Removing this container does not remove the nginx:1.27-alpine image; the image is the reusable read-only template, while the container is the specific runtime instance.

Example 3: Inspect a Still-Running Container from the Inside

docker rm -f debug-loop 2>/dev/null || true
docker run -d --name debug-loop alpine:3.20 sh -c "while true; do sleep 60; done"
docker exec debug-loop sh -c "pwd; id; ls -la /"
docker inspect --format 'pid={{.State.Pid}} restart={{.RestartCount}} command={{json .Config.Cmd}}' debug-loop

Output:

2c5d5d0552c2a111222333444555666777888999aaaabbbbccccddddeeeeffff0000
/
uid=0(root) gid=0(root) groups=0(root)
total 64
drwxr-xr-x    1 root     root          4096 Aug  3 12:00 .
drwxr-xr-x    1 root     root          4096 Aug  3 12:00 ..
pid=23175 restart=0 command=["sh","-c","while true; do sleep 60; done"]

docker exec starts an additional process inside the existing container. It is useful for checking paths, users, DNS, installed tools, and mounted files, but it only works while the container is running. The inspect line shows the host-side process ID, restart count, and configured command. For production images that intentionally omit shells and package managers, you may need to add diagnostics to a development image or reproduce the failure with a debug-friendly image.

How It Works Step by Step

  1. You start a container with docker run. Docker resolves the image, creates a thin writable layer, applies runtime options, attaches networks and mounts, then starts the configured process.
  2. The process writes to stdout and stderr. Docker captures those streams through the configured logging driver, commonly the readable json-file or local driver.
  3. If the main process exits, Docker records the exit code and timestamps in the container state. The container moves to exited unless a restart policy starts it again.
  4. docker ps -a asks the daemon for both running and stopped containers. This is why it finds failures that plain docker ps hides.
  5. docker logs asks the daemon for captured output. It does not read arbitrary files inside the container.
  6. docker inspect returns metadata stored by Docker: command, arguments, environment, mounts, port bindings, networks, health status, restart count, and state.
  7. docker exec asks the daemon to create another process inside an existing running container’s namespaces. It does not restart a stopped container.

This workflow is mostly read-only. The commands that change state are the setup and fix commands such as docker run, docker rm -f, and recreating a container with corrected options. Logs and inspect are safe to run repeatedly, although their output can reveal sensitive environment values and host paths.

Common Mistakes

Only Checking Running Containers

docker ps

This is not wrong, but it hides stopped containers. When a container fails during startup, use the all-containers view and then read the logs:

docker ps -a --filter "name=crash-demo"
docker logs crash-demo

Trying to Exec into a Stopped Container

docker exec -it crash-demo sh

This fails because docker exec needs a running target container. For startup failures, read logs and inspect state first. If you need an interactive reproduction, run a new container from the same image with a diagnostic command:

docker run --rm -it alpine:3.20 sh

Assuming EXPOSE Publishes a Port

docker run -d --name web-no-port nginx:1.27-alpine

The Nginx image documents port 80, but that metadata does not publish anything to the host. Fix it with an explicit port mapping:

docker run -d --name web-with-port -p 8080:80 nginx:1.27-alpine

Hiding the Real Command with an Override

docker run --name wrong-command nginx:1.27-alpine echo hello

This starts the image with echo hello instead of the normal Nginx command, so the container exits after printing one line. If a known-good image exits unexpectedly, inspect .Config.Cmd, .Path, and .Args to confirm the command you actually ran.

Sharing Full Inspect Output with Secrets

docker inspect may include environment variables, labels, command arguments, and mounted host paths. Do not paste full inspect output into public tickets or chat without checking it. Secrets should come from Docker secrets, mounted files, or an orchestrator secret store. Do not bake secrets into image layers; deleting a file in a later layer does not remove it from earlier layers.

Best Practices

  • Start with docker ps -a, not only docker ps, when a container appears to vanish.
  • Read docker logs --tail 100 CONTAINER before rebuilding. The application often tells you exactly what failed.
  • Use docker inspect --format for exact state, exit code, restart count, command, mounts, and port bindings.
  • Use specific image tags such as alpine:3.20 and nginx:1.27-alpine when reproducing a failure. latest is a moving target.
  • Remember that container data written only to the thin writable layer is disposable. Use named volumes for persistent data and inspect mounts before removing containers.
  • Prefer logging to stdout and stderr so docker logs can show failures.
  • Check whether a restart policy is hiding a crash loop. A repeatedly restarting container may be visible as Restarting rather than simply Exited.
  • Use docker exec for live inspection, not as the first tool for a stopped container.
  • Keep production images small, but maintain a repeatable debug path for cases where a minimal image has no shell or network tools.

Practice Exercises

  1. Create a container from alpine:3.20 that prints an error message and exits with code 7. Find it with docker ps -a, then use docker inspect --format to print only its status and exit code.
  2. Run nginx:1.27-alpine without -p. Confirm that it is running but has no host port binding, then recreate it so localhost:8081 reaches container port 80.
  3. Start a long-running alpine:3.20 container and use docker exec to check the current user and list the root directory. Expected end state: you can explain which command ran in the original container and which command was the extra exec process.

Summary

  • A container stops when its main process exits; the image can still be perfectly reusable.
  • docker ps -a shows stopped containers that plain docker ps hides.
  • docker logs reads captured console output, which is usually the fastest clue for startup failures.
  • docker inspect shows exit code, state, command, environment, mounts, networks, restart count, and port bindings.
  • EXPOSE is metadata only. Use -p or Compose ports: to publish a port.
  • docker exec is useful for live containers, but it cannot enter a container that has already stopped.
  • Debug runtime options separately from image contents so you do not rebuild when the real problem is a missing environment variable, mount, port, or command.