Using Docker in CI/CD Pipelines

Docker is useful in CI/CD because it turns build and runtime setup into files that automation can repeat. A pipeline can build the same image on every commit, run tests against the same container environment developers use locally, push the image to a registry, and deploy an immutable artifact instead of rebuilding on the server. This lesson shows how to wire Docker into that flow without making builds slow, flaky, or hard to audit.

Overview: how Docker fits into CI/CD

A CI/CD pipeline usually has four Docker-related jobs: build an image, test it, scan or inspect it, then push the image that passed. Docker helps because an image is a read-only stack of layers with a config object and manifest. The registry stores those compressed layer blobs and manifests; deployment systems pull the exact referenced tag or digest and start containers from it. A container is only a running or stopped instance of that image plus a thin writable layer, so the image is the portable artifact you should promote through environments.

In CI, the Docker client talks to a Docker daemon or a remote builder. With BuildKit and docker buildx, the build can run with better caching, multi-platform support, secrets mounts, SBOM/provenance options, and remote cache exporters. Each Dockerfile instruction creates or reuses a cached layer. If a line changes, Docker invalidates that layer and all layers after it. That is why a good CI Dockerfile copies dependency manifests first, installs dependencies, and only then copies application source.

A strong pipeline does not build different images for staging and production. It builds once, tags the result, records the digest, scans that exact image, and deploys the same digest later. Environment-specific configuration belongs outside the image: CI variables, orchestrator secrets, Compose environment files for non-secret values, or runtime secret stores. Baking configuration or secrets into image layers creates artifacts that are hard to rotate and easy to leak.

Docker also improves integration testing. A pipeline can start databases, queues, and the app with docker compose, run tests, then remove the stack. Compose creates an isolated project network where services can reach each other by service name. Named volumes can persist data, but most CI test stacks should use disposable volumes so every run starts clean.

Syntax

docker buildx build [OPTIONS] PATH
docker compose -f FILE up [OPTIONS]
docker push IMAGE[:TAG]
docker image inspect IMAGE[:TAG]

Command or option	Meaning in CI/CD
`docker buildx build -t NAME:TAG .`	Builds an image from the current build context and assigns a registry-ready tag.
`--push`	Pushes the image directly after a successful BuildKit build.
`--load`	Loads a single-platform image into the local Docker image store, useful before local test commands.
`--cache-from type=gha`	Imports cache from GitHub Actions cache storage when that backend is available.
`--cache-to type=gha,mode=max`	Exports build cache so later CI runs can reuse unchanged layers.
`--provenance=true`	Asks BuildKit to attach metadata about how the image was built.
`--sbom=true`	Attaches software bill of materials metadata for downstream inspection.
`docker compose up --abort-on-container-exit --exit-code-from tests`	Runs a test stack and makes the pipeline exit with the test container’s status.
`docker push`	Uploads image layers and a manifest to a registry.
`docker image inspect --format`	Extracts metadata such as image ID or RepoDigests for release records.

Examples

Build and test an image in CI

FROM node:20.18.1-alpine3.20 AS deps
WORKDIR /app
COPY package*.json ./
RUN npm ci

FROM deps AS test
COPY . .
RUN npm test

FROM node:20.18.1-alpine3.20 AS runtime
WORKDIR /app
ENV NODE_ENV=production
COPY package*.json ./
RUN npm ci --omit=dev
COPY --from=test /app/dist ./dist
USER node
EXPOSE 3000
CMD ["node", "dist/server.js"]

Output:

Dockerfile saved. The test stage runs npm test, and the runtime stage contains only production dependencies plus built output. EXPOSE records port 3000 as metadata only.

This multi-stage Dockerfile gives CI separate targets to test and ship. The deps stage caches dependency installation because it copies package*.json before the rest of the source. The test stage can fail the build before any image is published. The final runtime stage starts from a pinned base image tag, installs only production dependencies, runs as the existing node user, and documents port 3000. Remember that EXPOSE does not publish a port; a runtime command still needs -p or Compose ports:.

docker buildx build --target test --load -t inventory-api:test .
docker buildx build --target runtime --load -t inventory-api:ci .

Output:

[+] Building 24.8s (13/13) FINISHED
=> naming to docker.io/library/inventory-api:test
[+] Building 9.2s (15/15) FINISHED
=> naming to docker.io/library/inventory-api:ci

The first command builds through the test target and fails if npm test fails. The second command builds the production-shaped image and loads it into the CI runner’s local image store. A real pipeline would often run a container smoke test before pushing.

Run integration tests with Compose

services:
  db:
    image: postgres:16.6-alpine3.20
    environment:
      POSTGRES_DB: app_test
      POSTGRES_USER: app
      POSTGRES_PASSWORD: changeme
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U app -d app_test"]
      interval: 5s
      timeout: 3s
      retries: 10
  tests:
    image: inventory-api:ci
    depends_on:
      db:
        condition: service_healthy
    environment:
      DATABASE_URL: postgres://app:changeme@db:5432/app_test
    command: ["npm", "run", "test:integration"]

Output:

compose.ci.yml saved. The tests service can reach PostgreSQL at hostname db on the private Compose network.

This CI-only Compose file starts a disposable PostgreSQL service and a test container from the image built earlier. The password is the obvious placeholder changeme, suitable for an isolated throwaway test database, not a production secret. The depends_on health condition avoids starting tests before PostgreSQL accepts connections.

docker compose -f compose.ci.yml up --abort-on-container-exit --exit-code-from tests
docker compose -f compose.ci.yml down --volumes

Output:

[+] Running 2/2
 ✔ Container inventory-db-1     Healthy
 ✔ Container inventory-tests-1  Exited with code 0
[+] Running 3/3
 ✔ Container inventory-tests-1  Removed
 ✔ Container inventory-db-1     Removed
 ✔ Network inventory_default    Removed

The up command returns the exit status from the tests service, so CI can pass or fail correctly. The cleanup command removes containers, the network, and anonymous or named volumes declared by the project. In CI, cleanup matters because reused runners can otherwise accumulate state that changes later test results.

Push the image that passed

name: docker-ci
on:
  push:
    branches: ["main"]
jobs:
  image:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: docker/setup-buildx-action@v3
      - uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}
      - name: Build and push
        run: |
          docker buildx build \
            --cache-from type=gha \
            --cache-to type=gha,mode=max \
            --sbom=true \
            --provenance=true \
            -t ghcr.io/acme/inventory-api:${{ github.sha }} \
            -t ghcr.io/acme/inventory-api:main \
            --push .

Output:

Buildx pushes changed layers, writes a manifest for ghcr.io/acme/inventory-api, and attaches the commit SHA tag plus the main tag.

This workflow builds on pushes to main, authenticates to GitHub Container Registry, reuses BuildKit cache, and pushes two tags. The commit SHA tag is immutable in practice because it names one source revision. The main tag is convenient for humans but mutable, so deployment records should store the digest produced by the push.

docker image inspect --format '{{index .RepoDigests 0}}' ghcr.io/acme/inventory-api:main

Output:

ghcr.io/acme/inventory-api@sha256:3f4a2b0c9e1d7a6b5c8d9012e3f4567890abcdef1234567890abcdef12345678

A digest identifies exact image content. Tags can move; digests do not. Many production systems should deploy ghcr.io/acme/inventory-api@sha256:... after the image passes tests and scanning.

How it works step by step

The runner checks out source. The Docker build context is usually the repository directory, minus files excluded by .dockerignore.
BuildKit evaluates the Dockerfile. It hashes each instruction plus relevant files. Unchanged layers are reused from local or remote cache; changed layers and all later layers are rebuilt.
The test target runs. If a RUN npm test instruction exits nonzero, the build fails and the pipeline stops before publishing.
Compose creates a temporary environment. Services get a private network, service-name DNS, and clean containers. Test status becomes the pipeline status.
The final image is pushed. Docker uploads missing layer blobs, then writes a manifest and tag in the registry.
The pipeline records what shipped. The digest, scan report, SBOM, provenance, commit SHA, and deployment environment form the release trail.

Common Mistakes

Building again during deployment

ssh deploy@example.com "cd /srv/inventory && docker build -t inventory-api:latest . && docker run inventory-api:latest"

This is wrong because production rebuilds may use a different base image, cache, build argument, or source checkout than CI. Fix it by building once in CI, pushing that image, and deploying the tested digest.

Using latest as the release artifact

docker buildx build -t ghcr.io/acme/inventory-api:latest --push .

latest is only a tag name. It is not automatically the newest, safest, or tested build. Use version, commit, or build-number tags, then record the digest. A mutable convenience tag can exist, but it should not be the only release identifier.

Copying secrets into the image

FROM alpine:3.20
COPY production.env /app/production.env
RUN rm /app/production.env

This is unsafe because the secret remains in an earlier read-only layer even though a later layer deletes the file. Use CI secret variables for registry login, BuildKit secret mounts for build-only secrets, and runtime secret injection for application secrets.

Sending the whole repository as build context

docker buildx build -t ghcr.io/acme/inventory-api:slow .
# No .dockerignore, so logs, node_modules, test reports, and local env files enter the build context

Large contexts make CI slow and can accidentally send private files to the builder. Add a .dockerignore that excludes local dependencies, generated output, VCS metadata when not needed, reports, and env files.

Best Practices

Build once, test that image, scan that image, and deploy the same digest.
Use docker buildx and BuildKit cache exporters to keep CI fast without weakening reproducibility.
Pin Dockerfile base image tags such as node:20.18.1-alpine3.20; avoid bare image names and latest in production Dockerfiles.
Structure Dockerfiles so dependency installation layers come before frequently changing source layers.
Use multi-stage builds to keep compilers, test tools, and package caches out of the runtime image.
Keep secrets out of Dockerfiles, image layers, build arguments, and logs.
Use .dockerignore aggressively to shrink build context and reduce accidental leakage.
Run containers as non-root and publish ports explicitly with -p or Compose ports: only where needed.
Clean up CI Compose stacks with docker compose down --volumes for repeatable test runs.
Store image digests, scan reports, SBOMs, and provenance with release metadata.

Practice Exercises

Create a CI plan for a Python API that builds a test target, runs integration tests against PostgreSQL with Compose, then pushes only if tests pass. Hint: make the test container’s exit code control the job.
Given a Dockerfile that runs COPY . . before installing dependencies, rewrite the order so dependency cache survives ordinary source-code edits. Expected end state: lockfiles are copied before the full source tree.
Design a tagging scheme for an image pushed from every merge to main. Include one human-friendly tag and one immutable release reference.

Summary

Docker makes CI/CD more repeatable by turning the image into the artifact that moves through test, scan, registry, and deployment steps.
BuildKit and docker buildx support cache reuse, SBOMs, provenance, and direct registry pushes.
Compose is useful in CI for disposable integration-test environments with service-name networking.
Do not rebuild on production servers, depend only on latest, or bake secrets into image layers.
The safest release process records and deploys the digest of the exact image that passed the pipeline.

Using Docker in CI/CD Pipelines

Overview: how Docker fits into CI/CD

Syntax

Examples

Build and test an image in CI

Run integration tests with Compose

Push the image that passed

How it works step by step

Common Mistakes

Building again during deployment

Using latest as the release artifact

Copying secrets into the image

Sending the whole repository as build context

Best Practices

Practice Exercises

Summary

Table of Contents