Containerize Models and Manage Dependencies

Containerizing a model means packaging the serving code, model artifact, runtime libraries, operating-system packages, and startup command into an image that can run the same way on a laptop, in CI, and in a serving platform. Dependency management is the companion discipline: deciding exactly which Python wheels, native libraries, CUDA builds, tokenizer files, and configuration files are allowed into that image. The outcome is a repeatable serving unit whose behavior can be rebuilt, inspected, scanned, rolled back, and connected to model lineage.

In this MLOps course, this lesson sits in the serving section because a trained artifact is not deployable by itself. A model may depend on a specific scikit-learn release, a compiled BLAS library, a normalization file, or a tokenizer vocabulary. If those dependencies drift between training and serving, predictions can change without any model registry transition. Good containers make the runtime explicit and small enough to reason about.

What the Image Contains

A model-serving image normally has four layers of concern. The base image provides the operating system and language runtime, such as a slim Python image or a GPU image with compatible drivers. System dependencies add shared libraries needed by frameworks, image processing, or acceleration. Application dependencies install Python packages from a lock file. The final application layer copies the serving module, model artifact, schemas, and startup command.

Docker and similar builders create each instruction as a cached filesystem layer. If requirements.txt is copied and installed before the model file, changing only the model does not force every dependency to reinstall. That is why many model Dockerfiles copy dependency manifests first, install them, and copy larger application assets later. The ordering affects build time, cache correctness, and whether a dependency refresh is hidden behind a stale layer.

The running container is a process with an isolated filesystem view, environment variables, network namespace, and resource limits. It is not a virtual machine. The image digest identifies the filesystem content, while tags such as latest are movable names. For deployment evidence, promote immutable digests and record the model artifact checksum separately, especially when images are rebuilt by automation.

Dependency Anatomy

Dependencies should be split by role. Build tools such as compilers and header files are needed to produce wheels but should not remain in the final runtime image. Runtime packages are the minimum needed to import the serving application and execute inference. Model data files are dependencies too: a .joblib model, a vocabulary, a label map, and a feature schema all influence predictions and must move with the service or be fetched by a versioned startup step.

Pinning dependencies constrains versions so rebuilds are deterministic. A range such as scikit-learn>=1.4 allows a later compatible-looking release to change serialization behavior or numerical output. A lock file produced by pip-tools, Poetry, uv, or Conda records the full transitive set. Hash checking adds another guard by rejecting a package whose bytes differ from the approved artifact.

Example 1: Minimal Serving Image

This Dockerfile demonstrates a small CPU-serving image. It chooses a Python runtime, creates a fixed working directory, installs dependencies before copying the frequently changing model and source files, exposes the HTTP port, and starts Uvicorn. The deterministic behavior is that a container started from this image tries to import serve:app and listen on port 8080. If the module or ASGI app name is wrong, startup fails immediately.

FROM python:3.11-slim
WORKDIR /app
ENV PYTHONDONTWRITEBYTECODE=1 \
    PYTHONUNBUFFERED=1 \
    PIP_NO_CACHE_DIR=1
COPY requirements.txt .
RUN pip install --require-hashes -r requirements.txt
COPY model.joblib serve.py ./
EXPOSE 8080
CMD ["uvicorn", "serve:app", "--host", "0.0.0.0", "--port", "8080"]

The important design detail is instruction order. Rebuilding after a model refresh should reuse the dependency layer; rebuilding after a dependency change should invalidate it. For a larger project, use a multi-stage build so compilers and package caches are discarded before the runtime stage. Add a .dockerignore file so notebooks, local data, virtual environments, and credentials are not sent to the build context.

Example 2: Locked Python Inputs

The next fragment shows the shape of a hash-checked requirements file. The placeholder hashes must be replaced with real hashes generated by the locking tool. When pip install --require-hashes runs, every package, including transitive dependencies, must be pinned and hashed. The expected behavior is strict refusal: if a version is missing a hash, or if downloaded bytes do not match, the build stops before producing an image.

fastapi==0.110.0 \
    --hash=sha256:replace_with_hash_from_pip_compile
uvicorn[standard]==0.27.1 \
    --hash=sha256:replace_with_hash_from_pip_compile
joblib==1.3.2 \
    --hash=sha256:replace_with_hash_from_pip_compile
scikit-learn==1.4.1.post1 \
    --hash=sha256:replace_with_hash_from_pip_compile

Hash checking is stricter than many teams start with, but it is useful for model serving because dependency bytes affect numerical libraries and deserialization. The trade-off is maintenance overhead: every deliberate upgrade requires regenerating and reviewing the lock file. That review connects package changes to model evaluation, because a framework upgrade may require rerunning representative prediction tests even if the model artifact is unchanged.

Example 3: Model Contract in Code

This Python example models the serving contract without starting a web server. The bundle has a model name, version, labels, and a prediction method that validates feature shape before computing a score. It prints deterministic labels so the contract can be checked in CI. The first input has score 0.26 and prints keep; the second has score 0.81 and prints review.

from dataclasses import dataclass
from typing import Sequence

@dataclass(frozen=True)
class ModelBundle:
    name: str
    version: str
    labels: Sequence[str]

    def predict(self, features: Sequence[float]) -> str:
        if len(features) != 2:
            raise ValueError("expected exactly two numeric features")
        score = (features[0] * 0.7) + (features[1] * 0.3)
        return self.labels[1] if score >= 0.5 else self.labels[0]

bundle = ModelBundle("churn-demo", "2026-09-06", ("keep", "review"))
print(bundle.predict((0.2, 0.4)))
print(bundle.predict((0.9, 0.6)))

The same pattern applies to a real FastAPI or Flask service. Load the artifact once during startup, validate request shape before calling the model, and return a response that includes a model version or image digest. Avoid loading the model on every request; that turns latency into artifact I/O and makes failures intermittent under concurrency.

Example 4: Runtime Limits

This Compose fragment adds operational boundaries around the image. It sets the model path through the environment, makes the filesystem read-only, provides temporary space through tmpfs, and caps memory. The expected behavior is that the service can write transient files only under /tmp. Attempts to mutate application files fail, which helps detect code that tries to cache artifacts in the image directory.

services:
  model-api:
    build: .
    ports:
      - "8080:8080"
    environment:
      MODEL_PATH: /app/model.joblib
    read_only: true
    tmpfs:
      - /tmp
    mem_limit: 512m

These limits are not decoration. Many ML libraries create temporary files, memory maps, or thread pools. If the container is read-only without /tmp, startup may fail. If no memory limit exists, a batch request or large tokenizer can evict neighboring workloads. The right boundary is discovered by measuring realistic request sizes, cold start, warm latency, and peak memory during load.

Design Choices and Trade-offs

The first choice is whether the model artifact is baked into the image or downloaded at startup. Baking it in gives a single immutable deployable unit and simpler rollback. It can make images large, and every model promotion requires a new image build. Downloading at startup keeps images smaller and lets one image serve many versions, but startup now depends on object storage, credentials, checksum validation, and a clear rule for unavailable artifacts.

The second choice is base image size versus compatibility. Slim images reduce attack surface and transfer time, but missing native libraries can make scientific Python packages hard to install. Framework-provided images are convenient for TensorFlow, PyTorch, or GPU serving, but they can include many packages your service does not need. Use them deliberately when driver and accelerator compatibility matter.

The third choice is generic server versus specialized serving runtime. A FastAPI wrapper is flexible and easy to test for tabular models. Triton, TorchServe, or framework-specific runtimes can provide batching, GPU scheduling, model repositories, and protocol support. They also add configuration surface. Choose them when their mechanisms solve a measured serving problem.

Failure Modes and Troubleshooting

Symptom: the container builds but crashes with an import error at startup. Cause: the dependency was present in a developer virtual environment but missing from the lock file or final image stage. Diagnose: run docker run --rm IMAGE python -c "import package_name" and inspect the package list inside the container. Correct: add the dependency to the manifest, regenerate the lock, and rebuild without relying on local site packages.

Symptom: predictions differ between validation and served responses for the same record. Cause: dependency skew, missing preprocessing assets, or a feature schema mismatch. Diagnose: run a golden-record test inside the built image and compare output with stored evaluation evidence. Print model version, package versions, and schema version in the test log. Correct: package preprocessing assets with the model, pin framework versions, and reject requests whose feature names or order do not match the schema.

Symptom: the image is slow to build and huge to pull. Cause: the build context includes datasets, notebooks, cached wheels, or unused frameworks. Diagnose: inspect context size, image history, and layer sizes. Correct: add .dockerignore, split build and runtime stages, remove package caches, and avoid training-only dependencies in the serving image.

Symptom: the service passes health checks but times out on first real request. Cause: the health check does not force model loading, or lazy loading occurs under request traffic. Diagnose: compare cold-start logs with the first inference request and check memory during artifact load. Correct: load and validate the model during startup, make readiness fail until loading succeeds, and size startup timeouts accordingly.

Security, Performance, and Reliability

Model containers often deserialize binary artifacts, which can execute code depending on the format. Only load artifacts produced by trusted pipelines, verify checksums, and avoid accepting user-supplied model files in the serving process. Run as a non-root user when the base image and platform permit it, keep credentials out of the image, and scan both OS and language packages.

Performance tuning starts with the runtime. Set thread counts for numerical libraries if default oversubscription hurts latency. Separate readiness from liveness: readiness should reflect whether the model and dependencies are loaded, while liveness should detect a stuck process. Reliability improves when the image digest, model checksum, dependency lock, and evaluation run are stored together in deployment metadata.

Hands-on Lab

Prerequisites: Docker or a compatible container builder, Python packaging tools, and a small model artifact or placeholder file named model.joblib. Create a clean directory with serve.py, requirements.txt, Dockerfile, and optionally compose.yaml. For a real service, replace placeholder hashes with a generated locked requirements file before building.

  1. Write a serving module that loads the model at startup and exposes a health route plus one prediction route.
  2. Add a golden-record test that sends a fixed input and asserts the exact expected class or score range.
  3. Create the Dockerfile with dependencies installed before application files are copied.
  4. Build the image with an explicit tag and record the produced image digest.
  5. Run the container locally with a memory limit and required environment variables.
  6. Call the health route, then call the prediction route with the golden record.
  7. Stop the container and remove the local test image if it was only for the lab.

Verification is complete when the image builds from a clean checkout, the service starts without using your host virtual environment, the golden record returns the expected result, and logs identify the model version. Cleanup is docker rm -f for the local container and docker image rm for the lab tag. In a shared registry, rollback means redeploying the previous approved image digest, not retagging a mutable name.

Assessment Exercises

  1. A team wants to download the model from object storage at startup instead of baking it into the image. List two reliability checks they must add and explain what failure each one catches.
  2. Given a service that works on a laptop but fails in the container with ModuleNotFoundError, describe the diagnostic commands you would run and what evidence would prove the lock file is incomplete.
  3. Design a golden-record test for a model whose output is probabilistic. What should be exact, and what can be expressed as an allowed range?
  4. Compare a slim Python base image with a framework GPU image for a PyTorch model. Identify one benefit and one cost of each.
  5. Explain why promoting an image tag alone is weaker than promoting an image digest plus a model checksum.

Summary

Containerizing models makes serving reproducible when the image captures the runtime, application code, artifact, schema, and startup behavior deliberately. Dependency management keeps that image from drifting underneath the model. The practical standard is simple: build from a lock, package or verify every model asset, test golden predictions inside the image, constrain runtime resources, and deploy immutable digests tied back to evaluation evidence.