Deployment and Observability

Published by Programming Line in FastAPI. Last updated .

Deployment for FastAPI means turning an ASGI application object into a supervised network service. Observability means giving operators signals to answer three questions while traffic is flowing: is the process alive, is it ready to serve useful work, and why did a request become slow or fail?

The outcome is specific: run FastAPI with managed ASGI workers, terminate HTTPS at a proxy or platform edge, expose health endpoints for orchestration, emit machine-readable logs and metrics, and release in a way that can be rolled back. This completes the Production Engineering path by putting typed routes, dependencies, and exception handling under process managers, load balancers, probes, and dashboards.

ASGI Deployment Mechanics

A FastAPI instance is an ASGI callable. An ASGI server such as Uvicorn accepts socket connections, parses HTTP, builds an ASGI scope, and calls the application with receive and send callables. FastAPI delegates routing through Starlette, validates request data through Pydantic, runs dependencies, executes your path operation, serializes the response, and hands response events back to the server.

Production usually adds more processes. A request path is client, DNS, TLS terminator, reverse proxy or managed ingress, ASGI worker, FastAPI middleware stack, route handler, dependency clients, and response. TLS often ends before Python so certificate renewal and HTTP/2 negotiation are handled by a platform component. Python should trust forwarding headers only from that component; otherwise the application can be tricked about scheme, host, or client address.

Workers matter because one Python process has one event loop. Async endpoints can handle many waiting I/O operations, but CPU-bound work blocks the event loop until it yields. Multiple worker processes spread requests across CPU cores and isolate crashes. The trade-off is that in-memory state is per worker. A module-level counter is fine for a local metric example, but not for shared business state, rate limits, sessions, or job queues.

Configuration Anatomy

Piece FastAPI-specific role Common decision
app import path Identifies the ASGI callable, for example main:app. Keep startup side effects inside lifespan, not at import time.
worker count Controls how many event loops serve traffic. Start conservatively and measure latency, memory, and CPU saturation.
lifespan Runs startup and shutdown hooks. Mark the app ready only after required dependencies are initialized.
/livez Tells the orchestrator the process can answer HTTP. Do not check databases here or a dependency outage may cause restart loops.
/readyz Tells the load balancer whether this instance should receive traffic. Fail when required local initialization or critical dependencies are unavailable.
/metrics Exposes numeric time series for scraping. Use low-cardinality labels; never label by user id, raw path parameter, or token.

Gunicorn is often used as a process manager with Uvicorn workers when you need pre-fork worker supervision. Uvicorn alone is also common in containers where the platform restarts failed processes. The important design is that startup, shutdown, worker count, timeout, logs, and probes are explicit.

Example 1: Health Endpoints and Structured Signals

import json
import logging
import time
from contextlib import asynccontextmanager
from typing import Annotated
from uuid import uuid4

from fastapi import Depends, FastAPI, Header, HTTPException, Request, Response
from pydantic import BaseModel, Field

logger = logging.getLogger("api")
logging.basicConfig(level=logging.INFO, format="%(message)s")

class RuntimeState:
    def __init__(self) -> None:
        self.ready = False
        self.request_count = 0
        self.error_count = 0
        self.total_latency = 0.0

state = RuntimeState()

@asynccontextmanager
async def lifespan(app: FastAPI):
    state.ready = True
    logger.info(json.dumps({"event": "startup_complete"}))
    yield
    state.ready = False
    logger.info(json.dumps({"event": "shutdown_start"}))

app = FastAPI(title="FastAPI deployment example", lifespan=lifespan)

class WidgetIn(BaseModel):
    name: str = Field(min_length=2, max_length=40)
    quantity: int = Field(ge=1, le=100)

class WidgetOut(WidgetIn):
    id: str

async def correlation_id(x_request_id: Annotated[str | None, Header()] = None) -> str:
    return x_request_id or uuid4().hex

@app.middleware("http")
async def access_log(request: Request, call_next):
    start = time.perf_counter()
    request_id = request.headers.get("x-request-id") or uuid4().hex
    try:
        response = await call_next(request)
    except Exception:
        state.error_count += 1
        logger.exception(json.dumps({"event": "request_failed", "request_id": request_id}))
        raise
    elapsed = time.perf_counter() - start
    state.request_count += 1
    state.total_latency += elapsed
    response.headers["x-request-id"] = request_id
    logger.info(json.dumps({
        "event": "request_complete",
        "method": request.method,
        "path": request.url.path,
        "status_code": response.status_code,
        "latency_ms": round(elapsed * 1000, 2),
        "request_id": request_id,
    }))
    return response

@app.get("/livez")
async def livez() -> dict[str, str]:
    return {"status": "alive"}

@app.get("/readyz")
async def readyz() -> dict[str, str]:
    if not state.ready:
        raise HTTPException(status_code=503, detail="not ready")
    return {"status": "ready"}

@app.get("/metrics")
async def metrics() -> Response:
    average = state.total_latency / state.request_count if state.request_count else 0.0
    body = "\n".join([
        "# TYPE app_requests_total counter",
        f"app_requests_total {state.request_count}",
        "# TYPE app_errors_total counter",
        f"app_errors_total {state.error_count}",
        "# TYPE app_request_latency_seconds_average gauge",
        f"app_request_latency_seconds_average {average:.6f}",
    ])
    return Response(body + "\n", media_type="text/plain; version=0.0.4")

@app.post("/widgets", response_model=WidgetOut, status_code=201)
async def create_widget(payload: WidgetIn, request_id: Annotated[str, Depends(correlation_id)]) -> WidgetOut:
    logger.info(json.dumps({"event": "widget_create", "request_id": request_id, "name": payload.name}))
    return WidgetOut(id="wgt_001", **payload.model_dump())

This example separates liveness from readiness. /livez returns {"status": "alive"} as long as the app can process a basic request. /readyz returns {"status": "ready"} only after lifespan startup sets the readiness flag. During shutdown the flag is cleared so a load balancer can stop sending new work before the process exits.

The middleware records latency, status code, path, method, and request id as JSON. If the caller sends x-request-id: demo-1, the response carries the same value. The /metrics endpoint emits Prometheus-style text with request counters and average latency. This small collector is for teaching; in a real service use a maintained metrics library so concurrent updates, histograms, and process labels are handled correctly.

Example 2: Verifying Probes, Headers, and Metrics

from fastapi.testclient import TestClient

from main import app, state


def test_health_checks_and_metrics() -> None:
    with TestClient(app) as client:
        live = client.get("/livez")
        ready = client.get("/readyz")
        created = client.post("/widgets", json={"name": "router", "quantity": 2}, headers={"x-request-id": "demo-1"})
        metrics = client.get("/metrics")

    assert live.json() == {"status": "alive"}
    assert ready.json() == {"status": "ready"}
    assert created.status_code == 201
    assert created.headers["x-request-id"] == "demo-1"
    assert created.json() == {"name": "router", "quantity": 2, "id": "wgt_001"}
    assert "app_requests_total" in metrics.text
    assert state.request_count >= 4


def test_readiness_fails_when_startup_has_not_completed() -> None:
    state.ready = False
    with TestClient(app) as client:
        response = client.get("/readyz")
    assert response.status_code == 503
    assert response.json()["detail"] == "not ready"

The test enters TestClient as a context manager so FastAPI lifespan events run. Without lifespan, a readiness test may fail for the wrong reason because startup never happened. The deterministic widget response is {"name": "router", "quantity": 2, "id": "wgt_001"}, and the echoed request id proves the middleware is connected to the transport layer.

The second test flips readiness to false and expects HTTP 503 with detail not ready. That matters because orchestration depends on failure semantics. A readiness endpoint that always returns 200 makes a broken instance look healthy and keeps it in rotation.

Example 3: Worker Supervision Configuration

workers = 4
worker_class = "uvicorn.workers.UvicornWorker"
bind = "0.0.0.0:8000"
timeout = 30
graceful_timeout = 30
keepalive = 5
accesslog = "-"
errorlog = "-"

This Gunicorn configuration is a Python file, so it can be parsed and reviewed like code. worker_class selects Uvicorn’s ASGI worker implementation. bind listens on the container or host interface, while the proxy in front handles public HTTPS. timeout protects against stuck workers, and graceful_timeout gives in-flight requests a bounded chance to finish during deploys or scale-down.

The expected operational behavior is that the process manager forks four workers and writes access and error output to standard streams. Container platforms, systemd, or log agents can then collect those streams. If memory pressure rises after moving from one to four workers, each worker has its own interpreter, imports, connection pools, and caches.

Example 4: Reading a Deployment Incident

Assume a release changes a startup dependency from optional to required. After deployment, the load balancer shows many 503 responses from /readyz, while /livez remains 200. Logs contain startup_complete for some instances but not others. The symptom tells you the process is alive, but some workers are not ready to receive traffic. The likely cause is initialization that sometimes fails or exceeds the readiness budget.

The diagnostic path is concrete. First, compare readiness failures by version label. Second, inspect startup logs for the missing dependency or timeout. Third, query metrics for process restarts and request latency. Fourth, verify whether only new instances fail. The correction may be rollback, fixing configuration, or changing startup so noncritical integrations initialize lazily while /readyz reports only critical capability.

Design Choices and Trade-offs

Use more workers when requests are mostly I/O-bound and the process has available memory; use fewer workers when memory is tight or each worker owns expensive pools. For CPU-bound work, prefer a task queue, process pool, or separate service instead of blocking the event loop inside a FastAPI route. Async syntax improves concurrency only when the called libraries also yield control.

Health checks should be cheap. Liveness asks whether restarting this process is useful. Readiness asks whether this instance should get traffic now. Mixing those questions causes outages: a database outage should normally make readiness fail, but restarting every application process because the database is down adds noise and load.

Structured logs are better than formatted strings because a log backend can filter by request_id, status_code, path, or event. The trade-off is discipline: every field needs a stable meaning, and high-cardinality or sensitive values must be excluded. Metrics have the same constraint. A histogram of route latency is useful; a metric label containing every customer id is expensive and may expose data.

Failure Modes and Troubleshooting

Symptom Likely cause Diagnostic steps Correction
Requests hang until proxy timeout. Blocking CPU work or a dependency call without timeout inside an async route. Check latency metrics, worker CPU, and stack traces; reproduce with one slow dependency. Add client timeouts, move CPU work out of the event loop, and return bounded errors.
New deploy receives no traffic. /readyz fails because lifespan startup did not complete. Run the app locally with lifespan enabled and inspect startup logs. Fix required configuration, reduce startup work, or make noncritical setup lazy.
Client IP or HTTPS redirects are wrong. Forwarded headers are trusted from the wrong source or ignored. Inspect proxy headers and app server proxy settings in staging. Trust forwarded headers only from the ingress and configure allowed hosts.
Dashboard cost explodes after release. Metrics labels include raw ids, URLs, or request-specific values. List top label cardinality in the metrics backend. Use route templates and bounded status or outcome labels.

Security, Performance, and Reliability

Deployment settings are security controls. HTTPS termination, allowed hosts, proxy header trust, secret injection, and log redaction decide what the app believes and reveals. Never log authorization headers, cookies, raw personal data, or full request bodies by default. Prefer environment-provided secrets and give each deployed service only the credentials it needs.

Performance tuning starts with measurement. Increase workers only after watching p95 and p99 latency, CPU, memory, database pool usage, and queue depth. A faster application can still fail if every worker opens too many database connections. Reliability comes from bounded behavior: startup deadlines, request timeouts, graceful shutdown, readiness removal before termination, and rolling, blue-green, or canary releases.

Hands-on Lab: Run and Inspect a Deployable App

Prerequisites: Python with FastAPI, Uvicorn, and pytest installed; a shell; and permission to open a local port. Create main.py from Example 1 and test_main.py from Example 2 in a temporary directory.

  1. Run uvicorn main:app --host 127.0.0.1 --port 8000.
  2. Request http://127.0.0.1:8000/livez and expect {"status": "alive"}.
  3. Request http://127.0.0.1:8000/readyz and expect {"status": "ready"}.
  4. Post JSON {"name": "router", "quantity": 2} to /widgets with header x-request-id: lab-1; expect HTTP 201, the widget body, and response header x-request-id: lab-1.
  5. Request /metrics and verify that app_requests_total increased.
  6. Stop the server with the terminal interrupt signal and confirm the shutdown log appears.
  7. Run pytest and confirm both tests pass.

Cleanup is simple: stop the server and delete the temporary files. Rollback practice is to change the widget endpoint to return a different id, watch the test fail, then revert the change. That proves verification catches a contract change before deployment.

Assessment Exercises

  • A FastAPI service has 200 responses from /livez and 503 responses from /readyz. Explain why restarting the process may be the wrong first response.
  • Modify the metrics design so latency is measured per route template without using raw path parameters. What label values are safe?
  • Given a route that performs image processing for five seconds, decide whether adding more async keywords fixes throughput. Propose a better deployment design.
  • Design a canary release check using request logs, readiness status, and error metrics. What evidence would make you continue, pause, or roll back?
  • Write a test that proves the request id is generated when the client omits x-request-id without requiring a specific random value.

Summary

FastAPI deployment is the operational wrapper around the ASGI app: server, workers, proxy, lifecycle, probes, and release strategy. Observability is the app’s deliberate output about that wrapper and each request: logs, metrics, request ids, health, and failure categories. Keep liveness cheap, readiness meaningful, worker state isolated, labels bounded, and shutdown graceful. Those choices make typed FastAPI code behave predictably when it starts serving real traffic.