File Uploads and Background Tasks

Published by Programming Line in FastAPI. Last updated .

FastAPI file upload endpoints let a client send bytes as multipart/form-data instead of JSON. BackgroundTasks lets the endpoint register small pieces of follow-up work that run after the response has been sent. Used together, they are useful for flows such as accepting a profile image, storing it, returning a quick 202 Accepted, and then writing an audit record, extracting metadata, or notifying another internal component.

The outcome of this lesson is a concrete upload design: receive the file through UploadFile, validate it before trusting it, stream it to a controlled location, and decide whether post-response work belongs in FastAPI’s in-process background task hook or in a separate worker queue. In this API Design section, that matters because uploads change the shape of an API contract. The contract is not only fields and status codes; it also includes file size limits, media types, retry behavior, task visibility, and what the client can assume after the response returns.

How FastAPI Receives Uploaded Files

An upload arrives as an HTTP request whose body is divided into multipart sections. Each section has headers, usually including a field name and sometimes a submitted filename and content type. FastAPI delegates request parsing to Starlette. When an endpoint parameter is annotated as UploadFile and uses File(...), FastAPI knows that the value should come from multipart form data instead of JSON.

UploadFile is not just a bytes object. It wraps a spooled temporary file. Small uploads are kept in memory; larger ones roll over to a file-like object on disk. That behavior lets your endpoint read in chunks without loading the entire request into application memory. The object exposes filename, content_type, async methods such as read, seek, and close, and the underlying file attribute for code that expects a file-like object.

FastAPI also supports bytes = File(). That is convenient for very small files because the parameter already contains the full body. It is a poor default for user uploads because the whole file must fit comfortably in memory. UploadFile is the normal choice when size is not trivially bounded or when the endpoint will stream the content elsewhere.

How BackgroundTasks Runs

BackgroundTasks is a request-scoped collector. FastAPI injects it when an endpoint declares a parameter of that type. Your endpoint calls background_tasks.add_task(function, *args, **kwargs). FastAPI attaches those tasks to the response. After the response body is sent, Starlette runs the registered callables in the same application process.

This timing is important. A background task is not a durable job. If the process stops after sending the response but before the callable finishes, the work can be lost. If the callable is CPU-heavy, it still competes with the API process. If it raises an exception, the client has already received the response. Use BackgroundTasks for short, best-effort work such as logging, cache cleanup, sending a lightweight notification, or scheduling a real job in Redis, RabbitMQ, SQS, or another queue. Use a dedicated worker for thumbnail generation, antivirus scanning, document parsing, and anything that must be retried or tracked.

API Anatomy

The upload parameter usually looks like file: UploadFile = File(...). The name file becomes the multipart field name clients must send. Additional form fields can be declared with Form(...), but JSON bodies and multipart files do not share the same request body format. If an endpoint needs both file bytes and structured metadata, send the metadata as form fields or as a JSON string field that you parse explicitly.

Status codes should communicate the lifecycle. 201 Created fits when the resource is stored and immediately usable. 202 Accepted fits when the upload was accepted but follow-up processing is still pending. Validation failures should be specific: 413 for too large, 415 for unsupported media type, 422 for missing multipart fields, and 400 for malformed content that passed transport-level parsing but failed application checks.

Example 1: Inspect a Small Upload

This first endpoint shows the mechanics. It reads only the first 64 bytes and returns metadata supplied by the multipart parser. For a tiny uploaded file of 20 bytes, first_bytes will be 20. For a larger file, it will be 64 because the endpoint deliberately reads only one small chunk.

from fastapi import FastAPI, File, UploadFile

app = FastAPI(title="Upload basics")

@app.post("/avatar")
async def upload_avatar(file: UploadFile = File(...)):
    chunk = await file.read(64)
    await file.close()
    return {
        "filename": file.filename,
        "content_type": file.content_type,
        "first_bytes": len(chunk),
    }

The example is intentionally not a complete storage implementation. It demonstrates that filename and content_type are client-provided metadata, not proof. The server can report them, but it should not use them alone to make security decisions.

Example 2: Validate and Stream to Disk

The next endpoint adds the minimum production shape: allowed media types, a maximum byte count, generated storage names, chunked reading, and cleanup if validation fails mid-stream. A supported image under two MiB returns 201 with a generated filename and byte count. An unsupported content type returns 415. A file that crosses the size limit returns 413 and the partial destination is deleted.

from pathlib import Path
from uuid import uuid4

from fastapi import FastAPI, File, HTTPException, UploadFile

app = FastAPI(title="Validated upload")
UPLOAD_DIR = Path("/tmp/fastapi-upload-demo")
MAX_BYTES = 2 * 1024 * 1024
ALLOWED_TYPES = {"image/png", "image/jpeg"}

async def save_upload(file: UploadFile) -> dict[str, object]:
    if file.content_type not in ALLOWED_TYPES:
        raise HTTPException(status_code=415, detail="Unsupported file type")

    UPLOAD_DIR.mkdir(parents=True, exist_ok=True)
    suffix = Path(file.filename or "upload.bin").suffix.lower()
    destination = UPLOAD_DIR / f"{uuid4().hex}{suffix}"
    size = 0

    with destination.open("wb") as output:
        while chunk := await file.read(1024 * 1024):
            size += len(chunk)
            if size > MAX_BYTES:
                output.close()
                destination.unlink(missing_ok=True)
                raise HTTPException(status_code=413, detail="File too large")
            output.write(chunk)

    await file.close()
    return {"stored_as": destination.name, "bytes": size}

@app.post("/images", status_code=201)
async def create_image(file: UploadFile = File(...)):
    return await save_upload(file)

Generated storage names avoid collisions and prevent submitted filenames such as ../../app.py from controlling a path. The suffix is kept only as a convenience for later tools; the authoritative identity is the generated name. For stricter systems, inspect file signatures with a library that understands the format instead of trusting content_type.

Example 3: Add Post-response Work

This endpoint accepts a request to index an already stored image. It registers a simple function that appends a line to a log file after the response. The deterministic client-visible behavior is the 202 response. The side effect is the added CSV line, which happens after response sending in the same process.

from pathlib import Path

from fastapi import BackgroundTasks, FastAPI

app = FastAPI(title="Background task demo")
LOG_FILE = Path("/tmp/fastapi-upload-demo/tasks.log")

def write_index_record(image_id: str, stored_name: str) -> None:
    LOG_FILE.parent.mkdir(parents=True, exist_ok=True)
    with LOG_FILE.open("a", encoding="utf-8") as log:
        log.write(f"{image_id},{stored_name}\n")

@app.post("/images/{image_id}/index", status_code=202)
async def request_index(image_id: str, background_tasks: BackgroundTasks):
    stored_name = f"{image_id}.png"
    background_tasks.add_task(write_index_record, image_id, stored_name)
    return {"status": "accepted", "image_id": image_id}

The task function is synchronous because the work is ordinary file I/O. Background tasks may also call async functions. Keep arguments simple and serializable when the task represents a handoff to another system; passing open file handles or request objects makes the work tightly coupled to a request that has already ended.

Example 4: Test the Contract

Tests should prove both acceptance and rejection. With TestClient, background tasks are run before the test call returns, so the side effect can be asserted directly. The large upload test verifies a crucial ordering rule: reject invalid input before scheduling post-response work.

from pathlib import Path

from fastapi import BackgroundTasks, FastAPI, File, HTTPException, UploadFile
from fastapi.testclient import TestClient

app = FastAPI()
STATE = {"processed": []}
MAX_BYTES = 8

def mark_processed(name: str) -> None:
    STATE["processed"].append(name)

@app.post("/documents", status_code=202)
async def upload_document(background_tasks: BackgroundTasks, file: UploadFile = File(...)):
    data = await file.read()
    await file.close()
    if len(data) > MAX_BYTES:
        raise HTTPException(status_code=413, detail="File too large")
    safe_name = Path(file.filename or "document.bin").name
    background_tasks.add_task(mark_processed, safe_name)
    return {"accepted": safe_name, "bytes": len(data)}

def test_small_upload_runs_background_task():
    client = TestClient(app)
    response = client.post(
        "/documents",
        files={"file": ("note.txt", b"hello", "text/plain")},
    )
    assert response.status_code == 202
    assert response.json() == {"accepted": "note.txt", "bytes": 5}
    assert STATE["processed"] == ["note.txt"]

def test_large_upload_is_rejected_before_task():
    client = TestClient(app)
    STATE["processed"].clear()
    response = client.post(
        "/documents",
        files={"file": ("large.txt", b"too large", "text/plain")},
    )
    assert response.status_code == 413
    assert STATE["processed"] == []

Design Choices and Trade-offs

Choose UploadFile when files may be more than tiny because chunked reads limit memory pressure. Choose bytes only when the endpoint has a small hard limit and needs the whole body at once. Store files outside the application source tree. If storing locally, write to a directory with restricted permissions and record generated names in a database. If storing in object storage, stream from UploadFile to the storage client and keep the same validation rules.

Decide what the client receives. Returning the final public URL is convenient, but only correct when the file is already safe and available. Returning an upload id plus processing status is better when scanning, transformation, or moderation must happen first. For background work, decide whether losing a task is acceptable. If not, the background task should enqueue a durable job and return; the durable worker should perform the expensive work.

Failure Modes and Troubleshooting

Symptom: every multipart request returns an error before your endpoint logic runs. Cause: the multipart parser dependency is missing or the client is not sending multipart/form-data. Diagnose by checking the response detail and by inspecting the request in a client such as curl or an HTTP test. Correct it by installing the multipart parser package used by FastAPI deployments and sending the file under the exact field name in the endpoint signature.

Symptom: memory grows quickly during uploads. Cause: the endpoint used bytes or called await file.read() on unbounded input. Diagnose by reviewing the endpoint signature and looking for full-body reads. Correct it by using UploadFile, reading fixed-size chunks, enforcing a maximum size while streaming, and setting proxy or server body limits as an outer guard.

Symptom: uploaded files overwrite each other or appear outside the upload directory. Cause: the application trusted file.filename as a storage path. Diagnose by uploading names with duplicate values and path separators. Correct it by generating server-side names and using Path(filename).name only when you need a display name.

Symptom: clients receive 202, but processing never happens after a restart. Cause: BackgroundTasks stored work only in the web process. Diagnose by restarting the server immediately after a response and checking whether the side effect exists. Correct it by persisting a job record or enqueueing work in a durable queue before responding.

Security, Performance, and Reliability

Treat file bytes as hostile until validated. Limit accepted media types, extension handling, file size, total request size, and storage location. For public downloads, serve files through a controlled route or object-storage policy rather than exposing arbitrary filesystem paths. For sensitive uploads, encrypt at rest, restrict access by object id, and avoid logging raw filenames when they may contain personal information.

Performance depends on bounding resources. Chunk size is a trade-off: very small chunks add overhead, while very large chunks increase memory spikes per concurrent request. Background tasks improve perceived latency only for small follow-up work; they do not add capacity for CPU-heavy processing. Reliability depends on writing state in the right order: store the file, record its status, then schedule or enqueue processing. That gives operators a recoverable record if later work fails.

Hands-on Lab

Prerequisites: a FastAPI project, an ASGI server, pytest, httpx, and the multipart parser dependency used for FastAPI file forms. Create an endpoint named /lab/uploads that accepts one UploadFile field named file. Allow only text files, enforce a one MiB limit while reading chunks, store the file under a generated name in a temporary upload directory, and register a background task that writes stored_name,byte_count to an index file.

  1. Add constants for the upload directory, index file, maximum size, and allowed content types.
  2. Write a helper that creates the directory, streams chunks from UploadFile, counts bytes, deletes partial files on failure, and closes the upload.
  3. Return 201 if the file is fully stored and 413 or 415 for the relevant validation failure.
  4. Add a background task only after storage succeeds.
  5. Write tests for a valid text file, an oversized file, and an unsupported content type.

Verification: the valid test should assert the response contains a generated stored name and exact byte count. It should also assert the stored file exists and the index line was written. The failure tests should assert that no partial file remains and no index entry was added. Cleanup: delete the temporary upload directory and clear any in-memory state between tests so one test cannot pass because of another test’s side effect.

Assessment Exercises

  • An endpoint accepts file: bytes = File() and users begin uploading 200 MiB videos. Redesign the signature and read loop, and explain which limits belong in application code versus the reverse proxy.
  • A product manager wants the response to include a permanent download URL immediately after upload. List the checks that must complete before that is safe, and propose an alternative response for pending files.
  • A background task generates thumbnails. After deployments, some thumbnails are missing although clients received 202. Explain the likely cause and design a durable replacement.
  • Write a test that proves unsupported file types do not create storage objects or schedule follow-up work.

Summary

FastAPI uploads are multipart request parts mapped into UploadFile objects so endpoints can inspect metadata and stream bytes. BackgroundTasks attaches short follow-up callables to the response and runs them after sending, inside the same process. The practical design is to validate early, stream with limits, generate server-controlled storage names, expose honest lifecycle status, and move durable or expensive work to a real queue.