Design a Reproducible Project Structure

A reproducible ML project structure is a directory and naming scheme that lets another person, a CI job, or a future version of you rebuild the same training run from declared inputs. The outcome is not a pretty tree. The outcome is that code imports are stable, notebooks do not hide required logic, data is referenced without being casually copied, parameters are separated from source, and outputs can be traced back to the exact run that produced them.

In this MLOps foundations course, this lesson sits before orchestration, registries, and deployment because those systems assume a project already has repeatable boundaries. A pipeline scheduler cannot fix a repository where training reads an untracked spreadsheet from a desktop. A model registry cannot explain an artifact whose feature code lived only in a notebook cell. Structure is the first operational interface of an ML project.

Purpose and Outcome

A good project layout answers four questions quickly: where does importable code live, where are configuration files, where are data references documented, and where do generated artifacts go. It also separates source from derived state. Source includes Python modules, tests, lock files, small schemas, and configuration templates. Derived state includes trained models, reports, cached features, metrics, and temporary extracts. Derived state may be stored locally during development, but it should be ignored by version control or written to a managed artifact store.

The practical target is a project that can be cloned, installed, configured, tested, and executed with a small set of commands. Reproducibility does not mean every model score is bit-for-bit identical on every processor. It means the run declares enough evidence that differences are explainable: code revision, environment, data snapshot, parameters, random seed, and output location.

How the Structure Works Internally

The internal mechanism is path discipline plus metadata discipline. Path discipline means code uses project-relative locations or explicit URIs instead of implicit current working directories. Metadata discipline means every training run records the identifiers needed to reconstruct it. The repository root becomes the anchor. Under it, src/ holds the package that production and tests import. configs/ holds parameter files for named experiments. tests/ checks feature functions, data validation rules, and command entry points. data/ documents expected datasets and may contain tiny samples, but large or sensitive datasets are referenced by immutable snapshot URI. artifacts/ is a local landing zone for generated files and is usually ignored except for a placeholder.

The src/ layout matters because it prevents accidental imports from the repository root. If tests pass only because Python finds a loose train.py beside the test runner, packaging will fail later. A package such as src/churn_model/ forces development, testing, and batch jobs to import the same module path. Configuration files matter because source code should not be edited every time an experiment changes the learning rate, feature set, split date, or output destination.

Syntax and Anatomy

A typical small ML repository uses these roles:

Path	Responsibility
pyproject.toml	package name, dependencies, test settings, and command entry points
src/<package>/	importable feature, training, evaluation, and inference code
configs/	named parameter sets that can be selected by CI or an orchestrator
tests/	unit and integration tests that run without private local files
data/README.md	dataset contracts, snapshot naming, ownership, and access instructions
artifacts/	generated models, reports, metrics, and manifests during local runs

The most important convention is that executable code accepts inputs as arguments or config values. It should not silently read ~/Downloads/train.csv, depend on notebook execution order, or overwrite a shared artifact name such as model.pkl without a run identifier.

Example 1: Name the Project Surfaces

This first example models a minimal layout and prints the top-level surfaces. It is small, but it shows the design intent: each directory has one job.

from pathlib import PurePosixPath

LAYOUT = [
    "README.md",
    "pyproject.toml",
    "configs/train_baseline.yaml",
    "data/README.md",
    "notebooks/01_explore.ipynb",
    "src/churn_model/__init__.py",
    "src/churn_model/features.py",
    "src/churn_model/train.py",
    "tests/test_features.py",
    "artifacts/.gitkeep",
]

def top_level(path: str) -> str:
    return PurePosixPath(path).parts[0]

for name in sorted({top_level(path) for path in LAYOUT}):
    print(name)

The expected output is README.md, artifacts, configs, data, notebooks, pyproject.toml, src, and tests, one per line in sorted order. The files at the root are human and packaging entry points. The directories separate reusable code from exploration, configuration, tests, data documentation, and generated outputs.

Example 2: Validate Configuration Boundaries

The second example treats a training configuration as an API. It requires an explicit data URI, a target column, a deterministic seed, and a project-relative output path.

from dataclasses import dataclass
from pathlib import PurePosixPath

@dataclass(frozen=True)
class TrainConfig:
    data_uri: str
    target: str
    seed: int
    output_dir: str

    def validate(self) -> None:
        if not self.data_uri.startswith(("s3://", "gs://", "file://")):
            raise ValueError("data_uri must be an explicit URI")
        if not self.target:
            raise ValueError("target column is required")
        if self.seed < 0:
            raise ValueError("seed must be non-negative")
        if PurePosixPath(self.output_dir).is_absolute():
            raise ValueError("output_dir must be project-relative")

config = TrainConfig(
    data_uri="s3://mlops-course/churn/snapshot-2026-08-01.parquet",
    target="churned",
    seed=42,
    output_dir="artifacts/churn/baseline",
)
config.validate()
print(f"train {config.target} from {config.data_uri} into {config.output_dir}")

The deterministic output is train churned from s3://mlops-course/churn/snapshot-2026-08-01.parquet into artifacts/churn/baseline. The checks are deliberately simple. A missing target would make the run ambiguous. A negative seed is rejected because randomization should be controlled. An absolute output path is rejected because it would work on one machine and fail or overwrite the wrong location on another.

Example 3: Record a Run Manifest

The third example creates a compact manifest and hashes its canonical JSON representation. In a real project this manifest would sit beside the model artifact or be logged to an experiment tracker.

from dataclasses import dataclass
from hashlib import sha256
import json

@dataclass(frozen=True)
class RunManifest:
    code_revision: str
    data_uri: str
    config_name: str
    seed: int
    metrics: dict[str, float]

    def fingerprint(self) -> str:
        payload = json.dumps(self.__dict__, sort_keys=True, separators=(",", ":"))
        return sha256(payload.encode("utf-8")).hexdigest()[:12]

run = RunManifest(
    code_revision="git:3f2a19c",
    data_uri="s3://mlops-course/churn/snapshot-2026-08-01.parquet",
    config_name="train_baseline.yaml",
    seed=42,
    metrics={"auc": 0.873, "log_loss": 0.421},
)
print(run.fingerprint())

The expected output is 3f9b97e95c64. If the code revision, data snapshot, config name, seed, or metric values change, the fingerprint changes. This does not prove the model is correct, but it creates a stable handle for audit, comparison, and cleanup. It also prevents a vague artifact name such as latest.pkl from becoming the only clue.

Design Choices and Trade-offs

Keep notebooks, but do not make them the only executable path. Notebooks are useful for exploration and explanation; package modules are better for repeated training and serving. A common rule is that notebooks may call src/ code, but production jobs should not import from notebooks.

Keep small sample data in the repository only when it is non-sensitive and intentionally tiny. Sample data is valuable for tests and documentation. Full training data usually belongs in object storage, a feature store, or a warehouse with versioned snapshot identifiers. Storing large binaries in Git slows every clone and makes removal difficult.

Choose names that encode intent, not personal workflow. configs/train_baseline.yaml is better than final2.yaml. artifacts/churn/2026-09-06T120000Z/ is better than overwriting output/. The trade-off is verbosity, but explicit names reduce ambiguity when multiple runs exist.

Failure Modes and Troubleshooting

• Symptom: tests pass locally but fail in CI with ModuleNotFoundError. Cause: code was imported from the working directory instead of an installed package. Diagnose: run tests after installing the package in a clean environment. Correct: move reusable code under src/<package>/ and configure packaging in pyproject.toml.
• Symptom: a teammate cannot reproduce a model score. Cause: the script read a mutable data path or used an undeclared random seed. Diagnose: compare the run manifest with the command arguments and storage object version. Correct: require immutable data URIs, record seeds, and persist the resolved configuration.
• Symptom: a training job overwrites a previous model. Cause: every run writes to the same artifact path. Diagnose: inspect output paths in recent logs and object storage. Correct: write artifacts under a run identifier and promote by metadata rather than replacing files in place.

Security, Performance, and Reliability

Structure affects security because misplaced files are easy to commit. Keep secrets in environment variables or a secret manager, not in config files. Add ignore rules for local credentials, raw data, and generated artifacts. Structure affects performance because large data and model binaries in Git slow clones and CI. Use pointers, snapshot names, or artifact stores for large files. Structure affects reliability because stable entry points make automation possible: a scheduler can run the same command with a selected config, and a rollback can select a previous artifact by manifest.

This guard example detects absolute paths and unknown top-level locations before they become hidden dependencies.

from pathlib import PurePosixPath

PROJECT_FILES = [
    "configs/train_baseline.yaml",
    "src/churn_model/train.py",
    "tests/test_features.py",
    "artifacts/churn/baseline/model.pkl",
    "/tmp/manual_export.csv",
]

ALLOWED_TOP_LEVEL = {"configs", "src", "tests", "artifacts", "data", "notebooks"}

violations = []
for path in PROJECT_FILES:
    parsed = PurePosixPath(path)
    if parsed.is_absolute() or parsed.parts[0] not in ALLOWED_TOP_LEVEL:
        violations.append(path)

print("violations:", violations)

The expected output is violations: ['/tmp/manual_export.csv']. The absolute path is rejected because it cannot be reproduced from the project alone.

Hands-on Lab

Prerequisites: Python, Git, and a shell environment where you can create a temporary repository. The lab is intentionally local and does not require cloud credentials.

1. Create an empty directory and initialize Git.
2. Add pyproject.toml, README.md, configs/, src/churn_model/, tests/, data/README.md, and artifacts/.gitkeep.
3. Put one reusable feature function in src/churn_model/features.py and one test for it in tests/test_features.py.
4. Create configs/train_baseline.yaml with a data snapshot URI, target column, seed, and output directory.
5. Add a small script that reads the config, validates required fields, creates a run-specific directory under artifacts/, and writes a manifest containing code revision, data URI, config name, seed, and metrics.
6. Verify by cloning the repository into a second temporary directory, installing it, running tests, and executing the training script with the same config. The manifest fields should match except for values intentionally produced at run time, such as timestamps.
7. Cleanup by deleting the temporary repositories and any generated artifact directories. If you used remote object storage, remove test objects by run identifier rather than broad prefix deletion.

Assessment

1. A training script reads data/train.csv if it exists and otherwise downloads the newest warehouse extract. What reproducibility problem does this create, and how would you redesign the input?
2. Your team wants to commit a 600 MB model so demos work offline. What are two operational costs, and what alternative structure would preserve demo convenience?
3. A notebook contains the only feature transformation used by a deployed model. Describe a migration path that keeps exploration useful while making training repeatable.
4. Design a manifest schema for a batch scoring job. Which fields are required to explain a prediction file six months later?
5. How would you detect in CI that a new script introduced an absolute local path dependency?

Summary

A reproducible ML project structure turns a directory tree into an operational contract. Put reusable code in a package, keep configuration explicit, document data snapshots, isolate generated artifacts, and record run manifests. The structure is successful when another environment can rebuild the run or explain why it differs using declared evidence rather than local memory.