Parameters, Metrics, Artifacts, and Tags

Parameters, metrics, artifacts, and tags are the four record types that turn an experiment run into usable MLOps evidence. A training job may produce a model file, but the run record explains how that file was produced, how it behaved, what it belongs to, and whether it is comparable with another candidate. The practical outcome is simple: given two runs, you should be able to tell which values were chosen before training, which observations came out of evaluation, which files must be preserved, and which labels make the record searchable and governable.

In this MLOps course, the topic connects model development to deployment. Without these records, promotion decisions depend on notebooks, screenshots, memory, or ad hoc filenames. With them, an evaluation gate can ask precise questions: was this run trained on the approved dataset snapshot, did it exceed the validation threshold, does the model artifact match its digest, and who owns follow-up when production behavior changes?

What Each Record Means

A parameter is an input choice for a run. Learning rate, tree depth, embedding model name, feature set name, and split seed are parameters because changing them can change the result. A parameter should usually be immutable inside a run: if the value changes, you have a different run, not edited history.

A metric is a measured value emitted by training, validation, testing, or operation. Metrics may be scalar summaries such as accuracy, AUC, loss, latency, or cost. They may also be logged repeatedly with a step, timestamp, or epoch. A metric has direction, scope, and measurement method. Higher AUC is usually better, lower loss is usually better, and both are meaningless unless you know which dataset and evaluator produced them.

An artifact is a file or directory produced or consumed by the run. Common artifacts include a serialized model, tokenizer, feature schema, evaluation report, plot image, confusion matrix, sample predictions, environment lock file, or data profile. The experiment store keeps metadata while the artifact store keeps bytes.

A tag is descriptive metadata used for search, grouping, ownership, policy, and lifecycle decisions. Tags such as dataset=credit-risk-2026-08, owner=risk-ml, code_revision=git:a1, and stage=candidate do not normally affect training math directly, but they decide whether records can be found, compared, audited, and promoted.

Internal Mechanism

Most experiment trackers use a run as the central container. A client library starts a run and receives a run identifier. Calls such as log_param, log_metric, log_artifact, and set_tag write records under that identifier. The tracking backend persists structured metadata in a database or service. Artifact bytes are usually written to a filesystem, blob store, or object store, while run metadata stores artifact paths, sizes, content types, and sometimes digests.

The separation matters. Querying all runs where valid_auc > 0.90 should not require downloading every model file. Downloading a model for deployment should not require replaying the notebook that trained it. Comparing runs should use stable metadata filters, while reproducing or debugging a selected run should retrieve deeper artifacts such as configs, schemas, and reports.

Metrics often have history. Training loss logged at steps 1, 2, and 3 is a time series, while final validation AUC may be a single scalar. The latest value is not always the best value. For early stopping, the best validation score could occur before the final epoch. A useful run record identifies whether a deployment gate uses latest, minimum, maximum, or a named checkpoint metric.

Artifacts also need identity. A path such as model.pkl is not enough if later code can overwrite it. Reliable systems either place artifacts under immutable run-scoped locations or record a content digest. Promotion should reference a specific run and artifact digest, not a mutable local filename.

API Anatomy

The vocabulary is consistent across many tools even when method names differ. Parameter APIs take a key and value, convert the value into a persistable representation, and reject or discourage conflicting updates. Metric APIs take a key, numeric value, and optional step or timestamp. Artifact APIs take a local path or bytes plus a destination path. Tag APIs take key-value strings that support filtering and governance.

Record Typical key Typical value Comparison use
Parameter learning_rate 0.05 Group runs by input choice
Metric valid_auc 0.897 Rank candidates under a fixed evaluator
Artifact reports/confusion_matrix.txt Stored file bytes Inspect detailed evidence
Tag dataset credit-risk-2026-08 Filter to comparable runs

Example 1: Recording One Run

This first example builds a tiny in-memory run object. Parameters are fixed input facts, metrics accumulate by step, tags are searchable labels, and artifacts are addressed by a run-relative path.

from dataclasses import dataclass, field
from pprint import pprint

@dataclass
class Run:
    run_id: str
    params: dict = field(default_factory=dict)
    metrics: dict = field(default_factory=dict)
    tags: dict = field(default_factory=dict)
    artifacts: dict = field(default_factory=dict)

    def log_param(self, key: str, value) -> None:
        if key in self.params and self.params[key] != value:
            raise ValueError(f"parameter {key} is immutable for this run")
        self.params[key] = str(value)

    def log_metric(self, key: str, value: float, step: int = 0) -> None:
        self.metrics.setdefault(key, []).append({"step": step, "value": float(value)})

    def set_tag(self, key: str, value: str) -> None:
        self.tags[key] = value

run = Run("run-001")
run.log_param("learning_rate", 0.05)
run.log_param("max_depth", 6)
run.log_metric("valid_auc", 0.881, step=1)
run.log_metric("valid_auc", 0.897, step=2)
run.set_tag("dataset", "credit-risk-2026-08")
run.set_tag("owner", "risk-ml")
run.artifacts["reports/confusion_matrix.txt"] = "TN=920 FP=30 FN=44 TP=206"

pprint(run.params)
pprint(run.metrics["valid_auc"][-1])
pprint(run.tags)

The output is three lines: {'learning_rate': '0.05', 'max_depth': '6'}, {'step': 2, 'value': 0.897}, and {'dataset': 'credit-risk-2026-08', 'owner': 'risk-ml'}. That confirms numeric parameters were stored consistently, the latest validation AUC kept its step, and ownership plus dataset tags travel with the run.

Example 2: Comparing Only Compatible Runs

The second example shows why tags are not decorative. run-003 has the highest AUC, but it was evaluated on a different dataset tag. Selecting it would mix two problem definitions. The filter narrows candidates to the same dataset before ranking by metric.

runs = [
    {
        "run_id": "run-001",
        "params": {"learning_rate": "0.05", "max_depth": "6"},
        "metrics": {"valid_auc": [{"step": 2, "value": 0.897}]},
        "tags": {"dataset": "credit-risk-2026-08", "code_revision": "git:a1"},
    },
    {
        "run_id": "run-002",
        "params": {"learning_rate": "0.02", "max_depth": "8"},
        "metrics": {"valid_auc": [{"step": 2, "value": 0.904}]},
        "tags": {"dataset": "credit-risk-2026-08", "code_revision": "git:a1"},
    },
    {
        "run_id": "run-003",
        "params": {"learning_rate": "0.02", "max_depth": "8"},
        "metrics": {"valid_auc": [{"step": 2, "value": 0.913}]},
        "tags": {"dataset": "credit-risk-2026-09", "code_revision": "git:a1"},
    },
]

baseline_dataset = "credit-risk-2026-08"
comparable = [r for r in runs if r["tags"].get("dataset") == baseline_dataset]
winner = max(comparable, key=lambda r: r["metrics"]["valid_auc"][-1]["value"])

print(winner["run_id"])
print(winner["params"])
print(winner["metrics"]["valid_auc"][-1]["value"])

The deterministic output is run-002, its parameter dictionary, and 0.904. It beats run-001 inside the approved comparison group, while run-003 is excluded because its dataset tag is credit-risk-2026-09.

Example 3: Binding Artifacts to Promotion Metadata

The third example records a content digest and checks required tags before promotion. This pattern prevents a deployment process from approving an orphaned file with no dataset, code revision, or owner.

from hashlib import sha256

artifact_bytes = b"model coefficients:0.42,-1.13,2.09"
manifest = {
    "artifact_path": "models/model.bin",
    "sha256": sha256(artifact_bytes).hexdigest(),
    "content_type": "application/octet-stream",
    "producer_run_id": "run-002",
}

required_tags = {"dataset", "code_revision", "owner"}
run_tags = {"dataset": "credit-risk-2026-08", "code_revision": "git:a1", "owner": "risk-ml"}
missing = required_tags - set(run_tags)
if missing:
    raise ValueError(f"cannot promote without tags: {sorted(missing)}")

print(manifest["artifact_path"])
print(manifest["sha256"][:12])
print("promotion metadata complete")

The output begins with models/model.bin, then the digest prefix 9d382d921d30, then promotion metadata complete. If the owner tag were removed, the script would raise an error instead of printing the final line.

Design Choices and Trade-offs

Choose parameter granularity carefully. Logging every default from every library creates noise and makes queries hard to read. Logging only three hand-picked values can hide the true cause of a result. A practical compromise is to log values that materially affect model behavior, data selection, feature construction, evaluation, or deployment compatibility.

Metric naming needs discipline. auc, valid_auc, and test_auc are not interchangeable. Include the split or evaluation context in the name, and avoid changing metric definitions without changing names or tags. For threshold gates, document whether the gate uses a validation set, holdout set, shadow traffic, or post-deployment monitoring window.

Artifacts are expensive compared with scalar metadata. Large checkpoints, plots, and prediction dumps improve debuggability but increase storage cost and retrieval time. Store final deployable artifacts, compact evaluation reports, schemas, and enough samples to diagnose behavior. Use retention rules for intermediate checkpoints unless they are required for recovery or compliance.

Tags are flexible, which makes them easy to misuse. Free-form tags support fast experimentation, but production workflows need a small controlled vocabulary for dataset, code revision, feature view, owner, ticket, risk tier, and promotion state. Otherwise, prod, production, and stage=Production become three different truths.

Failure Modes and Troubleshooting

Symptom: a run cannot be reproduced even though the model artifact exists. Cause: the artifact was logged without parameters, dataset identity, or code revision. Diagnose: inspect run metadata for missing keys and required tags, then compare artifact time with source control and data catalog records. Correct: block promotion unless required parameters and lineage tags are present.

Symptom: dashboards show a model improvement that disappears after deployment. Cause: runs were ranked across different datasets, splits, or metric definitions. Diagnose: group candidates by dataset tag, evaluator version, and metric name. Correct: make comparison queries include dataset and evaluator filters, and rename metrics when their calculation changes.

Symptom: deployment downloads the wrong model file. Cause: artifact paths are mutable or shared across runs. Diagnose: compare the deployed file digest with the recorded manifest and check whether multiple runs wrote to the same destination. Correct: store artifacts under run-scoped prefixes and promote by run identifier plus artifact path.

Symptom: experiment tracking slows training. Cause: the job logs huge artifacts or high-frequency metrics synchronously inside the hot training loop. Diagnose: time logging calls, inspect artifact sizes, and compare training duration with logging disabled in a disposable environment. Correct: reduce logging frequency, batch metric writes, compress reports, and move large uploads outside the per-batch path.

Security, Performance, and Reliability

Experiment records can leak sensitive information. Do not put secrets, raw personal data, or confidential customer examples into tags, parameter values, metric names, or artifact filenames. If sample predictions are necessary, mask or synthesize sensitive fields and apply the same access controls as the source data.

Reliability comes from treating the run record as part of the build output. A training job should fail if required evidence cannot be logged, because a model without evidence is not promotable. At the same time, high-volume telemetry should be buffered or sampled so that the tracker does not become the bottleneck for numerical training work.

Hands-on Lab

Prerequisites: Python 3, a clean working directory, and permission to create a temporary folder. No external service is required. The lab uses the examples above as a local simulation of an experiment tracker.

  1. Create experiment_records.py and paste Example 1. Run it and confirm that the printed parameter dictionary contains learning_rate and max_depth.
  2. Change the script to call run.log_param("learning_rate", 0.10) after the original learning-rate line. Verification: the script should raise ValueError: parameter learning_rate is immutable for this run. Roll back that line.
  3. Create compare_runs.py with Example 2. Verify that run-002 wins. Change baseline_dataset to credit-risk-2026-09 and verify that run-003 wins for that different comparison group. Restore the original value.
  4. Create artifact_gate.py with Example 3. Verify the three printed lines. Remove the owner tag and confirm that promotion is refused. Restore the tag after the test.
  5. Cleanup: delete the three temporary Python files. If you used a virtual environment created only for this lab, deactivate and remove it.

Assessment

  1. You see two runs with identical valid_auc but different dataset tags. What additional evidence would you require before deciding which model to promote?
  2. A teammate logs learning_rate as a metric because it is numeric. Explain the practical bug this creates for run comparison.
  3. An artifact store contains latest/model.pkl for every experiment. Describe the failure this can cause and design a safer artifact path.
  4. A tracking database has millions of per-batch loss points. Which metrics would you keep, aggregate, or drop to preserve useful debugging while reducing overhead?
  5. Write a promotion rule that uses at least one parameter, one metric, one artifact property, and one tag.

Summary

Parameters capture chosen inputs, metrics capture measured behavior, artifacts preserve deployable and diagnostic files, and tags make runs searchable and governable. Together, they let a team reproduce a result, reject invalid comparisons, promote a specific artifact, and debug failures without guessing.