Install Python and Build a Reproducible Analysis Project

Module 1: Foundations and Reproducibility · Lecture 2 of 32

A reproducible project records the environment, input files, and execution order. Set up the downloadable course kit so that every lecture can run from a clean terminal without hidden notebook state.

What you will learn

Create an isolated Python environment on Windows, macOS, or Linux.
Locate data relative to the project rather than the current working directory.
Record versions and fingerprints for reproducible runs.

Understand the problem

Install and verify

Use Python 3.12 or a compatible newer Python supported by the pinned packages. Download the course kit from the link at the end of this page and extract it. In its root folder run python -m venv .venv. On Windows run .venv\Scripts\python -m pip install -r requirements.txt; on macOS or Linux run .venv/bin/python -m pip install -r requirements.txt. Activation is optional when you use the environment's Python explicitly. Where the command is named python3, substitute python3 for the initial environment creation.

Understand the folders

The data directory holds immutable fictional CSV exports. course_data.py contains the shared loading and validation rules. Numbered scripts in examples correspond to these lectures; solutions contains each full example plus its assignment answer. The runner puts the project root on Python's module search path, so use python run_course.py --lesson 2 for a portable launch. Running python run_course.py checks every example and solution. The outputs directory contains generated results and can be rebuilt.

Control notebook state

A notebook can display a result produced by a variable no longer defined by its visible cells. Restart its kernel and run all cells from top to bottom before sharing it. A script with explicit imports and deterministic inputs is an easier first reproducibility target. Record package versions rather than assuming that “latest” will behave identically next month.

Fingerprint the input

A SHA-256 digest identifies the bytes in a particular file. If the input hash changes, investigate whether a new export, changed encoding, line endings, or edited rows explain the difference. A hash is not proof that the data is accurate, anonymous, or complete. Store it alongside the metric contract and environment versions, not instead of them.

Run the worked example

Download and extract the course kit linked below. Open a terminal in the extracted folder, complete the setup in Lecture 2, and run python run_course.py --lesson 2 --examples-only. Each script starts from the shared dataset; it does not depend on a notebook cell from another lecture. All retail records are fictional and amounts are integer paise unless a chart explicitly says INR.

from pathlib import Path
import numpy as np
import pandas as pd
from course_data import load_orders, load_raw, load_customers, clean_orders, ROOT, OUT

import hashlib
import platform
import matplotlib
raw_path = ROOT / 'data' / 'orders_raw.csv'
digest = hashlib.sha256(raw_path.read_bytes()).hexdigest()
print('Python:', platform.python_version())
print('pandas:', pd.__version__)
print('NumPy:', np.__version__)
print('Matplotlib:', matplotlib.__version__)
print('Input SHA-256:', digest)
assert raw_path.is_file()
assert len(digest) == 64

Verified output

Python: 3.12.14
pandas: 3.1.0
NumPy: 2.5.4
Matplotlib: 3.11.2
Input SHA-256: e75574cca90fb1bcdd173775a64899b3b67e783748a58a62e25109a5cd788596

Walk through the result

The resolved ROOT path comes from the location of course_data.py, not from whichever folder your terminal happens to show. That eliminates a common class of file-not-found failures. The version lines identify the tested runtime; exact patch numbers may differ if you deliberately change the requirements. The digest is deterministic for unchanged input bytes. In your own project, retain the original export and give transformed files new names instead of overwriting the evidence.

Common mistakes and diagnosis

Installing packages into a different Python is common when several interpreters exist. Use python -m pip with the interpreter that will run the script. A ModuleNotFoundError for course_data usually means the script was launched without the project root on the search path; use run_course.py. Avoid embedding a personal absolute folder path in submitted code. Do not commit .venv, credentials, or real customer exports to a public repository.

Practice assignment

Create a JSON manifest containing the Python and pandas versions, the input filename, and its digest. Reload the JSON and assert that the digest matches. Explain what you would investigate if the same filename had a different digest tomorrow.

Worked solution and explanation

Add the following code below the worked example, or run the matching file in the solutions folder. Compare the result with the business definition before trusting a passing assertion.

import json
manifest = {'python': platform.python_version(), 'pandas': pd.__version__,
            'input': raw_path.name, 'sha256': digest}
manifest_path = OUT / 'environment-manifest.json'
manifest_path.write_text(json.dumps(manifest, indent=2), encoding='utf-8')
restored = json.loads(manifest_path.read_text(encoding='utf-8'))
assert restored['sha256'] == digest

Check your understanding

Does pinning package versions guarantee an identical analysis?

Answer guide

No. Inputs, Python version, operating system details, timezone assumptions, locale, algorithm choices, and random number generation can also matter. Pinning reduces one source of change. A stronger reproducibility record includes the input digest, transformation code, metric contract, environment, and a command that rebuilds the results from the original data.

Reference and next step

Download the complete course kit: datasets, all examples, solutions, and capstone.

Return to the full Data Analysis with Python and pandas curriculum.

Course index · Previous · Next