Module 7: Reliable Analysis Workflows · Lecture 28 of 32
An analysis needs tests for the assumptions that could change its conclusion. Protect key uniqueness, currency arithmetic, join cardinality, non-mutation, and rejection behavior with small executable tests.
What you will learn
- Distinguish example assertions from a reusable pytest suite.
- Test meaningful properties and adversarial inputs.
- Keep failures explicit instead of weakening the contract.
Understand the problem
Test the risk
A test that repeats the exact implementation expression can pass even when both copies encode the wrong rule. Start from a concrete risk: duplicate delivery inflates revenue, an invalid refund creates a negative amount, or a dimension join multiplies rows. Build a small input whose correct outcome can be reasoned about independently.
Use several kinds of evidence
A known small example checks arithmetic. A property such as nonnegative net revenue checks a broad invariant. A metamorphic test checks that adding an exact redelivery leaves the clean result unchanged. A failure-path test checks that a conflicting key raises instead of silently choosing a row. These tests complement one another.
Keep fixtures narrow
The supplied test_pipeline.py includes the fixed course export plus tiny altered inputs for invalid dates, fractional quantities, excessive refunds, missing required columns, duplicate customer keys, and conflicting orders. Run python -m pytest -q from the kit root. pytest discovers test functions and reports the assertion or expected exception that failed.
Do not confuse checks with production guards
Python can remove assert statements under optimization. Essential production validation belongs in explicit conditions that raise useful errors; assert is suitable for tests and teaching checks. When a test fails, inspect whether the source changed, the contract changed, or the implementation broke. Updating a snapshot blindly can approve a regression.
Run the worked example
Download and extract the course kit linked below. Open a terminal in the extracted folder, complete the setup in Lecture 2, and run python run_course.py --lesson 28 --examples-only. Each script starts from the shared dataset; it does not depend on a notebook cell from another lecture. All retail records are fictional and amounts are integer paise unless a chart explicitly says INR.
from pathlib import Path
import numpy as np
import pandas as pd
from course_data import load_orders, load_raw, load_customers, clean_orders, ROOT, OUT
orders = load_orders()
raw = load_raw()
before = raw.copy(deep=True)
cleaned, audit = clean_orders(raw)
pd.testing.assert_frame_equal(raw, before)
assert cleaned['order_id'].is_unique
assert int(cleaned['net_paise'].sum()) == 24112420
assert int(cleaned.loc[cleaned['status'].eq('cancelled'), 'net_paise'].sum()) == 0
bad = raw.drop(columns=['unit_price_paise'])
try:
clean_orders(bad)
except ValueError as error:
assert 'Missing columns' in str(error)
else:
raise AssertionError('Missing required column was accepted')
print('Non-mutation, money, uniqueness, cancellation, and missing-column checks passed.')
Verified output
Non-mutation, money, uniqueness, cancellation, and missing-column checks passed.
Walk through the result
The copy comparison verifies that cleaning preserves the raw evidence. The total is a reviewed fixture value for the supplied synthetic data, not a target that new data must be forced to match. The missing-column case expects an explicit error with an actionable message. The dedicated pytest suite expands those properties across separate cases so one failure does not conceal unrelated results.
Common mistakes and diagnosis
Do not weaken validation because a new source file fails it. Do not test only the happy path. A passing test suite cannot establish data completeness outside the tested assumptions. Avoid asserting exact floating-point equality for statistics that need a tolerance, while retaining exact equality for integer paise. Do not use optimization mode as a way to bypass a failed analysis assertion.
Practice assignment
Write a test proving that a valid order with a refund larger than discounted gross is rejected. Then write a test that exact redelivery leaves total revenue unchanged. Explain why those tests protect different failure modes.
Worked solution and explanation
Add the following code below the worked example, or run the matching file in the solutions folder. Compare the result with the business definition before trusting a passing assertion.
one = raw.iloc[[1]].copy()
one.loc[:, 'order_id'] = 'TOO-MUCH-REFUND'
one.loc[:, 'refunded_paise'] = '999999999'
rejected, rejected_audit = clean_orders(one)
assert len(rejected) == 0 and rejected_audit['rejected_rows'] == 1
repeated, _ = clean_orders(pd.concat([raw, raw.iloc[[0]]], ignore_index=True))
assert int(repeated['net_paise'].sum()) == int(orders['net_paise'].sum())
Check your understanding
Why should a test suite include expected failures?
Answer guide
A pipeline's reliability depends on refusing unsupported inputs as well as calculating valid ones. A cleaner that returns plausible numbers for a conflicting key or missing price can be more dangerous than one that stops. Expected-failure tests verify the boundary between accepted evidence and data that requires investigation.
Reference and next step
Download the complete course kit: datasets, all examples, solutions, and capstone.
Return to the full Data Analysis with Python and pandas curriculum.
