Module 6: Statistics and Sound Decisions · Lecture 22 of 32
A point estimate hides uncertainty under a sampling model. Build a bootstrap interval for mean observed customer revenue, resampling customers rather than correlated orders, and state exactly what the interval does and does not mean.
What you will learn
- Implement a percentile bootstrap with a fixed seed.
- Resample at the unit appropriate to the question.
- Explain uncertainty without claiming certainty or causality.
Understand the problem
Define the estimand
Here the quantity of interest is average net merchandise revenue per identified customer in the observed non-cancelled-order window. First aggregate all qualifying orders for each customer. This creates one value per customer, avoiding the mistake of treating several orders from the same person as independent customer observations.
Resample the observed units
A nonparametric bootstrap draws a sample of the same size with replacement from the observed units, calculates the statistic, and repeats. The 2.5th and 97.5th percentiles of those replicate statistics form a simple percentile interval. This method depends on the observed units and the sampling assumptions; it does not create information about people absent from the source.
Interpret the interval carefully
A frequentist 95% interval is a procedure designed to cover a fixed target in a stated proportion of repeated compatible samples. It is not a statement that a fixed population parameter now has a 95% probability of lying inside this particular realized interval. A percentile bootstrap is an approximation and can perform poorly for tiny, highly skewed, dependent, or boundary-constrained samples.
Know when uncertainty is unnecessary
If the goal is simply to total a complete historical export, the total is a descriptive fact about those rows; sampling uncertainty is not the same as missing-data or measurement uncertainty. The course dataset is synthetic and not a random sample of a real market, so this interval teaches a method. It must not be presented as a real business inference.
Run the worked example
Download and extract the course kit linked below. Open a terminal in the extracted folder, complete the setup in Lecture 2, and run python run_course.py --lesson 22 --examples-only. Each script starts from the shared dataset; it does not depend on a notebook cell from another lecture. All retail records are fictional and amounts are integer paise unless a chart explicitly says INR.
from pathlib import Path
import numpy as np
import pandas as pd
from course_data import load_orders, load_raw, load_customers, clean_orders, ROOT, OUT
orders = load_orders()
active = orders.loc[orders['status'].ne('cancelled') & orders['customer_id'].notna()]
customer_values = (active.groupby('customer_id')['net_paise'].sum() / 100).to_numpy(dtype=float)
rng = np.random.default_rng(22)
replicates = np.array([rng.choice(customer_values, size=len(customer_values), replace=True).mean()
for _ in range(2000)])
low, high = np.quantile(replicates, [0.025, 0.975])
estimate = customer_values.mean()
print('Mean observed customer revenue INR:', round(float(estimate), 2))
print('Illustrative percentile interval INR:', round(float(low), 2), round(float(high), 2))
assert low <= high
assert len(replicates) == 2000
assert np.isfinite(replicates).all()
Verified output
Mean observed customer revenue INR: 5017.19
Illustrative percentile interval INR: 3915.29 6160.93
Walk through the result
The resampling unit is a customer total, not an order. Each replicate contains the same number of customer draws as the observed table, with some customers repeated and others omitted. The interval summarizes variability under that empirical resampling model. It does not account for misrecorded refunds, missing exports, selection into the dataset, or a changing business process. Those limitations belong alongside any reported interval.
Common mistakes and diagnosis
Do not bootstrap rows blindly when observations are clustered or serially dependent. More bootstrap repetitions reduce simulation noise, not source bias. Do not read the interval as a guaranteed range for every customer's revenue. A confidence interval for a mean is different from a prediction interval for an individual. Avoid reporting many decimal places that imply more precision than the design supports.
Practice assignment
Run the bootstrap twice using the same seed and assert identical replicate arrays in this environment. Then compare 500 and 2000 repetitions. Explain which uncertainty the larger repetition count reduces and which it cannot reduce.
Worked solution and explanation
Add the following code below the worked example, or run the matching file in the solutions folder. Compare the result with the business definition before trusting a passing assertion.
def simulate(seed, repetitions):
generator = np.random.default_rng(seed)
return np.array([generator.choice(customer_values, len(customer_values), replace=True).mean()
for _ in range(repetitions)])
a = simulate(22, 500)
b = simulate(22, 500)
assert np.array_equal(a, b)
print('500-replicate interval:', np.quantile(a, [0.025, 0.975]).round(2).tolist())
Check your understanding
Why aggregate to customers before this bootstrap?
Answer guide
The question concerns average customer revenue, and orders from one customer can be dependent. Aggregation creates the intended customer-level units. If the target were an order-level quantity with customer clustering, a cluster bootstrap would instead resample customers while retaining their order groups and recalculate the appropriate order-level statistic.
Reference and next step
Download the complete course kit: datasets, all examples, solutions, and capstone.
Return to the full Data Analysis with Python and pandas curriculum.
