Machine Learning Introduction

Machine learning is a way to build behavior from examples instead of writing every decision rule by hand. In this foundations chapter, the practical outcome is simple: given a problem, you should be able to decide whether it needs supervised learning, unsupervised learning, reinforcement learning, or no machine learning at all.

That decision matters before Python, data frames, or model libraries enter the picture. A model is useful when examples contain a pattern that can generalize to future cases and when mistakes can be measured. A deterministic rule is better when the correct behavior is already known, legally mandated, easy to express, or too important to approximate. The rest of this course builds models in Python, but those models only make sense when the learning setup is clear.

What Machine Learning Learns

A machine learning system estimates a function from data. In supervised learning, the data contains inputs and known answers. The model learns a mapping from features to a target. A spam classifier, for example, uses features from an email and learns a target such as spam or not spam. In regression, the target is numeric, such as a house price. In classification, the target is a category.

In unsupervised learning, the data contains inputs but no answer key. The model searches for structure: clusters, lower-dimensional representations, unusual points, or associations. The output is not inherently right or wrong in the same way a labeled prediction is. It must be interpreted against a purpose, such as segmenting customers for analysis or finding unusual sensor readings for review.

In reinforcement learning, an agent takes actions in an environment and receives rewards. The agent is not simply matching labels. It learns a policy: a strategy for choosing actions that should produce higher long-term reward. This setup is appropriate when actions change the future state, such as game play, robotic control, or sequential allocation problems. It is usually overkill for ordinary tabular prediction.

Core Mechanism

The internal loop of supervised learning has four moving parts. Features represent the information available at prediction time. A model family defines the shape of patterns the learner can express. A loss function measures how wrong predictions are during training. An optimizer or fitting procedure adjusts model parameters to reduce loss on training data.

Generalization is the central idea. A model that memorizes training examples but fails on new examples has not learned the useful pattern. That is why data is split into training and evaluation sets. The training data is used to fit the model. The evaluation data is held back to estimate behavior on unseen cases. If the evaluation set leaks into feature engineering, model selection, or manual tuning decisions too heavily, the score becomes optimistic.

Terminology is precise. An observation or row is one case. A feature is an input variable. A label or target is the answer for supervised learning. A prediction is the model’s output. A parameter is learned from data, such as a weight. A hyperparameter is chosen outside fitting, such as the number of neighbors in a nearest-neighbor classifier or the number of clusters in k-means.

API Anatomy

Most Python machine learning APIs follow the same shape even when their internals differ. You prepare a matrix-like collection of feature rows, prepare a target vector when labels exist, create an estimator with configuration, call a fitting method on training data, and call a prediction or transformation method on new data. The key design requirement is that the same preprocessing used during training must be used during prediction.

Concept	Typical name	Question it answers
Features	X	What information is available before the answer is known?
Target	y	What should supervised learning predict?
Fit	fit	What pattern is estimated from examples?
Predict	predict	What does the fitted model output for new cases?
Score	accuracy, mae, loss	How is model quality measured?

Example 1: A Rule Beats a Model

Not every decision that uses data is machine learning. If the requirement says orders of at least 100 dollars receive free shipping, fitting a model would add error to a rule that is already exact. The expected behavior is deterministic: 99.99 is not eligible, 100.00 is eligible, and 125.50 is eligible.

def free_shipping(order_total: float) -> bool:
    if order_total < 0:
        raise ValueError("order_total must be non-negative")
    return order_total >= 100.0

for total in [99.99, 100.00, 125.50]:
    print(total, free_shipping(total))

This is the first design choice in machine learning: do not learn what you can specify. Rules are easier to inspect, test, explain, and change when the real policy is known. Machine learning becomes useful when the rule is unknown or too complex to maintain manually, such as predicting delivery delay from weather, distance, carrier, item type, warehouse congestion, and historical patterns.

Example 2: Supervised Classification

The next example uses a tiny nearest-neighbor classifier. It stores labeled examples and classifies a new point by finding the closest training point. This is supervised learning because every training example has a known label. The model family is simple: similarity in feature space implies similarity in label.

from math import dist

training = [
    ((1.0, 1.0), "basic"),
    ((1.5, 1.2), "basic"),
    ((4.0, 4.2), "premium"),
    ((4.4, 3.9), "premium"),
]

def nearest_label(point: tuple[float, float]) -> str:
    closest_point, label = min(training, key=lambda row: dist(point, row[0]))
    return label

for customer in [(1.2, 1.1), (4.2, 4.1)]:
    print(customer, nearest_label(customer))

The expected output labels the first customer as basic and the second as premium. The classifier has no understanding of customers. It only compares numeric feature values. Feature scaling therefore matters: if one column ranges from 0 to 1 and another ranges from 0 to 100000, distance will mostly reflect the larger-scale column unless values are normalized.

Example 3: Unsupervised Clustering

Clustering groups observations without labels. In this small k-means-style example, two centers are updated to the mean of their assigned points. There is no target column. The algorithm creates structure, then a human decides whether that structure is useful.

points = [1.0, 1.2, 1.4, 8.0, 8.2, 8.4]
centers = [1.0, 8.0]

for _ in range(3):
    groups = {0: [], 1: []}
    for point in points:
        nearest = min(range(2), key=lambda i: abs(point - centers[i]))
        groups[nearest].append(point)
    centers = [sum(groups[i]) / len(groups[i]) for i in range(2)]

print([round(center, 2) for center in centers])
print(groups)

The final centers are approximately [1.2, 8.2], and the groups separate the low values from the high values. This is not a proof that there are truly two natural populations. It is evidence that, under this distance measure and a request for two clusters, the data separates cleanly. If you ask for three clusters, you may get a different but still algorithmically valid answer.

Example 4: A Reinforcement Learning Signal

Reinforcement learning is different because the learner evaluates actions through reward. The following bandit example chooses between two actions using fixed observed rewards. A full reinforcement learner would balance exploration and exploitation over many rounds, but this miniature version shows the vocabulary: actions, rewards, values, and policy.

rewards = {
    "show_short_hint": [1, 0, 1, 1],
    "show_full_solution": [0, 1, 0, 0],
}

values = {
    action: sum(results) / len(results)
    for action, results in rewards.items()
}
policy_action = max(values, key=values.get)

print(values)
print(policy_action)

The expected policy action is show_short_hint because its average reward is higher. In a real learning environment, choosing the apparently best action too early can prevent discovery of a better action. That exploration trade-off is one reason reinforcement learning is more complex to evaluate than ordinary classification.

Design Choices and Trade-Offs

The first trade-off is between rules and learning. Rules are preferable when the decision boundary is known and stable. Learning is preferable when examples are plentiful, the pattern changes gradually, and occasional error is acceptable or can be reviewed. A credit-card expiration check should be a rule. Fraud detection usually needs learning because attackers adapt and signals interact.

The second trade-off is model complexity. Simple models are easier to debug and may generalize better on small data. Flexible models can capture richer patterns but need more data, stronger evaluation, and more care against overfitting. Accuracy alone is not enough. A medical triage model, a search ranking model, and a product recommendation model can all have different costs for false positives, false negatives, latency, and explanation.

The third trade-off is representation. The same algorithm can behave well or badly depending on features. Dates, categories, missing values, text, and counts need transformations that preserve meaning. Feature engineering should be fitted on training data only, then applied unchanged to validation and future prediction data.

Failure Modes and Troubleshooting

A common symptom is excellent training accuracy with poor evaluation accuracy. The usual cause is overfitting: the model has captured noise or memorized examples. Diagnose it by comparing train and validation scores, simplifying the model, increasing regularization, or adding more representative data. Correct it by reducing complexity, improving the split, or collecting examples that match the deployment setting.

Another symptom is a suspiciously high validation score. The cause may be data leakage. For example, a feature may contain information created after the target is known, such as a refund timestamp in a model predicting refunds. Diagnose leakage by reviewing when every feature becomes available and by removing features that are too directly correlated with the label. Correct it by rebuilding the dataset around prediction-time availability.

A third symptom is a model that worked last month but now fails for a subgroup or new pattern. The cause may be distribution shift: current data differs from training data. Diagnose it by comparing feature distributions, missing-value rates, and error rates across time and segments. Correct it by retraining on recent representative data, changing features, or falling back to a conservative rule while the model is repaired.

Reliability and Performance Implications

Machine learning adds reliability concerns that ordinary deterministic code may not have. The code can run correctly while the prediction is still poor. Therefore tests need two layers: software tests that check parsing, preprocessing, and API behavior, and model evaluation that checks predictive quality on held-out data. Reproducible splits, fixed random seeds for examples, and saved preprocessing configuration make results easier to compare.

Performance also matters. A nearest-neighbor model may need to compare a new point with every stored training example, which is understandable but can be slow at scale. A linear model may predict quickly after training because it only computes weighted sums. Larger models can improve quality but add memory use, latency, and operational cost. Choose the smallest model that meets the measured requirement.

Hands-On Lab: Choose and Test a Learning Setup

Prerequisites: a local Python interpreter and a terminal. This lab uses only the Python standard library. The goal is to classify small messages as promotional or normal using labeled examples, then verify that the classifier behaves as expected.

1. Create a file named intro_ml_lab.py.
2. Paste the script below.
3. Run python intro_ml_lab.py.
4. Verify that the promotional message is classified as promo and the meeting message is classified as normal.
5. Cleanup is optional: remove the file when finished.

from collections import Counter

examples = [
    ("limited discount offer", "promo"),
    ("buy now discount", "promo"),
    ("project meeting schedule", "normal"),
    ("team schedule update", "normal"),
]

def tokenize(text: str) -> set[str]:
    return set(text.lower().split())

def classify(message: str) -> str:
    scores = Counter()
    words = tokenize(message)
    for text, label in examples:
        scores[label] += len(words & tokenize(text))
    return scores.most_common(1)[0][0]

checks = {
    "discount offer now": "promo",
    "team meeting update": "normal",
}

for message, expected in checks.items():
    predicted = classify(message)
    print(message, predicted, predicted == expected)

The output should end each line with True. If both examples receive the same label, inspect token overlap by printing tokenize(message) and each training example’s tokens. The likely cause is that the message shares no useful words with the intended class. Correct it by adding representative labeled examples or by designing better features. If Python reports a syntax error, check that indentation uses spaces consistently and that every string quote is closed.

Assessment Exercises

• A tax form must reject negative income values. Should this be a model or a rule? Explain what would go wrong with the other choice.
• You have 50,000 labeled support tickets and want to route new tickets to billing, technical support, or account recovery. Identify the features, target, and evaluation metric you would start with.
• A retailer has customer purchase histories but no segment labels. Describe an unsupervised approach and explain how you would decide whether the result is useful.
• A tutoring system chooses between giving a hint, showing an example, or asking a simpler question. What makes this closer to reinforcement learning than ordinary classification?
• A model’s validation accuracy rises after you add a feature named closed_at to predict whether a ticket will be resolved. Explain the likely failure and how to test for it.

Summary

Machine learning is not a synonym for automation. It is a family of methods for estimating behavior from data. Supervised learning learns from labeled examples, unsupervised learning finds structure without labels, and reinforcement learning improves action choices through reward. A deterministic rule remains the right tool when the desired behavior is already explicit. Good machine learning begins by naming the target, checking what information is available at prediction time, choosing an evaluation method, and proving that the learned pattern generalizes beyond the examples used to fit it.