Module 7 · Efficient and Responsible DL

Bias, Privacy, Robustness, and Adversarial Risk

Lesson 27 of 32 12 min read Updated

Responsible deep learning is not a separate stage after training. In PyTorch work, bias, privacy, robustness, and adversarial risk are properties of the data pipeline, loss function, optimizer, evaluation slices, and deployment interface. The outcome of this lesson is practical: you will know how to measure whether a model treats groups differently, whether training data can be inferred, whether predictions survive realistic input changes, and whether gradients can be used to deliberately fool the model.

This chapter connects directly to efficient deep learning because every mitigation has a cost. Reweighting examples can improve a minority slice while changing the optimum found by SGD. Differential privacy can reduce memorization while adding gradient noise and lowering accuracy. Robust training can require extra forward and backward passes. The engineering task is to make these trade-offs explicit and testable instead of relying on a single aggregate validation score.

Purpose and Outcome

A model is biased when its errors, scores, or access to benefit differ meaningfully across relevant groups. A model has privacy risk when information about individuals in the training set can be recovered or inferred. A model lacks robustness when small, plausible input changes cause unstable predictions. An adversarial example is a deliberately chosen input, often built from model gradients, that causes a wrong prediction while staying close to a valid input.

For a PyTorch practitioner, these ideas become tensors and metrics. Group labels are tensors aligned with batches. Privacy checks often inspect per-example losses or gradients. Robustness tests transform inputs and compare output stability. Adversarial tests compute gradients with respect to inputs rather than parameters. The same autograd machinery that trains a model also exposes how to attack or harden it.

Mechanisms Inside the Model

Bias usually enters before the model sees a tensor. Sampling imbalance, label noise, proxy variables, and historical decisions shape the empirical distribution. During training, the loss is averaged over the mini-batch, so frequent patterns contribute more gradient steps than rare patterns. If group B has fewer examples or noisier labels, its signal can be drowned out even when the code is mathematically correct. The remedy may be stratified evaluation, group-aware sampling, per-group thresholds, reweighting, or collecting better data.

Privacy risk comes from overfitting and memorization. A network with enough capacity can assign unusually low loss to examples it trained on. Membership inference attacks exploit that gap: if a sample receives much lower loss than comparable held-out data, an attacker may infer it was in training. Differentially private SGD reduces this by clipping per-example gradients and adding calibrated noise before the optimizer step. Clipping bounds each example’s influence; noise makes individual contribution harder to distinguish.

Robustness concerns sensitivity. A classifier should not change from stop sign to speed limit because of a tiny lighting shift or a harmless crop. In PyTorch, this can be tested by applying transformations to validation tensors and comparing predictions, margins, or loss. Robustness is domain-specific: a crop may be valid for natural images, but deleting a token may change the meaning of a legal sentence.

Adversarial risk uses gradients directly. For an input tensor x and correct label y, the loss gradient x.grad says which input directions increase error fastest. The fast gradient sign method, or FGSM, creates x_adv = x + epsilon * sign(x.grad), then clamps the result to the valid input range. The perturbation may be visually tiny, but it is aligned with the model’s most sensitive direction.

API Anatomy

The PyTorch pieces are familiar. Use model.train() for mitigation training and model.eval() for deterministic evaluation. Keep group labels beside features and targets in the dataset so metrics can be sliced after each epoch. Use reduction='none' in losses when you need per-example loss for group metrics, privacy probes, or custom weighting. Use requires_grad_(True) on input tensors only when computing adversarial examples; ordinary evaluation should not retain input gradients.

A typical responsible evaluation record contains aggregate accuracy, per-group accuracy, worst-group accuracy, maximum group gap, calibration by slice, privacy probe signals such as train-versus-holdout loss separation, and robustness results under named transformations. The names matter because robust_to_noise_sigma_0_05 is interpretable while robustness_score hides the actual stress test.

Example 1: Measuring Group Error Gaps

This example computes overall accuracy, accuracy by group, and the maximum group gap. The deterministic result shows a model that looks acceptable in aggregate but is weaker on group B.

from collections import OrderedDict

def group_accuracy(preds, labels, groups):
    correct = [int(p == y) for p, y in zip(preds, labels)]
    overall = sum(correct) / len(correct)
    group_scores = OrderedDict()
    for group in sorted(set(groups)):
        indexes = [i for i, g in enumerate(groups) if g == group]
        group_scores[group] = sum(correct[i] for i in indexes) / len(indexes)
    gap = max(group_scores.values()) - min(group_scores.values())
    return {"overall_accuracy": overall, "group_accuracy": dict(group_scores), "max_gap": gap}

result = group_accuracy(
    preds=[1, 0, 1, 1],
    labels=[1, 0, 0, 1],
    groups=["A", "A", "B", "B"],
)
print(result)

Expected output: {'overall_accuracy': 0.75, 'group_accuracy': {'A': 1.0, 'B': 0.5}, 'max_gap': 0.5}. The design lesson is that aggregate accuracy hides the problem. A release gate based only on overall_accuracy would miss a 50-point slice gap.

Example 2: Per-Example Loss as a Privacy Signal

Privacy analysis often starts by comparing losses. This code marks samples with high loss, but the same mechanism can reveal unusually low loss on training members. A large separation between train and holdout losses is a warning sign for membership inference risk.

import torch
import torch.nn.functional as F

logits = torch.tensor([[4.0, -2.0], [0.0, 0.0], [-1.0, 2.0]])
labels = torch.tensor([0, 1, 0])
losses = F.cross_entropy(logits, labels, reduction="none")
high_loss_indexes = torch.where(losses > 1.0)[0].tolist()
print("high_loss_indexes:", high_loss_indexes)

Expected output: high_loss_indexes: [2]. In a real audit, you would compute distributions for training and holdout samples, then report overlap rather than a single row. If training examples are consistently much easier for the model than comparable nonmembers, regularization, early stopping, data deduplication, or differentially private training may be needed.

Example 3: FGSM Adversarial Perturbation

This example builds a tiny linear classifier and computes an FGSM perturbation. It shows the anatomy of an adversarial input without requiring an image dataset.

import torch
import torch.nn as nn
import torch.nn.functional as F

model = nn.Linear(2, 2, bias=False)
with torch.no_grad():
    model.weight.copy_(torch.tensor([[1.0, -1.0], [-1.0, 1.0]]))

x = torch.tensor([[0.8, 0.2]], requires_grad=True)
y = torch.tensor([0])
loss = F.cross_entropy(model(x), y)
loss.backward()
epsilon = 0.25
x_adv = torch.clamp(x + epsilon * x.grad.sign(), 0.0, 1.0)
print("clean_prediction:", model(x).argmax(dim=1).item())
print("adversarial_input:", x_adv.detach().tolist())
print("adversarial_prediction:", model(x_adv).argmax(dim=1).item())

Expected output: the clean prediction is class 0, the adversarial input is [[0.55, 0.45]], and the adversarial prediction remains 0. The perturbation moved the point toward the decision boundary but did not cross it. Increasing epsilon would create a stronger attack, but may leave the domain of plausible inputs.

Example 4: A Robustness Check Under Noise

Robustness testing compares predictions before and after a named perturbation. The code below uses a deterministic noise tensor so the result is repeatable.

import torch

def prediction_stability(clean_logits, noisy_logits):
    clean = clean_logits.argmax(dim=1)
    noisy = noisy_logits.argmax(dim=1)
    return float((clean == noisy).float().mean())

clean_logits = torch.tensor([[3.0, 1.0], [0.2, 0.8], [1.1, 1.0]])
noise = torch.tensor([[-0.1, 0.1], [0.0, 0.0], [-0.3, 0.3]])
noisy_logits = clean_logits + noise
print("stability:", prediction_stability(clean_logits, noisy_logits))

Expected output: stability: 0.6666666865348816. The third example flips because its margin was small. This suggests a useful diagnostic: track robustness by confidence margin, not only by average accuracy.

Design Choices and Trade-Offs

Bias mitigation can happen in data, objective, or decision policy. Resampling and group-aware batching change the gradient stream. Loss reweighting keeps all examples but changes their influence. Group-specific thresholds can reduce observed gaps without changing representation learning, but they require careful legal and product review. The correct choice depends on whether the measured harm comes from representation, calibration, thresholding, or missing data.

Privacy mitigation also has trade-offs. Removing direct identifiers is not enough because features can still be identifying in combination. Regularization and early stopping reduce memorization but do not provide a formal privacy guarantee. Differential privacy offers a clearer mathematical claim, but per-example gradient clipping and noise can slow training and reduce utility. Secure aggregation and federated learning can reduce central data exposure, but they do not automatically prevent leakage through model updates.

Robustness and adversarial training increase evaluation cost. Testing many corruptions, transformations, and attack strengths can dominate a training run. A practical approach is to define a threat model: what changes are plausible, who can choose inputs, how large perturbations may be, and what failure is unacceptable. Without that boundary, robustness becomes an open-ended benchmark chase.

Failure Modes and Troubleshooting

Symptom: validation accuracy is high, but complaints cluster around one subgroup. Cause: the validation report used only aggregate metrics or the subgroup has too few examples for a stable estimate. Diagnostic steps: compute per-group counts, confidence intervals, false positives, false negatives, and calibration curves. Correction: add slice metrics to every evaluation run, collect more representative data, or change sampling and thresholds based on the measured failure.

Symptom: a privacy probe separates training members from holdout samples. Cause: the model memorized rare or duplicated examples, or training continued after generalization stopped improving. Diagnostic steps: compare train and holdout per-example loss distributions, inspect duplicates, and check whether rare classes have extreme confidence. Correction: deduplicate, regularize, stop earlier, reduce capacity, or introduce differentially private training where the utility cost is acceptable.

Symptom: adversarial examples appear ineffective during testing. Cause: gradients were disabled, the model was left in an unexpected mode, or the adversarial tensor was detached before calling backward(). Diagnostic steps: confirm x.requires_grad, inspect x.grad after backpropagation, and verify that attack loss increases. Correction: enable input gradients only for attack generation, zero stale gradients, and clamp the perturbed tensor into the valid input range.

Symptom: robustness improves on synthetic noise but fails on real data. Cause: the perturbation does not match deployment conditions. Gaussian noise does not represent blur, compression, dialect shift, sensor failure, or prompt injection. Diagnostic steps: review real failure samples and categorize transformations by domain. Correction: build a stress suite from observed failures and plausible abuse cases, then report each named stressor separately.

Security, Performance, and Reliability

Security risk grows when users can adaptively query the model. Rate limits, input validation, output confidence controls, and abuse monitoring can reduce extraction and adversarial probing, but they do not replace model-side testing. Avoid logging raw sensitive inputs during audits; store bounded identifiers, metric summaries, and reproducible dataset versions.

Performance matters because responsible checks can be expensive. Per-group evaluation is cheap if group tensors are already present. Per-example gradients for differential privacy and adversarial training are much more costly. Reliability depends on making these checks part of the training and release process: a robustness notebook run once is weaker than a scripted evaluation that fails the build when worst-group accuracy or attack success exceeds a documented threshold.

Hands-On Lab

Prerequisites: Python with PyTorch installed, a working terminal, and a small classifier or the toy tensors from this lesson. No GPU is required for the toy lab.

  1. Create a script named responsible_eval.py and add the group accuracy function from Example 1.
  2. Add per-example cross entropy using reduction="none" for a fixed logits tensor and labels tensor.
  3. Add the FGSM code from Example 3 and print the clean prediction, adversarial input, and adversarial prediction.
  4. Run the script twice. The outputs should match exactly because the tensors are fixed.
  5. Change the group labels so the incorrect example belongs to group A. Verify that the maximum gap changes and explain why.
  6. Increase epsilon in the FGSM section to 0.75. Verify that the adversarial point crosses the classifier boundary.
  7. Cleanup by deleting the script or reverting it from version control if you added it only for practice.

Verification: the lab is complete when you can show one aggregate metric, one sliced metric, one per-example loss list, and one adversarial input. The important review question is whether each number maps to a named risk and a possible mitigation.

Assessment Exercises

  1. A model has 94% overall accuracy, 96% accuracy for group A, and 72% accuracy for group B. What additional metrics would you request before choosing between resampling, reweighting, and threshold adjustment?
  2. Why does reduction="none" matter for both privacy analysis and bias analysis? Give one example for each.
  3. An FGSM attack does not change any predictions. List three implementation checks before concluding the model is robust.
  4. Compare early stopping and differentially private SGD as privacy mitigations. Which gives a stronger privacy claim, and what cost might it impose?
  5. Design a robustness test for a text classifier where deleting a word may change the true label. What perturbations would be valid?

Summary

Bias, privacy, robustness, and adversarial risk are measurable properties of a PyTorch system. Bias work slices errors by meaningful groups. Privacy work studies memorization and individual influence. Robustness work tests stability under plausible changes. Adversarial work uses gradients to find sensitive directions. Treat each result as evidence tied to a threat model, not as a slogan, and record the trade-off between model utility, compute cost, and risk reduction.

Progress is saved in this browser.