pip install torch numpy) -- every snippet here runs on a plain CPU in a second or two, no GPU needed anywhere;Learn AI Series):I ended #148 by leaving a thread dangling on purpose. We spent that whole episode counting margins -- automate this task, capture that value, cut this cost, license that data -- and every one of those spreadsheet decisions lands on a real human being who was not in the room when we ran the numbers. Today we pick that thread back up. This is the episode where "can I build it" and "should I build it" stop being a clever line and turn into code you actually write.
Let me say up front what this episode is NOT. I am not going to lecture you about being a good person -- I assume you already are one, and a sermon from me would be both boring and useless. The uncomfortable truth about AI harm is that it almost never comes from villains twirling a moustache. It comes from well-meaning engineers with a deadline, a blind spot, and an incentive pointing slightly the wrong way. So the goal here is deliberately practical: give you tools to MEASURE harm before it ships, because a thing you can measure is a thing you can fix ;-)
Having said that, we settle last week's homework first. House rules.
#148 was the economics episode, and all three tasks were about pushing those toy economic models until they told you something the prose had glossed over.
Exercise 1 -- Tune the leash economically. Take the data_flywheel and make the leader PAY for the share it grabs (marketing spend proportional to the new share captured each round). Sweep a few cost levels and find the point where the cumulative spend stops being worth the lead it bought.
def data_flywheel_with_cost(rounds=8, leader0=1.0, follower0=0.9,
capture=0.15, cost_per_share=0.0):
"""Same flywheel as #148, but now the leader pays to keep winning share.
We compare the VALUE of the lead against the cumulative spend."""
leader, follower = leader0, follower0
cum_spend = 0.0
for r in range(rounds):
total = leader + follower
gain = capture * (leader / total) # new share the leader grabs this round
leader += gain
follower += capture * (follower / total)
cum_spend += cost_per_share * gain # marketing spend for that grabbed share
lead_value = leader - follower
worth_it = lead_value > cum_spend
print(f" round {r+1}: lead {lead_value:+.3f} spent {cum_spend:.3f} "
f"worth_it={worth_it}")
for cost in (0.5, 2.0, 8.0):
print(f"cost_per_share = {cost}")
data_flywheel_with_cost(cost_per_share=cost)
At a cheap cost_per_share the leader buys its runaway lead for pocket change and every round stays "worth it". Crank the cost up and the cumulative spend overtakes the value of the lead -- the flywheel still spins, but now the leader is setting money on fire to turn it. The one-sentence lesson: winner-take-most does NOT mean winning at any price, and a dominant position bought above its own value is just an expensive way to lose.
Exercise 2 -- Find the break-even user count. Using avg_cost_per_user, find the smallest number of users at which average cost drops below 5 cents, for a training bill of 100M and of 1B.
def avg_cost_per_user(train_cost, marginal_cost, users):
return (train_cost + marginal_cost * users) / users
def break_even_users(train_cost, marginal_cost, target):
# closed form: users > train_cost / (target - marginal_cost)
users = 1
while avg_cost_per_user(train_cost, marginal_cost, users) >= target:
users *= 2 # geometric search, close enough for this
return users
marginal, target = 0.002, 0.05
for train in (100_000_000, 1_000_000_000):
n = break_even_users(train, marginal, target)
print(f"train ${train:>15,}: break-even around {n:,} users")
Both break-even points are enormous -- billions of users -- and the 1B model needs an order of magnitude more of a crowd than the 100M model to reach the same 5-cent average. Two sentences on what that gap means: only a handful of companies on earth have the distribution to amortize a frontier training run, which is exactly why the foundation-model layer is an oligopoly. Everyone else should fine-tune (#69), where the fixed cost is thousands of dollars in stead of hundreds of millions, and the break-even is a rounding error.
Exercise 3 -- Race the cost curve. Extend the build-versus-buy comparison so the BUILD cost also falls, just slower (30% cheaper per year rather than 10x), and find the year buy overtakes build for good.
def buy_cost(monthly_tokens, price_per_1k, year):
price = price_per_1k * (0.1 ** year) # ~10x cheaper per year
return monthly_tokens / 1000 * price
def build_cost(year, base=8000):
return base * (0.7 ** year) # only 30% cheaper per year
monthly_tokens = 500_000_000
overtake = None
for year in range(8):
buy, build = buy_cost(monthly_tokens, 2.0, year), build_cost(year)
if overtake is None and buy < build:
overtake = year
print(f"year {year}: buy=${buy:>12,.0f} build=${build:>10,.0f}")
print(f"buy overtakes build permanently at year {overtake}")
Buy starts absurdly expensive and self-hosting wins for the first few years, then the 10x-per-year deflation drags the API price clean under the slowly-shrinking build cost and it never comes back up. Two sentences on why: a variable cost that drops tenfold every year will always, eventually, dive below a fixed cost that only trims 30%, no matter how far apart they start. The moment you bet on "we built our own to save money," you are quietly racing a curve that falls ten times faster than yours.
Right -- homework settled. Now let us go measure some harm ;-)
Bias in AI is not a vague moral vibe -- it is a number, and numbers you can compute are numbers you can argue about honestly. A bias audit systematically tests whether your model behaves differently for different groups of people. We drew the theory back in #35; here is the practical instrument. Point it at any classifier and it tells you where the model treats people unequally.
import torch
import numpy as np
from collections import defaultdict
class BiasAuditor:
"""Systematic bias auditing for classification models."""
def __init__(self, model, protected_attributes):
self.model = model
self.protected_attributes = protected_attributes # e.g. ['gender', 'race']
def audit(self, X, y_true, group_labels):
"""
Run a bias audit across all protected groups.
group_labels: dict mapping attribute -> tensor of group IDs
"""
self.model.eval()
with torch.no_grad():
y_pred = self.model(X).argmax(dim=1)
results = {}
for attr in self.protected_attributes:
groups = group_labels[attr]
unique_groups = groups.unique()
group_metrics = {}
for g in unique_groups:
mask = (groups == g)
group_preds = y_pred[mask]
group_true = y_true[mask]
tp = ((group_preds == 1) & (group_true == 1)).sum().float()
fp = ((group_preds == 1) & (group_true == 0)).sum().float()
tn = ((group_preds == 0) & (group_true == 0)).sum().float()
fn = ((group_preds == 0) & (group_true == 1)).sum().float()
group_metrics[g.item()] = {
'accuracy': ((group_preds == group_true).float().mean()).item(),
'positive_rate': (group_preds == 1).float().mean().item(),
'tpr': (tp / (tp + fn + 1e-8)).item(), # true positive rate
'fpr': (fp / (fp + tn + 1e-8)).item(), # false positive rate
}
rates = [m['positive_rate'] for m in group_metrics.values()]
tprs = [m['tpr'] for m in group_metrics.values()]
results[attr] = {
'group_metrics': group_metrics,
'demographic_parity_ratio': min(rates) / (max(rates) + 1e-8),
'equalized_odds_gap': max(tprs) - min(tprs),
}
return results
def print_report(self, results):
"""Print a human-readable bias report."""
for attr, data in results.items():
print(f"\n=== Attribute: {attr} ===")
for group, metrics in data['group_metrics'].items():
print(f" Group {group}: acc={metrics['accuracy']:.3f}, "
f"pos_rate={metrics['positive_rate']:.3f}, "
f"TPR={metrics['tpr']:.3f}, FPR={metrics['fpr']:.3f}")
print(f" Demographic parity ratio: "
f"{data['demographic_parity_ratio']:.3f}")
print(f" Equalized odds gap: "
f"{data['equalized_odds_gap']:.3f}")
The metric choices are not neutral -- each one encodes a different idea of what "fair" even means:
Demographic parity asks: does each group get positive predictions at the same rate? A hiring model that recommends 40% of male applicants but only 20% of female applicants fails demographic parity, full stop, regardless of who was actually qualified.
Equalized odds asks: does each group get the same true positive rate AND the same false positive rate? A medical model that catches 95% of a disease in one group but only 70% in another fails equalized odds -- it is quietly better at protecting one set of patients than another.
Predictive parity asks: when the model says "positive," is it equally likely to be correct for each group? This is the one a person on the receiving end of a decision cares about most -- "if this thing flagged me, how much should I trust it?"
Here is the part that trips up every engineer the first time, myself included. Those three definitions are not just hard to satisfy at once -- they are mathematically INCOMPATIBLE. Kleinberg, Mullainathan and Raghavan proved in 2016 (and Chouldechova showed the same result independently the following year) that when the true base rates differ between groups, you cannot have demographic parity and predictive parity at the same time, except in trivial cases that never happen in real data.
That sounds abstract until you watch it happen. Two groups, different true base rates, one honest noisy score, and we enforce demographic parity by selecting the same fraction of each group:
import numpy as np
def parity_vs_precision(base_rate_a=0.30, base_rate_b=0.10, select=0.20):
"""Two groups with different TRUE base rates. We enforce demographic parity by
selecting the same fraction `select` of each. Watch precision refuse to match."""
rng = np.random.default_rng(0)
for name, rate in (("A", base_rate_a), ("B", base_rate_b)):
y = (rng.random(100_000) < rate).astype(int) # who is actually positive
score = y + rng.normal(0, 1.0, size=y.shape) # a noisy, honest score
thresh = np.quantile(score, 1 - select) # take the top `select` fraction
pred = (score >= thresh).astype(int)
tp = int(((pred == 1) & (y == 1)).sum())
precision = tp / max(int(pred.sum()), 1)
print(f"group {name}: base rate {rate:.0%}, selected {select:.0%}, "
f"precision {precision:.2f}")
parity_vs_precision()
Same selection rate for both groups -- demographic parity satisfied by construction -- and yet the precision comes out different, because the group with the higher base rate has more real positives to find at the same threshold. You equalized one fairness and broke another, and no amount of clever engineering closes that gap, because it is not an engineering gap. It is a theorem.
So fairness is not a technical problem with one correct answer. It is a value choice, and the model cannot make it for you. You have to decide, out loud and in writing, WHICH fairness matters most for THIS application -- and a loan model, a medical triage model, and a content-ranking model will each answer differently. That decision belongs to humans, ideally humans who include the people affected. What the code CAN do is enforce your choice once you have made it -- for instance, picking a separate threshold per group to hit a target positive rate:
import numpy as np
def group_thresholds(scores, groups, target_rate=0.20):
"""One concrete way to ENFORCE demographic parity: a separate score threshold
per group so each group gets the same positive rate."""
thresholds = {}
for g in np.unique(groups):
s = scores[groups == g]
thresholds[g] = float(np.quantile(s, 1 - target_rate))
return thresholds
scores = np.random.default_rng(1).normal(size=1000)
groups = np.random.default_rng(2).integers(0, 2, size=1000)
print(group_thresholds(scores, groups))
Nota bene: per-group thresholds are themselves ethically loaded -- in some jurisdictions treating groups differently is exactly what the law forbids, even when you do it to be fairer. There is no free lunch here. The tool enforces a decision; it does not absolve you of making it.
Documentation is the most boring and the most effective ethical practice there is. If you build a model and do not write down its capabilities, limits, and intended use, somebody WILL misuse it -- not out of malice, but because they never knew where the edges were.
Model cards (Mitchell et al., 2019) document a model the way a nutrition label documents food. Here is one filled in for a realistic, uncomfortable case:
model_card = {
"model_name": "LoanApproval-v2.3",
"model_type": "Gradient Boosted Classifier (XGBoost)",
"intended_use": "Pre-screening loan applications for MANUAL review",
"not_intended_for": [
"Fully automated loan decisions without human review",
"Applications outside the US market",
"Applicants under 18 years old",
],
"training_data": {
"source": "Internal loan application data, 2019-2024",
"size": "2.4M applications",
"demographics": "US adults, skewed toward ages 25-55",
"known_gaps": "Underrepresented: rural applicants, "
"non-English speakers, recent immigrants",
},
"performance": {
"overall_auc": 0.87,
"by_demographic": {
"group_w": {"auc": 0.89, "fpr": 0.08},
"group_x": {"auc": 0.83, "fpr": 0.14},
"group_y": {"auc": 0.85, "fpr": 0.11},
"group_z": {"auc": 0.88, "fpr": 0.07},
},
},
"limitations": [
"Performance degrades on applicants with thin credit files",
"Not validated for economic downturns",
"6-point AUC gap between the best- and worst-served groups",
],
"ethical_considerations": [
"FPR disparity means some groups are more likely to be "
"incorrectly rejected; human review is MANDATORY",
"Model uses zip code, which correlates with protected attributes; "
"impact analysis runs quarterly",
],
"update_schedule": "Retrained quarterly, bias audit monthly",
}
This is not bureaucracy for its own sake. When the model makes a wrong call that damages someone's life, the card is what tells you whether that failure sat inside the known limitations (an accepted, documented risk) or outside the intended use (a misuse someone will answer for). That distinction is the whole ballgame in a courtroom, and more importantly it is the whole ballgame in the code review where you decide whether to ship.
Datasheets for datasets (Gebru et al., 2018) apply the same discipline one layer down, to the training data itself: who collected it, why, from whom, with what consent, and with what known biases. If you cannot answer those questions about your data, you cannot make an honest claim about your model's fairness -- you are just hoping.
Training big models has a real carbon cost, and hand-waving it away is its own little ethical failure. The training run for GPT-3 consumed an estimated 1,287 MWh of electricity -- roughly the annual draw of 120 average US homes -- and a single frontier run today uses a good deal more. For you and me, working at human scale, the number is smaller but not zero, and the nice thing is you can estimate it before you burn it:
def estimate_training_carbon(gpu_hours, gpu_type='A100',
region='us-west', pue=1.1):
"""Rough CO2 estimate for a training run.
PUE = Power Usage Effectiveness (datacenter overhead)."""
gpu_power = { # approximate board power draw, kW
'V100': 0.30,
'A100': 0.40,
'H100': 0.70,
}
carbon_intensity = { # kg CO2 per kWh, varies ENORMOUSLY by grid
'us-west': 0.21, # California, relatively clean
'us-east': 0.38, # more coal in the mix
'eu-west': 0.28, # Western Europe average
'eu-north': 0.05, # Nordic hydro / nuclear
'india': 0.71, # coal-heavy grid
}
power_kw = gpu_power.get(gpu_type, 0.40)
intensity = carbon_intensity.get(region, 0.40)
energy_kwh = gpu_hours * power_kw * pue
co2_kg = energy_kwh * intensity
print(f"GPU hours: {gpu_hours}")
print(f"Energy: {energy_kwh:.1f} kWh")
print(f"CO2: {co2_kg:.1f} kg ({co2_kg/1000:.2f} tonnes)")
print(f"Equivalent to roughly {co2_kg/0.255:.0f} km driven by an average car")
return co2_kg
# Example: fine-tuning a 7B model for 10 hours on 4 A100s
estimate_training_carbon(gpu_hours=40, gpu_type='A100', region='us-west')
Run it for the same job in eu-north versus india and the CO2 number swings by more than an order of magnitude -- the single biggest lever you own is WHERE the electrons come from. What actually moves the needle:
Where does the training data come from? This is, I argue, the most practically contentious question in the whole field right now, and "I found it on the internet" is not a license.
Large language models are trained on web-scraped text: copyrighted books, news articles, blog posts, forum threads, code repositories, social media. Most of the humans who wrote all that never agreed to feed a model, and the law has noticed. The New York Times sued OpenAI. Artists sued Stability AI. The EU AI Act now demands disclosure of copyrighted training data. The US still has no comprehensive federal statute, so the shape of the rules is being decided case by case in courtrooms as we speak.
As a practitioner you cannot settle the lawsuits, but you CAN behave decently upstream, and it is mostly a filtering problem. Honor opt-out signals BEFORE a page ever enters your corpus:
def respects_opt_out(page):
"""A cheap pre-training filter: honor opt-out signals and licensing
before a page is ever allowed into the training set."""
if page.get("robots_noai"): # e.g. X-Robots-Tag: noai
return False
if "noai" in page.get("meta_tags", []): #
return False
if page.get("license") in (None, "all-rights-reserved"):
return False # no clear license == not yours to train on
return True
pages = [
{"url": "a.com", "license": "cc-by", "robots_noai": False, "meta_tags": []},
{"url": "b.com", "license": None, "robots_noai": False, "meta_tags": []},
{"url": "c.com", "license": "cc0", "robots_noai": True, "meta_tags": []},
{"url": "d.com", "license": "cc-by-sa", "robots_noai": False, "meta_tags": ["noai"]},
]
keep = [p["url"] for p in pages if respects_opt_out(p)]
print("pages allowed into training set:", keep)
Only the cleanly-licensed, opt-in page survives the filter, and that is the point -- you default to EXCLUDE, not include. Beyond the filter, three habits keep you on the right side of this: know your sources and their licenses, prefer datasets that actually document where their text came from (The Pile, RedPajama and friends made a real effort here), and consider synthetic data (#133) where generating your own training examples sidesteps some -- not all -- of the consent problem.
Enough principles. Here is the checklist I actually run through, organized around WHEN the decision happens, because ethics you only think about at launch is ethics you got wrong months earlier.
Before you build. Who is affected by this system? Have you talked to any of them? What happens when the model is wrong -- and who pays the price for that error, the company or the user? If the answer to "who bears the cost of a mistake" is "not us," you have a problem baked in before line one.
While you build. Is the training data representative of the people the system will touch? Have you actually run the bias audit, or just intended to? Are your metrics aligned with real user benefit, or with whatever was easy to log? A model optimized for the wrong metric is a bias generator with good intentions.
Before you deploy. Does the model card document the limitations honestly, including the embarrassing ones? Is there a human in the loop for high-stakes decisions? Do users even know they are talking to an AI? A quiet deployment is where accountability goes to die.
After you deploy. Are you monitoring for drift and for bias that emerges as the world shifts under the model? Is there a real feedback channel for the people the system got wrong? Do you have a kill switch, and have you ever tested it?
The common thread through all four, the single question this whole episode reduces to: think hard about the people affected by your system who are NOT in the room while you build it. They never see your standup, they never read your metrics dashboard, and they carry every consequence of the choices you make without them. Keeping them in mind is not soft. It is the entire discipline.
Get your hands dirty before the next one. Three tasks, climbing in difficulty:
Add predictive parity to the auditor. Extend BiasAuditor so each group also reports precision (of the samples it predicted positive, how many were truly positive). Add a predictive_parity_gap alongside the existing disparity metrics, and have print_report flag the attribute with the widest gap. One sentence on which of the three fairness numbers you would put first for a medical-triage model, and why.
Enforce parity and pay for it. Take parity_vs_precision and, in stead of one shared selection rate, use group_thresholds to hit the SAME target positive rate in each group. Confirm demographic parity now holds exactly, then report the precision gap that stubbornly remains. Two sentences connecting what you see back to the impossibility result.
Write a model-card linter. Given a model_card dict, write check_card(card) that verifies every required key is present, that performance.by_demographic reports at least two groups, and that ethical_considerations is non-empty -- printing exactly what is missing. Two sentences on why a machine-checkable card beats a beautifully-written PDF that nobody can audit.
We open next episode with full solutions, same as always.
And the thread I will leave dangling for next time. We have spent this series building the thing brick by brick -- the math, the networks, the transformers, the agents, the economics, and now the ethics of shipping it. Next we lift our eyes off the workbench and look at where all of this is actually heading: the parts that are still half-rumor and half-research, the ideas that are not in any textbook yet because they are being written THIS year. It is the most speculative episode we will do, and for exactly that reason one of the most fun ;-)
@scipio