pip install torch numpy) -- and I mean it this time, because a couple of these snippets actually build small networks you can poke at on a plain CPU;Learn AI Series):150 episodes. One hundred and fifty. When I started this series I honestly did not know if I would make it past the linear-regression episodes without half of you wandering off, and here we still are ;-)
I ended #149 by promising you the most speculative episode we would do -- the one where I lift my eyes off the workbench and point at the horizon. So that is today. Everything up to now has been "here is a thing that works, here is the code, go build it." Today is different. Today is "here is a thing that MIGHT work, here is why smart people are excited, and here is my honest guess about whether you should care yet." I will be blunt about which of these I think is real and which is a lovely research paper wearing a business plan as a costume.
Having said that -- we settle last week's homework first, same house rules as always. No skipping ahead ;-)
#149 was the ethics-in-practice episode, and all three tasks were about turning a fairness principle into something a machine can actually check.
Exercise 1 -- Add predictive parity to the auditor. Extend BiasAuditor so each group also reports precision, add a predictive_parity_gap, and have the report flag the attribute with the widest gap.
import torch
class BiasAuditorPlus:
"""The #149 auditor, now with predictive parity (precision) per group."""
def __init__(self, model, protected_attributes):
self.model = model
self.protected_attributes = protected_attributes
def audit(self, X, y_true, group_labels):
self.model.eval()
with torch.no_grad():
y_pred = self.model(X).argmax(dim=1)
results = {}
for attr in self.protected_attributes:
groups = group_labels[attr]
group_metrics = {}
for g in groups.unique():
mask = (groups == g)
gp, gt = y_pred[mask], y_true[mask]
tp = ((gp == 1) & (gt == 1)).sum().float()
fp = ((gp == 1) & (gt == 0)).sum().float()
fn = ((gp == 0) & (gt == 1)).sum().float()
group_metrics[g.item()] = {
'positive_rate': (gp == 1).float().mean().item(),
'tpr': (tp / (tp + fn + 1e-8)).item(),
# NEW: precision == predictive parity per group
'precision': (tp / (tp + fp + 1e-8)).item(),
}
precisions = [m['precision'] for m in group_metrics.values()]
results[attr] = {
'group_metrics': group_metrics,
'predictive_parity_gap': max(precisions) - min(precisions),
}
return results
def print_report(self, results):
worst = max(results, key=lambda a: results[a]['predictive_parity_gap'])
for attr, data in results.items():
flag = " <-- WIDEST GAP" if attr == worst else ""
print(f"=== {attr} (pred-parity gap "
f"{data['predictive_parity_gap']:.3f}){flag} ===")
for g, m in data['group_metrics'].items():
print(f" group {g}: pos_rate={m['positive_rate']:.3f}, "
f"TPR={m['tpr']:.3f}, precision={m['precision']:.3f}")
Precision is the number the person on the receiving end of a decision feels most directly -- "when this thing flagged me, how often was it right?" -- so for a medical-triage model I would put predictive parity first, because a patient who gets told "you are fine" trusts that answer with their life, and that trust had better be worth the same across groups.
Exercise 2 -- Enforce parity and pay for it. Take parity_vs_precision from #149 and, in stead of one shared selection rate, use a per-group threshold so each group hits the SAME positive rate. Confirm demographic parity now holds exactly, then report the precision gap that stubbornly refuses to close.
import numpy as np
def parity_enforced(base_rate_a=0.30, base_rate_b=0.10, target_rate=0.20):
"""Per-group thresholds => demographic parity by construction.
Then we look at precision and watch the impossibility result bite."""
rng = np.random.default_rng(0)
precisions = {}
for name, rate in (("A", base_rate_a), ("B", base_rate_b)):
y = (rng.random(100_000) < rate).astype(int) # who is truly positive
score = y + rng.normal(0, 1.0, size=y.shape) # a noisy, honest score
thresh = np.quantile(score, 1 - target_rate) # THIS group's own cutoff
pred = (score >= thresh).astype(int)
tp = int(((pred == 1) & (y == 1)).sum())
precisions[name] = tp / max(int(pred.sum()), 1)
print(f"group {name}: positive_rate={pred.mean():.3f} "
f"precision={precisions[name]:.2f}")
gap = max(precisions.values()) - min(precisions.values())
print(f"demographic parity holds (both ~{target_rate:.0%}); "
f"precision gap stays at {gap:.2f}")
parity_enforced()
Both groups now get flagged at the same rate -- demographic parity satisfied exactly, by construction -- and yet precision refuses to match, because the group with the higher true base rate simply has more real positives sitting above any given cutoff. That is not a bug you can engineer away with a better threshold; it is the Kleinberg impossibility result from #149 showing up in ten lines of NumPy, telling you that equalizing one fairness spends another.
Exercise 3 -- Write a model-card linter. Given a model_card dict, write check_card(card) that verifies every required key is present, that performance.by_demographic reports at least two groups, and that ethical_considerations is non-empty -- printing exactly what is missing.
def check_card(card):
"""A machine-checkable model card. Prints exactly what is missing."""
required = ["model_name", "model_type", "intended_use", "training_data",
"performance", "limitations", "ethical_considerations"]
problems = [f"missing key: {k}" for k in required if k not in card]
by_demo = card.get("performance", {}).get("by_demographic", {})
if len(by_demo) < 2:
problems.append("performance.by_demographic needs >= 2 groups")
if not card.get("ethical_considerations"):
problems.append("ethical_considerations is empty")
if problems:
print("Card FAILED:")
for p in problems:
print(f" - {p}")
else:
print("Card OK: all required fields present and populated.")
return not problems
# a card missing its ethics section and with only one demographic group
check_card({
"model_name": "LoanApproval-v2.3",
"model_type": "XGBoost",
"intended_use": "pre-screening for MANUAL review",
"training_data": {"size": "2.4M"},
"performance": {"by_demographic": {"group_w": {"auc": 0.89}}},
"limitations": ["thin credit files"],
"ethical_considerations": [],
})
A machine-checkable card beats a beautifully-written PDF because the PDF gets read once, at launch, by the one person who already agreed with it -- while check_card runs in your CI pipeline on every retrain and fails the build the day someone quietly ships a model with no ethics section. Documentation that a computer can enforce is documentation that survives contact with a deadline ;-)
Right -- homework settled. Now let us go stand at the edge of the map.
Video generation today is roughly where image generation was two years ago: improving frighteningly fast, not yet reliable, and about to rearrange entire industries whether they like it or not.
The architecture behind systems like Sora extends the diffusion models we built in #84-85 from 2D into 3D. In stead of denoising a single image, the model denoises a whole stack of frames at once. The key piece is the spacetime transformer -- a transformer that chews on patches drawn from both the spatial dimensions (height, width) AND the time dimension. The first job is cutting a video into those spacetime patches:
import torch
import torch.nn as nn
class SpacetimePatchEmbed(nn.Module):
"""Cut a video into spacetime patches for a video transformer."""
def __init__(self, patch_size=16, temporal_patch=4,
in_channels=3, embed_dim=768):
super().__init__()
# A single 3D convolution does spatial AND temporal patching at once.
self.proj = nn.Conv3d(
in_channels, embed_dim,
kernel_size=(temporal_patch, patch_size, patch_size),
stride=(temporal_patch, patch_size, patch_size),
)
def forward(self, video):
# video: (batch, channels, frames, height, width)
x = self.proj(video) # (B, embed_dim, T', H', W')
x = x.flatten(2).transpose(1, 2) # (B, n_patches, embed_dim)
return x
Now the expensive question: how does attention work over all those patches? Full 3D attention over every patch against every other patch is quadratic in the TOTAL number of patches, which for even a short clip is ruinous. The trick that makes it affordable is factored attention -- do spatial attention within each frame first, then temporal attention across frames at the same position:
class VideoTransformerBlock(nn.Module):
"""Factored spacetime attention: spatial first, then temporal."""
def __init__(self, embed_dim=768, n_heads=12):
super().__init__()
self.spatial_attn = nn.MultiheadAttention(embed_dim, n_heads,
batch_first=True)
self.temporal_attn = nn.MultiheadAttention(embed_dim, n_heads,
batch_first=True)
self.norm_s = nn.LayerNorm(embed_dim)
self.norm_t = nn.LayerNorm(embed_dim)
self.mlp = nn.Sequential(
nn.Linear(embed_dim, embed_dim * 4), nn.GELU(),
nn.Linear(embed_dim * 4, embed_dim),
)
self.norm_m = nn.LayerNorm(embed_dim)
def forward(self, x, n_temporal, n_spatial):
B, N, D = x.shape
# Spatial: group patches by time step, attend within each frame
xs = x.reshape(B * n_temporal, n_spatial, D)
a, _ = self.spatial_attn(xs, xs, xs)
x = x + self.norm_s(a.reshape(B, N, D))
# Temporal: group patches by position, attend across frames
xt = (x.reshape(B, n_temporal, n_spatial, D)
.permute(0, 2, 1, 3).reshape(B * n_spatial, n_temporal, D))
a, _ = self.temporal_attn(xt, xt, xt)
x = x + self.norm_t(a.reshape(B, n_spatial, n_temporal, D)
.permute(0, 2, 1, 3).reshape(B, N, D))
return x + self.norm_m(self.mlp(x))
Many modern systems wrap this in DiT (Diffusion Transformer) blocks, where the conditioning -- your text prompt, the current denoising timestep -- modulates the network through adaptive layer normalization, exactly the flavour of conditioning we saw in the image-diffusion episodes.
What is NOT solved: temporal consistency (a dog should not grow a fifth leg between frame 12 and frame 40), physics (things should fall, collide, and pour like they mean it), and long-range coherence (a 60-second clip has to tell one story, not sixty). People are closing these gaps fast, but do not let a cherry-picked demo reel convince you they are closed. They are not, yet.
Generating a 3D object from a sentence sounds like it should need a giant 3D model trained on giant 3D datasets -- which barely exist. The clever escape hatch is Score Distillation Sampling (SDS): use a pretrained 2D image diffusion model as a critic, and let it teach a 3D thing to look correct.
The loop is genuinely elegant. Render your 3D object from a random camera angle, hand the 2D render to the diffusion model, and ask "does this look like the prompt?" Use its gradient to nudge the 3D representation, then rotate to a new angle and repeat -- thousands of times. Because you keep changing the camera, the only way the object can satisfy the critic from EVERY angle is to actually become a coherent 3D shape.
The 3D representation itself varies: Neural Radiance Fields (NeRFs, which we touched in #87), the newer and much faster 3D Gaussian Splatting, or plain old explicit meshes. The choice is downstream-driven -- games want meshes, visual effects might take NeRFs, real-time apps love Gaussian splats. This is not sci-fi; game-asset creation, product visualisation, architectural previews and virtual try-on are being built on text-to-3D pipelines right now, today.
I am going to be direct, because you deserve it: quantum machine learning (QML) is, for practical purposes, mostly hype at the moment. There. I said it. But the theory is genuinely interesting, so let me give you the honest version rather than either the breathless one or the dismissive one.
The premise is seductive. A quantum computer can represent and manipulate an exponentially large state space using superposition and entanglement, and a handful of ML problems -- high-dimensional distributions, certain nasty optimisation landscapes, specific kernel computations -- might one day get a real speedup from that. The current workhorse idea is the variational quantum circuit (VQC): a parameterised quantum circuit that behaves like a trainable model, with the parameters tuned by good old classical gradient descent.
import torch
class QuantumClassifier:
"""CONCEPTUAL hybrid quantum-classical classifier.
Not runnable without a quantum simulator (PennyLane / Qiskit)."""
def __init__(self, n_qubits=4, n_layers=3):
self.n_qubits = n_qubits
# trainable rotation angles for the quantum gates
self.params = torch.randn(n_layers, n_qubits, 3, requires_grad=True)
def quantum_circuit(self, x, params):
# 1. ENCODE: map classical features -> qubit rotations
# 2. PROCESS: apply layers of parameterised rotations + entangling gates
# 3. MEASURE: collapse the state; measurement probabilities ARE the output
raise NotImplementedError("needs quantum hardware or a simulator")
def forward(self, x):
return self.quantum_circuit(x, self.params)
Notice I left the circuit unimplemented on purpose -- that is not laziness, it is the honest state of the field for someone on a laptop. The blunt status: today's quantum machines are too noisy and too small to beat classical ML on any task you actually have. The theoretical advantages assume fault-tolerant machines with millions of clean qubits; we have hundreds of noisy ones. Might that change in 5-10 years? Maybe. Might it not? Also maybe. Keep half an eye on it, read a survey once a year -- but do NOT build your career on it this decade. If someone sells you a "quantum AI" product in 2026, keep your hand on your wallet ;-)
Conventional chips march to a synchronous clock and do floating-point arithmetic. Your brain does neither -- it fires asynchronous spikes between neurons and burns about 20 watts doing it. Neuromorphic computing builds hardware that works more like the brain, and its software counterpart is the spiking neural network (SNN).
In stead of the smooth continuous activations we have used all series, an SNN passes around binary spikes over time. The canonical unit is the Leaky Integrate-and-Fire neuron -- it accumulates incoming current, leaks a bit each step, and fires a spike when it crosses a threshold:
import torch
import torch.nn as nn
class LIFNeuron(nn.Module):
"""Leaky Integrate-and-Fire neuron -- one timestep at a time."""
def __init__(self, tau=0.9, threshold=1.0):
super().__init__()
self.tau = tau # leak: how much membrane potential survives
self.threshold = threshold
def forward(self, input_current, membrane=None):
if membrane is None:
membrane = torch.zeros_like(input_current)
membrane = self.tau * membrane + input_current # leak + integrate
spikes = (membrane >= self.threshold).float() # fire
membrane = membrane - spikes * self.threshold # reset what fired
return spikes, membrane
Stack those into a network and you process the input over many timesteps, then classify by counting output spikes:
class SpikingNetwork(nn.Module):
"""A small SNN for classification -- classify by spike count."""
def __init__(self, input_dim=784, hidden=256, n_classes=10, n_steps=25):
super().__init__()
self.n_steps, self.n_classes = n_steps, n_classes
self.fc1, self.lif1 = nn.Linear(input_dim, hidden, bias=False), LIFNeuron()
self.fc2, self.lif2 = nn.Linear(hidden, n_classes, bias=False), LIFNeuron()
def forward(self, x):
# rate-code the input: brighter pixel -> more likely to spike each step
trains = (torch.rand(self.n_steps, *x.shape) < x.unsqueeze(0)).float()
mem1 = mem2 = None
out = torch.zeros(x.size(0), self.n_classes)
for t in range(self.n_steps):
s1, mem1 = self.lif1(self.fc1(trains[t]), mem1)
s2, mem2 = self.lif2(self.fc2(s1), mem2)
out += s2
return out
The upside is real: spikes are binary, so neuromorphic silicon can be astonishingly frugal -- Intel's Loihi chip runs certain workloads at orders of magnitude less power than a GPU. The downside is also real: spikes are non-differentiable (the same wall we hit with binary activations, worked around with surrogate gradients), and the tooling and pretrained-model ecosystem is still thin. Where does it shine? Edge AI (#122) -- always-on, low-power sensing like keyword spotting, gesture detection and anomaly monitoring, where a milliwatt matters more than the last percent of accuracy. Niche, but a genuinely valuable niche.
MoE is not new -- the idea dates to the 1990s -- but it is having a full-blown renaissance because it answers one of the sharpest questions in modern AI: how do you make a model bigger WITHOUT making every forward pass proportionally more expensive?
The answer: in stead of one giant feedforward block, keep many smaller expert networks plus a learned router that picks which experts handle each token. Only a few experts fire per token, so your total parameter count can be enormous while the compute per token stays modest.
import torch
import torch.nn as nn
class MoELayer(nn.Module):
"""Mixture of Experts with top-k routing."""
def __init__(self, input_dim=512, expert_dim=2048, n_experts=8, top_k=2):
super().__init__()
self.n_experts, self.top_k = n_experts, top_k
self.router = nn.Linear(input_dim, n_experts)
self.experts = nn.ModuleList([
nn.Sequential(nn.Linear(input_dim, expert_dim), nn.GELU(),
nn.Linear(expert_dim, input_dim))
for _ in range(n_experts)
])
def forward(self, x):
B, S, D = x.shape
probs = torch.softmax(self.router(x), dim=-1) # (B, S, n_experts)
top_p, top_i = probs.topk(self.top_k, dim=-1) # pick k experts / token
top_p = top_p / top_p.sum(dim=-1, keepdim=True) # renormalise
out = torch.zeros_like(x)
for k in range(self.top_k):
idx, w = top_i[..., k], top_p[..., k:k+1]
for e in range(self.n_experts):
mask = (idx == e)
if mask.any():
out[mask] += w[mask] * self.experts[e](x[mask])
return out
Mixtral 8x7B is the classic example: about 46.7B total parameters but only ~13B active per forward pass (2 of 8 experts per layer), which buys it the quality of a much larger dense model at a fraction of the inference bill. The catches are engineering, not theory: load balancing (stop the router from lazily always picking the same two experts -- you add an auxiliary loss to spread the load) and communication overhead (when experts live on different GPUs, tokens have to travel). Both are solved well enough that MoE now sits inside a lot of frontier systems you have already used, whether the marketing told you or not.
If you make me pick the ONE idea on this page that already matters most in practice, it is this one -- and it is the least flashy of the bunch, which is exactly why I trust it.
For most of this series, scaling meant training-time compute: bigger model, more data, longer run. Test-time compute scaling adds a second dial. The insight (which ties straight back to reasoning in #142) is dead simple: harder problems deserve more thinking. So in stead of one forward pass and out, let the model chew -- generate several reasoning paths, check its own intermediate steps, search over candidate solutions, and keep the best one. Here is the cheapest honest version, best-of-N with a verifier:
def best_of_n(model, verifier, prompt, n=8):
"""Spend more compute at INFERENCE: sample n answers, keep the best-scored one.
`model` returns a candidate; `verifier` scores how good a candidate is."""
candidates = [model(prompt) for _ in range(n)] # n forward passes, not 1
scored = [(verifier(prompt, c), c) for c in candidates]
scored.sort(key=lambda pair: pair[0], reverse=True)
best_score, best = scored[0]
print(f"sampled {n} candidates; best verifier score = {best_score:.2f}")
return best
That is the whole trick in five lines: trade inference cost for quality. Systems like o1 and o3 do a far more sophisticated version of this, sometimes spending 10-100x the compute of a plain forward pass on a single hard query -- and the accuracy you get back follows a scaling law, just like training compute does. For a practitioner this quietly rewrites the economics. In stead of paying once to train a bigger model and eating that cost on every query forever, you can train a smaller model and spend extra compute ONLY on the queries that are actually hard. Easy questions stay cheap; hard questions get the thinking they need. That flexibility is why I think this is the frontier that changes your day job first.
Get your hands dirty before we start the final stretch of this series. Three tasks, climbing in difficulty:
Router entropy for MoE. Add a method to MoELayer that, given a batch, returns how often each expert was selected across all tokens, plus the entropy of that distribution. One sentence on why a HIGH entropy is what you want, and what a collapsing (low-entropy) router would mean for all those parameters you paid for.
A leaky-neuron dial. Take LIFNeuron and sweep tau from 0.1 to 0.99 while feeding it a constant input current. Plot or print how many timesteps pass between spikes at each tau, and write two sentences on what the membrane time constant is really controlling -- and why a spike-count classifier cares.
Budgeted best-of-N. Extend best_of_n so it takes a compute_budget and an easy/hard label per prompt, spending n=2 on easy prompts and n=16 on hard ones, and report the average verifier score per dollar of compute versus a flat n=8 baseline. Two sentences connecting what you find back to why test-time scaling beats "just train a bigger model" on a mixed workload.
We open the next episode with full solutions, exactly like always.
That is the horizon as I read it in the summer of 2026 -- part solid ground, part shifting sand, and I have tried to tell you honestly which is which. Next time we come back down off the mountain and put our hands to work: we take everything this series has taught and build something that is not a toy, something that actually MATTERS. Bring your editor and a pot of coffee ;-)
@scipio