Last time we did something that still feels a little bit like witchcraft: we stopped designing pictures directly and instead designed the pressure - a population, a fitness score, a rule for who breeds - and let evolution go find the picture for us. Nobody typed those final blobs, they were found. Allez, today we pull on that exact same thread, but from a completely different corner of the room. Because there's an even older idea in this series that also makes complexity out of nothing: cellular automata. Way back in episode 48 we built Conway's Game of Life, where every cell is one bit, alive or dead, and one tiny hand-written rule turns a random grid into gliders and guns and whole little machines. And in episode 92 we let a neural network run in the browser. Today we smash those two together and get one of the most beautiful ideas in the whole field: neural cellular automata. Cells that don't follow a rule we wrote - cells that follow a rule they learned.
I want to be honest about why this one grabbed me by the collar. For years I thought of Game of Life and neural networks as living on opposite planets. One is this crisp, discrete, almost mechanical toy. The other is all soft weights and gradients and matrix soup. Then I saw the "Growing Neural Cellular Automata" work (Mordvintsev and co, 2020) and my brain did a little backflip, because it turns out they're the same shape underneath. A CA is just "look at your neighbours, compute your next state". A neural net is just "take some numbers, compute some other numbers". So what if the CA rule is a tiny neural network? What if instead of me writing if neighbours == 3 by hand, a little net decides how each cell should change? Let me show you what I figured out, and by the end you'll have a grid of cells growing and healing themselves on your canvas.
Here's the mental jump you have to make first, because everything hangs off it. In Game of Life a cell holds one bit - alive or dead. That's it. In a neural CA, a cell holds a whole little vector of numbers. We call them channels. The first four are the ones you can see - red, green, blue, and alpha (how "alive" the cell is). The rest are hidden channels: secret scratch memory the cells use to talk to each other and coordinate. The cell can stash whatever it wants in there.
// a neural CA grid: W x H cells, each holding C channels (numbers).
// channels 0..3 are R,G,B,A (visible). 4..C-1 are hidden "memory".
const W = 96, H = 96, C = 12;
const grid = new Float32Array(W * H * C); // one flat buffer, all zeros to start
// tiny helpers to read/write channel k of cell (x, y)
function get(state, x, y, k) {
return state[(y * W + x) * C + k];
}
function set(state, x, y, k, v) {
state[(y * W + x) * C + k] = v;
}
Notice we store the whole thing as one flat Float32Array. That's not me being fancy, it's the only way this stays fast - we're going to touch every channel of every cell many times a second, and a flat typed array is about as quick as JavaScript gets. We used pixel buffers exactly like this back in the pixel-manipulation episode, so the shape should feel familiar. A grid is just a big bag of floats, and how you read it is what gives it meaning.
Game of Life starts from a random soup. Neural CA usually starts from something almost absurdly minimal: one single cell, sitting alive in the middle of a completely dead grid. Everything has to grow out from that one seed. That constraint is the whole magic - the pattern has to organise itself from nothing, using only local information.
// seed a single living cell in the centre.
// alpha = 1 means "fully alive". hidden channels start at 1 too, as a little kick.
function seed(state) {
state.fill(0);
const cx = (W / 2) | 0, cy = (H / 2) | 0;
for (let k = 3; k < C; k++) set(state, cx, cy, k, 1); // alpha + hidden = 1
}
seed(grid);
One dot. From that one dot, if the rule is any good, a whole structured pattern will bloom outward. Makes sense so far? Good, because now we need the two halves of the actual rule: how a cell senses its surroundings, and how it decides what to do about them.
A cell can't see the whole grid - it only knows itself and the eight neighbours around it, exactly like Life. But instead of just counting live neighbours, a neural CA "perceives" its surroundings through little filters. Three of them, per channel. First, the cell's own value (the identity filter - just "what am I right now"). Second and third, the gradient in x and in y, which tells the cell which way each channel is increasing. Those gradients are computed with Sobel filters, the same edge-detecting kernels photographers' software uses under the hood.
// the three perception kernels. identity = "myself".
// sobelX / sobelY = "which way is this channel sloping" (a gradient).
const IDENTITY = [0, 0, 0, 0, 1, 0, 0, 0, 0];
const SOBEL_X = [-1, 0, 1, -2, 0, 2, -1, 0, 1];
const SOBEL_Y = [-1, -2, -1, 0, 0, 0, 1, 2, 1];
Why gradients? Think about it for a second. If a cell knows "my alpha is climbing steeply to my left", it knows there's a wall of living tissue over there and empty space to the right. That's the cell figuring out where it sits on an edge, purely from local slope. Gradients are how a blind cell feels the shape it's part of. Now let me apply one of these 3x3 kernels to one channel at one cell - a plain little convolution.
// convolve one 3x3 kernel over channel k at cell (x, y).
// out-of-bounds neighbours are treated as 0 (dead border).
function conv(state, x, y, k, kernel) {
let sum = 0, i = 0;
for (let dy = -1; dy <= 1; dy++) {
for (let dx = -1; dx <= 1; dx++) {
const nx = x + dx, ny = y + dy;
if (nx >= 0 && nx < W && ny >= 0 && ny < H) {
sum += get(state, nx, ny, k) * kernel[i];
}
i++;
}
}
return sum;
}
Now the full perception step: for every channel, run all three filters, and stack the results into one long vector. If we have 12 channels and 3 filters each, that's a perception vector of length 36. That vector is everything the cell knows about its little neighbourhood, squeezed into a row of numbers.
// build the perception vector for one cell: 3 filters x C channels.
// [identity of every channel, then x-gradient of every channel, then y-gradient].
function perceive(state, x, y) {
const p = new Float32Array(3 * C);
for (let k = 0; k < C; k++) {
p[k] = conv(state, x, y, k, IDENTITY);
p[C + k] = conv(state, x, y, k, SOBEL_X);
p[2 * C + k] = conv(state, x, y, k, SOBEL_Y);
}
return p;
}
Here's the heart of it. In Life, the rule was hand-written: three neighbours and you're born, two or three and you survive, else you die. In a neural CA, the rule is a small neural network that takes the perception vector in and spits a delta out - a little nudge to add to each of the cell's channels. It's two dense layers: perception (36 numbers) into a hidden layer with a ReLU in between, then the hidden layer down to C numbers of change. If episode 92 is fuzzy, a dense layer is genuinly just "multiply the inputs by a matrix of weights and add a bias". That's the whole animal.
// a dense layer: out = ReLU(input . W + b) (optionally skip the ReLU).
// W is stored as a flat [inSize * outSize] array, row per input.
function dense(input, W, b, inSize, outSize, relu = true) {
const out = new Float32Array(outSize);
for (let o = 0; o < outSize; o++) {
let sum = b[o];
for (let i = 0; i < inSize; i++) sum += input[i] * W[i * outSize + o];
out[o] = relu ? Math.max(0, sum) : sum; // ReLU = clamp negatives to zero
}
return out;
}
That Math.max(0, sum) is the ReLU - the little bit of non-linearity that lets the network learn something more interesting than a straight line. Without it, stacking two layers would collapse into one and the whole thing could only ever do boring linear stuff. With it, the net can carve up its input space into regions and behave differently in each. It's a tiny ingredient with a huge effect. Now the update network itself, wiring two dense layers together:
// the update net: perception (3C) -> hidden (H) -> delta (C).
// last layer has NO relu, because a nudge can be negative (cells must shrink too).
const HID = 96;
function updateNet(p, net) {
const h = dense(p, net.w1, net.b1, 3 * C, HID, true);
const d = dense(h, net.w2, net.b2, HID, C, false);
return d; // the change to add to this cell's channels
}
Where do the weights come from? Ah, that's the whole story, and I'll be straight with you: in the real thing they come from training, exactly the gradient-descent process we met with neural nets in episode 92. You show the network a target picture, let the CA run for a few dozen steps, measure how far the result is from the target, and backpropagate the error to nudge every weight. Do that thousands of times and the weights slowly become a rule that grows that specific image from one seed. Training a big one really wants a GPU and a library, so I'm not going to pretend we'll do it live in a blog post. But here's the lovely part - the forward pass, the running of the thing, is completely runnable in plain JavaScript, and even with hand-set or random-ish weights you get living, twitching, self-organising texture. So let me build a small net we can actually run.
// make a small net with tiny random-ish weights.
// tiny weights = gentle nudges = the grid stays stable instead of exploding.
function makeNet() {
const rand = (n) => {
const a = new Float32Array(n);
for (let i = 0; i < n; i++) a[i] = (Math.random() * 2 - 1) * 0.5;
return a;
};
return {
w1: rand(3 * C * HID), b1: new Float32Array(HID),
w2: rand(HID * C), b2: new Float32Array(C),
};
}
Here's a subtle thing that turns out to be really important. In Game of Life, every cell updates at exactly the same instant - one global tick, everybody at once. Neural CA deliberately breaks that. Each step, only a random subset of cells actually apply their update; the rest sit still this round. Why on earth would we want that? Because perfect synchrony is fragile and, honestly, unrealistic - real cells in a real body don't share a global clock. Making updates asynchronous forces the pattern to be robust: it has to work no matter what order cells happen to fire in. It's a small change with a big payoff in stability.
// stochastic update mask: each cell flips a coin, ~50% update this step.
// returns true if THIS cell should apply its delta this round.
function shouldUpdate() {
return Math.random() < 0.5;
}
One more guard rail, and it's a pretty one. We don't want dead empty space spontaneously flickering to life all over the grid - growth should only happen at the edge of what's already alive. So we define a cell as "alive" if it or one of its neighbours has an alpha above a small threshold. Cells that are alive and cells next to alive cells get to update. Everything else stays firmly dead. This is what keeps the pattern a coherent growing blob instead of noise fizzing everywhere.
// a cell counts as alive if its 3x3 neighbourhood has any alpha > 0.1.
// this confines growth to the border of the living region.
function alive(state, x, y) {
for (let dy = -1; dy <= 1; dy++) {
for (let dx = -1; dx <= 1; dx++) {
const nx = x + dx, ny = y + dy;
if (nx >= 0 && nx < W && ny >= 0 && ny < H) {
if (get(state, nx, ny, 3) > 0.1) return true; // channel 3 = alpha
}
}
}
return false;
}
Now we glue it all together into a single step. Read from the current grid, write into a fresh one (double-buffering, so a cell never sees a half-updated neighbour). For each cell: check it's alive, perceive its surroundings, run the net to get a delta, and - only if the coin says so - add that delta to its channels. Then swap buffers. This function is the whole simulation.
// advance the world one step. reads `state`, writes into `next`, returns `next`.
function step(state, net) {
const next = state.slice(); // start next = copy of current
for (let y = 0; y < H; y++) {
for (let x = 0; x < W; x++) {
if (!alive(state, x, y)) continue; // dead + not on an edge: skip
if (!shouldUpdate()) continue; // stochastic: sit this round out
const p = perceive(state, x, y);
const d = updateNet(p, net);
for (let k = 0; k < C; k++) {
let v = get(state, x, y, k) + d[k]; // apply the nudge
v = Math.max(-1, Math.min(1, v)); // clamp so nothing blows up
set(next, x, y, k, v);
}
}
}
return next;
}
That clamp on the last line is doing quiet heroic work again, same as the gene clamp in the evolution episode. Neural feedback loops love to explode - a cell nudges its neighbour up, which nudges it back up, and within ten steps you've got infinities everywhere and a white screen. Pinning every channel into a sane range means the system can churn forever without detonating. Clean contracts between steps, ask me how many white screens taught me that one.
The grid is all abstract floats until we look at it, and looking is easy because we reserved the first four channels for exactly this. Channel 0,1,2 are red, green, blue; channel 3 is alpha. We walk the grid, pull those four numbers per cell, map them into 0..255, and blit the whole thing to the canvas through a pixel buffer - same putImageData trick from the Voronoi episode.
// render channels 0..3 (RGBA) of every cell into the canvas.
function render(ctx, state) {
const img = ctx.createImageData(W, H);
for (let y = 0; y < H; y++) {
for (let x = 0; x < W; x++) {
const a = Math.max(0, Math.min(1, get(state, x, y, 3))); // alpha
const idx = (y * W + x) * 4;
img.data[idx] = (get(state, x, y, 0) * 0.5 + 0.5) * 255 * a;
img.data[idx + 1] = (get(state, x, y, 1) * 0.5 + 0.5) * 255 * a;
img.data[idx + 2] = (get(state, x, y, 2) * 0.5 + 0.5) * 255 * a;
img.data[idx + 3] = 255;
}
}
ctx.putImageData(img, 0, 0);
}
Notice we multiply the colour by alpha, so dead cells (alpha 0) render as black background and only living tissue shows its colour. Little detail, big difference in how clean it looks.
Last piece - the loop. Each frame we take a step and paint the result. Because our canvas is only 96x96, we let CSS scale it up big and crunchy so you can actually see the cells. Open this and you'll watch that single seed cell start pushing colour and structure out into the void.
// run the world: one step + one render per animation frame.
const canvas = document.querySelector("canvas");
canvas.width = W; canvas.height = H;
canvas.style.width = "480px"; // scale up: 1 cell -> 5 screen px
canvas.style.imageRendering = "pixelated";
const ctx = canvas.getContext("2d");
const net = makeNet();
let world = grid;
function frame() {
world = step(world, net);
render(ctx, world);
requestAnimationFrame(frame);
}
frame();
Now, real talk about what you'll see. With random weights you won't grow a neat little lizard - you'll get living, crawling, self-organising texture that spreads from the seed and churns. It's mesmerising and it's alive in a way Game of Life never quite manages, because there's so much more state per cell. To grow a specific target image you have to train the weights, and that's a whole project on its own. But sit with this for a minute, because you're watching the same deep idea as last episode from a new angle: we didn't design the pattern, we designed the machinery and the pressure, and structure grew on its own. Evolution found pictures over generations; here a learned local rule grows one in a few hundred steps. Same music, different instrument.
There's one property of trained neural CA I have to tell you about even though we won't train one today, because it's the thing that made me sit back in my chair. Once a neural CA has learned to grow a shape, it doesn't just grow it and stop - it maintains it. And if you take a knife to the pattern, erase half the cells, delete a chunk out of the middle, the survivors notice their neighbourhood is wrong and regrow the missing part. Nobody programmed a repair routine. The same local rule that grows the thing from a seed also heals it from damage, because to every cell, "damaged" just looks like "not finished growing yet". Here's the tiny experiment - punch a hole and watch:
// erase a circular chunk of the world - then let step() run and watch it heal.
function damage(state, cx, cy, radius) {
for (let y = 0; y < H; y++) {
for (let x = 0; x < W; x++) {
const dx = x - cx, dy = y - cy;
if (dx * dx + dy * dy < radius * radius) {
for (let k = 0; k < C; k++) set(state, x, y, k, 0); // wipe this cell
}
}
}
}
Regeneration falling out of a rule that only ever wanted to grow - that's emergence at its most jaw-dropping, and it's the same lesson episode 61 hammered on: rich global behaviour from dumb local rules, with nobody in charge. A trained net turns that from a slogan into a living, self-repairing thing on your screen.
So that's neural cellular automata - we took Game of Life's "look at your neighbours and update", swapped the one-bit cell for a vector, and swapped the hand-written rule for a little neural network. The single takeaway, plainly: a CA rule doesn't have to be something you write, it can be something a network learns, and once it learns, it grows and heals like something alive. Get the forward pass running and just watch the texture crawl out of that one seed - it's honestly one of the most alive things you can put on a canvas with this little code.
And keep the thread in your back pocket, because it matters for what's coming. Twice now - evolution last time, learned CA today - we've built systems where we shape the machinery and the pressures and let structure emerge rather than drawing it ourselves. That way of thinking, of growing complexity instead of placing it, is exactly what we need when we stop making single images and start generating whole worlds, terrains and structures that build themselves. The tools that grow those worlds start with richer sources of controlled randomness than the plain noise we've used so far - and that's precisely where we head next. 't Was plezant to grow a little living grid with you :-).
Sallukes! Thanks for reading.
X
@femdev