Okay. Deep breath. This is the one all the audio episodes have been sneaking up on. For weeks now we've built pieces - synths, melodies, chord changes, drum patterns, drifting ambient beds, then a way to lock a picture to a sound within that magic 40-millisecond window, and last time a visualizer that pulls five separate musical features out of a track and paints each one on its own lane. Lovely bits, all of them. But bits. Today we stop making demos and make a piece. One thing, with a beginning and a middle and an end, that you could sit a friend down in front of and say "here, watch this for two minutes" - and the sound and the picture are so bound together that neither makes sense without the other. Allez, today we compose :-).
I want to be careful about a word here, because it matters. A demo runs the same forever - it's a loop, it's a texture, it never goes anywhere. A composition has form. It starts somewhere, it builds, it peaks, it lets you down gently. Music has known this for centuries (intro, verse, chorus, bridge), and the second you give a generative system a sense of form it stops feeling like a screensaver and starts feeling like it meant something. So that's really the whole lesson today: not new tricks, but how to arrange the tricks we already have into something that has a shape.
Back in episode 123 I showed you the most elegant way to marry sound and image - don't have one react to the other, generate both from a single source. A shared noise field feeding the melody's pitch and the picture's colour at the same time, so they're siblings from one parent rather than a graphic nodding along to a speaker. Today we take that idea and make it the whole architecture of the project.
Here's the shape of it. In the middle sits one little object I'll call the state - a handful of numbers describing "what is the piece feeling right now": how much energy, how bright, how dense, what key we're in. Nothing in that object knows anything about sound or pictures. It's just mood, as numbers. Then two engines read it: an audio engine that turns mood into notes, and a visual engine that turns the same mood into pixels. Neither engine talks to the other. They only ever look at the shared state.
// the heart of the whole piece: a shared "mood" that both engines read.
// nothing here knows about sound OR pixels. it's just the feeling, as numbers.
const state = {
energy: 0, // 0..1 - how much is happening (drives density + loudness)
brightness: 0.5, // 0..1 - dark/warm .. light/cool (drives timbre + colour)
density: 0, // 0..1 - how busy the melody + the particles are
root: 0, // 0..11 - the current key (a pitch class, episode 120)
section: "intro", // which movement of the piece we're in
};
Why bother with this middle layer instead of just wiring the synth straight to the canvas? Because it's the thing that lets a piece have form. If both engines only ever read state, then to change the whole feel of the piece - sound and picture together, in lockstep - I only have to change those five numbers. One knob turns, both worlds turn with it. That's the trick that makes the composition feel like a single living thing instead of two things that happen to be in the same tab.
So who changes those numbers? A little conductor I'll call the director. This is basically a state machine (hello episode 17) that walks the piece through its movements over time - and it's the single most important new idea today, because it's what turns a loop into a story.
I like to think of a piece in four movements: an intro that's sparse and quiet and makes you lean in, a build where energy climbs and things get busy, a peak where it's full and loud and everything's happening, and an outro that empties back out and lets you breathe. Each movement is really just a target for those state numbers, plus how long it lasts.
// the director walks the piece through movements. each is a TARGET mood + a duration.
// this is a state machine (episode 17) whose "states" are musical sections.
const score = [
{ name: "intro", bars: 8, energy: 0.15, brightness: 0.3, density: 0.2 },
{ name: "build", bars: 16, energy: 0.6, brightness: 0.5, density: 0.6 },
{ name: "peak", bars: 16, energy: 1.0, brightness: 0.8, density: 0.95 },
{ name: "outro", bars: 8, energy: 0.1, brightness: 0.4, density: 0.1 },
];
Notice I gave each movement a target, not a fixed value. Because if the piece just snapped from intro's energy of 0.15 straight to build's 0.6, it'd lurch - one bar sleepy, next bar frantic. What we actually want is for the state to ease toward each movement's target over the bars it's given. And you already know how to do that - it's our oldest friend, the lerp from episode 16, doing exactly what it always does: move a value smoothly toward a target a fraction at a time.
// ease the live state toward the current movement's target, a little each bar.
// this is lerp (episode 16). the piece GLIDES between moods, never lurches.
function approach(current, target, rate) {
return current + (target - current) * rate; // classic lerp step
}
function updateState(target) {
state.energy = approach(state.energy, target.energy, 0.08);
state.brightness = approach(state.brightness, target.brightness, 0.05);
state.density = approach(state.density, target.density, 0.08);
}
That 0.08 is a "how fast does the mood turn" dial - small numbers glide, bigger numbers snap. Different rates per property is a nice touch too: I let brightness drift slower than energy, so the colour of the piece lags a touch behind its intensity, which feels organic, like a room slowly changing temperature. Makes sense, right?
The director needs to know what bar we're on so it can pick the right movement and know when to hand over to the next one. And we already have a perfect clock for that from the whole Tone.js stretch - the transport. We count bars, figure out which movement that bar falls into, and feed its target into updateState. All of it fires through Tone.Draw.schedule so the mood changes land dead in sync with the music, exactly the lesson from episode 123.
// one loop per bar drives the whole director. counts bars, picks the movement.
let bar = 0;
function currentMovement(barNum) {
let acc = 0;
for (const mv of score) {
acc += mv.bars;
if (barNum < acc) return mv; // first movement whose range we're inside
}
return score[score.length - 1]; // past the end -> stay in outro
}
const conductor = new Tone.Loop((time) => {
const mv = currentMovement(bar);
Tone.Draw.schedule(() => {
state.section = mv.name;
updateState(mv); // ease the mood toward this movement
}, time);
bar++;
}, "1m").start(0); // "1m" = one measure/bar, our macro pulse from episode 123
That's the skeleton done, and honestly it's the part people skip and shouldn't. With just this - a shared mood and a director easing it through four movements - you already have form. Everything from here is dressing the skeleton: making the sound read the mood, making the picture read the same mood, and letting them feed a little energy back the other way.
Now the fun bit. The audio engine's whole job is: look at state, make notes that match it. We built every piece of this over the audio arc, we're just letting the mood drive the dials instead of us. Let me set up the voices first - a poly synth for melody and chords, a soft membrane for the kick, all the way back from episode 118.
// the voices. we built all of these across the audio arc; now the MOOD plays them.
const lead = new Tone.PolySynth(Tone.Synth).toDestination();
const pad = new Tone.PolySynth(Tone.Synth).toDestination();
const kick = new Tone.MembraneSynth().toDestination();
pad.volume.value = -14; // the pad sits underneath, quiet (episode 122's bed)
lead.volume.value = -6;
// a pentatonic scale can't sound wrong (episode 120). we transpose it by state.root.
const PENTA = [0, 2, 4, 7, 9]; // semitone offsets from the root
function noteFor(step, octave, root) {
const pc = PENTA[step % PENTA.length] + root; // pitch class within the key
return Tone.Frequency(48 + pc + octave * 12, "midi").toNote(); // 48 = C3
}
Now the melody. On each eighth note we maybe play a note - and how likely we are to play, plus how high and busy it is, comes straight from state.density and state.energy. Sparse intro? Density is low, so most steps stay silent and the melody breathes. Peak? Density is near 1, notes tumble out. The mood is literally deciding how much music happens.
// the lead reads the MOOD. density decides how often we play, energy how high.
const melody = new Tone.Loop((time) => {
if (Math.random() < state.density) { // sparse when calm, busy when peaking
const step = Math.floor(Math.random() * 5); // which scale degree
const octave = state.energy > 0.7 ? 1 : 0; // climb an octave at high energy
const note = noteFor(step, octave, state.root);
lead.triggerAttackRelease(note, "8n", time, 0.3 + state.energy * 0.5);
// stash the event for the visuals to pick up - shared state, not a direct call:
Tone.Draw.schedule(() => spawnNoteMark(note, state.brightness), time);
}
}, "8n").start(0);
See that last line? The melody doesn't draw anything. It drops a note-mark into the shared world and the visual engine will find it. The two never touch directly. And the kick follows energy the same way - below a threshold it stays quiet, and as energy climbs past it the beat kicks in, so the drums literally arrive when the build arrives.
// the kick only shows up once the piece has some energy. the beat ARRIVES with the build.
let flash = 0;
const beat = new Tone.Loop((time) => {
if (state.energy > 0.35) {
kick.triggerAttackRelease("C1", "8n", time, state.energy);
Tone.Draw.schedule(() => { flash = state.energy; }, time); // beat -> a pulse for the visuals
}
}, "4n").start(0);
And a slow chord underneath, changing key every few bars so the harmony drifts - this is the ambient-bed thinking from episode 122, layered on a much slower cycle than the melody so the piece feels like it's moving at two speeds at once.
// a slow pad chord, and every 4 bars we nudge the key. slow harmonic drift (episode 122).
const chords = new Tone.Loop((time) => {
const triad = [0, 2, 4].map((s) => noteFor(s, 0, state.root)); // root+third+fifth of the key
pad.triggerAttackRelease(triad, "1m", time, 0.4);
Tone.Draw.schedule(() => {
if (bar % 4 === 0) state.root = (state.root + 5) % 12; // move a fourth up, wrap in the octave
}, time);
}, "1m").start(0);
Right, mirror image. The visual engine reads the exact same state object and turns it into a picture - and because it's reading the same numbers the audio engine is reading, the two can't drift apart. When energy climbs, the sound gets busy and the screen gets busy, together, because they're both just looking at state.energy. That's the whole payoff of the shared-mood design.
First the note-marks that the melody was dropping. Each one is a little particle (episode 11), coloured by brightness at the moment it was born, that fades out over time.
// the visual side of a note. brightness at birth picks the hue (episode 127's centroid idea).
const marks = [];
function spawnNoteMark(note, brightness) {
const hz = Tone.Frequency(note).toFrequency();
// pitch -> height, in LOG space because pitch is logarithmic (episode 123):
const t = (Math.log2(hz) - Math.log2(130)) / (Math.log2(1050) - Math.log2(130));
marks.push({
x: Math.random() * width,
y: (1 - t) * height, // high notes ride high on screen
life: 1,
hue: 200 - brightness * 180, // dark mood -> warm, bright mood -> cool
});
}
Then the draw loop itself, which is pure rendering - it only reads state that the synced callbacks wrote, never makes a sound decision of its own. That clean split (audio writes, visuals read) is the single biggest thing keeping the whole piece locked and un-tangled.
// draw: PURE rendering. reads the shared mood + the marks. makes zero audio decisions.
function draw(p) {
// background darkness tracks brightness - the whole room dims and lifts with the mood
const bg = 8 + state.brightness * 22;
p.background(bg);
// beat flash, eased down every frame (episode 16). staccato hit, legato fade.
flash *= 0.88;
if (flash > 0.01) { p.noStroke(); p.fill(255, flash * 50); p.rect(0, 0, p.width, p.height); }
p.colorMode(p.HSB, 360, 100, 100, 1);
for (let i = marks.length - 1; i >= 0; i--) {
const m = marks[i];
m.life -= 0.012;
const size = (4 + state.energy * 26) * m.life; // energy makes every mark bigger
p.noStroke();
p.fill(m.hue, 70, 95, m.life);
p.circle(m.x, m.y, size);
if (m.life <= 0) marks.splice(i, 1);
}
}
Here's a lovely finishing move that ties the knot properly. So far the mood drives both worlds, which is already unified. But we can add one gentle feedback path so it feels alive: let the sound that came out nudge the visuals a hair, the way we learned to read a live signal in episode 127. I run the master output through an analyser, pull a single loudness number, and let it add a tiny bit of extra size to everything - so on top of the composed mood, the picture also breathes with the actual audio in real time.
// one feedback strand: the REAL output loudness nudges the visuals (episode 127).
// composed mood does the big shape; live audio adds the fine breathing on top.
const meter = new Tone.Analyser("waveform", 256);
Tone.Destination.connect(meter);
function liveLoudness() {
const buf = meter.getValue(); // -1..1 samples
let sum = 0;
for (let i = 0; i < buf.length; i++) sum += buf[i] * buf[i];
return Math.sqrt(sum / buf.length); // RMS - the "felt" loudness, same as last episode
}
You fold that into the draw loop wherever you want a touch of life - I usually add it into the mark size, (4 + state.energy * 26 + liveLoudness() * 40) * m.life, so the particles have a slow composed swell and a fast twitch on the actual sound. Two timescales again, the composed and the live. That's the same "sync the twitch and the arc" idea from episode 123, only now one strand is planned and the other is reactive.
One more thing, and it's what turns this from a toy into an artwork you can actually keep. Right now every run is different because we're calling Math.random() all over. Sometimes that's exactly what you want - a piece that's never the same twice. But sometimes you find a version you love and want to be able to play it again, or share the exact one. That's what seeds are for (episode 24) - swap the raw random for a seeded generator and the entire piece, sound and picture, becomes reproducible from one number.
// a seeded RNG (episode 24). same seed -> the EXACT same two-minute piece, every time.
function mulberry32(seed) {
return function () {
seed |= 0; seed = (seed + 0x6D2B79F5) | 0;
let t = Math.imul(seed ^ (seed >>> 15), 1 | seed);
t = (t + Math.imul(t ^ (t >>> 7), 61 | t)) ^ t;
return ((t ^ (t >>> 14)) >>> 0) / 4294967296;
};
}
const rng = mulberry32(20260720); // <- your seed IS your composition
// then replace every Math.random() above with rng(). seed 20260720 always plays the same piece.
The really nice thing: because both engines pull from the same seeded rng, a given seed pins down the notes and the visuals together. Seed 20260720 isn't "a random tune with some random dots" - it's this piece, this exact two minutes of sound-and-light, forever, and you can hand someone the seed like you'd hand them a record.
You'll want to keep the good ones, and we learned how in the recording episode (episode 20) and tightened it in episode 123 - grab the canvas as a video stream, grab the audio context as an audio stream, merge them, and let MediaRecorder write a single file with picture and sound already in sync.
// capture the whole piece - canvas + audio - into one .webm (episodes 20 & 123).
const vid = canvasEl.captureStream(60);
const aud = Tone.getContext().createMediaStreamDestination();
Tone.Destination.connect(aud);
const rec = new MediaRecorder(
new MediaStream([...vid.getVideoTracks(), ...aud.stream.getAudioTracks()]),
{ mimeType: "video/webm" }
);
const chunks = [];
rec.ondataavailable = (e) => chunks.push(e.data);
rec.onstop = () => saveBlob(new Blob(chunks, { type: "video/webm" }));
rec.start(); // start it, let the 48 bars play, call rec.stop() in the outro
Kick that off as the intro begins, let the director walk the whole 48-bar score, and stop it when the outro empties out - and you've got a self-contained little film of a piece that composed itself. That's genuinly a shareable thing, not a screenshot of a demo.
So here's the brief, and it's the biggest one we've done - but you have every single piece already. Build a two-minute generative audiovisual composition with real form: the shared state mood in the middle, the director easing it through intro-build-peak-outro, the audio engine turning mood into pentatonic melody plus a slow drifting pad plus a beat that arrives with the build, and the visual engine turning the same mood into note-marks and a background that dims and lifts. Seed it so you can replay the ones you love. Record your favourite.
The bar to clear isn't "does it look cool" - a random loop can look cool for ten seconds. The bar is: does it go somewhere? Sit and watch the whole two minutes. Does the intro make you lean in? Does the build actually feel like it's building - more notes, brighter colour, the kick sliding in? Does the peak feel full? Does the outro let you down soft instead of just stopping? If a friend watched it with no idea it was code, would they feel a shape? That's the whole game today, and when you hit it, it's a genuinly different feeling from anything we've made - because for once the thing isn't running, it's unfolding.
And once you've got a piece that unfolds on its own... you might start itching to reach into it while it plays. To nudge the energy up with a keypress right as the beat drops, to swap the scale live, to perform the system instead of just pressing go and watching. That itch - the leap from a composition that plays itself to one you play live, in front of people, changing the rules while it runs - is exactly the thread I want to pull next. Build your two minutes first, genuinly sit with the whole arc of it and get that outro landing soft, and then come back, because next time we stop being the composer and start being the performer :-).
state object (energy, brightness, density, key) sits in the middle knowing nothing about sound or pixels - an audio engine and a visual engine both read it, so they can never drift apart. This is episode 123's "one source, two faces" idea scaled up to the whole projectTone.Draw.schedule (episode 123) so mood changes land dead in sync with the music, on the transport's bar clockMediaRecorder merging canvas + audio (episodes 20 and 123) into one fileSo that's the mini-project, and really it's the whole audio arc landing in one place. Every episode since we first opened Tone.js was a brick; today we finally built the house. You can make a system that has a shape - that leans in, climbs, fills up, and lets go - and grow the sound and the picture from one shared mood so they move as a single living thing. That's a genuinly big place to have arrived: you've gone from "make a beep" to "compose two minutes of audiovisual art that composes itself", and it's all just the same handful of ideas - noise, lerp, state, sync - stacked with a bit of taste on top.
Sallukes! Thanks for reading.
X
@femdev