I left you last time with a promise that I've been itching to keep all week: we'd take sound apart, grain by grain. Everything in this whole audio arc so far has treated sound as something you make - you pick an oscillator, you shape it with an envelope, you play a note. Even when we placed sounds in space last episode, we were moving whole sounds around, complete notes picked up and set down somewhere. Today we do the opposite. Today we start with a sound that already exists - a recording, a sample, a bit of the real world - and we chop it into thousands of tiny fragments and rebuild it into something the original never contained. Stretched. Frozen. Smeared into texture. It's one of my favourite corners of audio, and honestly it's a bit magic the first time it works :-).
And here's the thing that made me want to teach it right after the spatial episode: this is the exact same move we made way back in episode 10, when we took an image apart pixel by pixel and rebuilt it. Back then a picture stopped being a picture and became an array of numbers we could push around. Granular synthesis does that to sound. A recording stops being a fixed thing you press play on and becomes a cloud of particles you can rearrange. Same idea, different sense. Allez, let's get a sample loaded first.
Everything we've done so far has been synthesis - building sound from scratch out of oscillators (episode 119). Sampling is the other half of the world: you record or download a bit of audio and play it back. Tone.js makes loading a file about as easy as it gets with Tone.Player. Point it at a URL, wait for it to load, press go.
// Tone.Player: load an audio file and play it back.
const player = new Tone.Player({
url: "https://tonejs.github.io/audio/berklee/gong_1.mp3",
autostart: false,
}).toDestination();
// files load asynchronously - wait for the buffer before you trigger it.
Tone.loaded().then(() => {
console.log("sample ready!");
// now it's safe to play
});
That Tone.loaded() bit matters more than it looks. Audio files load over the network, so if you try to play the thing the instant the page opens you'll get silence and a confused half hour of debugging (ask me how I know). Always wait for the buffer. Once it's loaded, a plain player already gives you a surprising amount to play with.
// basic sample manipulation - already fun before any granular trickery.
player.start(); // play from the top
player.playbackRate = 2; // twice as fast... AND an octave up (like a sped-up tape)
player.reverse = true; // play the buffer backwards
player.loop = true; // loop it forever
player.fadeIn = 0.1; // little fade so loops don't click
Reverse alone is worth the price of admission - a reversed cymbal is that classic "swelling up into a hit" sound you've heard a thousand times. But notice the comment on playbackRate: crank it to 2 and the sound plays twice as fast and jumps an octave higher. Speed and pitch are welded together, exactly like a tape or a record. Spin the record faster, everything goes chipmunk. And that welding is the exact problem granular synthesis was invented to solve. See where this is going?
Here's a thing you'll want constantly in creative work: take a 2-second sound and make it last 8 seconds - without dropping it three octaves into a slow-motion growl. Slow a sample down the normal way and it turns into a monster. Speed it up and it turns into a mouse. There's no way to pull time and pitch apart with a plain player, because playing a buffer slower literally means moving through it slower, which literally means longer wavelengths, which literally means lower pitch. They're the same knob.
So how do you stretch time while keeping the pitch? You cheat. You stop playing the sound as one continuous stream and instead chop it into tiny overlapping fragments - grains - and you control how fast you march through the recording seperately from how each little grain sounds. That decoupling is the whole idea, and it's genuinly one of the cleverest tricks in audio.
A grain is a very short slice of sound - typically somewhere between 10 and 100 milliseconds. That's short. At 20ms you can barely tell what a grain "is" on its own; it's more of a click or a blip than a sound with character. The magic is in quantity. Play hundreds of these tiny grains per second, overlapping, each fading in and out with its own little envelope, and your ear stops hearing individual grains and starts hearing a continuous texture again.
// a single grain is just a very short slice with a fade-in/fade-out envelope.
// on its own it's a blip. the trick is playing HUNDREDS of them, overlapping.
const grainConcept = {
grainSize: 0.03, // ~30ms - each slice of sound
envelope: "each grain fades in then out so it doesn't click at the edges",
overlap: "grains overlap so the stream sounds continuous, not choppy",
theBigIdea: "control HOW FAST you move through the sample seperately from PITCH",
};
This is the pixel thing again, I keep coming back to it because it's such a clean parallel. A photo up close is just thousands of coloured squares that mean nothing individually - step back and your eye fuses them into an image. A grain cloud is thousands of sound-blips that mean nothing individually - and your ear fuses them into a texture. Granular synthesis is pointillism for your ears. Once that clicked for me the whole technique stopped feeling like a black box.
Tone.js ships a GrainPlayer that does all the grain-chopping and envelope-fading internally. From the outside it looks almost like a normal player - but it hands you two knobs that a normal player can never give you: playbackRate (how fast you move through the sample) and detune (pitch), and they are finally, gloriously, independent.
// Tone.GrainPlayer: playbackRate and pitch are now SEPARATE knobs.
const grain = new Tone.GrainPlayer({
url: "https://tonejs.github.io/audio/berklee/gong_1.mp3",
grainSize: 0.1, // 100ms grains
overlap: 0.05, // how much neighbouring grains cross-fade
loop: true,
}).toDestination();
Tone.loaded().then(() => grain.start());
Two parameters do the heavy lifting here and they're worth really understanding, because they're the difference between "smooth stretch" and "robotic mush":
grainSize - how long each slice is. Big grains (100ms+) preserve more of the original character but smear transients; tiny grains (20ms) get glitchy and metallic. This is your main "texture" dial.overlap - how much each grain crossfades into the next. More overlap = smoother, more washed; less overlap = choppier, more rhythmic and grainy.Fiddling those two against each other is 90% of the art, and there's no "correct" setting - it's entirely about the sound you're chasing. But now watch what the decoupling actually buys us.
This is the payoff the whole technique was built for. With a GrainPlayer, playbackRate changes only how fast you travel through the recording. Set it below 1 and the sound stretches out - lasts longer - but every grain is still played at its original pitch, so nothing goes slow-motion-monster on you.
// TIMESTRETCH: slow the sample WAY down, pitch stays put.
grain.playbackRate = 0.25; // takes 4x longer to play through... but same pitch!
// the sound is stretched into a long, evolving drone - no chipmunk, no monster.
// speed through it fast, also without pitch change:
grain.playbackRate = 3; // races through the sample, pitch unchanged
The first time you drop playbackRate to 0.1 on a spoken word or a bit of music, it's a proper "ohhh" moment - a half-second sound unfurls into this long shimmering cloud, still recognisably itself but stretched into slow motion like time got taffy-pulled. That's the sound of a hundred experimental records you've half-heard. Now the other knob:
// PITCH SHIFT: change pitch WITHOUT changing length. detune is in cents (100 = 1 semitone).
grain.detune = 1200; // up one octave, same duration
grain.detune = -700; // down a perfect fifth, same duration
grain.detune = 300; // up a minor third
Time on one knob, pitch on the other, fully independent. That's the thing a tape deck physically cannot do and a grain cloud does for free. You can stretch a sound to four times its length and pitch it up an octave at the same time, a combination that simply doesn't exist in the physical world of spinning things. Makes sense, right?
Here's my favourite trick in the whole episode. If playbackRate controls how fast you move through the recording... what happens at zero? You stop moving. But the grain engine keeps firing grains - from whatever spot you're parked on. The result is a freeze: one instant of the recording, sustained forever, turned into a stationary drone. You've frozen a single moment of sound into a pad.
// FREEZE: park playbackRate near zero and the grain engine keeps sounding
// the CURRENT position forever - one frozen moment turned into an endless drone.
grain.loop = true;
grain.playbackRate = 0.0001; // effectively frozen in place
// now SCRUB through the recording by hand by nudging where we're frozen.
// (loopStart is in seconds into the buffer.)
function freezeAt(seconds) {
grain.loopStart = seconds;
grain.loopEnd = seconds + 0.1; // a tiny 100ms window we sit inside
}
freezeAt(1.2); // hold the texture found 1.2s into the sample
Wire that freezeAt to the mouse X position and you've built a scrub instrument: drag left and right and you sweep through the sound, freezing on whatever grain you land on, each spot a different frozen timbre. Drag slowly across a piano chord sample and you glide through the harmonics of a single struck note pulled out into eternity. This is the kind of thing that feels like a toy for five minutes and then like an instrument for five years.
The GrainPlayer hides the machinery, which is great for getting sound out fast - but I always understand a thing better once I've built the stripped-down version myself. So let's make a little grain cloud manually, scheduling short slices of a buffer over and over. This is exactly what's happening under the hood, just visible.
// a manual grain cloud: repeatedly play tiny slices from random positions in a buffer.
let buffer;
Tone.loaded().then(() => { buffer = player.buffer; });
function playGrain(positionSec, grainSec) {
const g = new Tone.Player(buffer).toDestination();
g.fadeIn = grainSec / 2; // envelope: fade in over half the grain...
g.fadeOut = grainSec / 2; // ...and out over the other half (no clicks)
g.start(Tone.now(), positionSec, grainSec); // play [positionSec .. +grainSec]
}
That start(time, offset, duration) call is the key - it plays just a duration-long window starting at offset seconds into the buffer. One call = one grain. Now we spray them: fire a new grain every few milliseconds, each from a slightly random spot, and the overlapping blips fuse into a shimmering cloud. Because we're scattering the read position, we get that lovely smeared, frozen-but-alive texture.
// spray grains: many per second, each from a jittered position around a centre.
let centre = 1.0; // read position in the buffer (seconds) - the "playhead"
const spread = 0.08; // how far grains wander from the centre
const grainSize = 0.08; // 80ms grains
const cloud = new Tone.Loop((time) => {
const jitter = (Math.random() - 0.5) * 2 * spread; // random offset around centre
playGrain(Math.max(0, centre + jitter), grainSize);
}, 0.02); // a new grain every 20ms = 50 grains/second
Tone.Transport.start();
cloud.start(0);
Fifty grains a second, each an 80ms slice jittered around a centre point. Move centre slowly and the cloud drifts through the recording; hold it still and you get a rich frozen texture that shimmers because of the random jitter rather than sitting dead. Widen spread and grains pull from further apart, smearing the sound into pure texture where you can't tell what the source even was. Narrow it and the original starts poking through. This handful of lines is granular synthesis - everything else is polish.
I want to zoom out for a second, because granular synthesis quietly changes how you think about sound. Up to now in this arc, sound has been made of notes - discrete pitched events with a start and an end (episode 120, when we taught the machine to choose them). Granular invites you to stop thinking in notes at all and start thinking in material. A grain cloud isn't a melody, it's a substance - a fog, a shimmer, a rough grinding surface, a soft wash. You sculpt it the way you'd sculpt clay, not the way you'd write a tune.
// the same source sample becomes wildly different TEXTURES depending on grain settings.
const textures = {
smoothWash: { grainSize: 0.2, overlap: 0.15, spread: 0.02 }, // creamy, blended pad
shimmer: { grainSize: 0.06, overlap: 0.04, spread: 0.05 }, // sparkly, glassy cloud
glitchGrind: { grainSize: 0.015, overlap: 0.0, spread: 0.1 }, // harsh, metallic, broken
frozenDrone: { grainSize: 0.1, overlap: 0.08, spread: 0.0 }, // one held moment
};
// SAME recording, four completely different worlds. that's the power of it.
The reason this matters for creative coding specifically: texture is generative gold. A note is a decision you have to make ("which pitch, when?"). A texture is a process you set running and shape with a couple of numbers - which is exactly the kind of thing we love to drive with noise fields, slow sine waves, mouse position, or data (episode 79). Wire a Perlin noise value (episode 12, all the way back) to spread and grainSize and your texture breathes and evolves on its own, never quite repeating. That's an ambient bed (episode 122) that came from a real recording instead of oscillators.
Let's build something you'll actually keep. The brief: load any sample, freeze into it, and let the mouse drive the whole thing. Mouse X scrubs the read position through the recording; mouse Y controls the grain size (top = tiny glitchy grains, bottom = big smooth ones). Draw the playhead position on screen so the eyes agree with the ears. It's a proper little instrument in about thirty lines.
// GRANULAR TEXTURE PAD - mouse X = position, mouse Y = grain size.
const pad = new Tone.GrainPlayer({
url: "https://tonejs.github.io/audio/berklee/femalevoice_aa_A4.mp3",
grainSize: 0.1,
overlap: 0.1,
loop: true,
playbackRate: 0.0001, // frozen - we scrub by hand
}).toDestination();
let ready = false;
Tone.loaded().then(() => { ready = true; });
Now the p5 side. On mouse move we map the cursor into the buffer and into a grain size, and we keep a tiny loop window so we're always frozen on the spot under the mouse. Click starts the audio (never forget Tone.start() - we've been burned by that one enough times this arc).
// map the mouse into the sound. click to start (Tone needs a user gesture).
function mousePressed(p) {
Tone.start();
if (ready) pad.start();
}
function scrub(p) {
if (!ready) return;
const dur = pad.buffer.duration; // length of the sample in seconds
const pos = p.map(p.mouseX, 0, p.width, 0, dur); // X -> position in the recording
pad.loopStart = Math.max(0, Math.min(dur - 0.05, pos));
pad.loopEnd = pad.loopStart + 0.05; // tiny window = frozen on this spot
// Y -> grain size: top of screen = tiny glitchy, bottom = big & smooth.
pad.grainSize = p.map(p.mouseY, 0, p.height, 0.015, 0.25);
}
And finally draw it, so there's something to look at while you play. A vertical line for the playhead, and let the grain size colour the background so the visual texture matches the sonic one - fine grains read as sharp and bright, coarse grains as soft and dark. Eyes and ears telling the same story, the thread that's run through this whole arc.
// draw the playhead + let grain size tint the scene, so picture matches sound.
function draw(p) {
scrub(p);
const fine = p.map(p.mouseY, 0, p.height, 1, 0); // 1 near top (fine grains)
p.background(20 + fine * 40, 20, 40 - fine * 20);
p.stroke(255, 180, 90);
p.strokeWeight(2);
p.line(p.mouseX, 0, p.mouseX, p.height); // the playhead, where we're frozen
p.noStroke();
p.fill(255);
p.text(ready ? "drag to scrub - click first" : "loading...", 12, 20);
}
Drag it around and you'll feel the instrument in your hands - the voice sample smears into vowels held forever, glitches into metal near the top of the screen, melts into a warm pad near the bottom. Sit with it. When you want to push it: add a second GrainPlayer on a different sample and cross-fade between them with a key, wire spread/jitter to a Perlin field so the texture drifts on its own, or run the whole thing through the Panner3D from last episode so your frozen cloud sits somewhere out in space around your head. Same handful of parts, endless places to take it. It's yours now :-).
Tone.Player and play it back. Always wait for Tone.loaded() before you trigger it, or you'll debug silence for half an hourplaybackRate = 2 plays twice as fast AND an octave up, like a sped-up tape. There's no way to pull them apart... normallyTone.GrainPlayer finally decouples the two knobs: playbackRate controls how fast you travel through the recording, detune controls pitch, and they're fully independent. Stretch a sound to 4x length with the pitch untouched, or pitch it up an octave without changing its length - things a tape physically can't dograinSize and overlap are the texture dials: big grains + lots of overlap = smooth wash; tiny grains + no overlap = glitchy metallic grind. There's no "right" setting, it's all tasteplaybackRate near zero and the engine keeps sounding the current spot forever - one frozen moment turned into an endless drone. Scrub the frozen position with the mouse and you've built an instrumentplayer.start(time, offset, duration) slices sprayed from jittered positions) shows the whole machine in a dozen lines - everything else is polishSo that's sound taken apart and put back together as texture. We've now built single sounds, chosen notes, added rhythm, drifted into ambient beds, locked visuals to sound in time and in space - and now we can grab any recording of the real world and melt it into whatever shape we want. That's a genuinly complete toolkit for making sound with code.
But there's a gap I keep bumping into, and maybe you've felt it too: everything we've built lives inside the browser tab, talking only to itself. Our code makes sound, our code makes pictures, and they hold hands - but nothing outside can reach in, and our sound can't reach out. What if a physical knob on a desk could drive that grain size? What if a note played on a real keyboard triggered our cloud, or our visuals sent messages to a completely separate program running the audio? There are standard languages for machines and instruments to talk to each other over, and once your creative code can speak them, the browser tab stops being a sealed box and becomes one instrument in a much bigger room. That's where we go next :-).
Sallukes! Thanks for reading.
X
@femdev