After lots of planning, lots of debating with myself, and tons of tests, the Buttercloud is alive and well.
I've been watching it ingest 3Speak videos all day, waiting for it to break my heart, but so far, so good.
Long live the Butter, cloud.
For those few who have been following my misadventures with IPFS, this is not exactly news. We've been approaching the limit of how many videos 3Speak can store, and the problem has just been snowballing daily.
When I first began helping Eddie with 3Speak two years ago, one of the first things we did was clean house. Back in those days, playback was so poor that people had abandoned the platform. But cleaning house, deleting terabytes of ripped-off films in foreign languages, and other unmentionable things, only kicked the can down the road.
As the IPFS repo grew, the hard drives of the supernode began to fall behind. Having to crawl through the blocks in order for the Kubo daemon to pin and store a new hash, a new wrapped directory, was getting harder with every upload.
The Buttercloud solves this problem in the simplest way.
No more IPFS for cold storage.
No more crawling through pins.
No more unmaintainable repos.
You see, it seems to be a known issue, a known problem. A video platform with thousands of tiny little segment files is pretty much the best way to make sure you can't scale past 20 terabytes of data without sacrificing video delivery in the process.
And the problem is there already, even before you take into account our RAID, which we need, of course, for reliability and redundancy.
To say we've extracted every last drop from the technology may not be accurate.
But only because we had a drop or two left.
Long live the Butter, cloud.
MenO