I've been working this week on finishing the Butter Cloud. The idea behind it all is that we could go back to implementing S3 technology in 3Speak without adding costs to our infrastructure.
I had begun working on Butter Cloud many months ago, and I had an MVP. But of course, one thing is to upload an image or two, and another one entirely is to serve videos.
With that said, the biggest challenge here is to incorporate S3 into our infrastructure without completely abandoning IPFS.
The ingest flow is done through IPFS, and it seems more than obvious that if we are to incorporate Speak Network in the future, IPFS is the only viable answer. But it's almost like dancing on top of rocks without shoes.
At every turn, with every decision, I have to keep in mind that whatever we build has to remain open enough to work with IPFS somewhere in the mix.
How do we inject, if you will, decentralization into centralization? A weird question, I admit, but a valid one right now.
The more I think about it, the more I think I need to build a new service. Something I'm thinking of calling Video Pulse. A service responsible for detecting organic plays on videos, both new and old, and keeping those videos on the Hotnode as long as they continue to get played, until they eventually go dormant.
I thought for a minute about incorporating this somewhere into our current stack, but it seems specialized enough to live on its own.
Another challenge that needs solving, and this is because we're literally running 3Speak on a shoestring budget, is how we can add redundancy to our S3 bucket.
IPFS, by its very nature, offers redundancy. Gateways can cache content, and that's before any CDN even gets involved. But S3? That's a different story.
This means that before going live, I'll need to decide how to double up our new MongoDB deployment. It needs a replica for safety. I'll also need to figure out how to double up the Butter Cloud Node server itself, because we can't trust a single VPS to never fail.
Seems like every step I take, I end up writing ten more issues for myself. Ten more questions that need answers.
Like a good friend once said to me:
These projects grow tentacles.
MenO