Video Subtitles on 3Speak

I’m gonna start this post off with a big bold statement: We need capability for video subtitles (CC) on 3Speak! My goal with this article is to make very clear why this feature should be a priority to implement.

I will also offer a solution for how to achieve the benefits of this functionality a bit further down with concrete examples, as a workaround. I will make suggestions on how the technical implementation could be. So keep reading!

Open Captions vs. Closed Captions

Before we dive into the arguments, there are a couple of terms that need clarification, so that we’re all on the same page as to what exactly we’re talking about.

There are two types of captioning: Open Captions and Closed Captions.

Open Captions is when the subtitles are “burned” into the video.
Closed Captions (usually known simply as “CC”) is the kind of subtitles that the user can turn on and off at will.

In the case of Open Captions, the subtitles are embedded into the video as images, which means that we get a sequence of images of text rather than cleartext. This has many downsides, as it makes the subtitles unavailable for any type of further processing after they’ve have been embedded into the video.

For these reasons it is preferable to use CC in almost all cases.

Why even bother with this though?

The main key word is Accessibility. How do you know the words spoken in a video if there are no subtitles and you cannot hear? If you’re actually deaf, then forget about it. But it doesn’t need to be that dramatic. Perhaps you’re in a crowded/noisy place and got no headphones. Or maybe you just have trouble focusing on the contents of audio alone. In all those cases, the the presence of a CC-button is crucial.

What if you’re a non-English speaker (or a non-spanish speaker for that matter — many videos on 3Speak are in spanish). You’d be out of luck.

A decentralized video platform should by definition be for everyone. However, with today’s situation regarding CCs, it is arguably not.

Accessibility is a defined requirement

It is defined in the Web Content Accessibility Guidelines (WCAG) that all videos should have a CC alternative, to make its contents as accessible as possible. For this reason, public institutions across Europe is actually required by law to have this option for all videos they publish. This of course extends to educational institutions – both public and private.

The fact that it’s required by law does of course not oblige 3Speak as a decentralized platform to implement it in any way. But it does say a lot about how big of a deal this is. And it sadly also excludes 3Speak from serving as the hosting platform of any kind of video in a public/formal manner anywhere in Europe – and most likely in other places too.

Benefits of Closed Captions

Flexibility: The ability to turn subtitles on and off at will, makes the video more available to watch in different settings.
Translation/Across language barriers: Because the subtitles/transcript is available in cleartext, users can easily translate it to their own languages using browser plugins. See example.
For the deaf/hearing impaired: In this situation the content is simply not available to you if you don’t have subtitles.
AI training: This is actually a huge deal; There are currently several different AI solutions that are currently in the process of being implemented (or already are) on various Hive front-ends. Without cleartext video transcriptions, the contents of these videos is more or less unavailable for the AI. With all the insights and valuable discussions taking place in the many podcasts being published to 3Speak, this would be a huge untapped potential/loss.

Workaround Pilot Project: Lion’s Den

If you didn’t already know, there’s a weekly HIVE/LEO podcast called “Lion’s Den” that takes place every Friday, and is hosted by @taskmaster4450. Many important discussions take place there every week, with many interesting guest appearances from prominent community members. These discussions should be available to all, not just for English speakers with perfect hearing.

So, as a pilot I have – in agreement with him – started posting these episodes to 3Speak with embedded Open Captions + the full transcription in the comments section of the posted episode. We’re already seeing the benefits of this through the lens of Portugese speaking community member @coytelation’s recent thread, where he shows us how he’s using a browser plugin to translate the entire episode to Portugese. All because of the presence of the episodes’ transcript in cleartext.

Suggestions for Implementation of CC’s on 3Speak

There are - the way I see it - two different sides/parts of the implemention off CCs on 3Speak, and one is a lot more important (and hopefully easier) than the other.

1. The ability to attach subtitle files to videos

This is the most crucial feature. With the rise of speech-to-text generation tools like like Whisper AI, any person with a decent graphics card are now able to run speech-to-text detection on videos they are posting, creating very good subtitle files without much trouble. It needs to be possible to manually upload subtitle files (SRT, VTT, SUB for instance), and attaching them to the videos as you upload them to 3speak - as Closed Captions (CC).

Note: Whisper AI happens to be the one I’m using to create the subtitles for the episodes, in combination with my self-made open source application Hive ASR Dictionary. (Feel free to ask me about any of this btw!)

2. Transcription nodes running Speech to Text software

This is more of a dream scenario. What if you could incentivize nodes to do speech-to-text transcriptions of any video. You may have noticed that subtitles is automatically generated on any video uploaded to YouTube. How amazing would it be to make this a part of the incentive structure on SPK Network?

This is of course not crucial, and I realize it probably would require a lot of work to implement. But nonetheless I think it’s an idea worth entertaining, at the very least.

If you found this insightful, feel free to leave a comment, upvote or reblog.