Part of the process within the technology world is when information moves from the highly technical/science pages to the mainstream media.
When it comes to data, the growing problem is now appearing on the latter. Even the NY Times is getting into the act, covering what we have detailed.
Artificial intelligence, specifically large language models (LLM), require a great deal of data. This comes in the form of text, photos, and video.
Fortunately, the Internet, which we can say started in 1991 with the World Wide Web, is a major data center. Over the last 3 decades, enormous volumes of data was posted. This really took off with the popularity of social media. When we consider the fact there are 500 million tweets added per day, billions of photos on Instagram, and roughly 275K hours of video uploaded to YouTube.
If this is the case, how can there be a data shortage?
This is what we will dive into with this article while also detailing how we can alter this outcome.
Image generated by VeniceAI
For years, users of social media knew they were the product. Any data posted to a social media platform is property of the company behind it.
Over much of the Internet life, these entities processed data and sold it. The most common buyer were advertisers, where targeted messages became the norm.
With the advent of LLM, things changed a great deal. Suddenly these companies were sitting on a gold mine of information. They had decades worth of content to feed through the system.
At the same time, the Internet was basically open. Starts ups such as OpenAi could go and scrape the Internet. This means taking the information from other sites and using it to train these models.
It is this move that is getting OpenAi sued by just about everyone. They are facing mountains of lawsuits, something that is far from resolved.
The value of data was realized.
For this reason, we see the response:
The study, which looked at 14,000 web domains that are included in three commonly used A.I. training data sets, discovered an “emerging crisis in consent,” as publishers and online platforms have taken steps to prevent their data from being harvested.
The researchers estimate that in the three data sets — called C4, RefinedWeb and Dolma — 5 percent of aLL data, and 25 percent of data from the highest-quality sources, has been restricted. Those restrictions are set up through the Robots Exclusion protocol, a decades-old method for website owners to prevent automated bots from crawling their pages using a file called robots.txt.
This is very concerning.
We saw what happened last year when both Elon Musk and Reddit caused an uproar by limiting access to their APIs. This was done in an effort to reduce the ability to scrape their systems. While some high volume users were likely affected, those trying to extract data were mostly stopped.
This allowed Reddit to turn around and sell the data. One deal was for $60 million with Google.
The challenge here is that we are rapidly moving to a time when many companies are excludes.
Essentially, Big Tech as all the data.
But widespread data restrictions may pose a threat to A.I. companies, which need a steady supply of high-quality data to keep their models fresh and up-to-date.
They could also spell trouble for smaller A.I. outfits and academic researchers who rely on public data sets, and can’t afford to license data directly from publishers. Common Crawl, one such data set that comprises billions of pages of web content and is maintained by a nonprofit, has been cited in more than 10,000 academic studies, Mr. Longpre said.
Of course, the ones who are protected the data are not the users who actually created it. Instead, it is the mega corporations that own and run the platforms. After all, the data is on their servers which makes it theirs.
That said, even these entities are likely to hit a wall. Here is where a major problem arises.
If much of the newer data is locked behind pay walls, then we see how the information provided by the LLMs is going to be inaccurate.
Then there is the bigger problem: the exclusion of smaller firms (start ups).
How many companies can afford to give Reddit $60 million? This is where Big Tech has another advantage. If Silicon Valley behemoths have to pay for the data, not a major issue. However, this basically excludes new entrants, making the LLM world limited to a handful of companies.
The advantage to most blockchains is they are open, permissionless databases. When dealing with a blockchain like Hive, anyone can set up an API.
Here is where people need to start questioning what they are doing on a daily basis.
Unfortunately, people do not realize what they are dealing with. If data is fully in the control of a few major technology firms, what do you think the world will look like. Many are fearful of the "skynet" scenario. If that is the case, why feed them with more power?
Ultimately, this is what we are dealing with.
When adding to an owned, controlled database, we are simply feeding that entity. Data is lining up to be power. This is something that we must keep in mind.
How do we spread the power of the future around? This is done by simply adding to database that are not owned by the usual suspects. Here is where Web3 offers an alternative.
This is a topic that is going to keep getting more attention over the next 12-18 months. It is not a situation that will magically improve. The value of data is going on, meaning the industry will become fragmented along the lines of the haves and have nots.
If this does not sound appealing to you, then it is time to stop feeding Big Tech as much as possible and divert your data to those networks which are open and can be used by anyone.
Here is where Web 3.0 is democratizing data.
