Yesterday I listened to a very interesting podcast [source] ... It was one of those self-reflection ones in which a journalist wrote about how people, including journalists, are falling into traps with their eyes open.
The title:
"There's more data than ever. But when does it actually make you smarter?"
The podcast itself is a spoken version of an article [source] published in my most favourite digital newspaper 'De Correspondent'.
Why my most favourite? It is a platform with tons of highly professional journalists without an ad-revenue model. They earn their money with subscriptions and 'side-hustles' such as book publishing. Many of their own journalists write books and publish through their own publishing house. Perhaps not much of a side hustle anymore, thinking about it. The crux of what I wanted to say: "Nobody outside the platform can drive these journalists to conform, no censoring, no pressured bias, unless they do this to themselves, for which no proof points exist thus far."
In less than 15 minutes, Simon van Teutem (the author and podcast voice) tells the story that even he and his colleagues, in and outside the company, are falling into the trap of believing statistics. Statistics in whatever subject and statistics about whatever topic, without questioning the research behind it.
Although I was about to fall asleep when this article reached my ears, my beauty sleep had to wait. Instantly and automatically, the words by this guy pulled all my attention.
Why?
Well, because I tend to do the same! Most often, I do ask myself the question when some statistical number comes to me: "Is this really the case?" But then I leave the question unanswered and continue with the next statistic about another topic or subject, instead of spending the time to research the source, the situation and the conditions in which the research was carried out. Turns out: Often the research behind the numbers is much different to what that single number, encapsulated in a headline sentence, tries to convey and/or what we take away from such a number.
70% of the youngsters are depressed
Some time ago, a good friend told me: "Did you know that about half of the Dutch population is poor?" She took this from mainstream media that used headlines such as:
Since I kinda knew the official figures about how many people in my country are considered poor, I immediately wanted to go against this statement. But I realised this would quickly turn into: "I just read it this morning!", "Where do you get your numbers from?" kinda discussion. Quickly, I decided to lay low and responded with: "Owww Geee, that is a mighty bunch of people!" or something along those lines. Yesterday I was reminded of this event and today I did some serious digging into the 42-page research report [source].
Turns out the numbers are based on a survey, a list of questions asked to 2.515 citizens/households. Many questions ask for the 'feeling'. Not many questions were about exact numbers. Interesting to read: 85% of respondents feel they didn't have any trouble paying their bills last year, while 39% of respondents are classified as having some to extreme difficulties paying their bills.
The report itself does not analyse these discrepancies. The writers seem to focus on a section of questions at a time, without analysing conflicting results from different sections of questions in the total survey.
Uhm ... "Perhaps a standard method in survey analyses?" ... was what jumped into my mind. I don't know. I am not at all a survey specialist. Perhaps I shall investigate this as well. Darn, so much to dive into before we can judge a statistical summary in the form of a single parameter/value pair.
Owww BTW, official numbers for the amount of people in poverty in my country, is: 3.1% [source] - Be very very careful. You may want to research the report behind this number 😱 ... and YES, I know: Poverty isn't the same as the feeling of people not being able to pay their bills. Also, to classify people living in poverty is to follow a list of criteria, which may not be the same criteria as the 'list of criteria' - in quotes, because it was all about feeling instead of exact numbers - in the survey under discussion here. Interesting to see is the fact that a decade ago the stats were kinda the same as for this most recent survey. In between, society did a bit better, with a trendline to the positive side of things, followed by the dip in 2026 back to levels of 10+ years ago.
Long story short, lesson learned: Never just assume some numbers or stats are correct! Always question these. Or at least, use with care. When such numbers are used in situations that can lead to considerable effects, always research before driving those effects.
I wish I were the prime minister of my country right now. That would've given me the chance to drive considerable effects. But maybe I am able to drive some effects? Here in our HIVE community? The realisation with you - the reader - that statistical data thrown at us day in day out, shall be handled with care?
