The core mission of SBM comes down to a few things – examining the complex relationship between scientific evidence and healthcare, exploring issues of how optimally to regulate health care and health products, and fighting misinformation. Over the years I think we have made some solid strides on the first category. The medical infrastructure has been trending increasing towards higher standards of scientific evidence and greater awareness of the many pitfalls of bad science or pseudoscience. More of my colleagues know what p-hacking is, and the higher quality journals are exploring ways to improve the quality of scientific evidence.
The other two broad areas have not been trending well. Regulations remain a mixed bag and overall have been ratcheting in the wrong direction. The forces of quackery are relentless at promoting their interests, and have the ill-gotten gains to back their efforts. These efforts include watering down science-based standards, carving out exceptions for specific pseudosciences, and legitimizing pseudoscientific practitioners, practices, and products.
Regarding misinformation, it’s hard not to feel as if we are massively losing the battle. I do think the situation would be far worse were it not for the efforts of fact-checkers and analysts trying to set the record straight, but at least over the medium term misinformation is clearly on the rise. A lot of this is likely due to social media and algorithms optimized for engagement rather than quality of information. Also, the purveyors of misinformation are just getting better at what they do, and have been building infrastructure to extensively spread their misinformation and disinformation and work it into the mainstream conversation.
I think they have also figured out what we have known for a long time at SBM (and the broader skeptical community) – that misinformation is the endgame. Unfortunately, mainstream institutions have not figured this out, although they might be slowly waking up to this reality. It doesn’t matter if your claims or practice are backed by super high quality science, or if you have thoughtful and effective regulations, if the public believes abject nonsense. Carl Sagan warned about this in his Cosmos series in the 1980s – if the public does not understand and support science, eventually the barbarians will come and tear it all down. I think that is what we are experiencing right now.
Therefore, understanding misinformation and how best to limit and counter it is critical. Along those lines, a recent study looks at who is most susceptible to misinformation. A recent study uses a validated online test, the MIST – misinformation susceptibility test, which includes information on education, gender, age and political leanings, to see which demographics are most susceptible. You can take the test yourself here.
In the study they looked at data from 66,242 individuals from 24 countries. They found:
“Multilevel modelling showed that Generation Z, non-male, less educated, and more conservative individuals were more vulnerable to misinformation. Furthermore, while individuals’ confidence in detecting misinformation was generally associated with better actual discernment, the degree to which perceived ability matched actual ability varied across subgroups. That is, whereas women were especially accurate in assessing their ability, extreme conservatives’ perceived ability showed little relation to their actual misinformation discernment. Meanwhile, across all generations, Gen Z perceived their misinformation discernment ability most accurately, despite performing worst on the test. “
These results create more questions than answers, because the why is what is most important. There are a few points of interest. One, there was no apparent “Dunning-Kruger” effect, meaning that overall people who rated their ability to discern fake from true headlines were actually better at it. In fact, Gen Z performed the worst but were the most accurate in their discernment. Women also were below average in performance but above average in discernment.
The most difficult category to interpret, in my opinion, is the political leanings. While the test seemed politically balanced, it’s hard to account for all possible confounding factors. They used, for example, a large language model to generate the headlines, and the training data may have been biased. What I think this means is that we cannot know if there is something inherent about conservatives that makes them more believing of fake headlines, or if current conservative ideology simply aligns more with the kinds of fake headlines that would be created by an LLM trained on the internet. To be clear, I think this is a real effect (because it seems to keep cropping up in research) but it likely reflects the political culture rather than the people.
The fact that the younger generation is more susceptible could simply reflect less life experience, but it may also be a generational effect. Spending lots of time of social media, for example, there appears to be a “post-truth” culture in the younger generation. We may simply be seeing the effects of a generation growing up with social media.
A recent meta-analysis of 31 experiments using the fake headline paradigm found similar results:
“We found that older adults, Democrats (compared to Republicans), and those with higher analytical thinking skills show greater discrimination ability. Ideological congruency (alignment of participants’ ideology with news), motivated reflection (higher analytical thinking skills being associated with a greater congruency effect), and self-reported familiarity with news were linked with a true-news bias.”
This data also finds that being older and more liberal predicts better discriminatory ability. Higher analytical skill also did, which aligns with higher education found in the current study. But also they found that if someone was familiar with the claim being made in the headline they were more likely to believe it (whether it was true or not). This means, unfortunately, that simply repeating misinformation can be extremely effective.
These types of studies are useful, but they are just a starting point. There is also a body of research about countering misinformation. The quick version is – it requires not only fact checking but scientific literacy, critical thinking skills, and media savvy. You essentially have to give someone a new way to understand the world. You can’t just tell them their belief is wrong.
But the bigger question of how we “win” the misinformation wars (whatever a win looks like) is a much tougher nut to crack. Social media companies have essentially given up (and I don’t think their hearts were ever truly in the effort). There is no political will or consensus for any serious measures. And artificial intelligence is about to make the situation orders of magnitude worse.
For now the best we can to is to fight misinformation with good information, and to do what we can to promote scientific literacy and critical thinking skills.
The Misinformation Wars