We have to deal with artificial intelligence (AI). We cannot simply dismiss it or criticize it, and we certainly shouldn’t just embrace it without significant skepticism. But I feel it is necessary for any science communicator to do their best to become AI-literate – even if AI itself is not your field of communication. AI now touches everything.
For example, increasingly people are searching the internet using AI. About 43-48% of Google searches trigger an AI summary, AI chat tools handle over 50% of search volume, and 37% of users begin their search with an AI tool. Further, many people have “discussions” with chatbots about a topic they are interested in to learn more about it. Increasingly we are getting our information filtered through AI, or explicitly composed by AI. This can have several concerning effects.
Several problems are well-documented, including that chatbots can hallucinate. It is important to understand that large language models (LLMs), the technology on which chatbots are built, are not really logic machines, they are prediction machines. Their output can be uncanny, but they do not really understand anything. They are really good at predicting language responses, partly because they are heavily trained on massive sets of data. LLMs are generative – they generate text based on their training data and internal algorithms. What this means it they can entirely make stuff up by generating text that is not true but resembles information which is out there on the internet and in their training data. They can even make up fake references, all properly formatted.
Even when information they provide is not a strict hallucination, it is just reflecting the weight of information on the internet. So in essence you are asking – what are most people saying about this on the internet? This may be fine for simple facts or uncontroversial topics, but don’t necessarily expect a rigorous analysis of a controversial topic.
I say “necessarily” because you might get a good analysis. It largely depends on your prompt and your history with your chatbot. LLMs not only reflect the biases in their training data, they reflect the biases of the user. Often this amounts to what is called the “sycophancy problem” – LLMs are programed to please their users, they are effusive with praise, and tend to “yes and” their users. This means you can have a conversation with a chatbot, and think that you are getting useful feedback and even criticism, but there is a massive bias toward affirming your biases and praising your ideas. In the end you may be convinced that you have created something brilliant and useful, when the AI has essentially just gaslighted you into believing so.
All of the above amounts to a serious limitation of current chatbots, but they can be mitigated to a large degree by a savvy user. You can prompt a chatbot to be skeptical, to not be sycophantic, to make the strongest case for the other side, to be harshly critical – and all of this will affect the outcome. AI, ultimately, is a powerful but flawed tool, and can be useful if used carefully, thoughtfully, and skeptically. I am not saying you should use it, just that outcomes vary wildly based upon the habits of the user.
Which leads to the final problem with AI, which I call the oracle problem. Many users are not savvy, and they use chatbots as if they are an unassailable oracle of definitive information. Increasingly I am getting e-mails or messages from those defensing a specific pseudoscience, and their “evidence” is output from a chatbot. Here is an example, from Dana Ullman, a homeopathy apologist we have discussed here many times.
Ullman, in my opinion, is a classic pseudoscientist – he tries to portray what he is doing as scientific, but he gets it consistently an horribly wrong. What prompted his most recent e-mail to me was an accusation that we are “censoring” him because we are afraid. In actuality his comment was simply flagged as spam by Discus (a common occurence, as is the knee-jerk accusation of fear-based censorship).
One of his e-mails contained this statement:
“One of the strongest statements verifying the efficacy of homeopathic medicines was the confirmation that four of the five leading previous systematic reviews of homeopathic research found a benefit from homeopathic treatment over that of placebo: ‘Five systematic reviews have examined the RCT research literature on homeopathy as a whole, including the broad spectrum of medical conditions that have been researched and by all forms of homeopathy: four of these ‘global’ systematic reviews reached the conclusion that, with important caveats, the homeopathic intervention probably differs from placebo.’
Mathie RT, Lloyd SM, Legg LA, Clausen J, Moss S, Davidson JR, Ford I. Randomised placebo-controlled trials of individualised homeopathic treatment: systematic review and meta-analysis. Systematic Reviews 2014; 3:142. doi:10.1186/2046-4053-3-142″
Essentially he is proving our case while naively thinking he is proving his own. All he is demonstrating is that he does not understand the nature of scientific evidence. He thinks this is the “strongest” case for homeopathy – a luke warm review from 2014. He did not quote the statement from the conclusion: ” The low or unclear overall quality of the evidence prompts caution in interpreting the findings.”
All he can do is quote “low or unclear quality” evidence, as the best case for homeopathy. I recount this to demonstrate his clear biases. He also states in this e-mail chain:
“I challenge you to debate ChatGPT on the subject of explaining how homeopathic nanodoses CAN have biological and physiological effects. YOU can even do some homework by reading 3 of my previous published conversations with ChatGPT at my blog.
I am not asking you to debate me (you’re too afraid)…but heck, are you AFRAID of ChatGPT?”
What do you get when a pseudoscientist prompts a chatbot – pseudoscience. But always game, I prompted ChatGPT with “Is there any clinical evidence that any specific homeopathic treatment has efficacy beyond placebo?” ChatGPT’s conclusion statement reads:
“There are positive clinical signals and isolated positive trials for several homeopathic treatments, but no specific homeopathic remedy–condition pairing currently has convincing, reproducible evidence of efficacy beyond placebo.
That is slightly more precise than saying “there is absolutely no clinical evidence,” but it still does not support prescribing any homeopathic treatment as an evidence-based therapy.”
I wonder if the nuances of my prompt resulted in a more science-based conclusion than Ullman’s rabbit hole of sycophancy.
I next prompted: “is there any basic science to support that homeopathic-level nanodoses can have biological effects?” And its response”
“The short answer is yes, but only in a very limited sense. There is basic science showing that extremely small numbers of molecules (even single molecules, in some contexts) can have biological effects. However, there is no convincing basic science supporting the core claims of homeopathy, particularly for the high dilutions used in classical homeopathy (typically beyond about 12C or 24X, where no molecules of the original substance are expected to remain).”
In homeopathic products, there is typically no active ingredient remaining. Originally the homeopathic preparation was supposed to contain the “essence” of the starting material, but such claims have not aged well scientifically. So modern homeopaths moved on to structured water – the original substance is not there but the water “remembers”. They still use this rationalization, but the evidence is not there – structures in water are fleeting and could not survive to have biological effects. There is also no evidence that such structures reflect the original substance in any meaningful way. Now they have moved on to “nanodoses”, but as ChatGPT correctly reflected, most homeopathic products don’t even contain nanodoses. Further, “biological effects” is not the same thing as clinically meaningful effects. None of this handwaving even addresses the other scientific problem with homeopathy – that the starting materials are fanciful (sometimes are not even real) and have no plausible connection to remedies.
So Ullman likely was just getting reflected back from ChatGPT his own pseudoscientific biases, and selectively interprets the results with those same biases, as he did above.
All of this is why we must do our best to understand and deal with AI. AI is increasingly the medium through which pseudoscience is being spread, and it is engendering an unearned confidence in its biased results.
