One can only imagine how annoyed Twain would have been had he known about opinion polls.
A new Harris Poll on “alternative medicine” nicely demonstrates some of the problems with polls. The biggest problem is how you frame the questions. You can dramatically affect the results of the poll by exactly how a question is phrased, which other questions come before or after the question, and the overall context of the poll.
In essence a poll is a two-way form of communication. While it is meant to derive information from the subjects of the poll, those subjects are also receiving information from the poll itself. That is the very reason for “push polling” – which is the practice of disguising campaigning messages as a poll. (“On a scale from 1-5, how much are you bothered by the fact that candidate X is a misogynist?”). Yes, Prime Minister has an amusing example of this practice in action.
Interpreting the results of a poll is also not straightforward. You have to know how the subjects of the poll interpreted the questions, and what factors might affect their response.
The Harris Poll
Polls looking into attitudes and use of so-called “alternative medicine” are automatically tricky because the definition of “alternative medicine” is deliberately vague. What counts as alternative? What is the operational definition?
I have argued that it is a false category, designed for marketing and propaganda purposes. That is why proponents have been shifting the phrase over the years, from alternative to complementary, and now to integrative medicine. In practice it seems to include anything that is not adequately based on science and evidence, plus anything else that proponents want to include to make the category seem more popular and legitimate.
The biggest weakness of the poll is that it makes no attempt to define “alternative.” Does this include exercise, taking vitamins, prayer, getting a massage, or relaxation?
Further, it contrasts “alternative medicine” with “conventional Western medicine.” This is buying into the false dichotomy even further. Calling science-based medicine “Western” is an insult to our colleagues in the East who are just as capable and dedicated to science as their Western counterparts.
While the poll does not define the key concept of the entire poll, it does suggest to those taking the poll what might be included in the definition by asking them about specific alleged “alternative” interventions.
Here is a quick summary of the key results from their press release:
Overall, about seven in ten Americans (71%) have used some kind of alternative therapy before. Over one third state they have used herbal medicines (37%) or chiropractic care (34%). For Matures (those aged 70+), chiropractic care is the predominant type of alternative therapy used, with nearly half having ever used it (45%). Herbal medicines appeal much more to a younger audience, with 40% of those under 50 having used it.
Usage of alternative therapies is higher among those who don’t have insurance – nearly half of uninsured Americans (48%) use alternative therapies as often as or more than conventional treatments.
When it comes to how likely Americans are to use alternative options for various conditions, a majority of all adults (64%) are likely to use these options to treat physical pain, and Millennials and Gen Xers are particularly likely to consider such alternatives for things like addiction (60% and 57%, respectively) and mental health (57% each) as well.
The 71% is a typical inflated statistic of alternative medicine use when using the broadest definition. If you dig a little deeper into the poll you see that this is mainly comprised of chiropractic, supplements/vitamins, and massage. Aromatherapy and meditation also are over 20%.
What does it mean?
Chiropractic is always near the top because it has been institutionalized in the US for a century now. Some patients use chiropractors as their physical therapists. Chiropractors also practice a lot of blatant pseudoscience, making this also a complex category.
Generations of aggressive marketing and the deregulation of supplements in 1994 has also exploded the supplement market, so again, no surprise there. The popularity of aromatherapy and essential oils is a bit of a recent phenomenon, and to me is mostly an extension of the supplement phenomenon.
Massage has always been popular, and is a perfectly reasonable intervention for muscle tension. It is a great example of an intervention that is not really alternative, unless you make pseudoscientific claims for its effects (such as squeezing out toxins from the tissues).
Meditation, in my opinion, is relaxation. Studies looking into its benefits simply show the benefits of relaxation, and often make no attempt to compare meditation to other forms of relaxation, and so do not establish that there is any specific benefit to meditation itself.
As you can see, the category is full of sloppy thinking, loose definitions, and dubious science. But, as you can also see, the vast majority of what people are using under the label of “alternative medicine” comes down to essentially some kind of supplements, physical therapy, or relaxation.
When we get to what I consider the hard core “alternative” treatments, ones that are not just a rebranding of lifestyle factors, diet, and physical therapy, the numbers drop to very low levels. The most popular such modality in this poll barely cracks double digits – acupuncture at 11%. The rest are all single digits.
This is also not surprising given the degree to which the scientific research on acupuncture has been systematically misreported to the public. What the research actually show, quite clearly, and consistently after thousands of clinical trials, is that acupuncture simply does not work, for anything. If you define acupuncture as sticking thin needles into acupuncture points to address specific symptoms or health issues, then acupuncture does not work. The evidence clearly shows it does not matter where you stick the needles, or even if you stick the needles. Efficacy depends only on the ritual surrounding acupuncture, and having a positive interaction with the acupuncturists. In other words, acupuncture is an elaborate placebo, which is a scientific way of saying it does not work.
Meanwhile, the same research which shows it does not work is systematically reported to the public (and presented to academics and professionals who apparently fail to do their due diligence), as if it does work. They essentially say that placebo acupuncture works also. This is identical to a pharmaceutical company arguing that the placebo of their drug also works to relieve subjective symptoms, and therefore their drug works (because there was no difference between the two).
All of this deception has pushed acupuncture to 11% (at least in this poll) in terms of people saying they have ever used it. From there it goes: electrotherapy (also vague) – 9%, reflexology – 5%, hypnotherapy – 4%, Reiki – 3%, cupping – 3%, other – 3%. Homeopathy did not even make the cut, apparently.
It is also important to note what the poll does not ask. For example, a previous survey found that the phrase, “what is proven to work best” was the highest ranked, at 79%, as a description of what patients want in their medical treatments.
The public does not appear to be turning away from science, or from science-based standards of care. They want their treatments to be safe and effective. They are simply being misinformed about what this means. They are often being lied to and directly deceived, often with an immediate profit motive on the part of those selling snake oil.
Conclusion: PR, not polling
This Harris poll on “alternative medicine” is so terrible it is practically a pro-alternative medicine push poll. They do not define their key terms, use loaded language and false dichotomies, and use vague categories.
I should also point out that this was an online poll of those who agreed to take the poll. The results can therefore not be generalized to the population as a whole. It is quite possible that people who are frequently online are more exposed to pro-alternative medicine marketing. Those willing to take the poll may also be more pro-alternative medicine.
Even without these weaknesses, the design of the poll itself was very poor, in my opinion. The reporting of the results was also highly biased, reflecting the bias in constructing the poll itself.
This poll represents one piece of the big picture – that of a self-fulfilling narrative. Those with an ideological agenda, or who make money by selling dubious treatments to the public, have been engaged in a propaganda campaign to rebrand what was previously known as health fraud as if they were a legitimate alternative. This campaign, unfortunately, has been very successful. They have controlled the narrative to the point that it is not even questioned, except by those who are dedicated to high standards of science within health care.
A Harris Poll on “Alternative Medicine”