“A person is smart. People are stupid.”
– Agent K (Tommy Lee Jones), Men In Black
Regular readers of my blog know how passionate I am about protecting the public from misleading health information. I have witnessed first-hand many well-meaning attempts to “empower consumers” with Web 2.0 tools. Unfortunately, they were designed without a clear understanding of the scientific method, basic statistics, or in some cases, common sense.
Let me first say that I desperately want my patients to be knowledgeable about their disease or condition. The quality of their self-care depends on that, and I regularly point each of them to trusted sources of health information so that they can be fully informed about all aspects of their health. Informed decisions are founded upon good information. But when the foundation is corrupt – consumer empowerment collapses like a house of cards.
There is growing support in the consumer-driven healthcare movement for a phenomenon known as “the wisdom of crowds.” The idea is that the collective input of a large number of consumers can be a driving force for change – and is a powerful avenue for the advancement of science. It was further suggested (in a recent lecture on Health 2.0), that websites that enable patients to “conduct their own clinical trials” are the bold new frontier of research. This assertion betrays a lack of understanding of basic scientific principles. In healthcare we often say, “the plural of anecdote is not data” and I would translate that to “research minus science equals gossip.” Let me give you some examples of Health 2.0 gone wild:
1. A rating tool was created to “empower” patients to score their medications (and user-generated treatment options) based on their perceived efficacy for their disease/condition. The treatments with the highest average scores would surely reflect the best option for a given disease/condition, right? Wrong. Every single pain syndrome (from headache to low back pain) suggested a narcotic was the most popular (and therefore “best”) treatment. If patients followed this system for determining their treatment options, we’d be swatting flies with cannon balls – not to mention being at risk for drug dependency and even abuse. Treatments must be carefully customized to the individual – genetic differences, allergy profiles, comorbid conditions, and psychosocial and financial considerations all play an important role in choosing the best treatment. Removing those subtleties from the decision-making process is a backwards step for healthcare.
2. An online tracker tool was created without the input of a clinician. The tool purported to “empower women” to manage menopause more effectively online. What on earth would a woman want to do to manage her menopause online, you might ask? Well apparently these young software developers strongly believed that a “hot flash tracker” would be just what women were looking for. The tool provided a graphical representation of the frequency and duration of hot flashes, so that the user could present this to her doctor. One small problem: hot flash management is a binary decision. Hot flashes either are so personally bothersome that a woman would decide to receive hormone therapy to reduce their effects, or the hot flashes are not bothersome enough to warrant treatment. It doesn’t matter how frequently they occur or how long they last. Another ill-conceived Health 2.0 tool.
When it comes to interpreting data, Barker Bausell does an admirable job of reviewing the most common reasons why people are misled to believe that there is a cause and effect relationship between a given intervention and outcome. In fact, the deck is stacked in favor of a perceived effect in any trial, so it’s important to be aware of these potential biases when interpreting results. Health 2.0 enthusiasts would do well to consider the following factors that create the potential for “false positives”in any clinical trial:
1. Natural History: most medical conditions have fluctuating symptoms and many improve on their own over time. Therefore, for many conditions, one would expect improvement during the course of study, regardless of treatment.
2. Regression to the Mean: people are more likely to join a research study when their illness/problem is at its worst during its natural history. Therefore, it is more likely that the symptoms will improve during the study than if they joined at times when symptoms were not as troublesome. Therefore, in any given study – there is a tendency for participants in particular to improve after joining.
3. The Hawthorne Effect: people behave differently and experience treatment differently when they’re being studied. So for example, if people know they’re being observed regarding their work productivity, they’re likely to work harder during the research study. The enhanced results therefore, do not reflect typical behavior.
4. Limitations of Memory: studies have shown that people ascribe greater improvement of symptoms in retrospect. Research that relies on patient recall is in danger of increased false positive rates.
5. Experimenter Bias: it is difficult for researchers to treat all study subjects in an identical manner if they know which patient is receiving an experimental treatment versus a placebo. Their gestures and the way that they question the subjects may set up expectations of benefit. Also, scientists are eager to demonstrate positive results for publication purposes.
6. Experimental Attrition: people generally join research studies because they expect that they may benefit from the treatment they receive. If they suspect that they are in the placebo group, they are more likely to drop out of the study. This can influence the study results so that the sicker patients who are not finding benefit with the placebo drop out, leaving the milder cases to try to tease out their response to the intervention.
7. The Placebo Effect: I saved the most important artifact for last. The natural tendency for study subjects is to perceive that a treatment is effective. Previous research has shown that about 33% of study subjects will report that the placebo has a positive therapeutic effect of some sort.
In my opinion, the often-missing ingredient in Health 2.0 is the medical expert. Without his/her critical review and educated guidance, there is a greater risk of making irrelevant tools or perhaps even doing more harm than good. Let’s all work closely together to harness the power of the Internet for our common good. While research minus science = gossip, science minus patients = inaction.