Playing by the Rules

I recently read Flock of Dodos (no relation to the movie of the same name). It’s a hilarious no-holds-barred send-up of the lies and poor reasoning of the intelligent design movement. I was particularly struck by a quotation from William Dembski’s Intelligent Design.

We are dealing here with something more than a straightforward determination of scientific facts or confirmation of scientific theories. Rather we are dealing with competing world-views and incompatible metaphysical systems.

That cuts to the essence of what we are confronting on Science-Based Medicine. We are trying to evaluate the science behind claims that are often not based on science but on beliefs that are incompatible with science. The claimants are happy to use science when it supports them, but when it doesn’t they are likely to unfairly critique the science or even to dismiss the entire scientific enterprise as a “materialistic world view” or “closed-minded.” We are talking at cross purposes. How can we communicate if we say “this variety of apple is red” and they insist “it feels green to me”?

We get frustrated when we show these folks the scientific evidence and they refuse to accept it. Homeopathy is not only implausible, but it has been tested and has failed the tests. Yet proponents refuse to acknowledge those failures and still want to talk about data from the 19th century and make claims for the memory of water. We have to realize we are not even speaking the same language. We are trying to play a civilized game of gin rummy and they are dribbling a basketball all over the card table. Before getting into a debate, doesn’t it make sense to define what game you’re playing and what the rules are?

Science has been a very successful self-correcting group endeavor. It wouldn’t be successful if it didn’t follow a strict set of rules designed to avoid errors. [Note: there are no rules written in stone; I’m talking about conventions that are generally understood and accepted by scientists, conventions that grow naturally out of reason and critical thinking.] If proponents of alternative medicine want to play the science game, they ought to play by the rules. If they won’t play by the rules, they effectively take themselves out of the scientific arena and into the metaphysical arena. In that case, it is useless for us to talk to them about science.

If you want to play the science game, here’s what you do:

1. Submit your hypothesis to proper testing. Testimonials, intuitions, personal experience, and “other ways of knowing” don’t count.
2. See if you can falsify the hypothesis.
3. Try to rule out alternative explanations and confounding factors.
4. Report your findings in journal articles submitted to peer review.
5. Allow the scientific community to critique the published evidence and engage in dialog and debate.
6. Withhold judgment until your results can be replicated elsewhere.
7. Respect the consensus of the majority of the scientific community as to whether your hypothesis is probably true or false (always allowing for revision based on further evidence).
8. Be willing to follow the evidence and admit you are wrong if that’s what the evidence says.

If you want to play the science game, here are some of the things you don’t do:

1. Accuse the entire scientific community of being wrong (unless you have compelling evidence, in which case you should argue for it in the scientific journals and at professional meetings, not in the media).
2. Design poor-quality experiments that are almost guaranteed to show your hypothesis is true, whether it really is or not. Use science to show THAT your treatment works, not to ask IF it works.
3. Keep using arguments that have already been thoroughly discredited. (The intelligent design folks are still claiming the eye could not have evolved because it is irreducibly complex; homeopaths are still claiming homeopathy cured more patients than conventional medicine in the 19th century epidemics).
4. Write books for the general public to promote your thesis – as if public opinion could influence science!
5. Form an activist organization to promote your beliefs.
6. Step outside the scientific paradigm and appeal to intuition and belief.
7. Mention the persecution of Galileo and compare yourself to him.
8. Invent a conspiracy theory (Big Pharma is suppressing the truth!).
9. Claim to be a lone genius who knows more than all the other scientists put together.
10. Offer a treatment to the public after only the most preliminary studies.
11. Set up a website to sell products that are not backed by good evidence.
12. Refuse to admit it when your hypothesis is proven wrong

Changing Our Minds

Scientists will change their minds when the evidence warrants. Before we waste time arguing, one thing we can do is ask our opponents what it would take to change their minds. One woman I asked said no amount of evidence could change her mind because she knew from personal experience that her claim was true, so any evidence that said otherwise would have to be false and fabricated. End of discussion. She’s out of the game.

The rules of science are pretty clear about what it takes to change our minds. I’ll use the example of Helicobacter and ulcers. We used to think that stress and too much stomach acid caused ulcers; now we think a bacterium causes ulcers. Here’s a summary of why we changed our mind:

1. Scientists noticed bacteria in biopsy samples from ulcers.
2. They identified the bacteria as Helicobacter pylori
3. They found a strong correlation between ulcers and the presence of the bacteria
4. One of the researchers who was healthy and not a Helicobacter carrier was able to induce an ulcer in himself by ingesting the bacteria.
5. They found that treating patients with antibiotics cured ulcers.
6. They found that antibiotics were superior to previous ulcer treatments.
7. The studies were replicated and studies done in different ways corroborated each other.
8. The bacterial hypothesis was not inconsistent with the rest of scientific knowledge.

If we had that quantity and quality of evidence for homeopathy, we’d gladly accept it. In fact, if we had 1 through 7 we’d provisionally accept it while we kept checking the data and tried like crazy to figure out the mechanism behind it.

There are two issues that are often misunderstood: scientific consensus and prior plausibility.

Prior Plausibility

Homeopathy is completely implausible. We would have to accept robust evidence that it worked, but we would require much stronger evidence than we would for, say, a new antibiotic. Because if the claims for homeopathy were true, we would have to revise much of what we know about physics, chemistry, and physiology.

The crossword analogy is helpful. If you think the answer to 1-across should be “library” but the clue to 1-down is a five-letter word for the author of Tom Sawyer and the clue to 2-down is a four-letter word for the name of Eve’s husband in Genesis, you have to reject “library” and keep looking for a word that starts with T-A. You have to recognize that no matter how strong your conviction that 1-across must be “library” you must be wrong and that there must be another answer that you just haven’t thought of.

Consensus

It’s easy to dismiss the scientific consensus as a popularity contest, a vote on opinions. But it’s far more than that. The body of evidence stands or falls on its own merits, and when the weight clearly tips the balance to one side everybody can see it. The scientific community is made up of experts who know how to evaluate the evidence and who thrash out disagreements in medical journals and scientific conferences. It is easy for the scientific community to reach an agreement based on clear evidence. There are times when the outcome is uncertain and controversy among scientists is appropriate, but there comes a time when it would be perverse not to accept the evidence, just as it is perverse to deny evolution or germ theory. The scientific consensus on evolution and the germ theory is a recognition of reality, not a matter of opinion.

A reasonable default assumption is that the scientific consensus is usually right, and if it isn’t, it will change as the evidence becomes clearer. Truth will prevail. It does no good to attack the scientific consensus as prejudiced or closed-minded. The consensus will only change when it incorporates new and better evidence. One of the irrational tactics we’ve seen over and over on this blog is for opponents to cite one or a handful of studies to support their belief, in the ridiculous assumption that it was new information that the people who reached the scientific consensus had failed to consider, or that it somehow outweighs all the other studies that found the opposite.

Play by the Rules or Go Play Your Own Game

There’s no point in arguing scientific facts with someone whose world-view is metaphysical and non-scientific. There’s no point in presenting geological age data to someone who “knows” the age of the Earth from the Bible. Before we get into a useless debate, maybe we should find out what game our opponents are really playing. If they are playing pingpong, it’s silly for us to bring a football to the table. It would be handy if we could get them to say up front what game they are really playing, but all too often they have deluded themselves into truly believing they are following the rules of science.

If they won’t play the science game by the rules, we are justified in crying “foul” and disqualifying them. Then they can go away somewhere else and play their own game by whatever rules they want and we won’t be able to refute them. If they are relying on beliefs unsupported by evidence, let them say so. Wouldn’t it be refreshing to hear a homeopath say “I believe homeopathy works based on my personal experience and on non-scientific evidence like testimonials, and I categorically reject the results of any scientific trial that fails to support my beliefs. Homeopathy cured my neighbor’s uncle’s cousin of cancer. Trust me. I’m a nice guy so you should believe whatever I tell you.”

If they’d say that up front, we wouldn’t waste any of our valuable time re-hashing scientific evidence that they would just ignore. They would be out of the game, permanently. And patients would have a better basis for giving truly informed consent