Ian Harris, a Professor of Orthopaedic Surgery at the University of New South Wales in Sydney, Australia, wrote a book titled Surgery, The Ultimate Placebo. I haven’t read the book, but I watched his excellent YouTube video lecture on the subject. It is an eye-opening evaluation of commonly performed surgical operations that have been tested and shown to be no more effective (and arguably worse) than placebo, or that have never even been tested. He covers the history of sham surgery studies, talks about placebo effects, and explains why so many surgeons ignore the evidence and continue to do ineffective operations. In the process, he provides a valuable education in critical thinking.
His introduction to skepticism at age 18 was a documentary on television where dowsers were subjected to an elegant controlled test by James Randi and Dick Smith. They buried ten pipes and challenged dowsers to determine which pipe the water was flowing through. When they knew which pipe was active, the dowsers all identified it correctly; but when they didn’t know, they were only right 12% of the time. He was impressed by how a simple test could easily refute a claim.
When he watched the same documentary again recently, he was impressed by the reaction of dowsers when shown that dowsing didn’t work. He recognized it as the same reaction he was seeing in surgeons when their operations were tested and shown not to work. They made up excuses, hypotheses about why the experiment might not have given the results they expected, and said they wouldn’t give up doing the operation, “because it works”.
In the early 20th century, a common operation for heart disease was internal mammary artery ligation. It seemed obvious that it worked: patients felt better. In 1939, a sham surgery experiment showed that an equal number of patients got better if the surgeon just made an incision but didn’t ligate the arteries. Surgeons heeded the evidence and stopped doing that operation.
He takes us on a historical tour of surgical operations that have been tested and shown not to work. Surgeons often reject the evidence and make up rationalizations as to why it might not have worked in those studies, but they remain convinced that it does work in their hands. When they refuse to stop doing that operation, their excuses sound just like those of the dowsers. They insist on continuing to do the operation “because it works” (in their experience). Billions of dollars are being wasted on hundreds of thousands of operations that have been shown not to work or that have never even been tested to see if they work better than doing nothing.
False analogies and faulty reasoning
Theory often says an operation should work, but biological plausibility means absolutely nothing unless tests can show that it really does work. Dr. Harris says “You can make up a biologically plausible mechanism for anything you want.”
The parachute analogy says you don’t need a randomized controlled study of parachutes to find out if they work. Dr. Harris says he has seen it frequently invoked, usually for procedures that are later shown to be rubbish.
Comparative effectiveness studies are done comparing two different methods of something like steroid injections for back pain, but they don’t tell us whether either of them work. They would be better called comparative ineffectiveness studies. The first step should have been to use a control group and compare steroid injections to no injections.
Perceived effectiveness
Perceived effectiveness is a combination of two things: the treatment’s specific effects plus its placebo effects.
There are shortcuts in thinking that all humans are hard-wired to take. They assume correlation means causation, and they fall for the post hoc ergo propter hoc fallacy (assuming that if B follows A that must mean that A caused B). These shortcuts in thinking often lead to errors. Improvement might be due to something other than the surgery, like the natural history of the disease, regression to the mean, or concomitant treatments.
Improvement can be perceived by the patient (sometimes it is a false perception due to placebo factors or misinterpretations) and/or by the doctor, due to factors like measurement error, reporting bias, confirmation bias, etc. Doctors rate the effectiveness of their treatments higher than their patients do. The perception of the doctors is wrong; they consistently overestimate the benefits and underestimate the harms of their treatments.
Treatment is not always needed
He tells the story of Archie Cochrane, for whom the Cochrane Collaboration of systematic analyses was named. As the only doctor in a POW camp in World War II, he was responsible for 10,000 prisoners, many suffering from open wounds, dysentery, typhoid, and other serious diseases. His requests for doctors and medicines were denied by his German captors, who said doctors were superfluous. In six months, only 4 prisoners died, each of them shot while trying to escape. The rest all got better, without treatment.
Harris’ story distorts the truth, but it serves as a vivid illustration of an important fact: recovery often occurs without any treatment. Cochrane’s experience led him to question much of what was being done in the name of medicine. Patients were kept in bed for a week after a heart attack not because of any evidence, but just “because it makes sense”. Cochrane did a controlled study and found that bedrest was actually harming people.
Surgery often acts as a placebo
Harris covers the many factors that affect a patient’s response to a placebo. A systematic review found that placebo was just as effective as surgery in over half of the cases studied, and all of the recent trials comparing surgery to placebo have found that surgery was no better than placebo.
He demolishes all the arguments surgeons give for continuing to do these operations that have been tested and shown not to work. The real reason is that they continue to believe the procedure is effective, just as the dowsers continued to believe they could find water with a forked stick. Tradition and personal experience triumph over science and reason.
Science is just a systematic way to reduce error. Imperfect, but better than any other way. Blinded trials are the least biased way to determine effectiveness.
Ethical concerns
Some people object that doing sham surgeries is unethical. Placebos are unethical in clinical practice for treating an individual, but not in research where they are essential to finding the truth and balancing harms and benefits for large groups of people and for preventing future harms. Surely, it’s unethical to perform ineffective surgeries. On Science-Based Medicine, we are constantly questioning the ethics of selling dietary supplements with claims that have never been properly tested. Some herbal medicines might be harmless, but surgery always carries a risk of infection, tissue damage, and adverse effects. And it is expensive: millions of dollars are being wasted worldwide on procedures that expose patients to risk for no possible benefit.
Conclusion: Evidence is essential
He concludes by saying we should treat new surgical procedures like new drugs, and only pay for those that are part of a trial to find out if they work. Most surgical procedures being done today have not been subjected to blinded trials.
Advice to patients: it’s OK to ask for a second opinion, and you should always ask your surgeon for the evidence showing the benefits and risks of the procedure, and what you can expect to happen if surgery is not done.