Peer review, a flawed but vital part of the scientific process.
When I lecture about the need for science-based medicine (SBM), I have to pause about half-way through my list of all the things wrong with the current practice of medical science, and I balance my discussion by emphasizing what I am not saying: I am not saying that medical science is completely broken. It is just really challenging, we need to raise the threshold for what we consider reliable higher than most people think, and there are some practical fixes we can do, some of which are already in the works.
It is easy, however, to “demonize” any person, institution, or philosophy by taking all the negative aspects that are inevitably present and wrapping them up in a frightening package, perhaps throwing in some conspiracy thinking or sensational alarmism.
Take, for example, a recent article by F. William Engdahl, “Shocking Report from Medical Insiders“. The headline alone warns you that you may be in for some sensationalism.
For background, Engdahl is a journalist with some ideas that are out of the mainstream. He believes the theory that oil is not biological in origin but geological, and therefore “peak oil” is a myth, specifically designed to create panic in the public for political control. He also believes that the warnings about global warming are exaggerated. He is the author of Seeds of Destruction, which in my opinion is a propaganda hit piece against the technology of genetic modification.
In other words, Engdahl appears to be a journalist with an ideology, with the result being some skillfully written but ultimately misleading pieces.
His recent article is based upon recent statements made by medical editors about the state of medical research. He quotes Dr. Richard Horton, Editor-in-chief of The Lancet:
Much of the scientific literature, perhaps half, may simply be untrue. Afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses, and flagrant conflicts of interest, together with an obsession for pursuing fashionable trends of dubious importance, science has taken a turn towards darkness.
Actually I think 60% may be a better figure than half. This should not be news to regular readers of SBM. We often highlight the problems of poor methodology: overreliance on P-values and P-hacking, researcher bias, publication bias, testing implausible hypotheses, failure to replicate, citation bias, and conflicts of interest.
We don’t deny these very serious problems exist within the institution of medical science. Our point in exploring them, however, is two-fold:
- To provide a helpful guide to distinguishing reliable science from unreliable science. There is still good science out there, and with it we can come to confident conclusions.
- We want to improve the practice of science by fixing the problems. You can either point to these known issues as if they are part of some dark conspiracy, or you can point to them in a constructive way in order to advocate for change.
Horton, to his credit, is doing the latter. He goes on to write:
Instead of changing incentives, perhaps one could remove incentives altogether. Or insist on replicability statements in grant applications and research papers. Or emphasise collaboration, not competition. Or insist on preregistration of protocols. Or reward better pre and post publication peer review. Or improve research training and mentorship. Or implement the recommendations from our Series on increasing research value, published last year. One of the most convincing proposals came from outside the biomedical community. Tony Weidberg is a Professor of Particle Physics at Oxford. Following several high-profile errors, the particle physics community now invests great effort into intensive checking and rechecking of data prior to publication.
That list should also be familiar to regular readers of SBM. Engdahl doesn’t quote that part of the paper or even mention its existence, because it does not serve his conspiracy alarmist narrative.
Engdahl also quotes Dr. Marcia Angell, previous editor-in-chief of the New England Journal of Medicine (which he misnames the “New England Medical Journal“).
It is simply no longer possible to believe much of the clinical research that is published, or to rely on the judgment of trusted physicians or authoritative medical guidelines. I take no pleasure in this conclusion, which I reached slowly and reluctantly over my two decades as an editor of the New England Journal of Medicine.
Angell, of course, is referring in part to the many problems with published medical science as discussed above, and I agree with much of what she says. However, in my opinion she overstates the situation as well. I shared a stage with her about ten years ago at an alternative medicine conference. She seemed far more interested in taking on the pharmaceutical industry than pseudoscience itself, and said as much.
There is much to criticize in how the pharmaceutical industry has tried to subvert medical research to their advantage. This is why they need to be carefully monitored and regulated. They are not the only ones, however. I would argue that the entire supplement and CAM industries are far, far worse, and they are not as well regulated.
The problems in science can be fixed
There are many endemic problems with the science of medicine as it is currently practiced. It is possible to draw the wrong lessons from these problems, often influenced by one’s ideology.
While a great deal of published research is preliminary, implausible, or of low methodological rigor, not all of it is. We can still pick through the research to find the most reliable studies that have survived independent replication. I agree that, with any lower threshold, you might as well flip a coin.
In addition to looking at the literature with open and critical eyes, a valuable lesson is that we need to make serious efforts to fix identifiable problems with the institutions of science (as outlined above).
The big challenge, as Horton himself points out, is not that we don’t know what to do, but that no one has the power to do it. There is no central power hierarchy of science, and when power is distributed it is often difficult to accomplish major changes. Everyone thinks it is someone else’s responsibility.
What we need is for the issue to become well known, to cross that nebulous threshold of political will. We need many voices pointing out the problems that need fixing, and we need to put pressure on those with any power.
The problem with conspiracy-mongering is that it is often not constructive — there is no call to action, just the sense that the world is hopelessly corrupt. Engdahl concludes his article with:
Corruption of the medical industry worldwide is a huge issue, perhaps more dangerous than the threat of all wars combined. Do we have such hypnosis and blind faith in our doctors simply because of their white coats that we believe they are infallible? And, in turn, do they have such blind faith in the medical journals recommending a given new wonder medicine or vaccine that they rush to give the drugs or vaccines without considering these deeper issues?
I conclude this article, as I do many of my essays here at SBM, with a call to action. These problems can easily be fixed. They do not require new resources or even great sacrifices. They just require a change in standard practice. Universities, journal editors, regulators, and professional organizations can make it happen. In many cases they know exactly what to do — they just need to do it.
