News bulletin on BBC NEWS International version, 8 Feruary 2008:“Acupuncture ‘boosts IVF chances.’ Acupuncture may increase the success rates of fertility treatment, according to a study. “
(Manheimer E, Zhang G, Udoff L, Haramati A, Langenberg P, Berman BM, Bouter LM. Effects of acupuncture on rates of pregnancy and live birth among women undergoing in vitro fertilisation: systematic review and meta-analysis. BMJ. 2008 Feb 7)
First off, how plausible is the claim? The press release states that acupuncture had been used in China fior thousands of years for infertility. Has it? No medical historian writing I have seen made such an interpretation of ancient texts. Maybe I missed something…possible. But acupuncture was not used for specific disorders or purposes, but was used as a sort of panacea to cause balance of either the Yin and Yang or of the relationship of the individual with the 5 elements and the cosmos and the earth. There is nothing specific in claims of acupuncture in traditional Chinese Medicine history. Who gave the news people that misleading lead-in?
Second, what is the plausibility that acupuncture could possibly affect a laboratory procedure on tissue removed from the subject, regardless of timing? Negligible to none. There is no consistent and credible information that acupuncture is effective for anything, except as a conditiong agent for perception of symptoms.
So, does acupuncture increase the success of IVF?
This study, making international news wires, was not another randomized clinical trial (RCT,) but a Systematic Review (SR) of previous trials. We have several layers of potential error to contend with: 1) The quality and accuracy of each RCT, 2) The quality and accuracy of the SR itself, 3) The accuracy of the news release, which emanates from publicity departments of the respective universities, 4) The relevance and selection of the quotations and the data.
Individual RCTs display yet another layer of potential error. Some of the difficulties in evaluating RCTs were delineated in Dr. Atwood’s review of homeopathy. They include the biases of the sponsoring organization, the heterogeneity of treatment and control subjects from one trial to another, differing end points, differing methods of pain and other symptom measurements, the problems of quantifying symptoms – which are multiple and difficult to overcome, differing (in this case) acupuncture techniques, and differing statistical analyses and expressions. Acupuncture studies are particularly prone to problems of breaking of the blinding. None of these is known to have been adequately controlled for in RCTs to date. Ioannidis in his now classic paper in PLoS Medicine 1996 listed a number of similar faults responsible for his thesis that most clinical results are false. (I would say, just erroneous.) Nevertheless, most positive acupuncture trials are defective and their results are unreliable.
As for sponsoring organizations, RCTs sponsored by pharmaceutical companies are generally suspect and show inordinate degrees of “efficacy.” So far, RCTs sponsored by the National Center for Complementary and Alternative Medicine, supplement companies, sectarian method treatment centers, ideologically oriented granting agencies, have not received the same attention for conflict of interest. They should. They all show an excess of “positive” trials for ineffective treatments.
As for SRs, no SR system accounts for those embedded biases. Nor is there an accepted, proved methodology for performing SRs. One count of different SR methods came to over twenty. Some have entry criteria for RCTs, which eliminate lower quality individual studies. Other SRs include as many studies as possible. Whichever general method is used, most SRs differ from one another as to which studies qualify for inclusion.
Not only that, there are no current universally accepted standards for RCT evaluation, or as to how the evaluators judge and negotiate their differences. Nor is there an accepted level or measure for when to declare a method effective or ineffective or indeterminate. Add to that the facts that there are no qualifications for systematic review reviewers – and worse, most reviewers of “CAM” or sectarian methods are the practitioners and advocates of those methods. Talk about biases…
We have been collecting Srs and analyzing the conclusions from varying subjects and treatment methods.
Comparing acupuncture Srs with those of anti-hypertensive drug efficacy, or with other ineffective methods shows an apparent bias in how results are reported, and their conclusions.
Regarding this BBC report, so far I have not seen the original paper, just the news release. Nevertheless, the release states an obvious clue that contradicts the headline conclusion, which I assume is the conclusion of the SR authors. The standard known success rate in the best of hands for in vitro fertilization (IVF) is about 25 percent. We went through this material in analyzing and reporting on the infamous Cha/Wirth report of positive results on IVF from distant prayer, now with mountains of evidence to have been fraudulent. In this acupuncture SR, the authors admit that in the most successful labs, there was no improvement in IVF success when subjects received acupuncture. Apparently, the successes came in labs in China which had lesser success rates before the study. There are several reasons why a success might be increased in a low success rate lab. Data selection would rise to the to the top, as would any study that allowed a timing bias – with comparisons of later vs. earlier data, which would allow improvement with experience and expertise development. Editors should require all raw data from those overseas labs.
One might ask, in view of that latter fact alone, why this article was accepted for publication, and why such a headline conclusion was allowed to be broadcast. The British Medical Journal seems to have been printing reports on sectarian medicine methods more frequently than have many other journals. The editorship recently changed and we had hoped for a significant change in that policy. Change is slow.