I read this Reuters Health article on MedlinePlus, and then I read the study the article referred to (The impact of acupuncture on in vitro fertilization) and now my head hurts. The study found that acupuncture was not effective in increasing the pregnancy rate (PR) during in vitro fertilization (IVF). As quoted on MedlinePlus, the lead author, Alice Domar, seems to blame her patients (the presumably poor quality of their embryos) rather than acupuncture for the lack of success, and then she recommends using acupuncture even if it doesn’t work. That was bad enough, but “poor quality embryos” is a hypothesis that was actually tested and rejected in the study itself. Has Domar forgotten?
The headline of the MedlinePlus article says “acupuncture doesn’t boost IVF success for all” – suggesting that it boosts success for some? Then the first sentence says the study suggested that acupuncture doesn’t work, period. But wait…
The lead researcher says acupuncture may not have worked in her study because, unlike past research, her investigation wasn’t limited to women who had good quality embryos available for transfer. “I’m wondering if my sample was just not a good sample, in that most of the patients in my study were probably not the best-prognosis patients,”
Domar and her team say the most likely explanation for the lack of an acupuncture effect in their study was the fact that they included many women who didn’t have good quality embryos available for transfer. While acupuncture may help a woman become pregnant after the transfer of a healthy embryo, the researcher noted in an interview, it can’t repair an embryo with chromosomal defects or other abnormalities.
Hold the boat!! In the Discussion section of the paper itself, Domar et al point out that previous research has included mostly patients with good quality embryos. They ask if perhaps acupuncture only works for good quality embryos? They test that hypothesis by separately analyzing the subjects in this study who had good quality embryos. There was no increase in PR with acupuncture in this sub-group; the results were the same as for the entire sample.
This study not only had an objective endpoint (pregnancy) but it also had several subjective psychological endpoints (optimism, confidence, and anxiety as measured by perceived relaxation). The women who received acupuncture felt more relaxed and enjoyed the IVF procedure more, the researchers found. They were also more optimistic about getting pregnant, but not more confident that they would get pregnant.”
Despite the results of my own study, I still recommend acupuncture to women going through IVF because there’s no downside,” Domar added, aside from the $150 an acupuncturist would typically charge — a small fraction of the $12,000 to $14,000 couples typically spent on a single round of IVF.
It seems to me this translates as: Acupuncture works. It didn’t work in this study, but that can’t be the fault of acupuncture, because acupuncture works. So it must be the fault of the patients for producing poor quality embryos, (our data don’t support that hypothesis, but let’s just ignore that). Acupuncture is harmless and people like it, so let’s use it on every patient whether it works or not. Patients will have to pay $150 extra, but I’m willing to decide for them that the expense is worth it. What?!
Domar’s study was intended to replicate an earlier study by Paulus from 2002, the first randomized, controlled, prospective study on the impact of acupuncture in 160 IVF patients with good quality embryos. The treatment group got 25 minutes of acupuncture before and after the IVF procedure, and the control group rested for 25 minutes. The pregnancy rate was significantly higher in the acupuncture group.
Domar also mentions a subsequent study that the Paulus’ group did with the same protocol but using sham acupuncture as a control. There was no significant difference. They proposed an “acupressure” effect of sham acupuncture, assuming that both groups achieved a greater pregnancy rate than they would have with no treatment; but the data don’t support that hypothesis since there was no comparable control group of untreated patients selected for good quality embryos. For some reason, they opted not to publish this second, negative study. It was only presented orally at a conference in Madrid in June 2003. (Paulus WE, Zhang M, Strehler E, Seybold B, Sterzik K. Placebo-controlled trial of acupuncture effects in assisted reproduction therapy. 2003: Oral presentation, ESHRE, Madrid, Spain, June 2003;18(Suppl1): xviii18.)
Domar’s intent was to repeat Paulus’ study and also investigate the potential of a placebo effect. Double blind acupuncture studies are impossible: the patients know they are getting acupuncture and the acupuncturist knows he’s giving it. But for the first time Domar tried to blind the rest of the IVF treatment team so no one else knew which group a patient was in. She expected the acupuncture patients not only to have a higher pregnancy rate (PR), but also to be more optimistic and confident that their cycle would be successful, thus demonstrating a placebo effect. The acupuncture patients were indeed less anxious and more optimistic, but they were not more confident, and the PR was not higher.
They were asked “how optimistic do you feel” and “how confident do you feel” about the outcome of the current cycle of IVF. I’m not sure where I would draw the line between optimism and confidence; the whole thing is a bit nebulous. And one could argue that false optimism may not be a good thing. If a patient is confident that the IVF will work this time, she will have to deal with greater disappointment when it doesn’t work.
They can’t understand why they got negative results when previous studies got positive results; but to my mind if acupuncture is no better than placebo, the very fact that the IVF treatment team was blinded would tend to reduce part of the apparent placebo effect. They point out that this is the first study from the United States. In R. Barker Bausell’s book Snake Oil Science, he shows why studies from other countries are more likely to get positive results. (Anecdotally, I recently heard from a friend with contacts in China that a political officer in at least one Chinese research institution reviews all studies before they are submitted for publication and routinely vetoes any studies with negative results.) It is common for a new study in the US to contradict previous studies from elsewhere. And it is common for a well-designed study to contradict results of previous studies.
Now here’s the part that really blew my mind. In 2005, Alice Domar wrote “Acupuncture and infertility: we need to stick to good science.” She pointed out flaws in existing IVF/acupuncture research and mentioned several studies that got negative results (studies that are not mentioned in the review of the literature in the introduction to the current study). She also mentioned one where the acupuncture group had greater pregnancy loss after a second session. She clearly postulated that the apparent effects of acupuncture might be only a placebo effect, and she suggested that there could even be harmful effects. Then she recommended that future studies include a sham treatment. Why, oh why, did she ignore her own advice and do this new study without a sham treatment? She offers no explanation. Even her psychological findings are suspect because she compared a relaxing acupuncture treatment (“how relaxed did you feel during the sessions?”) to an inadequate control: no treatment at all. Would a relaxing massage have produced even better psychological results than acupuncture?
The whole thing smells of “I want to use acupuncture no matter what.” Indeed, Domar heads the Domar Center for Mind/Body Health, where acupuncture and other kinds of “mind/body medicine” are offered.
She does not question the choice of acupuncture points. They were chosen for their sedative effect as well as to increase uterine blood flow. How can she be sure they have these effects? She offers no references to support those claims. If they do have these effects, is there any evidence that sedation and increased uterine blood flow increase fertility? How would that work? Do we have any evidence that anxiety or deficient uterine blood flow impair IVF? (In fact, this study found that the patients in the acupuncture group report reported less anxiety but did not have better outcome).
Isn’t this an example of what I have called Tooth Fairy Science? We can study the amount of money left by the Tooth Fairy in different settings, but since we haven’t determined that there is really a Tooth Fairy, any conclusions we reach will be falsely attributed to an imaginary being rather than to the real cause (parental behavior). In acupuncture studies, the acupuncture points/meridians/qi may be imaginary and we may be studying an elaborate placebo rather than a real physiologic phenomenon.
This study fits a pattern. The original study by Paulus showed an effect of acupuncture on PR but only compared to a no-treatment control group. When he repeated the study with a sham acupuncture control group, there was no difference between the “real” acupuncture and the sham acupuncture groups. When Domar improved on the methodology by blinding the treatment team, acupuncture had no effect on PR. As controls are tightened, the effect disappears.
If this same pattern appeared in successive studies of a prescription drug, I doubt if Domar would be recommending it for all patients, even if the drug had no reported side effects and the patients felt better when they took it – and even if it “only” cost $150. Of course, if the same pattern appeared in drug studies, the drug would not even be on the market.
Domar et al have tried to do good science and have contributed to the evidence against acupuncture. Unfortunately, they seem not to want to accept the implications of their own results; and the poor reporting in the MedlinePlus article only muddied the waters further.