If there’s a characteristic that’s common among proponents of alternative medicine, it’s tenacity. The willingness to stick with an idea, no matter the evidence, must give one a certain clarity. The naturalistic fallacy is often the foundation. Natural is good, synthetic is bad, no matter the evidence. In some cases, in spite of the evidence. How one deals with contradictory evidence is an effective tool to differentiate between medicine and alternative medicine – given sufficient convincing evidence, medicine changes its practices. The same can’t be said for alternative medicine, where few treatments are ever discarded. Otherwise practices like homeopathy, acupuncture, reiki, and chiropractic would have disappeared long ago. It’s also why disproven products continue to have occasional resurgences in interest. Cranberry is one. It has been touted as a treatment and a preventative for urinary tract infections for years. And it doesn’t seem to work – not well, and not reliably, if you look at all the trials. And that’s being generous, considering the poor quality of the evidence with its inherent biases. I know a dead parrot when I see one. Yet its advocates, mainly manufacturers (is there a Big Cran?) keep insisting it’s alive, supported by the occasional positive report that appears. With a new systematic review and meta-analysis that declares it’s effective, it’s time to update our review.
Mark Crislip did a nice overview of cranberry juice back in 2011. It’s most commonly used as a supplement (as juice, extract, or capsules) to prevent and treat urinary tract infections (“acute cystitis”). These infections are generally very responsive to antibiotic treatment. If untreated, it can worsen and move up into the kidneys and beyond. Some women (UTIs are uncommon in men) have repeated UTIs, and there are a number of non-drug strategies that can be used to reduce them. Cranberry juice has been a popular folk treatment for years. A decent reason why cranberry might prevent or treat urinary tract infections is still lacking despite five decades of study. An ingredient in cranberry, proanthocyanidins, is thought to block some variants of E. coli bacteria from adhering to bladder cells, which might reduce the risk of infection. But these “fimbriated” bacteria are uncommon in species isolated from urinary tract infections. So even if the theoretical mechanism of action is correct (which remains unproven), the real-world effect, based on this assumption, should be minimal. And that’s largely what the evidence shows, when you account for the poor research quality. But we don’t need to know how it works, if it works. And there is no good evidence that cranberry is an effective treatment for UTIs. The data are less clear with prevention. There are lots of trials and the results are mixed. The best reviews of clinical evidence are systematic in nature, and happily Cochrane has a perspective. A 2012 review noted the following:
Cranberries (usually as cranberry juice) have been used to prevent urinary tract infections (UTIs). Cranberries contain a substance that can prevent bacteria from sticking on the walls of the bladder. This may help prevent bladder and other UTIs. This review identified 24 studies (4473 participants) comparing cranberry products with control or alternative treatments. There was a small trend towards fewer UTIs in people taking cranberry product compared to placebo or no treatment but this was not a significant finding. Many people in the studies stopped drinking the juice, suggesting it may not be a acceptable intervention. Cranberry juice does not appear to have a significant benefit in preventing UTIs and may be unacceptable to consume in the long term. Cranberry products (such as tablets or capsules) were also ineffective (although had the same effect as taking antibiotics), possibly due to lack of potency of the ‘active ingredient’.
The Cochrane review included both juice and capsules of cranberry. Most trials were about 6 months in duration, which is probably the minimum time period in which a difference could be identified. Of the 24 trials Cochrane identified, 13 could be combined in a meta-analysis. The meta-analysis found no significant difference in the rates of recurrent urinary tract infections. There was a suggestion of benefit, with a relative risk of 0.86 but a confidence interval of 0.71 to 1.04. Trials that could not be included in the meta-analysis were also studied, with the majority finding no benefit. Is there any reason to be optimistic? Probably not, given the overall quality of the research. There was little follow-up and lots of patient dropouts, likely a result of the difficulty in taking the juice daily. Perhaps not surprisingly, cranberry manufacturers and their affiliated scientists insist cranberry is effective, despite the evidence, and blamed poor quality products and poor quality trials for hiding what they felt was a benefit. One researcher, Amy Howell, even dismissed the idea that cranberry could be evaluated in a Cochrane review:
“At Cochrane they typically review pharmaceutical drug studies and their prevention of disease. They do not have anyone on their staff that specializes in supplements and functional food products. This is not their strong point,” Howell said. A European commentary in 2005 had already cited problems with Cochrane review of nutrition studies. That review, Howell said, found that, “you really cannot really use drug criteria to review these types of foods. It just doesn’t work. You can’t look at just one thing.” Howell quoted the European review as saying: “It’s hard to even imagine a human clinical trial in which one half of a large group of middle age people agreed to avoid vegetables for five years and then agreed to be followed up to see how many developed of them developed cancer.”
Handwaving and a straw man argument. Studies required the consumption of a specific supplement for about 6 months – not unlike any other clinical trial of an intervention. If cranberry works, we should be able to measure its effects when we test it. Now I’m in agreement with the other contributors to this blog that Cochrane can get CAM wrong. But it’s because they don’t consider prior plausibility – not because the the technique of a systematic review isn’t applicable. If anything, Cochrane’s methodology is somewhat biased towards accepting implausible treatments, not against them. (Consider this evaluation of a homeopathic remedy, for example.)
Now there’s another systematic review of cranberry with a different conclusion. This one is by Wang and colleagues, and it’s entitled, Cranberry-containing products for prevention of urinary tract infections in susceptible populations. It, like Cochrane, is also systematic review and a meta-analysis. This study also identified 13 trials (mainly juice, some capsules) and included 1616 subjects in its dataset. The authors looked at the risk of bias in each trial, and all but two were rated “High Risk” in at least one category.
Their conclusion: a relative risk of 0.62 (with a confidence interval of 0.49-0.80). Just better enough to achieve statistical significance. How did this occur? Some statistical sleight-of-hand. The authors put all the trials into a Galbraith plot, a test to measure statistical heterogeneity (non-uniformity). One is strongly positive (A) and one is strongly negative (B). The authors declare that the negative trial by Babosa-Cesnick is too heterogeneous to be included, so the exclude it. Yet the positive trial by Ferrara, which is also heterogeneous, is included. Guess what happens to the result?
Dropping the negative trial makes the overall benefit from cranberry look positive. So the authors announce that cranberry “seemed to be effective in prevention of UTIs”. The authors then go on to illustrate how questionable the cranberry data set really is. They look at the statistical significance of the combined by removing other trials, one by one. You can see how excluding the negative Babosa-Cesnik data slides the risk ratio to the left (see the bottom line), making the result look much more compelling:
Perhaps not surprisingly, the cranberry producers and their scientists like this study a lot more. The same researcher, Amy Howell, who dismissed the Cochrane methodology seems to find this one a lot better:
Commenting independently on the review’s conclusions, Amy Howell, PhD, from the Marucci Center for Blueberry Cranberry Research at Rutgers University told NutraIngredients-USA: “The results of this latest meta-analysis of clinical trials in a highly regarded journal provide further convincing support for the use of cranberry in the prevention of recurrent UTI in susceptible populations. Results indicate that despite the challenges of utilizing cranberry in different formulations, dosages, and target populations, overall positive outcomes were obtained in many cases.”
In other words, this meta-analysis supports my confirmation bias, so it’s a good paper.
My personal take is that if there is an effect from consuming cranberries, it’s so slight that the clinical trials so far haven’t been large enough to pick it up. Whether it’s an attractive approach, despite the evidence, depends on a a consideration of consequences, alternatives, and costs. The excellent Therapeutics Education podcast did a clinical review of the evidence and included a brief dietary and economic analysis. They estimated that even if we accept the positive data, we need to consider the following:
- You need to consume about 500mL of cranberry cocktail to obtain the studied amount of the pure juice. This costs $0.45-$0.66 per day and could provide 120-150 calories.
- Over 6 months, in order for a 1 in 12 chance of avoiding a UTI (that’s the efficacy), you’d spend about $180 and consume about 45,000 calories which (holding everything else equal) could cause a weight gain of several pounds.
I calculated 61 grams of sugar per day in two cups of Ocean Spray cocktail, which means 24 lbs of sugar consumption over 6 months. So while it’s “natural”, and may not have many acute side effects, it’s not without the potential for unwanted effects.
Finally, let’s look at the alternatives to the alternative. Antibiotics taken for the prevention of UTIs are very effective, reducing the risk of an infection by up to 95%. You only need to treat two people for one year to prevent 1 recurrence. An NNT of 2 is pretty impressive, anywhere you see it in medicine. It’s not a panacea, though. There are justifiable concerns about the risk of antibiotic resistance developing, and there are drug-related side effects, including more yeast infections. However, long-term antibiotics appear to be far more useful as a strategy for those experiencing many infections per year. Some choose just to take antibiotics when infections recur – managing the infection effectively while minimizing overall exposure to these drugs.
Medicine isn’t unlike any other aspect of our lives. We see what we want to see. These two systematic reviews, and their interpretation, illustrate this nicely. Methods matter if we want to minimize bias. Cochrane has the most rigorous approach to data collection, synthesis and analysis. Their conclusion is a reasonable analysis of the evidence base. There is no persuasive evidence that cranberry can prevent or treat urinary tract infections. In comparison, the Wang paper looks more like an exercise in data-dredging, where the analysis was manipulated until a positive result could be identified. This post-study rationalization is akin to throwing darts at the wall, drawing the target around the darts afterwards, and then declaring victory – or in this case, clinical effectiveness. Even taking the analysis at face value, cranberry is only modestly effective, expensive, and may come at a high dietary cost. And, it’s substantially less effective than conventional treatments. So is this the final word on cranberry? It probably should be, but that’s unlikely. It’s a zombie, and it will be back.
Jepson R.G., Williams G. & Craig J.C. (2012). Cranberries for preventing urinary tract infections., Cochrane database of systematic reviews (Online), PMID: 23076891
Wang C.H. (2012). Cranberry-Containing Products for Prevention of Urinary Tract Infections in Susceptible PopulationsA Systematic Review and Meta-analysis of Randomized Controlled Trials, Archives of Internal Medicine, 172 (13) 988. DOI: 10.1001/archinternmed.2012.3004