“One of the most important discoveries I believe we’ve made that will help you burn fat – green coffee bean extract” – Dr. Oz, September 10, 2012, Episode “The Fat Burner that Works”
Dr. Mehmet Oz may be biggest purveyor of health pseudoscience on television today. How he came to earn this title is a bit baffling, if you look at his history. Oz is a bona fide heart surgeon, (still operating 100 times per year), an academic, and a research scientist, with 300+ or 400+ (depending on the source) publications to his name. It’s an impressive CV, even before the television fame. He gained widespread recognition as the resident “health expert” on Oprah, and went on to launch his own show in 2009. Today “The Dr. Oz Show” is a worldwide hit, with distribution in 118 countries, a massive pulpit from which he offers daily health advice to over 3 million viewers in the USA alone. For proof of his power to motivate, just look at the “Transformation Nation Million Dollar You” program he launched in 2011, enrolling an amazing 1.25 million participants. Regrettably, what Oz chooses to do with this platform is often disappointing. While he can offer some sensible, pragmatic health advice, his show’s content seems more focused on TV ratings than medical accuracy, and it’s a regular venue for questionable health advice (his own, or provided by guests) and poorly substantiated “quick fixes” for health issues. (And I won’t even touch Oz’s guests like psychic mediums.) One need only look at the number of times the term “miracle” is used on the show as a marker of the undeserved hyperbole. Just this week, Julia Belluz and Stephen J Hoffman, writing in Slate, itemized some of the dubious advice that Oz has offered on his show, with a reality check against what the scientific evidence says. It’s not pretty.
I know I’m not alone in my dismay with the Dr. Oz show – As a health professional, Oz should know better than this. He has the education and training that should allow him see through the pseudoscience that he now embraces on a regular basis. I took a close look at Dr. Oz’s show on green coffee beans a few months ago as a case study in pseudoscience, after being inundated with questions about it. In short, green coffee bean extract is derived from unroasted coffee beans and is touted as a source of chlorogenic acid, purported to be the active ingredient in what Dr. Oz called a “miracle” and a “fat burner”. All the clinical trials done on green coffee beans have been small and inconclusive, and a recent meta-analysis noted these limitations, and called for better conducted studies. On his show, Dr. Oz introduced a naturopath touting the product (who also happens to sell his own line of green coffee bean extract, but never disclosed his conflict of interest.) The naturopath and Dr. Oz discussed a small clinical trial by Joe A. Vinson that studied just 16 patients in India. Even from a cursory review of the study, the trial design and reporting was so poor that there was no way to make any definitive conclusions about its efficacy.
Instead of describing the problems with the trial, and cautioning the audience not to get suckered in by cures that look too good to be true, Oz did the exact opposite. He did a case study. He asked two members of his audience to take the supplement for five days. One reported a 2 pound loss, the other, a six pound loss. A five day, unblinded evaluation of a supplement in women who are about to be profiled on television is quite possibly the worst possible way to evaluate a treatment for obesity. But it was good enough for Dr. Oz.
“It’s important for everyone to understand what green coffee bean is, and why I think it’s so important for weight loss.” – Dr. Oz, September, 2012
Perhaps stinging from the well-deserved criticism he endured with his first show and his laughable case study, Dr. Oz recently dedicated another show to green coffee bean. He called it The Green Coffee Bean Project:
To test the effects of the green coffee bean extract for The Dr. Oz Show, the show’s Medical Unit conducted a project under the guidance of experts including Dr. Caroline Apovian and registered dietician Kristin Kirkpatrick. It is the largest project the show has ever done.
So what was the “project”? The website describes the undertaking:
The show’s Medical Unit enlisted 100 women between the ages of 35 and 49 who were overweight with BMIs between 25 and 45. The women were not pregnant or breastfeeding and didn’t have any major medical problems including diabetes, or history of heart attack or stroke. These women received either the green coffee bean extract or a placebo, and were instructed to take 400mg capsules, three times a day – 30 minutes before each meal. No participant knew which pill they were given. Furthermore, the participants were instructed to not change their diet, and to keep a food journal that would help The Dr. Oz Show learn what these women were eating. After two weeks, the study participants were weighed.
In two weeks, the group of women who took the green coffee bean extract lost, on average, two pounds. However, the group of women who took the placebo lost an average of one pound – possibly because they were more aware of their diet for that two weeks because of the required food journal.
There is a bit more detail in the episode clips. All women started with an examination with extensive measurements, including blood pressure, weight, heart rate, and waist size. The bottles are shown on screen for a moment, and the label says “Green Coffee Bean Project”, “For investigational use only” and what looks like “There are no known side effects.” Dr. Oz mentions that the placebo is “almost identical” in appearance. The study product (green coffee bean or placebo) was mailed to participants. All participants were also asked to complete a daily food diary, which appeared to be completed online. The study lasted two weeks.
Oz then moves to discuss the evaluation with the studio audience, which is made up entirely of the participants in the study. Overall, everyone (approximately 100 women, it’s not clear) is reported to have lost 123 lbs over the 2 weeks. Oz then notes that the group taking the green coffee bean extract group lost 81.5 pounds collectively, while the placebo group lost 42 pounds. Assuming 50 participants per group, that’s 1.6lbs in the green coffee bean group, and 0.84 pounds in the placebo group. No final numbers of participants are provided, so it’s not clear what the average weight loss really was.
Does 1.6 lbs versus 0.84 pounds sound convincing enough? There’s not a lot of detail here, but what is reported raises some serious questions about the approach and methodology.
Oz calls this a “project” but we should be clear on what this really was: a randomized clinical trial. He recruited participants, randomized them, gave them a treatment, and measured an outcome. Oz is presumably the principal investigator here, so my first question is if he obtained clearance to conduct this trial from an Institutional Review Board (IRB), either from Columbia University, or elsewhere. As staff at Columbia University, their requirements would presumably apply. Ensuring clinical trials are conducted in ethically sound ways is no trivial matter. Institutional review boards emerged in part as consequence of the horrific medical abuses of the second world war, to protect the right of research subjects. While IRB approval doesn’t guarantee a trial is credible and safe for participants, it does provide an external validity check of its overall approach and ethics. Now it could be the case that the show did obtain IRB approval from some organization who reviewed the protocol, consent forms, and overall methodology. I sincerely hope so, from a research ethics and patient protection perspective.
Another quality indicator is trial registration, a critical issue in research and a means of holding investigators more accountable for reporting their results in a transparent and unbiased manner. Public registration serves as international resource that catalogs human experiments, and allows for more systematic evaluations of treatments. Disappointingly, there are no clinical trials registered at clinicaltrials.gov that refer to green coffee bean. There are trials that are studying chlorogenic acids – but in coffee. Yes, despite what Dr. Oz said on his show, brewed coffee is also a source of chlorogenic acids.
Taking a closer look at the results that Oz reported, there’s even more missing data. No information is provided on how women were recruited, how they were randomized, and how comparable the two group were. The only condition for enrollment seems to have been BMI, which is not a good measure of obesity on an individual basis. There is no discussion of informed consent. Then we come to the blinding and the group allocation (placebo or active treatment). It’s noted that participants didn’t know which group they were in. Was this a”single-blind” trial where the staff collecting data knew of the allocation, raising the risk of bias? Again, it’s not clear. (One need only look at acupuncture trials to see how single-blinded studies generate misleading results.)
Now let’s look at the treatment itself. Two weeks is insufficient for a clinical trial of a drug for weight loss. Real obesity studies last months – not days. And multiple measures are collected before, during, and after the trial, including changes in figures like BMI, waist size, lipid levels, and weight. Reporting only overall weight loss per group, as Oz has done, is both uninterpretable and suspicious. What about all of the other measurements collected? Even looking only at weight loss, we need both the average and the standard deviation. We then need a statistical analysis of the results, to see if the results differ from random chance. None are reported, or even mentioned.
Finally let’s look at the safety. There’s little mention of side effects other than a question that Oz asks the audience. Were participants asked to report on side effects? Were there any differences at all? Again, it’s not reported. Two weeks is insufficient to evaluate safety, in any case. And if the bottle really says, “There are no known side effects,” should we really expect participants to report any negative effects? What still hasn’t been answered with green coffee bean is the long-term safety. Concerns have been raised about the potential for chlorogenic acid to raise levels of homocysteine. While the relationship between elevated homocysteine and cardiovascular disease isn’t clear, it does suggest that the safety profile should be confirmed, and not assumed.
“This stuff really seems to work” – Dr. Oz
This clinical trial makes a mockery of good research methodology. As a clinician scientist, Dr. Oz knows the evidence bar for good research. This made-for-TV trial doesn’t even come close. The idea of doing a clinical trial on green coffee bean extract is a good one. And the Dr. Oz show certainly has the resources to fund a proper evaluation. So why design it in a way that will guarantee meaningless results? If he truly believes that this was a fair and ethical test of green coffee bean extract, I invite the Dr. Oz show to prove it: Show us the IRB approval, show the consent forms, give us the pre- and post-test comparison of the two groups, describe the randomization, show all the collected data, and show us the statistical analysis that let to the conclusion that green coffee bean actually works as a weight loss product. And I’ll happily write up a follow-up post heralding the first non-prescription product ever to generate meaningful weight loss despite no changes in dietary consumption and exercise. I’m not going to hold my breath.
Despite the impression given by Oz, the laws of thermodynamics hold, even on daytime television. There is no magical product that allows you to eat whatever you want, hold your calorie expenditure constant, and still lose weight. Weight changes are driven by by overall calorie intake and expenditure, so weight loss is always a consequence of a negative balance. In the case of the Dr. Oz study, weight loss in both groups was almost certainly due to food journaling of calories, and a heightened awareness of calorie intake simply from being in a clinical trial. In fact, Oz goes on to recommend the food diary as an acceptable alternative to those that don’t want to purchase green coffee bean extract, which may be the most sensible statement he makes in the entire episode.
Conclusion
A year ago green coffee been was a little known product showing some promising but unverified results in small clinical trials. Then came the Dr. Oz effect. Lots of publicity, lots of use of the term “miracle”, and then a made-for-TV clinical trial. Despite all this hype, the evidence base is no better today than it was one year ago. Yet sales have skyrocketed, will millions likely trying green coffee bean, in the absence of good efficacy or safety data. Thanks to Dr. Oz, many consumers have the impression that green coffee bean has been evaluated, it’s safe, and it offers a meaningful benefit for weight loss. In short, it’s a panacea. Unfortunately, there are no panaceas for weight loss, and there are no short cuts when it comes to science. Of course, these uncomfortable realities don’t lend themselves well to daytime television.