I thought I’d do a little SBM 101 series for our lay readers. Forgive me if this information is too basic… at least it’s a place to start for those who are budding scientists and critical thinkers. 🙂

***

Which news source do you trust more: The New York Times or The National Enquirer? Which news reporter would you trust more: Charlie Gibson or Jerry Springer? As it turns out, medical journals and science researchers run the gamut from highly credible and respected to dishonest and untrustworthy. So as we continue down this road of learning how to evaluate health news, let’s now turn our attention to pillar number one of trustworthy science: credibility.

In medical research, I like to think of credibility in three categories:

1. The credibility of the researcher: does the researcher have a track record of excellence in research methodology? Is he or she well-trained and/or have access to mentors who can shepherd along the research project and avoid the pitfalls of false positive “artifacts?” Has the researcher published previously in highly respected, peer reviewed journals?

2. The credibility of the research: does the study design reflect a clear understanding of potential result confounders and does it control for false positive influences, especially the placebo effect?

3. The credibility of the journal that publishes the research: top tier journals have demonstrated a track record of careful peer review. They have editorial boards of experts who are trained in research methodology and are screened for potential conflicts of interest that could inhibit an objective analysis of the research that they review. The importance of careful peer review must not be underestimated. Some say that the quality of a product is only as good as its quality control system. Top tier journals have the best quality control systems, and the articles they publish must undergo very careful scrutiny before they are published.

So as a lay person, how do you evaluate the credibility of a health news report? In practical terms, here’s what I’d recommend:

1.  Look at the name of the journal reference – where was the research published? Is it from a top tier journal? R. Barker Bausell considers the following journals to be “top tier:” The New England Journal of Medicine (NEJM), The Journal of the American Medical Association (JAMA), Annals of Internal Medicine, Nature, and Science. I might cast a slightly larger net, but no one will argue that these are certainly some of the most respected journals in medicine and science.

2. Look at the study design described in the research article abstract. Was it a randomized, controlled, double-blind, placebo-controlled trial? Were there more than 50 subjects in each group? Did the authors overstate their conclusions? This sort of analysis is challenging to the lay person – so do it if you can, but if it proves too difficult, fall back on credibility check #1.

3. Look at the primary author of the research. Search for his/her name in the National Library of Medicine’s Medline database and see what other research he or she has done, and where it was published.

If the news report is based on credible research, you may feel confident in taking the results more seriously (so long as the media is representing them accurately). But before you hang your hat on a journal’s reputation (they have their own publication biases), let’s take a look at the other 2 pillars of trustworthy science: plausibility and reproducibility. These two will help you navigate your way through the vast gray zone, where the credibility check doesn’t pass with flying colors – or maybe you’re dealing with neither Charlie Gibson nor Jerry Springer.

Posted by Val Jones

Val Jones , M.D., is the President and CEO of Better Health, PLLC, a health education company devoted to providing scientifically accurate health information to consumers. Most recently she was the Senior Medical Director of Revolution Health, a consumer health portal with over 120 million page views per month in its network. Prior to her work with Revolution Health, Dr. Jones served as the founding editor of Clinical Nutrition & Obesity, a peer-reviewed e-section of the online Medscape medical journal. Dr. Jones is also a consultant for Elsevier Science, ensuring the medical accuracy of First Consult, a decision support tool for physicians. Dr. Jones was the principal investigator of several clinical trials relating to sleep, diabetes and metabolism, and she won first place in the Peter Cyrus Rizzo III research competition. Dr. Jones is the author of the popular blog, “Dr. Val and the Voice of Reason,” which won The Best New Medical Blog award in 2007. Her cartoons have been featured at Medscape, the P&S Journal, and the Placebo Journal. She was inducted as a member of the National Press Club in Washington , DC in July, 2008. Dr. Jones has been quoted by various major media outlets, including USA Today, The Wall Street Journal, and the LA Times. She has been a guest on over 20 different radio shows, and was featured on CBS News.