Laboratory Rats

Suffer the lab rats

Elsevier has announced that they are retracting the infamous Seralini study which claimed to show that GMO corn causes cancer in laboratory rats. The retraction comes one year after the paper was published, and seems to be a response to the avalanche of criticism the study has faced. This retraction is to the anti-GMO world what the retraction of the infamous Wakefield Lancet paper was to the anti-vaccine world.

The Seralini paper was published in November 2012 in Food and Chemical Toxicology. It was immediately embraced by anti-GMO activists, and continues to be often cited as evidence that GMO foods are unhealthy. It was also immediately skewered by skeptics and more objective scientists as a fatally flawed study.

The study looked at male and female rats of the Sprague-Dawley strain of rat – a strain with a known high baseline incidence of tumors. These rats were fed regular corn mixed with various percentages of GMO corn: zero (the control groups), 11, 22, and 33%. Another group was fed GMO corn plus glyphosate (Round-Up) in their water, and a third was given just glyphosate. The authors concluded:

The results of the study presented here clearly demonstrate that lower levels of complete agricultural glyphosate herbicide formulations, at concentrations well below officially set safety limits, induce severe hormone-dependent mammary, hepatic and kidney disturbances. Similarly, disruption of biosynthetic pathways that may result from overexpression of the EPSPS transgene in the GM NK603 maize can give rise to comparable pathologies that may be linked to abnormal or unbalanced phenolic acids metabolites, or related compounds. Other mutagenic and metabolic effects of the edible GMO cannot be excluded.

Sounds pretty scary. Now let’s look at the multiple criticisms:

The biggest criticism of the study is the combination of two features – the small sample size and lack of statistical analysis. The entire study is premised on comparing various dose groups with control groups that were not exposed to GMO or glyphosate. And yet, the authors provide no statistical analysis of this comparison. Given the small number of rats in each group, it is likely that this lack of statistical analysis is due to the fact that statistical significance could not be reached.

In other words – the results of the study are uninterpretable. In the retraction statement Elsevier wrote:

Ultimately, the results presented (while not incorrect) are inconclusive, and therefore do not reach the threshold of publication for Food and Chemical Toxicology.

The retraction reads like a long excuse for the editorial failure of the journal, and is disappointing. But at least they ultimately reached the correct conclusion – this paper should never have been published. It slipped through the cracks of peer review.

If you look at the survival curves for the various groups, I think you will see that the results are all over the place. This is a typical scatter of data with no clear pattern. In the male groups, the GMO and glyphosate groups tended to do better, if anything. In the female groups they did worse, but there is no clear dose-response effect evident, and the overall results are a wash. Inconclusive is being polite – the data do not show anything, especially absent any statistical analysis.

The study has also been criticized for their choice and treatment of animals. Choosing a strain with a very high background rate of tumor is asking for lots of noise in the data. In fact, a study of the strain found:

The total tumor incidences were 70 to 76.7% and 87 to 95.8% in males and females, respectively.

Further, many scientists charged that the rats were not treated ethically. It is standard practice in such studies to establish an endpoint, such as tumor number and size, at which point the animal with be euthanized. In this study the rats were allow to die of their tumors. The more cynical critics of the study speculate that this was done to generate graphic images in order to have the intended effect on public opinion.

Seralini responds

Despite the fact the the article has been withdrawn, Seralini is not backing off his support of the results. In a detailed response to critics, Seralini and his coauthors write in the introduction:

This may explain why 75% of our first criticisms arising within a week, among publishing authors, come from plant biologists, some developing patents on GMOs, even if it was a toxicological paper on mammals, and from Monsanto Company who owns both the NK603 GM maize and R herbicide.

They are immediately trying to frame the discussion as that between vested corporate interests and scientists. This is a diversion, however, and is irrelevant to the substance of the scientific criticism of their paper. Further, much substantive criticism came from the scientific community, not Monsanto. The most recently published, for example, concludes:

We and many others have criticized the study, and in particular the manner in which the experiments were planned, implemented, analyzed, interpreted and communicated. The study appeared to sweep aside all known benchmarks of scientific good practice and, more importantly, to ignore the minimal standards of scientific and ethical conduct in particular concerning the humane treatment of experimental animals.

They also call for Monsanto to release the raw data for the 2004 Hammond study, which was a 13-week trial of Roundup Ready corn. This is a reasonable request in itself – I think all such studies should disclose the raw data for independent review. But the placement of this request in the Seralini response still serves as another diversion from criticism and a way of calling into question the results of the 2004 study, which showed no health effects from the GM corn.

Much of their defense of their protocol consists of pointing out deficiencies in other similar research. While this does address the standards typical in this type of research, it does not address the main criticism, that the cumulative deficiencies in the Seralini study make interpreting the results impossible.

They even defend the lack of statistical analysis of the survival curves, which were the main outcome. They stated that they wanted to be “simply factual” and then again criticize the statistics used in other studies.

Conclusion: Seralini’s study was unethical, poorly-done, and ideologically motivated

The Seralini study suffers from small sample size, lack of statistical analysis, ambiguous results, a questionable selection of rat strain which maximizes noise in the data, and dubious ethical treatment of the animals for possible dramatic effect. At this point anyone referencing this study as support for their position that GMO has health risks sacrifices their credibility.

It helps that the study has now officially been withdrawn, but references to the study in the anti-GMO literature are spread across the internet. The damage is done.

The study is similar in quality to the Carman pig stomach study – which was also worthless but was presented as evidence that GMO is bad. This study also took a random scatter of data and then hunted for any possible illusion of a signal in the noise.

Meanwhile, systematic reviews of the research show no evidence for any health risks from GMO foods.

Seralini et al.’s response to their critics is not convincing, and mostly consists if the tu quoque logical fallacy. They also call for more rigor and transparency in further GMO research. This is hard to disagree with – we can always use more rigor and transparency. This does not, however, answer the deficiencies in the Seralini study.

Calls for more rigorous research also echo what we hear from the anti-vaccine movement. Safety data consists of not finding a health risk, and negative findings are difficult to prove. They are only ever as good as the amount and rigor of our current data. We can always benefit from more and more rigorous safety data, and this makes for an easy criticism for anti-GMO activists or anti-vaccinationists (or anti-fluoridation, or whatever) to make.

At some point we have to set reasonable thresholds for safety data. To the critics, it will never be enough. Vested interests, of course, want the bar to be very low (they would probably be happy with the bar being eliminated, as with the supplement industry). The scientific community, in collaboration with regulators, need to set reasonable, science-based thresholds for safety data, which will never be perfect or guarantee zero risk.

For current GMO foods the evidence seems to be above a reasonable threshold for safety. More data is always welcome, but that does not mean holding an industry hostage to endless calls for more data.

Posted by Steven Novella

Founder and currently Executive Editor of Science-Based Medicine Steven Novella, MD is an academic clinical neurologist at the Yale University School of Medicine. He is also the president and co-founder of the New England Skeptical Society, the host and producer of the popular weekly science podcast, The Skeptics’ Guide to the Universe, and the author of the NeuroLogicaBlog, a daily blog that covers news and issues in neuroscience, but also general science, scientific skepticism, philosophy of science, critical thinking, and the intersection of science with the media and society. Dr. Novella also contributes every Sunday to The Rogues Gallery, the official blog of the SGU.