Flag of the World Health Organization (WHO)

Flag of the World Health Organization (WHO)

The World Health Organization (WHO) has recently released a new position statement on mandatory reporting of all interventional clinical trials. This is a positive step in the trend towards higher quality and greater transparency in clinical trials.

The underlying ethical concept here is that the public has a right to data that results from experimentation on humans. The researchers do not ethically own that data. They have been granted the privilege of performing research on humans as part of a social contract that includes the timely public reporting of the data.

The WHO statement calls for all registered clinical trial results to be made public in an open-access public forum in a searchable format within 12 months, and publication in the peer-reviewed literature within 24 months. They urge more rapid dissemination, but state that the 12/24 month time frame is the upper limit of when results should be reported.

The WHO statement is part of a larger trend toward greater transparency but also quality in scientific research. It is part of a recognition that we cannot understand how to best practice medicine and allocate resources based upon individual studies. We need to look at the entire scientific literature as a whole. Even when individual studies may be of high quality, there are many potential factors that can distort the scientific literature and therefore misinform doctors, scientists, and regulatory agencies.

Dissemination bias is one such factor. In the past certain pharmaceutical companies have even argued that they own the data from clinical trials they funded. This argument has been soundly rejected. The debate resulted in the formation of clinicaltrials.gov, a site where all human clinical trials must be registered. This registration has allowed for tracking of how clinical trials are reported.

A 2010 study found that trials with positive results were more likely to be published, and are published earlier, than trials with negative results. A 2015 study published in the NEJM found:

Only 13.4% of trials reported summary results within 12 months after trial completion, whereas 38.3% reported results at any time up to September 27, 2013.

Corporate funded studies with FDA oversight were more likely to be reported than those funded by the government or academic institutions – so this is not a problem isolated to the pharmaceutical industry.

A 2014 study in the Annals of Internal Medicine also found many discrepancies between the trials registered in clinicaltrials.gov and what ultimately is reported. These discrepancies included reporting of secondary outcomes, reporting of adverse events, and even reports of deaths.

Conclusion

One of the missions of Science-Based Medicine is to explore the relationship between scientific evidence and the practice and regulation of medicine. We have learned a great deal over the last couple of decades about this relationship, which is complex and evolving.

It is important to understand how to evaluate the quality of individual studies and how the type of information provided relates to the practice of medicine.

Perhaps even more complex, however, is the relationship between the body of scientific evidence and our ultimate conclusions about practice and regulation. We now know that most individual published studies are wrong. You have to look at the overall pattern in the published literature to make a reliable conclusion.

Often we rely upon systematic reviews to assess the overall pattern in the literature. However, such reviews are likely to reflect any biases in the publication and dissemination of research results. There is a remarkable bias toward publishing positive studies – among researchers, funders, and journal editors. There also appears to be a bias toward provocative studies, which are the very ones that are most likely to be false positives. Exact replications, perhaps the most important type of study, are seen as uninteresting. Some journals, as Richard Wiseman discovered, may even have a policy of not publishing boring exact replications.

What is clear is that we need a global effort to improve the technology of tracking and disseminating clinical trial results to minimize any biases. Fortunately we have a relatively new technology, the internet, to help with this effort. There is no longer any excuse of the pages of a print journal being a limited resource. An unlimited amount of data can be published online.

Ideally we would have a situation in which all human clinical trials are registered before they begin, including details about methods, outcome measures, number of subjects, and comparisons (to reduce researcher degrees of freedom). Once registered, trial progress and results will be tracked and reported in a timely fashion. All this data will exist in a public, open-access, and searchable database so that researchers can easily access the information. Negative results will not be swept under the rug. Exact replications will be encouraged and enthusiastically reported.

As a result, systematic reviews addressing a specific medical question will be much more likely to capture the true answer free of systematic biases. Ethics demand nothing less.

Without thorough and transparent reporting, the rights of subjects in clinical trials are violated. They allow themselves to be experimented on for the advancement of science, and if their results are not reported in a timely fashion then that trust is betrayed. Further, any subject in a future clinical trial, that may have been unnecessary if only previous trials were properly reported, are also being treated unethically. Dissemination bias wastes resources, distorts the practice of medicine, and is a disservice to patients, research subjects, and physicians and regulators who rely upon published data.

The WHO statement is a step in the right direction, although it needs to be backed up with more teeth. We now have the technology to make all clinical trial results transparent and accessible. This has the power to dramatically improve the science of medicine. The ethical imperative to do so seems clear.

 

Posted by Steven Novella

Founder and currently Executive Editor of Science-Based Medicine Steven Novella, MD is an academic clinical neurologist at the Yale University School of Medicine. He is also the president and co-founder of the New England Skeptical Society, the host and producer of the popular weekly science podcast, The Skeptics’ Guide to the Universe, and the author of the NeuroLogicaBlog, a daily blog that covers news and issues in neuroscience, but also general science, scientific skepticism, philosophy of science, critical thinking, and the intersection of science with the media and society. Dr. Novella also contributes every Sunday to The Rogues Gallery, the official blog of the SGU.