THREATS TO SCIENCE-BASED MEDICINE
The theme of this blog is science-based medicine. It’s even the name given the blog by our fearless leader, Steve Novella. By “science-based” medicine we generally mean medicine that is both grounded in scientific plausibility based on our best understanding of human physiology and disease as well as in strong evidence from well-designed clinical trials, both of which are extremely important We SBM bloggers tend to concentrate mainly on so-called “alternative,” “complementary and alternative,” or “integrative” medicine because it does indeed represent a major threat to the consensus among medical professionals that medicine should be science- and evidence-based. Moreover, the infiltration of pseudoscientific and antiscientific woo into medical schools, academic medical centers, and medicine at large, coupled with large amounts of money going to promote CAM, both from the government and wealthy private foundations, does represent an extremely worrisome trend that makes all of us, who range from mid-career to retired physicians, fear for the future generation of physicians and their ability to apply science and critical thinking to the evaluation of implausible health claims, such as reiki, homeopathy, applied kinesiology, and the large variety of woo that falls under the rubric of CAM. Worse, this trend began not long after a concerted push to make medicine more science- and evidence-based and less dogma- and authority-based.
Unfortunately, though, the antiscience of implausible health claims is not the only threat that science-based medicine faces. We bloggers here at Science-Based Medicine concentrate on it because its resurgence and infiltration into the very heart of academic medicine represent a sea change in the culture of scientific medicine, which once rightly and without reservation rejected much of what CAM represents as quackery. Also, I can’t speak for others, but pseudoscience interests me; it brings up questions of why people believe irrational and clearly false propositions. That being said, at the risk of ruffling a few feathers among my co-bloggers, I have observed that, if there is one thing that this blog has not to this point emphasized sufficiently, it’s that the commerce of medicine, the very manner in which we develop new therapies, can, if not carefully observed and regulated, represent a threat to science-based medicine even more potent than Andrew Weil, David Katz, and their all-out assault on the very foundations of scientific medicine and drive to return medicine to the days of anecdote-based rather than science-based medicine.
I’m talking about pharmaceutical companies. I’m also about to destroy any opportunity I might ever have to work for or receive any funding from Merck & Company. C’est la vie. A skeptical doc’s got to do what a skeptical doc’s got to do. Not that I won’t at least partially protect myself by adding the disclaimer that the following represents my opinion, and my opinion alone. It does not represent the opinion of my university, cancer institute, or partners.
Now that that’s taken care of, let’s start with a little primer on a pernicious phenomenon known as the “seeding trial.”
PHARMACEUTICAL “SEEDING” TRIALS
What provoked my ire is a study that was published in the Annals of Internal Medicine last week, to which my attention was directed by a friend of mine, PalMD over at denialism blog, as well as over at Health Care Renewal. It is the first article in the medical literature ever to provide “smoking gun” evidence for the existence of a heretofore suspected but never truly demonstrated variety of clinical trial known as a “seeding trial.”
The clinical trial in question is what is now realized in retrospect to have been a seeding trial carried out by Merck for Vioxx. Seeding trials are a special breed of trial that is carried out either shortly after the Food and Drug Administration (FDA) approval of a new drug or while a drug is being considered for approval by the FDA. Normally, one would think that a trial shortly after FDA approval would be for the purposes of post-approval surveillance of a drug for safety and efficacy. After all, the phase III clinical trials that are used pre-approval to establish the value of a new drug are not and cannot cover nearly as many patients as will be taking the drug after it’s approved. There are sometimes rare effects that show up only after a drug has been administered to hundreds of thousands or millions of people. Given that it’s utterly impractical to test a drug on that many people before its approval, and post-approval monitoring is essential.
That’s not the kind of clinical trial we’re talking about here.
What we’re talking about is a study that appears on the surface to be a valid clinical trial–and that might actually be a valid clinical trial–but that is in reality is designed primarily not to obtain new knowledge about a drug, but rather for a different, unspoken and unrevealed purpose. This type of trial, known as a seeding trial, involves coming up with a hypothesis to be investigated that is either not that important or is scientifically redundant (i.e., studied before adequately) and then designing a study in such a way as to get the new drug into the hands of as many primary care physicians as possible in order to influence them to use the drug after it is approved.
Community primary care doctors rarely participate in clinical trials. There are many reasons for this, not the least of which is a lack of training in clinical trial methodology and ethics, but the primary reason is because they are very busy. Signing up patients for clinical trials is very time-intensive. It takes hours to check patient eligibility, explain the trial, obtain informed consent, register a patient with the clinical trial office doing the trial, and doing the necessary documentation. Most community physicians have not been trained in clinical research and do not have the skill set to deal with clinical trial issues themselves, not to mention that individual private offices, even fairly large group practices, lack the necessary infrastructure to support all the documentation and regulatory requirements of clinical trials. Moreover, many community primary care practitioners operate on razor-thin margins; spending that much time signing up patients can seriously jeopardize the financial health of their practice.
Enter the pharmaceutical company and the seeding trial. In such trials, many, many physicians are signed up as investigators, so that each doctor only has to sign up a relatively small number of patients. Thus the effort is not nearly as onerous. Not only that, but the drug company pays physicians for each patient to cover (or more than cover) lost clinical revenue and provides all the support necessary for monitoring and paperwork. Meanwhile the reward for physicians participating is the prestige that comes with being an investigator in a clinical trial, coupled with in essence no financial penalty to the doctor’s practice and not a lot of work. In reality, however, the design of the trial is usually very inefficient for purposes of asking the putative question being asked. For a real scientific question being tested in a clinical trial, more often than not it would be far more efficient to concentrate the patient accrual in a few large academic centers that could find patients much more quickly and already have the infrastructure to do clinical trials. For seeding trials, however, the exact scientific question being asked is almost besides the point, an afterthought. The real, unstated purpose of such trials is to expose as many doctors as possible to using the drug and thereby make them comfortable using it. The real purpose of seeding studies is to make these physicians advocates for the new drug. The real purpose of seeding trials is marketing, not science.
THE ADVANTAGE TRIAL FOR VIOXX
The Annals article drives this point home quite effectively with a case study of the ADVANTAGE (Assessment of Differences between Vioxx and Naproxen To Ascertain Gastrointestinal Tolerability and Effectiveness) clinical trial run by Merck. The stated purpose of this trial, which involved 5,557 patients at 600 different sites, was to compare gastrointestinal side effects and tolerability of Merck’s new drug Vioxx with those of an established drug (Naproxen). The results of the ADVANTAGE study were published in 2003 in–of all journals–the Annals of Internal Medicine. (Yes, I do take into account that some of the venom directed at Merck in the article and the accompanying editorial might come from outrage at having been played by Merck.) Not surprisingly, the results of the ADVANTAGE trial showed that Vioxx was superior to Naproxen when it came to gastrointestinal side effects. However, there was a huge problem.
As Merck internal documents clearly show, the ADVANTAGE trial was primarily a marketing tool. Think of it as a direct-to-physician sales pitch, as demonstrated quite clearly by this document, which I urge you to take the time to read. Here’s a key passage from this document originating from Merck’s marketing department:
The ADVANTAGE (Assessment of Differences between Vioxx and Naproxen To Ascertain Gastrointestinal Tolerability and Effectiveness) trial is the largest ever initiated prior to the launch of a Merck product. The objectives were to provide a product trial among a key physician group to accelerate uptake of VIOXX as the second entrant in a highly competitive new class and gather data important to this customer group. The trial was designed and executed in the spirit of the Merck marketing principles, as described below.
First, the trial was targeted to a select group of critical customers. The clinical trial program for VIOXX focused primarily on specialists. While they would be critical to early uptake and advocacy for VIOXX, the large majority of prescriptions in the A&A market (~60%) come from primary care physicians. The ADVANTAGE trial utilized this important group of prescribers as investigators. In addition to gaining experience with VIOXX, many of these physicians gained a highly coveted introduction to clinical research.
In other words, the introduction of these physicians to the joys and glory of clinical research was secondary, and the actual scientific question to be answered was dead last in Merck’s considerations. Indeed, the most important part of the memo appears to be the part where it is emphasized how careful the tracking of marketing-related results; i.e., numbers of Vioxx prescriptions written by study physicians during and after the study. There’s also this slide from a Merck presentation, which described the role of the marketing department in the design and execution of ADVANTAGE thusly:
- Design protocol and oversee execution of trial
- Select investigator sites
- Run investigator meetings
- Choose and manage CRO (clinical research organization)
- Perform data analyses
- Prepare publications
That’s right. Merck’s marketing department ran virtually every aspect of this trial.
Worse, even though the first author of the main publication resulting from ADVANTAGE was Dr. Jeffrey R. Lisse, an academic physician not employed by Merck, as reported in the New York Times, he did not have a role in data collection or analysis and was quoted thusly:
“Merck designed the trial, paid for the trial, ran the trial,” Dr. Lisse said. “Merck came to me after the study was completed and said, ‘We want your help to work on the paper.’ The initial paper was written at Merck, and then it was sent to me for editing.”
Clearly, Dr. Lisse was seriously compromised ethically by Merck, as well. He should have either refused to help or demanded a much greater role in the analysis of the data. To do otherwise was to let oneself become in essence a front man for a big pharma-produced marketing document. On the other hand, the inducements are quite hard to resist. What harried academic physician can resist being listed as the first author on a manuscript describing the results of a large, important-appearing clinical trial without having to do all that much work? I’d like to think that, if I were ever put in Dr. Lisse’s position, I could do the right thing, but I’ve never been put in that situation.
Even some officials at Merck were reportedly uneasy about this trial. For example, Dr. Edward M. Scolnick, Merck’s top scientist from 1985 until 2002, was unhappy about the ADVANTAGE trial:
In e-mail messages on April 7, 2001, to Dr. Douglas A. Greene, an executive vice president at Merck Research Laboratories, Dr. Scolnick wrote that he was especially angry because the Advantage trial had no scientific purpose. In theory, Merck set up the trial to show that Vioxx caused fewer stomach problems than naproxen. But Merck had already demonstrated that with the Vigor trial, which tracked more than 8,000 patients for a year.
He also wrote in an e-mail:
[T]he reason we have resisted doing large marketing clinical studies is just this. It opens a lot of data to FDA that compromises the large clinically meaningful trials. Small marketing studies which are intellectually redundant are extremely dangerous and the PAC [Products Advisory Committee] system with the marketing emphasis in CDP [Clinical Development Program, a part of Merck’s Marketing Division] on all their studies opens Pandora’s box which we have urged against from the beginning of time. Their budget is now 179 million for CDP—as much as our phase 2/3 new chemical entities used to be. I have told [another Merck colleague] I think it is wasteful.
Dangerous and wasteful indeed. It also brings up several questions, as asked by MedinformaticsMD:
- If the President of the Research Labs, ostensibly the head of R&D at the company, could not stop such a “seeding trial” study of a drug not yet released and/or its publication, then who was actually in charge of R&D?
- Was the go-ahead for the study approved by erstwhile CEO Gilmartin? The Board?
- Was there internal conflict at high levels over this and similar issues of control?
- Why were “seeding studies” that the head of R&D himself called “intellectually redundant” and “extremely dangerous” being funded, while the needs of R&D and new drug discovery in particular were being underfunded?
The answers to these questions are critical to understanding how controls could have broken down at Merck to the point where such a travesty of a clinical trial could have been greenlighted in the first place.
SEEDING TRIALS: SUBVERTING SCIENCE-BASED MEDICINE
To boil it all down, here’s my perspective on why seeding trials such as this are inherently unethical, regardless of whether the clinical scientific question under study is worthy or the design of the trial can produce an answer to that scientific question. It all comes down to one issue: Deception. There is no way to carry out a seeding trial without deceiving virtually everyone involved below the level of study designers. First, the physicians recruited as investigators can’t be told that they are in fact study subjects as well, given that a main purpose of a seeding study is to determine whether exposing them to the use of the drug under study will change their prescribing habits and turn them into advocates for the glories of the drug. The renumeration and stroking by the drug company also compromise their objectivity to an unacceptable degree. While it’s true that no clinical investigator can ever be 100% objective because all clinical investigators have at least some degree of an emotional investment in the outcome of the studies they design and perform, that’s one reason why we have so many safeguards in the regulations governing clinical trials. Second, few, if any, reputable clinical researchers would be willing to sign on as co-investigators for a trial if they knew the primary purpose of the trial was marketing. Third, the patients recruited as human subjects can’t know one of the major questions of the study, again whether physicians’ prescribing habits can be changed. This alone means that truly informed consent on the part of the study subjects is impossible. Finally, the very committee charged with protecting the interests of human subjects in clinical trials, the Institutional Review Board (IRB), must also be kept in the dark regarding the marketing aspect of the study. The reason is obvious. Virtually no IRB would approve a human subjects study whose major purpose is clearly more for marketing than answering an important scientific and clinical question.
As an accompanying editorial by Harold C. Sox, MD, Editor of Annals of Internal Medicine, and Drummond Rennie, MD puts it:
No one told Annals the true purpose of ADVANTAGE. We learned about it when we received a letter to the editor from Dr. David Egilman, who was a consultant to the plaintiffs’ attorneys in the civil suits against Merck (7). He had access to publicly accessible trial documents, which included Merck employees’ e-mail messages that disclosed the true intent of the ADVANTAGE trial. These messages are the meat of the article about seeding trials published in this issue by Hill and colleagues (8). To our knowledge, this article is the first to provide documentary evidence that proves the existence of seeding trials. Other than an excerpt from a single industry document cited in an article by Kessler and colleagues (9), we have not had “smoking gun” evidence, in which the perpetrators are on public record about why they conducted a trial like ADVANTAGE. The article provides clear evidence that the intent of ADVANTAGE was to increase prescriptions of Vioxx (the study outcome of greatest interest to Merck seems to have been Vioxx prescribing rates). However, despite the large body of documents searched by the authors, they discovered few details about exactly how Merck’s marketing division carried out ADVANTAGE.
The documents do tell us that deception is the key to a successful seeding trial. That information–once it becomes general knowledge–could be the fatal blow for seeding trials. Institutional review boards, whose purpose is to protect humans who participate in research, would probably not likely approve an action that places patients in harms’ way in order to influence physicians’ prescribing habits. If they knew, few established clinical researchers would participate as coinvestigators. Few physicians would knowingly enroll their patients in a study that placed them at risk in order to provide a company with a marketing advantage, and few patients would agree to participate. Seeding trials can occur only because the company does not disclose their true purpose to anyone who could say “no.”
The problem for clinical investigators and busy clinicians, as well as IRBs, is identifying a seeding trial. It’s not as easy as it may seem. In fact, it can be quite difficult without knowledge that is withheld. Keep in mind that the ADVANTAGE trial is the first drug company-sponsored clinical trial for which strong evidence has come to light supporting the contention that it was in fact a seeding trial, and that evidence only came to light because there was a problem with the drug under study that led to a rash of lawsuits. Indeed, that the ADVANTAGE trial was a seeding trial did not become apparent until long after it was published, and an article in the New York Times suggested that its purpose was more for marketing than for science. One possibility is for investigators and IRBs to ask pointed questions about the intent of a trial. However, asking about the intent of the trial may not be enough, and the very issue of “intent” brings up a number of thorny issues in itself, as described again in the editorial:
Asking about intent may be the wrong approach. Does the goodness of a trial inhere in its intent or in something else? An adequately powered trial is good if it tries to answer an important scientific question on which patients should be in equipoise, even though participating physicians will become familiar with a new drug and be more likely to prescribe it. Is the same trial bad if its main purpose is to increase drug sales by habituating trial physicians to prescribing a new drug? Couldn’t the answer to this question depend on the scientific importance of the question that it addresses? The trial would be bad if it addressed a question that lacked merit but good, despite its intent, if the question had intrinsic merit. Does the motivation for the trial make it wrong to participate in it, or does addressing an important, unsettled question make the study worthwhile, despite its intent? Perhaps physicians should focus less on intent and more on the scientific question. The ADVANTAGE physicians should have been asking whether the trial addressed an issue that previous trials had already answered—or should have answered.
Personally, I would answer that a trial is “good” if it asks an important question, is well-designed and adequately powered to answer that question, and the answer to that important question is the primary purpose of the trial. If such a trial habituates its physician-investigators to using the drug being tested and persuades them of its superiority as a secondary outcome, that is not a bad thing–especially if the drug is indeed superior to older remedies. However, habituating physicians to using a new drug so that they prescribe more of it should not be considered in the design of a trial.
With regards to the scientific value of a clinical trial, further complicating the issue, it’s usually anything but a simple matter to ascertain whether the scientific hypothesis to be tested is reasonable or redundant. For example, ten years ago, when the ADVANTAGE trial was being planned and submitted to IRBs, the scientific question to be answered by the trial was not an unreasonable one–if one was not sufficiently aware of the results of the Vigor Trial to point out that the question being asked had already been answered with a high degree of confidence. Moreover, as a matter of general clinical trial design for testing new drugs, there are perfectly valid reasons for doing a trial using many community practitioners rather than academic centers, inefficiencies of this approach notwithstanding. The key advantage is to obtain “real world” data about side effects and efficacy when a drug is prescribed “in the real world,” so to speak. Academic medical centers that do a lot of clinical trials are places with a “rarified” atmosphere, so to speak, as well as resources not available to the vast majority of community physicians not affiliated with an academic medical center. It is often of considerable interest to know how a drug will work “in the wild.” In addition, some inducement is necessary to get community physicians on board, given that the constraints on their time and finances usually make being an investigator in a clinical trial too onerous to undertake. Otherwise, it takes a Herculean devotion to clinical research for a busy private clinician to take the time it takes to enroll patients in clinical trials. With that in mind, I’ve come to the conclusion that, if there is one characteristic that is common in a seed trial, it’s usually that the hypothesis to be tested is redundant. In other words, the trial is asking a question that has already been answered. Certainly, this was the case with the ADVANTAGE trial, as the incidence of gastrointestinal side effects had already been assessed and shown to be lower than those due to Naproxen in an earlier trial known as the Vigor Trial. However, it may not always be the case that the question being asked has alrady been answered. It may be that the question is simply not particularly important, which is admittedly a scientific and value judgment.
FINDING THE BALANCE BETWEEN MAMMON AND SCIENCE-BASED MEDICINE
Given the huge expenses involved in developing new medications and bringing them to market, I am under no illusions that drug development will ever be done to a significant degree by any other entities than for-profit companies, and I would still partner with a pharmaceutical company if that’s what it took to develop a future discovery I might make into a viable drug or treatment. After all, what good is “translational” research if it never gets translated, and pharmaceutical companies will always have a major role to play in such translation. Indeed, the profit motive, when harnessed and regulated properly, is an incredibly powerful incentive to develop better therapies. Unfortuantely, it can also be an equally powerful incentive for drug companies to launch trials like the ADVANTAGE trial. Moreover, pharmaceutical companies are more highly regulated than the CAM practitioners and supplement hawkers who are also threats to science-based medicine.
Nor am I under any illusion that the authors of this study are as pure as the driven snow with respect to objectivity. They clearly are not. Each and every one of them was compensated for participation in the current ongoing litigation against Merck at the request of plaintiffs. Indeed, their access to internal Merck documents reproduced in the Annals paper came about through the discovery process in this massive litigation. In other words, each and every author of this study is a hired gun for the plaintiffs suing Merck, and no doubt this Annals article will feature prominently in the many lawsuits against Merck. I’ve railed against “litigation-driven” research before in the context of, for example, the dubious and pseudoscientific “research” that antivaccinationists generate to support lawsuits against vaccine manufacturers, and this article, although not as scientific article, cannot help but be touched the same sort of taint. I acknowledge that, just as I acknowledge that this article represents only one side of the story. Even so, that particular side of the story is so compelling and so disgusting that it is very difficult to imagine a defense or documents that would come close to balancing the clear marketing purpose of the ADVANTAGE study with a legitimate scientific purpose. Although it is (barely) conceivable that that mitigating documents might come to light during Merck’s defenses against Vioxx lawsuits internal documents largely speak for themselves; res ipsa loquitur, if you will. They make it very clear that the ADVANTAGE trial was a seeding trial using a previously answered scientific question as the basis of a redundant clinical trial whose real main purpose was to “seed” experiences prescribing Vioxx to several hundred primary care physicans.
For science-based medicine to be accepted and trusted by the public, the public must be confident that the clinical trials designed to test the safety and efficacy of new drugs conform to the highest standards of science and clinical research. More importantly, they must know that these trials are as free from bias and hidden agendas as is humanly possible to achieve. No clinical trial will ever be completely free from such potential confounders, but without transparency it is much more likely that such hidden agendas will find their way into them. Also, a personal standpoint, I detest Merck’s actions because it will make it that much more difficult for honest clinical investigators like myself to win patients’ trust and persuade them to agree to participate in clinical trials, particularly trials in which a pharmaceutical company is involved. The distrust will spill over to all clinical trials, even NIH-funded trials. A final reason why seeding trials like ADVANTAGE are so pernicious is because they undermine faith in science-based medicine and contribute to the distrust of “conventional” medicine by corrupting academic physicians, community physicians induced to take part in them, and even clinical science itself. I predict–no, I know–that advocates of unscientific and implausible treatment modalities will draw sustenance for their paranoia and conspiracy-mongering about “conventional” medicine from the ADVANTAGE trial trial. I know that they will frequently shove my nose into the ADVANTAGE trial whenever I argue for scientific medicine.
Whether we advocates of scientific medicine realize it or not, Merck just made our jobs a lot harder.
REFERENCES:
1. Hill, K.P., Ross, J.S., Egilman, D.S., Krumholz, H.M. (2008). The ADVANTAGE Seeding Trial: A Review of Internal Documents. Annals of Internal Medicine, 149(4), 251-258.
2. Sox, H.C., Rennie, D. (2008). Seeding Trials: Just Say “No”. Annals of Internal Medicine, 149(4), 279-280.