NB: If you haven’t yet read Part 1 of this blog, please do so now; Part 2 will not summarize it.
At the end of Part 1, I wrote:
We do not need formal statistics or a new, randomized trial with a larger sample size to justify dismissing the Gonzalez regimen.
In his editorial for the JCO, Mark Levine made a different argument:
Can it be concluded that [the] study proves that enzyme therapy is markedly inferior? On the basis of the study design, my answer is no. It is not possible to make a silk purse out of a sow’s ear.
That conclusion may be correct in the EBM sense, but it misses the crucial point of why the trial was (ostensibly) done: to determine, once and for all, whether there was anything to the near-miraculous claims that proponents had made for a highly implausible “detoxification” regimen for cancer of the pancreas. Gonzalez himself had admitted at the trial’s inception that nothing short of an outcome matching the hype would do:
DR. GONZALEZ: It’s set up as a survival study. We’re looking at survival.
SPEAKER: Do you have an idea of what you’re looking for?
DR. GONZALEZ: Well, Jeff [Jeffrey White, the director of the Office of Cancer Complementary and Alternative Medicine at the NCI—KA] and I were just talking a couple weeks ago. You know, to get any kind of data that would be beyond criticism is—-always be criticism, but at least three times.
You would want in the successful group to be three times — the median to be three times out from the lesser successful groups.
So, for example, if the average survival with chemo, which we suspect will be 5 months, you would want my therapy to be at least — the median survival to be at least 15, 16, 17 months, as it was in the pilot study.
We’re looking for a median survival three times out from the chemo group to be significant.
Recall that the median survival in the Gonzalez arm eventually turned out to be 4.3 months.
It would not surprise me if Dr. Levine, the author of the JCO editorial, were to take issue with what I’ve written so far. I would be remiss, for example, to leave readers with the impression that he called for another, larger trial, performed not as a cohort study but as a randomized, controlled trial, in order to prove whether (or not) “enzyme therapy is markedly inferior” to chemotherapy for cancer of the pancreas. He specifically did not make that recommendation; instead he argued, reasonably, “Given the scarcity of resources for cancer research, there are many more important questions to address.” I’ll come back to that statement, because although it hints that Dr. Levine has intuitively recognized the need for invoking prior probability when considering health research policy, he still appears confused by a strict, EBM perspective.
Human Studies Ethics: why Science Matters
Dr. Levine may also feel misrepresented by my emphases on the pseudoscientific nature of the Gonzalez regimen and the ethics of the trial. Near the beginning of his editorial, possibly to distance himself from those very issues, he wrote:
The goal of this discussion is not to debate the merits of conventional medicine versus CAM.
Let’s cut to the chase: “the merits of conventional medicine versus CAM,” at least insofar as the phrase refers to public perceptions or individual choices of patients or even physicians, are not the issues here. The overwhelming issues are ethical and scientific, and the two are not distinct: human studies ethics must be informed by science. I’ve explained this in some detail elsewhere in this long series. In summary, the Gonzalez trial repeatedly violated the Helsinki Declaration and other human studies treatises, including those established by the NIH itself and offered by NIH ethicists: it made Gonzalez, who is neither “a scientifically qualified person” nor “a clinically competent medical person,” responsible for human subjects; it did not minimize risks or enhance potential benefits, nor did “the potential benefits to individuals and knowledge gained for society … outweigh the risks” (it was a “trifling hypothesis”); it did not have “respect for enrolled subjects,” as evidenced by the horrible experience of at least one subject whose story was told by his friend, mathematician Susan Gurney; it did not “conform to generally accepted scientific principles…based on a thorough knowledge of the scientific literature, other relevant sources of information, and on adequate laboratory and, where appropriate, animal experimentation”; instead, it justified itself by citing a flawed, non-consecutive case series offered by Gonzalez himself, which both Dr. Peter Moran and I deconstructed with only a bit of effort (see also here), and by the “popularity” fallacy.
Perhaps most disturbing, even to people who aren’t familiar with the literature of human studies ethics, is that the trial investigators almost certainly failed to provide prospective subjects with the knowledge necessary for informed consent. Such knowledge would have consisted of the cancer biology known at the time, which overwhelmingly predicted that Gonzalez’s regimen would have no beneficial effect on cancer of the pancreas; of an accurate, honest assessment of his case series; and of his history of peddling pseudoscientific claims and providing incompetent care. I’ve written “almost certain” because I haven’t seen more than a short excerpt possibly from the trial’s consent form, although I’ve tried to do so (hint: if anyone reading this has a copy please send it on). I can’t imagine, however, that the consent form included comprehensive information because if it had, no competent IRB would have approved the study and very few subjects would have enrolled.
The history told by Susan Gurney about her friend, moreover, strongly suggests that he had not been adequately informed:
I told him that I was going to attend the annual conference of the American Society of Clinical Oncology (ASCO) and would report on other options to him. Once at ASCO, I learned quickly and definitively that the Gonzalez protocol was a fraud; no mainstream doctors believed it was anything else and they were surprised that anyone with education would be on it…
By remaining neutral about the Gonzalez regimen, physicians at Columbia Presbyterian who place patients in this trial effectively preclude them from starting other options, because of the demands it places on patients and their families. If physicians believe they are truly being neutral by not fully explaining the Gonzalez protocol’s nature to cancer patients, it is they who are in denial.
Michael Specter, who wrote about Gonzalez for the New Yorker in 2001, reported similar faux neutrality from Karen Antman, then the chief of Columbia’s division of medical oncology and a past president of the American Society of Clinical Oncology, who would be a co-author of the eventual JCO report. First, she showed him the excerpt that I mentioned above, which Mr. Specter characterized as “instructions that Columbia gives patients interested in the Gonzalez study”:
Many Americans who develop advanced cancer for which standard treatments have little to offer, turn to alternative or complementary therapies….There is no current conventional medical support for the theories and assumptions underlying the use of Nutritional Therapy. The Columbia College of Physicians and Surgeons does not support its use except as part of a properly conducted clinical trial.
If that bland statement was all prospective subjects were told about “conventional medical support” for the Gonzalez regimen, it hardly constituted a responsible explanation. Mr. Specter himself, who is not a physician or scientist, sensed that as well:
I asked if it would be right to infer that she thought the trial wouldn’t work. She shook her head. I asked if she had an idea why it might work. She said no. Did she have any opinions at all about the potential of nutritional therapy or the Gonzalez regime? “I have lots of opinions,” she told me, “but none of them matter.”
Except that such opinions do matter, as any human study investigator is expected to know. Susan Gurney explained why they would have mattered to her friend:
He was an artist—a painter and a sculptor—and he had little scientific knowledge. When Dr. Chabot was neutral about the Gonzalez protocol, and when Dr. Antman said nothing adverse about it, my friend assumed that they must genuinely believe that the treatment could work.
The trial’s Principal Investigator, John Chabot, was also reticent to offer subjects accurate information about the Gonzalez regimen, as evidenced by a 1999 “Dear Prospective Patient” letter. These findings—that oncologists in general thought that “the Gonzalez protocol was a fraud,” but that the Columbia investigators failed to disclose that fact, and the reasons for it, to subjects—exposes another ethical violation that I’ve previously discussed: the trial lacked clinical equipoise.
Finally, it is clear, as discussed here and elsewhere in this series, that the underlying impetus for the trial was political, not scientific. Shouldn’t that have also been included in the consent form?
EBM Gets it Wrong
Keeping all that in mind, let’s revisit the quotation from Dr. Levine explaining the goal of his editorial, this time putting it into the context of the entire paragraph:
I am fortunate to have spent my entire academic career at McMaster University (Hamilton, Ontario, Canada), the birthplace of evidence-based medicine, and to have had the privilege of learning from colleagues such as David Sackett, MD, and Gord Guyatt, MD. The goal of this discussion is not to debate the merits of conventional medicine versus CAM. Rather, the objectives are to consider whether these two paths of medicine should be held to the same standards of evidence in terms of clinical and policy decision making and to determine whether a model that historically has been based on weak evidence can be reconciled with the new paradigm of the requirement for high-quality evidence.
Yes! A thousand times yes! They should be held to the same standards of evidence. But that means that they should be required to meet the same preliminary standards before being accepted for high-quality human trials, and those standards are about scientific evidence: evidence from basic science, evidence from the biology of the disease in question, evidence from animal studies, and evidence from whatever preliminary human case reports or uncontrolled trials there may be. The Gonzalez regimen failed all of those requirements, and the trial was thus scientifically unjustified and unethical from the start.
Dr. Levine, however, is misled by the limited EBM understanding of “evidence,” as is suggested by the last line of the paragraph. Later, he leaves no doubt:
This is one of the most challenging editorials I have had to write. Wearing the hat of a clinical epidemiologist, it is difficult not to find fundamental flaws in both of these trials. Not too long ago, I would have dismissed them out of hand. However, as a clinician, I recognize that many of my patients seek CAM therapies, and many are using them and afraid to tell me. I am troubled by the lack of evidence for many of these therapies and the costs that patients incur in using them. In the past, there was a reluctance to subject CAM therapies to the same standards of evaluation as those for conventional therapies. Should CAM therapies undergo the same type of rigorous evaluation to which conventional therapies are subjected? Absolutely! When authors submit clinical trials to JCO, they are instructed to follow specific guidelines. These guidelines should be the same whether the intervention is chemotherapy, radiation, an herbal remedy, or a psychosocial intervention.
Yes, there are specific guidelines for publishing reports in the JCO, and in the case of the Gonzalez report, they were violated. I wonder why Dr. Levine didn’t mention CONSORT? Clearly, the “type of rigorous evaluation” to which he is referring is a clinical trial. Otherwise, there would be little trouble subjecting scientifically incoherent methods (not ‘CAM therapies,’ which begs the question) to the same standards of evaluation as those for scientifically coherent methods (not ‘conventional,’ which has irrelevant social connotations). The beginning of such an evaluation is to judge how plausible the method might be, and the way to do that is by considering scientific evidence. This, Dr. Levine, is evidence.
Similarly, there was not merely a “lack of evidence” for the Gonzalez regimen; there was abundant evidence against it. As suggested above, Dr. Levine must understand this at some level: although he argued, incorrectly and irrelevantly, that the trial had failed to prove that “enzyme therapy is markedly inferior” to gemcitabine, he nevertheless opined that “Given the scarcity of resources for cancer research, there are many more important questions to address.” How could he assign even a qualitative measure of “importance” without considering prior probability?
Yet again he retreats to the safe haven of EBM, failing to recognize the ethical mischief wrought by ignoring science:
Chabot et al should be congratulated on their persistence and determination to compare pancreatic enzymes versus [sic] chemotherapy.
No! A thousand times no! Compare Dr. Levine’s statement with my own:
The authors of the report should be scrutinized by the OHRP and the NIH. They should be banned from being investigators in any NIH-sponsored trial for some finite period, and their involvement in the Gonzalez trial should become a noticeable smear on their reputations.
Here is Dr. Levine’s final paragraph. Rather than comment further, I’ve provided pertinent hyperlinks:
Recently, there has been a shift from single CAM modalities for cancer management to a more comprehensive approach called integrative oncology, which is “an evolving, evidence-based specialty that uses CAM therapies in concert with biomedical cancer treatments to enhance its efficacy, improve symptom control, alleviate patient distress and reduce suffering.” I am encouraged that the leaders of the Society of Integrative Oncology (Dundas, Ontario, Canada) have developed evidentiary levels to gauge the strength of evidence for CAM therapies. It is not surprising that these levels are based on the foundational principles of evidence-based medicine established by Sackett et al. I look forward to future well-designed clinical trials that provide high-quality evidence on how CAM therapies can improve the quality of life of our patients.
*The “Gonzalez Regimen” Series:
† The Prior Probability, Bayesian vs. Frequentist Inference, and EBM Series:
16. What is Science?