Arguably the most difficult aspect of science-based medicine is where the rubber hits the road, so to speak. That’s where scientists and physicians take the results of preclinical studies performed in vitro in biochemical assays and cell culture models and in vivo in animal models to humans. There are numerous reasons for this, not the least of which that preclinical models, contrary to what animal rights activists would like you to believe, do not predict human responses to new therapeutic agents as much as we would like. However, the single biggest reason that we cannot answer questions in human studies as easily as we can in cell culture and animal studies is ethics. Of course, answering questions using cell culture and animal studies is not “easy,” either, but performing studies using human beings as subjects is an order of magnitude (at least) more difficult because the potential to cause harm exists, and if harm is caused by the experimental treatment under study, that harm will be done to human beings, rather than cells in a dish or mice bred for research.
The “gold standard” type of study that we do to test the efficacy of a new drug is known as the randomized, placebo-controlled, double-blinded study, often abbreviated RCT. Indeed, this remains the gold standard and is accorded the highest level of “power” in the framework of evidence-based medicine. Of course, as we have argued time and time again, using the RCT to test therapies that are incredibly implausible on a strictly scientific basis (homeopathy or reiki, for instance) inevitably leads to numerous “false positives” in which the therapy appears to produce results statistically significantly better than the control. John Ioannidis has done numerous clever analyses that demonstrate how easily clinical research is led astray if it is not grounded in scientific plausibility. Indeed, the probability of false positive studies increases, the more improbable the modality. It is for these very reasons that we have proposed the concept of science-based medicine, which takes into account estimates of prior probability based on preclinical studies and basic scientific principles, rather than evidence-based medicine, which does not. Indeed, Wally Sampson has even proposed a “plausibility scale” for rating RCTs, and Steve Novella has pointed out how difficult it can be to interpret the medical literature.
Leaving aside all of these issues, to which a whole series of posts could be devoted (and, indeed, to which a whole series of posts has been devoted by Kimball Atwood, Steve Novella, and Wally Sampson), I want to look at the ethical precepts that guide clinical trials and why they render it much more difficult to answer some questions. To the shame of the medical profession, these ethical precepts are a relatively recent phenomenon dating back only a few decades at most. Some of the impetus for these rules derived, not surprisingly, from the horrific medical experiments that the Nazis and Japanese performed on prisoners during World War II. Virtually everyone’s heard, for instance, of Dr. Mengele and his experiments on twins, but there was more–so much more–that the Nazis did, including testing the effects of immersion in ice water to simulate what sailors forced to abandon ship or pilots shot down over the ocean might experience and to test ways to rewarm them and save their lives; placing prisoners in vacuum chambers to test the effects of high altitude (which had obvious applications to how their pilots could deal with high altitude); sewing glass, foreign objects, or gauze impregnated with bacteria into wounds in order to test means of preventing and treating gangrene; smashing prisoner’s bones with a hammer in order to simulate more realistically war wounds; and the use of radiation aimed at the ovaries in order to sterilize quickly Jewish women and other “undesirables,” at the cost of horrific radiation injury to the surrounding bowels because the technology to aim radiation beams was very crude then. The Japanese were no slouches at depraved medical experiments, either, performing “dissections” on prisoners while they were still alive; testing phosgene gas on prisoners; decapitating prisoners to test the sharpness of blades; amputating limbs to study shock due to hemorrhage; performing gastrectomies and testing reattachments of the esophagus to various parts of the intestinal tract; and testing the effect of various pathogenic microorganisms on prisoners to see what inoculum was necessary to cause disease.
Then there was the Tuskegee syphilis experiment, formally known as The Tuskegee Study of Untreated Syphilis in the Negro Male. Carried out between 1932 and 1972 by the U.S. Public Health Service, the study involved 399 African-American men with syphilis who were studied to determine the effects of untreated syphilis in black men as opposed to whites, the hypothesis being that whites experience more neurological complications due to syphilis, whereas blacks were thought to be more susceptible to cardiovascular damage. What horrified the nation in 1972, when details of the experiment were first reported Jean Heller of the Associated Press, was that the study continued after the 1940s, when penicillin had been validated as curative for syphilis. The experiment also involved considerable deception:
By the end of the experiment, 28 of the men had died directly of syphilis, 100 were dead of related complications, 40 of their wives had been infected, and 19 of their children had been born with congenital syphilis. How had these men been induced to endure a fatal disease in the name of science?
To persuade the community to support the experiment, one of the original doctors admitted it “was necessary to carry on this study under the guise of a demonstration and provide treatment.” At first, the men were prescribed the syphilis remedies of the day —bismuth, neoarsphenamine, and mercury— but in such small amounts that only 3 percent showed any improvement.
These token doses of medicine were good public relations and did not interfere with the true aims of the study. Eventually, all syphilis treatment was replaced with “pink medicine” —aspirin.
To ensure that the men would show up for a painful and potentially dangerous spinal tap, the PHS doctors misled them with a letter full of promotional hype: “Last Chance for Special Free Treatment.” The fact that autopsies would eventually be required was also concealed.
As a doctor explained, “If the colored population becomes aware that accepting free hospital care means a post-mortem, every darky will leave Macon County…” Even the Surgeon General of the United States participated in enticing the men to remain in the experiment, sending them certificates of appreciation after 25 years in the study
It wasn’t until 25 years after the revelation of the ethics violations that had occurred during the Tuskegee syphilis experiment were revealed that the U.S. government acknowledged how wrong the experiment had been and then President Clinton formally apologized.
In response to these sorts of abuses, efforts were undertaken, both in the U.S. and internationally, to codify ethical principles that would prevent such atrocities and abuses. In the aftermath of World War II, the first major effort to regulate medical research was the Nuremberg Code, which was developed from the verdict of the Doctors’ Trial in 1947. There were ten points
- The voluntary consent of the human subject is absolutely essential. This means that the person involved should have legal capacity to give consent; should be so situated as to be able to exercise free power of choice, without the intervention of any element of force, fraud, deceit, duress, over-reaching, or other ulterior form of constraint or coercion; and should have sufficient knowledge and comprehension of the elements of the subject matter involved as to enable him to make an understanding and enlightened decision. This latter element requires that before the acceptance of an affirmative decision by the experimental subject there should be made known to him the nature, duration, and purpose of the experiment; the method and means by which it is to be conducted; all inconveniences and hazards reasonable to be expected; and the effects upon his health or person which may possibly come from his participation in the experiment. The duty and responsibility for ascertaining the quality of the consent rests upon each individual who initiates, directs or engages in the experiment. It is a personal duty and responsibility which may not be delegated to another with impunity.
- The experiment should be such as to yield fruitful results for the good of society, unprocurable by other methods or means of study, and not random and unnecessary in nature.
- The experiment should be so designed and based on the results of animal experimentation and a knowledge of the natural history of the disease or other problem under study that the anticipated results will justify the performance of the experiment.
- The experiment should be so conducted as to avoid all unnecessary physical and mental suffering and injury.
- No experiment should be conducted where there is a prior reason to believe that death or disabling injury will occur; except, perhaps, in those experiments where the experimental physicians also serve as subjects.
- The degree of risk to be taken should never exceed that determined by the humanitarian importance of the problem to be solved by the experiment.
- Proper preparations should be made and adequate facilities provided to protect the experimental subject against even remote possibilities of injury, disability, or death.
- The experiment should be conducted only by scientifically qualified persons. The highest degree of skill and care should be required through all stages of the experiment of those who conduct or engage in the experiment.
- During the course of the experiment the human subject should be at liberty to bring the experiment to an end if he has reached the physical or mental state where continuation of the experiment seems to him to be impossible.
- During the course of the experiment the scientist in charge must be prepared to terminate the experiment at any stage, if he has probable cause to believe, in the exercise of the good faith, superior skill and careful judgment required of him that a continuation of the experiment is likely to result in injury, disability, or death to the experimental subject.
(Taken from the Wikipedia entry reprinted from Trials of War Criminals before the Nuremberg Military Tribunals under Control Council Law No. 10, Vol. 2, pp. 181-182. Washington, D.C.: U.S. Government Printing Office, 1949.)
Next was the Declaration of Helsinki developed by the World Medical Association in 1964 and The Belmont Report in the United States. It in essence expanded upon the principles of the Nuremberg Code (more on how later). In the aftermath of the Tuskegee experiment and in response to other ethical violations in human subjects research, the former United States Department of Health, Education, and Welfare (renamed to Health and Human Services) produced a document entitled “Ethical Principles and Guidelines for the Protection of Human Subjects of Research,” more commonly called The Belmont Report, for the Belmont Convention Center, where the National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research (1974-1978) met to draft the report.
The Belmont Report identified three essential aspects of any human subjects research:
- Respect for persons. This principle demands that persons be treated as autonomous agents with self-determination and that persons unable to exercise autonomous self-determination (children, the mentally ill) or who could be easily coerced (prisoners) are entitled to protections for their interests. In brief, this means that research subjects must be accorded full understanding of the risks and potential benefits of participating in a research study, as well as full autonomy (i.e., the right to refuse to participate without fear of repercussions or that their medical care may be compromised).
- Beneficence. This principle can be boiled down to (1) do not harm and (2) maximize possible benefits and minimize possible harms. Thus, the scientific design of the experiment must not be excessively risky for participants and must have sound science behind it suggesting that a therapy is likely to be of benefit.
- Justice. Research must be fair, including (and potentially benefiting) as many groups as possible. Key quote: “For example, the selection of research subjects needs to be scrutinized in order to determine whether some classes (e.g., welfare patients, particular racial and ethnic minorities, or persons confined to institutions) are being systematically selected simply because of their easy availability, their compromised position, or their manipulability, rather than for reasons directly related to the problem being studied. Finally, whenever research supported by public funds leads to the development of therapeutic devices and procedures, justice demands both that these not provide advantages only to those who can afford them and that such research should not unduly involve persons from groups unlikely to be among the beneficiaries of subsequent applications of the research.”
Based on the Belmont Report, the U.S. Government eventually formulated the Common Rule (full DHHS regulations here), which mandates that all federally-funded human research and all human research designed to obtain FDA approval must be overseen by committees known as Institutional Review Boards, independent bodies that guarantee that human subjects are protected according to the Belmont Report and all applicable federal regulations.
Finally, the Helsinki Declaration represents an ongoing international effort to come up with a set of ethical principles to guide human subjects research. It is not a legally binding treaty but nonetheless now serves as the cornerstone of document in human research ethics. Despite its not being codified as a treaty, the Helsinki Declaration draws its power and authority based on how much of it has been codified into legislation and regulations in a large number of countries. The first version, released in 1964, promulgated ten principles based upon the original ten principles of the Nuremberg Code. Since then the Helsinki Declaration has undergone six revisions, the most recent of which occurred in October 2008. The first revision, in 1975, introduced the concept of an independent oversight board for all human subjects research, which influenced the requirement for an IRB in the Common Rule in the United States. In its current form, the Helsinki Declaration has introduced additional protections that are relevant to this blog:
12. Medical research involving human subjects must conform to generally accepted scientific principles, be based on a thorough knowledge of the scientific literature, other relevant sources of information, and adequate laboratory and, as appropriate, animal experimentation. The welfare of animals used for research must be respected.
We have discussed this time and time again on SBM: the concept of prior probability. It is the very reason that so much research on complementary and alternative medicine (CAM) is, at the very least, of dubious ethics, especially for highly improbable modalities like homeopathy and reiki. The reason is that testing these modalities on human subjects does not conform to generally accepted scientific principles and is not based on proper preclinical experimentation. Indeed, a discussion of this very issue has broken out in the comments of a post here at SBM about a trial of massage therapy for HIV/AIDS, and I expressed grave concern over the ethics of a trial of homeopathy for infectious childhood diarrheal diseases in a Third World country.
More importantly, here’s one reason why placebo-controlled research is becoming more and more uncommmon:
32. The benefits, risks, burdens and effectiveness of a new intervention must be tested against those of the best current proven intervention, except in the following circumstances:
- The use of placebo, or no treatment, is acceptable in studies where no current proven intervention exists; or
- Where for compelling and scientifically sound methodological reasons the use of placebo is necessary to determine the efficacy or safety of an intervention and the patients who receive placebo or no treatment will not be subject to any risk of serious or irreversible harm. Extreme care must be taken to avoid abuse of this option.
This is the very reason why most clinical trials now examine the addition of a new drug to the standard of care or a new drug versus the standard of care, rather than true placebo-controlled trials. Exceptions occur for self-limited conditions, such as hot flashes during menopause or headach therapies, but not for more serious conditions and certainly not for life-threatening conditions like cancer. Indeed, very few cancer therapy trials have a placebo control.
Considering the totality of ethical considerations that must be applied to human subjects research, it is easy to understand why we cannot always do the “definitive” experiment to answer a scientific question about the efficacy of a new therapy. For example, in surgical trials, with rare exceptions, it is not ethically acceptable to do “sham” surgery on a control group because that subjects patients to all risks and no benefit, as I have discussed before for trials of total mesorectal excision for rectal cancer and a clinical trial testing the routine use of nasogastric tubes for GI surgery.
That’s not to say that sham surgery controls haven’t been used before. Indeed, they have. For example, a randomized, sham surgery-controlled trial is how it was originally determined that poudrage for revascularizing the heart and treating angina was no better than placebo. First developed in the 1930s, poudrage was based on the concept that irritating the pericardium would provoke an inflammatory response that would lead to the influx of blood vessels into the heart muscle, and this irritation was accomplished by opening the chest and sprinkling sterile talcum powder in the pericardial sac surrounding the heart. (In the era before heart-lung bypass technology, it was not possible to consider revascularizing the heart directly.) As is the case for so many CAM modalities, results, as far as relieving the symptoms of angina, looked good–-in uncontrolled studies. Poudrage was the rage as a treatment for angina for at least a decade. Then, in the 1950s, a randomized trial was performed with a sham surgery control arm in which one group of patients had their chests opened, but no powder sprinkled in the pericardial sac around the heart. The other got standard poudrage. There was no difference in symptoms or other measures of cardiac symptomatology between the two groups. (Surgery itself is a very powerful placebo.) This was one of the very first and most famous randomized trials of a surgical intervention with sham surgery as the control. Poudrage was abandoned, and it is now relegated to being a historical curiosity in the history of surgery. It was not the only procedure for angina pectoris found when actually tested to have effects based primarily on the placebo effect of surgery.
As you may imagine, based on the latest iteration of the Helsinki Declaration, it is unlikely–to say the least–that such a study would be approved by an IRB in the U.S., although two more recent trials have been done, as this excellent discussion of the ethics of sham surgery in surgical clinical trials mentions:
There have been very few sham controlled surgical trials to date. In 1959, Cobb published a study showing no difference in improvement between patients undergoing internal mammary artery ligature versus a sham operation for treatment of angina pectoris. In recent years, two studies were done to evaluate the intracranial implantation of fetal neural cells for Parkinson’s disease. These studies had some of the study patients randomized to a sham operation that required simulating all aspects of the surgery, including the drilling of burr holes on the skull under anesthesia. In the field of orthopaedics, Moseley et al. in 2002 evaluated the effectiveness of arthroscopic surgery for arthritis of the knee. In this study, one group received a full arthroscopic debridement, one group underwent arthroscopic lavage with irrigation fluid alone, and the last group had three one-centimeter sham incisions but no actual procedure performed. This study concluded that arthroscopic surgeries done for advanced arthritis were no more effective than the sham operation.
The ethical rules laid down by a combination of the Nuremberg Code, the Belmont Report, and the Helsinki Declaration are also the reason why one of the latest talking points of antivaccinationists, a “randomized” study of unvaccinated versus vaccinated children, is ethically not permissible. The reasons are obvious. it would require that the control group of children in the placebo-controlled group be intentinoally left completely unprotected against vaccine-preventable diseases. There would also be the practical issue that antivaccine parents wouldn’t sign up for such a study because their irrational fear of vaccination would prevent them from agreeing to the possibility of randomization to the vaccination group, and parents grounded in science-based medicine would never be foolish enough to allow their children to be randomized to the no vaccine placebo group. Indeed, not too long ago an antivaccine commenter on this very blog proposed just such a study, and I told him why it would be unethical in no uncertain terms.
Meanwhile, true to form, after making a brain-meltingly specious analogy that he seems to think a slam-dunk, Generation Rescue founder and anti-vaccine zealot extraordinaire J.B. Handley makes a huge straw man out of the ethical and scientific arguments against doing a “vaxed versus unvaxed” study:
- Looking at unvaccinated kids is either impossible (due to confounders) or unethical
- Looking at adverse events based on giving children 6 vaccines, the way we do in the actual schedule, has never happened and never will, because of the “Helsinki Declaration”
No one ever said that comparing the rates of neurodevelopmental disorders like autism in unvaccinated children compared to vaccinated children is impossible or unethical. What we argue is that looking at unvaccinated children in the manner demanded by anti-vaccine zealots like J.B. Handley is unethical (the randomized, placebo-controlled “vaxed versus antivaxed” study some anti-vaccine zealots demand) or far more difficult to control for confounders than those as full of the arrogance of ignorance as J.B. Handley assume. Indeed, Prometheus has written three excellent posts [1, 2, 3 (of the three, #3 is the most detailed) describing what it would take to do a scientifically and epidemioligcally rigorous study comparing neurodevelopmental outcomes in vaccinated versus unvaccinated children. Suffice it to say, that such a rigorous study is not what the antivaccine movement wants and that doing such a study would be far more complicated, expensive, and difficult than the simplistic view of our commenter or J.B., who seems to view ethical concerns as nothing more than made-up impediments to his getting his way rather than what they are: Legitimate rules designed to protect research subjects. I also suspect that J.B. knows this but keeps demanding such a study because to the ignorant it gives him the appearance of being on the scientific high ground. An alternate explanation for the insistence by anti-vaccine zealots is that the truly believe that vaccines are so harmful that it’s more ethical not to administer them than it is to administer them. In other words, to them the unvaccinated group would be less likely to incur harm) than to administer them.
As I’ve said time and time again, where the “rubber meets the road” (so to speak) in science-based medicine is the clinical trial. Part of that ethical framework, in my view and I daresay the views of my co-bloggers, is that, before subjecting any human being to the potential risks of a clinical trial, there must be nothing but the most rigorous science supporting the hypothesis to be tested, backed up by preclinical data in the form of biochemistry, cell culture, and animal studies. In addition, because the subjects of such trials are human beings, even when there is a scientifically compelling and well-supported hypothesis being tested, the welfare of the subjects must take precedence over rigorous scientific design whenever there is a conflict between the two. Over 100 years of surgical research shows that it is possible to produce compelling evidence for the efficacy of a surgical procedure without necessarily having a sham surgery placebo control group and double-blinding of the participants (given that it’s nearly always impossible to blind the operating surgeon to the operation done), but it is much more difficult. Non-blinded prospective trials or sometimes retrospective trials need to be used, requiring many more trials to show a pattern that can overcome the confounding factors inherent in such designs.
Similarly, when it comes to issues like vaccination, although we do rigorously test each new vaccine added to the current recommended vaccination schedule in accordance with the Helsinki Declaration precepts, we cannot ethically do a randomized, double-blind, placebo-controlled trial of “vaccinated versus unvaccinated” children because that would involve leaving the control group unprotected against vaccine-preventable disease. That is a failure to provide the best standard of care as the minimum that research subjects receive. In addition, such a trial would fail from an ethical standpoint in another way. Because the hypothesis that vaccines cause autism is not supported by the weight of current evidence, such a trial would be testing a scientifically dubious (at best) hypothesis, which also goes against the Helsinki Declaration. What we are left with, then, are retrospective trials or prospective non-randomized trials requiring large numbers of subjects and careful controls for confounders. The good news is that enough of these trials can be as convincing as the gold standard. Indeed, critics of the RCT sometimes argue that sound epidemiological studies, properly done, can in some cases replace the need for RCTs.
As trite as it may seem, what this all boils down to is the observation that science-based medicine is hard. Even if you leave aside the questions of how to come to a conclusion regarding the best treatment for a patient based on science, just answering a single question requires not just loads of preclinical data in cell culture and animal models, but well-designed clinical trials. But a well-designed clinical trial is not enough; it has to be ethical. Moreover, the bioethics has been in general evolving towards ever more protection of human research subjects. Indeed, I’m reminded of an HBO movie Something The Lord Made (an excellent, albeit a bit conventional and predictable, movie about one of the “gods” of surgery Alfred Blalock and the poor black carpenter whom he hired as an assistant whose surgical skills were invaluable to Dr. Blalock in achieving what he did). The movie dramatized surgical research in the 1940s, the development of the Blalock-Taussig shunt, and the relationship between Blalock and Thomas. Watching the movie, I remember being amazed at how little it took in the way of discussion and permissions for Dr. Blalock to try out his procedure on infants. Of course, there was no procedure to save such infants back then, who inevitably died, and his assistant Vivien Thomas and he had worked out the procedure in dogs, but the difference between then and now was striking. Indeed, studies that were approved 20 or even 10 years ago might not be approved today.
Although no one, least of all I, would argue that human subjects protections are what they should be (which may be the topic for a future post), I find that comforting.