A recent meta-analysis of eye movement desensitization and reprocessing (EMDR) therapy concludes that the evidence “confirms” EMDR is effective in treating depression. It is a great example of the limitations of meta-analysis, and how easy it is to create essentially a false narrative using poor quality research.
EMDR was “developed” by Dr. Francine Shapiro in 1987. It is the notion that bilateral eye movements stimulate the frontal lobes which alters the processing of memory information. This allows subjects to better process traumatic memories to lessen their negative effects. There are two ways to look at the claims of EMDR, the proposed neurological mechanism and the clinical evidence for efficacy. Both are severely lacking.
The neurological explanation for EMDRs putative effects has always been very hand-wavy – a just-so story without hard evidence in neuroscience. Admittedly it is difficult to reverse engineer the neuroanatomical correlates of complex cognitive entities, and there are many clinically accepted conditions we still do not fully understand.
The difference is that EMDR is a treatment based upon and proposed underlying neurological mechanism, and yet there was never any real basis for proposing such a mechanism. It was always essentially wild speculation. Later research aiming to backfill the evidence for an EMDR mechanism amounts to what I call “stuff happens” evidence. Whenever subjects do anything cognitive stuff is happening in the brain, but that does not mean that whatever brain activity is detected is having the specific effects that are claimed for it, in this case changing how subjects process traumatic or painful memories.
The big problem with the alleged mechanism is that it is very gimicky, like a “brain hack” that alters brain function with “one simple trick”. That’s just not how the brain works. Research on mechanism and clinical efficacy also suffer from the same problem – the research lacks sufficient controls.
This gets to the fundamental question we tackle at SBM – how do we know anything in medicine? We are dealing with complex, messy, and variable systems (humans) with lots of subjective outcomes. In order to truly advance our knowledge of how the human machine works, the mechanism of disease and disorder, and the efficacy of treatments, we need to define, isolate, and control variables as much as possible. Otherwise it is very easy to create the illusion of a real effect that is actually nothing more than a cultural story.
We see this with acupuncture, for example. If we operationally define acupuncture according to acupuncture points and manipulation with needle insertion, it becomes exceedingly clear that acupuncture is not a real phenomenon. The points don’t exist in any genuine scientific way, and it does not seem to matter if or how these fictitious points are manipulated. The entire clinical effect of acupuncture can be explained as a subjective placebo response to the ritual surrounding the administration of acupuncture. It is all smoke and mirrors.
EMDR suffers from the exact same problem. Let’s look at this recent meta-analysis to demonstrate. If you go through the individual studies you will see that none of them are compelling evidence for the efficacy of EMDR, and therefore adding a lot of weak studies together does not create a strong study. This is the perfect example of garbage in-garbage out.
For example, studies like this one have no control intervention. The control group was “watchful waiting” – in other words, nothing. This study used a medication treatment control group, so of course of completely unblinded. You could also argue the duration of medication treatment was too short, and a comparison is meaningless without any therapy intervention in the control. In this study the control group was relaxation therapy. At least this is a somewhat reasonable control, although not completely and not blinded. Also in this study both EMDR and relaxation were effective.
What we don’t see in the EMDR research to any significant degree are studies that actually isolate the variable that is the very definition of EMDR itself. What we are given is excuses – how difficult it is to blind such interventions. Helpful trial are entirely doable, however. You could, for example, have one group in which the practitioner goes through the entire process of EMDR (which includes a lot of actual cognitive therapy) but without the eye movements. You could also substitute some other similar physical intervention, such as having the subjects tap their hands, or perhaps do a task on one side of the body only.
If EMDR researchers were doing what scientists should be doing – trying to prove their hypothesis wrong with studies able to do so, by isolating specific components of EMDR – then we could start to build a case for or against EMDR. We could see if eye movements themselves matter, if bilateral movements matter, if movements matter. This, of course, feeds back to the putative mechanism.
Instead what we have are weak clinical trials that are not properly controlled or blinded and do not isolate specific variables. Just as will acupuncture, this opens the door for non-specific therapeutic effects – the ritual surrounding EMDR. With therapy there are also many real therapeutic effects that come from the bond between therapist and client. In fact, that seems to be the variable that matters most.
It’s tempting to say – why does any of this matter as long as people feel better? But it does matter. Without the ability to determine which specific elements of mental illness intervention (or any intervention) has specific efficacy, then we cannot make real progress. We are stuck chasing nonspecific placebo effects.
Further, not only would a lack of specific knowledge about efficacy hamper our ability to understand how the brain works and what is happening to cause mental disorder, but it can lead to the creation of false narratives or the illusion of knowledge (which is far worse than mere ignorance). That is what we have with things like acupuncture and EMDR – a feedback loop of non-specific or placebo effects being falsely interpreted as specific effects feeding into speculations about underlying mechanisms that are unmoored from reality. We get a cultural narrative pretending to be science.
Then, of course, this false cultural narrative becomes institutionalized. Once we have an “Institute of Fake BS” the fake BS is never going away. Rather we end up with research articles and even entire journals which are dedicated to “cargo cult”, “tooth fairy” (pick your metaphor) fake science. They do studies which are meant to show that their preferred treatment works, and never to examine whether or not it works.
EMDR is therefore not likely going away anytime soon, even though it is a house of card built on nothing.
