A great deal of electronic ink has been spread on these pages discussing the question of what it means to be science-based. While we have developed and iterated an operational definition, like many complex phenomena there is no sharp demarcation line. Practices occur along a spectrum from rigidly science-based to blatant quackery.

There are plenty of practices, however, that are in the middle. Further, an individual practice can range across the spectrum depending on the claims that are being made for it. “Nutrition” as an approach to health can be rigidly scientific (folate for pregnant women to reduce the incidence of neural tube defects) or pure snake oil (“superfoods” to cure what ails you).

One practice that I think straddles this middle-zone is mindfulness meditation (or just mindfulness). The “dictionary” definition of mindfulness is, “a mental state achieved by focusing one’s awareness on the present moment, while calmly acknowledging and accepting one’s feelings, thoughts, and bodily sensations, used as a therapeutic technique.” But before we get into more detail about this practice, let’s review what we mean by “science-based.”

What is science-based practice?

This, of course, is the core question of Science-Based Medicine and a topic of deep thought by its proponents. I will try to give the quickest definition I can here. SBM acknowledges that medical interventions which are safe and effective are preferred over practices which are unsafe or ineffective. Further, the best way to evaluate practices is by considering all of the available scientific evidence in the most thorough context.

That last bit is where the complexity is – each piece of scientific evidence needs to be evaluated for its rigor, context, implications, and relation to other relevant pieces of scientific evidence. Further, scientific research needs to be evaluated in total as a process. It is the arc of a research question that tells us what the best current conclusion is, not any one piece of evidence.

SBM also endeavors to understand patterns in the research to detect biases, flaws, and errors. We also use the history of science (and especially medical science) to understand how well the research predicts whether an intervention will ultimately be useful or not.

In short, SBM takes a detailed meta-scientific view of all the relevant research to make the best prediction possible about whether or not any individual practice is safe and effective for any specific indication.

Another way to look at the SBM approach is this – what does it take before a medical claim is considered compelling? What are the thresholds of scientific validity in various contexts before we would recommend an intervention? This question is important because medicine is an applied science. We have to make decisions about what to do with actual people.

These are all complex questions requiring much longer discussion (and you can find plenty of discussion in other articles on this site), but again to quickly review: Scientific evidence is only compelling when there are specific terms used that can be operationally defined (so we know what it is we are studying and discussing). The claims made should also be plausible, meaning that at the very least they do not defy basic scientific knowledge. We also need clinical evidence to show that specific variables which are adequately controlled for in a blinded fashion have an effect which is consistent, replicable, and both statistically and clinically significant.

The history of medicine and systematic reviews of the evidence show that less than something like this threshold and an intervention probably doesn’t work or is not a net benefit for patients. Most new ideas in medicine turn out to be wrong, and there is a massive positive bias in evaluating treatments (at every stage from early observation to citing published research). Only when we get to a relatively high threshold of evidence as outlined above can we be reasonably confident that an intervention actually works.

Mindfulness

With that review – let’s apply these basic SBM principles to the question of mindfulness. A recent review of the topic, “Mind the Hype: A Critical Evaluation and Prescriptive Agenda for Research on Mindfulness and Meditation,” helps this task considerably.

As the authors point out, there are significant problems with the mindfulness research and the overhyped claims that are being made for this intervention. Problems begin with the very first criterion of science-based practice – lack of an operational definition. This may seem nitpicky, but it is absolutely critical to be being science-based.

Good science is all about controlling for specific variables so that we can make firm conclusions about the effects of those isolated variables. If we cannot define the variables, let alone isolate them, then we cannot study them scientifically.

Mindfulness research, as the authors point out, is plagued by a lack of a consistent definition, or one that can be operationalized for research. What is/are the critical component(s) of mindfulness? Is it relaxation, introspection, being in the moment, directing one’s attention, avoiding mind-wandering, or something else? Granted, this is a frequent generic problem with psychological science because human psychology is complex. We often don’t understand psychological phenomena in a fundamental-enough way to operationalize it for research. But still, we need to acknowledge the limitations of any individual research question.

To show by analogy how critical this feature is, let’s turn to acupuncture research. If we define acupuncture as placing thin needles through the skin at acupuncture points, we can confidently conclude based upon the research that acupuncture does not work. It does not matter where or even if you place needles through the skin. However, proponents continue to claim that acupuncture does work, citing research results that use a deliberate loose definition of “acupuncture.” Acupuncture, when necessary, can mean just poking the skin in random locations, or using electrical stimulation.

Proponents have essentially said – acupuncture works, it just doesn’t seem to matter where or if you stick the needles. Then what is “acupuncture?” And if specific variables don’t seem to matter, how can you control for non-specific effects? Proponents respond by essentially saying it doesn’t matter. The non-specific or placebo effects are real (which misunderstands the nature of placebo) and that is all that matters.

Having followed the mindfulness research for years and discussed it with many proponents, the response is largely the same. The details of specific variables are either poorly understood or debated, and don’t seem to really matter. As long as there is a perceived subjective benefit, that is sufficient justification.

The authors of the current review agree. They write:

However, the aforementioned complexity, confounding, and confusion that surrounds empirical research on “mindfulness” limits the potential of the method to inform broad questions and inform specific theories. The extent to which a specific model is supported or disconfirmed by particular sets of empirical data or systematic observations depends on the meaning of “mindfulness” that inspired data acquisition.

Mindfulness, in various studies, can refer to alleged specific mental phenomena such as: “psychological distancing/reperceiving, decentering and inhibitory control, nonconceptual discriminatory awareness, acceptance and reintegration, or focused attention, decentering, and meta-awareness.” That is a pretty big umbrella.

Proponents have also argued that mindfulness is partly relaxation, partly cognitive therapy, and partly developing vaguely defined mental skills related to the above.

This creates a further problem – how do you control for nonspecific results. This gets to the second major issue that the reviewers and others have, most studies of mindfulness are poorly controlled or methodologically weak. Some lack a control group at all. The best studies have active controls, attempting, for example, to control for simple relaxation. But these best studies tend to have the most modest results (a meta-pattern important to note).

The reviewers point out, for example:

In a recent review and meta-analysis commissioned by the U.S. Agency for Healthcare Research and Quality, MBIs (compared to active controls) were found to have a mixture of only moderate, low, or no efficacy, depending on the disorder being treated.

MBI=Mindfulness based intervention. Results are mixed and unimpressive. That is a pattern with which we are very familiar, and does not exactly inspire confidence.

When I have expressed skepticism about mindfulness in the past, the response (even from skeptics) is often – well, it worked for me. The obvious response is – what worked, and how did it work, and how do you know it worked?

In order to answer these questions we need the kind of clinical research that can answer them – specifically defined variables controlled for non-specific effects. We need to know that mindfulness is a specific thing that has an effect beyond the non-specific effects of relaxation, or taking a mental break from the stress of the day, or just exercising your executive function.

So what’s the bottom line? It’s hard to say, and that is part of the point of the current review. Existing research has not yet clearly defined what mindfulness is and what effect it has. They make recommendations for future research to clarify these questions.

I think the best current summary is to consider mindfulness like yoga, or a specific form of exercise. There is evidence that doing yoga has specific health benefits. However, those benefits are likely not specific to yoga and are universal to exercise. It is therefore more accurate to say that exercise has many health benefits, and yoga is a form of exercise.

Likewise I would say that relaxation has health benefits and cognitive therapy potentially has psychological benefits. Mindfulness, taken broadly, seems to be a reasonable practice for relaxation and may have some effects similar to cognitive behavioral therapy.

But I am not convinced by the existing research that there is any other phenomenon at work here, that there is something specific to mindfulness, or that it has benefits beyond other similar behaviors. There may be – perhaps there is something specific about certain mindfulness practices that we can isolate and study. We are simply not there yet.

Further, public hype supporting a billion dollar industry has gone way passed the existing evidence.

Posted by Steven Novella

Founder and currently Executive Editor of Science-Based Medicine Steven Novella, MD is an academic clinical neurologist at the Yale University School of Medicine. He is also the president and co-founder of the New England Skeptical Society, the host and producer of the popular weekly science podcast, The Skeptics’ Guide to the Universe, and the author of the NeuroLogicaBlog, a daily blog that covers news and issues in neuroscience, but also general science, scientific skepticism, philosophy of science, critical thinking, and the intersection of science with the media and society. Dr. Novella also contributes every Sunday to The Rogues Gallery, the official blog of the SGU.