NOTE: I get a lot of emails asking me whether treatment X is evidence-based or a scam. This one was different. Zachary Hoffman had done his homework and had already answered the question for himself (at least, as well as it could be answered with the existing published evidence). I asked him to write up his findings as a guest post for SBM. This is a great example of how a layman can figure things out for himself using little more than google-fu and critical thinking skills. I hope it will be an inspiration to others who may not have thought they were qualified to do what we do on SBM.
Recently a friend alerted me to something called “Whole-Body Cryotherapy” which has been making the rounds on Facebook and is being promoted by many athletes and celebrities. I had only heard of cryotherapy in the context of freezing off a wart, but I was about to find out there was so much more. She explained that subjecting your entire body to extreme cold (-200˚F!) for a few minutes a day was a virtual panacea, with weight-loss, tissue repair, and beauty treatments as the target market. My limited background in biology hadn’t quite prepared me for understanding why subjecting oneself to cold air could possibly help treat any illness.
For instance, up here in Boston, I ride my bike all winter long, and on a particularly cold day, after a 5 degree ride, no one has commented that I seem particularly trim, or that my face is looking unusually beautiful. Unfortunately, a few days ago while riding my bike, I took a spill and mushed my hand pretty good. However, the cold winter air hasn’t done much to alleviate that pain or stop my right hand from being twice the size of the left. In any case, it seemed to me that I’d have to give this a closer look before I made any comments.
A quick search on Google led me to a website, Cryohealthcare, Inc. The website is aesthetically pleasing and has plenty of information about how this treatment can transform your life. To top it off, there are lots of endorsements from professional teams and athletes. It appears that for about $65 a pop you can subject yourself to unfathomably low temperatures and enjoy a whole-body tingle when you step out (when I was younger I used to jump in the snow and then get into a hot tub, so I get the appeal). A quick scroll down and we see indications for injury recovery, pain mitigation, and athletic performance, among others, followed nicely by the FDA quack Miranda warning.
They helpfully provide us with a full page of research, essential for understanding how this is really a scientific treatment and not just some recent fad (really!). On the research page we are confronted with an impressive page of peer reviewed articles. I pull up the second article in the list, “Ice Freezes Pain? A Review of the Clinical Effectiveness of Analgesic Cold Therapy.” This paper is from 1994, and is a literature review attempting to understand how appropriate cryotherapy is for the reduction of pain related to the musculoskeletal system. The conclusion states:
Cooling (usually of small areas) of the body surface seems an attractive approach to treating musculoskeletal pain and it is often used for this purpose. Unfortunately, there is little scientific evidence to show that it is effective. All clinical studies that have evaluated this modality are severely flawed. Thus, further trials should be initiated to provide the scientific proof of its effectiveness and to test which type of pain responds best.
I could feel a silly smirk forming uncontrollably across my face. Here is the second paper listed and it is concluding the exact opposite of what was being claimed on the previous page. But to be fair, there are nearly 50 others on this page. A quick browse and I found that there are a few positive studies, but they nearly all have severe methodological flaws and small sample sizes. Not to mention that they are dealing with extremely complex subjects such as depression, where citing one paper for clinical applications is totally inappropriate. Further browsing led me to an almost hilarious number of negative conclusions such as:
The analgetic effects demonstrated so far and the high percentage of adverse events observed with this challenging method currently do not support its routine use in clinical practice.
…all changes due to cold were relatively mild.
Based on our results, whole-body cryotherapy at -110 degrees C is not superior to local cryotherapy commonly used in RA patients for pain relief and as an adjunct to physiotherapy.
This last one takes the cake, because it actually considers the exact treatment that the website is selling, rather than attempting to extrapolate from related studies. The most bizarre part is that I found all three of these statements in the abstracts presented directly on their webpage; I didn’t even have to make the effort to sift through the linked PDFs to find them. Honestly, and not surprisingly, it would appear that they were attempting to make this page as long as possible rather than provide any helpful information. It seemed that this was nothing more than another pseudo-scientific marketing ploy to get people to pay money to freeze their butts off. Fortunately, they were nice enough to provide me with all of the evidence against their method right there on their website.
Getting back to that swollen purple hand of mine; as I giggled giddily at the “research” presented on this webpage, I adjusted the ice pack that was helping to control the swelling of my hand… and then it hit me, I’m using cryotherapy to treat my hand. My motivated reasoning kicked into full gear in an attempt to legitimize what I was actually practicing, versus the conclusions that I had reached from reading the research articles. Cognitive dissonance at its finest.
I could come up with justifications: “Lots of sports trainers from professional teams would pay tons of money to ensure their athletes heal quickly… The same athletes that use cryotherapy?” and “It’s different because the ice pack is… You know, different!” But after much struggle to resolve the dissonance, I started to think about what evidence I had that ice packs actually work for reducing inflammation or expediting the healing process. Where did I actually learn this? It seemed it was just something people always told me and I assumed was true. You know, like the way your mother puts rye bread on your elbows to prevent bruising? Of course that only sounds ridiculous because you didn’t grow up with it, but 8 year old me wouldn’t have it any other way. And besides, I can easily rationalize away the differences with the fact that rye bread is inert and ice packs use physics, or something like that. So, now that I’m an adult, one is ridiculous and the other isn’t… Right?
Everywhere I went people had a suggestion about how to help my hand, almost always consisting of some sort of ice treatment. So I found myself with an issue: I’ve got two very related therapies, but I’ve also got two very opposite conclusions about them. To complicate my dilemma, while I do have a science background, my experience with biology and medicine is very much limited. It feels very uncomfortable to question conventional wisdom like this, but I think it’s probably worse to have to live with a constant contradiction.
I started by attempting to find out whether any reputable groups had made a formal statement about the subject. In general, groups such as the American Medical Association, American Medical Society for Sports Medicine, and others are at least a good starting point for trying to form a well-informed opinion and finding an aggregate of reasonable papers on the subject. Unfortunately, there didn’t appear to be much information from any of these groups. General web searches for cryotherapy and inflammation yielded plenty of explanations about how ice numbs the skin and stops blood flow, but the sources were dubious. Further, I questioned how superficial cooling at the skin could have an impact on blood flow in subsurface tissue. I decided to try my luck over at PubMed to see what I could find there.
When I am doing research in my own field of optics, it can be a daunting task to wade through the amount of information that turns up on PubMed. The problem of separating the signal from the noise in a subject that I am wholly unfamiliar with seemed nearly insurmountable. In order to get some semblance of structure, I did my best to narrow down the question that I wanted to ask. In particular, subjective outcomes such as pain can be extremely difficult to assess. The fact that papers on elaborate placebos continue to get published and gain legitimacy seems to verify this and I honestly didn’t expect to be able to sort out any of these subjective outcomes. I tried to focus on objective outcomes such as swelling size and healing time.
Applying the search terms “cryotherapy,” “healing,” “swelling,” “ice pack,” and “inflammation” yielded dozens of papers. The first thing that I found was that there was very little controlled research done on cryotherapy, certainly less than I would have thought considering how much money there is in professional sports. Rather than comparing ice to no ice, a number of the papers that I found consisted of comparing differences in hot and cold temperatures, or applying cold for various time intervals, all with the assumption that icing is beneficial.
Eventually I was able to sort out a number of meta-studies that had exactly what I was looking for. Surprisingly, the conclusions I found were not what I had expected: many of them were totally neutral or even negative. I was able to find one recent meta-study (2012), where the authors concluded[1]:
Despite its widespread clinical use, the precise physiologic responses to ice application have not been fully elucidated. Moreover, the rationales for its use at different stages of recovery are quite distinct. Using cryotherapy to manage acute soft tissue injury is based largely on anecdotal evidence.
Another meta-study from 2004 concludes [2]:
There was marginal evidence that ice plus exercise is most effective, after ankle sprain and postsurgery. There was little evidence to suggest that the addition of ice to compression had any significant effect, but this was restricted to treatment of hospital inpatients.
A study from 2015 concludes [3]:
The physiological effects of cold therapy include reductions in pain, blood flow, edema, inflammation, muscle spasm, and metabolic demand. There is limited evidence from randomized clinical trials (RCTs) supporting the use of cold therapy following acute musculoskeletal injury and delayed-onset muscle soreness (DOMS).
And another study from 2015 finds [4]:
Cryotherapy or icing, as currently practiced, will not likely be successful in cooling muscle sufficiently to have any significant influence on muscle repair regardless of the degree of injury. However, based on studies in animal models, it may be that if sufficient muscle cooling could be achieved in humans, it could actually delay recovery and increase muscle scarring following significant muscle damage.
Furthermore, I found that most studies expressed their dissatisfaction with the overall availability and quality of studies and found that ice therapy isn’t necessarily a clear-cut solution in dealing with muscle injury. I was very confused about how it is possible that such a universally accepted practice could really have such a dearth of evidence behind it.
So there I sat, slowly moving the ice pack further and further from my hand as I considered whether the whole icing after an injury or icing sore muscles theory was wrong. Could this entire paradigm be incorrect and did I just debunk this myth? I think the answer is no, probably not. While many of these papers draw conclusions that are negative, it is important to remember that the authors of these papers almost uniformly complained about the insufficient number of high-quality randomized controlled trials, which precludes reaching a very firm conclusion one way or the other. These are the same deficiencies that have left us scratching our heads when it is claimed that whole-body cryotherapy is capable of curing your depression, based on a single point of “evidence.” It would be totally dishonest to make the opposite claim that it “doesn’t” cure depression based on the same deficient evidence.
Does whole-body cryotherapy work? Does icing a sore muscle help? Maybe, maybe not. There probably isn’t enough high quality evidence to support either in clinical practice and it is dishonest to state otherwise.
While on one hand, I would love to have been able to remove all doubt, it’s all too easy to forget that like most scientific subjects, medicine is extremely complex. In fact, for most subjects that we don’t fully dedicate our lives to, we know so little that we don’t even understand what we’re ignorant of. This often leads us to be overconfident in whatever caricature we’ve created in an attempt to oversimplify a complex subject (i.e. using four sketchy studies to make a conclusion about ice therapy or a few studies to justify whole-body cryotherapy). In our desire to have an answer we often just accept one side of the conversation or the other, without admitting that we just don’t know. By not conceding our ignorance, we allow “conventional wisdom” (whether right or wrong) to triumph over scientific reality.
Just because we’ve put in all this work with no solid conclusion, I don’t want this to sound like a deterrent from learning about and researching a complex or controversial topic. Here I am, a non-expert empowered with the ability to access and consider the latest research on any given subject. And while I have no intention of telling everyone who ices their hand that “they’re doing it wrong,” it is nice to get a handle on how tenuous the evidence is behind this practice. As a member of the public I have an obligation to understand the basic science and know when to hand over the questions to the experts.
Finally, for fear of this being misused, I feel an obligation to point out that this isn’t a call to question everything and demand more research unnecessarily. There are topics, such as vaccines, where the overwhelming consensus is rooted both in a mountain of scientific plausibility and evidence. These conclusions are rooted in decades of high quality studies, from a vast pool of experts in the field teasing out nuances of the subject that I can’t even begin to understand. Further, there are topics that have been put to rest long ago. This also isn’t an attempt to demand for “more high quality studies” on practices such as homeopathy, which are built on a framework of magical thinking and egregiously defy the laws of chemistry and biology. Most importantly, when stepping outside of our own fields, it is critical to understand our limitations and biases, ensuring that our actions follow the scientific conclusions and not just the evidence that we’ve found to support a particular position.
References
- van den Bekerom, Michel PJ, et al. “What is the evidence for rest, ice, compression, and elevation therapy in the treatment of ankle sprains in adults?.” Journal of athletic training 47.4 (2012): 435-443.
- Bleakley, Chris, Suzanne McDonough, and Domhnall MacAuley. “The use of ice in the treatment of acute soft-tissue injury a systematic review of randomized controlled trials.” The American journal of sports medicine 32.1 (2004): 251-261.
- Malanga, Gerard A., Ning Yan, and Jill Stark. “Mechanisms and efficacy of heat and cold therapies for musculoskeletal injury.” Postgraduate medicine 0 (2014): 1-9.
- Tiidus, Peter M. “Alternative treatments for muscle injury: massage, cryotherapy, and hyperbaric oxygen.” Current reviews in musculoskeletal medicine (2015): 1-6.
About Zachary Hoffman
Zach with a fish
Zachary Hoffman is a PhD student at Northeastern University in Boston, Massachusetts. Focusing in optics, his research attempts to develop and improve methods of structured illumination microscopy for resolving subsurface information in-vivo. He is also working as an engineer at Draper Laboratory, developing light based inertial sensors. He is currently living in Boston, and his hand is feeling much better.