Science, Evidence and Guidelines

Disclaimer:  I am a paid Medscape  blogger and writer, and since they are in part supported by advertisements from the Pharmaceutical companies,  indirectly I am in the thrall of Big Pharma.

I found Harriet’s post on the Medscape Connect topic of How do you feel about Evidence-Based Medicine? interesting.

I wondered about the breakdown of the comments by both specialty and opinions about SBM.  So I read the 226  comments and classified them by field and response.  I classified each response as disapprove, approve or nuanced.  It is not, obviously, a legitimate survey and there was more than a little subjective interpretation in deciding how to classify the responses.  I have no doubt that others would get different results; it is not methodologically sound analysis. The discussion was in the Family Medicine & Primary Care section, so it is unlikely to be representative of any population, including that of Family Practitioners and Primary Care Physicians.  I would bet, as in alternative medicine and most topics, Shruggies predominate and are the silent majority.

Even though I belong to what  a commentator referred to as the not so silent “militant wing” of SBM, I was surprised at my results:

It was the lack of nuance that amazed me. That docs had issues with EBM/SBM should be given.  I have issues with SBM, but like democracy, it is superior to all the alternatives.  And the inability to differentiate between EBM/SBM and the guidelines upon which they are based flabbered my gaster.  I have never been able to come to grips with the concept that people who should know better, don’t appear to.  Being perfect myself, I am taken aback when others are not.

There is an intellectual hierarchy in medicine, and at the top, of course, are infectious disease doctors.   I leave it the commentators to rank the various specialities based on their response to EBM, since my referral base may be reading this and work is slow enough as is thanks to all the quality initiatives to decrease hospital acquired infections.

Besides, any list would be based entirely on confirmation bias and the personality of the specialists I work with.  I also work in an internist dominated, inpatient medicine teaching hospital, so my bias is towards evidence and science, since I have be able to quote chapter and verse when I teach the residents. What makes me expert in Infectious Diseases is partly experience, which has made me a better diagnostician every year I am in practice, and my futile attempt to master the ID literature, which makes me better at choosing a treatment.  But does the greatly extolled (at least for the EBM naysayers) experience make me better at treatment? I don’t know for sure, but I doubt it

Some examples.  P. acnes is an anaerobic bacteria that causes occasional post-operative back infections and artificial joint infections.  It is an odd bug, being such a part of the normal flora it can fester for years with little in the way of classic inflammatory signs and symptoms.  It is resistant to metronidazole, my usual go to drug for anaerobes.  The treatment is usually high dose penicillin, and I see maybe a case a year, tops.  I have a smattering of cases where penicillin did not appear to be effective, but when changed to clindamycin the patient rapidly improved. Perhaps they would have improved anyway; most do.  But as a result I have this nagging unsupported by the literature ‘feeling’ that clindamycin is better for P. acnes.

Or MRSA.  From the published data,  all the current treatment options, when compared to a beta lactam  for susceptible organisms, suggests that all antibiotics for MRSA stink on ice.  From the amount of data, i.e. preponderance of information, the rank order, and rank it is,  for treating MRSA  is perhaps vancomycin = daptomycin = linezolid => quinolone or TMP/Sulfa plus rifampin > ceftaroline (for MRSA pneumonia it is linezolid > vancomycin > ceftaroline = quinolone or TMP/Sulfa + rifampin  with no daptomycin on the list).  What I think,  in my Eldunarí, that the real order is ceftaroline > daptomycin > linezolid  = quinolone or TMP/Sulfa + rifampin with vancomycin last, with lots of caveats depending on the specific staph and the organ infected.  I have no doubt that other ID docs would rank the drugs differently. My experience, bias, and interpretation of the literature can be conflicted depending on how I approach what I am reading and the patients I have treated in the past; although I bet the literature will catch up with me someday. It always does 🙂

It is an interesting challenge when treating a given individual since the study population may not be reflected in my particular patient and then applying bias and hubris becomes very tempting.  I am very careful when I talk to house staff to differentiate my recommendations that are based on studies, based on extrapolation from known data and those that are my opinion, based in part on the three most dangerous words in medicine (in. my. experience).

It requires remarkable intellectual effort to not give in to my bias and not come up with rationalizations to justify my opinion.  Working in a teaching hospital makes that easier, since I have to make sure the residents know the ‘right’ answers so they will pass their boards.  And someday they will be in practice, having to make decisions without the support of the faculty, and have to make the ‘right’ decisions that will benefit their patients as well.  Right being in quotes as a recognition that medicine changes, and I get to bore/amuse the residents about what we did in my day, when every PVC received lidocaine, asthma was treated with aminophylline, we had no CT scans and I wore an onion on belt, as was the style of the day.

I have long noticed that many docs do not think like me (i.e. an internist) and I was influenced years ago by an article in the Annals  (sorry Harriet) called Reflections on Internal Medicine and Family Medicine  by an FP doctor. Obviously, training for breadth will have a different emphasis than training for depth.  A metropolitan subspecialist needs a vastly different skill set than a small town generalist and that  will be reflected in their training and approach to medicine

The money quotes

Internists are trained in the classic deductive reasoning model of differential diagnosis. Family physicians are trained in this model of problem solving on internal medicine services, but, in practice, they often take a less structured and more empiric approach to clinical reasoning that is based in clinical epidemiology.

and

Internists, especially those in academic medicine, may underestimate the degree of nonconformity and rebellion required of U.S. medical students who entered family medicine in the 1970s and 1980s. It is these students who now fill the faculty ranks of family practice programs around the country. Nowhere is this nonconformity more evident than in the ambivalence with which family physicians have integrated themselves into academic medicine. The importance of this counterculture mentality in the character of the family physician can best be understood by reading the works of Gayle Stephens, which are considered classics in family medicine. Stephens is one of the philosophical fathers of the family practice movement, and his book, The Intellectual Basis of Family Practice, has inspired a generation of family physicians. An understanding of the culture of family practice will help to explain much of the independence and cynicism that so often seem to characterize the family physician.

So I should not surpassed the overwhelming antipathy revealed in the Medscape comments by those in the field of FP and GP, which were particularly dismissive of evidence, often erroneously confusing evidence with guidelines.

In my own field the guidelines have remarkably little evidence to back them up, although in ID we do have the distinct advantage in treating individual patients that it is often the purpose to eradicate large swaths of microbial life; I am glad there is no Karma or I would dread the payback in the next life. Outcomes in ID are often more binary than most specialties; either I cured your infection or I did not.

Still the guidelines in ID, as noted in the ID literature, are lousy with opinion instead of evidence:

Approximately one-half  of the recommendations in the current guidelines are supported by level III evidence (derived from expert opinion). Evidence from observational studies (level II) supports 31% of recommendations , whereas evidence based on >1 randomized clinical trial (level I) constitutes 16% of the recommendations… The IDSA guideline recommendations are primarily based on low-quality evidence derived from nonrandomized studies or expert opinion.

They say that the IQ of a committee is the average IQ of the committee divided by the number on the committee and it occasionally shows in the guidelines.  Sometimes the advice is stupid, or at least brain dead.  The pneumonia guidelines suggest, with no data, “In general, when switching to oral antibiotics, either the same agent as the intravenous antibiotic or the same drug class should be used.”  That simple comment is a sure way to jack up costs and breed resistance.  Not that my opinion is necessarily better, although my IQ is divided by one, but most patients with uncomplicated pneumonia who respond promptly probably require no more than amoxicillin or doxycycline, although I can pepper that statement with several pages of caveats.  It still amazes me that for a common infection like pneumonia, we still do not really know the optimal treatment:

Most patients with CAP have been treated for 7–10 days or longer, but few well-controlled studies have evaluated the optimal duration of therapy for patients with CAP, managed in or out of the hospital. Available data on short-course treatment do not suggest any difference in outcome with appropriate therapy in either inpatients or outpatients.

Or take the  acute bacteria rhinosinusitis (ABRS) guidelines

High-dose” (2 g orally twice daily or 90 mg/kg/day orally twice daily) amoxicillin-clavulanate is recommended for children and adults with ABRS from geographic regions with high endemic rates (≥10%) of invasive penicillin-nonsusceptible (PNS) S. pneumoniae.

High dose amoxicillin high dose should be enough to treat penicillin-nonsusceptible S. pneumoniae, at least for a relatively trivial infection like sinusitis.  But here is the deal: S. pneumoniae doesn’t make a beta lactamase, an enzyme that degrades amoxicillin, which they point out in the text.  For S. pneumoniae, clavulanate, which inhibits the beta-lactamase, adds nothing to the treatment over amoxicillin except diarrhea and cost  and is recommended for the odd H. influenza in the sinus of adults, which does make a beta lactamase,  as they mention:

H. influenzae was isolated in 36% of patients with positive bacterial cultures consistent with ABRS, compared with 38% for S. pneumoniae and 16% for M. catarrhalis. Unfortunately, the rate of β-lactamase–producing H. influenzae was not reported in this study.

After their convoluted argument, they finally conclude

Thus, the recommendation of amoxicillin-clavulanate in adult patients with ABRS is primarily based on in vitro susceptibility data and the current prevalence rates of β-lactamase production among H. influenzae.

Why even bother?  Not confidence inspiring, although I understand the reasoning, it is awkward at best: You want to give more amoxicillin if you have high endemic rates  of invasive penicillin-nonsusceptible (PNS) S. pneumoniae but due to the potential for  H influenza you want  amoxicillin-clavulanate instead.   I suspect the commentators on Medscape who complain about guidelines were not doing so due to a fine reading of the content in the context of an understanding of an extensive literature, but more in rebellion against being told what to do by the man: insurance companies and the smarting pants academics.  I share their concerns, but for different reasons.

There is science and evidence, and often I do not really know what do with the results, even when impressive.  Recently the NEJM  published an article that demonstrated that azithromycin, when compared to other agents, was associated with ‘excess deaths’  The overall increase in deaths was small, although not if you were one those in the excess.

“Relative to amoxicillin, azithromycin was associated with an increased risk of cardiovascular death (hazard ratio, 2.49; 95% CI, 1.38 to 4.50; P=0.002) and death from any cause (hazard ratio, 2.02; 95% CI, 1.24 to 3.30; P=0.005), with an estimated 47 additional cardiovascular deaths per 1 million courses; patients in the highest decile of risk for cardiovascular disease had an estimated 245 additional cardiovascular deaths per 1 million courses.”

The paper is EBM/SBM, an epidemiologic study.  How do I approach such a study?  First, I will admit, is a gut check.  Was I surprised at the headline?  Nope.  Quinolones, azoles and macrolides are all known to have effects on the cardiac conduction system.  This is in contrast with a report that quinolones were associated with an increase in retinal detachment.  While quinolones are known to affect connective tissue, I am a bit more skeptical about this one.  I would like to see it confirmed.

The azithromycin study I am more likely to take to heart.  The problem with the study is that while it demonstrated those with cardiac problems were more likely to have the adverse reaction,the main indication for the antibiotics was sinusitis or bronchitis.  Will the same effects happen for other infections?  What is the risk and benefit?

For example, in both sepsis and pneumonia, receiving a macrolide is associated with increased survival compared to those who do not.  Also, acute pneumonia, where there is survival benefit from a macrolide, is one of many infections with increased vascular events, including arrhythmias and heart attacks.  Since macrolides are part of the guidelines for the treatment pneumonias, are we seeing more deaths from the drug or the infections for which it is being used?   That being said, I would be less likely to use azithromycin for ‘trivial’ infections like sinusitis and bronchitis that I would for more life threatening infections.

What do you do if you think experience trumps data and evidence?  The risk was increased from 47 to 245 per million doses azithromycin given, making it unlikely that an experienced provider, with 3 to 6000 patients will ever be likely to inadvertently kill a patient with azithromycin.  In my experience, I never killed a patient with azithromycin. That you know of.

That is the problem with experience based medicine.  Experience will always be insufficient to recognize rare yet important complications or the beneficial effects of therapies applied to populations such as treating hypertension.  SBM should be the basis of medicine, but its application will often require the ‘art’ of medicine.