Publication bias

Although we’re all (presumably) better off when we try to answer questions by turning to the literature rather than, say, a Magic 8 ball, it behooves us to remember that it’s still an imperfect source for truth. And while you can hone your ability to peruse a study for flaws or poor quality, one foible may never be detectable: publication bias.

In other words, no matter how careful you are, you can only read the studies that are published — you never know about the ones that aren’t. And odds are, the ones that weren’t published aren’t just a random selection… they’re the ones that weren’t favorable to the sponsors or didn’t find interesting results.

So when you look over the five studies that support an intervention and the one that doesn’t, bear in mind there may be fifteen other studies producing negative results that never saw the light of day. (Oh, and take a look at the conflicts of interest declaration — and compare them to who might stand to benefit from a positive study.)

Here’s a good, brisk introduction to the subject

And here’s the exchange Goldacre mentions between the Cochrane group and Roche Pharmaceuticals, if you can stomach some truly shameless evasion.

OR, ARR, RRR, NNT, WTF?

Hopefully you’re out there reading the research and trying to figure out what you should be doing and why. But inevitably, when you get to the results, they’re reported using a baffling array of acronymic metrics. Some are fairly intuitive; some are truly confusing; but you should understand them all. Here’s a quick run-through on the most common terms, particularly odds ratios, absolute risk reduction, relative risk, and NNT, using two recently-added studies (Kudenchuk 1999 and Jacobs 2011) as examples.

DRL Update

DRL update: 12 new additions: 6 Cardiac Arrest, 2 Spinal Immobilization, 4 misc. (all Fluid Resuscitation)

 

Some good reviews on the evidence behind fluid resuscitation strategies, as well as a historical piece (nearly 100 years old!). Under Spinal Immobilization, there’s the 2013 update to the AANS/CNS guidelines, and a nice review on immobilization techniques. And in Cardiac Arrest there’s the seminal Valenzuela casino paper (38% all-comers survival to discharge! if you’re going to die, die in a casino), a couple studies on devices (Lucas implementation and ACD-CPR), high-dose epi, and two of the best studies on thrombolytics during cardiac arrest (spoiler alert: they don’t help). Lots of great stuff.

As always, search for *** to view new material.

Determining Risk/benefit Using Test Thresholds

A quick illustration of how to drill down on the group of people in a cohort who actually benefits from a course of care, and compare it against the group who is harmed. Important, important stuff when it comes to reading any study.

 

The Legend of the Unstable Spine

So the AANS/CNS released an update to their 2003 guidelines for acute spinal care (Theodore 2013), and this is in the introduction to the prehospital chapter:

Pathologic motion of the injured cervical spine may create or exacerbate cervical spinal cord or cervical nerve root injury.9-11,16,51,52

Although immobilization of an unstable cervical spinal injury makes good sense and Class III medical evidence reports exist of neurological worsening with failure of adequate spinal immobilization, there have been no randomized trials or case-control studies that address the impact of spinal immobilization on clinical outcomes after cervical spinal column injury.3,4,6,11,12,15,16,27,31,32,53

Actually, if you read through enough papers on spinal immobilization, pretty much every introduction reads this same way. First they say “spinal fractures suck, this is their prevalence”; then they say, “not immobilizing these patients can increase morbidity and disability, which is catastrophic”; then they say, “there’s actually no evidence for immobilization, but still.

It’s the third part that chafes my nuts, because “the exception proves the rule” — by acknowledging the inadequacy of evidence for our immobilization techniques, it seems to confirm that the other stuff is well-supported. In particular, it seems clear that “pathologic motion may create or exacerbate” cord injury. Just look at all those citations!

But in reality, it’s all just hand-waving. In philosophy, there’s a school called “coherentism,” which essentially states that a system of beliefs is “true” if everything you believe mutually agrees — regardless of whether the lot of it has any connection to reality. This is pretty much what we’re dealing with, because if you start looking through those citations — and this is true for every one of those spine papers with carbon-copy introductions — you find that there’s no “there” there. Most of the pieces cited (they’re all in the DRL if you want to explore) only support the idea of neurologic deterioration in a single sense: that they also made this same unsupported statement in their introduction. In some cases they mix it up and say it in the conclusion. But either way there’s no actual connection with reality, except in the sense that they cite a bunch of similar papers that made the same unsubstantiated claim in their introduction. Round and round it goes.

The threadbare actual evidence for this consists of examples of neurologic deterioration that happened after hospital arrival, either in the ED, in the OR, on the floor, or in at least one case while moving someone to the x-ray table. These are exclusively case reports or case series, but at least go to show that the concept is not completely fictional. However, there are no examples of this happening prior to hospital arrival, which is presumably what “prehospital” immobilization is meant to address. (There are a few cases of people hurting themselves, walking around for a few hours or days, and then self-presenting because they developed tingling in their arm or something, but that’s not really the catastrophic event we’re trying to prevent and is not an EMS matter either.)

Really the only primary “evidence” of this happening is from two or three very old papers that claim it happens all the time, but as a rule, give no further support or details to back that statement. For instance Geisler 1966:

These 29 comprise those patients who at first had little or no spinal cord involvement although they had suffered spinal column injury of great severity. The subsequent development of serious spinal cord involvement is confirmed by clinical observation or circumstantial evidence. The evidence for this statement derives in each case from the recorded witness of a reliable observer or from the account given by a credible patient who is able to report activities such as walking or definite use of the hands following spinal injury, and who subsequently had lost the ability to use these members owing to developing paralysis. The paralysis occurred in each case as a consequence of failure to recognize the injury to the spinal column and to protect the patient from the consequences of his unstable spine. He may thus have essayed to walk, or even sit up for a drink. It may have occurred when the patient was being moved from the site of accident to a hospital.

Or the true classic, Toscano 1988, who claimed that a whopping 28% of cord-injured patients deteriorated between injury and hospital admission, due entirely to inadequate immobilization. But he describes none of these patients in detail, the actual published paper is incredibly terse, and for further information you’d have to read his med school thesis, which only exists on paper in the University of Melbourne library. Good luck with that.

Anecdotally, of course, everyone in EMS has heard about the patient who moved wrong and was forever paralyzed. But in reality, if this happened, it would be publishable, because that person doesn’t exist anywhere in the literature. (Of course, maybe if that happened, the medics involved would just button their lips and never speak of it, so you could make an argument for reporting bias.)

Howdy and Welcome

Welcome to Lit Whisperers, a new side-blog of EMS Basics.

Several months ago myself and several colleagues launched the Digital Research Library, essentially an organized index of peer-reviewed research useful to prehospital medicine. The goal was to make it easy — or at least possible — for the average EMT or paramedic to find studies relevant to their work, and use them to provide better, more evidence-based care.

The project has exploded, and continues to evolve as we find bigger and better ways to sort, codify, and bring together the wide world of medical literature. But something else happened, too. Because inputting a study involved reading it from cover to cover and analyzing it deeply for its methods, limitations, conclusions, and relation to other work, after going through a bunch of these, we started to notice things. You notice patterns, clusters between certain studies, and gaps where there should be more but isn’t. You notice odd results that aren’t trumpeted, but might have important ramifications. You get a pretty good grasp on the depth and breadth of what’s out there, and while some of that information was pumped into the library resources, some of it went nowhere, because our goal was to provide basic tools for objective research, not to share our opinions and personal musings.

So to make a long story short, this blog is a place to share our opinions and personal musings. All existing editors and librarians who contribute to the DRL can use this space to share whatever thoughts, observations, or points of interest they care to.

We’re humans and we have the same biases and fallibilities as anyone, and you’ll probably see those showcased here. There’s no posting schedule; take ’em as they come. But hopefully this soapbox will add some context and additional layers to your understanding of the research, and shine extra light that helps guide you somewhere good.