Aiming to place a daunting 18-foot model of Stonehenge on the stage, Nigel Tufnel drew the structure on a napkin. Ian, manager of the mockumentary band Spinal Tap, took the napkin to a prop designer, who then built what she saw. Unfortunately the drawing used double quotation marks for size, indicating inches rather than feet. Too frightened to tell the band of the flub, Ian surprised them mid-concert by unveiling the tiny Stonehenge (and tiny people to dance around it).
Last week the New England Journal of Medicine published the research equivalent of tiny Stonehenge. The flu drug being studied was embarrassingly ineffective, unable to keep the flu from spreading within households. This result was clear enough that anyone who read the study (or the abstract) knew it. Based on the broad failure of drugs in the same class, like Tamiflu, this shouldn’t have been surprising. But also unsurprising was how the study authors applied multiple layers of lipstick to their shiny new pig.
For instance, they magnanimously concluded the drug “led to a lower incidence of transmission of influenza virus to close contacts than placebo.” Which is weird, because the drug did not prevent illnesses in close contacts.
Here’s what happened. To determine whether the antiviral drug baloxavir could reduce household transmission, the researchers randomly assigned 1,457 people ill with influenza to baloxavir or a placebo. Then, for 5 days the researchers recorded the nasal swab results and symptoms experienced by 2,681 people living in the same households.
By reporting the swab results as a primary outcome, the study authors were able to claim that 4% less in the baloxavir households tested positive than in the placebo households (9% vs 13%). The study was large enough that this measly (and clinically unimportant)1 difference was statistically significant.
But much, much more importantly the rate of flu illness was 5.8% vs 7.6% in the two groups—statistically the same. In other words baloxavir didn’t prevent close contacts from falling ill, but it did slightly reduce the chance of finding traces of the virus in a person’s nose.
Which one, would you guess, matters to the study participants?
It is borderline hilarious to have the researchers claiming the drug worked, when we all know what the participants would say. It’s a lot like having little people dance around a little Stonehenge, while a band manager pretends it’s exactly the effect they were hoping for.
There’s a darker and more disturbing note, however. The trial was sponsored, executed, and authored primarily by employees of the manufacturer (Hoffman-Laroche). We expect drug company employees to distort and spin results. But in this case they deliberately planned an 18-inch Stonehenge. The study was designed to show a difference in nasal swab results—illness, the thing that matters to people, was an afterthought. In other words researchers knew the drug couldn’t prevent illness, so they targeted a meaningless surrogate instead.
But they would only do such a thing if they also knew the New England Journal of Medicine would both publish their study, and allow them to claim victory. And indeed, the Journal went further, preparing and posting a video to hype and advertise the result, a promotional service offered only to select papers hand-picked by the editors.
In 2005 Richard Smith, a longtime editor-in-chief of the prestigious British Medical Journal, wrote that medical journals act as “an extension of the marketing arm of pharmaceutical companies.” The baloxavir study is a brazen and odious example. For not only did the editors fail to point out the emptiness of preventing a nasal swab result; not only did they allow the authors to claim unqualified success; not only did they publish an editorial suggesting the drug might be great for a pandemic (stockpile, anyone?);2 they also took special steps to highlight and amplify the deceptive findings.
When Spinal Tap unveiled their wee Stonehenge, accompanied by threatening music and an ominous narration, the crowd guffawed. Only Ian, the band manager, could even pretend it worked. Similarly, when a drug company claims breakthrough success for a drug that any educated reader can tell helps no one, the Journal should be the crowd—not Ian. Instead, their peer review is ineffectual and their editing incompetent, as it was decades ago for Tamiflu, and continues to be for Alzheimer’s drugs, cardiac risk, paxlovid, stents, and more.
This is why the conclusions from most New England Journal of Medicine studies should be taken about as seriously as an itty-bitty Stonehenge.
I am calling this unimportant based on judgment. My judgment is that, first, the difference is about test results and not human illness. But second, and perhaps more importantly, even if we valued the nasal swab results more than the actual illnesses, this study suggests that at best 1 in every 25 close contacts (i.e. 4%) could benefit from their family member taking the drug, while 24 out of 25 close contacts wouldn’t be affected. An effective prevention doesn’t fail to help 24 out of 25 people. Even the relative reduction, an often misleading number that magnifies the perceived effect of the drug, was less than a third. This means more than two thirds of the unlucky household contacts who were destined to get sick, still did. (9.5% vs 13.4% tested positive in the two groups, which means 9.5/13.4 is the relative reduction, about 30%, while the absolute reduction is 13.4 - 9.5, or about 4%).
If this sounds hyperbolic consider that the marketing of Tamiflu, a drug originally developed by Gilead, a drug company once led by Donald Rumsfeld (later the US Secretary of Defense) was carefully directed at military and public health authorities—and it worked. The Department of Defense stockpiled billions worth of Tamiflu, boosting the company’s sales and stock prices. Still doubting? Check out the discussion section of the baloxavir report, which quickly dismisses the lack of any clinical effect of the drug, then goes on to spend a full paragraph explaining how “In a pandemic, development of a vaccine would take time,28 and the availability of an antiviral drug that reduces disease severity and person-to-person transmission could serve as a new element to combat such a pandemic.” So, yeah. Their marketing goals seem pretty clear.
Share this post