Aiming to place a daunting 18-foot model of Stonehenge on the stage, Nigel Tufnel drew the structure on a napkin. Ian, manager of the mockumentary band Spinal Tap, took the napkin to a prop designer, who then built what she saw. Unfortunately the drawing used double quotation marks for size, indicating inches rather than feet. Too frightened to tell the band of the flub, Ian surprised them mid-concert by unveiling the tiny Stonehenge (and tiny people to dance around it).

Last week the New England Journal of Medicine published the research equivalent of tiny Stonehenge. The flu drug being studied was embarrassingly ineffective, unable to keep the flu from spreading within households. This result was clear enough that anyone who read the study (or the abstract) knew it. Based on the broad failure of drugs in the same class, like Tamiflu, this shouldn’t have been surprising. But also unsurprising was how the study authors applied multiple layers of lipstick to their shiny new pig.

Research Translation is 100% reader-supported. Pleeeeease subscribe if you dig.

For instance, they magnanimously concluded the drug “led to a lower incidence of transmission of influenza virus to close contacts than placebo.” Which is weird, because the drug did not prevent illnesses in close contacts.

Here’s what happened. To determine whether the antiviral drug baloxavir could reduce household transmission, the researchers randomly assigned 1,457 people ill with influenza to baloxavir or a placebo. Then, for 5 days the researchers recorded the nasal swab results and symptoms experienced by 2,681 people living in the same households.

By reporting the swab results as a primary outcome, the study authors were able to claim that 4% less in the baloxavir households tested positive than in the placebo households (9% vs 13%). The study was large enough that this measly (and clinically unimportant) difference was statistically significant.

But much, much more importantly the rate of flu illness was 5.8% vs 7.6% in the two groups—statistically the same. In other words baloxavir didn’t prevent close contacts from falling ill, but it did slightly reduce the chance of finding traces of the virus in a person’s nose.

Which one, would you guess, matters to the study participants?

It is borderline hilarious to have the researchers claiming the drug worked, when we all know what the participants would say. It’s a lot like having little people dance around a little Stonehenge, while a band manager pretends it’s exactly the effect they were hoping for.

There’s a darker and more disturbing note, however. The trial was sponsored, executed, and authored primarily by employees of the manufacturer (Hoffman-Laroche). We expect drug company employees to distort and spin results. But in this case they deliberately planned an 18-inch Stonehenge. The study was designed to show a difference in nasal swab results—illness, the thing that matters to people, was an afterthought. In other words researchers knew the drug couldn’t prevent illness, so they targeted a meaningless surrogate instead.

But they would only do such a thing if they also knew the New England Journal of Medicine would both publish their study, and allow them to claim victory. And indeed, the Journal went further, preparing and posting a video to hype and advertise the result, a promotional service offered only to select papers hand-picked by the editors.

In 2005 Richard Smith, a longtime editor-in-chief of the prestigious British Medical Journal, wrote that medical journals act as “an extension of the marketing arm of pharmaceutical companies.” The baloxavir study is a brazen and odious example. For not only did the editors fail to point out the emptiness of preventing a nasal swab result; not only did they allow the authors to claim unqualified success; not only did they publish an editorial suggesting the drug might be great for a pandemic (stockpile, anyone?); they also took special steps to highlight and amplify the deceptive findings.

When Spinal Tap unveiled their wee Stonehenge, accompanied by threatening music and an ominous narration, the crowd guffawed. Only Ian, the band manager, could even pretend it worked. Similarly, when a drug company claims breakthrough success for a drug that any educated reader can tell helps no one, the Journal should be the crowd—not Ian. Instead, their peer review is ineffectual and their editing incompetent, as it was decades ago for Tamiflu, and continues to be for Alzheimer’s drugs, cardiac risk, paxlovid, stents, and more.

This is why the conclusions from most New England Journal of Medicine studies should be taken about as seriously as an itty-bitty Stonehenge.

Get full access to Research Translation at researchtranslation.substack.com/subscribe