Secret Study, Revealed

If it seems like it was just yesterday that I wrote about The Secret Studies of Paxlovid, it was. But last week, less than a month after my piece and nearly two years after it was completed, the most important secret Paxlovid study was published. Let’s dig in to it.

When I was a kid my dad brought home a Rubik’s Cube. I liked challenges so I receded into my room and a little later (read: way later) came out with a solved Cube. Mom and dad gushed, and I felt like a bag of chips, so in the morning I showed off the Cube at school. My 5^th grade nemesis Marcus immediately grabbed it, scrambled it, and handed it back.

“Do it now, Newman.” He and three other boys stood, waiting.

 “Right. Um, yeah. Ok. Cool.” I hadn’t planned for this. And I had no clue how I’d done it.   

They let me fumble, then started giggling, then blew up.

“I call bulls---!”

“Nice try!”

“Faker!”

“Oh SNAP!” (Always in there).

In science, when a group, lab, or company generates a seemingly miraculous result, it’s meaningless until it’s been replicated. This is why for new drugs, the FDA says “the drug maker will submit results from two well-designed clinical trials, to be sure the findings from the first trial are not the result of chance or bias.”

The philosopher Karl Popper noted that, given the theory “all swans are white”, discovering even a single black swan proves the theory false. This path to knowledge, called ‘falsification’, is the backbone of good research because it leads to a conclusion, “all swans are not white,” that is irrefutably true, a quality otherwise hard to come by.

The FDA’s policy therefore asks drug makers to make an earnest attempt to find black swans, by performing at least two tests of a hypothesis, and replicating any positive findings. If either trial fails, an irrefutable truth emerges: the drug does not work as hoped and planned. Back to the drawing board.

Failure to replicate is not the exception, it’s the rule. A few years back a group reported miraculous results with a Vitamin C-based cocktail for life-threatening sepsis. But multiple rigorous trials were unable to replicate the results, including at least one in which the cocktail increased deaths. This pattern of initial excitement (white swan) then clear cut failure (black swans), is so common that entire research papers have been written on it. Just last week the first rigorous trial results for the ALS drug Relyvrio were made public. After the FDA authorized it in 2022 based on promising early data, 4,000 people began taking the drug at $158,000 per year (each). Results: same as a placebo. Black swan.

Which brings us to Paxlovid. Last month I pointed out Paxlovid’s multiple hidden (i.e. unpublished) black swan studies. But I couldn’t solve the mystery of why the FDA approved the drug, since they’d seen the failed studies. Last week Paxlovid’s second, and most important, trial was finally published and the report’s authors—bless their hearts—let the cat out of the bag.

Here’s how the trickery worked: In the first Paxlovid trial the primary outcome, the trial’s focus, was reducing serious illness due to COVID. This appeared to succeed. Then, the second trial claimed a different focus, reducing symptoms of COVID. This failed. But the two different primary outcomes gave the impression of two different trials testing two different hypotheses. Therefore Pfizer never had to admit they failed to replicate the results of their first trial.

Imagine if, when Marcus handed me the scrambled Cube, I set it down and picked up a crossword instead. Then, after I flubbed the crossword, I pointed out that crosswords are different than Rubik’s Cubes—and since they only saw me fail at the crossword, I must still be a master at the Cube. That’s how Pfizer convinced the FDA.

But as inane as that sounds, it gets worse: Pfizer actually was attempting to replicate the first study. They were pretending to do a crossword while failing to solve a Cube.

Buried deep in the statistics section of the paper is this gem: “The protocol was later amended to include more participants to allow for better evaluation of the secondary efficacy end point [serious illness].”

In other words they altered the second trial’s size hoping to show the drug reduced serious illness. And despite the misleading language (“secondary endpoint”), no study alters enrollment for a secondary outcome. Indeed, Pfizer’s own press release announcing the results in June 2022 openly concedes they halted the second trial “due to low rates of hospitalization or death [i.e. serious illness].” Halting a study is another rare and extreme maneuver, used based exclusively on a study’s primary outcome.

Therefore Pfizer’s second Paxlovid trial tested the same primary outcome as the first—it was a replication trial, and it failed. Classic black swan.

Based on the drug’s FDA label (see paragraph 2, p37, “secondary endpoint”) it appears Pfizer fooled the FDA. Or perhaps the FDA understood exactly what happened. Not sure which one’s worse.

The evidence is finally public. By the most basic FDA standard, Paxlovid is officially a failed drug.