Why the Alzheimer's Drugs Won't Make Your Burger Better

Over-powered studies are forensic-level proof that a drug doesn't work

I like a burger. With cheese. And lettuce and tomato and maybe onion and ketchup or sauce (house choice). I prefer it rare. Really rare. Like, true blue.

This last part is the downfall of many a burger joint where, fearing liability, kitchens won’t make a rare burger. To me, besides corrupting its taste, this means dubious beef. Sushi, steak tartare, and raw bar items are much riskier than anything cooked, but they’re still common menu items. A kitchen that won’t make a rare burger doesn’t trust its sourcing and prep for burger meat. Sad!

But crucially, rare burgers aren’t just hard to find, they’re also tricky to make because they must—MUST—be seared on the outside and lightly cooked on the inside. This delicate contrast, a pillowy but juicy inside with a firm outer layer, is the mouth-watering soul of a great burger.

So for home burgering, I got a meat thermometer.

To my surprise, the first time I used it the reading was 122.68°. Impressive precision! But this number was, as the kids say, TMI (not to mention sus). Because while my gifted palate is a wonder of epicurean virtuosity, I cannot tell a 1° difference in temperature, much less 0.01°. Those decimal points are not helping me.

I can tell rare from medium-rare (I think). That’s about 10° different. Which means somewhere between 1° and 10°, for me, the difference becomes perceivable and thus potentially meaningful. If I was bored and curious I might experiment to find the threshold, and in research parlance we would call it the ‘clinically significant’ difference. This principle, also known as the minimal clinically important difference (or ‘MCID’), defines the smallest improvement patients and doctors are able to perceive, a starting point for judging whether treatments provide a meaningful benefit.

Curiously, however, all three FDA-approved Alzheimer’s drugs were tested in studies that are the equivalent of a 2-decimal point thermometer.

Each trial enrolled way more people than needed to find improvements in dementia. Why? Because the more participants, the smaller a difference a study can find. But my thermometer, for instance, can detect differences that are much too tiny to matter. And so can studies. In research this is called over-powering, but in trials it’s very unusual. After all, genuinely helpful drug effects, even lowly MCIDs, aren’t tiny. So why would researchers go looking for tiny differences? Who even has the money and resources to enroll way too many people in a randomized trial??

Oh. Wait.

As discussed in these pages before, a jarring attribute of the Alzheimer’s drugs is that trials proved they do not make a perceivable difference. One hapless employee at Eli Lilly even spent years studying (and writing at least seven reports) to establish the MCID for cognitive impairment. He found it was at least 5 points on a 144-point scale for mild disease and 9 points for early Alzheimer’s. But his company’s drug Kisunla produced scores within 3 points of a placebo—in other words, indistinguishable from placebo. Meanwhile Biogen and Eisai’s drug Leqembi was within 0.45 points of placebo on an 18-point scale—also less than half the established MCID of 1 point.

How could this happen? Were the studies purposefully over-powered? Or did they stumble upon tiny differences while seeking meaningful ones?

In the Kisunla report Eli Lilly says they aimed to find a 3-point difference (!!!). The authors calculated that 1,000 participants would give them a 95% chance, or ‘power’, to find it. So they enrolled 1,000 people. Then they enrolled 736 more.

What about Leqembi? The FDA approval trial also, amazingly, targeted 0.4 points better than placebo (!!!). This, they found, required 1,566 participants—and they enrolled nearly 1,800.

This is over-over-powering. The original plans were over-powered, and then each study ADDED extra enrollments. These are confessions, hidden in plain sight in the methods sections of the papers, and they mean the companies knew before the trials started that their drugs don’t help. So they targeted statistical—not perceivable—differences, then convinced the FDA to approve the drugs.¹

Bonus question: How could the companies feel confident the studies would generate a statistical advantage for their drug? Answer: The double-dog placebo.

In their brilliant, deeply researched investigation Jeanne Lenzer and Shannon Brownlee describe a woman who was convinced, and whose friend was convinced, Leqembi was rapidly improving her memory. But she was receiving a placebo. As placebo researcher Irving Hirsch explains in his wonderful book, placebo effects are common, but they’re reliably larger when people experience drug side effects. Side effects effectively unblind study participants, letting them know they’re on a real drug and amplifying their expectations, excitement, and thus placebo effects.

One of the great ironies of modern scientific rigor, therefore, is that ineffective drugs in double-blinded trials can generate statistical advantages if they have harmful side effects. People given Leqembi in trials had infusion reactions 19% more often than those given placebos, and brain swelling 11% more often. That’s a lot of unblinding, easily enough double-dog placebo to virtually guarantee an advantage.²

My burger thermometer gives me too much information, and so did the trials for the new Alzheimer’s drugs. Deliberately over-powered trials are a smoking gun, telling us what the companies knew all along: their drugs don’t work.

1. There are loopholes for approval that drugs with ‘breakthrough’ status (meaning they may be the first to change the course of a disease) like Leqembi and Kisunla can aim for. One is the use of surrogate markers—rather than clinically meaningful differences—as their measure of success. In this case the surrogate marker is amyloid plaques, which are indeed consistently removed by the new drugs. But rather than proof that the drugs help people, this dual finding from both trials (reduced plaques with no meaningful cognitive improvement) confirms the findings of many other studies of the same drug class essentially proves the amyloid plaque theory of Alzheimer’s Disease was wrong all along. See here, here, and here for more on that.

2. I would argue this double-dog placebo impact was probably far greater than the tiny advantages favoring the Alzheimer’s drugs in trials, and therefore likely masked a significant harmful impact of the drug on cognitive performance.