Excellent explanation of the untrustworthiness of most observational studies. Some of the most sophisticated epidemiologists repeatedly get this wrong. The most glaring example was the Nurses' Health Study Finding (published in the New England Journal of Medicine in 1997), which found that using hormone replacement therapy (HRT) led to women living longer. When the dust finally settled after the randomized controlled trial of HRT, the Women's Health Initiative, was stopped prematurely because the risk of breast cancer, heart attack, stroke, and blood clots was significantly greater than the benefits of decreased hip fractures and colon cancers. The key to understanding why the Nurses' Health Study investigators got it wrong is that they did not appreciate the impact of the inherent differences between women whose doctors put them on HRT and those whose doctors did not. The truth of the matter is that being on HRT was a marker of being predisposed to better health (wealthier, better educated, whiter...) rather than the reverse.
This elementary misinterpretation of the validity of observational data continues--especially when it is commercially advantageous. For example the first COVID boosters were approved based on a 94% reduction hospitalization and death from Covid among people who chose to get the booster. Two years later it came out that this same "benefit" was seen in the non-Covid death rate in people who got the booster. There is no conceivable way that Covid boosters could prevent non-Covid death. This just shows how powerful (and distorting) reliance on observational rather than RCT data can be.
John Abramson MD MSc
Author of Sickening: How Big Pharma Broke American Health Care and How We Can Repair It
Such an important point, John. The problem truly is rampant and drug companies have used it to their advantage in claims about boosters, vaccine data more generally (Israeli and other studies), Paxlovid and more. Thanks for the comment.
Excellent explanation of the untrustworthiness of most observational studies. Some of the most sophisticated epidemiologists repeatedly get this wrong. The most glaring example was the Nurses' Health Study Finding (published in the New England Journal of Medicine in 1997), which found that using hormone replacement therapy (HRT) led to women living longer. When the dust finally settled after the randomized controlled trial of HRT, the Women's Health Initiative, was stopped prematurely because the risk of breast cancer, heart attack, stroke, and blood clots was significantly greater than the benefits of decreased hip fractures and colon cancers. The key to understanding why the Nurses' Health Study investigators got it wrong is that they did not appreciate the impact of the inherent differences between women whose doctors put them on HRT and those whose doctors did not. The truth of the matter is that being on HRT was a marker of being predisposed to better health (wealthier, better educated, whiter...) rather than the reverse.
This elementary misinterpretation of the validity of observational data continues--especially when it is commercially advantageous. For example the first COVID boosters were approved based on a 94% reduction hospitalization and death from Covid among people who chose to get the booster. Two years later it came out that this same "benefit" was seen in the non-Covid death rate in people who got the booster. There is no conceivable way that Covid boosters could prevent non-Covid death. This just shows how powerful (and distorting) reliance on observational rather than RCT data can be.
John Abramson MD MSc
Author of Sickening: How Big Pharma Broke American Health Care and How We Can Repair It
Such an important point, John. The problem truly is rampant and drug companies have used it to their advantage in claims about boosters, vaccine data more generally (Israeli and other studies), Paxlovid and more. Thanks for the comment.