Perhaps the most important attribute in science and scientists is humility. A scientist must be passionate about finding truth, but dispassionate about judging it.
That’s hard to do, particularly with issues like politics or religion. For instance in a recent NY Times piece titled “As Congress Debates Cutting Medicaid, a Major Study Shows It Saves Lives.”
We are in a complicated and fraught moment of political history. Things are changing and it isn’t clear where we’re headed in the USA—which means big feelings. Really big.
But translating research properly means setting aside big feelings and dispassionately judging what a study can tell us. And one of the worst mistakes one can make in interpreting research is to adopt the hubris of certainty. For instance “… Shows It Saves Lives.”
Reader, may I ask you to search your soul for the type of research data that can provide certainty? In the pages, notes, comments, and discussions of Research Translation we have tackled the issue more than once (perhaps infinitely). What say you? What kind of research offers certainty?
One reasonable answer would be ‘none’. Arguably, virtually any research result can and will be overturned or at least seriously challenged at some point. Another possible answer would be ‘randomized trials’. More pointedly, randomized trials that have been replicated, showing the same results.
So is the ‘Major Study’ a randomized trial replicating previous work? Well no. It isn’t a trial at all. Actually, it’s not even peer-reviewed and would therefore not be considered a published or vetted study in the scientific community. It’s a ‘working paper’ by the nonprofit National Bureau of Economic Research. The NBER describes the paper as “circulated for discussion and comment purposes,” and confirms it is neither peer reviewed nor reviewed by the group’s directors (i.e. Don’t blame us!).
And what exactly does the paper examine? The paper looks at over a decade of mortality data in states that did and did not expand Medicaid from 2010 to 2022. In states that did, they report, fewer people living under the federal poverty level died than in states that didn’t.
From the many discussions of studies in RT, I feel sure my brilliant readers know that since this was not a randomized trial it cannot speak to causation, only association. Therefore this observational paper cannot “show Medicaid saves lives.” Incredibly, the authors of the paper seem illiterate on this fact. Their opening sentence begins “We examine the causal effect of health insurance on mortality… .”
Why can this observational inquiry not claim causation? One reason is people living in states that did and didn’t expand Medicaid were different before 2010, through the next decade, and they’re different now. Mortality trends are affected by car accident rates, Covid deaths, the opiate crisis, obesity rates, violence and crime rates, and a million other things known to vary wildly from community to community and state to state. Claiming Medicaid as the cause for these differences is unscientific and, frankly, daft.
It’s a shame it wasn’t a randomized trial, since there is one previous trial of Medicaid (just one). When Oregon expanded Medicaid in 2008 they had just enough resources to insure half of new applicants, so they randomly assigned Medicaid by lottery, then followed outcomes for two years. Medicaid appeared to reduce personal bankruptcies, increase ER visits, increase diabetes diagnoses, decrease diagnoses of depression, and increase the use of preventive services. Perhaps because it only lasted two years with a relatively small cohort, however, there was no statistical change in physical health or death rates.
Bottom line, there is no ‘Major Study’ and the un-vetted paper the Times relied on did not show Medicaid saves lives. The question of whether insurance saves lives in the U.S. is simply not answerable with study data. Instead, you’ll have to do what the Times did—use your gut to answer that question.
But since you, as a reader and listener of RT, are far more scientific than the Times journalists perhaps you’ll preface your opinion by noting that this is a question of politics and ideology, and the implications of the data are uncertain.
After which, you may feel free to go ahead and indulge your very biggest feelings. Enjoy!
Share this post