How Do We Know What Really Works in Healthcare? (Ep. 201)
Our latest Freakonomics Radio episode is called “How Do We Know What Really Works in Healthcare?” (You can subscribe to the podcast at iTunes or elsewhere, get the RSS feed, or listen via the media player above. You can also read the transcript, which includes credits for the music you’ll hear in the episode.) The gist of the episode: a lot of the conventional wisdom in medicine is nothing more than hunch or wishful thinking. A new breed of data detectives is hoping to change that.
Imagine there’s one elementary school in one district where the kids do much better than all the other nearby schools. This also happens to be the only school that serves its kids breakfast every day (in addition to lunch). It’d be tempting to conclude that the school’s good grades are due to the breakfast — and that if you simply started serving breakfast at all the other schools, their grades would also shoot up. But how can you tell for sure? Maybe breakfast is one of 10 things this school does differently — or maybe the kids are different, or the parents, or the teachers, or the curriculum. Maybe this is the only school where dodge ball is played every day at recess. So how do you find out; how do you isolate the effect of the breakfast? You set up an experiment – a randomized controlled trial or RCT, like the ones traditionally used in bench science, in drug studies, and elsewhere. You take one population, randomly divide it into groups, and give some groups a treatment that the others don’t get. Then you can measure whether the treatment group came out any differently than the control group. Here’s what Steve Levitt has to say in the podcast about the use of RCTs in research:
LEVITT: So I think the randomized trial is the very best way to learn about the world around us. And that’s for a couple of reasons. One is because randomization is just your best friend when you’re trying to find causality. Because absent randomization, you always have to tell stories about why what we observe in the world — which are correlations — actually can be mapped into causal relationships. But the beauty of randomization, if done well, and at least in large numbers with large samples, is that because you’ve randomized, on average, you expect the outcomes to be exactly the same for the treatment group and the control group.
As we’ve regularly noted in the past, economists and other academic researchers have increasingly been using RCTs to study all sorts of things, including how to best fight poverty. At the forefront of this movement is J-PAL, or the Abdul Latif Jameel Poverty Action Lab, at MIT. The award-winning economist Esther Duflo, one of J-PAL’s founders, has helped run many RCTs in India, Kenya, and elsewhere, trying to learn how best to prevent teen pregnancy and anemia, and drunk driving; and how to better incentivize nurses, small-business growth, and modern farming techniques.
In this episode, we turn our attention to the U.S. and J-PAL’s efforts to learn about what really works in healthcare delivery. We focus on research done by the MIT economist Amy Finkelstein and several colleagues, whose growing body of work in this realm is fascinating. As Finkelstein tells us in the podcast, RCTs are far too rare in healthcare delivery — which is a shame, for the link between healthcare and poverty is strong:
FINKELSTEIN: We take a rather broad view of poverty alleviation. And so anything that improves the efficiency of healthcare delivery, I think is important for the public for two reasons. First, you know, the poor are disproportionately unhealthy and therefore have the burden of healthcare relative to less poor people. Also, given that healthcare spending is currently about a fifth of public-sector budgets at the state and federal level, anything one can do to improve the efficiency of healthcare delivery frees up more money to spend on other programs as well. Or to spend on, you know, getting even better health.
You will hear about Finkelstein’s research on a Medicaid expansion plan in Oregon. While there was no RCT attached to this project, Oregon did use a lottery to determine who would and wouldn’t receive healthcare coverage, so the effect was essentially the same. Finkelstein and her colleagues looked into how this new supply of healthcare coverage affected clinical outcomes, emergency-room use, and employment. (Perhaps not surprisingly, liberals and conservatives all leaped at the chance to cherry-pick and spin these findings. Here, for instance, are left and right views of the findings on clinical outcomes; and left and right views of the finding that Medicaid led to a rise, not the suspected fall, in ER visits.)
We also talk about an RCT that Finkelstein and J-PAL are currently working on with a New Jersey group called the Camden Coalition of Healthcare Providers. Its focus is low-income “super utilizers,” the kind of patients who might show up in ERs dozens or even hundreds of times a year. The mission is to help them get better treatment while also cutting through some of the grotesquely inflated costs that come with modern healthcare. The Coalition’s executive director and founder is Jeffrey Brenner, a family doctor, healthcare maverick, and MacArthur “genius” who has strong and bracing views on the medical business model:
BRENNER: So we learned that 1 percent of the patients is 30 percent of the payments to the hospitals, and that 5 percent of the patients is about 50 percent of the payments to the hospital. So a very small sliver of patients are driving all of the revenues to the system. … And you know, the question really is this the fault of the patients or is this a system failure? And I think our journey over the last couple of years has really demonstrated to use that it’s a system failure and that we could be doing much, much better for these patients.
The conversation with Brenner was so fascinating that we will put out a follow-up episode next week that continues some of the themes he raises here. We’ll look into why Americans are consuming more and more healthcare; whether all that extra care is actually improving outcomes; and what happens when a significant portion of American cardiologists go away at the same time to a medical conference. Do you think there is a huge increase in heart deaths during their absence — or maybe, just maybe, the opposite?