How Do We Know What Really Works in Healthcare? Full Transcript

This is a transcript of the Freakonomics Radio podcast “How Do We Know What Really Works in Healthcare?

[MUSIC: Jessica Lurie, “Pudding” (from !Zipa Buka!)]

Amy FINKELSTEIN: My parents are both biologists and I did not follow their route. I didn’t go into the natural sciences.

That’s Amy Finkelstein.

FINKELSTEIN: In college, I was studying political science and most of my work was qualitative. I’d bring home a term paper that I was proud of and I’d have made some argument in it. My father would read it and say, “Well, that’s nice. But you could’ve just as easily argued the other side.”

“You could’ve just as easily argued the other side.” In a family of researchers, of empirical thinkers, those were fighting words. Amy Finkelstein eventually came around to her father’s view. Today, she’s as impatient with a certain kind of argument as he was:

FINKELSTEIN: If I talk to a friend of mine, for example, who’s a law professor, they’ll often describe their research as, “I’m trying to make the following argument.” I never hear my fellow empirical economists ever describing their research as, “I’m trying to make the following argument.”

That’s right, Finkelstein wound up entering the dismal science.

FINKELSTEIN: I’m a professor of economics at M.I.T. and I’m one of the scientific directors of J-PAL North America.

We’ll explain J-PAL later. But first, how Finkelstein and her fellow economists do describe their research:

FINKELSTEIN: They say, “I’m trying to figure out something.” Right? “I’m trying to figure out what is the effect of health insurance.” Or, “what is the effect of this hospital discharge program.” Ultimately, the more we can learn from data, the more we can figure out what’s going on, rather than just make rhetorical arguments that may or may not win the day because you’re a clever debater.

That sounds good, right? “Let the data do the talking.” That is essentially the gospel we preach on this program every week. But when you’re trying to answer certain tough questions, in the real world, the data you need aren’t necessarily just sitting there in a neat little pile for you to gather up. So what are you supposed to do about it? We’ll tell you what to do about it right after the — here, sing it with me …


*      *      *

[MUSIC: Stinky and the Rabbits, “Sunny Beach” (from Small Thin But Soft)]

Stephen J. DUBNER: Okay, so Levitt, how important, in your work as an economist are R.C.T.s, randomized control trials?

Steven LEVITT: Randomized trials have not historically been that big a part of economics. Certainly, 25 years ago, when I was in graduate school, it was barely even thought of as being a part of the toolkit we have. Over the last ten or fifteen years, it’s become increasingly important.

DUBNER: Well, you answered the question [of] how prominent they are but I want to know how valuable they are.

LEVITT: The randomized trial is the very best way to learn about the world around us. That’s for a couple of reasons: one is because randomization [are] your best friend when you’re trying to find causality. Because absent randomization, you always have to tell stories about why what we observe in the world — which are correlations — actually can be mapped into causal relationships. But the beauty of randomization —  if done well and at least in large numbers with large samples — is that because you’ve randomized, on average, you expect the outcomes to be the exactly the same for the treatment group and the control group. Any difference you observe, you can very plausibly attribute to being causal effects of the treatment — an incredibly simple idea, incredibly powerful idea.

There’s no better way of learning about the world than through a randomized trial.

DUBNER: It makes me wonder, Levitt … Your dad is a doctor, a medical researcher, whose concentration is intestinal gas, therefore he’s known as the King of Farts around the world. How influential was it for you as an economist — who now does like to do randomized trials when you can — to grow up in a home where medical randomized trials were the norm? That’s the way he learned what he learned, right?

LEVITT: Yeah, absolutely. My father from almost birth trained me to think about the power of randomized experiments. But what’s so interesting about this — as an eight or ten year old, I understood the power of randomization. What is really telling, though, is when I turned into an economist eventually, it just seemed like, “How am I going to run an experiment on the issues I care about?” I couldn’t really see an obvious way. I thought, “I’ll do the best I can to approximate randomized experiments by looking for natural experiments or the accidental experiments.” All the things that in my early research were the focus of what I did.

It was motivated by the idea that a randomized trial would be wonderful if you could do it, but somehow the limitation that it would be absolutely impossible for me to do randomized trials as an economist, was one that was just fundamental. It felt too binding.

DUBNER: On the other hand, there’s some things that you’ve looked at where it would be impossible to do randomized trials. If you want to look at the impact of abortion on crime or if you want to look at whether imprisonment really deters crime, you can’t let a bunch of prisoners go in one place and lock up a bunch of innocent people in another. You can’t mandate that people get abortions in one place and have no access in another. Therefore, the natural experiment is the best that you can ever hope for, isn’t it?

LEVITT: Absolutely. Given binding constraints, where you can’t run randomized experiments, then the natural experiments that I’ve used are a good second best. Bu a mistake I made, looking back on my own life, was I really came to adopt the view that this accidental experiment methodology was the right way to think about the world. In that, a much better way to think about the world would be, “The first thing I should do every time I come up to a problem is I should say, “Can I somehow manage to run a randomized experiment?” And having failed that, then I should say, “Given that, is there a way to find accidental experimental variation?”” It absolutely isn’t the way I looked at the world.

But then other people came along, John List and others like him. They really changed my view to open the horizons, that like a scientist, an economist could generate data.

[MUSIC: Christopher Norman, “Can’t Let Go” (from Ep 1)]

An economist could generate data. That doesn’t sound like such a radical idea, does it? But, as Steve Levitt points out, it wasn’t until quite recently that economists like John List and others have turned the world into one big econ laboratory. Because you can gather all the found data you want, and analyze it to death, but it can’t necessarily answer every question. Let’s say there’s one elementary school in one state where the kids do really well much better than all the other schools nearby. This also happens to be the only school that gives its kids breakfast every day, in addition to lunch. It’d be tempting to conclude that the good grades at that school are due to the breakfast, and that if you could only serve breakfast at all the other schools, their grades would shoot up too.

But how can you tell for sure? Maybe breakfast is one of 10 things this school does differently or maybe the kids are different, or the parents, or the teachers, or the curriculum. Maybe they’re the only school that plays dodgeball at recess. So how do you find out, how do you isolate the effect of the breakfast? You set up an experiment a randomized controlled trial, or R.C.T., like the ones used in bench science, in drug studies. You take one population, you randomly divide it into groups, and you give some groups a treatment that the others don’t get. Then you can measure whether the treatment group came out any differently than the control group. These days, economists and other researchers are using R.C.T.s to learn more about everything from altruism to dieting to fighting poverty. And that’s where Amy Finkelstein and J-PAL come in.

[MUSIC: Euforquestra, “Obatala” (from Explorations in Afrobeat)]

DUBNER: J-PAL standing for?

FINKELSTEIN: The Abdul Latif Jameel Poverty Action Lab.

DUBNER: Okay. Whose mission is?

FINKELSTEIN: J-PAL is a center at M.I.T. that’s a network of high-quality academics around the world. It’s composed of regional offices. I run J-PAL North America. J-PAL’s mission is to promote and encourage randomized evaluations on important public policy issues and then to disseminate the results of those randomized evaluations to, you know, key actors and decision makers.

J-PAL was established in 2003. For years, its primary focus was overseas. Esther Duflo, an award-winning economist and one of J-PAL’s founders, has helped run many R.C.T.s in India, Kenya, and elsewhere, trying to learn how best to prevent teen pregnancy and anemia and drunk driving; and how to better incentivize nurses and small-business growth and modern farming techniques. In the U.S., Amy Finkelstein and her J-PAL colleagues were interested in health care delivery.

FINKELSTEIN: We take a rather broad view of poverty alleviation. And so anything that improves the efficiency of health care delivery, I think is important for the public for two reasons. First, you know, the poor are disproportionately unhealthy and therefore have the burden of health care relative to less poor people. Also, given that health care spending is currently about a fifth of public sector budgets at the state and federal level, anything one can do to improve the efficiency of health care delivery frees up more money to spend on other programs as well. Or to spend on, you know, getting even better health.

Along with her colleague Sarah Taubman, Finkelstein found that randomized trials are very much the norm in medicine in drug studies, medical treatment, and so on. But when it came to health care delivery how those drugs and treatments are actually consumed by patients R.C.T.s were quite rare.

FINKELSTEIN: So that, to us, was a striking disparity. While we don’t think everything can or should be randomized, we would like randomized evaluations to be closer to the norm rather than the exception in health care delivery the way they are in medical research.

DUBNER: A lot of people, when they hear you or other economists talk about, “If you take 5,000 people, you can randomize them. Give half of them insurance and the other half not — they’re your control group — and then you measure depression and job outcome.” That’s great and that’s exciting. There are some people out there who hear that and say, “Man, no. I don’t want any part of that. I don’t want to hear about it. It is wrong to do that experiment in a realm where people’s livelihoods, health and so on are involved.” Can you talk about that, I don’t know what you call it moral or…



FINKELSTEIN: The ethics of randomized evaluation…

DUBNER: Yeah, yeah.

FINKELSTEIN: Let me start by telling you an interesting story, which is the Oregon Health Insurance Experiment. The Oregon Health Insurance Experiment was not something that came about because a bunch of researchers dreamed it up and convinced the state to do it. We found out about the lottery after the state had decided to do it. They decided to do it, not for research reasons — although it turned out to be wonderful for research — but for fairness reasons.

[MUSIC: Pearl Django, “Saskia” (from Modern Times)]

Like many states, Oregon was expanding its Medicaid program. In addition to covering the standard patients like poor children and pregnant women and the disabled it was now offering free or low-cost health care coverage to some able-bodied low-income adults.

FINKELSTEIN: At the start of 2008, the state of Oregon realized that they had enough money in their next budget cycle to cover an additional 10,000 individuals. However, they also correctly realized that if they just reopened the program to low-income, uninsured, able-bodied adults in Oregon that there were many more than 10,000 who would be eligible. In fact, we’ve subsequently estimated that about 200,000 such individuals would have been eligible for the program. So they decided the fairest thing to do was actually assign eligibility for Medicaid based on a lottery. They ran a huge public relations campaign for two months. They got about 75,000 individuals to sign up. Then they literally ran a random number program to determine which of them could apply for Medicaid and which couldn’t.

[MUSIC: Ruby Velle & The Soulphonics, “The Man Says” (from It’s About Time)]

This gave Finkelstein a great and rare opportunity to learn what really happens when a bunch of poor, uninsured people get access to health insurance. Because now there were no confounding factors. What you had were two statistically equivalent groups of people those who were eligible for Medicaid and won the lottery and got it, and those who were eligible for Medicaid and lost the lottery and didn’t get it. So what did Finkelstein learn? That’s coming up on Freakonomics Radio along with another health care project she’s working on:

FINKELSTEIN: It sounds so simple and yet this can really potentially make the difference between ending up back in the hospital and not.

And even if you don’t much care about people generally, why you might want to care about improving health care delivery:

Jeffrey BRENNER: As we’ve gone around the country we have heard many stories of patients up to $1 million a year bouncing around the health care system for pretty absurd reasons.

*      *      *

[MUSIC: The Amity Front, “The Beat(Nik)” (from Highway Bound)]

The economist Amy Finkelstein believes in the power of a randomized trial to figure out things like what happens when a bunch of poor, previously uninsured people suddenly get health insurance. She stumbled upon a gold mine: a Medicaid expansion in Oregon that had been randomized by lottery. Finkelstein and her colleagues began analyzing the data; and they started to release their findings as the Obamacare debate was raging.

FINKELSTEIN: One of the results that came out that surprised a lot of people is that when you cover the low-income uninsured people with Medicaid, they use the emergency room more rather than less.

DUBNER: Which is very counter to everyone’s assumption, I guess. Was it counter to even yours going in?

FINKELSTEIN: Well, I didn’t know what I would find and that’s one of the fun things about doing research. You’re right, everyone assumed emergency room use would go down. In fact, one of the arguments people make for covering the uninsured is to get them out of the expensive emergency room and into primary care clinics and other services. We found that covering the uninsured increased emergency room use by 40%.

DUBNER: That is so fascinating, right? What does that say — maybe you don’t have an answer to this — about the underlying assumption of why we thought that uninsured people, or how often they thought they were using the emergency room already? In other words, do you start to think “Maybe people who are uninsured don’t even think about access to the emergency room as a right?”

FINKELSTEIN: That’s one possibility, right? Because it is the case that the law requires hospitals to treat people who show up in the emergency room, right? But they can charge them if they’re uninsured. You do see the uninsured using the emergency room, you just see them using it more when they get insurance. The premise for why covering the uninsured with Medicaid would get them out of the emergency room is that Medicaid makes the doctor’s office free and so now people would go to the doctor instead of going to the emergency room. But what you forget is that Medicaid also makes the emergency room free.

To go to your point — while you’re allowed to go to the emergency room and you have to be treated even if you don’t have insurance, you can be charged for it after the fact, which means you either have to pay it out of pocket or you may have collection agencies harassing you. That can affect your credit rating or just be unpleasant.

[MUSIC: Chauncey Canfield, “Incandescence” (from The Long Walk Around)]

Obamacare opponents seized on the evidence that the expansion of Medicaid led to substantially more emergency-room visits:

FOX NEWS: Well, there’s a new study out that says that the expansion of Medicaid under Obamacare will actually increase E.R. visits. Of course, the Obama Administration repeatedly claiming that the health care law would actually reduce those costs.

Another Finkelstein paper found, and I quote: “no significant effect of Medicaid coverage on the prevalence or diagnosis of hypertension or high cholesterol levels or on the use of medication for these conditions.”

Steve DOOCY in a clip from Fox & Friends: Bad news for Democrats who support Obamacare. A study out of Oregon suggests that the government’s health care push for all does not necessarily make people any healthier.

FINKELSTEIN: On the other hand, another set of results we found is that, again, through this randomized evaluation covering low-income people with Medicaid reduces depression. We got about a 30% reduction in depression. Also, and perhaps most interesting to me as an economist, covering the uninsured with Medicaid improves their financial stability. It reduces the risk of catastrophic, out-of-pocket expenditures to virtually zero. It’s providing important financial security, as well.

As you might have guessed, Obamacare supporters played up this finding:

Chris HAYES in a clip from All In: Can I just also say the thing about the Oregon study — which has been litigated a tremendous amount, and I don’t want to litigate here. The one thing, the big takeaway, was that people were much happier and much less depressed.

Avik ROY in a clip from All InBecause they were more financially secure!

Chris HAYES in a clip from All In:Right! And you know what I say? That’s a huge boon!

So you’ve got both sides in the Medicaid-expansion debate finding evidence in Finkelstein’s study that supports their position, which may not sound like much of an improvement over the standard political shouting match but, remember, at least they’re talking about evidence here, rather than hunch or pure ideology. And that, says Steve Levitt, is what makes a randomized trial so useful.

LEVITT: Randomized trials can tell you — in the setting you’re in — whether or not a change that you make makes outcomes better or worse. Okay? It tells you that. But that’s only one of the things it does. It also is an incredibly effective way of changing the process. Even if -you could hire a consulting firm to carry out an analysis which would tell you with exactly as much certainty that you should do something differently, it’s harder to get people to actually change when they can use arguments like, “I think the analysis might not have been done right.” Or “That analysis was paid for by the doctors and the doctors don’t like the nurses and so it can’t be believed.” It really is the ex ante agreement that, “If the study comes out like this in a randomized trial, we will agree to change.”

That is incredibly powerful in organizations.

So people are still free to argue their positions. But as the researcher, you can leave the arguments to the arguers, to the people with horses in the race. As the researcher, you get to prove a real causal relationship that people were only guessing at before. And that is how you start to actually solve problems  by learning the real cause, which is especially valuable in a realm as complicated and as expensive  as health care.

[MUSIC: Tango Alpha Tango, “All Mine Blues” (from Black Cloud)]

BRENNER: I’m going to tell you about a patient who is up in Trenton, New Jersey.

That’s Jeffrey Brenner.

BRENNER: I’m a family doctor. I’m also the executive director and founder of the Camden Coalition of health care providers in Camden, New Jersey.

He’s also a MacArthur “genius” grant winner and a medical iconoclast, but we’ll get there later.

BRENNER: We have a sister organization, a similar nonprofit doing work up there. They found a patient who had been 450 times in a single year.

DUBNER: To what? To an E.R., to a hospital?

BRENNER: To the local emergency rooms and hospitals. Occasionally a patient was admitted as well.

DUBNER: Wow, so I don’t even understand how that’s possible. That’s more than once a day.

BRENNER: Yep, more than once a day to the local hospitals and emergency rooms.

DUBNER: How do you do that?

BRENNER: It’s a lot of work. It’s not easy.

DUBNER: Okay, so 450 times.

BRENNER: This was a woman who was in her 50s who had severe alcoholism, mental illness, significant medical issues as well, and was literally bouncing form emergency room to emergency room, hospital to hospital. She was also homeless as well and very cut off from her family.

DUBNER: At the risk of sounding cruel, what did it cost for those 450 visits? And who paid for that?

BRENNER: Let’s take $500 to $1,000 a visit. No matter how you do the math on that, that’s pretty expensive, right? This is could be $100,000, $200,000 care. As we’ve gone around the country, we have heard many stories of patients up to $1 million a year bouncing around the health care system for pretty absurd reasons.

DUBNER: Can you talk for a minute about what you began to learn about those — as you call them, “super utilizers” — people who use that health care so much?

BRENNER: Yep. We learned that 1 percent of the patients [are] 30 percent of the payments to the hospitals and that 5 percent of the patients [are] about 50 percent of the payments to the hospitals. A very small sliver of patients are driving all of the revenues of the system. This top 1 percent, we use the term super utilizers. The question really is, “Is this the fault of the patients or is this a system failure?” Our journey over the last couple of years has really demonstrated to use that it’s a system failure and that we could be doing much better for these patients.

[MUSIC: Black Oil Brothers, “Trust Me.” (from The High Road to Ruin)]

Brenner did not have this understanding of the problem from the outset. He came to it through a side door.

BRENNER: About 10 years ago, the city of Camden became the most dangerous city in the country. I lived here and it was [a] terribly frightening time. My wife and I had had a baby here. Daily, I would have patients coming into my practice who’d been hurt in crimes. They wouldn’t even call the police department because the police wouldn’t even come or, if they did, they weren’t helpful. I had patients telling me horrific things about being beaten up by the police department. The police department was in free-fall and the city was in free-fall. It was a very frightening time to be here. It got to [a] point that the state took over the Camden police department, put it in receivership. I got appointed as one of two city residents to be on a police reform commission and got a chance to be behind closed doors where state officials are having all these really intense conversations about the police department and just realizing how this thing was really melting down.

They brought in a whole crew of people who had helped reform the New York City police department under Bratton. They took me under their wing and started teaching me what had been done up there; learned a lot about Compstat, also all the management theory and data theory behind it, and was just in awe of it.

[MUSIC: Dave Pezzner, “Closed Cover” (from Dave Pezzner)]

Brenner was exposed to a data-driven approach to policing that potentially found trouble before it happened, by identifying patterns in the data. By identifying hot spots. Brenner began to wonder: if hot spotting could help the police, why not him? At the very least, there was one big parallel: just as a relatively tiny number of criminals cause a whole lot of trouble and expense for everyone, a relatively tiny number of sick people were gunking up the health care delivery system. He got hold of some hospital data:

BRENNER: It was a stunning data set. We found out incredible things about the rates of falls in the city, about assaults in the city that concurred with our hypothesis, which is that people are being hurt at an incredible rate in the city far higher than even the reported police rate. We expanded the data set. After this initial data analysis, we were really turned on by this idea of mapping, graphing, and charting the data. We would sit for hours and hours, cutting the data up in different ways. My patients were in the data. I recognized names in the data of people I knew and was taking care of. I was just shocked by how much money was being made over on the hospital side of this care, and how often people were going back over and over.

The number one reason to go to the hospital in Camden was head colds. Number two was viral infection. Number three was sore throat. There were 12,000 visits for head colds over five years. That’s shocking.

DUBNER: What’s a head cold visit cost?

BRENNER: It can cost $300, $500, $800, $1,000. It’s ridiculous.

As Brenner saw it, the problem was twofold at the very least. One: a certain kind of patient was consuming a ton of health care treatment but still not getting healthy. And two: the health care delivery system, the hospitals in particular, were set up to profit from these super utilizers and profit they did, at the expense of the taxpayers. So those were the problems. What was the solution? In 2007, Brenner got his group, the Camden Coalition, to start focusing on super utilizers. Five years later, they set up a program called Link2Care.

BRENNER: It includes all of the local stakeholders — so primary-care providers, hospitals, long-term care, behavioral health, addiction, the homeless shelter. My feeling was this was a game-theory problem of, “How do I get people to collaborate instead of compete?” Competing over market share of homeless, medically complex, addicted patients is not a great way for society to exist and not a great set of rules for the health care system to function by.

So Link2Care would be an aggressive, proactive plan meant to help these super utilizers by making sure they got the attention they needed, whatever kind of attention that might be. Which sounds potentially wonderful but also potentially expensive, and, who knows, potentially not wonderful. Furthermore, how would you measure the success of the program? By randomizing it, of course. Jeffrey Brenner and his people got together with Amy Finkelstein and her people to set up Link2Care as a randomized controlled trial.

[MUSIC: Roses & Revolutions, “Take Me With You” (from Roses & Revolutions)]

BRENNER: We have real-time data feeding in from all the local hospitals of demographics, utilization, labs, radiology and hospital-discharge summaries.

FINKELSTEIN: They don’t have unlimited resources. Also, they’re trying to figure out the kinks as they go and learn from their experiences as they scale up.

BRENNER: Every morning we’re alerted to who’s been admitted to the local hospitals. We log in remotely to the electronic health records system of each hospital. We do a triaging to identify patients who are medically complex, socially complex and have been readmitted more than two times in six months. Then, we have teams located in the hospital who go right to the bedside, explain the program to the patient.

FINKELSTEIN: What was really striking is you saw people who were quite ill and quite lonely. They were usually alone in the hospital and they were extremely eager to talk to the people from Dr. Brenner’s team who came in, who are people from the community, who are sensitive and understand the needs. They were just very happy to have someone listen to their story, and hear —much more than a physician can in the brief health care visit — what was going on in their lives.

BRENNER: If they’re interested, we consent them. We then walk out of the room and it’s a randomized control trial. We’re testing the intervention and we hit the random button. They randomly get assigned to either the intervention, which is either 90 to 120 days of care coordination or to default care, which is just the normal discharge process. If they end up in the intervention, we have a whole multidisciplinary team made up of community health workers, social workers, nurses, AmeriCorps volunteers who go right to the bedside and do a care plan. We spend a whole lot of time really trying to get to know the patient and understand what their needs are and where they want to see their life go.

We then go to their house within 72 hours, go with them to their primary care appointments, with them to their specialty appointments — if they’re homeless, help them get housing.

FINKELSTEIN: You see these people, often with unstable housing, family, and employment situations who are coping with really serious illnesses and pages of instructions from having left the hospital, bags of medication from previously prescribed and trying to keep it all straight. They’re supposed to be taking six different things a day, some with food, some without food. Just seeing the health coach — it’s something so simple, sit with them, and very calmly and methodically go through, “Let’s see what the list of instructions are you’re supposed to take. Can you find me that medication? Let’s look at it. Is it expired? Let’s put it here. And let’s take them.”

BRENNER: If they have outstanding warrants, help them settle them. If they’ve got social service needs, if they need help signing up for insurance, whatever the issues are across 15 different domains, we work with them for about 90 to 120 days on independence, autonomy, and self-efficacy. And then graduate them.

We tagged along one day this winter on a Link2Care visit in Camden. It is one of the poorest cities in the U.S. Jeneen Skinner is a nurse and a nurse care coordinator with the Camden Coalition. She has now driven up to a housing complex with yellow, two-story buildings.

[MUSIC: Kenneth Ward Lovell Jr., “Shine a Light” (from Swamp Blues)]

VOICE: Hi, Susan. How are you?

Susan COLEMAN: Good. It’s cold out there.

VOICE: It’s freezing out there! How are you feeling today?

COLEMAN: Good. A little sleepy (chuckles). Yeah, I’m tired. My name is Susan Coleman and we’re at my home in Camden.

Jeneen SKINNER: Did the cardiologist change that or did the doctor change that?

COLEMAN: The doctor.


COLEMAN: I’m a diabetic — and high blood pressure, congestive heart failure. They help arrange my appointments, get me to the right doctors that I need to go to, monitor my blood pressure and my sugars, tell me what to eat and stuff like that. The treatment is very helpful and they feel like a family.

SKINNER: You mind if I check your blood pressure?

COLEMAN: Not at all.

SKINNER: Have you been checking it with your blood pressure machine?

COLEMAN: No. I know I should…

SKINNER: Did they take the other one? Did the nurses take the other — there was another one, right?

COLEMAN: Yeah, they took that.

SKINNER: They took that one already? Okay.

Jeffrey Brenner again:

BRENNER: This does not work for every patient. Not everyone is ready to change. Not everyone’s ready to make different choices. But we’ve found that this works for many patients.

But for how many patients? And at what cost? Might there be better alternatives? These are the kind of questions that Brenner, and Amy Finkelstein, and all their colleagues will try to answer through the data.

BRENNER: Yep, so we’re looking to randomize 800 patients. We have randomized about 240 so far. It has been an enormous amount of work.

FINKELSTEIN: We’re looking at hospital readmissions and other things that are regularly collected in health care data. If we don’t find an effect on readmissions, we want to know, “Is that because they’re not actually achieving the intermediate goals that they think are important? Or is it because they’re achieving them and they turned out not to be important?”

BRENNER: Amy’s team has been so helpful. There’s no way that we would have been able to launch something like this ourselves.

FINKELSTEIN: We also want to look at prescription drug use.

BRENNER: We’ve wanted to do this for a long time, but we just didn’t have the sophistication, the structure, and the knowhow to be able to do it.

DUBNER: What do you know so far or is it too early to say anything?

BRENNER: It’s really early. The data’s very noisy early on. There are positive trends, but we have to wait until it reaches statistical significance.

DUBNER: How long will it take for these 800 patients to go through the system then?

BRENNER: One of our biggest challenges is that our federal grant runs out sometime in the fall and maybe winter. We’re a jigsaw puzzle of 28 different sources of funding. There’s a special place in heaven for delivering better care at lower cost. But there’s no business model for it. There’s certainly no business model for how to prove it or do research on it. I don’t want to say we’re holding bake sales but we’re in a race against time to get enough patients through the study in order to get to the statistical significance. The other problem is that even if we prove this with the perfect New England Journal or JAMA article, that still doesn’t change Medicaid policy.

DUBNER: I would have to think of all the players involved here, Medicaid, Medicare are those who are most eager to find real ways to cut cost for this kind of care, yeah?

BRENNER: I don’t know about that —

DUBNER: Oh no, tell me — why would that not be the case?

BRENNER: There’s a really well-funded randomized control trial. It’s the largest randomized controlled trial in my field that’s ever been done. It was 1,700 elderly Medicare patients over a 10-year period. Iit was actually part of a contest that was launched as part of the Balanced Budget Act about 12, 14 years ago. Congress got tired of all these disease-management companies coming and saying they can save Medicare money. They launched a contest. They hired Mathematica to be the judge. They were randomly given a group of patients. Then there was a random control group that just got normal, default care in the Medicare system. There were about 14 different entities across the country that applied and got into it.

Within a year the Mathematica and the Federal Government pulled the plug on it. All but three of them had the plug pulled on them. It turned out that they had driven costs up. The three that were allowed to move forward were messy boots on the ground, difficult programs to run. They were nurses out in the field banging on doors. Two of them dropped out because it was so hard to do. There was only one still left standing in Doylestown, Pennsylvania. They are a community nonprofit a lot like us that works with all the local providers and hospitals to do care coordination for very complex Medicare patients. They have 10 years’ worth of data that’s been published in journals and published in the Congressional Record.

Basically they reduced the death rate by 25 percent. They didn’t just save money, they reduced the death rate just by having a nurse come out to your house every week or every other week in a highly structured, well-organized intervention to make sure that your mom or grandmother were having coordinated services, wellness interventions, exercise, fall prevention. People over 85 with complex chronic illnesses, they had a 49 percent reduction in the death rate. This is sustained over 10 years. They also had a reduction of about, not for the middle cohort, but the most complex patients. The middle cohort they broke even. The most complex cohort, they reduced costs and hospitalization by about 20 to 30 percent. That’s a stunning accomplishment.

I don’t understand, given the evidence that this project works, why Medicare hasn’t scaled it. I’m not confident in the ability of government to make the best decision on the best evidence the way they should.

DUBNER: One cynical guess might be that when you’ve got a budget baked into your universe — whether it’s a governmental universe, a for-profit or not-for-profit universe — that when there’s something that threatens to diminish that budget by 10, 20, 40 percent, even though it makes a lot of sense for a lot of people involved, taxpayers for instance, it might not make sense for you, right?

BRENNER: Absolutely. Frankly, Medicare’s comment was that it’s really hard to do. We’re not sure we could scale it. Well, we fucking scaled open heart surgery. We scaled separating Siamese twins. We scaled transplanting hearts and lungs, curing complex cancers. We’re sequencing the human genome. You’re telling me we can’t have a nurse go out and check on your mom or grandmother in a highly organized, well-structured, well-trained intervention for which someone’s already doing it for hundreds and hundreds of patients every day?

DUBNER: It sounds as though even if you, all your colleagues, Amy Finkelstein and all her colleagues come to the conclusion that this project that you’re working on is both effective and very cost-effective, you don’t sound very optimistic that it will move the needle in the direction at least that you want it to move.

BRENNER: No. At this juncture in life, based on the medical history and social history that I’ve read — that evidence alone is not enough to fix a complicated problem. You need to have the combination of evidence and advocacy. At some point the American public needs to stand up and say we’re sick and tired of being cut, scanned, zapped and hospitalized in a $2.8 trillion industry that’s running out of control, and is not taking good care of us.

[MUSIC: The Diplomats of Solid Sound, “My Place BBQ” (from Let’s Cool One)]

At the beginning of this episode, we proposed the idea that the R.C.T., the randomized controlled trial, is a valuable tool in sorting out hard problems, including health care delivery. I think our conversation today proved that value – but that rather narrow issue led us into a much larger argument. Jeffrey Brenner — an M.D., a health care delivery innovator, an R.C.T. believer — has, as you just heard, very strong feelings about how our health care system is set up and who it’s really serving. And so next week, we’ll continue this conversation, push it even further in that direction. You’ll hear more from Brenner — about what it would take to truly revolutionize health care delivery:

BRENNER: No one in the C.I.A. could imagine Mubarak ever being out of power, right? Complex, adaptive systems go through state changes. They do it in very complex and unpredictable ways where one day they’re one way and the next day there’s been a dramatic shift.

We’ll also talk about whether the central tenet of medicine should perhaps be “less is more.”

Anupam JENA: The perception of health care is that by doing more we can improve health. What we need to recognize is that so much of health care, so many of the clinical decisions that we make operate in this gray zone. It’s not black and white. It could very well be the case that in the gray, less may be more.

That’s coming up next time, on Freakonomics Radio. If you like what you hear, please tell your friends about this free podcast. You can even tell your enemies. Spread the word on Twitter, Facebook. Feel free to rate or review our show on Apple Podcasts or wherever you get your podcasts.

This is a transcript of the Freakonomics Radio podcast “How Do We Know What Really Works in Healthcare?