How Can You Choose the Best Doctor?

I was chatting with my producer, Jessica, recently and we got to talking about her daughter, who is 12 and wants to remove a small birthmark on her cheek. So now, Jessica’s researching options, trying to figure out where to take her daughter. Should she go with the plastic surgeon at the local hospital, who trained at a foreign medical school and takes her insurance? Or maybe she should try the cosmetic surgeon in the nearest city, about an hour and a half away, who did his residency at the fancy Cleveland Clinic and specializes in face lifts and tummy tucks. Anyway, it’s a big decision, and after all, this is her daughter’s face, so, any scar could be there for life. So, what should Jessica do? You know, maybe there’s some other, better doctor if she just does a bit more research. Asks a few more questions.

Jessica’s conundrum got me thinking. We all have some version of this struggle to choose a doctor. Sometimes it’s about finding a new primary care physician in the town we’ve just moved to. Sometimes it’s about treating a life-threatening cancer. Whatever the case is, it’s important. We’re talking about our health here. Our bodies. Our minds. Our lives.

Well, luckily, a lot of researchers have had the same questions. And while some of their findings might confirm what we’d expect, not all of them do.

*          *          *

From the Freakonomics Radio Network, this is Freakonomics, M.D.

*          *          *

I’m Bapu Jena. I’m an economist and I’m also a medical doctor. Each episode, I dissect an interesting question at the sweet spot between health and economics.

Today: when it comes to deciding on a doctor, what matters most? Where they went to medical school? Where they did their residency training? The hospital they work at? Does age matter? How do you know if you’re choosing a good doctor?

Let’s think about one possible way to find the best doctor: figure out where they did their residency. After medical school, all doctors complete at least one residency. It could range from three years to sometimes seven years. It’s a big deal. It’s intense, on-the-job training in the area you want to specialize in — internal medicine, pediatrics, radiology, surgery. Residency is a lot of hours, nowadays it’s often up to 80 hours a week, and it used to be a lot more. I spent three years in residency at Mass General Hospital in Boston, where I still work now — at least when I’m not recording podcasts and doing research.

Anyway, residency programs can be really competitive to get into. And because of this, we may tend to think that doing a residency at a place like Johns Hopkins or the Mayo Clinic or some other prestigious institution may somehow make a person a better doctor. Or maybe those institutions, because they’re so selective, just pick the very best doctors fresh out of medical school and, so, training at one of those places might just be a marker of a doctor’s quality. People on both ends of the stethoscope think this way.

But is that justified?

A while back, David Asch, a medical and health policy professor at the University of Pennsylvania, and a colleague, Sean Nicholson, an economist now at Cornell, started wondering if there was any way to know whether one residency program was better than another.

David ASCH: We came upon the idea that we should evaluate medical training by its ability to produce doctors who deliver good care or good outcomes to patients. And we were pretty enthusiastic about that idea. It seems pretty logical, but I will say that at the time a lot of people had not been thinking along those lines. In most cases, I would say that people were evaluating residency programs by reputation, something I’d probably have to put into air quotes because I’m not really sure how reputation is created or whether it has any meaning compared to what we fundamentally care about, which is good clinical care.

That might sound kind of crazy, but at the time, about 15 years ago, it wasn’t. You see, there wasn’t any good data on the outcomes of physicians who’d trained in quote unquote prestigious residency programs. Sure, their teachers may have been world-class scientists and doctors, or authors of leading medical textbooks. And maybe the residents themselves had studied at prestigious colleges or medical schools or had great test scores. But all of that was really about “reputation,” or our perception of quality, and not about hard outcomes on whether the doctors who were training in the most prestigious programs ended up being any better doctors.

So, the big question is: is that reputation valid? When it comes to finding the best doctor, should we care about where they trained? It turns out it’s really hard to dig under the surface here, for lots of reasons.

Doctors treat lots of different conditions. A general internist, for example, sees patients with everything from diabetes to heart failure. So, which outcomes would you even pay attention to if you wanted to figure out whether or not that doctor was a good one? And if you’re trying to compare doctors, how would you make sure that one doctor didn’t have patients that were just healthier to begin with, making their outcomes look better.

All of these questions make it really hard to figure out whether the residency program matters when it comes to the quality of care that a doctor provides. But David and his colleagues realized that if they focused on one narrow field of medicine, they might get some answers.

ASCH: And so, we’ve chose obstetrics as the field to try this out in, because we thought that would be an example of a clinical area where, basically you have one job, which is to make sure that pregnancies do well and babies are delivered well, and that the mothers do well as well.

David, Sean, and their colleagues, focused specifically on what happened to the mothers.

ASCH: We use discharge data from New York state and Florida, two large and very fertile states. And the only data we had were the data about the maternal outcomes — what happens to the mothers.

The researchers looked at issues like lacerations, hemorrhages, infections, problems with blood clots, and, whether a C-section was performed and if so whether any problems occurred during the operation. Their analysis included about 5,000,000 deliveries between 1992 and 2007 performed by physicians who’d completed an O.B.-G.Y.N. residency and had performed at least one hundred deliveries. It was a massive amount of data that included a lot of information about the doctors.

ASCH: So, we knew for example, where they went to medical school and where they did their residency training. We knew how old they were. We knew all sorts of things about them. And we knew a lot about the medical complications of the women or about the pregnancy.

I mentioned earlier that one of the big challenges in comparing one doctor to another is: how do you compare an O.B. who treats higher-risk pregnancies with one who treats lower-risk pregnancies? You need detailed data on patients to account for those sorts of differences. You also need data on the outcomes: the complications that arise from childbirth. Without both pieces of data, you can’t credibly compare the quality of doctors, which means that you can’t say anything about the quality of the residency programs that trained them.

Now, lucky for David and his colleagues, they did have both those sets of data.

ASCH: And so, we were really in a good position to try to answer the question of: how much do the outcomes that women achieve depend on the residency program of the obstetrician who was delivering them? Could we say that someone who went to residency program “X” consistently outperforms someone who graduated from residency program “Y”, and we had a lot of data to address that question.

So, I’m curious, what’s your guess? You think there’s a relationship between where an O.B. trained and the outcomes of the mothers they treat?

ASCH: Well, it turns out A residency program matters.

Now I hate to disappoint you, but the researchers didn’t name names. We don’t know which programs produced O.B.s with the best patient outcomes. But David did share that the top quintile included several “prestigious” programs. So, there was some overlap between a hospital’s reputation and the performance of doctors who trained at that hospital’s residency program. But not every program in that group was a flashy one. Unfortunately, that was all the information we could get out of David. I know, not very juicy.

But to be fair, though, the purpose of the study wasn’t about helping individuals choose an obstetrician. The point was to actually answer a broader scientific question: does where a doctor trains matter for the care they provide? And as importantly, to develop a statistical approach that would allow researchers to answer that question: to show the role that a residency program can play when it comes to patient care. David and his colleagues showed that where a doctor trained could matter.

Now, I want to dig in a little deeper. The researchers focused on the roughly 12 percent of deliveries in the study that had some sort of complications. They counted up the complications according to which residency program the obstetricians had attended. Then they ranked the residencies in order of total complications. Using that list, they then grouped the programs into five categories, or quintiles, ranging from the highest to lowest in terms of maternal complications.

ASCH: residency programs that were in the worst-performing quintile had a complication rate of 13.6 percent. The patients who are cared for by obstetricians who trained in the programs with the highest quality had a complication rate of 10.3 percent. So, the difference between that best-performing quintile and that worst-performing quintile was around 3.3 percent.

Now, that might not sound like that big of a difference. But let’s look at this in terms of actual numbers. The researchers included nearly 5,000,000 deliveries in their study. So, if you averaged that out to 1,000,000 deliveries per quintile, then the worst-performing group would have about 130,000 deliveries with maternal complications, and the best-performing group would have about 100,000. That’s about 30,000 more deliveries with some kind of problem — could be mild, could be severe — in the worst-performing group. The exact numbers in the study are slightly different, but this shows you the range we’re talking about here.

ASCH: A difference of 3.3 absolute percentage points is a big deal given roughly 4,000,000 babies being delivered each year.

So, how do we know that the benefit of attending a top residency program came from the education or training that doctors received during their time there? Maybe the best medical students all gravitate to the same programs. In that case, the lower complication rates that their obstetricians have later on is simply due to the doctors’ own smarts, not something specific about how they were trained. Well, David and his colleagues had the same concern. So, how did they sort that out?

ASCH: What we did was we looked at board scores. And when we adjusted for the standardized test results, our results were unchanged.

Standardized test scores are obviously not a perfect measure of a doctor’s knowledge, or more importantly, what kind of doctor they’ll become, but they’re still useful. The fact that accounting for these test scores didn’t change the findings means we can conclude the differences in how doctors ended up performing really was due to the residency program.

And by the way, I should note that the researchers also didn’t spot any differences between doctors who trained at better or worse residency programs in terms of the women that they treated. For example, the lower complication rate group wasn’t full of healthier women who were extra committed to prenatal vitamins and pregnancy diets and birth plans.

As David mentioned, the nature of obstetrics lent itself to this proof-of-concept study. But the same phenomenon seems to hold true for some other fields of medicine.

ASCH: Other researchers have used the exact same principles, for example, in general surgery and looked at post-operative complications and length of stay and other measures of quality and have found that residency program matters also in the production of general surgeons. And I think if you look carefully enough, you’d probably find that it matters in other conditions as well.

Now, the importance of the residency program wasn’t the only factor that the researchers could study — remember they had lots of other data about the doctors.

ASCH: We asked the question, does experience matter? And for experience, we just used years of experience: how many years have they been practicing since they completed residency program?

This question wasn’t addressing how many babies they’d actually helped deliver, only how many years they’d been practicing. On first thought, if you’re like me, you might consider that doctors get better for a while after they’re done with their training, but at some point, they peak, and then their skills decline. I mean, that would make sense — you get a little stuck in your ways, you find it hard to keep up with the latest medical information, and at some point, your mind might not even be as sharp. That’s what David thought, too.

ASCH: That is not what we found at all. We found that experience matters hugely, and it continues to matter for decades. Now, I’m not an obstetrician, but I sort of don’t understand what you’re learning in your third decade of delivering babies that you didn’t learn in the first two decades, but the results were totally robust. It was a very, very powerful and profound, and frankly, counterintuitive result. A lot of people think that, eventually, doctors get worse as they get older. We found that in obstetrics, they just get better and better and never stop getting better. What that would suggest is that if you’re a woman looking for an obstetrician, you should pick the absolute oldest one you can find.

But before you go looking for the oldest doctor you can find, you know, hold on for a second. Turns out that what David and his colleagues found in obstetrics may not apply to all fields of medicine. Let me give you an example. A few years ago, we looked at a similar question among internists who provide care to hospitalized patients. These doctors are called hospitalists and nowadays if you’re hospitalized with a general medical condition like pneumonia or heart failure, chances are you’ll be treated by one of these types of doctors. So, my colleagues and I looked at whether the age of a hospitalist physician was tied to outcomes among elderly patients. We looked at about 730,000 hospital admissions by nearly 19,000 physicians and we found that patients treated by older physicians had a higher rate of mortality within a month of their admission date — it’s called “30-day mortality” — compared to patients treated by younger physicians. More patients died under the watch of doctors who were age 60 or older. And interestingly, doctors who were just five years out from their training lost the fewest patients. That finding appears to contradict David’s data showing that obstetricians continued to improve even 30 years into their career.

It’s hard to know why our findings differ. The study subjects are, of course, really different — new mothers compared to patients aged 65 and over. Another factor may be the work itself. Obstetrics is a surgical specialty, whereas internal medicine is not, and surgeons use their hands in a way that internists don’t. It’s just a really different approach to the body.

A related study that my colleagues and I did might offer a clue. We looked at the relationship between surgical outcomes and surgeon age for general surgeons and found that surgical outcomes improved over time — similar to what David found — but did deteriorate amongst the oldest surgeons, like those in their 70s.

But back to David’s study: his goal wasn’t to provide healthcare consumers with practical information about what to consider when choosing a doctor. He was primarily interested in whether the quality of training programs may differ and if so, how to show that. But let’s be honest, we’d all like to know which schools trained the best doctors so we could use that information the next time we have to choose a doctor. But what if that potential benefit in care came at a price? How much would you pay for a slightly better outcome? That’s after the break.

*          *          *

Hey there. A quick note before we dive into the second half of this episode. I love getting your feedback — what we’re doing well, things we’ve missed, ideas for new studies to dig into and questions to explore.

Lately, I’ve been thinking though, what a shame it is that no one else gets to see your ideas but me! So, what if you follow and share your feedback and thoughts with me on Twitter at DrBapuPod. That way, your fellow listeners can read your ideas and chime in. We can brainstorm together about the latest episode or the latest research that caught your eye!

That’s DrBapuPod — D-R B-A-P-U P-O-D — on Twitter. I’m looking forward to it. And now, back to the show.

*          *          *

David Asch’s study of the outcomes of different residency programs came out just a few months after a landmark article in The New Yorker by the surgeon and writer Atul Gawande called “The Cost Conundrum.” It examined the vast differences in Medicare costs across the country. People often refer to this story as “the McAllen, Texas, article,” because it largely focused on this one town, where Medicare spending per person was about double the national average. The article became a lightning rod for discussions about the Affordable Care Act because it captured the shocking extent to which the amount of money our country spends on healthcare is driven by sometimes unnecessary and ineffective care.

But for one researcher and her team, that lightning strike was a little different.

CHEN: Could we look at whether where you train and whether you train in a low-spending area, whether you train in a high-spending area, then influences how you go on to practice?

That’s Candice Chen. Candice teaches health policy at George Washington University. She also leads research to improve the relationships between healthcare professionals’ and their communities, especially those that are underserved. And she’s a pediatrician.

CHEN: The very purpose of residency programs is to train future doctors. And so, it didn’t seem unreasonable that you might also be training people to practice in certain ways that might be good and in certain ways that might not be so good.

There are lots of ways to think about good care and not-so-good care. Some doctors may not offer enough care to their patients because of, say, the kind of insurance the patient has or how the doctor’s paid. Other doctors may offer interventions that really aren’t necessary – say, sending a patient with back pain to an orthopedic surgeon before first trying physical therapy.

Candice and her colleagues had a very specific question: how did where a doctor trained influence their spending patterns later on — things like how many diagnostic tests they ordered or what medications they prescribed? Did physicians who attended residency programs in high-spending areas, like Miami or McAllen, Texas, later practice in ways that cost Medicare more, compared to physicians who trained in lower-spending places? Did doctors take the spending habits of their residency program with them when they left?

The thing is, the amount of money spent on healthcare in a given geographic area often isn’t explained by just how sick the population is or their preferences for care. So, the big question is: where do doctors’ practice habits come from? They don’t fall from the sky. A natural place to look is where a doctor trained. Maybe doctors who train in areas with higher spending get “imprinted” and those practice styles remain with them long after they leave.

To study this idea, Candice and her colleagues looked at Medicare claims from about 3,000 physicians who treated nearly 500,000 patients. What they did first is look at the average amount of money that each primary care provider’s patients spent per year. They wanted to classify doctors as being high- or low-spending based on how much care their patients received.

CHEN: Then we looked at where they trained and whether they trained in low spending areas or whether they trained in high spending areas. And then we looked at how they practiced once they were out in practice. basically, did where they trained and the spending characteristics of where they trained then influence how they were practicing once they were beyond their residency training programs.

So, what do you think? You think that the spending level of where a doctor trained influenced their spending patterns later on?

CHEN: What we found was it does. So, if you train in a low=spending area, you are more likely to spend less per Medicare beneficiary once you’re in practice. If you train in a high spending area, you’re more likely to spend more.

But the researchers wanted to be sure the spending habits were definitely a holdover from the residency program in which doctors trained. To do that, they looked specifically at doctors who had trained in one kind of spending region — high or low — and then moved to the opposite kind — low-or high — to start their careers.

CHEN: You could see that the people who started off in low-spending areas and went into high-spending areas, continued to practice in ways that were less costly. and when we looked in the opposite direction of people who are trained in high-spending areas, going into low-spending areas, we did see suggestion that they also continued to spend more than their counterparts in the low-spending practice areas.

Okay, so what kind of numbers are we talking about here? It turns out that the difference in health care spending among doctors who trained in high-spending regions and those who trained in low-spending regions was $522 per patient. That’s actually a lot. That figure controls for other factors that could otherwise affect how much healthcare a patient gets – like their age, sex, race and ethnicity, chronic medical conditions, where they live, their provider’s specialty, all of it. The only thing contributing to this difference is where the doctor trained.

CHEN: What our study shows is that where you train will then influence so many potentially different communities as physicians move around the country and locate in different areas. Also, it affects our overall healthcare spending. And if we trained a bunch more people in low-spending areas that might, in the long run, help us with our healthcare spending overall.

Now, a study that came out after this one looked at whether low-spending doctors were compromising on quality. Maybe their patients were getting subpar care and that was the explanation for the lower cost per patient. But that wasn’t the case. That study found no difference in quality of care or how well the patients did. My own work on the relationship between doctors’ spending patterns and their outcomes, which we talked about in Episode 19, has found something similar.

Now, I should mention that Candice found that even though physicians seemed to adopt the spending patterns of where they trained, that did change over time — slowly. Year by year, many of the physicians included in the study gradually adopted the spending habits of the regions they moved to. The doctors who were initially high-spending — because they’d trained in regions that were high-spending — became low-spending when they moved to low-spending areas. And vice versa. But it took time.

CHEN: It was about 15 years out that we started to see that the effect goes away.

Candice wasn’t alone in her findings. The economist David Molitor, who you heard from in the episode about Hurricane Katrina, has studied the evolution of practice styles among cardiologists. Like Candice, he found that where a cardiologist trained impacted their practice patterns, but unlike Candice, he found that those practice patterns quickly adjust to where they end up moving. I think this is an area where we could learn much more.

It’s worth noting that Candice and her colleagues did notice one striking trend: primary care doctors spent less. On the one hand, that may seem obvious. Primary care typically doesn’t involve as many costly interventions as the specialty medicine required to treat, say, cancer or heart disease.

But this finding also hints at something potentially important when it comes to understanding healthcare spending. Communities with a lot of specialists and fewer primary care physicians tend to spend more on health care. That raises the possibility that the supply of specialists creates the demand.

CHEN: I think that there’s a real concern in the healthcare system, that when there are more providers, when there are excess beds and services available, the healthcare system has demonstrated that that might actually drive demand.

I do want to say though that the jury is actually still out on why it is that areas with more specialists have greater health care spending. It’s certainly possible that the supply of specialists induces demand, but it’s also the case that specialists locate in areas where the demand — the clinical need — is greater. What you need to answer this question is a clever natural experiment that leads some areas to have, by chance, more specialists than others. If you have any good ideas, you let me know!

Now, let’s get back to how we started today’s show: your search for a doctor. So, first David Asch showed us that the quality of training a doctor receives during their residency might impact how well their patients do later on. But Candice Chen told us that, in some cases, there might be a price to pay — literally — for that residency, if that program was in an area that tends to spend a lot on healthcare.

So, what else might you want to consider when choosing a doctor? Maybe you’re worried about whether your doctor trained in the U.S. or went to school in another country? My mom is a doctor. She trained in India and growing up she used to always tell me — in a motherly Indian accent — that a good doctor didn’t need labs, they didn’t need imaging. They just needed to know a patient’s story. Now, I won’t try to replicate her accent here, but there’s probably a lot of truth to that. Coming to this country to practice medicine was a huge step for her, and I can tell you that both she and many of her colleagues who trained in other countries struggled when they got here. Struggled to convince people — sometimes colleagues, sometimes patients — that they were good enough.

I wanted to dig into that thought. So, a few years ago, my colleagues and I compared patient outcomes among doctors who attended foreign medical schools and those who attended U.S. programs.

Again, I’m curious what you think we found? Which patients did better? Well, we looked at about 1,200,000 hospital admissions treated by about 44,000 general internists, and again, all of the patients were age 65 or older. We found that the 30-day mortality rate was lower among patients whose doctors were international graduates. These were doctors who were born and trained abroad.

A key part of this study was that it compared patients treated by international or domestic medical graduates who worked in the same hospital. The patients that those two groups of doctors treated were similar, which meant that the reason more patients of international medical graduates lived longer had to be because of the doctors, not because the patients were somehow healthier.

In light of my mom’s story, which reflects the experience of many people like her, maybe we shouldn’t be surprised by this finding. The doctors who train abroad and leave their home to come to the U.S. face immense competition for limited residency training spots, which means they have to stand out from their stateside counterparts. And some often repeat residencies in this country after finishing a residency in their own country, which means extra years of training. And, of course, the group is highly selected for people who are incredibly motivated.

You may be noticing a trend here. Sometimes the traits people prize as the hallmarks of a good doctor – the training program, years of experience, where they’re from – do matter. But sometimes those assumptions are wrong. Sometimes the doctor from the so-called better residency program is the right choice. But sometimes the only difference is the amount of money their choices cost the healthcare system, which can negatively affect all of us.
So, how do we decide on a doctor? Well, Candice Chen says maybe set aside the doctor’s resume and ask yourself: is the doctor you’re considering someone you feel like you can talk to?

CHEN: And if you’re a patient, I think that the way that that feels is that you’re in partnership with your healthcare providers, that they’re giving you information, that they’re explaining when you don’t understand that they try again. What I am looking for and what many people are looking for is oftentimes represented in communication.

I’d agree with that and maybe I’ll end today with what I look for in a doctor. Things like good communication and bedside manner are clearly important. Understanding what your preferences are for care is also key. I’m personally sort of the “less-is-more” type when it comes to my own care and the care of my kids. So, a doctor with enough experience to feel comfortable just waiting, “observing” a problem, rather than ordering a bunch of tests or jumping into a treatment. That’s important to me. But others may want something different. They may want a doctor who does more. Where a doctor trained is less important to me but experience, especially ongoing experience, matters to me.

The last thing I’ll say is that I tend to value very highly what other doctors recommend. If I need to see a doctor, the first thing I do is ask my colleagues who they’d recommend. Now, that obviously isn’t possible for most of us, but there is a nugget of wisdom there that might apply to all of us. What if we tried to measure the quality of doctors by looking at who doctors choose when they need medical care? Maybe we’ll talk more about that idea another day.

Anyway, that’s it for Freakonomics, M.D. this week. I hope you enjoyed our discussion today. You can find links to all the studies we mentioned at freakonomics.com.

As always, I want to thank you for listening. It’d be great if you could give us a review on Apple Podcasts or wherever you’re listening. It helps new people discover the show.

And like I said earlier, if you have any thoughts on the show, follow Freakonomics, M.D. on Twitter at DrBapuPod — that’s D-R B-A-P-U P-O-D — and send me a tweet! You can also still shoot me an email at bapu@freakonomics.com.

*          *          *

Freakonomics, M.D. is part of the Freakonomics Radio Network, which also includes Freakonomics Radio, No Stupid Questions, and People I (Mostly) Admire. All our shows are produced by Stitcher and Renbud Radio. You can find us on Twitter and Instagram at @drbapupod. This episode was produced by Jessica Wapner and mixed by Eleanor Osborne. Original music composed by Luis Guerra. The supervising producer was Tracey Samuelson. Our staff also includes Alison Craiglow, Greg Rippin, Rebecca Lee Douglas, Morgan Levey, Zack Lapinski, Ryan Kelley, Mary Diduch, Jasmin Klinger, Emma Tyrrell, Lyric Bowditch, Jacob Clemente and Stephen Dubner. If you like this show or any other show in the Freakonomics Radio Network, please recommend it to your family and friends. That’s the best way to support the podcasts you love. As always, thanks for listening.

ASCH: Well, my strategy in choosing a doctor is to not get sick in the first place. And I would advise everyone to follow that particular rule.