Edward Miguel on Collecting Economic Data by Canoe and Correlating Conflict with Rainfall

One thing I’ve learned over my career as an academic economist is that it’s hard to do good research. I started out hoping to do important research but I figured out quickly, I wasn’t very good at important stuff, so I settled for at least doing research that was interesting. I’ve always admired economists who do important work, even if it’s boring. But the economists I admire the most are the ones who manage to do work that is both interesting and important. There are only a handful of those folks. And my guest today, Ted Miguel, is right at the top of that list.

Edward MIGUEL: This kind of human capital investment could actually help break the intergenerational transmission of poverty.

Edward Miguel, his friends call him Ted, was an undergrad at M.I.T. and did his Ph.D. in economics at Harvard, working with Nobel Prize winner Michael Kremer. He spent the last 20 years in the economics department at the University of California at Berkeley. Unfortunately, my many attempts to convince him to come to the University of Chicago, they all failed. 

Ted has been one of the pioneers in bringing large-scale randomized experiments into economics, but he’s also remarkably creative in exploiting what economists call natural experiments. My only concern today is that Ted can be quite serious and earnest. That’s a great trait in a researcher, but a bad one in a podcast guest. So my challenge is to try to get him to have some fun. With that in mind, I know exactly how I want to start our conversation.

Steven LEVITT: How about we start with your least glamorous topic? Can you tell us about worms and why an economist would ever study worms?

MIGUEL: I guess, somebody has to be the worms guy and that’s me. I started working on a project to treat kids who had worm infections, intestinal parasites. I went to Kenya in 1997 to help set up a project that would provide health treatments to kids and then studied what happened to their education, their test scores, their school attendance — and what was really cool about that project at the time is it was one of the first handful of randomized control trials, so field experiments, in development economics. It’s definitely a topic that at the time was a bit out of the mainstream of what economists were studying. 

LEVITT: Yeah, I’ll say, yeah. Most people don’t know anything about worms. So what kind of worms are they? And how bad are they?

MIGUEL: Yeah, there’s a bunch of different worms that are common in East Africa, including in the areas that we worked in Kenya: Hookworm, roundworm, whipworm, and schistosomiasis. These kinds of parasites are incredibly prevalent around the world. Between one to two billion people today still have these worm infections. And they used to be really widespread in the U.S. a hundred years ago. So in the U.S. South, there were very high rates of hookworm infection estimated to be 40 or 50 percent of the population. 

So these are worms that you get through bad hygiene and sanitation, and they bore their way into your intestines and they can affect your nutrition. They can make you anemic, make you weak, affect absorption of nutrients. There’s recent research showing they can affect the gut microbiome, which can have all kinds of health effects. They’re not the kinds of things that’ll kill you, or only in very rare circumstances, but they can really weaken kids and affect their growth and affect their learning in school. 

So the only really good thing about these infections is that the treatment is super easy. It’s very cheap. Costs today 30 cents per kid, per year when delivered at scale. They only need to be delivered — the drugs — once or twice a year. And they can be done on school health days where every kid just gets a pill. They stand in line and they get a pill. 

LEVITT: Okay, wait, so hold on. That implies that you have no idea if these kids have worms to begin with, you’d be giving a pill to every kid, just because it’s too hard to figure out if any individual kid has the worms?

MIGUEL: That’s exactly right. So the recommendation from the World Health Organization for the last 30 or so years has been to mass treat with deworming drugs in areas where there’s at least a certain minimum level of prevalence usually above 20 or 30 percent. And the reason why it’s exactly what you said, which is finding out who’s infected is actually pretty complicated. You have to get stool samples, you have to take those samples to a lab, and figure out who has which worm infections. 

But because the drugs are so cheap and have such minimal side effects, it’s much more cost-effective to mass treat kids in areas where there’s some minimal prevalence and, in our study area, in Western Kenya, in the late 1990s, over 90 percent of the kids had worm infections. Basically everybody had worms. So, it made sense.

LEVITT: One pill does it for life, Ted? Or you have to take this pill over and over?

MIGUEL: You have to take it over and over. So if you’re in a setting where there is bad hygiene, bad sanitation, worm eggs from people’s stool enter the environment. And if you come into contact with those worm eggs in fecal matter you can just keep getting reinfected over and over again. And so you need to be treated, either every year or every six months, for as long as you’re in a setting with lots of worms. So it isn’t like a shot or a vaccination where, hopefully, when you get it, you’re protected for a long time. Although, I guess we’ve seen with vaccines, that’s not always the case.

LEVITT: Okay. And we see in the U.S. that lots of people don’t want to take the Covid vaccine. Do you have a lot of people in your study saying, “No. I don’t want to take this pill.” Or does everybody just take the pill?

MIGUEL: We were really concerned in the first year. We just didn’t know what the reaction would be in these communities together with a non-profit organization we were working with and with the ministry of health. We held meetings in all the communities where deworming was going to get rolled out to explain the drugs to them, explain the health problem. And we had a take-up of about 75 percent. 

So it’s actually not that different than when we think about vaccine hesitancy in the U.S. now we think 30 percent of people are pretty reluctant. The vast majority of people did end up getting treated and of course that’s really important because as I mentioned, it’s really other people’s worm infections that enter the environment and fecal matter that re-infect you. So treating a large share of the community becomes very important for control.

LEVITT: It’s what economists call an externality — your being treated for the worms has a benefit to everyone else in the same way that your being vaccinated helps everyone else. But that creates some problems for study design because the typical randomized experiment, you randomize across people. And then you compare the people who got the treatment to the people who didn’t get the treatment. But obviously you can’t do that in this setting. So you had to do something different in your randomization, right?

MIGUEL: Exactly. Some of the early studies before us, there were a handful of small-scale experiments on deworming that did randomize at the individual level. But those studies are problematic methodologically for exactly the reason you said, if we’re treating some folks in a community or in a school and not others, the untreated are benefiting from the treatment status of people nearby because they’re getting infected less. So the design that we used was what’s called a cluster-randomized design, where all the kids in a given school community would get treated. Those would be treatment schools. And then there were control schools. Now to be clear, the way the design was set up is eventually all the schools in the area received deworming, but it was staggered in over about three years. So there were early treatment schools and then these late treatment schools are like our control. In the early treatment schools, we tried to treat everybody and that allowed for the benefits of externalities to be realized and also allowed us to compare effects more reliably between the treatment and control schools, better than if we had randomized at the individual level.

LEVITT: So the way you’re describing it, it sounds like at the time you were doing this deworming was controversial and not that well known — or not very widespread? Is that true or it just hadn’t really been studied in an experimental way?

MIGUEL: The W.H.O. had already issued a recommendation in the early nineties for mass deworming treatment but the research on deworming at scale was really limited. So we worked across 75 different school communities with over 30,000 kids in the sample, so a pretty large sample, to be collecting data on. The other thing that was different about what we did at the time is we weren’t just looking at health outcomes. We did look at those, but we were very interested in the educational benefits, and the broader human capital formation effects of improving child health. So we measured child school attendance, dropouts, test scores to see if improving child health in a setting where basically all these kids have worm infections, if that would lead to improvements in schooling. And that was a really novel aspect of the research at the time.

LEVITT: Yeah. So you looked over a three or four-year period. And what did you find? 

MIGUEL: First of all, in terms of schooling, there was a sharp reduction in school absenteeism in the schools that received deworming. So absenteeism was quite high in these schools on average. About 25 percent of kids were out of school on a given day over the few years we collected data. But that fell by about six percentage points. So about a quarter, in the treatment schools. And that was a really striking reduction, especially for a treatment that was so cheap. 

The second thing that we found, and this is where the externalities come in, is we were able to estimate the effects of the treatment on the health outcomes, even of kids who were not living in or attending these treatment schools. We found that providing deworming drugs in a particular elementary school actually reduced infection levels of kids that lived up to four kilometers away. So we also provided some new evidence on the importance of treatment externalities for deworming drugs. And that’s something that even though externalities has been central to public economic theory and core-economic theory for a long time, actually demonstrating the magnitude of those externalities, empirically, had been challenging.

LEVITT: So those results sound pretty good, but they don’t sound revolutionary exactly. I’ve had the impression that deworming was roughly the single best thing we have going in development economics. Am I wrong about that?

MIGUEL: No. Well, that the first study that looked at schooling was really just the beginning. So, once we found those schooling estimates, we kept studying and tracking this sample of kids over time. And we had the idea that if we do improve kids’ health, we do improve their schooling, that could really affect their life in many different ways. So we put in place a data collection project called the Kenya Life Panel Survey, K.L.P.S., that has now been tracking these kids for 23 years. So it started in 1998 with the school deworming project and now instead of being 10-years-old, eight years old, 12 years old, these folks are in their mid-30s. 

And we’re actually able to see what the effect was of that child-health investment on their adult labor-market outcomes, earnings outcomes. So the kids who received deworming when they’re in elementary school are more likely to go on to secondary school. And actually, those effects are larger for females. The girls who got deworming ended up getting more schooling. When we track them even farther — 10, 15, and 20 years down the line, we see large benefits in terms of labor market earnings. So they earn on average something like 10 to 14 percent more per year, even 20 years later.

LEVITT: Woah. Wow. Okay. So that way, let me stop you there because that’s huge. So you only dewormed these kids two or three times, right?

MIGUEL: For two or three additional years in childhood.

LEVITT: Okay. So that temporary treatment, which you said costs — what did you say? Thirty cents per kid?

MIGUEL: Yes. Per year.

LEVITT: Okay. For a dollar per kid, you did an intervention that led to a 10-percent increase in lifetime income. That sounds crazy, right?

MIGUEL: It’s a huge effect. And when you look at the rate of return of that investment, it’s massive because obviously, folks in Kenya earn less than folks in the U.S. but even for someone who’s relatively poor, who’s earning, let’s say $500 a year, if they’re earning 10 percent more, that’s $50 a year. Every year. So the rate of return is just huge for this intervention and it’s like we kept going back into the field, Steve — we went back after five years and then after 10 years and then after 15 years, and every time we go back, we’re wondering, okay, at what point does this peter out? 

In the third round out, we started measuring agricultural productivity in a more detailed way. Then we started administering very detailed consumption surveys and in development economics, what people consume, what they spend money on, is seen as a really good measure of living standards. So we keep going out in time and measuring things better and better. And we keep replicating this gain of roughly 10 percent in earnings, in consumption, in assets. So for us it’s really been a pretty staggering finding and it shows that improving kids’ health for a few critical years of childhood can really make a big difference.

LEVITT: Yeah, that’s awesome. And so one would hope that given this evidence, deworming is standard practice across the world. What’s the state of deworming right now?

MIGUEL: The state of deworming is different than it was 30 years ago. There’s a lot more than there used to be. And certainly the evidence that we’ve put on the table and that other researchers have produced about deworming have contributed. Michael Kramer and I had presented our results a lot in Kenya to ministry officials, to local officials, to the World Bank, to donors, and they got pretty excited about the possibilities of expanding deworming. 

So starting in 2009, they rolled out a national deworming program across elementary schools in the whole county. Which they’ve continued since then. So in Kenya, rates of worm infections, just because there’s been like year after year of deworming over more than a decade now, rates are just way down from where they used to be, which is a huge success. 

LEVITT: The rates were 90 percent when you started, at least in your communities. What are they now?

MIGUEL: Yep. In the parts of Western Kenya that we work in, the latest data I’ve seen is from a couple of years ago, but they’re down to something like 20 percent from 90 percent. 

LEVITT: You noted that economists didn’t really do randomized field experiments in the mid 1990s. And I think that will be really surprising to many listeners who are outside of the field. Because it seems so obvious, if economists want to learn about worms, they should be out doing the kind of studies you’re doing. But strangely, you and I are about the same age, when we were in grad school it wasn’t normal. 

I remember the first time someone told me about a randomized experiment, my reaction was — I didn’t realize you could do that. I thought you had just live with the data you’re given by the government. It’s so stupid in retrospect, it’s so easy to live within constraints. I just want to highlight that this sounds mundane now to the modern ear, but it was really revolutionary what you were doing.

MIGUEL: Well, I agree. And I remember the comments in grad school from faculty and students alike of something like, “That isn’t economics. If you run an experiment, that’s not economics.” And again, it’s almost laughable now. 

LEVITT: Now, the second thing that I don’t think people appreciate is just how hard it is to, number one, implement the study. But number two, especially to track these folks for 30 years. I did a study with Roland Fryer and John List where we started a preschool in the city of Chicago Heights. And we wanted to track these kids. This is much easier. We’re in the U.S. We had all the advantages. Oh my God, we failed so badly. They just vanished. They just disappeared. I think the effort, the grit that it takes to do what you did is really commendable. 

MIGUEL: Well, thanks Steve. From the beginning, certainly the field work was a challenge. We started the project in 1997, ’98, which was a year when there was a very large El Niño phenomenon in the world. There was a lot of flooding in the area — getting around the study area was challenging. There were times in the baseline data collection when so many roads were flooded out that we were getting to schools via canoe, literally. Like these roads had flooded. So we were just canoeing. 

LEVITT: You had to be careful not to tip over those canoes and lose your fecal samples that you collected into the water, spreading worms to other areas down the way.

MIGUEL: You joke, but that’s true. And then the water was infected with schistosomiasis. It was, it was real. It was real.

LEVITT: The other thing that stands out about this project is the incentives in economics and academic economics are so much towards running a pilot study, showing that something works, getting it published in a great journal, and then moving onto something else. And I think that’s one of the biggest criticisms someone could launch at the field of development economics. But it’s heartening that you stuck with this and that the public policy people really embraced it. And now deworming is done at such a broad level. 

MIGUEL: We did a survey of published research and development that came out a couple of years ago. Now there’s been thousands of development economics R.C.T.’s, randomized experiments. And we did a survey of them to figure out how many had done a follow-up for at least eight to 10 years. And there’s really only a handful out of those thousands of projects. So it’s true, very few studies are designed that way. 

I do hope that what we’re doing with K.L.P.S. and with long run deworming follow-ups will inspire other development economists. And I think it has already to continue tracking samples. I hope that donors and funders will see the value of it because some results really only emerge over time. And I’ll give you one more result from this project. This is not yet published but we’ve been working on it a lot the last six months. 

Now, Steve, we’re able to collect data on the outcomes of the children of our original respondents because now they’re in their mid-30s. Folks in Kenya, on average, have four kids. And, our samples had about three on average, by the time they get to their mid-30s. And the first thing we measured was the most basic health outcome for children, which is survival. So unfortunately, child survival is a big issue in Subsaharan Africa, including in Kenya. 

And at the start of our study period about 80 kids out of 1,000 would die before age five. Eight percent is really high — an order of magnitude above what we would have in the U.S. Now over time, it’s fallen. Kenya has developed and there’s been improvements in health. What we find is in our sample, there’s a reduction of a quarter in under-five mortality for the kids of people in the treatment group. 

So the kind of broader social benefits just go way beyond even those earning gains, I mentioned before. We’re starting to measure cognitive effects among three to six-year-olds and six to eight-year-olds. We’re finding some gains there. The indication is this kind of human capital investment could actually benefit the next generation and maybe help break the intergenerational transmission of poverty.

LEVITT: So let me change the topic, specifically to armed conflict in Africa. So in recent history, in any given year, what share of African countries are involved in an armed conflict? Either fighting another country or some kind of civil war?

MIGUEL: The rates are very high. So in any given year, 20 percent of African countries can be in some form of armed conflict. But if we go back to 1980, so the last 40 years or so, over 80 percent of countries have had at least one year of armed conflict. It’s ubiquitous. The vast majority of countries have experienced conflict recently.

LEVITT: Okay. So I’d like to pause and ask listeners to think about the factors that are most important in predicting where there will be conflict. So if you, the listener, are feeling inspired or you like the challenge, hit pause, take a minute to come up with the three or four factors that you think will be most predictive. I’m going to rattle off a few of the factors that come to my mind and Ted, I know you’re an expert in this era. I want you to tell me one by one, whether empirically these different factors turn out to be important. Okay? So the first one is how much ethnic diversity there is in a country. I’d expect the countries with many different ethnic groups would have much more conflict than those that are more homogeneous. Is that right?

MIGUEL: It’s a contested research literature, but there’s no clear evidence that more diversity leads to more conflict. Certain studies say if there’s certain alignments of ethnic groups, there could be more, but as a first order, kind of fact, no. It’s not the case that more linguistically, ethnically diverse countries have more conflict. That’s very surprising to people, I think.

LEVITT: It is surprising. How about democracy? Every American is raised to believe that democracy is good and dictatorship is bad. Does this hold true when it comes to armed conflict in Africa?

MIGUEL: There’s a very weak relationship and a lot of democracies in Sub-Saharan Africa revert to autocracy quite quickly. So it’s a kind of unstable situation, in terms of which countries are democratic and there’s no strong smoking gun correlation with conflict.

LEVITT: Okay. How about a country’s colonial history? There’s been tons of research in economics suggesting that the more extractive and ruthless the colonial times, the worse things are today. Is that predictive of more armed conflict?

MIGUEL: No, no strong relationship. These are all the things like everybody looked at, including myself. And we were like, wait, where are the relationships we thought were there? 

LEVITT: Okay. So I totally failed. We could talk about other possible factors, maybe education levels, or be an ally of the U.S., geography. But why don’t you tell us about the fact that you’ve shown to be incredibly important? And I will say it is a factor that not only would I personally never have thought of, but when you first proposed the hypothesis, I honestly thought you were wrong and over the years, I’ve been really convinced of the evidence. 

MIGUEL: So about 20 years ago, I started working in this area and found evidence, strong evidence, that rainfall patterns and also temperatures, high temperatures, are very strongly predictive of conflict. In the original research, we focused on rainfall and we found that in African countries in the year after a drop in rainfall — so basically like when countries enter a drought situation and there’s a big decline in rainfall, there’s a very large increase, immediately, within a year, in the risk of armed conflict. 

LEVITT: So what do you mean by large? 

MIGUEL: So in the year after a major drop in rainfall, which is associated with a drought, you could see an increase in conflict risk of something like 50 percent. So going from say, a 20-percent baseline risk of armed conflict up to 30-percent risk.

LEVITT: So a 10-percentage point increase. So one in 10 countries that experience a drought will have an extra armed conflict one year later?

MIGUEL: Yeah, following a large drop in rainfall. Exactly. It was a very striking finding for us.

LEVITT: I can see why I didn’t believe it when you first came up with it. It sounds so crazy, but what do you think the mechanism — it’s, obviously, people are poorer, but is any G.D.P. change going to lead to the same thing or is this a very particular mechanism that leads to conflict?

MIGUEL: At the time with just the rainfall data on hand, it was hard for us to pin down the exact mechanism. We speculated though that rainfall mattered a lot because so much G.D.P. was driven by the agriculture sector in Sub-Saharan African countries. And most agriculture is rain-fed agriculture. Even today, there isn’t nearly as much irrigation and water storage and water management for farming in Africa, relative to Asia, say. So when the rains fail, people’s crops fail and rural populations are poor. They’re very often living on the edge of subsistence. So an extreme drop in rainfall can translate into an economic shock and political instability.

LEVITT: What’s a little bit weird about that is that these are the poorest people. They’re — obviously have very little political power. Is the idea that they become so desperate that they’ll follow populists who are doing crazy things? Or they could become refugees? And so they’re crossing over borders and destabilizing the neighbors. I don’t know why things happen, always hard, but do you have thoughts on that?

MIGUEL: Yeah, since we published our paper, there’s a whole literature that’s emerged looking at the effect of extreme rainfall, but also extreme temperature on conflict outcomes in Sub-Saharan Africa and globally. And some of the more recent literature finds that in the years following a big drop in rainfall or of an extremely hot year where, again, crop growth would be hit, you see an increase in many different forms of violence from cattle raids to property crime, all the way up to civil war. So it really feels like a kind of social breakdown in periods where there is extreme weather, just because so many people’s livelihoods depend on the weather in an agrarian society.

LEVITT: The obvious extension of this is thinking about climate change. And I’m sure these models say that Africa will get hotter. Are the models precise enough to tell us what to expect about rainfall and drought? That really matters for the future of these areas.

MIGUEL: For sure. The climate models, as you said, are much more precise and confident about temperature increase. But model to model, there is more variability in terms of their predictions for rainfall. But when we — and this is work with colleagues here at Berkeley and at Stanford — we have done an exercise where we take the estimates on the relationship between rainfall and temperature today in the contemporary period and we kind wind it forward, given climate change predictions, to predict how armed conflict will change in the future. 

And when we do that, the models do predict with some degree of precision that there will be a large increase in conflict risk over the next 50 years in Sub-Saharan Africa, driven by the temperature increase. So in addition to variable rainfall, high temperatures are associated with bad crop outcomes, and that’s something that the climate models and the climate scientists are just really confident about that the world is going to warm at this point, by at least a couple degrees Celsius, if not more, unfortunately.

LEVITT: And you focus there on the idea of armed conflict, but as you said before, armed conflict is just the tip of the iceberg, right? It really is a proxy for the breakdown of civility and of the social norm. So it sounds from that, you’re not so optimistic about the future of Africa?

MIGUEL: It’s such a hard question. What you said is right, Steve, in that our research has spawned some other research and other folks have been working on this wide range of consequences of high temperatures and extreme climate. There’s studies showing that domestic violence goes up. There’s studies showing crime goes up, and that is terrifying. It’s really scary, especially for those of us who are working in and care a lot about economic development and political development in Sub-Saharan Africa. The last 20 to 25 years have been a period that’s been pretty good in terms of economic development in most African countries. 

LEVITT: Wait, hold on. Wait, hold on. So I do know in 2009 you wrote a book and you were very optimistic about Africa’s future and it was called Africa’s Turn. And I will say, I shared your optimism. Maybe even you influenced my own optimism. And in 2012, I invested a chunk of my retirement in a mutual fund, dedicated to companies listed on the African Stock Exchange. 

So, in preparation for this interview, I went back and looked at the performance of that thing. And it turns out that in the last nine years, my index fund is down 30 percent. And If I had invested that in the S&P 500, I would have tripled my money! So then having done that, I looked at information on G.D.P. per-capita growth in Africa. Look, and maybe I misread the data, but it looked terrible. It looked like the last decade Africa has been a disaster. Is that not true?

MIGUEL: It’s really all relative. The 80s and 90s per-capita income growth in Sub-Saharan Africa as a whole was negative for multiple decades. From the mid-70s to about 2000 was a really bleak period, tons of armed conflict, even more than today, G.D.P. was declining. And since about 2000, G.D.P. rates are positive. So there’s been this turnaround from negative to positive with roughly three percent G.D.P. growth per year across Subsaharan Africa. 

LEVITT: But per capita — but aren’t populations growing two or three percent a year, as well?

MIGUEL: Yeah, maybe two percent a year. So, there’s positive G.D.P. per capita growth the last 20 years, rather than negative growth. So I think that’s where things are better. It’s certainly not— Growth has not been at east Asian levels. but economies aren’t shrinking. 

LEVITT: Only my portfolio is shrinking.

MIGUEL: Yeah, I’m so sorry to hear that. There’s certainly a long way to go. And you throw climate change on top of that and you start thinking about all the volatility that’s going to come from extreme temperatures in a context where countries are just digging themselves out of the hole of the 80s and the 90s. And it’s terrifying.

LEVITT: So overall, if you had to rate your optimism about Africa’s prospects, mildly bullish or not so sure?

MIGUEL: I still am mildly bullish just because there are a lot of other changes that are really positive, even beyond the G.D.P. figures. Levels of schooling have gone through the roof in a lot of African countries, there’ve been health gains. Child survival rates have improved. The H.I.V./ AIDS epidemic has been brought under some control relative to 20 years ago. In a lot of different dimensions, there has been real progress. 

In a bunch of countries gender gaps in education have really narrowed. Not all African countries, but certainly in some. I think those are the areas where I see a lot of positive trends. There’s also the beginnings of some really exciting new sectors in Africa. There’s a tech sector in Nairobi, there’s an entertainment industry in Nigeria that’s becoming globally known. So when I see all these trends, they really give me a lot of hope for the future. But again, climate change and the risk of extreme temperatures is just hanging over everybody’s head.

LEVITT: Okay. I’m going to hang on to my mutual fund. I’ll probably regret it, but I’m going to stick with it. 

*      *      *

LEVITT: Morgan, what do we have on tap today?

Morgan LEVEY: So Steve, we had a listener write in with the observation that you often use the fact that you’re an economist as rationale for why you think or act a certain way. This person went on to ask if your economic-style of thinking has ever led you astray?

LEVITT: Well, it’s not that I try to think like an economist. I just don’t know any other real way to think. But where I found that economics is not helpful at all, in fact, it’s actively harmful, is in thinking about the best way to get to whatever point I’m trying to get to. The more I think like an economist when I think about the process or how to convince someone or how to get my way, the worse things go. 

LEVEY: Has there ever been a situation where trying to get to the right answer has forced you to think in a different manner?

LEVITT: Yeah, it goes back to when I was in college and we were trying to figure out how to allocate roommates in our rooming assignments. So we had about 14 people in our rooming group, as we tried to apply for rooms in our dorm. And the reason we had 14 was because the incentives were set up so that the bigger the group, the better. And I had figured out as an economist that we needed a really big group, but the problem was we didn’t really even like each other very much. So I had solved one problem with economics, which was how would we get a good lottery number, but I hadn’t figured out how to solve the issue of: how can we actually allocate these 14 people across three or four rooms to make everybody happy? 

And then it hit me. Well, look, utility theory tells us how to do this. I will just set up a computer program where each of us will anonymously enter our own preferences about who we like and don’t like in the rooming group, and I’ll write an algorithm in the background, which will do all the calculations to figure out what the optimal allocation of the people is across three or four different rooms. Okay, so that all sounded great. And I told them, as soon as it’s done, it’ll destroy all the data afterwards. So no one has to worry about their preferences being known. 

So everyone typed in their data. And I went into look at the results. And as soon as I looked at it, I realized I’d made a terrible error in logic because what had happened was that it had decided that the most efficient way to allocate people was to put the four people that nobody liked all in the same room, because that was the way to maximize overall utility. But the thing is those four people were going to be miserable because they actually hated each other more than other people hated them.

LEVEY: Were you one of those four people?

LEVITT: Oh, no, I was one of the popular ones. 

LEVEY: Sorry.

LEVITT: It was a rare case where I was one of the popular ones. And so I saw that result and I knew that anyone else who saw the result would know exactly what had happened too, and I did one of the smartest things I ever did, which is I turned off the computer and I walked out of the room to the waiting group. And I said, “I’m so sorry, something went wrong. And there was no output. It just didn’t work.” 

But that was a case where thinking like an economist, but not doing a very good job of really solving the problem well, led me to a really bad outcome. I remind myself all the time when I try to get too clever with economics. Is that when you’re too clever, oftentimes you get outsmarted.

LEVEY: Well, thank you for writing in. If you have a question for us, you can reach us at pima@freakonomics.com. Steve and I both read every email that’s sent and we look forward to reading yours.

LEVITT: So there’s a whole different side of your research we haven’t talked about where you ask questions that at first glance would seem impossible to answer. But then you come up with an incredibly clever twist. So one of your papers in that spirit that I love, because it is so ridiculous on the surface, but actually quite deep is your analysis of unpaid traffic violations among diplomats in New York City. Can you describe what that research is all about?

MIGUEL: Yeah, this is a project with Ray Fisman at Boston University. I have been interested in corruption for a long time. Certainly working in development, working in Kenya, it’s something that’s very present in policy-making. I also grew up in the New York area. And I remember as a kid always hearing about U.N. diplomats in New York not paying for parking. Basically, parking on the sidewalk, double parking. It was like a big issue in the local news in New York City in the 1980s that there were all these diplomats who, because they have diplomatic immunity could get away with anything. 

Diplomatic immunity really shields diplomatic personnel from any form of legal sanction. They can be expelled by the host country, but other than that they could commit crimes and they can’t be prosecuted. And, of course, a parking ticket isn’t a major crime, but it turns out that these U.N. diplomats living in New York of which there are many, racked up massive unpaid parking-ticket bills that the city of New York always wanted to collect on but couldn’t. And the idea crossed my mind:, which country’s diplomats are doing this? And is it the case that diplomats from countries that are more corrupt are really the ones who are taking advantage of the situation? 

LEVITT: So you take this little microcosm of parking tickets and the idea behind it is you want to use that as a window into the kind of corruption we would care more about.

MIGUEL: Exactly. And we realized that it was more than just this kind of cute, interesting case. What you have in the case of New York City with U.N. diplomats is you have diplomats from all over the world, from a hundred-something countries, all living in the same place, facing the same parking environment. And that’s really rare. 

Typically, when we observe the actions of government officials in Nigeria, they’re in Nigeria. And government officials in Norway are in Norway. So if we’re comparing corruption levels between Norway and Nigeria, there’s just so many things that are different. The legal environment is different. The likelihood they’re going to get caught is different. Their ability to get away with it is different. Cultural norms may be different, et cetera. 

You bring everybody to New York and you solve a lot of problems. First of all, they’re all facing the same legal environment. Second, they’re on the same streets facing the same parking meters. So all of a sudden we’ve taken people from around the world into a common environment. And we can ask the question: when you remove legal sanctions, when basically you give diplomats this immunity, is there something restraining their behavior that kind of prevents them from rule-breaking? Is it either a cultural norm or some psychic costs that they face that leads them to obey the rules even when they don’t have to?

LEVITT: Okay. So just to put this into perspective, what’s the scale of this activity?

MIGUEL: When we reached out to the parking division in New York City, they were really eager to share the data with us. They were like, “Please do something with this data. Publicize it.” And it was striking to us that from the start, there was huge variation across countries in how many tickets they got but the typical diplomat could get dozens of these tickets a year. 

LEVITT: Okay. So let’s name names. What were the countries that were the big offenders and what were the countries that were really rule abiding even without any punishment?

MIGUEL: So the country that had the highest number of parking tickets per diplomat was Kuwait. And there was like one license plate belonging to one diplomat who was racking up 246 unpaid tickets per year on average. So it was like multiple tickets per workday. There were diplomats from Egypt, Syria — Nigeria was in the top 10 list. So a number of Sub-Saharan African and Middle Eastern countries were towards the top getting 50, 80, 100 parking tickets per diplomat per year. Basically never paying for parking. 

The flip side is there were some countries that had zero unpaid parking tickets. Japan, Norway. Some other wealthier countries, but also countries that are characterized by lower levels of corruption. And that’s really where the study came in, is we wanted to go beyond just looking at a couple of cases or looking at a couple of countries to find something systematic. 

And when we gathered this parking ticket data, the number of unpaid tickets per diplomat, and we correlated it with measures of government corruption back in the home country, there was a very strong relationship. So really the strongest predictor of whether there were unpaid parking tickets among diplomats in New York was how corrupt government officials were in the home country.

LEVITT: Could you just do one paragraph, Ted, on how we measure corruption? People outside our field wouldn’t even begin to understand how we classify governments as being corrupt or not.

MIGUEL: So the measurement of corruption is always challenging because corruption is an illegal activity. One of the most common tools is through expert surveys, where folks who are researchers or business people, who have experience working in a particular country are asked to answer a bunch of specific questions about what the expectations are regarding bribe paying and other forms of corruption. And then those survey answers are aggregated and added up to come up with a country corruption score. So it’s an imperfect measure because corruption is so hard to measure directly. 

LEVITT: If it wasn’t hard to measure corruption, you wouldn’t be looking at parking tickets. So it’s not surprising. But you find this really strong relationship at the country level between what these experts who’ve been surveyed about doing business in a country — how corrupt they say a country is and then the diplomats in New York, how many unpaid tickets they have.

MIGUEL: That’s exactly right. It really feels like there’s something about the norms of behavior among government officials in the home country that come with these officials to New York. So even if they’re in New York for multiple years, the behavior of diplomats from a country where there is a culture of corruption or where corruption is widespread, really persists and in countries where there’s really minimal corruption, they don’t get these unpaid parking tickets.

LEVITT: So one notable shortcoming of this study is that you don’t know anything about the behavior of Americans because American diplomats don’t have immunity in New York City. Are you able to look at these data in any other place to see if Americans are well-behaved or big offenders?

MIGUEL: We’re always asked that. And again, as you said, we can’t get at it in New York. There was a similar case in London that does shed some light on this, and it doesn’t actually turn out the best for Americans. So around the same time in the city of London, they introduced a congestion charge for vehicles driving into the central city. 

And diplomats from a number of different countries early on protested and they said, “Hey, we’re not going to pay these congestion charges,” making the diplomatic immunity argument. Eventually, most countries ended up paying and complying with the regulation, but the country that was the largest offender that had accumulated the most unpaid congestion charges were U.S. diplomats in London. So, it doesn’t look the best for the U.S.

LEVITT: There’s a lot of talk these days in academics about a replication crisis, about the fact that lots of published academic studies are wrong. Researchers made errors in the analysis, or they consciously or unconsciously made modeling choices that made the results seem more significant or exciting than they really were. Or even maybe outright fabrication. So you’ve been talking about this for years long before others got on the bandwagon. How big a problem do you think this is in economics?

MIGUEL: People often ask about fraud per se. And the cases where there’s outright fabrication, data was made up — my own sense in economics is those cases may not be that common, but we don’t know for sure. What we do know is a lot of other practices that are questionable are really common. When we were in grad school, Steve, we were taught to analyze data and analyze it again and analyze it a hundred times. And when folks wrote papers, they would write up a paper showing a few regressions they had run, but they may have run like 5,000 regressions before they got to those three. 

And there was really no indication of the paper about the extent of specification searching and data mining that went into our analysis. And, we just know statistically, if you look at enough variables enough times, you’re going to find some relationships that are spurious — there are going to be some false positives. And the fear is that too much of the published literature, not just in economics, but in lots of other fields are just littered with these false positives due to data mining. And I think that’s a kind of pervasive problem in the culture of empirical social science research. 

LEVITT: Related to that one thing that has always confused me was the disconnect between how economists do empirical research, database research, and how we describe what we do in academic publications. What we do typically is we have some brilliant hypothesis that inspires us to go collect a bunch of data and we analyze the data. And at least I know in my case, almost always, my brilliant hypothesis is totally wrong. It just doesn’t fit the data at all. But along the way — “Oh, now I see.” I learned a bunch of things from the data and eventually interesting patterns emerged. And I think I finally understand what’s going on. So that’s all fine. 

But here’s what makes no sense at all to me. The way that people write data-based economic papers is they almost always start with an introduction and a literature review, and then they sketch out a theoretical model that yields a series of predictions. And then they have a data section that tests those predictions. Now, the thing is the theory almost every time has been written after the economist knows what the results are. The models have been reversed engineered to generate the predictions in the data. But authors never say this. They pretend the model came first and everybody knows that this is how we do it. And it is so unbelievably unscientific that I don’t understand. Has that struck you as being as crazy as it strikes me?

MIGUEL: It always has. And I think because I was coming up in grad school and getting my training working on experiments, I saw another way of doing things. So I was writing some papers in the traditional way, but I was also working on these experiments where you lay out your design in advance, you lay out your data collection in advance, you lay out your hypotheses in advance. And in some way that design, and the treatment arms, really lay out the kind of analyses that are natural in the data. They constrain a bunch of the analyses you might do with the data. 

Now, that’s not to say that exploratory research isn’t really useful. We’ve discovered so much in the social sciences and the biological sciences through serendipity through exploration. What I think we need to do as economists is just delineate — was this an exploratory study where we looked at the data in a lot of different ways? These are some cool patterns that we found. Versus confirmatory research where we lay things out in advance. We say, “These are the hypotheses we’re going to test and show the results, come what may.” 

Again, sometimes our theories are wrong, but you wouldn’t know that by reading most economics papers. And then the advantage of some of the experimental research again, when you’re locked into a design is you start getting lots of null findings. You start getting lots of strange patterns that demand more research. And I think that’s a really good thing for economics.

LEVITT: So I agree. One way to go is to lay out ahead of time, what you plan on doing and being held accountable to it. Approach I adopted unilaterally many years ago, over a decade ago, was to just continue to do exploratory work but in a very different way that felt more honest. And so what I do in my papers, usually, is I have an introduction and then I go right into the data. And I just try to describe the patterns in the data without trying to claim I know anything about why these patterns appear. Because, in principle, everybody should be able to agree on the patterns in the data. They’re just correlations. 

And then I just try to lay out all of the possible competing theories that might explain what’s going on. So very different from a typical economics paper, where you really hitch your wagon to one particular theory, and then you try to defend it to the death. Instead, I just say, “Look, there are all these theories and either they fit with the data or they don’t. And sometimes at the end of the day, none of the theories fit very well. And sometimes I might have two or three theories that all fit the data. So I can’t even tell the difference. I can’t really distinguish between those two theories.” So anyway, I’ve just found that to be a much more sensible way to write up research, but not very many of our colleagues have followed my lead on that one. And I rarely see a paper written up that way.

LEVITT: You once told me that your mom was Polish and your dad was from Uruguay. And they were both immigrants. What was it like growing up in that kind of multicultural family?

MIGUEL: I loved it. It’s a different background than most other folks. I think most other immigrants — they have the American experience outside the home, and then they have a particular experience from the home country within the home. But because my folks are from different countries and different continents, and they always had friends from many different countries, it was like a little bit of the United Nations at home — lots of languages, lots of foods, and just a tolerant feeling. And I’m so grateful for that. 

I also got to travel to visit family in South America and in Eastern Europe. And of course, I was a kid in the ‘70s and ‘80s when Poland was still communist, and that was a different experience. It really illustrated how different our lives were in the U.S. Same thing going to Uruguay. I started going to Uruguay when Uruguay was still a military dictatorship. And so I saw a lot of different things at a young age and I always wonder why is my life so different from my first cousins? And it gave me a different, maybe a broader, perspective than some other people.

LEVITT: How are you raising your own kids? Are you trying to reproduce that multicultural environment?

MIGUEL: I really hope so. I think the Bay Area environment is pretty diverse, which is great. But I think over time, what many of us find — children of immigrants and grandchildren of immigrants — it’s a little harder to keep the direct connections to the home countries in the same way. But I certainly have traveled with my kids. I’ve actually taken my kids both to Poland and Uruguay. And the most memorable thing we’ve done is I took my kids to an elementary school in Kenya a couple of years ago when they were still in elementary school. And it was special for them to see kids their own age, obviously in a different place. They were all curious about each other. And I do hope it has a positive effect on just opening their mind to the world.

If you like what you heard from Ted Miguel today, check out his book, it’s called Economic Gangsters and it’s co-authored with another one of my favorite economists, Ray Fisman. If you buy the hardcover version, you’ll see my blurb read on the front and it perfectly captures my opinion of the book. It reads: “Rarely has a book on economics been this fun and this important.” Also, if you’re interested in learning more about corruption and how it’s measured, there’s a new Freakonomics Radio episode out, and it’s called “Is the U.S. Really Less Corrupt Than China?”— and it might give you a clue about why those American diplomats in London weren’t much better behaved than some of their U.N. counterparts in New York.

*      *      *

People I (Mostly) Admire is part of the Freakonomics Radio Network, which also includes Freakonomics Radio, No Stupid Questions, and Freakonomics M.D. This show is produced by Stitcher and Renbud Radio. Morgan Levey is our producer and Jasmin Klinger is our engineer. Our staff also includes Alison Craiglow, Greg Rippin, Tricia Bobeda, Emma Tyrrell, Lyric Bowditch, Jacob Clemente, and Stephen Dubner. Theme music composed by Luis Guerra. To listen ad-free, subscribe to Stitcher Premium. We can be reached at pima@freakonomics.com, that’s P-I-M-A@freakonomics.com. Thanks for listening.

LEVITT: I didn’t name names because this particular daughter does not like it when at dinner parties I begin talking in her presence about this five-inch worm that was inside of her. 

MIGUEL: I can imagine.