To Test or Not to Test

Many folks always ask me what the impact of randomized trials are on development. We at Innovations for Poverty Action and the M.I.T. Jameel Poverty Action Lab are dedicated to randomized trials to help push forward evidence-based policymaking. Yet what is the evidence that evidence shifts views? Not always so easy to do. I’ve done some work on the donor side, which I’ve reported on here before.  Here is a meta-study that uses two of my studies that found fairly different results. One found that access to credit in South Africa led to increased income, the other found that access to credit in the Philippines had no discernible impact on income.

The researchers sent off about 1,500 mailers to microfinance institutions around the world, telling them about the positive study, the negative (or non-positive, technically) study, or a placebo (no mention of a study), and asked them if they wanted to participate in a randomized trial to measure the impact of their organization.  They then saw which microfinance leaders responded, and whether they responded favorably or negatively.

The choice of my research I assume was simple: I have results that contradict themselves (in a simple outcome measure of income; I’m not sure I’d say they actually contradict each other once context is taken into account). So this allowed them to hold researcher identity constant, to keep as much the same across treatment groups. Glad Jonathan Zinman and my potentially confusing results could serve a greater purpose!

Check out here for the results here.

Bottom line: hearing about positive results elsewhere helps bring in more partners for more research. Negative results drives folks away.

(HT: Chris Blattman)