To Test or Not to Test

Many folks always ask me what the impact of randomized trials are on development. We at Innovations for Poverty Action and the M.I.T. Jameel Poverty Action Lab are dedicated to randomized trials to help push forward evidence-based policymaking. Yet what is the evidence that evidence shifts views? Not always so easy to do. I’ve done some work on the donor side, which I’ve reported on here before.  Here is a meta-study that uses two of my studies that found fairly different results. One found that access to credit in South Africa led to increased income, the other found that access to credit in the Philippines had no discernible impact on income.

The researchers sent off about 1,500 mailers to microfinance institutions around the world, telling them about the positive study, the negative (or non-positive, technically) study, or a placebo (no mention of a study), and asked them if they wanted to participate in a randomized trial to measure the impact of their organization.  They then saw which microfinance leaders responded, and whether they responded favorably or negatively.

The choice of my research I assume was simple: I have results that contradict themselves (in a simple outcome measure of income; I’m not sure I’d say they actually contradict each other once context is taken into account). So this allowed them to hold researcher identity constant, to keep as much the same across treatment groups. Glad Jonathan Zinman and my potentially confusing results could serve a greater purpose!

Check out here for the results here.

Bottom line: hearing about positive results elsewhere helps bring in more partners for more research. Negative results drives folks away.

(HT: Chris Blattman)

Leave A Comment

Comments are moderated and generally will be posted if they are on-topic and not abusive.

 

COMMENTS: 1

View All Comments »
  1. Jeff Rold says:

    Recently there has been a lot of press, especially in business media, about the decline in mortgage applications and home sales. This decline has been attributed solely to the increase in interest rates (particularly government bond rates). There’s a great deal of evidence, both anecdotal and statistical that indicates the real reason is the extreme restrictions banks and mortgage originators have placed on potential borrowers – for both new mortgages and re-financings. A near perfect credit score, perfect mortgage payment history (in some cases going back 30+ years), huge liquid assets in excess of the desired mortgage, abundant cash flow, property appraisal nearly double the requested mortgage doesn’t guarantee success. This process drags on for many months and requires mountains of personal financial documentation, some as insignificant as a few hundred dollars, and some going back 35 years. It’s a near impossibility for most people and many get tired of the process and just don’t proceed with the financing out of frustration. I’ve spoken to a number of people who confirm all of what I’m saying. I conclude this must be having a significant negative affect on the overall economy as well as the housing market. Any thoughts?

    Thumb up 0 Thumb down 0