Hypotheses for an Impact Study on a For-Profit Microlender

Through Innovations for Poverty Action, I am co-Principal Investigator on a randomized trial of the impact of Compartamos, a for-profit microlender in Mexico. Compartamos was the first microcredit organization to go public, and at IPO time had a market capitalization of US$1.5 billion.  Needless to say, that created a lot of buzz.  Several years later, we will soon be finishing a randomized trial to measure the impact on communities in the Nogales area in northern Mexico. We will be posting our hypothesis before we do the analysis, and encourage readers to do the same, for three reasons:

1. Avoid data mining: Pre-specifying hypotheses is a helpful way of addressing a data mining concern when multiple outcomes are measured. By pre-specifying primary and secondary outcomes, it is transparent whether 5 out of 10 hypotheses came true or 5 out of 100. With proper statistics, 5 out of 10 is not a fluke, but 5 out of 100 is exactly what one would call a fluke and is a kin to finding no changes at all.  So pre-specifying is helpful: when it comes time for publication, people reading the research know that any significant results found were not simply as a result of mining the data for the magic significant relationships. Development economists have been slow to do integrate this step into our research; folks have pointed this out in several posts around the web (see here and here, for example). A registry system is indeed underway, and we’re quite supportive of this.
2. Comparing results to prior opinions: The goal of research is to produce knowledge. But we aren’t in a vacuum beforehand, and we of course have our opinions. Suppose we are measuring where something is on a scale of -10 to 10. We have a prior opinion that it is 7. We lack evidence on that, so we are uncertain, but we still have an opinion that it is 7. If the result comes out and is 7, of course the study should not change our opinion. If on the other hand the result comes out and is 4, or 0, or -3, we ought to update our opinion at least a bit, perhaps not all the way. In this spirit, we are also welcoming the posting of predictions, preferably signed, so that if you are right you can show the world “I told you so” and if not, well then maybe it’ll encourage you to shift your opinion a bit. If this works, maybe we’ll start a website called www.itoldyouso.com (although that URL is taken by someone not using it).
3. Crowdsourcing:  We’d like to hear from people their thoughts on which of the outcome measures they think will be particularly important. A simple way of engaging with the microcredit community ex-ante, rather than ex-post, with a hope of creating a better dialogue than one typically sees on studies after the results are posted. Plus we’re interested to hear thoughts from people, that may shape our analysis (although pre-specifying the hypotheses we’ll post of course!).

You can look forward to more blog posts with our hypotheses on different subjects, coming out in the next few weeks.

The Study

IPA has partnered with Compartamos Banco in Mexico to evaluate the social and economic impact of Crédito Mujer, their principal village banking loan product. The product offers individual women access to credit from $1,500 to $27,000 Mexican pesos (1 US$ = 12.8 Mexican pesos). Compartamos employs a group process, with women organized in groups of ten to fifty to act as solidarity guarantors.

This study took advantage of Compartamos’ decision to open three new branches in northern Sonora, where it had not previously operated. Within the region, we created 250 clusters and randomly assigned 125 of them to receive direct promotion of Crédito Mujer, while the remainder served as the control and did not receive any direct promotion. To ensure differential take-up between treatment and control groups, Compartamos restricted loan access exclusively to treatment clusters; loan officers, coordinators, and branch managers were responsible for physically verifying the addresses of all potential clients before they could take out credit.

After Compartamos offered credit to women in northern Sonora treatment areas for at least 18 months, we launched a survey to measure the impact of Crédito Mujer, through a survey which asks respondents’ about income, well-being and businesses. We started fieldwork on surveys in November, and plan to complete interviews with 16,500 women by the end of March.

The Sample Population

The women in our sample range in age from 18-60 years old. At the time of our baseline survey in 2010, the average age was 41 (st dev: 15). Half of the sample was married, and the women had, on average, 1.1 children (st dev: 1.1).

Before Compartamos Banco began marketing, according to our baseline survey, 24.5% of the sample owned a business. Two-thirds of the sample reported that they were unlikely to take out credit in the next 6 months, and one-quarter said they were likely to take out credit. 18.9% reported having taken a loan in the previous year, with the majority of those loans (44%) coming from a bank or finance institution, and the rest coming from other sources, like moneylenders, relatives, store credit, or a friend. Compartamos Banco’s Crédito Mujer product introduced as a new opportunity for the majority of the sample.

The Survey Instrument

Here is the survey instrument:

• in English (pdf)
• in Spanish (pdf)

There are 22 main sections:

I. Personal characteristics
II. Household population
III. Health
IV. Household characteristics
V. Children
VI. Migration inside and outside Mexico
VII. Household consumption and assets
VIII. Savings
IX. Household assets
X. Business information
XI. Business experience
XII. Overall satisfaction
XIII. Income
XIV. Social networks
XV. Community and political engagement
XVI. Decision-making
XVII. Locus of control
XVIII. Unexpected expenses and events
XIX. Sources of credit
XX. Bank account
XXI. Credits
XXII. Mood in the last week

Feedback from the Microcredit Community

Please email or post one or two of the following:

1. Which outcome measures do you consider to be the most important, i.e., the primary ones on which the overall success (or failure) of the program should be judged?
2. Do you have any specific predictions on any particular outcomes? Or, put it the other way, are there any outcome measures for which if the results are outside of a certain range you will change your opinion?