A Good Instrument Is Hard to Find

(Photo: kelsey e.)

Phoebe Clarke recently posted a Deadspin article about an article that we just published in The Journal of Socio-Economics. The article, “The Chastain Effect: Using Title IX to Measure the Causal Effect of Participating in High School Sports on Adult Women’s Social Lives,” adopts an ingenious methodology pioneered by Betsey Stevenson (whose research is frequently featured here) in her 2010 study “Beyond the Classroom: Using Title IX to Measure the Return to High School Sports.” Stevenson estimates the effects of participating in high school sports on women’s economic lives, and finds that sports participation leads women to attain higher levels of education and earn more. I apply the same methodology to social outcomes, and find that sports participation causes women to be less religious, more likely to have children, and, if they do have children, more likely to be single mothers.

As explained in the Deadspin piece, estimating the effect of sports participation is more difficult than it sounds. Comparing women who participated in sports to those who did not does not generate causal estimates:

Women choose to participate in sports, and their decision is likely a function of pre-existing individual characteristics. Some of these characteristics, such as race, socio-economic position, and school quality, can be controlled for. Others, such as ambition, self-esteem, and ability, cannot, because measures for such intangibles are not readily available.

If these unmeasurables affect not only a woman’s decision to play sports, but also her behavior later in life, the differences between ex-athletes and non-athletes cannot necessarily be attributed to sports participation.

Ideally, economists would avoid this self-selection problem by randomly assigning women to two different groups: one that plays sports and one that does not, just as scientists assign patients to “treatment” and “control” groups when testing a new drug. Once randomized, women in the sports group would be expected to have the same levels of unmeasurables than those in the non-sports group such that differences between the two groups could be attributed to athletic participation. However, unfortunately for economists, randomizing sports participation is less practical than randomizing drug treatments.

Stevenson’s solution (and the one that Phoebe and I redeploy) uses the passage of Title IX as a natural experiment. The methodology effectively splits women into two groups (sports and non-sports) depending on whether they went to school before or after Title IX came into full effect—in the same way that the randomized experiment would. It also avoids any self-selection issues that remain within this natural experiment by using a creative proxy: the rate of male sports participation in the woman’s state just before Title IX was passed. Why is this a good “instrument” for female sports participation?  Stevenson shows empirically that the rate of male participation before Title IX corresponds strongly with post Title IX increases in female participation.  Title IX required states to provide girls with the same opportunities to participate in sports as boys, thereby placing pressure on states to bring their female sports participation levels up to their male levels. Women in states with 90 percent historic male participation who went to high school after Title IX were more likely to see increases in female sports participation than those living in states with 20 percent historic male participation.

And crucially, the instrument satisfies the exclusion criterion.  As a theoretical matter, the rate of variation in historic male participation is not related to post Title IX changes in male participation except through its impact on the Title IX variable.  A good instrument (which satisfies this exclusion criterion) is hard to find.  Good theorists can almost always come up with a story for why an instrument fails the criterion (and if pushed, I can think of some even here), but Betsey’s instrument is about as good as they come.

The abstract of our article captures its central results:

We find that a ten percentage-point increase in state-level female sports participation generates a five to six percentage-point rise in the rate of female secularism, a five percentage-point increase in the proportion of women who are mothers, and a six percentage-point rise in the proportion of mothers who, at the time that they are interviewed, are single mothers. While our results appear to paint a picture of independence from potentially patriarchal institutions (church and marriage), further research is necessary to understand whether our results can be attributed to a single story such as this one or whether they are the products of multiple causal mechanisms.