The answer to the quiz on beer prices and violence.

The winner of the quiz goes by the tag KZ. If I can find out more about KZ, I will post that info.

KZ recognized that there do not seem to be any controls for a time trend in the paper. While the authors control for all sorts of other factors like unemployment and seasonality, I think they have left out potentially the most important confounder.

Why is it important to include a time trend, or even better, a separate “dummy” variable (i.e. a variable equal to 1 if it is say 2003 and zero otherwise)? Because my guess is that a lot of the variation in beer prices is coming from nationwide shifts over time. All the fancy econometrics in the world won’t change the fact that if the variation in beer prices that they are using is coming from national shifts over time, then ultimately what they are doing more or less boils down to plotting one point per year on a graph, with the X axis being the national beer price and the Y axis being national levels of violence. You don’t need 30 pages of econometrics, you need one paragraph and a graph if that is true.
My strong conjecture — and it is only a conjecture because my quick internet search did not turn up the data to do it — is that if you plot those few points on the graph and compute the slope, you will get an answer very similar to what is in the paper: a 1% increase in beer price lowers violence by 2%. For someone who really has extra time on his or her hands, I would be interested in seeing the answer.

My second strong conjecture is that if the authors simply add year dummies to their specifications, their standard errors will explode and all their results will disappear. My guess is that there will not be any real price variation left.

Economists are rightfully skeptical of correlations that are based purely off national time series data because so many things can be going on at once that are not controlled for. What is dangerous about this paper is that it gives the strong impression that it is using a different sort of variation (it seems like it is comparing idiosyncratic fluctuations in beer prices across regions from year to year) that economists tend to like, but really it is usually a completely different source of variation that economists tend to hate. To figure that out, though, you really have to dig into it.

So what was it about reading the article that made me sense something might not be as it seems? First, I’ve played around with beer price data from the U.S. and I know there is not much variation there of the kind these authors seemed to be using (but really weren’t). Second, the results were too good and too consistent. When they ran very different specifications like “fixed effects” and “random effects” models (these terms will mean nothing to most readers, but rest assured they are very different), the results were almost unchanged, as were the standard errors. That made me think that they couldn’t be only using the minute differences in prices that would occur within a place over time.

There may very well be a strong relationship between beer prices and violence. Dubner and I have even blogged about a fascinating paper on grain prices and crime in 19th century Germany which likely operates through beer and written about this relationship in one of our New York Times columns. The paper that was the subject of the quiz, however, probably tells us little about the question it tries to answer.