The Folly of Prediction, Cont'd.

Our "Folly of Prediction" podcast included an interview with Joe Prusacki, who directs the statistics division at the USDA’s National Agricultural Statistics Service. This means he helps make crop forecasts (read a primer here). As hard as the USDA works, the fact is that predicting the future of even something as basic as crop yield can be maddeningly difficult. The Wall Street Journal has the latest in an article headlined "Erroneous Forecasts Roil Corn Market":

Government reports about the U.S. corn crop have become increasingly unreliable of late, contributing to wild swings in corn prices, a Wall Street Journal analysis shows.

Over the past two years, the Department of Agriculture's monthly forecasts of how much farmers will harvest have been off the mark to a greater degree than any other two consecutive years in the last 15, according to a Journal analysis of government data. This year's early-season forecasts also appear to have been way off. The next monthly report is due on Friday.

This Week in Corn Predictions: The USDA Got it Right (Almost)

We've been having some fun recently at the expense of people who like to predict things. In our hour-long Freakonomics Radio episode "The Folly of Prediction" -- which will be available as a podcast in the fall -- we showed that humans are lousy at predicting just about anything: the weather, the stock market, elections. In fact, even most experts are only nominally better than a coin flip at determining a future outcome. And yet there remains a huge demand for professional predictors and forecasters.

Earlier this week, Stephen Dubner and Kai Ryssdal chatted about this on the Freakonomics Radio segment on Marketplace. The question remains: "should bad predictions be punished?"

As mentioned in the segment, the U.S. Department of Agriculture's August crop yield report came out today. The result? Not bad actually. The corn yield forecast was revised downward by just 1.3% from its estimate last month. That's a considerable improvement over last year's big miss, when the August corn yield report had to be revised downward by almost 7%.

Should Bad Predictions Be Punished?

What do Wall Street forecasters and Romanian witches have in common? They usually get away, scot-free, with making bad predictions. Our world is awash in poor prediction -- but for some reason, we can't stop, even though accuracy rates often barely beat a coin toss.

But then there’s the U.S. Department of Agriculture's crop forecasting. Predictions covering a big crop like corn (U.S. farmers have planted the second largest crop since WWII this year) usually fall within five percent of the actual yield. So how do they do it? Every year, the U.S.D.A. sends thousands of enumerators into cornfields across the country where they inspect the plants, the conditions, and even "animal loss."

This week on Marketplace, Stephen J. Dubner and Kai Ryssdal talk about the supply and demand of predictions. You'll hear from Joseph Prusacki, the head of U.S.D.A's Statistics Division, who's gearing up for his first major crop report of 2011 (the street is already "sweating" it); Phil Friedrichs, who collects cornfield data for the USDA; and our trusted economist and Freakonomics co-author Steven Levitt.

Corn in My Coffee, Lead in My Pot

Doctors at several hospitals in Leipzig, Germany, could not figure out the cause of a recent rash of lead poisoning. Was there an environmental disaster underway? They kept seeking the source and, after several weeks, as they write in the New England Journal of Medicine: … we detected a common pattern: the patients were young, […]