Should Bad Predictions Be Punished?

Government corn predictions are based on the work of people like Phil Friedrichs, gathering data in a corn field in Hiawatha, Kansas. (Photo: Stephen Koranda)

What do Wall Street forecasters and Romanian witches have in common? They usually get away, scot-free, with making bad predictions. Our world is awash in poor prediction — but for some reason, we can’t stop, even though accuracy rates often barely beat a coin toss.

But then there’s the U.S. Department of Agriculture’s crop forecasting. Predictions covering a big crop like corn (U.S. farmers have planted the second largest crop since WWII this year) usually fall within five percent of the actual yield. So how do they do it? Every year, the U.S.D.A. sends thousands of enumerators into cornfields across the country where they inspect the plants, the conditions, and even “animal loss.”

This week on Marketplace, Stephen J. Dubner and Kai Ryssdal talk about the supply and demand of predictions. You’ll hear from Joseph Prusacki, the head of U.S.D.A’s Statistics Division, who’s gearing up for his first major crop report of 2011 (the street is already “sweating” it); Phil Friedrichs, who collects cornfield data for the USDA; and our trusted economist and Freakonomics co-author Steven Levitt.

We’ll also hear from journalist Vlad Mixich in Bucharest, who tells us why those Romanian witches might not be getting away with bad fortune telling for much longer.

Here’s where to find Marketplace on the radio near you.


Siddhartha Herdegen

Unless there is a consequence, people will continue to spout blather. Predictions without cost are just opinions with no weight.

Daisy

Watching this obsession with 'predictions' is interesting. What is the average period of time one takes discussing a topic they are obsessed about before they move on? I would 'guess' another 6 months, but never fully moving on for another year and a half.

Defenetly not to be punished, as it is not their fault other people presume they are good at making good predictions.

Tom

I would add all those Fox news blowhards who have been so colossally wrong over and over to the list of those deserving of punishment.

rationalrevolution

FOX spouts so many incorrect predictions because they aren't actually doing any predicting, they are simply engaging in propaganda. That's the issue, is that in most cases the predictions we here in the mass media aren't actually forecasts at all, they are simply propaganda meant to drive behavior.

Eric M. Jones.

When I was young I asked my dad--"If people have a system that can predict something, why don't they just shut their mouths and take all the marbles instead of writing books and giving seminars about it?"

My dad explained that there was obviously more money in getting people to buy your books than in using your own system. Otherwise that's exactly what they would do.

JimFive

Your dad is only partially right. Imagine a poker game with 5 players, each with a $100 stake. The most anyone can walk away with is $500. In order to get more, you have to get more people into the game.

One of the purposes of those books is to get more suckers playing the game so that the professionals can take home more money.

Nosybear

Anyone who believes a prediction, especially concerning the future, is punishing themselves quite enough. The problem is the same as I witness in business processes: We trumpet successful prediction and never talk about the ones that fail. Their punishment? Anyone who makes a public prediction, be it the local weather person or a high-powered Wall Street firm, should be required once the date stamp of the prediction is arrived to publish the actual value next to the predicted value in very big letters. Robust processes survive the future, most predictions do not.

Of course, if you wait long enough, any plausible prophesy will be fulfilled just by random chance.

rationalrevolution

The reason that so many "predictions" are bad is that most of these aren't actually predictions in the first place, they are either A) wishes or B) attempts to manipulate the present.

Look at the housing bubble issue. Look at the "predictions" made by the National Association of Realtors. They said in 2005 that a national housing bubble was "impossible", that housing prices "would not decline nationally", and that the fact that housing prices were growing faster than incomes "did not signal unsustainable price growth".

These were all absurd statements at the time they made them, but they made these statements because they had an immediate interest in maintaining housing sales. So they weren't really making "predictions", they were engaged in a marketing campaign and posing marketing pitches as "predictions".

What you see when you look at predictions made by private, for profit, entities is that the predictions suit their economic interests, so the reason they are wrong isn't that they are poor at making predictions, its that they aren't actually making predictions at all.

This is why, the example that you use of the highly accurate prediction rate for a government entity makes sense. It's a non-profit entity that is largely removed from biases and its not engaged in marketing.

Read more...

John B

You are partially right; your statement "The reason that so many “predictions” are bad is that most of these aren’t actually predictions in the first place, they are either A) wishes or B) attempts to manipulate the present." is right on point.

However, ascribing all good predictions to government and all bad predictions to for-profit entities is way off base.

Government officials constantly use "predictions" to manipulate support for their policies. The obvious example is the $70o plus billion stimulus that was going to "create" millions of jobs. Not a good prediction.

Another example is the annual hurricane precitions by the government. Huge headlines are written when they predict a large number of hurricanes. Even after being wrong several years in a row, the predictions still get headlines. When the hurricanes don't happen, they give excuses which say, in effect, that they were right - but nature was wrong.

Oh, and by the way, government officials said the same things as the realtors about the impossibility of a crash. Think Fannie Mae, etc.

And while most predictors are not punished or fired, it is a good bet that of those who are, none of them work for the government.

Read more...

rationalrevolution

I agree with you, and that wasn't my intention. Certainly there are government agencies and politicians who have incentives to produce biased predictions. Anything that is politicized will end up being biased. The issue with the Ag Dept predictions is not just that its a "government agency", but that its a non-political issue they are doing forecasting on, for which there is no economic incentive for the predictors to introduce biases.

Essentially, in order to get actual "predictions" the folks doing the predicting can't have an incentive to bias the prediction, and there are few cases where this is the case when it comes to political and economic issues.

J Bryk

Good explanation.

Unfortunately, there are very few things which aren't politicized today. But it is refreshing to see how much more accurate the predictions are when they don't have an agenda.

Doug M

Why are there twice as many buy recommend stocks as sell recommended stocks. Is it because a lot more stocks go up than down?

The firm making the recommendation hopes that a buy recommendation will lead to a better banking relationship with the firm being recommended?

The firm making the recomendation has more clients who are long then who are short?

The analyst develops a fondness for the companies that they research and so they want them to go up in price?

Mary

I make a living off of predicting. It started with weather and now includes financial markets. If I'm right, I make money. If I'm wrong, I lose. Very pure way to make a living.

Being wrong is a constant part of the game but we simply need to be right more often than wrong and keep the losses from being catastrophic.

We ignore all other blowhards, doomsayers, pundits, academics, and other so-called forecasters who are publicly offering their opinion. At the same time, we scan the web for those with verifiable track records. We assess the statistical significance of their record and monitor the winners and also monitor folks that are consistently wrong.

In the US of A, stating your prediction is an opinion and the right to do so is constitutionally protected.

Max

IARPA recently launched Forecasting Ace to study how effectively crowdsourcing can be used to predict the future. Most English-speaking college grads are eligible to participate.
http://forecastingace.com

It's not a prediction market, and participants aren't required to put their money where their mouth is, so they aren't punished for making bad predictions.