Are Predictions Getting Better?

If you’re the kind of person who cares about “The Folly of Prediction” and The Signal and the Noise, you may want to read Amy Zegart‘s Foreign Policy piece about predictions. Making predictions within the intelligence community, for example, is a different game than betting on basketball:

In March Madness, everyone has access to the same information, at least theoretically. Expertise depends mostly on how geeky you choose to be, and how much time you spend watching ESPN and digging up past stats. In intelligence, however, information is tightly compartmented by classification restrictions, leaving analysts with different pieces of data and serious barriers to sharing it. Imagine scattering NCAA bracket information across 1,000 people, many of whom do not know each other, some of whom have no idea what a bracket is or the value of the information they possess. They’re all told if they share anything with the wrong person, they could be disciplined, fired, even prosecuted. But somehow they have to collectively pick the winner to succeed.

In other spheres, however, predictions just keep getting better. “Smart people are finding clever new ways of generating better data, identifying and unpacking biases, and sharing information unimaginable 20 or even 10 years ago,” writes Zegart.

Joe Dokes

The short answer is no, predictions at least in the political arena are not really getting better. For example, how many people predicted the Arab Spring, and just as importantly the Arab Fall (i.e. the clear move away from democratic institutions that seemed so inevitable at the height of the Arab Spring)?

We can't even predict the fallout from the Eurozone Crisis and the financial issues created and exacerbated by the PIGS.

We can't even predict if the government is going to get its stuff together and prevent the sequester.

Elections are easy to predict, particularly in a stable country like the US.



I think most of us predicted the Arab Fall, even if we didn't come up with the catchy name for it.

tung bo

Inexplicably, Zegart relegated the fact that the "intelligence" business have no clear cut metrics to the fourth factor. I'd thought that in any venture, that should be the #1 factor! It reflects the ill defined and changing nature of the project. If we don't know what the terms we use mean, what good is getting mountain of data?

"Is Al Queda advancing or declining?" Who is meant by Al Queda? Does it include self proclaimed affiliates? What if a group claimed to be an affiliate is running a very popular clinic for refugees? Is that an advancement of their brand? Or is that a mutation of their focus and energy which makes the group more managable?

With the ill-formed questions that politicians often asks, it is no wonder that there are often no sensible answers to them. Not to speak of prediction even.