Sign Up for a Prediction Tournament

Photo: Spike Mafford

You may remember Phil Tetlock from our Freakonomics Radio hour-long episode “The Folly of Prediction.” He’s a psychologist at Penn and author of the deservedly well-regarded book Expert Political Judgment. Tetlock and some colleagues have embarked on an ambitious new study of prediction — and even better, they’re looking for volunteers. Specifically, they’re looking for people “who have a serious interest in and knowledge about world affairs, politics, and global economic matters and are interested in testing their own forecasting and reasoning skills.”

Doesn’t that sound like you? You need to be 18 or older, with a college degree. The project even pays a small honorarium. The start date has been pushed back to September, so you better act fast if you want in.

Here’s more information from Tetlock and colleagues:

 

We are writing with an unusual but, we hope, intriguing request. We are in the process of recruiting knowledgeable people to participate in a quite unprecedented study of forecasting sponsored by the Intelligence Advanced Research Projects Activity (“IARPA”) and focused on a wide range of political, economic and military trends around the globe. The goal of this unclassified project is to explore the
effectiveness of techniques such as prediction markets, probability elicitation, training, incentives and aggregation that the research literature suggests offer some hope of helping forecasters see further and more reliably into the future.

To join, go to www.goodjudgmentproject.com. You do not need to read anymore of this message, but, if you are curious, further details follow.

***

There are several teams recruiting participants. Ours builds on Phil Tetlock’s work described in his book Expert Political Judgment; his co-PIs are Barb Mellers and Don Moore, with an advisory board that includes Daniel Kahneman, Robert Jervis, Scott Armstrong, Michael Mauboussin, Carl Spetzler and Justin Wolfers. It involves a multi-disciplinary effort to understand how people use the knowledge they have to develop expectations about the future and what sorts of processes and strategies lead to success.

We need to recruit as many as 2500 people who have a serious interest in and knowledge about world affairs, politics, and global economic matters and are interested in testing their own forecasting and reasoning skills. So please consider visiting the project web-site at goodjudgmentproject.com. You will find what you need to register there and more information about the project.

We are committed to maintaining high standards for admission to this special program. And we would greatly welcome your participation if you are so inclined (please be advised that the minimum time commitment would be several hours in passing training exercises, grappling with forecasting problems, and updating your forecasting response to new evidence throughout the year).

The primary motivation for participating should be Socratic: self-knowledge. But we can also offer a token honorarium of $150 to each participant for completing each year in the forecasting tournaments. And, although all participants will be given key anonymity, the winners of the forecasting tournaments will know who they are and will be free to go public if they so wish.

Of course, we understand if you yourself do not have the time to engage in an exercise of this sort. But we would be very grateful if you are willing to pass this request on to colleagues and readers of your work who might be interested in participating in this program and who would be likely to qualify for admission.

We can promise the following: this will be an intellectually stimulating experience (indeed, should you be bored, you should drop out); participants will have the opportunity to work with state-of-the-art techniques (training and incentive systems) designed to augment accuracy; and participants will receive feedback on how well calibrated (among other things) their subjective probability judgments are in relation to others for various categories of problems.

In short, we think it will be fun. If we were not running it, we would volunteer ourselves.

Many thanks for considering this, and for passing this invitation on to others who might also be interested.

Philip Tetlock, Psychology and the Wharton School, University of Pennsylvania
Barb Mellers, Psychology and the Wharton School, University of Pennsylvania
Don Moore, The Haas School, University of California, Berkeley

Leave A Comment

Comments are moderated and generally will be posted if they are on-topic and not abusive.

 

COMMENTS: 10

View All Comments »
  1. Craig Willis says:

    Sounds very interesting although one cannot take predicting the future purely in terms of your knowledge of a particular subject. Especially politics and World affairs.

    The accuracy of predicting political outcomes may have as much to do with the persons influence in that particular area than their ability to apparently predict the future. For example, an academic on the subject may have less chance of predicting a revolution than say an office worker that is actively promoting said revolution through social networks.

    What will be interesting is how the participants predictions are shaped by their environment and past experiences. Unfortunately I am not a US citizen but I look forward to hearing the results!

    Thumb up 3 Thumb down 0
    • Joshua Northey says:

      Well said. Almost all the best predictions have an aspect of insider trading to them.

      Thumb up 2 Thumb down 0
      • Craig Willis says:

        And of course in technology predictions, a well known commentator is essentially throwing down the gauntlet to the technology enablers. :)

        Thumb up 0 Thumb down 0
  2. niel says:

    Why not open to non-US residents or citizens?

    Thumb up 2 Thumb down 0
  3. avinash says:

    The official website (http://www.goodjudgment.info) doesn’t say anything on the US citizenship requirement. Please do confirm!!!

    Thumb up 0 Thumb down 0
  4. Hemant says:

    No, you do not need to be a US citizen. Freakonomics just copy-pasted from an earlier post. That requirement has been removed now.

    http://www.freakonomics.com/2011/03/29/calling-all-predictors-to-a-new-forecasting-tournament/

    Thumb up 1 Thumb down 0
  5. Jaime says:

    I am so so so saddened by the “only U.S. citizen requirement”.

    Is it like IARPA thinks that only those of U.S. citizenship have the required influence to affect prediction markets.

    Thumb up 0 Thumb down 0
  6. Eveline says:

    So, only a degree is needed to enroll? No checks on native speakers or something like that?

    How well you predict will be influenced by how well you understand the question, I guess…

    Thumb up 0 Thumb down 0
  7. leo Piccioli says:

    Why do americans get all the fun???
    I am from Argentina and meet 99% of the requisites…

    Thumb up 0 Thumb down 0
  8. Jacob Silver says:

    Generally speaking, if a certain class of events follows a fixed pattern of action, then trend analysis of those events will produce a future prediction of high accuracy. However, there are few classes of events with such fixed patterns of action. The moon and tides, the gestation of foeti, are a few of such events, but these would have limited applicability for human strategic or tactical action. And, generally, to see any future, or to live it, is precluded by the Standard Model of physics. If one could engage one’s body, or vehicle in which one’s body is, with time, the only direction you would be able to move would be to the past. Such engagement is immensely difficult, and, if it could be achieved, much havoc with the present could result.

    Thumb up 0 Thumb down 0