Why Don’t Sports Teams Use Randomization? A Guest Post

Here’s the latest guest post from Yale economist and law professor Ian Ayres. His past posts can be found here and here.

In a recent post, I mentioned that when playing poker, I use my watch as a crude random number generator to tell me when to bluff. While there are lots of sports in which it’s best to play a somewhat random strategy, that doesn’t mean that every possible play is equally likely. But it does mean, for example, that when it’s third-and-2 in football, the offense wants to have some possibility of passing to keep the defense honest.

Levitt and others have tested the degree to which professional tennis and soccer players are successful at playing randomized strategies. But it remains a mystery to me why coaches don’t have random number generators (any laptop would do) to help them pick the next pitch in baseball, or the next play they will call in football. Norv Turner would pick the probability of running or passing, and then let the computer decide which it would be.

But an even bigger puzzle is why teams don’t exploit the other powerful use of randomization. To my knowledge, no sports team in the history of humankind has ever run a random control trial to figure out which strategies work the best. (I make this extravagant claim in hopes of provoking you all into providing some counterexamples.) Randomized studies are the gold standard of medical testing, and they’re now the hottest thing in Internet ads.

Want to know whether your Web banner for beer should say “Tastes Great” or “Less Filling”? Run a randomized test in which half the people see one and half the people see the other at random, and then sit back and watch whether one ad generates more sales. I ran just this kind of test on Google Adwords to help choose the title of a book (shameless pitch) I was writing. When I started writing it, I loved the title, “The End of Intuition.” But in a randomized test, “Super Crunchers” had a 63 percent higher click-through rate.

So why don’t sports teams run (more) randomized experiments? The Boston Red Sox are famous for relying on number crunching to gain a competitive edge. But why don’t they proactively make some powerful data by creating randomized treatment and control groups? They could use their minor league teams, for instance, to figure out whether catchers or pitchers make better calls.

They could even have a randomized trial of randomization — they could randomly assign the pitches for half the at-bats to be called in the traditional way (by the coach or the catcher) and the other half could be called by a random strategy established in advance. It would be a double-blind study, because neither the pitcher nor the hitter would need to know which system called the pitch.

If it turned out that the random strategy reduced the batting average of your opponents, that would be pretty strong evidence that it was a better strategy.

Or you could run an experiment to find out whether football teams should go for it more often on fourth down. Economist David Romer has crunched numbers to suggest that professional football teams should go for it fourth down a lot more than they currently do. His proposed optimal strategy is summarized in the following graph, found at the end of his paper:

graph

Amazingly, the data suggests that if it’s fourth down and your team has the ball on the opponent’s 33-yard line, you should go for it even if you have 9 yards left for a first down. NFL coaches have resisted Romer’s advice (though Pulaski Academy has started acting on it). But this is another area in which a little randomized testing could go a long way to help figure out what works. There are thousands upon thousands of college and high school games, but we collectively go for decades without figuring out whether simple changes in strategy could really produce better outcomes.

If you know of any randomized tests of sports strategy, please let me know. And if you are a coach and want to run a test, feel free to contact me. I’d be happy to help design and evaluate a test.

Leave A Comment

Comments are moderated and generally will be posted if they are on-topic and not abusive.

 

COMMENTS: 110

View All Comments »
  1. TaintMasterMitch says:

    Really neat post, I’d like to see some tests in sports come out of it. Thanks for putting the time and effort into a thought-provoking read.

    Thumb up 0 Thumb down 0
  2. TaintMasterMitch says:

    Really neat post, I’d like to see some tests in sports come out of it. Thanks for putting the time and effort into a thought-provoking read.

    Thumb up 0 Thumb down 0
  3. Nathan Whitehead says:

    Wow, that is a thought provoking post. I think the actual answer as to why teams don’t use randomization or A/B testing is that they don’t understand the benefits. Humans have an innate bias against randomness and the scientific method. We like to control things and find patterns that make sense, not add randomness and perhaps discover facts that we don’t understand fully.

    Thumb up 0 Thumb down 0
  4. Nathan Whitehead says:

    Wow, that is a thought provoking post. I think the actual answer as to why teams don’t use randomization or A/B testing is that they don’t understand the benefits. Humans have an innate bias against randomness and the scientific method. We like to control things and find patterns that make sense, not add randomness and perhaps discover facts that we don’t understand fully.

    Thumb up 0 Thumb down 0
  5. Barb says:

    Maybe Mike Tomlin could use this advice… Anyone know how to reach him? He’s not hiding from his fans yet, is he?

    Thumb up 0 Thumb down 0
  6. Barb says:

    Maybe Mike Tomlin could use this advice… Anyone know how to reach him? He’s not hiding from his fans yet, is he?

    Thumb up 0 Thumb down 0
  7. Craig Kocur says:

    This post, and last week’s about football coaches not going for it on 4th down, are doing a lot for my ego. For years while watching games I’ve daydreamed about being an NFL coach and going for it on every 4th down (except in the most dire circumstances, 4th and 26 from your own 10 for example). It always struck me that any NFL team should be able to manage a 2.5 yard per play average and longer and sustained drives would do a lot to wear down opposing defenses. Also, offenses would play looser knowing they have a larger margin for error.

    I’ve also thought that sports teams don’t take advantage enough of computers. I’ve had this image in my head for awhile of 2 coaches in the coaches’ box, one pulling randomized plays from a laptop and relaying them to the field and the other inputting the results of plays in another laptop to help make the randomizations smarter by accounting for the variables, down and yardage, defensive formations, wind speed and direction, day or night, real or artificial grass, jersey colors (you never know what influences outcomes until you test for them), etc.

    Thumb up 0 Thumb down 0
  8. Craig Kocur says:

    This post, and last week’s about football coaches not going for it on 4th down, are doing a lot for my ego. For years while watching games I’ve daydreamed about being an NFL coach and going for it on every 4th down (except in the most dire circumstances, 4th and 26 from your own 10 for example). It always struck me that any NFL team should be able to manage a 2.5 yard per play average and longer and sustained drives would do a lot to wear down opposing defenses. Also, offenses would play looser knowing they have a larger margin for error.

    I’ve also thought that sports teams don’t take advantage enough of computers. I’ve had this image in my head for awhile of 2 coaches in the coaches’ box, one pulling randomized plays from a laptop and relaying them to the field and the other inputting the results of plays in another laptop to help make the randomizations smarter by accounting for the variables, down and yardage, defensive formations, wind speed and direction, day or night, real or artificial grass, jersey colors (you never know what influences outcomes until you test for them), etc.

    Thumb up 0 Thumb down 0