When Winning Leads to Winning: A Response

Here are two interesting follow-ups to Tuesday’s post, in which I described how basketball teams who are behind at half-time fare a bit better than might be expected.

First, my friend Lionel Page points me to a related study of his, which analyzes tennis. Lionel uses a similar approach to arrive at a different conclusion, but I think his results point to the same psychological factors. Let me explain.

Lionel analyzes over half a million tennis matches, but focuses on that subset of matches where the first set is played to a tie break (there are about 70,000 of them). In fact, he focuses on those matches where the first set is awfully close to a draw — a long and drawn-out tiebreak, lasting more than 20 points. But eventually someone wins, and it turns out that player is much more likely to win the second set. And so, winning leads to winning in tennis, but losing leads to winning in basketball.

Are these results in conflict? I don’t think so: the effect of falling a little behind in a tennis tiebreak is very different to basketball. In tennis, falling behind in a tiebreak means losing the entire set, which is a big hurdle to overcome. Recall that Berger and Pope ran laboratory experiments in which they tried to figure out the (de-) motivating effect of being a little behind, a long way behind, tied, a little ahead, or a long way ahead. They found that being a little behind led to a burst of extra motivation, which describes the basketball finding nicely. But being a long way behind led to no extra motivation, and so the tennis player who just lost the first set doesn’t get these motivating benefits. In fact, Lionel’s finding suggests that it might actually be de-motivating. Because basketball scores are continuous, and tennis outcomes are so sharply discrete, the psychological impacts of small differences in early play are very different.

Second, Freako commenters showed that they are among the most demanding consumers of statistical analyses, and were pretty vocal in what they perceived to be shortcomings in the Berger-Pope analysis (and see more from Andrew Gelman, here). I passed on your comments to the authors, and here’s their response:

In our sample of N.C.A.A. basketball games, teams that were losing by one point at halftime were more likely to win the game than teams winning by one point. This is indisputable. However, is this finding statistically significant given the noise in the data?

The key issue here is understanding exactly what hypothesis we are testing to see whether losing by a small amount increases motivation. Directly comparing the winning percentage of teams down by one with teams up by one is problematic because these are different types of teams in different situations. Teams are not randomly assigned to be up or down by one. On average, teams down by one tend to be worse (they have a lower season winning percentage). Furthermore, it is mechanically harder for teams down by one to win. They have to score at least two more points than their opponent to win, while their opponents can win even if the teams trade baskets the rest of the game.

This means that we shouldn’t expect teams down by one to win 50 percent of games. What should be expected? This is where reasonable people may begin to differ on the right way to construct a counterfactual. Many different curves can be fitted to the data. One may argue (as many did in the comments) that a linear line should be fitted; Andrew Gelman suggests a logistic function. It ends up that it doesn’t really matter what curve is fit.

For example, consider the figure below (the exact figure requested by Andrew Gelman) which indicates the winning percentage for the home team as the halftime point difference for the home team ranges from -10 to 10. Also, note the inclusion of standard error bars for sophisticated readers. The dotted line represents the fitted curve from a simple logistic function when including the halftime score difference linearly. Focus on the winning percentage when either the away team was losing by a point, or the home team was losing by a point. In both of these situations, the losing team did better than expected. For example, when the home team is ahead by one point, they end up only winning 57.5 percent of games while we would have expected them to win 65.6 percent of games. This difference in actual versus expected performance (8.1 percent) is statistically significant at conventional levels and provides evidence in favor of our hypothesis that losing can be motivating.

INSERT DESCRIPTION

 

This difference persists when controlling for home-team advantage, possession arrow to start the second half, prior season winning percentage, and team fixed effects (see Table 1 of the paper).

Further, supplementary analyses show that teams losing by one point closed the gap the most in the first few minutes after halftime (supporting our motivation hypothesis). Laboratory studies, using random assignment, also demonstrate that merely telling people they are slightly behind halfway through a competition leads them to exert more effort.

Taken together, these findings indicate that being slightly behind motivates people to work harder and be more successful.

Finally, let me say just why I like this paper. It’s easy to mine sports data to find interesting anomalies, but sometimes it is hard to see what it means. And it is easy to get students in an experimental setting to do weird things that have no relevance to the real world. It is the juxtaposition of suggestive data from the field with a well-designed experiment that leads me to conclude there’s some interesting social science here.

My friends at the Association for Professional Basketball Research have collected some interesting discussion threads on this controversy, here, including data suggesting a similar pattern in the NBA.


kman

Looking at this graph, I see that if a team is behind by 7 points at halftime, it has a better chance of winning than if it's behind by 6 points (or 5). How do the authors explain that? If that's just noise, why is the -1 vs. +1 not noise?

I also think they need to normalize for who gets the ball in the second half first. If the team behind gets an extra possession, their chances of winning while down by 1 at halftime increases quite a bit I think.

jonathan

Perhaps this post is an example of how larger review may improve work. This material should be in the paper. In about 2 pages of printed text, it addresses the most obvious statistical objections. Clarity, gentlemen, clarity.

Peter Douglas

In tennis, even if the result of the first set has no psychological effects, if there is a 50/50 chance of winning the next set, and then again on the third set, the chances are that the winner of the original set will win the match. Once one set is won, there is simply less work needed to win the match.

Does that explain the discrepancy?

Snidely

I would be interested to see whether other sports with a half time break had similar outcomes. Do football games have a loser's advantage when one team is down by 3 at the half? You could contrast this with baseball in the 5th inning and a team down by a run. Does the half time pep talk by the coach show any positive impact over baseball where there is no break?

In tennis, the psychological factors of being alone on the court with no coach and no breaks are far more difficult to overcome than a 1 point margin in a team game.

Mike

How abaout a little humor. As with all statistical analyses, points of view may be different and still valid. However, I think we can all agree that the frequency of winning a game with the score tied at haltime for each team is a solid 50%.

Kyle

Another cool graph that could be distilled from this data is final score difference given halftime score difference. It would be interesting to see the distribution of final scores.

Also, could someone point me to a reference for calculating the standard errors for this type of data? It's been a while since my last stats class and I'm having a hard time visualizing what the standard error means for binary data points (wins, losses).

Dave

@Mike (#5)

I don't think the chances should be 50/50 at halftime as you suggest. The home team should have a slight psychological/emotional advantage (although that could be debated as well). The data fits this hypothesis although the advantage seems quite strong in this data set.

Interesting read here: http://en.wikipedia.org/wiki/Home-field_advantage#Factors_of_home_advantage

BC

Standard error is calculated as the standard deviation divided by the square root of the number of observations. Extracting anything meaningful from the 'standard error bars' is impossible without knowing the number of observations. So congratulations on telling us nothing, but using fancy symbols and words to do it.

Peter

Tennis and Basketball are also fundamentally different because one is a team sport typically played in a season and the other is an individual sport typically played in tournaments.

There is a motivation for a basketball coach to play his team at less than 100% in every game except the championship, because he wouldn't want to risk injuring his best players. In other words; if you're coaching the old-school Bulls and your up by enough, why even put Jordan in the second half? Particularly because a single loss won't ruin your season.

What motivation would a tennis player have to ever play at less than 100%, when if they lose any match they are out of the tournament?

MRW

This graph is much, much better.

However, there are several other points that deviate from the fit by comparable amounts to the +1 and -1 points.

asmart

Agree with comments 1 and 10. I think the author is missing the important point. This plot, though it represents a great improvement over the original, does not present compelling evidence that the regression discontinuity model does a better job of explaining the data than the logistic model. Several points, not just +/- 1, appear to be outliers in want of explanation.

jagorev

Chad Orzel wrote an extremely good response excoriating the earlier graph:

http://scienceblogs.com/principles/2009/03/teams_who_are_ahead_win_more_f.php

Raymer Díaz

About the second paragraph, kman, I have my reserves. Because basketball`s own dynamics, extra possession really doesn`t matter because of the steals in a game, which can turn a game around in a matter of seconds, even between possessions.

In this sense, I think a good variable to consider is the amount of steals in a game for the winning teams that trail by one point. I haven't read the paper yet and don't know if this statistic is considered, but I think a motivated team would increase efforts when trailing by one point, so the amount of steals (and maybe rebounds) will increase as well.

Jim

The study itself may well be statistically accurate and the authors may have a valid conclusion, at least with respect to the dynamics of NCAA basketball games. Many of the comments to the original article, mine included, point out suspected statistical errors. In truth, my statistical skills are too rusty for me to fairly evaluate the technical merits of the study. The authors are much better statisticians than I and probably much better than most of the others who commented. They will certainly have reasonable responses to any technical objections to the statistics, as today's article demonstrates.

On the other hand, it is clear to me is that the authors very deliberately presented the data so as to emphasize their conclusion. They consciously decided to exclude 0 from the Trend line data. They used a 5th order polynomial when a lower order polynomial probably would have fit just as well but would not have shown such a dramatic discontinuity. They projected the Trend lines to -.5 and +.5 when they had actual data for -1, 0, and 1, again increasing the appearance of a discontinuity. In graph 1B of the academic paper they chose to display the range -3 to +3, which just happens to be the range that maximizes the appearance of discontinuity. While their conclusion may be perfectly valid statistically, the added emphasis in the presentation suggests authors who are more interested in supporting their thesis than objectively reporting the results of their study.

The authors would be well served to recall that statistical significance does not prove causation in fact. At most, the data suggests there might be a very small winning effect from being behind by one. Given the weakness of the findings and noting that there are other outlier points on the graph which are not explainable by the authors' thesis, it seems that the most that could be said is that there is a possibility that something interesting might be going on.

On the other hand this might just be noise even if it is statistically significant. The most that should come out of these studies is an observation that further study might be fruitful. Instead, the authors trumpeted their conclusion in the New York Times and claimed that it has significant application to situations that are far beyond the scope of their study. Perhaps they are so "bright" that they see significance in the results that the rest of us aren't smart enough to see. Or perhaps they were lured by the prospect of a catchy headline and failed to see how marginal their conclusion really might be. That is not the standard that I would hope for from "two of the brightest young behavioral economists around".

Read more...

Hugh Critz

I have 25,000 college basketball games in my database, with scores in the halves as well as pointspread lines. I do not have the same results as the authors of this working paper. Here are my resutls (no home/away breakdown)

Team down by 5 wins game 27.4%
down by 4 wins 34.9%
down by 3 wins 38.1%
down by 2 wins 42.5%
down by 1 wins 48.0%
tied wins 50%

There were an average of 975 games in each line.

Then I looked at games where the pointspread for the game was 7.5 or less. There were fewer games - only about 640 games per line.

Team down by 5 wins game 29.7%
down by 4 wins 38.1%
down by 3 wins 40.9%
down by 2 wins 45.0%
down by 1 wins 48.1%
tied wins 50%

I think I have more data and games than the authors of the paper, but they have more information on each game. It is possible my data is flawed, I am not in academics so I am not as worried about accuracy as people in academics may be. However I do my best to gather accurate data and I have no reason to believe my data is not accurate.

For the academics out there, what are the chances that the data in the working paper and the data I have can be reconciled by the difference in sample size?.

Read more...

Guy

"what are the chances that the data in the working paper and the data I have can be reconciled by the difference in sample size?"

Pretty good. Brian Burke, using another NCAA dataset, also finds that +1 teams win about 52% of the time. See his article with link to spreadsheet at Wages of Wins:
http://dberri.wordpress.com/2009/03/05/modeling-win-probability-for-a-college-basketball-game-a-guest-post-from-brian-burke/.

I think it probably is true, looking at all this data, that a 1 point lead is not half as valuable as a two point lead. But it's very unlikely that being one point behind is actually better than being tied or up one. And to the extent there is a "-1 effect," it's not at all clear this has anything to do with "effort."

Lawrence

In addition to all the above suggestions, the authors may want to modify footnote 3, which refers to a "discreet" point.

Lawrence

Even though the NCAA results are driven by the elimination of the tie observation and the quintic line fit, the experiments provide evidence for the authors' theory about being a little bit behind. This may result from the fact that (a) the experiment did not concern a team game, and (b) experiment participants were only given feedback at the midpoint of the game. Difference (b) is probably more important, because someone in the lead would have no incentive to change strategy or exert more effort, since the only information available is that what they are doing is working. Contrast this with NCAA games, where players get constant feedback, unless you buy the implication on page 5 that halftime is the only instance where all players are aware of the score.

Matt B

Also, be sure to checkout Goal Theory (Locke & Latham) instead of just Prospector Theory.

It's premise: feedback direction (points up or down) to one's goal dictates adjustments to future efforts. Negative feedback indicates more effort is required to accomplish the goal, while positive feedback suggests a little less effort may be needed for the same results. Caveats (moderators) are the strength of the feedback (i.e. points down/up) which could lead to withdraw as feedback becomes increasing negative and goal commitment (probably not an issue with NCAA basketball).

The most successful individuals are the ones who continue to put forth effort despite continuous indications of extreme positive feedback. Think Tiger Woods.

Goal theory has typically been studied at the individual level, however. It's interesting to see symmetry within a group setting and collective goal.

Also, for what's it's worth, I agree that a logistic criterion is more appropriate for the analysis than a linear examination even though they produce corroborating results.

Read more...

pat toche

what would you get if you tossed a fair coin? I mean, most of the focus here is on games that are, essentially, 50-50.