Is Changing the Coach Really the Answer?

(Photo: Steve Burns)

Much of the focus today on college football is on the teams at the top.  Will Notre Dame win the national title and finish undefeated? Can Alabama win another championship?  Then there are the 34 other bowl games.  In all, 70 teams have an opportunity to finish the year as a winner.

For those without this opportunity, though, this past season was a disappointment.  For these “losers,” the focus these past few weeks has been strictly on preparing for the next season.  And part of that preparation appears to be changing the head coach.

Already, at least 25 schools have announced that the head coach from 2012 will not be on the sideline in 2013.  For some, this is because a successful team lost their coach to another program.  In many instances, though, teams have asked a coach to depart in the hope that someone else will alter their team’s fortunes.

A recent study published in the Social Science Quarterly suggests that these moves may not lead to the happiness the fans envision (HT: the Sports Economist). E. Scott Adler, Michael J. Berry, and David Doherty looked at coaching changes from 1997 to 2010.  What they found should give pause to  people who demanded a coaching change (or still hope for one). Here is how these authors summarize their findings:

…we use matching techniques to compare the performance of football programs that replaced their head coach to those where the coach was retained. The analysis has two major innovations over existing literature. First, we consider how entry conditions moderate the effects of coaching replacements. Second, we examine team performance for several years following the replacement to assess its effects.

We find that for particularly poorly performing teams, coach replacements have little effect on team performance as measured against comparable teams that did not replace their coach. However, for teams with middling records—that is, teams where entry conditions for a new coach appear to be more favorable—replacing the head coach appears to result in worse performance over subsequent years than comparable teams who retained their coach.

So these authors find that if you are a bad team, changing your coach didn’t make a difference.  And if you are “not bad,” a new coach makes it worse.

This result is consistent with studies of other sports. This year, Mario De Paola and Vincenzo Scoppa published a study of coaching in Italian football (a.k.a. soccer) in the Journal of Sports Economics. Like the study of college coaching, these authors also fail to find evidence that changing the coach helps:

From our analysis, it emerges that coach replacement does not produce statistically significant effects on team performance. This result turns out both when we estimate the impact of coach change including among controls team fixed effects and when using a matching estimator, in which selection on the treatment depends on team performance in the latest rounds. This finding confirms results obtained by some recent studies (for example, Balduck & Buelens, 2007; Bruinshoofd & TerWeel, 2004)

The 2007 and 2004 studies also looked at soccer.  And as noted, these studies produced similar results.

But this story goes beyond soccer.  Back in 2006, Rick Audas, John Goddard, and W. Glenn Rowe looked at coaching in the NHL.  As these authors note in their conclusion, they also failed to find evidence that changing a coach helps a team win more games.

The effect of a change of coach on team performance in the NHL has been estimated in a parametric model, also based on match-level data. Ordered probit regression has been used to represent the discrete and hierarchical structure of the ‘win-tie-lose’ match-results-dependent variable. The use of lagged match results data provides a control for the phenomenon of mean-reversion in team performance. The empirical results suggest teams that changed their coach within-season tended to perform worse subsequently in the short term than those that did not. However, the detrimental effect appears to be short-lived, and over a longer time horizon the effect is almost neutral. In the broader context of the debate concerning the managerial influence on organizational performance, the results suggest that a change of management in the midst of a crisis is unlikely to improve performance by more than might have been expected through the natural tendency for mean-reversion after a spell of poor performance.

Each of these studies looked at what happens to team outcomes when a coach is changed.  Back in 2009, Michael Leeds, Eva Marikova Leeds, Michael Mondello and I published a study of NBA coaching in the International Journal of Sport Finance.  This paper took a different approach, examining how NBA coaches impacted the productivity of individual NBA players.  Although the approach was different, the results for most NBA coaches were similar.  In other words, we found that most NBA coaches have no statistical impact on the productivity of individual players.

What does all this mean?  Henry Abbott – of ESPN’s True Hoop – suggested in 2008 that the argument that NBA coaches don’t tend to change player productivity indicated that coaches could be replaced with “deck chairs.”   These studies, though, don’t indicate that teams are better off without a coach.  That is because none of these studies looked at a team with and without a coach.  What these studies did is look at teams or players with different coaches and failed to find much of a difference.   That suggests that coaches in sports are not very different from each other. It may be true (and more than likely very true) that you are better off with a professional coach than with a random person grabbed from the stands (or no one at all).  But it doesn’t appear that the choice of professional coach matters much.

Such an argument echoes something that was noted by Adam Smith in 1776. In An Inquiry into the Nature and Causes of the Wealth of Nations, Smith argued that the daily operations of a firm are run by “principal clerks” and such clerks are essentially homogenous.  Or as Smith put it “their labour of inspection and direction may be either altogether or very nearly the same.”

Smith’s view of those charged with the “labour of inspection and direction” certainly runs counter to the view people have of coaches and managers in professional sports.  But the sports data appears to be consistent with Smith’s view, even if he wasn’t explicitly talking about sports.

So why do principle clerks and coaches appear to be the
same?  Essentially, coaches appear to receive similar training, face similar information sets, and ultimately make similar decisions. The results – perhaps not surprising when you consider these similarities – are that outcomes with different coaches are quite similar.

And that means, if it costs a small fortune to fire your coach – and often it does – then a team is probably better off just keeping who they have on the sideline.  Yes, this may not make the fans of the losers very happy today.  But it doesn’t make sense for universities to make decisions that cost the school money and don’t systematically change the outcomes we see on the field.