Is Changing the Coach Really the Answer?

(Photo: Steve Burns)

Much of the focus today on college football is on the teams at the top.  Will Notre Dame win the national title and finish undefeated? Can Alabama win another championship?  Then there are the 34 other bowl games.  In all, 70 teams have an opportunity to finish the year as a winner.

For those without this opportunity, though, this past season was a disappointment.  For these “losers,” the focus these past few weeks has been strictly on preparing for the next season.  And part of that preparation appears to be changing the head coach.

Already, at least 25 schools have announced that the head coach from 2012 will not be on the sideline in 2013.  For some, this is because a successful team lost their coach to another program.  In many instances, though, teams have asked a coach to depart in the hope that someone else will alter their team’s fortunes.

A recent study published in the Social Science Quarterly suggests that these moves may not lead to the happiness the fans envision (HT: the Sports Economist). E. Scott Adler, Michael J. Berry, and David Doherty looked at coaching changes from 1997 to 2010.  What they found should give pause to  people who demanded a coaching change (or still hope for one). Here is how these authors summarize their findings:

…we use matching techniques to compare the performance of football programs that replaced their head coach to those where the coach was retained. The analysis has two major innovations over existing literature. First, we consider how entry conditions moderate the effects of coaching replacements. Second, we examine team performance for several years following the replacement to assess its effects.

We find that for particularly poorly performing teams, coach replacements have little effect on team performance as measured against comparable teams that did not replace their coach. However, for teams with middling records—that is, teams where entry conditions for a new coach appear to be more favorable—replacing the head coach appears to result in worse performance over subsequent years than comparable teams who retained their coach.

So these authors find that if you are a bad team, changing your coach didn’t make a difference.  And if you are “not bad,” a new coach makes it worse.

This result is consistent with studies of other sports. This year, Mario De Paola and Vincenzo Scoppa published a study of coaching in Italian football (a.k.a. soccer) in the Journal of Sports Economics. Like the study of college coaching, these authors also fail to find evidence that changing the coach helps:

From our analysis, it emerges that coach replacement does not produce statistically significant effects on team performance. This result turns out both when we estimate the impact of coach change including among controls team fixed effects and when using a matching estimator, in which selection on the treatment depends on team performance in the latest rounds. This finding confirms results obtained by some recent studies (for example, Balduck & Buelens, 2007; Bruinshoofd & TerWeel, 2004)

The 2007 and 2004 studies also looked at soccer.  And as noted, these studies produced similar results.

But this story goes beyond soccer.  Back in 2006, Rick Audas, John Goddard, and W. Glenn Rowe looked at coaching in the NHL.  As these authors note in their conclusion, they also failed to find evidence that changing a coach helps a team win more games.

The effect of a change of coach on team performance in the NHL has been estimated in a parametric model, also based on match-level data. Ordered probit regression has been used to represent the discrete and hierarchical structure of the ‘win-tie-lose’ match-results-dependent variable. The use of lagged match results data provides a control for the phenomenon of mean-reversion in team performance. The empirical results suggest teams that changed their coach within-season tended to perform worse subsequently in the short term than those that did not. However, the detrimental effect appears to be short-lived, and over a longer time horizon the effect is almost neutral. In the broader context of the debate concerning the managerial influence on organizational performance, the results suggest that a change of management in the midst of a crisis is unlikely to improve performance by more than might have been expected through the natural tendency for mean-reversion after a spell of poor performance.

Each of these studies looked at what happens to team outcomes when a coach is changed.  Back in 2009, Michael Leeds, Eva Marikova Leeds, Michael Mondello and I published a study of NBA coaching in the International Journal of Sport Finance.  This paper took a different approach, examining how NBA coaches impacted the productivity of individual NBA players.  Although the approach was different, the results for most NBA coaches were similar.  In other words, we found that most NBA coaches have no statistical impact on the productivity of individual players. 

What does all this mean?  Henry Abbott – of ESPN’s True Hoop – suggested in 2008 that the argument that NBA coaches don’t tend to change player productivity indicated that coaches could be replaced with “deck chairs.”   These studies, though, don’t indicate that teams are better off without a coach.  That is because none of these studies looked at a team with and without a coach.  What these studies did is look at teams or players with different coaches and failed to find much of a difference.   That suggests that coaches in sports are not very different from each other. It may be true (and more than likely very true) that you are better off with a professional coach than with a random person grabbed from the stands (or no one at all).  But it doesn’t appear that the choice of professional coach matters much.

Such an argument echoes something that was noted by Adam Smith in 1776. In An Inquiry into the Nature and Causes of the Wealth of Nations, Smith argued that the daily operations of a firm are run by “principal clerks” and such clerks are essentially homogenous.  Or as Smith put it “their labour of inspection and direction may be either altogether or very nearly the same.”

Smith’s view of those charged with the “labour of inspection and direction” certainly runs counter to the view people have of coaches and managers in professional sports.  But the sports data appears to be consistent with Smith’s view, even if he wasn’t explicitly talking about sports.

So why do principle clerks and coaches appear to be the same?  Essentially, coaches appear to receive similar training, face similar information sets, and ultimately make similar decisions. The results – perhaps not surprising when you consider these similarities – are that outcomes with different coaches are quite similar.

And that means, if it costs a small fortune to fire your coach – and often it does – then a team is probably better off just keeping who they have on the sideline.  Yes, this may not make the fans of the losers very happy today.  But it doesn’t make sense for universities to make decisions that cost the school money and don’t systematically change the outcomes we see on the field.

Leave A Comment

Comments are moderated and generally will be posted if they are on-topic and not abusive.

 

COMMENTS: 17

View All Comments »
  1. Alex B. says:

    One point on college football, in particular – there is a carousel aspect of this. The notion that 25 schools will have new coaches is true, but misleading in the sense that you imply 25 schools think they could get better by hiring a new coach. Some of those schools had little choice.

    Consider the chain reaction: Arkansas fires their successful coach Bobby Petrino for cause, after he was exposed in an affair with a staff member and after lying to his superiors about it. Arkansas has an interim coach, but still has an open job. They hire Wisconsin head coach Bret Bielema, which opens up a job at UW. UW was not looking to change their coach, but now they have no choice. UW hires Utah State’s Gary Andersen, and now Utah State is looking for a new coach…

    I don’t doubt the point about new coaches failing to improve a school’s performance, but a better metric for the sense of the scale of schools doing that would be to look at the number of coaches fired, not the number of coaches hired, as a lot of the hires are due to the chain reactions. One firing at a big-name school can easily trigger 2 or 3 more job openings at other schools.

    Well-loved. Like or Dislike: Thumb up 11 Thumb down 1
  2. Steve Nations says:

    More teams should keep the coach and fire the GM. The ultimate determiner of success is the quality of the players, so whoever picks the players (via draft, free agency, whatever) should take the fall more often.

    Thumb up 1 Thumb down 0
  3. Caleb B says:

    This entire article could replace the word ‘coach’ with CEO, and ‘team’ with ‘company,’ and the results would be the exact same.

    Well-loved. Like or Dislike: Thumb up 10 Thumb down 1
    • J S says:

      Oh! You beat me to the very same comment.
      Expand on such a CEO article including CEO pay and it would be link-bait nirvana for the author and web site.

      Thumb up 0 Thumb down 0
    • Jim says:

      Caleb, I respectfully disagree with your premise – in most cases I’ve seen, firing a CEO does result in improved performance for a company. But, in keeping with the spirit of Freakonomics – I’d say that someone should examine the data to see if your hypothesis is actually correct.

      Thumb up 0 Thumb down 0
  4. AaronS says:

    The issue is not winning more games, I don’t think (though that is certainly to be desired). Rather, it is the desire to please team fans who, upset at losses, want the coach gone. So the Athletic Directors sacrifice an otherwise good coach (who will likely go to another program and do a great job), bring in someone else…and know that they have 2-4 years of breathing room.

    If the first year is a flop, it’s because the athletes are learning a new system. If the second year is a flop, we’re rebuilding. By the third or fourth year, well, you just fire that coach and start the process over.

    If the coach does great, however, you look like a hero.

    Thumb up 3 Thumb down 0
  5. Molly says:

    The only problem is, there are times when replacing your coach is extremely successful. For example, the University of Utah replaced their middling coach with Urban Meyer. In one year, the U went from a good team to a BCS buster. Although this is obviously contrary to the statistics, everyone hopes THIS coach will take the team to the money-winning BCS, THIS coach will be the Savior of the Sidelines, THIS coach will return Alma Mater U to its rightful place in football history. Winning is so important, logic is almost irrelevant.

    Thumb up 3 Thumb down 0
    • James McAllister says:

      A proper study would compare replacements that are new hires vs. those promoted organically. Meyer was succeeded by his assistant Kyle Wittingham, who continued to build a very strong program. So much so that Utah literally bust into the BCS as a new member of the PAC-12. There’s a lot to be said for continuity and incremental improvements.

      Thumb up 2 Thumb down 0
  6. Gary Drake says:

    Thank you for doing the research to support my gut feeling. Changing coaches is generally nothing more than appeasement to the fans, while a disruption to the players. There are some coaches who can probably change the arc of a team—but less than a handful, and only a handful of teams can afford them. Therefore most teams are generally just moving around average coaches to no positive end.

    What I never understand is when a coach is fired after a very short time. I’ve never heard an owner say, “Sorry, I was dead wrong to hire that guy. Let me try again.”

    Thumb up 0 Thumb down 0
  7. BC says:

    I’m not sure that the interpretations of these studies’ results are quite correct. In sports, as in all zero-sum competitions, no matter what teams and coaches do and no matter how much or little dispersion there is in their abilities, the average team will finish with a 0.500 winning percentage (win and lose 50% of games). Not all teams can be above average and neither can all coaches. Similarly, not all teams can be above average in identifying and recruiting the above average coaches. Finally, all teams can improve in absolute terms on average, even if there is no net improvement in relative terms.

    So, we could very well have a situation where teams that hire new coaches end up with a better coach than the one they fired in absolute terms, yet those teams, on average, do not improve in relative terms. For example, suppose the fired coaches are replaced by new coaches, who were not previously coaches for other teams but are better than the fired coaches. Overall coaching quality goes up, but if these new coaches are not better than the unfired coaches of other teams, the relative performance of teams that hired new coaches may not necessarily improve. Any given team that hired a new coach, though, could still be better off than if they had kept their old coach. Given the overall increase in coaching quality, keeping the old coach could have resulted in even worse performance relative to other teams.

    Also, consider the case of Team A “poaching” a coach from Team B. Suppose, Team A does get better (in absolute and relative terms) as a result of the coaching change. Team B lost a good coach, and might have hired a lesser coach and gotten worse. Both Team A and Team B are included in the sample of teams that made coaching changes and, on average, the teams did not improve. Yet, it was a good decision for Team A to change its coach.

    There could also be a selection bias in the teams that search for new coaches. These could be teams that tend to have the worst coaches and might be expected to do worse in the future if they keep those coaches than teams that are satisfied with their existing coach. Thus, the fact that teams that make coaching changes perform about the same as, or maybe slightly worse than, those teams that keep their coaches could actually mean that the coaching-change teams do better than they would do if they keep the old coach.

    In interpreting these studies’ results, it seems like we need to distinguish between (1) a coach not being better than a given team’s other alternatives (previous coach, cheaper coach, etc.) and (2) coaches not being better, on average, than the cross-section of his peers because the average coach has to be average.

    Well-loved. Like or Dislike: Thumb up 8 Thumb down 2
  8. Voice of Reason says:

    I think that rewarded coaches for success with contract extentions and punishing them by making them clean out their office after failure is just good operant condition. Coaches need to know that the AD and school are serious about winning, and that there is a direct incentive to perform well now, or at least show signs of life. The school has no idea if the coach has some master that he is about to unleash in five years, or if he’s just some bum along for a free ride. Supervision and putting a fire under his butt are necessary evils.

    Thumb up 0 Thumb down 0