College Football Polls Aren’t What You Think

It may not be surprising to you that Trevon D. Logan, an economics professor at Ohio State University, is interested in college football. Ohio State is, after all, a football mecca (as we experienced first-hand some time ago).

What may surprise you, however, is what Logan has concluded about college football polls. In a new working paper, Logan used 25 years of AP poll data to test if a variety of conventional wisdoms were correct. He found out that often, they are not. From the abstract:

In particular, I test (1) whether it is better to lose early or late in the season, (2) whether teams benefit from playing stronger opponents, and (3) whether teams are rewarded for winning by large margins. Contrary to conventional wisdom, I find that (1) it is better to lose later in the season than earlier, (2) AP voters do not pay attention to the strength of a defeated opponent, and (3) the benefit of winning by a large margin is negligible.

If you care at all about college football, this is pretty interesting. And if you care at all about college football, you also know that Ohio State plays Michigan this weekend. It is good to see that Michigan has bounced back after an early-season whomping by my alma mater.

Finally, lest you think Logan is wasting research dollars by worrying about college football poll results, you should know that his specialties are economic history, economic demography, and biodemography. This paper, “The Transformation of Hunger,” looks particularly interesting. It finds that poor people today are less hungry than yesteryear’s wealthy industrial workers.


John

From everyone's comments, I am guessing that how he ranked how much the late vs. early losses hurt is the number of spots that the team dropped in the polls (number of points or positions). However, the correct measurement would be the percentile drop compared to teams with the comprable number of losses. A lost in the third game might drop a #2 team by 10 spots since there are many 2-1 teams, but they will become the highest ranked of the 2-1 teams because they had the #2 rating previously.

If a #2 team loses in the tenth game to become 9-1 loss, they might only drop four spots, behind the rest of then 9-1 teams. They will drop fewer spots because there will be fewer teams with one loss at that time to move them behind. You may not want to move them behind a two loss team but would move them behind teams that lost their only game earlier in the season (see Ohio State). So while the number of spots may be smaller later in the season, their position relative to competitors "Championship" or "BCS" competitors will be greater.

Also, since the end rankings determine bowl position, there is less time to make up the drop in the polls (for both them winning and other teams losing). Value of the loss is not determined by the next week's poll, but by the final poll.

A telling (but not perfect) study would be to find if there is a correlation between when (i.e. what week) a team lost and where they finished relative to other teams (either ranking or points) with the same record. That would allow you to compare early weeks to late weeks. That would show the number of teams would will need to jump to get into the Championship or BCS game.

Read more...

mubic

I read some of the article. Assuming all the other things are equal, I'd guess the main reason a late loss results in a smaller poll drop has to do with the quality of the opponents of the teams ranked behind you. In the early season, ranked teams are mostly playing crappy squads who have no chance, so if you lose, you're going to drop a long ways because everyone behind you probably won. In the late season, the teams ranked right behind you are playing major conference teams, and possibly conference championship games, and are more likely to lose themselves (in the moment).

I agree with some of you guys that he's kind of misinterpreting what college football analysts value when they talk about conventional wisdom. The goal of these powerhouses is the championship. The heavyweights dont care about how many poll points they have for any other purpose than making the title game. I have some ideas for an analysis of this.

Gooch Retunrs:
You say running up the score is "playing it safe". One reason teams hold back is because they don't want their stars to get hurt. so i guess you weigh the chance of a polling abberation vs the chance of someone getting knocked out for the season. I don't know if you watch the games, but football's pretty violent.

Read more...

Dan Limbach

Let's walk through a season backwards and estimate the chance of winning the national championship (rank=1) based on where your loss occurred.

Let's define every ranked team's final game of the season (BCS Bowl Championship Week) as f, for “final game.”

Game f: The chances of ending the season with rank=1, if your loss is in your last game (game f) is essentially zero. The team that loses the BCS title game has never, nor ever will, be ranked higher than the team who beat them in that game. So let's agree losing your very last game of the season (game f) gives you a 0% chance of rank=1 at the end of the season. As you can guess, your rank=1 chances will most certainly be greater than 0% if your one loss occurs anywhere earlier than game f.

Game f-1: Teams who lose their second to last game will likely take a lethal hit in the rankings, falling below #2 in the rankings with one game left. If they were 11-0 and ranked #1 going into game f-1, it is not impossible for them to slide only down to the #2 rank and still have a place in the BCS title game in game f. If Michigan and Ohio State were ranked #1 and #2 respectively going into game f-1, and Ohio State beats Michigan in game f-1, Michigan could have a tiny sliver of a chance to only slide to #2 and still be in the BCS title game for a rematch with Ohio State. Therefore, the chances of ending the season with rank=1 after a loss in game f-1 is a tiny bit greater than losing in game f.

Game f-2: Losing game f-2 can severely cripple a team's chances at a rank=1 at the end of the season, as demonstrated by Ohio State this season. There may be more than one team at 11-0, and several highly ranked teams at 10-1. The odds of winning a national championship after losing game f-2 is slightly better than losing in game f-1.

Game f-3: Losing game f-3, you have slightly greater chance of winning the national championship than if you lose in game f-2. You can still win your last 3 games (f-2, f-1, f), and everyone ahead of you still has a chance to lose up to 3 games, which gives you a chance at rank=1.

Game f-4: Losing game f-4 gives still gives you a chance to win 4 more games, and the teams ahead of you could still lose up to 4 games.

And so on. If you have to account for a loss, losing game f-12 gives you the greatest chance of eventually winning the national championship, because you have a chance to win your next 12 games, and every team above you has a chance at losing 1 or more games in the next 12.

So statistically, losing early is way better than losing late, especially if your loss is in games f, f-1, or f-2. I would be shocked if the data points in the other direction. I must admit that my explanation pertains to obtaining the #1 ranking, but I believe it has to have merit for ending at #2, #3, and so on.

Game of Loss Chance of final rank=1
f 0%
f-1 greater than 0%
f-2 > losing in game f-1
f-3 > losing in game f-2
. .
f-12 > losing in game f-11

Read more...

the Gooch

Since the BCS began (1998 season), six teams have played in the title game with a loss during the regular season.

Here is that list and the date on which they lost:

2006 Florida Gators (Oct. 14 to LSU)
2003 Oklahoma Sooners (Dec. 6 to Kansas State in Big XII Championship)
2003 LSU Tigers (Oct. 11 to Florida)
2001 Nebraska Cornhuskers (Nov. 23 to Colorado)
2000 Florida State Seminoles (Oct. 7 to Miami)
1998 Florida State Seminoles (Sept. 12 to NC State)

Two of those six had their loss come in their final game before bowl season.

However, both times those teams were not in the top 2 spots in the AP. Oklahoma was #3 in 2003; Nebraska was #4 in 2001.

Dan Limbach

10 years (accounting for 20 teams) is not a very reliable database, but it's all we have. Most of those six teams mentioned by the Gooch who had one loss, lost their first game with 4 or more weeks; enough time to climb back to earn a spot in the title game.

Therefore, only 2 teams out of 20 possible, in the brief history of the BCS have overcome a late loss (Nov. or Dec.) and made it to the title game. Both of those teams lost in the title game.

2003 Oklahoma Sooners: Lost late and lost the title game
2001 Nebraska Cornhuskers: Lost late and lost the title game

Teams who lost earlier have been more successful at obtaining the #1 ranking at the end of the season (winning the national championship).

4 teams have made it to the big dance after an early loss (before Oct. 15), and half of them won the title game.

2006 Florida Gators: Won the title game
2003 LSU Tigers: Won the title game
2000 Florida State Seminoles: Lost the title game
1998 Florida State Seminoles: Lost the title game

Read more...

Jim Norman

If Ohio St. loses how can a 12-2 LSU become a national champion with a loss to an unranked Arkansas? The BCS computer system has failed us again.

steven

won't technology mean that the "natural selection" of genes in the rich will select the relatively weaker genes. in poor people the weak won't be able to bread so those genes won't get passed on. If the rich continue getting weaker and the poor continue getting stronger then won't the superior species be the poor, who will take over and in turn become the rich?

Andrew

What about the timing of voting during each week. Cal's loss to Arizona State was its third in a row. Yet, some people still voted for Cal to be in the top 25. Perhaps, because many Pac-10 games end after 11pm EST, several of the voters submitted their votes at halftime when Cal still had a lead. Such a result would create bias against teams that play late games.

Jim

The math that Mr. Logan uses to determine whether it is better to lose early or late is correct, but the logic is flawed.

This year is a perfect example. LSU lost early to Kentucky and was set back several places in the polls. However, as the weeks went on, and teams ahead of them continued to lose, they gradually creeped up in the polls. Meanwhile, Ohio State won their first 10 games only to lose to Illinois this weekend. The result? Now there are 6 teams ahead of them in the polls, with no time to move up. It has nothing to do with team quality, how teams are rated, or whether pollsters evaluate early or late losses differently. It has to do with the timing of the penalty (invariably teams move down when they lose)- that is the critical factor, and it was ignored in the analysis.

While teams may be penalized less in terms of number of spots later in the season, the effect remains- losing early is better than losing late.

Read more...

Darrel

Ignoring strength of schedule and margin of victory are cardinal sins in evaluating teams.

schadenfreude

I didn't RT whole FA, but at first glance, it looks like Logan either doesn't understand or is deliberately misrepresenting conventional wisdom on early vs. late season losses. The theory is not that and early season loss will have less effect the following week, the theory is that the rankings weight recent games more heavily than older games, and therefore, in the end of season rankings, it is better to have your losses in the voters distant memory than fresh on their minds.

A better way to test this theory would be to look at pre-season rankings versus post-season rankings relative to season record. The conventional wisdom is that if you took two teams that started the season with the same ranking and they each had, say, 11-1 records, the team with the early season loss ends up higher in the polls than the team with the late season loss.

Charles

Hawaii and Boise State beg to differ about strength of defeated opponents.

Badger Fan

Michigan may have "bounced back," but that didn't stop them from losing to Wisconsin last weekend--granted, both teams were really depleted by injuries though.

Toni

I'll admit now I dropped out of the Masters in Econ program after one semester (for non-academic reasons). So explain to me how otherwise thoughtful readers (whose level of expertise I admittedly don't know) can, in minutes, attack a study that takes into account 25 years of data and months of analysis--based on hunches, beliefs, and anecdotal evidence.

the rest of the world

Slow down there. Polls? You determine a championship by polling?

Why not just roll some dice? Then you could get rid of the whole annoying, and expensive, actual sport, and leave even more time for adverts.

When are you guys going to start playing real sport? Divisions. Promotion. Relegation.

Josh

Conventional wisdom is that if a team wants to contend for the national championship or a BCS berth (roughly the top 12 or so ranked teams), it is far better to lose early than late.

Logan's argument is correct, but only against a strawman argument. It is absolutely not correct when applied against the top contending teams. For the small universe of one-loss teams that typically are in the mix for the national championship and BCS games, getting that loss early and then climbing the ranks over the season is far better than losing late in the season and having no time to climb.

One only need look this season at LSU, Oregon and Oklahoma compared to Ohio State. How did the timing of each of these team's single loss impact them in the rankings?

oddTodd

Apparently people don't like to read the papers before commenting.

#1, see page 3: "Even if one wished to argue that an early loss gives teams more time to make up ground in the rankings, the results here suggest that late losses leave teams with less ground to make up."

#3, see page 9: "Also, since ranking in
the polls reflects recent performance, it is better to avoid losses late in the season. Similarly, the
wisdom holds that voters view late losses unfavorably as they are a signal of low team quality."

the Gooch

I agree with schadenfreude. Pre-season rankings have a large effect on post-season rankings, which is a crying shame.
The obvious example this year is that, last week, the Ohio State University was ranked number 1 simply because they were undefeated and played in a major conference (though their conference is week this year and their out-of-conference schedule was a joke). This week, Kansas remains undefeated, plays in a major conference (that's not having a great year, but is better than Ohio State's conference), and had a joke of a non-conference schedule, but they are ranked #3.
The difference between the two teams is that the Ohio State University was ranked in the preseason and Kansas was not.

schadenfreude

Toni-

Actually, myself (and the other haters) are not attacking his analysis. We are attacking his view of conventional wisdom. I have never heard anyone argue that an early season loss results in a smaller drop in the next poll. And yet, this is what he measured to determine that conventional wisdom is wrong.

oddTodd-

Actually, I did read that part of the paper. Had he measured the effect of losses on end of season rankings, then he would have been testing that hypothesis. But how can analyzing a week one loss in the week two polls versus a week ten loss in the week eleven polls be a measure of the hypothesis that polls reflect recent performance? Wouldn't the effect of a week one loss versus a week ten loss in the week eleven poll be a better measure of whether or not ranking in the polls reflects recent performance?

Charles

Pointing to Ohio State as a reason to counter the early/late loss is very weak. Ohio State came to be #1 by default, they had not proved they were the best but did prove they weren't the worst by not losing, so they moved up as a preseason #8 undefeated team. They rested on the thin line of a weak schedule(which is in the range of top 5-10 weakest schedules depending on the weak), which propelled them much further down than when LSU lost to a ranked team after defeating numerous teams while they were in the top 10.

And to reply to #7, this is why we love college football. Sports are a means to conversation. If you break it down it really doesn't matter who wins, we just want something to talk about. There is no better discussion than who should be ranked where in college football since there are endless amounts of data you could look at to support your argument.