College Football Polls Aren’t What You Think

It may not be surprising to you that Trevon D. Logan, an economics professor at Ohio State University, is interested in college football. Ohio State is, after all, a football mecca (as we experienced first-hand some time ago).

What may surprise you, however, is what Logan has concluded about college football polls. In a new working paper, Logan used 25 years of AP poll data to test if a variety of conventional wisdoms were correct. He found out that often, they are not. From the abstract:

In particular, I test (1) whether it is better to lose early or late in the season, (2) whether teams benefit from playing stronger opponents, and (3) whether teams are rewarded for winning by large margins. Contrary to conventional wisdom, I find that (1) it is better to lose later in the season than earlier, (2) AP voters do not pay attention to the strength of a defeated opponent, and (3) the benefit of winning by a large margin is negligible.

If you care at all about college football, this is pretty interesting. And if you care at all about college football, you also know that Ohio State plays Michigan this weekend. It is good to see that Michigan has bounced back after an early-season whomping by my alma mater.

Finally, lest you think Logan is wasting research dollars by worrying about college football poll results, you should know that his specialties are economic history, economic demography, and biodemography. This paper, “The Transformation of Hunger,” looks particularly interesting. It finds that poor people today are less hungry than yesteryear’s wealthy industrial workers.


Snot Rag Dave

Note to posters:
'weak' - 1: lacking strength: as a: deficient in physical vigor : feeble, debilitated
'week' - 1 a: any of a series of 7-day cycles used in various calendars; especially : a 7-day cycle beginning on Sunday and ending on Saturday
(Courtesy www.merriam-webster.com/dictionary/)

Blue Number 2

I think Gooch is on to something. In particular, how does the history of the program come into play? I would argue that the historical success of the program and its stature play a big role especially in preseason rankings but also throughout the season. And, as Gooch says, it's harder to move up if you are low in the preseason poll or not in it at all.

Should not a factor be included in the analysis to control for the "value" of the program? There is obviously a lot of subjectivity in this...but then again...the polls are repleat with subjectivity.

jacknjacks

Seems rather obvious to me that the good professor got lost in the numbers. If he wants to test a conventional wisdom he first needs to know what the conventional wisdom is. Relative to early/late losses, the "everybody knows" wisdom is that a team's net end-of-season penalty is worse for a late loss than an early loss. Along the lines of Jim and schadenfreude's points, such a hypothesis would be tested by comparing the week 11 polls for a team that had their only loss in week one vs a team that had their only loss in week 10. If Logan wants to argue that conventional wisdom is saying something about the immediate poll drop after a loss, he needs to cite some examples.....I've never heard anyone say that.

the Gooch Returns

A big test of whether a late loss hurts more is in the pudding of the BCS title game matchup.

Most years, there is at least one team in the championship game. Off the top of my college-football addled head, I can only think of one of those teams that lost its final game in the regular season--Nebraska in 2001 played in the 2002 Rose Bowl. The caveat there is that Nebraska's season ending loss didn't happen in the last week of the season--it gave them a de facto bye week while the other 0 and 1 loss contenders had to play in conference championship games (and the key ones lost).

The paper is limited in that it only studies 25 programs that won at least a piece of a national championship. Keep that in mind with all of its implications. It does not and cannot address the polling attitudes towards the Boise States and Kansases of the world.

Also, the study statistically shrugs off the effect of not blowing out lesser opponents as not significant. But, it mentions one example (Penn State in 1994) where that clearly happened. As a coach, what should I use to guide my decision-making in a game I am clearly winning? Should I run up the score or not? The statistics say in general, running up the score doesn't help you. The statistics say in general, winning by only a small margin doesn't hurt you. Yet, I can look at examples where that phenomenon clearly took place. To be safe, I'd run up the score. I don't want to have to face my team Sunday afternoon when the polls came out and say, "The fact that we dropped in the polls after beating Indiana by only 6 is merely anecdotal, not statistical, so don't worry about the fact we won't be playing for the national title this year."

Read more...

Oliver Dziggel

Well, we can all thank Logan for demonstrating why "Freakonomics" does not always work, or rather, demonstrating how "Freakonomics" is misused and misinterpreted

John

#6 and #9; I totally agree. He addresses most of the concerns being raised here in his paper.

Coming up with evidence to refute what his evidence shows is one thing, but basing your argument on circumstantial evidence, opinions, etc. is not the way to do it.

Nate

No matter what the analysis is, I am just glad that Notre Dame is having an awful year. It makes everything else in college football just a little better.

John

From everyone's comments, I am guessing that how he ranked how much the late vs. early losses hurt is the number of spots that the team dropped in the polls (number of points or positions). However, the correct measurement would be the percentile drop compared to teams with the comprable number of losses. A lost in the third game might drop a #2 team by 10 spots since there are many 2-1 teams, but they will become the highest ranked of the 2-1 teams because they had the #2 rating previously.

If a #2 team loses in the tenth game to become 9-1 loss, they might only drop four spots, behind the rest of then 9-1 teams. They will drop fewer spots because there will be fewer teams with one loss at that time to move them behind. You may not want to move them behind a two loss team but would move them behind teams that lost their only game earlier in the season (see Ohio State). So while the number of spots may be smaller later in the season, their position relative to competitors "Championship" or "BCS" competitors will be greater.

Also, since the end rankings determine bowl position, there is less time to make up the drop in the polls (for both them winning and other teams losing). Value of the loss is not determined by the next week's poll, but by the final poll.

A telling (but not perfect) study would be to find if there is a correlation between when (i.e. what week) a team lost and where they finished relative to other teams (either ranking or points) with the same record. That would allow you to compare early weeks to late weeks. That would show the number of teams would will need to jump to get into the Championship or BCS game.

Read more...

mubic

I read some of the article. Assuming all the other things are equal, I'd guess the main reason a late loss results in a smaller poll drop has to do with the quality of the opponents of the teams ranked behind you. In the early season, ranked teams are mostly playing crappy squads who have no chance, so if you lose, you're going to drop a long ways because everyone behind you probably won. In the late season, the teams ranked right behind you are playing major conference teams, and possibly conference championship games, and are more likely to lose themselves (in the moment).

I agree with some of you guys that he's kind of misinterpreting what college football analysts value when they talk about conventional wisdom. The goal of these powerhouses is the championship. The heavyweights dont care about how many poll points they have for any other purpose than making the title game. I have some ideas for an analysis of this.

Gooch Retunrs:
You say running up the score is "playing it safe". One reason teams hold back is because they don't want their stars to get hurt. so i guess you weigh the chance of a polling abberation vs the chance of someone getting knocked out for the season. I don't know if you watch the games, but football's pretty violent.

Read more...

Dan Limbach

Let's walk through a season backwards and estimate the chance of winning the national championship (rank=1) based on where your loss occurred.

Let's define every ranked team's final game of the season (BCS Bowl Championship Week) as f, for "final game."

Game f: The chances of ending the season with rank=1, if your loss is in your last game (game f) is essentially zero. The team that loses the BCS title game has never, nor ever will, be ranked higher than the team who beat them in that game. So let's agree losing your very last game of the season (game f) gives you a 0% chance of rank=1 at the end of the season. As you can guess, your rank=1 chances will most certainly be greater than 0% if your one loss occurs anywhere earlier than game f.

Game f-1: Teams who lose their second to last game will likely take a lethal hit in the rankings, falling below #2 in the rankings with one game left. If they were 11-0 and ranked #1 going into game f-1, it is not impossible for them to slide only down to the #2 rank and still have a place in the BCS title game in game f. If Michigan and Ohio State were ranked #1 and #2 respectively going into game f-1, and Ohio State beats Michigan in game f-1, Michigan could have a tiny sliver of a chance to only slide to #2 and still be in the BCS title game for a rematch with Ohio State. Therefore, the chances of ending the season with rank=1 after a loss in game f-1 is a tiny bit greater than losing in game f.

Game f-2: Losing game f-2 can severely cripple a team's chances at a rank=1 at the end of the season, as demonstrated by Ohio State this season. There may be more than one team at 11-0, and several highly ranked teams at 10-1. The odds of winning a national championship after losing game f-2 is slightly better than losing in game f-1.

Game f-3: Losing game f-3, you have slightly greater chance of winning the national championship than if you lose in game f-2. You can still win your last 3 games (f-2, f-1, f), and everyone ahead of you still has a chance to lose up to 3 games, which gives you a chance at rank=1.

Game f-4: Losing game f-4 gives still gives you a chance to win 4 more games, and the teams ahead of you could still lose up to 4 games.

And so on. If you have to account for a loss, losing game f-12 gives you the greatest chance of eventually winning the national championship, because you have a chance to win your next 12 games, and every team above you has a chance at losing 1 or more games in the next 12.

So statistically, losing early is way better than losing late, especially if your loss is in games f, f-1, or f-2. I would be shocked if the data points in the other direction. I must admit that my explanation pertains to obtaining the #1 ranking, but I believe it has to have merit for ending at #2, #3, and so on.

Game of Loss Chance of final rank=1
f 0%
f-1 greater than 0%
f-2 > losing in game f-1
f-3 > losing in game f-2
. .
f-12 > losing in game f-11

Read more...

the Gooch

Since the BCS began (1998 season), six teams have played in the title game with a loss during the regular season.

Here is that list and the date on which they lost:

2006 Florida Gators (Oct. 14 to LSU)
2003 Oklahoma Sooners (Dec. 6 to Kansas State in Big XII Championship)
2003 LSU Tigers (Oct. 11 to Florida)
2001 Nebraska Cornhuskers (Nov. 23 to Colorado)
2000 Florida State Seminoles (Oct. 7 to Miami)
1998 Florida State Seminoles (Sept. 12 to NC State)

Two of those six had their loss come in their final game before bowl season.

However, both times those teams were not in the top 2 spots in the AP. Oklahoma was #3 in 2003; Nebraska was #4 in 2001.

Dan Limbach

10 years (accounting for 20 teams) is not a very reliable database, but it's all we have. Most of those six teams mentioned by the Gooch who had one loss, lost their first game with 4 or more weeks; enough time to climb back to earn a spot in the title game.

Therefore, only 2 teams out of 20 possible, in the brief history of the BCS have overcome a late loss (Nov. or Dec.) and made it to the title game. Both of those teams lost in the title game.

2003 Oklahoma Sooners: Lost late and lost the title game
2001 Nebraska Cornhuskers: Lost late and lost the title game

Teams who lost earlier have been more successful at obtaining the #1 ranking at the end of the season (winning the national championship).

4 teams have made it to the big dance after an early loss (before Oct. 15), and half of them won the title game.

2006 Florida Gators: Won the title game
2003 LSU Tigers: Won the title game
2000 Florida State Seminoles: Lost the title game
1998 Florida State Seminoles: Lost the title game

Read more...

Jim Norman

If Ohio St. loses how can a 12-2 LSU become a national champion with a loss to an unranked Arkansas? The BCS computer system has failed us again.

steven

won't technology mean that the "natural selection" of genes in the rich will select the relatively weaker genes. in poor people the weak won't be able to bread so those genes won't get passed on. If the rich continue getting weaker and the poor continue getting stronger then won't the superior species be the poor, who will take over and in turn become the rich?