College Football Polls Aren’t What You Think

It may not be surprising to you that Trevon D. Logan, an economics professor at Ohio State University, is interested in college football. Ohio State is, after all, a football mecca (as we experienced first-hand some time ago).

What may surprise you, however, is what Logan has concluded about college football polls. In a new working paper, Logan used 25 years of AP poll data to test if a variety of conventional wisdoms were correct. He found out that often, they are not. From the abstract:

In particular, I test (1) whether it is better to lose early or late in the season, (2) whether teams benefit from playing stronger opponents, and (3) whether teams are rewarded for winning by large margins. Contrary to conventional wisdom, I find that (1) it is better to lose later in the season than earlier, (2) AP voters do not pay attention to the strength of a defeated opponent, and (3) the benefit of winning by a large margin is negligible.

If you care at all about college football, this is pretty interesting. And if you care at all about college football, you also know that Ohio State plays Michigan this weekend. It is good to see that Michigan has bounced back after an early-season whomping by my alma mater.

Finally, lest you think Logan is wasting research dollars by worrying about college football poll results, you should know that his specialties are economic history, economic demography, and biodemography. This paper, “The Transformation of Hunger,” looks particularly interesting. It finds that poor people today are less hungry than yesteryear’s wealthy industrial workers.


Andrew

What about the timing of voting during each week. Cal's loss to Arizona State was its third in a row. Yet, some people still voted for Cal to be in the top 25. Perhaps, because many Pac-10 games end after 11pm EST, several of the voters submitted their votes at halftime when Cal still had a lead. Such a result would create bias against teams that play late games.

Jim

The math that Mr. Logan uses to determine whether it is better to lose early or late is correct, but the logic is flawed.

This year is a perfect example. LSU lost early to Kentucky and was set back several places in the polls. However, as the weeks went on, and teams ahead of them continued to lose, they gradually creeped up in the polls. Meanwhile, Ohio State won their first 10 games only to lose to Illinois this weekend. The result? Now there are 6 teams ahead of them in the polls, with no time to move up. It has nothing to do with team quality, how teams are rated, or whether pollsters evaluate early or late losses differently. It has to do with the timing of the penalty (invariably teams move down when they lose)- that is the critical factor, and it was ignored in the analysis.

While teams may be penalized less in terms of number of spots later in the season, the effect remains- losing early is better than losing late.

Read more...

Darrel

Ignoring strength of schedule and margin of victory are cardinal sins in evaluating teams.

schadenfreude

I didn't RT whole FA, but at first glance, it looks like Logan either doesn't understand or is deliberately misrepresenting conventional wisdom on early vs. late season losses. The theory is not that and early season loss will have less effect the following week, the theory is that the rankings weight recent games more heavily than older games, and therefore, in the end of season rankings, it is better to have your losses in the voters distant memory than fresh on their minds.

A better way to test this theory would be to look at pre-season rankings versus post-season rankings relative to season record. The conventional wisdom is that if you took two teams that started the season with the same ranking and they each had, say, 11-1 records, the team with the early season loss ends up higher in the polls than the team with the late season loss.

Charles

Hawaii and Boise State beg to differ about strength of defeated opponents.

Badger Fan

Michigan may have "bounced back," but that didn't stop them from losing to Wisconsin last weekend--granted, both teams were really depleted by injuries though.

Toni

I'll admit now I dropped out of the Masters in Econ program after one semester (for non-academic reasons). So explain to me how otherwise thoughtful readers (whose level of expertise I admittedly don't know) can, in minutes, attack a study that takes into account 25 years of data and months of analysis--based on hunches, beliefs, and anecdotal evidence.

the rest of the world

Slow down there. Polls? You determine a championship by polling?

Why not just roll some dice? Then you could get rid of the whole annoying, and expensive, actual sport, and leave even more time for adverts.

When are you guys going to start playing real sport? Divisions. Promotion. Relegation.

Josh

Conventional wisdom is that if a team wants to contend for the national championship or a BCS berth (roughly the top 12 or so ranked teams), it is far better to lose early than late.

Logan's argument is correct, but only against a strawman argument. It is absolutely not correct when applied against the top contending teams. For the small universe of one-loss teams that typically are in the mix for the national championship and BCS games, getting that loss early and then climbing the ranks over the season is far better than losing late in the season and having no time to climb.

One only need look this season at LSU, Oregon and Oklahoma compared to Ohio State. How did the timing of each of these team's single loss impact them in the rankings?

oddTodd

Apparently people don't like to read the papers before commenting.

#1, see page 3: "Even if one wished to argue that an early loss gives teams more time to make up ground in the rankings, the results here suggest that late losses leave teams with less ground to make up."

#3, see page 9: "Also, since ranking in
the polls reflects recent performance, it is better to avoid losses late in the season. Similarly, the
wisdom holds that voters view late losses unfavorably as they are a signal of low team quality."

the Gooch

I agree with schadenfreude. Pre-season rankings have a large effect on post-season rankings, which is a crying shame.
The obvious example this year is that, last week, the Ohio State University was ranked number 1 simply because they were undefeated and played in a major conference (though their conference is week this year and their out-of-conference schedule was a joke). This week, Kansas remains undefeated, plays in a major conference (that's not having a great year, but is better than Ohio State's conference), and had a joke of a non-conference schedule, but they are ranked #3.
The difference between the two teams is that the Ohio State University was ranked in the preseason and Kansas was not.

schadenfreude

Toni-

Actually, myself (and the other haters) are not attacking his analysis. We are attacking his view of conventional wisdom. I have never heard anyone argue that an early season loss results in a smaller drop in the next poll. And yet, this is what he measured to determine that conventional wisdom is wrong.

oddTodd-

Actually, I did read that part of the paper. Had he measured the effect of losses on end of season rankings, then he would have been testing that hypothesis. But how can analyzing a week one loss in the week two polls versus a week ten loss in the week eleven polls be a measure of the hypothesis that polls reflect recent performance? Wouldn't the effect of a week one loss versus a week ten loss in the week eleven poll be a better measure of whether or not ranking in the polls reflects recent performance?

Charles

Pointing to Ohio State as a reason to counter the early/late loss is very weak. Ohio State came to be #1 by default, they had not proved they were the best but did prove they weren't the worst by not losing, so they moved up as a preseason #8 undefeated team. They rested on the thin line of a weak schedule(which is in the range of top 5-10 weakest schedules depending on the weak), which propelled them much further down than when LSU lost to a ranked team after defeating numerous teams while they were in the top 10.

And to reply to #7, this is why we love college football. Sports are a means to conversation. If you break it down it really doesn't matter who wins, we just want something to talk about. There is no better discussion than who should be ranked where in college football since there are endless amounts of data you could look at to support your argument.

Snot Rag Dave

Note to posters:
'weak' - 1: lacking strength: as a: deficient in physical vigor : feeble, debilitated
'week' - 1 a: any of a series of 7-day cycles used in various calendars; especially : a 7-day cycle beginning on Sunday and ending on Saturday
(Courtesy www.merriam-webster.com/dictionary/)

Blue Number 2

I think Gooch is on to something. In particular, how does the history of the program come into play? I would argue that the historical success of the program and its stature play a big role especially in preseason rankings but also throughout the season. And, as Gooch says, it's harder to move up if you are low in the preseason poll or not in it at all.

Should not a factor be included in the analysis to control for the "value" of the program? There is obviously a lot of subjectivity in this...but then again...the polls are repleat with subjectivity.

jacknjacks

Seems rather obvious to me that the good professor got lost in the numbers. If he wants to test a conventional wisdom he first needs to know what the conventional wisdom is. Relative to early/late losses, the "everybody knows" wisdom is that a team's net end-of-season penalty is worse for a late loss than an early loss. Along the lines of Jim and schadenfreude's points, such a hypothesis would be tested by comparing the week 11 polls for a team that had their only loss in week one vs a team that had their only loss in week 10. If Logan wants to argue that conventional wisdom is saying something about the immediate poll drop after a loss, he needs to cite some examples.....I've never heard anyone say that.

the Gooch Returns

A big test of whether a late loss hurts more is in the pudding of the BCS title game matchup.

Most years, there is at least one team in the championship game. Off the top of my college-football addled head, I can only think of one of those teams that lost its final game in the regular season--Nebraska in 2001 played in the 2002 Rose Bowl. The caveat there is that Nebraska's season ending loss didn't happen in the last week of the season--it gave them a de facto bye week while the other 0 and 1 loss contenders had to play in conference championship games (and the key ones lost).

The paper is limited in that it only studies 25 programs that won at least a piece of a national championship. Keep that in mind with all of its implications. It does not and cannot address the polling attitudes towards the Boise States and Kansases of the world.

Also, the study statistically shrugs off the effect of not blowing out lesser opponents as not significant. But, it mentions one example (Penn State in 1994) where that clearly happened. As a coach, what should I use to guide my decision-making in a game I am clearly winning? Should I run up the score or not? The statistics say in general, running up the score doesn't help you. The statistics say in general, winning by only a small margin doesn't hurt you. Yet, I can look at examples where that phenomenon clearly took place. To be safe, I'd run up the score. I don't want to have to face my team Sunday afternoon when the polls came out and say, "The fact that we dropped in the polls after beating Indiana by only 6 is merely anecdotal, not statistical, so don't worry about the fact we won't be playing for the national title this year."

Read more...

Oliver Dziggel

Well, we can all thank Logan for demonstrating why "Freakonomics" does not always work, or rather, demonstrating how "Freakonomics" is misused and misinterpreted

John

#6 and #9; I totally agree. He addresses most of the concerns being raised here in his paper.

Coming up with evidence to refute what his evidence shows is one thing, but basing your argument on circumstantial evidence, opinions, etc. is not the way to do it.

Nate

No matter what the analysis is, I am just glad that Notre Dame is having an awful year. It makes everything else in college football just a little better.