ESPN's New QB Rating System: Who Benefits?

This season, ESPN has decided to challenge the NFL and roll out its own system for rating the play of quarterbacks. Its Total Quarterback Rating (QBR) is meant to be an improvement on the NFL’s official quarterback passer rating system, which was designed in the early 1970s and grades QB’s on four basic metrics: completion percentage, passing yards, touchdowns and interceptions.

The idea behind the QBR is to offer a more nuanced approach that teases out how a quarterback contributes to the success (or failure) of a particular play, and ultimately how he impacts the outcome of a game. For example, under the passer rating system, a ten-yard throw that a receiver turns into a 50-yard touchdown, rewards the quarterback exactly the same had he thrown the ball 50 yards into the endzone for a touchdown. The new system differentiates the two by taking into account the run after the catch, a familiar stat known as RAC to fantasy football players. The QBR also accounts for dropped passes, QB rushing yards, avoiding sacks, giving up fumbles, and something called a Clutch Index — which gives extra weight to plays when the game is on the line.

ESPN has crunched the numbers to determine who, under the new QBR system, were the top performers from 2008 to 2010. While QBR does take into consideration quarterback rushing yards, consummate pocket passers Peyton Manning, Drew Brees and Tom Brady (not to mention statue-like Chad Pennington) still come out on top in the new system. One early casualty? Bears quarterback (and noted gunslinger) Jay Cutler looks a lot less effective under QBR.


Jonathan

Anyone know how they are coding the specifics? Are they using one coder or multiple coders. Depending on the mechanisms they should also be reporting inter-coder reliability metrics. All of this seems to be pretty subjective...

assumo

I'd say that all quarterbacks win. With even more artificial importance placed on the playcaller, every one of them is going to get more scrutiny and face time. While it will put some further in the doldrums (see:Derek Anderson), it still solidifies QB as the most followed position in the sport. Couldn't similar treatment be given to every position to describe their effectiveness at gametime? By what metric did Marshawn Lynch improve when he bowled over 7 defenders en route to a game winning touchdown in the playoffs against the defending champs last year?

RGJ

Yeah, how are they coding the specifics? Actually, I don't even understand that question. But this sounds Bill Jamesian, who is awesome in baseball (much more of an individual sport) but difficult to translate to football.

Too many factors in football:

Do you play in a dome like Peyton, or in a cold weather stadium like his brother? Maybe it doesn't matter, because Eli's eight home games were nice sunny days. Can't really build a field factor like James does out of 8 home games instead of 81.

How is your left tackle? Is he giving you 3.1 seconds, or are you scrambling for your life? And even if you are scrambling and ESPN can count that, are you doing it out of loss of nerve (Jaworski), poor grasp of your job (early McNabb), or because you are a 1,000 rusher whose team builds you into their game plan? (Vick, 2006).

Do you have a running game the linebackers fear, or are they trying to play kill the quarterback on every second and seven?

Clutch Index? What if your team has mostly blowouts up or down?

Always willing to listen, but I'm going to focus on the stats in my fantasy league :-)

Read more...

Hassan A.

Would be interesting to see if they added a component to calculate wins (not losses though) as a way to gain extra points. I have a sneaky suspicion that this would boost the prevalence of African American QB's (Young and Vick) who have been one numerous teams that ended with winning records but were criticized for accomplishing wins for utilizing non-traditional weapons such as speed, and scrambling ability (specifically only the scrambles the break result in yardage beyond the line of scrimmage).

Also, interesting enough according to the ESPN Rankings of past QB performance, Vince Young appears to be a better QB on paper. Looks like the Eagles did not get too bad of a deal.

Joshua Northey

They are definitely more valuable than QB rating would show, but no one with a brain has taken QB rating serious for 10 years.

In pretty much every "advanced" attempt to create a statistical system that actually measures value I have seen scrambling quarterbacks end up being less valuable than the conventional wisdom. This is because people will remember outliers disproportinately. Scrambling quarterback outcomes tend to be more 3,1,50,4,5. Whereas regular quarterback outcomes tend to be 17,10,8,30,22. The latter is much more valuable, particularly in football where the limited downs make consistency more important then total value (and the scrambling quarterbacks tend to have less total value anyway, just more peak value). I don't think it has anything to do with them being African American, the numbers cannot see that.

Don't make the mistake of trying to attribute wins to the QB. Yes the QB is important, but there are 21 other starters. Young in particular had the benefit of a great defense, and the Falcons under Vick were not winning because of him if the numbers are to be believed. Vick was extremely valuable last year, but last year he was also passing a lot better. Passing is the core of the NFL, and good scrambling simply doesn't make up for poor passing.

Read more...

KJ

I like the concept but I still think it's missing something (other than what you noted). What about the DEFENSE itself? Should throwing 4 touchdowns on, let's say, the Raiders count as much as if you do the same against the Steelers defense? I think not.

Vince

Much less the Packers. :)

Vince Skolny

Math aside, if the QBR somehow correlates to what I see with my eyes during the game and makes some remote sense with nothing but an understanding of football stats, then the fans are the winners.

Mike

One nice thing about the ESPN system - the ratings are between zero and a hundred, with an average of fifty.

The passer rating starts at 39.6 and "perfect" is 158.3 Even if passer rating is a better measure of a QB's performance, I think its easily supplanted by a number that's easy to understand and assess.

To put it another way, I watch a lot of football and when I hear that someone's QB rating is (say) 89.5, I still have to think "is that good or bad"? ESPN's system makes it easy and its likely to succeed on that basis alone.

Anthony

I don't know jack squat about the finer math here, but as a gridiron fan, I think running back is the position most in need of a qualitative measurement overhaul.

Two yards gained on 4th & 1 is much more meaningful than 9 yards gained on 3 & 15, yet the tired, generic variants of yards-from-scrimmage still seem to be the last word on a running back's performance.

Joshua Northey

Check out a site like Football Outsiders, or Brian Burke's "Advanced NFL Stats" blog. It is not perfect but it is 1000x better than the announcers.

I cannot even listen to the announcers anymore.

Jim

It also seems fairly arbitrary to discount yards-after-catch from a QB's rating. Maybe the QB correctly diagnosed that the 2 yd screen pass was going to net big yards, but the receiver was going to get immediately tackled after catching the ball on the 15 yd slant pattern.

Ulysses

I notice QBR doesn't take into account incompletes and picks caused by Devin Hester and Johnny Knox route-running.

Dan

Not that much of a difference from the old system.......Manning is on top and JaMarcus Russell is the worst :p

Gary

The best statistical evaluations of NFL players and teams can be found at the Cold Hard Football Facts website.

Joshua Northey

Ummm no. That site is a Patriots homer site, and doesn't hold a candle to site like Football Outsiders, or Brian Burke’s “Advanced NFL Stats” blog.

Just my opinion, but there are a lot of things I don't like about Football Outsiders and I still vastly prefer it to CHFF.

hemp

.Doctor Zs Campaign Against the NFLs Quarterback Rating.Doctor Z at Sports Illustrated has gone on the warpath.Apparently he really really really dislikes the NFL quarterback rating system. As he notes and his the NFLs metric is based on how the game was played in the early 1970s when this metric was put in place. Obviously the game has changed a bit in the past three decades so maybe its time for a new formula..Beyond the change in standards overtime though is a problem for decision-makers. It affects what is written what is discussed what becomes the basis in some cases of salary structure and bonuses for players and coordinators..Steve Young who has the highest career passer rating in history admits that hes not quite sure how the system works..Charley Casserly who as Redskins general manager was quite aware that some clauses were built into contracts that reflected the rating points says No I couldnt tell you exactly how they determine the ratings..Bill Parcells whose 11-point dictum to quarterbacks came from years of study of the position says I dont know how they arrive at their ratings and I dont care. I have my own system for evaluating quarterbacks..The confusion is understandable since this may be the most complicated metric commonly cited in all of sports. Here is how the calculation of the quarterback rating is described in .First one takes a quarterbacks completion percentage then subtracts 0.3 from this number and divides by 0.2.

Read more...

Ryan

The take away from this should be that ESPN is actually promoting more useful statistics other than the out dated ones we see now. This is a step in the right direction, and football has been the last sport to really embrace the change. The funny thing is that because it is the NFL, ESPN actually cares enough to promote this, even though it does have some flaws and because they want to profit off of this metric they wont release all of the data it calculates. Hopefully this stat will be used and embraced by the public, so that we can move forward with advanced stats as a whole. Pretty much I'm excited that there is now a tiny possibility that Mark Schlereth will be proven to be a total idiot that we all know he is and be taken off the air because the common fan will actually see how wrong he is.