Wednesday, December 08, 2010

Random Musings on NFL Passer Rating

After years of having my laziness overpower my wonderment over how the QB Passer Rating stat is determined, I finally took a look. I won't bore you with the mathematical minutia (feel free to do so yourself by clicking here), but will instead let my random observations do that job:

Basic Composition - First, a summary of the four components that go into the calculation:

  1. % of pass attempts completed
  2. % of pass attempts that are touchdowns
  3. % of pass attempts intercepted
  4. yards gained per pass attempt
So everything is normalized by # of pass attempts. Furthermore, extremely high or low performance in each component are capped or floored (more on this later).

Weighting - To frame the relative importance of each component, I did a simple trade-off analysis. This is what I found*:

    1 TD pass = 4 completions = -0.8 INT = 20 passing yards

In other words, all else being equal, a player with 1 TD pass would have same rating as player without a TD but 4 additional completions.

Overall I think it's a pretty good metric, with two exceptions:

Complaint #1: Overweighting of Completion % vs Yds/Att - Consider the following two games from Brett Favre, 13 years apart:

Brett Favre
Date
Compl
Att
Yds
TD
Int
9/9/1996
17
31
261
3
0
9/20/2009
23
27
155
2
0

They're both good games, but which would you say is better? To me the 1996 game is clearly better. He passed for more than 100 additional yards on only 4 additional attempts, plus he threw one additional TD pass. Yet the 2009 game yielded a (slightly) higher Passer Rating (115.3 vs 115.1) because of the higher completion %. I get why completion percentage should have some weight - after all, I'd rather have a guy who gained exactly 5 yards every play than 25 yards unevenly distributed every 5 plays - but it's already inherently a component of yds/attempt, so in a sense it's getting double-counted.

To take it to a further extreme, someone who went 2-for-2 for 15 yards would have a higher QB rating (97.9) than someone who went 1-for-2 for 80 yards (95.8). What??

Complaint #2: Capping Extremes - Why cap extremes, particularly at the component level (rather than on the overall score)? Here are the extremes above or below which accumulating further stats don't matter:
Component
Lower Limit
Upper Limit
Completion %
30.0%
77.5%
TD's per pass attempt
0%
11.875%
INT's per attempt
9.5%
0%
Yards per attempt
3.0 yds
12.5 yds

So going 5-for-10 for 225 yards is no better than going 5-for-10 for 125 yards. I generally understand the logic behind omitting outlying data points when aggregating a collection of data points (I think it's to guard against one-time anomalies or bad data...correct?), but these components are already aggregates in themselves, so I don't see how that applies here. Completing 8 of 10 passes (90% completion rate) or 3 TD's in 20 attempts (15% TD percentage) are not one-off data points. If someone can complete 80% of their passes over the course of a game or season, why not give them credit for it? I'm not a statistician so anyone qualified to speak on this (P Ditty?), let's hear from you.

Anyhow, this capping also produces the ugly situation where the weighted average of one's individual performances does not equal his overall QB rating. Take for example this actual two-game stretch from Steve Young:

Steve Young
Date
Compl
Att
Yds
TD
Int
Rating
9/21/1997
17
24
336
2
0
141.0
9/29/1997
16
24
152
1
0
97.9
Total
33
48
488
3
0
122.6

In each game, Young attempted the same number of passes, yet the rating for the two games combined is not equal to the simple average of each game (122.6 vs 119.4). And he didn't even need to 'max out' his game rating to 158.3 before this anomaly occurs. Effectively, the last 36 yards he gained in Game 1 didn't count in his score for that game, yet it did count when aggregating over a season. That bugs me.

Them's my initial thoughts to anyone who read through this. Your thoughts?


*Based on and using as a baseline the 2009 league-wide average statline of 20.3-for-33.3 for 218.5 yds with 1.4 TD 1.0 INT.

11 comments:

Fred's Brim said...

I think they use NFL Passer Rating as the BCS formula

Fred's Brim said...

I can't remember where I saw it (NFL Hall of Fame maybe) but years ago, I saw a listing of all-time NFL Passer Rating leaders. Neil Lomax was number 5 or 6 and I remember being shocked by this. I guess he was "lucky" to be felled by his hip injury before he turned into a suckass veteran. He has been passed by 20-25 "current" QBs so is only at 30 now

spank said...

*head explodes*

Steve Sax said...

The WSJ had an article last month about how the NFL Passer Rating also biases against scrambling QBs, such as Michael Vick. I suppose it is a "passer rating", but still.

Eric Karros said...

Read the WSJ article, I sort of disagree with it...omitting rushing considerations from the calculation doesn't shortcharge running quarterbacks any more than slugging percentage shortchanges good-fielding sluggers. You just need to know what it measures and not treat it as anything more.

Interestingly, the college rating does factor in a bit of rushing ability because yards lost from getting sacked is netted out of passing yardage.

Eric Karros said...

But I do agree it's a passer rating, not a QB rating.

Josh S. said...

THREADJACK:

Dioner again? Instead of Russ?

WHAT IS GOING ON?!

Josh S. said...

On the upside, Flat Breezy is officially the Braves' problem now.

Fred's Brim said...

I think they meant Dave Navarro

He never belonged in the Red Hot Chili Peppers anyway

Nostradamus said...

Navarro? Seriously?

He's sucked way harder than Russ over the last few years. What gives?

Steve Sax said...

you can't spell "Navarro" without "NO!"

NPUT