Our primary goal here at Pro Football Focus is to evaluate players to the best of our ability. We spend countless hours making our assessments because we know that so much of the current way that players are evaluated is flawed and only a more detailed inspection will help. In essence, we want to make football statistics that can more accurately estimate just how good a player really is.
The only way to truly understand why we need better metrics is to have a look at what’s wrong with current football statistics. Just about every number found in a box score can (and has been) used in ways it shouldn’t be. The faults of some statistics have been mentioned around here before, but I think it’s time we shine some light on the flaws of even the most commonly used data. In this series, I will dig into specific statistics and show you why they don’t exactly say what you think they do.
I’ll begin with a look at completion percentage. On the surface, we’d like to think it tells us how accurate a passer is, but so much is beyond the quarterback’s control, that it fails to paint the real picture.
Not All Passes Are Created Equal
When a quarterback throws a pass a few feet away to his running back, it’s relatively easy. However, when the same quarterback’s team is losing late and he’s heaving a deep ball, hoping for it to end up in the right hands, that’s obviously a much tougher challenge. When we simply add up attempts and completions on our way to finding completion percentage, it’s assumed that all passes are equally difficult to complete, which simply isn’t true.
Quality of the players around the quarterback, pass routes, game situations and quality of the defense are among the factors that affect the difficulty of a throw. Another aspect, one that we can quantify, is the distance a pass is thrown. As you would guess, the further a pass is travels in the air, the less likely it is to be completed. In 2010, passes thrown behind the line of scrimmage were completed 87.2% of the time. Passes between 0 and 10 yards were completed 72.3% of the time, between 10 and 20 yards the number dropped to 55.8%, and passes beyond 20 yards were completed just 33.8% of the time.
Then, naturally, if a quarterback throws a large percentage of their passes short, they’ll have a much better shot at a higher completion percentage. In 2010, Jimmy Clausen, Sam Bradford, Jon Kitna, Tom Brady and Alex Smith all threw 69% or more of their passes 10 yards or less, so their completion percentage is likely inflated to some degree. The quarterbacks with less than 57% of their passes aimed at targets within 10 yards were Vince Young, Troy Smith, Ben Roethlisberger, Derek Anderson and Bruce Gradkowski so their completion percentages would be expected to suffer.
We could also look at quarterbacks who have better offensive lines, since it’s typical to have a better completion percentages when not under pressure. We could look at strength of schedule in terms of defenses faced. Passes could be broken down by down, distance, game score, time remaining … the number of factors affecting the difficulty of a throw is seemingly endless. The main point is that passes have varying levels of difficulty and some quarterbacks operate in more difficult situations more often than others, yet completion percentage treats all throws the same.
Quarterbacks Wrongfully Penalized
While completion percentage assumes all passes are equally difficult, it also assumes that quarterbacks are completely responsible for whether a pass is completed or not.
The chief evidence against this is the dropped pass. Of the 6762 incomplete passes that occurred in the 2010 regular season, we attributed 15% (1041) to drops by receivers. On traditional stat sheets, the quarterback is credited with an attempt and an incompletion and, in turn, his completion percentage takes a hit. Receivers, historically, haven’t had a mark against them despite fully owning the mistake. Only recently has the mainstream begun to focus on drops.
Tom Brady, Shaun Hill, Colt McCoy and the Manning brothers all had 20% or more of their passes fall incomplete as a result of their receivers dropping the ball. This clearly shouldn’t be held against them and greatly hurts their completion percentage.
Another problem, though smaller, is spikes. The goal of a spike is to stop the clock, so unless something goes terribly wrong, the ball is smashed into the turf, the clock stops, and all the play is a success. However, on the stat sheet this is also considered an incomplete pass and further lowers the quarterback’s completion percentage. A little over 1% of incomplete passes in 2010 were on spike plays.
This last part is a little more debatable. There were 602 passes that we marked as “throw aways” this season. While it is still an incomplete pass, it could be argued that this is often a smarter decision than forcing a throw into solid coverage or choosing a target hastily in the face of a heavy rush. A quarterback should be penalized for the poor throws he makes, but in many situations a throw away is not a poor throw.
Overall, 16-25% of all incomplete passes in 2010 shouldn’t have been blamed on the quarterback. That is a sizable error that should be considered when looking at a quarterback’s completion percentage.
While completion percentage may not be quoted as often as touchdowns, passing yards, or interceptions, it is a frequently used statistic in quarterback debates, and is one of the four factors used in determining the Passer Rating (which will be saved for another day.) It’s fair to judge quarterbacks on a number of their throws, but with so much of their success so heavily influenced by the situation, and so much necessary information left out, completion percentage isn’t as useful as it’s thought to be.
Below, we have a short table highlighting a new PFF stat called “Accuracy Percentage.” It takes the basic completion percentage idea and adjusts for the problems we’ve just discussed. All the passes a quarterback delivered on target (caught or not) are divided by the passes he truly attempted. It doesn’t yet account for the difficulty of the pass, but it’s a step in the right direction. It’s derived:
(completions + drops) / (attempts – spikes – throw aways)
Most notable in this Top 10 from 2010, are the jumps that Shaun Hill and Colt McCoy make when when their dropped passes, throw aways and spikes are accounted for. (Matt Schaub and David Garrard are who they bumped from the top group.) Also note that the two Dallas QBs experience a lesser change in this new look as they combined to have only 27 of their 531 passes adjusted.