Stat Sheet Misconceptions: Completion Percentage

| March 22, 2011

Our primary goal here at Pro Football Focus is to evaluate players to the best of our ability. We spend countless hours making our assessments because we know that so much of the current way that players are evaluated is flawed and only a more detailed inspection will help. In essence, we want to make football statistics that can more accurately estimate just how good a player really is.

The only way to truly understand why we need better metrics is to have a look at what’s wrong with current football statistics. Just about every number found in a box score can (and has been) used in ways it shouldn’t be. The faults of some statistics have been mentioned around here before, but I think it’s time we shine some light on the flaws of even the most commonly used data. In this series, I will dig into specific statistics and show you why they don’t exactly say what you think they do.

I’ll begin with a look at completion percentage. On the surface, we’d like to think it tells us how accurate a passer is, but so much is beyond the quarterback’s control, that it fails to paint the real picture.
 

Not All Passes Are Created Equal

When a quarterback throws a pass a few feet away to his running back, it’s relatively easy. However, when the same quarterback’s team is losing late and he’s heaving a deep ball, hoping for it to end up in the right hands, that’s obviously a much tougher challenge. When we simply add up attempts and completions on our way to finding completion percentage, it’s assumed that all passes are equally difficult to complete, which simply isn’t true.
 
Quality of the players around the quarterback, pass routes, game situations and quality of the defense are among the factors that affect the difficulty of a throw. Another aspect, one that we can quantify, is the distance a pass is thrown. As you would guess, the further a pass is travels in the air, the less likely it is to be completed. In 2010, passes thrown behind the line of scrimmage were completed 87.2% of the time. Passes between 0 and 10 yards were completed 72.3% of the time, between 10 and 20 yards the number dropped to 55.8%, and passes beyond 20 yards were completed just 33.8% of the time.
 
Then, naturally, if a quarterback throws a large percentage of their passes short, they’ll have a much better shot at a higher completion percentage. In 2010, Jimmy Clausen, Sam Bradford, Jon Kitna, Tom Brady and Alex Smith all threw 69% or more of their passes 10 yards or less, so their completion percentage is likely inflated to some degree. The quarterbacks with less than 57% of their passes aimed at targets within 10 yards were Vince Young, Troy Smith, Ben Roethlisberger, Derek Anderson and Bruce Gradkowski so their completion percentages would be expected to suffer.
 
We could also look at quarterbacks who have better offensive lines, since it’s typical to have a better completion percentages when not under pressure. We could look at strength of schedule in terms of defenses faced. Passes could be broken down by down, distance, game score, time remaining … the number of factors affecting the difficulty of a throw is seemingly endless. The main point is that passes have varying levels of difficulty and some quarterbacks operate in more difficult situations more often than others, yet completion percentage treats all throws the same.
 

Quarterbacks Wrongfully Penalized

While completion percentage assumes all passes are equally difficult, it also assumes that quarterbacks are completely responsible for whether a pass is completed or not.
 
The chief evidence against this is the dropped pass. Of the 6762 incomplete passes that occurred in the 2010 regular season, we attributed 15% (1041) to drops by receivers. On traditional stat sheets, the quarterback is credited with an attempt and an incompletion and, in turn, his completion percentage takes a hit. Receivers, historically, haven’t had a mark against them despite fully owning the mistake. Only recently has the mainstream begun to focus on drops.
 
Tom Brady, Shaun Hill, Colt McCoy and the Manning brothers all had 20% or more of their passes fall incomplete as a result of their receivers dropping the ball. This clearly shouldn’t be held against them and greatly hurts their completion percentage.
 
Another problem, though smaller, is spikes. The goal of a spike is to stop the clock, so unless something goes terribly wrong, the ball is smashed into the turf, the clock stops, and all the play is a success. However, on the stat sheet this is also considered an incomplete pass and further lowers the quarterback’s completion percentage. A little over 1% of incomplete passes in 2010 were on spike plays.
 
This last part is a little more debatable. There were 602 passes that we marked as “throw aways” this season. While it is still an incomplete pass, it could be argued that this is often a smarter decision than forcing a throw into solid coverage or choosing a target hastily in the face of a heavy rush. A quarterback should be penalized for the poor throws he makes, but in many situations a throw away is not a poor throw.
 
Overall, 16-25% of all incomplete passes in 2010 shouldn’t have been blamed on the quarterback. That is a sizable error that should be considered when looking at a quarterback’s completion percentage.

 
Closing Thoughts

While completion percentage may not be quoted as often as touchdowns, passing yards, or interceptions, it is a frequently used statistic in quarterback debates, and is one of the four factors used in determining the Passer Rating (which will be saved for another day.) It’s fair to judge quarterbacks on a number of their throws, but with so much of their success so heavily influenced by the situation, and so much necessary information left out, completion percentage isn’t as useful as it’s thought to be.
 
___________________________________________
 

Below, we have a short table highlighting a new PFF stat called “Accuracy Percentage.” It takes the basic completion percentage idea and adjusts for the problems we’ve just discussed. All the passes a quarterback delivered on target (caught or not) are divided by the passes he truly attempted. It doesn’t yet account for the difficulty of the pass, but it’s a step in the right direction. It’s derived:

(completions + drops) / (attempts – spikes – throw aways)

Most notable in this Top 10 from 2010, are the jumps that Shaun Hill and Colt McCoy make when when their dropped passes, throw aways and spikes are accounted for. (Matt Schaub and David Garrard are who they bumped from the top group.) Also note that the two Dallas QBs experience a lesser change in this new look as they combined to have only 27 of their 531 passes adjusted.
 

Accuracy Percentage

Player
Team
Accuracy %
Acc%Rk
Completion%
Comp%Rk
%diff
Rkdiff
Drew BreesNO76.15%168.09%28.06%+1
Tom BradyNE76.14%265.85%510.29%+3
Tony RomoDAL75.12%369.48%15.64%-2
Aaron RodgersGB74.02%465.68%78.33%+3
Shaun HillDET73.80%561.78%1412.02%+9
Peyton ManningIND73.48%666.18%37.31%-3
Philip RiversSD73.31%765.99%47.32%-3
Jon KitnaDAL72.99%865.72%67.27%-2
Colt McCoyCLE72.86%960.81%2012.05%+11
Eli ManningNYG72.52%1062.89%109.62%0
  • tom

    The whole passer rating concept is especially flawed. Dr Z of Sports Illustrated wrote an article on it a few years back. A brief synopsis is that once all the catagories are totalled, the result is normalized to a standard measure of passer rating established in the 70′s or 80′s. Since then offenses throw shorter and shorter passes, which adds the completion percentage, keeps drives going and adds to yardage totals. Also, at present it is not unusual for an offense to throw a 5 yard pass on 3rd down when they need 10 yards, hoping the receiver might break a tackle; this also adds to competion pct and total yardage. Back when the rating was established it was extraordinary to have a single passer with a 100.0 rating, now it is the norm, also completion percentages were around 50-55, touchdowns in the 20′s and a lot more interceptions as QB’s threw the ball further downfield. If the league were to adjust the normalization factor to modern times; figuring in the normal 60% completion pct, 25-30 touchdowns, 5-10 interceptions then the passer rating would mean something. As it stands now, mediocre QB’s can have 100 passer ratings.

    • Nathan Jahnke

      I agree with you on all of those points. I’ll be examining the passer rating at some point during this series of articles.

  • andrewglover87

    Since you are accounting for drops (ie, passes that should make it), shouldn’t you also account for passes that shouldn’t be completed but are? Situations like deflections, spectacular one handed grabs, last minute hail marys (e.g. Garrard last season) etc. I realise this would be hard to judge, and is probably less common than drops, but may still lower the % of some of the luckier QB’s with great receivers.

    • Nathan Jahnke

      In an ideal world, yes we would. However we don’t keep track of those situations like we do drops, since that’s more subjective. In terms of the overall PFF ratings we do, but we don’t have separate data for it like we do drops. Like I said the accuracy percentage is a step in the right direction, but it’s not perfect and those are some examples of why not.

  • stringer bell

    Where can one find the full list of accuracy %?

    • Nathan Jahnke

      It’s possible we’ll have the full list somewhere in the future. However right now if you wanted the data for a specific quarterback, or the ability to make the list you would need to purchase the premium statistics.