Cumulative Effect: A Deeper Look

| May 20, 2014

cumulative-effect-2Yesterday I examined the cumulative effect of hits on quarterbacks. If you haven’t read it already, it would be good to do so now in order to understand the rest of this article. There were a number of concerns or questions brought up in the comments or on twitter that deserved a response.

One concern is that good quarterbacks are better at avoiding sacks/hits than bad quarterbacks are. Peyton Manning is an example of a great quarterback who is known for avoiding getting sacked and hit. If this assumption that good quarterbacks are better at avoiding sacks/hits is true, then the sample at the high sack/hit marks would only consist of below average quarterbacks.

In order to check if this was true, I decided to find the weighted expected average Accuracy Percentage at each sack/hit grouping. I did this by taking the Accuracy Percentage of each quarterback in the sample size and found the average Accuracy Percentage among the quarterbacks in the sample. The more throws a quarterback had in the sample, the more their Accuracy Percentage was weighted in.

For example, there have been 27,273 passes thrown the last six years where the quarterback had yet to be sacked or hit in the game. Peyton Manning in 2013 had an Accuracy Percentage of 77.3% and accounted for 333 of those passes. His Accuracy Percentage of 77.3% represented 333/27273 of the average. Once you taken into account all quarterbacks involved, the average is 70.2%. Therefore, if sacks/hits had no effect on a quarterback, I would expect the Accuracy Percentage of those who haven’t been sacked or hit to have an Accuracy Percentage of 70.2%.

Here is the Accuracy Percentage by the entire league compared to this expected Accuracy Percentage.

In general, the expected graph hovers around 71%, with a very slight downward tendency. While some quarterbacks like Manning are good at avoiding sacks and hits, other great quarterbacks get sacked and hit often. Ryan Tannehill, Russell Wilson, Matt Ryan, Ben Roethlisberger, Carson Palmer and Tom Brady were all positively-graded quarterbacks last year that were also in the Top 10 in terms of sacks taken. While the later sacks/hit grouping won’t have Peyton Manning in them, there are also poor quarterbacks not included, while good and bad quarterbacks remain in the higher sample size grouping as well.

Another issue brought up was that quarterbacks who were sacked/hit a lot are more likely to be losing late in a game than one that has not been sacked/hit a lot. Quarterbacks who are losing late in a game often try difficult passes in order to catch up. The concern was that the reason quarterbacks performed worse with a lot of sacks and hits is that they were throwing more difficult passes rather than the sacks/hits effecting them.

While there are multiple factors that impact the difficulty of a throw, one major one is the depth of the pass. Below is a graph of how accurate quarterbacks are at different depths and different number of sacks/hits taken prior to the throw. In order for the data point to be included, it needed a sample size of at least 375 throws which is why each line is a different length

At each depth grouping, the more sacks/hits a quarterback takes, the less accurate they are. The rate of decline at each depth is similar to the rate of decline without accounting for depth. It doesn’t appear that this concern factors in.

Another way to check that is simply by accounting for the game situation by looking at this by quarter. Typically in the first half, a team’s strategy is not dependent on the score of the game. Once the second half starts a team’s strategy is somewhat dependent on the score, and in the fourth quarter a team’s strategy is very dependent on the score. Because strategy is rarely changed in the first half due to score, that is grouped together in the following graph

In the first half there is a very clear downward slope where the more sacks/hits you take, the worse the accuracy is. When you take out the situation where a team’s strategy is dependent on the score, the results for those first few sacks/hits are very clear.

We see an interesting effect in the third and fourth quarter. If a quarterback hasn’t taken a sack/hit yet, they aren’t as accurate as someone with one sack/hit. One part of why this could be true is the lower sample size. On average, a quarterback had taken three sacks/hits by the end of the first half. The best theory I have for this is the sample of who the quarterbacks are who haven’t taken a sack or hit yet.

A lot of the quarterbacks in this sample played the entire first half and had things go right for them and they stayed clean. The other quarterbacks in the sample are backups. Some of the quarterbacks in the sample in the third quarter came in due to injury or a switch due to the starter playing poorly. Once you get into the fourth quarter, the sample gets even more clouded by backups coming in during garbage situations. Looking at the adjusted expected Accuracy Percentage described above, this seems to play some role.

In both the third quarter and fourth quarter, once you get to the three sack/hit range, there is a general downward trend, although the data is a little bit noisier likely due to situation. In general, it looks like while things like situation and depth of throw have a clear effect on a quarterback’s accuracy, even in those specific situations sacks/hits taken still looks to factor in.

 

Follow Nathan on Twitter: @PFF_NateJahnke

  • KittyTheBear

    Wow, Nate, this is truly superb. You actually read my comment on your last article and provided some great feedback. I really appreciate it, man. Keep up the good work.

    • Chris from Cape Cod

      Agreed: Awesome stuff.

  • Colin William Weaver

    Yeah, this is really cool. This is so $()&%#ing cool.

  • ZWK

    There are a lot of things that you might want to control for to examine this effect – it looks like you controlled for some of them here but not all of them (for example, it looks like you didn’t control for the quality of the defense, or the weather). There is one analysis that you could do which would control for a bunch of things at once:

    Repeat your original analysis on time-reversed games, starting at the end of the game and running the clock backwards.

    So a quarterback’s last dropback of the game would count as coming with 0 sacks/hits. Then you go back to the time in the game when the latest sack/hit took place, and the dropbacks which happened immediately before that would count as coming with 1 sack/hit. Etc.

    The total number of sacks/hits in each game would be the same in the time-reversed analysis as it is in the original analysis, so if game-specific factors like the quarterback, the defense, the weather, etc. are having an effect (which leads to lots of sacks/hits in that game and also poor accuracy) then the time-reversed analysis should show the same pattern as the original analysis. But if there really is a cumulative effect of sacks/hits that adds up over time, then the time-reversed analysis should not show the pattern that you found in the original analysis (and should probably even show the opposite pattern, with quarterbacks getting more accurate as you subtract more hits while going back in time).

  • Oli Richford

    Thanks Nate – two excellent articles. Very eye opening.