Last offseason we introduced you to the concept of subset scouting and explored the limitations of a methodology which attempts to judge a player only by looking at three or four games without placing those games within the context of that player’s whole season.
For most players you could take their best four games of the season and see a Pro-Bowler or you could take their worst four games and see at best a subpar starter or at worst a player who should barely be scraping a roster spot, let alone starting.
A prime example of such polarizing subset performances from 2013 would be Miami defensive end Olivier Vernon who earned a creditable +10.3 overall grade in his best four games but turned in an abysmal -17.0 overall grade in his worst four.
Now that is one of the wider grade ranges from this season but along with the Stock Report series which we introduced this season, it goes to illustrate how individual player performance fluctuates and is a constant only in extremely rare cases.
Scouting on minimal games on its own can be dangerous and lead to wild misinterpretations of a player’s abilities or performance level. However, turning this around and developing on the concept of subset scouting this offseason, we are going to use it to dig deeper into our overall grades to give some context of how those grades are put together through a season and how consistent a player is.
There is always more value in looking into our grades than simply taking them at face value. Did a player put together his +16.0 overall grade with 16 games of +1.0 or did he put it together with four +6.0’s, four 0.0’s and eight -1.0’s? They both end up in the same place, but clearly the grade constructions paint the picture of two very different players.
A Comparison on the Edge
The first two players under the microscope for this new look at Subset Scouting are our top two free agent edge defenders who will hit the open market untagged when free agency kicks off on Tuesday. For the second straight season Michael Bennett tops our list while a year removed from playing under the franchise tag, Michael Johnson of the Bengals comes in at No. 4 behind the tagged Greg Hardy and Brian Orakpo.
A year ago Bennett was our poster boy for the perils of Subset Scouting with his best four games earning a +21.2 overall grade second only to Cameron Wake (+32.8) for the best “Subset+” grade among 4-3 defensive ends. On the other end of the scale, however, were his worst four games (Subset-) that amassed a -11.5 overall grade, a figure that only 16 4-3 defensive ends (playing at least 10 games) surpassed.
Depending upon which games you selected to judge Bennett on last season you could have come away with a stark contrast in your opinion of him as a player.
Though a more consistent player last season, Johnson offered his own pitfalls in your assessment of him on a subset basis. His best four games were disproportionately boosted by a four-sack game against the Washington Redskins in Week 3 where he beat up on Jordan Black after Trent Williams left 14 snaps into the game.
You credit him for taking advantage of a matchup against favorable competition, but looking beyond that, his Subset+ (+8.8 of +17.2 in one game) was mediocre, as were his “Middle-8” games (+2.7). The best thing about Johnson’s 2012 season, thanks in large part to his consistent play against the run, was that he didn’t have the extremely poor games that dogged Bennett’s 2012. His Subset- of -5.8 overall was far better than Bennett’s, offering at least some contribution in every game.
The Bengals then made a shrewd judgment call in tagging Johnson last year and got a solid one-year rental from a player who they clearly deem as not being worth a third big contract on their defensive line. Overall, Johnson was better and more consistent this season, aided largely by some stellar work against the run, and his Pass Rushing Productivity more moved around (61 total pressures, up from 55) than fell away as his decline in sack numbers (13 to 5) would suggest.
Johnson showed more consistent upside as a pass rusher this season (Subset+ of +11.4, 11th-best in the league) and made strides in his Middle-8, but his Subset- of -8.9 (five games of -1.3 grade or worse) was among the league’s worst, and very similar to Bennett’s last season in Tampa Bay in that regard with too many games with no impact as a pass rusher.
That weakness and inconsistency was something that Bennett eradicated in a spectacular season with the Seahawks on their 2013 Super Bowl run. A year on from being the poster boy for the pitfalls of Subset Scouting, Bennett is now the poster boy for how to play during a “Prove It” one-year contract.
The Buccaneers dabbled with Bennett at defensive tackle in 2012 (11.0% of 985 snaps) but Pete Carroll and Dan Quinn made it a focus of his deployment, nearly tripling his playing time on the inside (32.3% of 619 snaps). With a focus on getting the best out of Bennett rather than leaving him out there at defensive end to struggle against the best right tackles he faced (-10.2 Subset- pass rush in 2012), the Seahawks got positive contributions from him week in and week out in 2013.
Bennett’s Subset+ pass rush grade fell away a little in 2013, but his Middle-8 was much improved and most importantly his Subset- marking out a consistently destructive and productive season. Only Robert Quinn (+26.5) had a better Middle-8 grade as a pass rusher than Bennett’s +10.6 among 4-3 defensive ends as a measure of the games you get most consistently from a player, while Bennett tied Quinn for the best Subset- as a pass rusher last season (-2.0).
Betting on Improvement
Signing a player to a free agent contract isn’t about what they have done to this point; it is about (with that prior performance in mind) what you think they will do for you over the course of the contract that you are offering them.
In 2012, Bennett had five games with a pass rush grade of -1.0 or worse. A year later, in Seattle, he had none. In his last two seasons in Cincinnati, Johnson has had five in each season (including the playoffs) and it is this inconsistency that he must remove from his game in order to take the next step that Bennett did.
The two players show similar profiles against the run (consistently among the league’s best in each grading subset), so it’s the development of consistency as a pass rusher that Bennett has over Johnson as both enter free agency for the second season in a row.
So the question in terms of development for Johnson is whether you can get him to make that leap as a pass rusher and how you do it? If he is a player limited in terms of his ability to making a consistent impact as a pass rusher (and his career track record would suggest he is) then can you deploy him inside to draw out the consistent production that Seattle got from Bennett in 2013 or is he simply that limited edge player?
Bennett’s displays in 2013 showed his versatility as a defensive lineman and the production and consistency that he developed should ensure that this year he finds the sort of market that he would have hoped and might have been expecting to find 12 months ago.
For Johnson that inconsistency in his pass rush paired with his precipitous decline in sacks from 2012 to 2013 may see a reduced market compared to that which many expected him to have a year ago before the Bengals tagged him. A solid multi-year deal may still be waiting for Johnson on March 11th, but his inconsistency and limitations on the edge is something that his suitors should be aware of and have a plan to supplement if they are expecting him to be an elite defender for them.
Follow Ben on Twitter @PFF_Ben