All News & Analysis

Home Cookin' - How NFL Stats Crews Affect IDP, Pt. 1

Have you ever been watching an NFL game as your IDP (individual defensive player) star is making plays all around the field only to be disappointed when you check the stats? Sure, he's registered a handful of solo tackles, but only one assisted tackle? You're sure you saw him contribute to many more stops. Chances are the game is being scored by an assist-stingy crew like the ones in St. Louis or Jacksonville. Unfortunately, there's nothing you can do about it except wait until next week.

The NFL defines tackles, assists, and passes defensed as unofficial statistics. In your fantasy football IDP league, however, these stats are very much official and often the difference between victory and defeat. I understand the NFL's position. These stats are much more subjective than others and can be very difficult to score, especially in real time.

Still, the inconsistency between NFL stat crews affects our fantasy football franchises. Therefore, in this three-part series I'll identify these inconsistencies and discuss what they mean for your IDP league. I've conducted five- and 10-year studies on each NFL team comparing home and away statistics to determine which NFL stats crews are biased in their application of solo tackles, assisted tackles and passes defensed.

Review

The word “bias” can be a bit tricky.  Most of the time when used in conversation, some level of motive is implied. From a statistical sense, motive is irrelevant. Bias strictly means that an outcome is systematically favored, no matter the reason. NFL stats crews aren't biased due to motive. Bias stems from the differences in definition of solo tackles, assists and passes defensed.

According to Jene Bramel at Footballguys, between the 2006 and 2007 seasons the league sent a video to its stats crews. The focus was to “credit only those defensive players who had a material effect on the outcome of the play” (Bramel, 2008). The league's goal was to improve consistency in scoring these statistics along with sacks. It worked to some degree. Assists and passes defensed numbers did go down across the board in 2007.

However, one video is hardly enough to standardize the way these statistics are scored. Rachel Cohen of ESPN detailed how crews are hired by each NFL team's public relations staff and how the size of the crew varies. The crews all flew to New York in 2000 to learn how to use the current Game Statistics & Information System, but haven't had any league-wide sessions since (unless you count the 2006 offseason video sent by the league). Jon Stratton, the Washington crew chief, would like to see crews trained more (Cohen, 2011). Another issue for crews is that many have now been moved to the endzone to make more room for sideline luxury boxes.

Methods

The five- and 10-year studies compare the differences between home and road statistics. A t-test was used to see if the differences were significant or simply a matter of normal variation. The results of these tests are given as a p-value.

The p-value is a percentage that displays how often these results would occur if there was no bias among stats crews. A very low p-value means the stats crew is scoring the statistic differently from the rest of the league. A standard value for significance is less than 5 percent. Therefore, if the p-value is less than 5 percent, there's evidence that the stats crew is biased.

I entered the home and away splits for the season and calculated the difference between the two for easy viewing. By using only the team home and away splits I've eliminated the team variable. The statistics compared are all from the same team. Due to divisional play, three of eight home and road opponents are the same.

Some may question whether stats crews award a higher number of statistics to the home team due to familiarity or favoritism. Based off the limited research done on this subject, it doesn't seem to be the case. For example, if a stats crew awards a higher number of assists to the home team, they usually also do so for the visiting team, as compared to the league average (Sitzmann, 2011 & 2012).

The goal of this study is to glean information that could lead to IDP fantasy football success. At the very least this study will raise awareness to the inconsistencies in IDP scoring.

Results

In Part 1 of this series we're going to focus on assists. In the five-year test, 14 teams had p-values less than 5 percent, showing significant evidence of bias when scoring assists. Therefore, it follows that NFL stats crews are biased when it comes to assists, systematically favoring certain outcomes.

Eight teams scored significantly more assists over the time period. They were Buffalo, Cincinnati, Detroit, Green Bay, New England, New York Giants, Pittsburgh and Washington.  Six teams scored significantly less assists over the time period. They were Arizona, Baltimore, Jacksonville, Miami, Philadelphia and St. Louis.

In 2009 the St. Louis Rams were credited for 17 assists in their eight home games at the Edward Jones Dome. In their eight road games they were credited for 119.  Take a look at the results for St. Louis:

St. Louis Year 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 10yr p-value
Home 72 45 53 37 45 35 17 50 47 68 0.01%
Assists Away 83 139 137 63 106 134 119 100 154 125 5yr p-value
Diff -11 -94 -84 -26 -61 -99 -102 -50 -107 -57 0.24%

As you can see the Rams defense recorded much lower assists numbers when they played in front of the home stats crew. This has occurred every year for the last 10.  Also consider that a stats crew that awards less assists, doesn't always award more solos. The Rams crew actually awarded less solos in 2008 and 2009. I'll focus on solos and their relationship to assists in Part 2 of this series. Now take a look at Washington and Cincinnati:

Washington Year 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 10yr p-value
Home 80 101 81 173 152 211 159 306 280 288 1.57%
Assists Away 101 101 79 97 134 93 121 105 137 108 5yr p-value
Diff -21 0 2 76 18 118 38 201 143 180 0.87%
Cincinnati Year 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 10yr p-value
Home 154 191 190 253 202 248 202 222 260 306 0.06%
Assists Away 119 165 124 141 149 169 129 130 92 113 5yr p-value
Diff 35 26 66 112 53 79 73 92 168 193 0.82%

Both of these teams have recorded many more assists from the home stats crew. Remember, I'm not saying that the stats crew awards more assists to the home team and this study is not testing for that. In these cases the Cincinnati and Washington stats crews are awarding more assists to both teams due to bias in how they define the assists statistic.

Assists Differences (Home – Away) for All 32 Teams

[table id=821 /]

|

Conclusions

Almost half the teams in the NFL (14) showed significant evidence of bias when recording assists. This bias comes from how each stats crew defines solo tackles and assists and the relationship of one to another. In Part 2 of this series we'll take a look at solo tackles and discuss this relationship further. Because of the complexities of awarding these statistics, it would be foolish to jump to conclusions based on a quick glance of this data.

For example, it would be premature and simplistic to conclude that players playing in Washington and Cincinnati will be better IDP plays based off of this study. At times these defenses were awarded significantly less solo tackles, so there's a balancing effect.

But it would be foolish to dismiss these results as well. In fact, last season Bengals linebacker Rey Maualuga had exactly 61 solo tackles and 61 assists. Vontaze Burfict posted 72 solos and 52 assists. Washington Redskins linebacker London Fletcher has carried IDP franchises with assists, posting 71 and 63 the past tw seasons.

In St. Louis, middle linebacker and IDP staple James Laurinaitis posted 118 solo tackles and only 24 assists in 2012. In Laurinaitis' four seasons for the Rams, his lowest solo number has been 98, while his highest assists total has been 37. That's quite a difference in distribution when compared with Cincinnati and Washington linebackers.

The low numbers in St. Louis could explain why a WLB has never posted consistent IDP numbers despite the Rams running a 4-3 system. It could just be that the WLBs just haven't been very good. Either way, it's a consideration. In Part 2 I'll break down the relationship between solos and assists and pinpoint which stats crews may benefit certain types of players. I'll take a look at the way a stats crew may approach scoring different types of plays and how these philosophies lead to the bias evident in the data.

The goal of this study was to glean information that would be beneficial to IDP fantasy football owners in the future. As we analyze the data further in Parts 2 and 3, I think there will be conclusions that do just that. At best, IDP owners will be able to use the data as a metric to assist in drafting and lineup decision making. At worst, I think the location of an NFL game could serve as a tiebreaker when making final IDP lineup decisions. In Part 2, I'll look at solo tackle numbers specifically, how the solo tackle values and assists are related, and how a stats crew's “tackle philosophy” impacts how they score certain plays. In part 3, I'll briefly look at the passes defensed numbers and draw some overall conclusions for IDP fantasy football owners.

References

Jene Bramel.  A Review of the NFL's Guidelines for Individual Defensive Statistics.   Available:  <http://subscribers.footballguys.com/2008/08bramel_idpstats.php>, 6 Aug. 2008.

Rachel Cohen.  Stats-taking at NFL games more art than science.  Available:  <http://sports.espn.go.com/wire?section=nfl&id=7399087>, 31 Dec. 2011.

Ryan Sitzmann.  2012 Tackles Issued by Home Scorekeepers.  Available:  <http://www.idpguru.com/2012/09/2012-tackles-issued-by-home-scorekeepers.html>, 2012.

All Featured Tools

Subscriptions

Unlock the 2023 Fantasy Draft Kit, with League Sync, Live Draft Assistant, PFF Grades & Data Platform that powers all 32 Pro Teams

$31 Draft Kit Fee + $8.99/mo
OR
$89.88/yr + FREE Draft Kit