How much stock can we put in Pro Football Focus rankings? Part 1: The process

As the NFL offseason presses on, fans are evaluating how players from their favorite team performed in 2019 in order to determine how they view the position going into both free agency and the draft. While some positions have much easier data to compare, others are very difficult to quantify with statistics. With many of these positions, many often turn to the Pro Football Focus (PFF) player grades they issue for the season. There are many fans who swear by the rankings while others dismiss them completely. So how much can fans truly trust these grades?

The first thing which is needed is to understand how PFF breaks down each player in order to come up with an overall score. According to their website pff.com they give following explanation as to how they determine their grades:

On every play, a PFF analyst will grade each player on a scale of -2 to +2 according to what he did on the play.

At one end of the scale you have a catastrophic game-ending interception or pick-six from a quarterback, and at the other a perfect deep bomb into a tight window in a critical game situation, with the middle of that scale being 0-graded, or ‘expected’ plays that are neither positive nor negative.

Each game is also graded by a second PFF analyst independent of the first, and those grades are compared by a third, Senior Analyst, who rules on any differences between the two. These grades are verified by the Pro Coach Network, a group of former and current NFL coaches with over 700 combined years of NFL coaching experience, to get them as accurate as they can be.

From there, the grades are normalized to better account for game situation; this ranges from where a player lined up to the dropback depth of the quarterback or the length of time he had the ball in his hand and everything in between. They are finally converted to a 0-100 scale and appear in our Player Grades tool.

Season-level grades aren’t simply an average of every game-grade a player compiles over a season, but rather factor in the duration at which a player performed at that level. Achieving a grade of 90.0 in a game once is impressive, doing it 16 times in a row is more impressive.

It is entirely possible that a player will have a season grade higher than any individual single-game grade he achieved, because playing well for an extended period of time is harder to do than for a short period.

Similarly, playing badly for a long time is a greater problem than playing badly once, so the grade can also be compounded negatively.

It’s important to understand that PFF is not grading talent in these numbers, rather strictly performance on the field. Talented players can have bad games, runs, or even seasons, and often players without nearly as much talent can put together impressive play on the field. We are not necessarily telling you who the best players are. Our rankings are more of a performance evaluation, and a reflection of how efficiently a player made plays in the time he was on the field.

Another key benefit to PFF’s grading system is building a complete picture of a player’s performance.

Even watching a game closely can result in something of a mental highlight reel on a player – remembering the few good and bad plays and making an overall judgement based on the balance of those – but it can be the other 40 snaps in the game that are being ignored that held the true key to his performance. Was he gaining a slight edge every one of those snaps, or was he struggling just to maintain parity?

By recording performance on every single snap, we come to a more complete conclusion and evaluation. Just because a player was quiet during the game does not mean he played poorly, and in certain positions it could mean he played very well, but had little to feature on the highlight reel.

One thing which PFF does in its rankings is looks at every play a player is on the field and not just the “splash plays” which can lead to a great statistical performance. Their explanation of their PFF score versus game stats is explained as follows:

PERFORMANCE VS PRODUCTION

We aren’t grading players based on the yardage they rack up or the stats they collect. Statistics can be indicative of performance but don’t tell the whole story and can often lie badly. Quarterbacks can throw the ball straight to defenders but if the ball is dropped, you won’t see it on the stat sheet. Conversely, they can dump the ball off on a sequence of screen passes and end up with a gaudy looking stat line if those skill position players do enough work after the catch.

PFF grades the play, not its result, so the quarterback that throws the ball to defenders will be downgraded whether the defender catches the ball to notch the interception on the stat sheet or not. No amount of broken tackles and yards after the catch from a bubble screen will earn a quarterback a better grade, even though his passing stats may be getting padded.

With a greater understanding of how PFF rankings work, they may seem like a fantastic tool to help evaluate every player in the NFL. While they can be useful, it is also important to remember that those individuals judging how a player performed on a given play do not have the inside knowledge as to what the call was which determine the players responsibility. Did the receiver run the wrong route or did the quarterback make the wrong read? Should the corner have had safety help over the top who did not come through with their responsibility? These are things that can sometimes be indicated on plays, but cannot always be known for sure. It is a large part of what makes judging these plays so difficult.

While I like to use PFF grades in helping to evaluate the performance of various Steelers’ players from 2019, many times it depends on the position they play as to how much stock I take in the score. My personal determination many times comes down to how much contact a player receives or gives or any play. For example, when an offensive lineman goes to block a defender, making the contact and how the player performs based on the person they are in contact with makes it much easier to grade the play. The same is often true on the opposite side of the ball when a defensive lineman is trying to shed the block of the offensive player. It is also somewhat effective when looking at edge rushers when they are attempting to get to the quarterback. These types of players have more reliable grades, in my opinion, since the intention is much easier to determine.

Where things get tricky is looking at receivers and secondary players as they are operating in open space. Exactly what a player is supposed to do when not in contact with another one is more difficult to judge and could make these numbers less reliable.

The bottom line is each person asked to determine for themselves how much stock they put in to player grades. While they are not perfect, I believe Pro Football Focus does help put things in perspective as they are evaluating players across the league on every team when I am just watching all the snaps with the Steelers. I may think a particular Steelers’ player is having a great year, but there may be other players I am not focusing on other than highlights to see how they really stack up. So while player grades can be an interesting point of discussion, they are also some of the best quantitative numbers associated with players at certain positions and can be useful in evaluating someone’s overall performance for the season.

After seeing how this process works, next time we’ll dive in deeper into some rankings of our favorite Steelers’ players as well as others around the NFL from 2019 to see if their scores seem appropriate.

So do you think PFF grades are reliable source in evaluating a player’s performance based on their process of determining grades? Please vote in the pole and leave your thoughts in the comments below.