My previous two posts assessing the Steelers’ drafting success since Kevin Colbert fully took the reins of the Front Office in 2001 focused on the first round (Part 1) and the second round (Part II ). In theory this is the post assessing the lower rounds, but is as my wont lately I went into full stats geek mode, loaded all the relevent information into a spreadsheet, and started digging around. As a result I have the definitive pronouncement on the relative value of the Steelers drafts for each round in comparison to the rest of the league. At least until someone else looks at the numbers and disagrees with my interpretation of them.
As I looked at the some of the overall draft assessments one thing which seemed to be missing from the analysis is the position teams were drafting. That doesn’t make a lot of sense if you think about it. Teams drafting at the lowest end of the first round are choosing from a pool of players who more closely resemble the second-round pool for the teams picking at the top of each round. So it makes little sense to assume the same value for Pick #32 and Pick #1, for instance. Yet this seems to be the way round-by-round comparisons are typically done.
The Steelers’ success in the past decade has guaranteed a lower round position for many of the years. On the other hand, teams do trade around, and the Steelers have done a bit of horsetrading themselves, most notably to take Troy Polamalu. So I decided to determine the success of each team in each round using the Pro-Football-Reference Career Approximate Value (hereafter CarAV) for the players drafted by each team. Adding these together for the ten years gave me the total value of the players each team had picked during those ten drafts, and I averaged it out for the number of picks in that round a team had during that time. (Some teams had more than ten, some less, for various reasons.) This gave me a list by draft round of how well each team did, relative to the rest of the league. For the first round, here were the top ten teams:
But then I averaged the position of the Round One picks each team had during the decade. The Colts had the highest average pick by far: 26.75. (In other words, on average they were picking around the 27th pick of Round One.) Cleveland had the lowest, at 11.5. So let’s look at where those top ten teams picked on average:
Now let’s redo the league list, compensating for the pick number. (I multiplied the CarAV by the team’s average pick number, then divided by the average pick league-wide for that round. During this particular 10 years that number happened to be 16.53.) Here’s the new set of top-ten teams:
You can see the Steelers, Ravens and the Colts move up, as teams on the list who on average were picking at the lower end of the league. The Falcons and Vikings drop off the list, to be replaced by the Cowboys and the Giants.
It is reasonable to ask whether this is entirely fair. After all, it isn’t as if every player in a given round played the same position. Then it would clearly be an enormous advantage to have a higher pick. When a number of different positions are being chosen, though, there may still be a few top players at less-popular positions available. Nonetheless, I think it is a fair way to adjust the scale. The Advanced NFL Stats studies have demonstrated the top couple of players at a given position are generally a good bit better than those who follow, at least in the short term. It is very advantageous to pick in the upper part of a round—otherwise there would just be a lottery.
There are lots of other things one could adjust by, and I messed about some of those. Number of years the player was a starter, number of Pro Bowls and all-Pro, and so on, as well as a "bust rate" I invented and assigned. But ultimately it seems to me a perfectly good evalution can be made from just the CarAV for each player. Pro Bowls, for instance, are partially a matter of popularity and visibility for a given play and/or team. Number of years as a starter has at least something to do with the policies and management efficiencies of the team(s) the player is on and how much need there is for the player to start immediately. And at least some of those things are already taken into account in the CarAV. So in the end, I stuck with the CarAV.
Rather than try to make a bar chart for the whole league, which would be practically unreadable, I decided to plot the Steelers, the Ravens, and the two best and worst teams in each round. (If PIT or BAL was in either the top or bottom two in a given round I threw in another nearby team.) I wanted to answer two questions. The first question: is any team drafting consistently well (or poorly) throughout the draft? But the really interesting question for denizens of the AFC North is the second: is Ozzie Newsome, the Ravens’ GM, or Kevin Colbert, the GM for the Steelers, really the Wizard?
Here's the results. As you can see, I plotted each round both with the original numbers and the numbers adjusted for pick. The numbers after each team name in the legend are where they placed in the CarAV and where they placed when the numbers were adjusted for average draft position. All charts are the same height for ease of reading, but the top number is considerably less in each round.
Obviously IND is the clear winner when adjusted for pick number. The Steelers and Ravens both move up, the Steelers more than the Ravens, but unfortunately the Ravens are still ahead by a nose. Round One: Advantage Newsome.
Interestingly, things don’t change as much in Round Two, and the order doesn’t change at all. (And before anybody asks, I did recalculate the average pick number for each round.) Advantage: Colbert.
In Round Three, adjusting for pick number puts New England just ahead of Carolina. Pittsburgh and Baltimore both benefit from the adjustment, Baltimore more than Pittsburgh. Colbert is the clear winner.
Once again the top two switch places, but the order is otherwise unaffected. Kevin Colbert is still the man...
Round Five is where the Steelers really fell short during between 2001 and 2010. And if you’re going to fail consistently, better to do it in the lower rounds where the consequences are much less. Ozzie Newsome forges ahead.
Ozzie Newsome is still dominating, I fear.
Dallas and the Giants switch places, but otherwise everyone stays the same. I’m guessing Pittsburgh is 17th place rather than far worse almost entirely because of Brett Keisel, but I haven’t checked that opinion.
So how about those two questions? It looks like Kevin Colbert has beaten out Ozzie Newsome, but is this true? And which teams are drafting consistently well, or poorly? Once again I ran the numbers both ways. I added up the total CarAV for each team, and ranked the teams accordingly, and did the same for the CarAV adjusted for average pick number. (This has the advantage over the ranking numbers of weighting for the earlier draft rounds.) Here are the results:
One judge of this metric is how well it lines up with how the teams performed during the ten-year period in question. (For example, this Forbes Magazine article evaluating the draft used what was clearly a flawed metric, because their three worst teams were the Pittsburgh Steelers, the St. Louis Rams, and the New England Patriots.) I think a perusal of the rankings for the CarAV adjusted for draft pick shows a very good correlation between how well a team did in the last decade and where they rank in my chart. For example, the Saints may have won a Super Bowl, but they were bad a lot of the time. Teams like the Cowboys and the Eagles didn't make it to the big game but they were generally competitive.
How well a team drafts and how well they use the players are two different issues, of course. But I believe we can state this with confidence: the Steelers have been one of the very best teams in the league at identifying talent in the draft, particularly in the upper rounds where it counts the most. Their record during this time indicates they are also one of the best teams in evaluating the talent they drafted and utilizing them well, or cutting their losses if necessary.
And although Pittsburgh beats Baltimore, it isn't by a lot, although it is significant. So if Ozzie Newsome is the Wizard, I guess that makes Kevin Colbert into Harry Potter. An older and badder-looking Harry Potter, especially in the shades...
How much do the Steelers beat the Ravens by? Here are the numbers. Judge for yourselves. And note how large the difference is between the best teams and the worst ones.
Things can change relatively rapidly in the league, and new management can infuse a great deal of hope into a franchise—witness San Francisco and Detroit. So all of this may look quite different in five years. But for the past ten years, it looks as if 50% of the AFC North really knew how to draft.