Assessing the Steelers' Drafting Success During the Kevin Colbert Era: The Final Word

PITTSBURGH - SEPTEMBER 18: Director of football operations Kevin Colbert watches the Pittsburgh Steelers warm up prior to the game against the Seattle Seahawks on September 18, 2011 at Heinz Field in Pittsburgh, Pennsylvania. (Photo by Jared Wickerham/Getty Images)

My previous two posts assessing the Steelers’ drafting success since Kevin Colbert fully took the reins of the Front Office in 2001 focused on the first round (Part 1) and the second round (Part II ). In theory this is the post assessing the lower rounds, but is as my wont lately I went into full stats geek mode, loaded all the relevent information into a spreadsheet, and started digging around. As a result I have the definitive pronouncement on the relative value of the Steelers drafts for each round in comparison to the rest of the league. At least until someone else looks at the numbers and disagrees with my interpretation of them.

As I looked at the some of the overall draft assessments one thing which seemed to be missing from the analysis is the position teams were drafting. That doesn’t make a lot of sense if you think about it. Teams drafting at the lowest end of the first round are choosing from a pool of players who more closely resemble the second-round pool for the teams picking at the top of each round. So it makes little sense to assume the same value for Pick #32 and Pick #1, for instance. Yet this seems to be the way round-by-round comparisons are typically done.

The Steelers’ success in the past decade has guaranteed a lower round position for many of the years. On the other hand, teams do trade around, and the Steelers have done a bit of horsetrading themselves, most notably to take Troy Polamalu. So I decided to determine the success of each team in each round using the Pro-Football-Reference Career Approximate Value (hereafter CarAV) for the players drafted by each team. Adding these together for the ten years gave me the total value of the players each team had picked during those ten drafts, and I averaged it out for the number of picks in that round a team had during that time. (Some teams had more than ten, some less, for various reasons.) This gave me a list by draft round of how well each team did, relative to the rest of the league. For the first round, here were the top ten teams:

1st_rnd_cav_medium

But then I averaged the position of the Round One picks each team had during the decade. The Colts had the highest average pick by far: 26.75. (In other words, on average they were picking around the 27th pick of Round One.) Cleveland had the lowest, at 11.5. So let’s look at where those top ten teams picked on average:


1st_rnd_avg_pick_medium

Now let’s redo the league list, compensating for the pick number. (I multiplied the CarAV by the team’s average pick number, then divided by the average pick league-wide for that round. During this particular 10 years that number happened to be 16.53.) Here’s the new set of top-ten teams:


1st_rnd_adj_car_chart_medium

You can see the Steelers, Ravens and the Colts move up, as teams on the list who on average were picking at the lower end of the league. The Falcons and Vikings drop off the list, to be replaced by the Cowboys and the Giants.

It is reasonable to ask whether this is entirely fair. After all, it isn’t as if every player in a given round played the same position. Then it would clearly be an enormous advantage to have a higher pick. When a number of different positions are being chosen, though, there may still be a few top players at less-popular positions available. Nonetheless, I think it is a fair way to adjust the scale. The Advanced NFL Stats studies have demonstrated the top couple of players at a given position are generally a good bit better than those who follow, at least in the short term. It is very advantageous to pick in the upper part of a round—otherwise there would just be a lottery.

There are lots of other things one could adjust by, and I messed about some of those. Number of years the player was a starter, number of Pro Bowls and all-Pro, and so on, as well as a "bust rate" I invented and assigned. But ultimately it seems to me a perfectly good evalution can be made from just the CarAV for each player. Pro Bowls, for instance, are partially a matter of popularity and visibility for a given play and/or team. Number of years as a starter has at least something to do with the policies and management efficiencies of the team(s) the player is on and how much need there is for the player to start immediately. And at least some of those things are already taken into account in the CarAV. So in the end, I stuck with the CarAV.

Rather than try to make a bar chart for the whole league, which would be practically unreadable, I decided to plot the Steelers, the Ravens, and the two best and worst teams in each round. (If PIT or BAL was in either the top or bottom two in a given round I threw in another nearby team.) I wanted to answer two questions. The first question: is any team drafting consistently well (or poorly) throughout the draft? But the really interesting question for denizens of the AFC North is the second: is Ozzie Newsome, the Ravens’ GM, or Kevin Colbert, the GM for the Steelers, really the Wizard?

Here's the results. As you can see, I plotted each round both with the original numbers and the numbers adjusted for pick. The numbers after each team name in the legend are where they placed in the CarAV and where they placed when the numbers were adjusted for average draft position. All charts are the same height for ease of reading, but the top number is considerably less in each round.

1st_rnd_adj_cav_medium

Obviously IND is the clear winner when adjusted for pick number. The Steelers and Ravens both move up, the Steelers more than the Ravens, but unfortunately the Ravens are still ahead by a nose. Round One: Advantage Newsome.

2nd_rnd_bar_chart_medium

Interestingly, things don’t change as much in Round Two, and the order doesn’t change at all. (And before anybody asks, I did recalculate the average pick number for each round.) Advantage: Colbert.

3rd_rnd_bar_chart_medium

In Round Three, adjusting for pick number puts New England just ahead of Carolina. Pittsburgh and Baltimore both benefit from the adjustment, Baltimore more than Pittsburgh. Colbert is the clear winner.

4th_rnd_bar_chart_medium

Once again the top two switch places, but the order is otherwise unaffected. Kevin Colbert is still the man...

5th_rnd_bar_chart_medium

Round Five is where the Steelers really fell short during between 2001 and 2010. And if you’re going to fail consistently, better to do it in the lower rounds where the consequences are much less. Ozzie Newsome forges ahead.


6th_rnd_bar_chart_medium

Ozzie Newsome is still dominating, I fear.


7th_rnd_bar_chart_medium

Dallas and the Giants switch places, but otherwise everyone stays the same. I’m guessing Pittsburgh is 17th place rather than far worse almost entirely because of Brett Keisel, but I haven’t checked that opinion.

So how about those two questions? It looks like Kevin Colbert has beaten out Ozzie Newsome, but is this true? And which teams are drafting consistently well, or poorly? Once again I ran the numbers both ways. I added up the total CarAV for each team, and ranked the teams accordingly, and did the same for the CarAV adjusted for average pick number. (This has the advantage over the ranking numbers of weighting for the earlier draft rounds.) Here are the results:

Full_league_ranking_medium

One judge of this metric is how well it lines up with how the teams performed during the ten-year period in question. (For example, this Forbes Magazine article evaluating the draft used what was clearly a flawed metric, because their three worst teams were the Pittsburgh Steelers, the St. Louis Rams, and the New England Patriots.) I think a perusal of the rankings for the CarAV adjusted for draft pick shows a very good correlation between how well a team did in the last decade and where they rank in my chart. For example, the Saints may have won a Super Bowl, but they were bad a lot of the time. Teams like the Cowboys and the Eagles didn't make it to the big game but they were generally competitive.

How well a team drafts and how well they use the players are two different issues, of course. But I believe we can state this with confidence: the Steelers have been one of the very best teams in the league at identifying talent in the draft, particularly in the upper rounds where it counts the most. Their record during this time indicates they are also one of the best teams in evaluating the talent they drafted and utilizing them well, or cutting their losses if necessary.

And although Pittsburgh beats Baltimore, it isn't by a lot, although it is significant. So if Ozzie Newsome is the Wizard, I guess that makes Kevin Colbert into Harry Potter. An older and badder-looking Harry Potter, especially in the shades...

How much do the Steelers beat the Ravens by? Here are the numbers. Judge for yourselves. And note how large the difference is between the best teams and the worst ones.

Full_league_numbers_medium

Things can change relatively rapidly in the league, and new management can infuse a great deal of hope into a franchise—witness San Francisco and Detroit. So all of this may look quite different in five years. But for the past ten years, it looks as if 50% of the AFC North really knew how to draft.

X
Log In Sign Up

forgot?
Log In Sign Up

Please choose a new SB Nation username and password

As part of the new SB Nation launch, prior users will need to choose a permanent username, along with a new password.

Your username will be used to login to SB Nation going forward.

I already have a Vox Media account!

Verify Vox Media account

Please login to your Vox Media account. This account will be linked to your previously existing Eater account.

Please choose a new SB Nation username and password

As part of the new SB Nation launch, prior MT authors will need to choose a new username and password.

Your username will be used to login to SB Nation going forward.

Forgot password?

We'll email you a reset link.

If you signed up using a 3rd party account like Facebook or Twitter, please login with it instead.

Forgot password?

Try another email?

Almost done,

By becoming a registered user, you are also agreeing to our Terms and confirming that you have read our Privacy Policy.

Join Behind the Steel Curtain

You must be a member of Behind the Steel Curtain to participate.

We have our own Community Guidelines at Behind the Steel Curtain. You should read them.

Join Behind the Steel Curtain

You must be a member of Behind the Steel Curtain to participate.

We have our own Community Guidelines at Behind the Steel Curtain. You should read them.

Spinner.vc97ec6e

Authenticating

Great!

Choose an available username to complete sign up.

In order to provide our users with a better overall experience, we ask for more information from Facebook when using it to login so that we can learn more about our audience and provide you with the best possible experience. We do not store specific user data and the sharing of it is not required to login with Facebook.

tracking_pixel_9341_tracker