Process over results.
If you have ever coached a sport, or (more likely) critiqued the job a coach was doing, you’ve come across these words. It is almost certain that at some point in their careers, all 32 general managers and all 32 head coaches in the NFL have called themselves a “process over results guy.” They will tell you that the way to build up to consistent success is through proven methods or informed experimentation married with consistent good habits and execution, even if it doesn’t add up to winning immediately.
That process is more and more visible to fans every year. The stereotype of the best coaches as gruff know-it-alls who condescended to media whom they assumed had zero understanding of the sport they were covering is rapidly giving way to coaches as people who are willing, even eager, to give a lesson in the game every time they’re asked a productive question.
For me, it’s made following the sport more fun and more fruitful, to have a chance to learn something new every time I watch a press conference and then be able to apply what I learned to watching games, or evaluating draft prospects, or even judging other teams and coaches.
The NFL Draft, though, remains a pretty murky endeavor. Teams meet with prospects in private visits where nobody really knows what is said (unless, of course, we’re talking about Xavier Legette and the Panthers), conduct research that’s pretty much unavailable to the public, and conduct their own medical investigations, leading to the inevitable assumed conclusion that teams know significantly more about who they’re drafting and not drafting than the public. Certainly, that’s helped by the fact that basically every single draft evaluator in the public eye has had more than one truly awful and visible miss.
But recently, that assumption has reason to be challenged. Arif Hasan’s work in compiling Consensus Draft Boards has led him and others to conclude that the masses aggregate into a pretty dang good talent evaluator, and thus a pretty dang good outcome predictor. Whether you’re tracking which picks get second contracts (and how big those contracts are) or using on-field analytics (Timo Riske at PFF used Wins Above Replacement, according to Hasan), the Consensus Big Board performs at least as well as the draft itself at organizing the eligible talent, which means it performs at least at the level of an average NFL team — which is to say better than a fair few of them. More importantly, Hasan notes that “it very much seems like NFL teams that consistently reach against the Consensus Big Board have poor draft classes.”
We may not have all the information, but we do have a powerful tool to guide us as fans on Draft Night. Individual guides and scouting reports are awesome (and shamelessly, I think the 2PC’s is particularly fantastic), but just by nature of the endeavor, are going to be wrong a lot. The Consensus Board isn’t infallible, but by balancing out any individual’s most-likely-to-be-wrong takes with the wisdom of the rest of the crowd, it gets as good as we could reasonably expect.
And thus, I get to wondering about draft grades. They’re a popular online target in the aftermath of the draft every year, and most of the complaints can be summarized thus: Assigning grades to draft picks and draft classes immediately after the draft without seeing how those picks and classes play out on the field is at best mindless clickbait and at worst an inherently self-aggrandizing exercise for a bunch of writers and wannabe ball-knowers. Maybe it also feels a little mean at times, as if the prospects themselves are being graded for things out of their control rather than the organizations. And sometimes, perhaps there is truth to this. But it appears to me that this complaint implies that results-based analysis is the proper way to judge a draft. And to that, I say…
Process over results.
The Consensus Board gives us a method to judge teams’ process that doesn’t just rely on our own evaluation. Once we have that, the job of grading can still be more complex and interesting than simply calculating which teams reached and which teams found steals. It’s much easier to check out a team’s roster, staff, and previous results to see their needs and trajectory than it is to forecast an individual’s chances of success, and you still get variance on how different people weigh those things differently. So we have a way to consistently judge individual teams against history and the masses (other teams and the public), and we have multiple people’s perspectives on the way those judgements reflect teams’ conditions and situations. To me, that’s a recipe for interesting content.
There is of course value in looking back on draft classes years after the fact and marveling at how good or bad they ended up being. I’m still equal parts awed and annoyed by the New Orleans Saints getting 7 players who played 90+ NFL games (including 3 All-Pros, though one didn’t earn the honor with them) in their 2017 class. But that’s a different exercise, and it implicates coaching staffs as much as it does front offices. Grading drafts in the immediate aftermath is pure thought experiment: Einstein, not Newton. It’s the one time the public has a reasonably fair opportunity to match wits with NFL decision-makers. That’s an encounter that I, for one, want to see every year.
