Yesterday, Chris Harris Jr. and other Broncos took much issue to an article on ESPN that projected the defending Super Bowl champions to go 7-9 in 2016. The author of the article in question was Aaron Schatz, founder of Football Outsiders and the concept of DVOA. FO has been a read of mine for a long time, as long as Schatz started up DVOA anaylsis in 2003.
Schatz’s bearish forecast on the 2016 Broncos hits on some common points he’s made in the past: extreme sides of the ball, good or bad, tend to regress to the mean, and performance on defense is more inconsistent than offense from season to season. Throw in a projection of Paxton Lynch more likely to “bust” than not, and Schatz sees a team that’s more like the Rams teams of recent years than a repeat champion.
As I said above, DVOA projections have been run for the past 13 seasons. So given the disrespect that Schatz has inadvertently injected in the Broncos’ locker room, let’s take a look at that track record.
First, it’s important to note that what’s being dealt with here are mean wins. FO runs many simulations of the season in question based upon their data, and averages the resulting records together. So a team projected to have around 10 mean wins will have a few projections in which everything goes right and the team goes 13-3 or better, and a few in which everything goes wrong and the team has a losing record.
As a result, when FO projects mean wins based upon DVOA, it’s unlikely to see teams with either a very good or very bad record. After rounding to whole numbers, I count only 13 of 416 team seasons in which a team was projected to have more than 11 mean wins or fewer than 5. For this reason, as I evaluate DVOA mean win projections of the past, I will normalize on a scale that has wins, both projected and actual, in a range from 11 to 5. Think of it as a table like this:
|Wins||Type of Season|
Using this normalized scale, how do the DVOA mean win projections look? Well, it means that FO overrates the entire league by about 0.024 wins. That’s a pretty thin margin. So is it time for Broncos fans to start gulping? Well, let’s take a look at the team-by-team breakdown for a rosier picture. The section on the left indicates the difference in mean wins to actual wins. Positive numbers mean that DVOA mean wins underrated the team, while negative numbers means it overrated the team. On the right, the standard deviation of the differences are listed, sorted from lowest to highest. The lower the standard deviation, the more consistent the team was underrated or overrated.
You’ll see Denver listed prominently there as the 7th most underrated team, and the 3rd most consistent, which is a good thing if you’re underrated. That looks better, doesn’t it? The breakdown of each Broncos season is as follows:
|Year||Mean Wins||Actual Wins
The one interesting point I’ll make here is that in 2009 DVOA projected the Broncos to be the worst team in the league, yet they finished 8-8. One year later, it projected around 8 mean wins, and Denver was the second-worst team in the league. Just one year too late on that one.
After having taken you on an adventure with numbers, this is where I break from that form and say to step away from the numbers. Thousands of simulations can be conducted, but only one of them will become reality. Shit happens in the NFL, there are tons of unknowns when rookies and former second stringers step on the field, and every prognosticator is going to get some correct and some incorrect due to not much more than sheer luck.
Meanwhile, there’s nothing wrong with some subjective guesses on how each team in the NFL will fare in the upcoming season, especially since those are better understood to be guesses. The FO writers themselves do this every year. Which is why I thought it was curious that the point of disagreement Schatz had the most with the mean wins DVOA spit out was not with the Broncos, but the Raiders (the ESPN article had them at 6-10).
Lots of response about our early forecast in Todd McShay 2017 mock draft. I'm writing more next wk. Yes, I disagree w/ system about Raiders.
— Aaron Schatz (@FO_ASchatz) May 6, 2016
Schatz isn’t the only one–I’m seeing a growing trend among analysts that are bullish on the Raiders. If I may enter my own subjective opinion, I sense some underdog bias in play among the masses. The Raiders have been terrible for the past 13 years, never generating a winning record, and now that they’ve finally shown some glimmers of competency, I feel like heartstrings are being tug to give the poor Raiders a chance. More likely, the Raiders could vary anywhere from very good to very bad, because again, shit happens. But I’ll conclude with a couple statements:
- If DVOA is right about the 2016 Broncos, I sure as hell hope they’re also right about the 2016 Raiders. Denver missing the playoffs is easier to swallow when Oakland is even worse.
- The Raiders are the 2nd most overrated team in DVOA mean win projection history.