Which are More Accurate: Fantasy Football Rankings or Projections?

When deciding how to compare players in fantasy football, managers have an important decision: whether to compare players with rankings or projections.  In this article, we compare the accuracy of fantasy football rankings vs. projections.

There are a few possible reasons to expect that rankings may be more accurate than projections, but there are also several reasons to expect that projections may be more accurate than rankings.  In the case of rankings, many more analysts provide rankings than projections, so the “wisdom of the crowd” includes a larger crowd for rankings, which may reduce error.  Some people argue that the top RB/QB/WR scores about the same number of points every year, so their position ranking matters more their projections.  However, there is often considerable variability in actual fantasy points from year to year for the same position rank.  For instance, there was a 60+ point difference in the top RB from 2008 to 2009 and a 50+ point difference in the top RB from 2014 to 2015 (according to Pro-Football-Reference.com).  This is just one example of variability in actual performance for the same position rank, but there are many other examples.  Moreover, what matters for fantasy performance is not whether you had the top ranked RB, but rather, how many more points your RB scored relative to other RBs.

As a result, there are several reasons to expect that projections may be more accurate than rankings.  First, projections can be customized to your league settings (unlike rankings).  Second, projections tell you how much players are better than each other, whereas rankings do not.  Moreover, you can always calculate rankings from projections but you cannot reverse engineer projections from rankings.  You can also account for variability and injury risk when examining projections, as we do in our tools.

Method

We were interested in seeing whether rankings or projections were more accurate for predicting actual performance, so our criterion was actual fantasy points scored.  We compared the accuracy of 2015 rankings versus projections using standard league scoring settings.  We used R-squared (R²) to evaluate prediction accuracy rather than MASE, because we were interested in relative accuracy, not absolute accuracy (i.e., position rank is not on the same metric as actual fantasy points scored, so estimates of absolute accuracy would not be meaningful).  For rankings across positions, we used FantasyPros Expert Consensus Rankings (i.e., expert rankings: ECR) and Average Draft Position (i.e., crowd rankings; ADP).  We also determined within-position rankings from the ECR and ADP.  For projections, we calculated the average of projections across sources using our Projections tool.

R Script

You can download the R script for the analysis here:

https://github.com/FantasyFootballAnalytics/FantasyFootballAnalyticsR/blob/master/R%20Scripts/Posts/Accuracy%20of%20Rankings%20vs%20Projections/AccuracyOfRankingsVsProjections.R

Results

FantasyPros had 167 sources of rankings to include in their “wisdom of the crowd” for expert rankings.  We had 11 sources of projections to include in our “wisdom of the crowd” for projections, except for K and DST (7 sources).

Despite having considerably fewer sources, projections (R² = .53) were more accurate than expert rankings (R² = .27) and crowd rankings (R² = .23) in predicting actual performance.  Projections were nearly twice as accurate as rankings.

When examining within-position rankings versus projections:

QB: Projections (R² = .50) were more accurate than expert rankings (R² = .37) and crowd rankings (R² = .29)

RB: Expert rankings (R² = .41) were more accurate than projections (R² = .36) and crowd rankings (R² = .32)

WR: Projections (R² = .48) were more accurate than expert rankings (R² = .44) and crowd rankings (R² = .29)

TE: Projections (R² = .47) were more accurate than expert rankings (R² = .43) and crowd rankings (R² = .25)

K: Neither expert rankings (R² = .09) nor crowd rankings (R² = .03) nor projections (R² = .02) were accurate, but rankings were more accurate

DST: Neither expert rankings (R² = .22) nor crowd rankings (R² = .17) nor projections (R² = .08) were accurate, but rankings were more accurate

Conclusions

In general, projections were more accurate than rankings, especially for QBs, WRs, and TEs.  Projections were nearly twice as accurate as rankings.  Interestingly, crowd rankings were slightly less accurate than expert rankings.  Neither rankings nor projections were accurate for Ks and DSTs (consistent with our prior findings), but were slightly more accurate for rankings than projections.  This may be because there were simply few sources of projections for Ks and DSTs.  We expect projections to continue to be considerably more accurate than rankings as we continue to add more sources of projections.  Hopefully, this results in an increased accuracy for predicting Ks and DSTs.

The bottom line is that we suggest using projections instead of rankings because 1), projections can be adapted to your league settings (unlike rankings) 2) projections tell you how much players are better than each other, 3) you can calculate rankings from projections but you cannot calculate projections from rankings, 4) you can account for variability risk and injury risk when using projections (as we do in our tools), and 5) projections are generally more accurate than rankings.  You can use customized projections for your league settings using our Projections tool.