Who Has the Best Fantasy Football Projections: ESPN, CBS, NFL.com, or FantasyPros?
12In prior posts, I demonstrated how to download, calculate, and compare fantasy football projections from ESPN, CBS, and NFL.com. In my last post, I demonstrated how to download FantasyPros projections, which aggregate projections from many different sources to increase prediction accuracy. In this post, I will compare fantasy football projections from ESPN, CBS, NFL, and FantasyPros, including our average and latent projections to determine who has the best fantasy football projections.
The R Script
The R Script for downloading fantasy football projections from FantasyPros is located at:
https://github.com/FantasyFootballAnalytics/FantasyFootballAnalyticsR/blob/master/R%20Scripts/Posts/Evaluate%20Projections.R
To compare the accuracy of the projections, we will use various metrics including:
- R-squared (R2) – higher is better
- Harrell’s c – higher is better
- Somer’s Dxy – higher is better
- Intraclass correlation (ICC) – higher is better
- Mean absolute error (MAE) – lower is better
- Root mean squared error (RMSE) – lower is better
- Mean absolute percentage error (MAPE) – lower is better
- Mean absolute scaled error (MASE) – lower is better
Whose Predictions Were the Best?
| Source | R-squared | Harrell’s c | Somers’ Dxy | ICC | MAE | RMSE | MAPE | MASE |
|---|---|---|---|---|---|---|---|---|
| ESPN | .497 | .725 | .450 | .695 | 44.18 | 56.23 | 43.66 | .596 |
| CBS | .607 | .775 | .549 | .754 | 41.37 | 53.63 | 59.07 | .518 |
| NFL.com | .487 | .743 | .486 | .655 | 48.80 | 62.47 | 38.39 | .701 |
| FantasyPros | .667 | .775 | .549 | .816 | 32.66 | 45.36 | .434 | |
| Average | .657 | .776 | .551 | .810 | 33.62 | 46.18 | .447 | |
| Latent | .661 | .779 | .559 | .810 | 34.31 | 46.96 | 76.38 | .441 |
Note: MAPE was unable to be calculated for FantasyPros and the average because of values of zero in the series (for a discussion on this topic and for reasons to prefer MASE to the other error metrics, see here).
Here is how the projections ranked when focusing on R-squared and MASE:
- FantasyPros
- Latent
- Average
- CBS
- ESPN
- NFL.com
In general, projections from FantasyPros were more accurate than projections from ESPN, CBS, and NFL.com, and were also more accurate than our average and latent variables. FantasyPros projections explained about 67% of the variance in the actual points scored in my Yahoo league in the 2012 season. Interestingly, the average of the sources was more accurate than any of the individual sources. Even better than the average was a latent variable representing the common variance of the sources, which discards the unique, error variance.
Here is a scatterplot of the FantasyPros projections in relation to the actual points scored:
Conclusion
Hi,
How hard would it be to create a wisdom of the crowd projections utilizing other projections other than espn, yahoo, etc?
Great question. If you’re looking for projected player rankings based on wisdom of the crowd, you might try fantasyfootballcalculator.com, which publishes ADP data from thousands of mock drafts (http://fantasyfootballcalculator.com/adp.php). If you’re looking for projected points, I’m not familiar with any sites that calculate consensus projections across thousands of sources. The closest site I’ve come across is fantasypros (http://www.fantasypros.com/nfl/projections/qb.php). That being said, if you find any sites with a wisdom of the crowd approach to consensus projections, let me know!
-Isaac
The problem I run into when I try to rate projections is how far to go down in the list of players. The players at the top are the best and score the most and are the hardest to predict closely in absolute terms and yet they’re also the ones people are most likely to put into their lineup. If you predicted Peyton Manning would throw four TDs in Week 1 against the defending Super Bowl champsion Ravens then you would have done a fantastic job. And yet you also would have been off by three TDs.
OTOH when you go further and further down the list to players that are less and less important it becomes easier and easier for your predictions to become more and more accurate. You predict the backup RB to have 4 rushes for 15 yards and 1 reception for 7 yards and unless the starting RB gets hurt and the backup RB has to play a lot, your prediction is likely to be within couple points. It is hard to make a prediction within a couple points for a starter but it’s a lot easier to do it for a backup.
So if you include too many players in your assessment of projections then you’re including players that are easy to predict and who don’t matter much anyway. And as a result a method that doesn’t predict important players as well may seem good just because it can accurately improve the lesser important players. Or so it seems to me. What sayeth you?
I like the site but I’m a SAS user slowly trying to learn R via Robert Muenchen’s book. We’ll see how that goes.
Hey “Nick”,
It’s true that higher-tiered players are more difficult to predict than lower-tiered players, as supported by the wider confidence interval for players with more projected points. Part of this effect may reflect the greater difficulty in predicting the points a player will score on a game-by-game basis, as you point out. This difficulty in projecting points is attenuated when projecting points over an entire season, which allows some of the “random” variability to smooth out. It’s a good point, though, and one should not only focus on a player’s projected points, but also the expected points based on the player’s positional rank (see below):
https://fantasyfootballanalytics.net/2013/07/expected-points-by-position-rank-in-fantasy-football.html
Combining a player’s projected points, expected points based on positional rank, and risk, you have the data to make a more informed decision about who to draft.
Hope that helps!
-Isaac
Great site Isaac – I’m an engineer, so I really appreciate the data driven approach you’ve taken to combine the nerd aspect with the jock (albeit fantasy) aspect! Question I have for you is if there is a way to get a correlation of overall team performance versus fantasy player impact for that team? Obviously it makes sense that the better performing teams have the better scoring players, but I’m curious if there are any outliers or interesting trends that could be gleaned…
Is the projection app still being supported for the 2016 season? I’m trying to customize my projections for the 2016 league year based on league specific settings but keep getting a runtime error when I click on the link. Thanks
Apps should be up now. For more info on why they were down, see here:
https://fantasyfootballanalytics.net/app-down-heavy-traffic
given this analysis, why is FFP not weighted the most heavily in your draft optimizer?
https://fantasyfootballanalytics.net/2017/03/best-fantasy-football-projections-2017.html
Is there any update on this through the 2021 season?
Hello Andrew. Individual weeks from 2021 are available through the projections tab but altering the year and week settings. Good luck.
I meant is there any update on “Who Has the Best Fantasy Football Projections: ESPN, CBS, NFL.com, or FantasyPros?” for 2018-2021? The latest update I see is for 2017.