A number of comments yesterday wondered how last year’s prescribed Big Board projections mix performed vs. the rest of the projection systems, so here we go with a quick followup! Let me state right at the top – my projections mix evolves every year, and I recommend you use the newly calculated mix from yesterday’s post, NOT last year’s version. Projections have gone through a tumultuous few years with the introduction of juiced balls and statcast metrics, so our understanding is constantly evolving.
As a reminder, last year’s mix was:
-
Hitter Playing Time: 45% ZiPS, 30% ATC, 25% FGDepth
-
Hitter Rate Stats: 45% Steamer, 30% ATC, 25% ZiPS
-
Pitcher Playing Time: 60% Steamer, 40% Fans
-
Pitcher Rate Stats: 50% Steamer, 30% ZiPS, 20% PECOTA
See yesterday’s post for what all these numbers mean, but the R squared and RMSE values for each category were as follows:
Hitters –
So how does that rank?
On the hitter side, the correlations are better than any individual system did in 2018 – next best overall was Steamer, at 8% above average, and next best on a rate basis was ATC, at 6% above average. The RMSE is also better than all other systems, with the next best being a tie between several systems at 3%.
On the pitcher side, the correlation for playing time was somewhat poor, but it still manages to be second only to ATC overall, at 14% above average. I cannot overstate how much ATC just blew everyone out of the water on playing time projections last year. On a rate basis, my mix again beat all systems, with ATC next best at 6% above average. The RMSE is again better than all other systems, with the next best being a tie between several systems at 2%.
" I cannot overstate how much ATC just blew everyone out of the water on playing time projections last year."
But only in hitting, right?
No, only in pitching (that whole paragraph is about pitching). On the hitting side, ATC’s PA projections were just okay.
My bad. I meant to say pitching.
It’s kind of interesting bc Cohen uses weighted averages of third party projections himself in generating ATC (https://fantasy.fangraphs.com/the-atc-projection-system/). In theory, his projections should not add any unique value that could not be obtained via optimizing weightings of other projections. It’s possible, though, that he does PT manually, which could be valuable.
"my projections mix evolves every year, and I recommend you use the newly calculated mix from yesterday’s post, NOT last year’s version."
Yes, but are you determining that mix at the beginning of the season and seeing how it fares, or coming up with the ideal mix after the fact? That wasn’t clear from either the original post or this follow-up.
Hmm it’s strange because I’ve written this piece for several years now and never gotten this question, and especially with this followup I think it’s pretty clear that the mix is determined after the season. What this post shows is that the 2018 mix, which I came up with after the 2017 season, performed very well, better than most systems.
I can’t speak to what you may or may not have specified in prior years, as I just discovered your site a few days ago. Perhaps nobody else asked because they assumed one way or another. Explicit is always better than implicit, so thanks for clarifying.