Page 10 of 15
Re: 2016-17 Team Win Projection Contest / Discussion
Posted: Sat Jan 21, 2017 11:43 am
by Mike G
Since b-r.com's page is blank today, here we are relative to the
538 forecast
Code: Select all
1-21 RMSE avg abs err SMRE
trzu 4.44 BaDo 3.55 BaDo 2.50
lnqi 4.57 RyRi 3.63 RyRi 2.91
RyRi 4.65 lnqi 3.66 amp5 2.96
amp5 4.70 amp5 3.67 kmed 3.09
ATCt 4.83 trzu 3.67 trzu 3.14
sbs. 4.84 sbs. 3.85 sbs. 3.16
BaDo 4.90 kmed 3.90 lnqi 3.16
vegas 5.01 cali 3.98 Crow 3.23
Crow 5.04 vegas 4.01 cali 3.25
Mike 5.13 ATCt 4.01 shad 3.34
kmed 5.18 Crow 4.03 vegas 3.41
cali 5.19 Mike 4.07 yoop 3.42
ncsD 5.27 yoop 4.21 Mike 3.48
taco 5.30 shad 4.26 ATCt 3.51
ncsB 5.35 nrfo 4.33 jg34 3.52
nrfo 5.39 jg34 4.36 538 3.56
sndi 5.39 sndi 4.41 nrfo 3.68
shad 5.43 ncsD 4.41 ncsD 3.72
jg34 5.59 ncsB 4.46 sndi 3.74
yoop 5.65 taco 4.46 ncsB 3.77
GK5. 5.75 538 4.59 taco 3.81
538 5.91 GK5. 4.79 GK5. 4.15
EDIT: Andrew Johnson's numbers: 4.99, 4.24, 3.78
Back to b -r.com
Code: Select all
1-22 RMSE avg abs err SMRE
lnqi 4.31 lnqi 3.27 BaDo 2.35
RyRi 4.34 BaDo 3.31 Mike 2.74
amp5 4.38 amp5 3.37 lnqi 2.75
trzu 4.40 Mike 3.39 amp5 2.79
sbs. 4.55 RyRi 3.47 sbs. 2.97
ATCt 4.66 sbs. 3.57 RyRi 3.00
BaDo 4.68 trzu 3.58 shad 3.06
Mike 4.71 kmed 3.74 trzu 3.09
AnJo 4.73 vegas 3.77 kmed 3.11
vegas 4.80 ATCt 3.77 nrfo 3.17
Code: Select all
1-23 RMSE avg abs err SMRE
lnqi 4.18- lnqi 3.17 BaDo 2.32
RyRi 4.21- BaDo 3.20 lnqi 2.70
trzu 4.27- amp5 3.29 amp5 2.74
amp5 4.27- RyRi 3.34 Mike 2.74
sbs. 4.41 Mike 3.36 RyRi 2.83
BaDo 4.51 trzu 3.45 sbs. 2.89
ATCt 4.54 sbs. 3.45 trzu 2.96
Mike 4.61 kmed 3.62 kmed 2.96
AnJo 4.64 vegas 3.65 shad 3.05
Crow 4.65 ATCt 3.65 Crow 3.06
The 4 best RMSE today are better than any previously.
Re: 2016-17 Team Win Projection Contest / Discussion
Posted: Mon Jan 23, 2017 8:54 pm
by tarrazu
BR and 538 projections look pretty similar at this point but with BR more condensed/smaller range.
Re: 2016-17 Team Win Projection Contest / Discussion
Posted: Tue Jan 24, 2017 12:11 pm
by Mike G
For a few days, BasketDork had an astonishing lead in the 3rd column, SMRE -- some .40 better than anyone else.
Yesterday I found he had 7 guesses within
0.3 of the b-r.com projections. Nobody else had more than 3 such direct hits.
Something like half of projections fell within 1.5 of his predictions. I wonder if this is largely luck, even as 538 gave him an equally commanding lead.
The trouble with having such perfect guesses is that either a win or a loss will move a team off that perfect spot.
Code: Select all
1/24 RMSE avg abs err SMRE
lnqi 4.11- lnqi 3.10- lnqi 2.55
RyRi 4.17- BaDo 3.27 BaDo 2.67
trzu 4.24 Mike 3.32 Mike 2.68
amp5 4.25 amp5 3.33 amp5 2.80
sbs. 4.40 RyRi 3.34 sbs. 2.80
BaDo 4.50 trzu 3.41 RyRi 2.85
Mike 4.57 sbs. 3.44 taco 2.92
ATCt 4.57 Crow 3.58 trzu 2.93
vegas 4.59 kmed 3.60 kmed 2.93
Crow 4.60 vegas 3.61 shad 3.01
sndi 4.64 taco 3.65 Crow 3.02
AnJo 4.64 ATCt 3.70 nrfo 3.05
ncsD 4.68 shad 3.72 cali 3.06
kmed 4.68 sndi 3.77 ATCt 3.12
taco 4.72 cali 3.79 vegas 3.14
cali 4.82 nrfo 3.85 jg34 3.22
shad 4.85 jg34 3.89 sndi 3.29
ncsB 4.94 AnJo 3.99 538 3.37
jg34 4.95 yoop 4.07 GK5. 3.42
nrfo 5.02 ncsD 4.07 yoop 3.51
yoop 5.12 538 4.12 ncsB 3.59
GK5. 5.29 ncsB 4.15 AnJo 3.61
538 5.46 GK5. 4.18 ncsD 3.66
Some more season bests marked by
-
Code: Select all
1/25 RMSE avg abs err SMRE
lnqi 4.21 lnqi 3.24 Mike 2.63
RyRi 4.27 Mike 3.31 BaDo 2.73
amp5 4.32 BaDo 3.37 lnqi 2.76
trzu 4.39 amp5 3.43 amp5 2.92
sbs. 4.50 RyRi 3.44 RyRi 2.95
Mike 4.56 trzu 3.50 trzu 2.96
BaDo 4.62 sbs. 3.57 Crow 3.01
ATCt 4.68 Crow 3.63 sbs. 3.01
sndi 4.69 kmed 3.72 kmed 3.09
vegas 4.69 vegas 3.75 shad 3.11
Re: 2016-17 Team Win Projection Contest / Discussion
Posted: Wed Jan 25, 2017 1:54 pm
by nbacouchside
well, this is the worst i've done in a while.
Re: 2016-17 Team Win Projection Contest / Discussion
Posted: Wed Jan 25, 2017 3:41 pm
by Mike G
You aren't doing worse; everyone else is doing better.
On 1/19, your D version RMSE made the top 10 at 4.83. Now at 4.78, good for 14th.
On 1/17, the B side RMSE was 8th from the bottom at 5.18; now 5.08 is 5th worst.
I keep thinking we can't all keep looking better, but it keeps going there. Phenomenal.
Re: 2016-17 Team Win Projection Contest / Discussion
Posted: Wed Jan 25, 2017 7:04 pm
by sndesai1
wow 538 has really fallen off a cliff moving to their new system this year
Re: 2016-17 Team Win Projection Contest / Discussion
Posted: Thu Jan 26, 2017 10:46 am
by Mike G
A new season best in RMSE
Code: Select all
1/26 RMSE avg abs err SMRE
lnqi 4.09- BaDo 3.207 BaDo 2.49
RyRi 4.18 lnqi 3.210 Mike 2.64
trzu 4.23 Mike 3.33 lnqi 2.76
amp5 4.23 RyRi 3.34 RyRi 2.77
sbs. 4.41 trzu 3.39 Crow 2.85
BaDo 4.47 amp5 3.41 trzu 2.89
Mike 4.49 sbs. 3.49 sbs. 2.89
ATCt 4.54 Crow 3.49 amp5 2.89
Crow 4.59 kmed 3.66 shad 3.01
sndi 4.60 vegas 3.72 kmed 3.04
Abrupt new lows:
Code: Select all
1/28 RMSE avg abs err SMRE
lnqi 3.94- lnqi 3.01- lnqi 2.50
RyRi 4.04- RyRi 3.20 BaDo 2.63
trzu 4.04- BaDo 3.24 Mike 2.69
amp5 4.10 amp5 3.24 shad 2.70
sbs. 4.29 trzu 3.26 RyRi 2.71
BaDo 4.37 Mike 3.37 amp5 2.73
sndi 4.45 sbs. 3.40 kmed 2.79
ATCt 4.45 Crow 3.43 trzu 2.79
AnJo 4.46 vegas 3.48 Crow 2.84
vegas 4.47 kmed 3.51 sbs. 2.93
More new lows, and an avg of our 20 guesses leads all fields.
Code: Select all
1/30 RMSE avg abs err SMRE
avg 3.73 avg 2.89 avg 2.34
lnqi 3.78- lnqi 2.96- BaDo 2.42
trzu 3.79- trzu 3.01 RyRi 2.47
RyRi 3.86- BaDo 3.03 trzu 2.54
amp5 3.95 RyRi 3.03 lnqi 2.54
sbs. 4.06 sbs. 3.13 sbs. 2.55
BaDo 4.13 amp5 3.14 yoop 2.65
vegas 4.20 Crow 3.21 Crow 2.66
ATCt 4.25 vegas 3.28 amp5 2.66
Crow 4.25 Mike 3.31 Mike 2.72
sndi 4.28 kmed 3.32 kmed 2.72
AnJo 4.29 sndi 3.42 vegas 2.79
kmed 4.32 yoop 3.45 sndi 2.82
ncsD 4.36 shad 3.50 shad 2.85
Mike 4.36 cali 3.52 cali 3.03
cali 4.37 ncsD 3.65 ncsD 3.10
shad 4.49 jg34 3.67 taco 3.10
taco 4.56 ATCt 3.70 jg34 3.14
yoop 4.59 taco 3.70 538 3.28
jg34 4.64 AnJo 3.78 nrfo 3.29
nrfo 4.78 nrfo 3.87 ATCt 3.39
ncsB 4.78 GK5. 3.98 GK5. 3.39
GK5. 4.95 ncsB 4.07 AnJo 3.48
538 5.31 538 4.09 ncsB 3.68
RMSE are especially shrinking because outlying successes Hou and Mil have been losing.
Code: Select all
1/31 RMSE avg abs err SMRE
trzu 3.83 trzu 3.04 RyRi 2.46
lnqi 3.84 lnqi 3.05 BaDo 2.55
RyRi 3.94 RyRi 3.07 trzu 2.58
amp5 3.96 BaDo 3.10 sbs. 2.61
sbs. 4.14 amp5 3.14 amp5 2.62
BaDo 4.16 sbs. 3.20 Crow 2.65
vegas 4.22 Crow 3.24 kmed 2.66
ATCt 4.31 vegas 3.28 lnqi 2.66
sndi 4.31 kmed 3.31 yoop 2.78
AnJo 4.32 Mike 3.38 vegas 2.78
Re: 2016-17 Team Win Projection Contest / Discussion
Posted: Tue Jan 31, 2017 1:43 pm
by GK5
Glad to see I have moved from the basement in all three categories
Thanks for keeping this updated Mike!
Re: 2016-17 Team Win Projection Contest / Discussion
Posted: Tue Jan 31, 2017 4:59 pm
by Mike G
The worst RMSE today was about the best just 3 weeks ago.
Code: Select all
date - err + - RMSE +
11/17 5.19 6.91 6.42 8.47
11/22 5.08 6.28
11/29 3.99 4.98
12/03 4.55 5.53 5.54 6.83
12/13 3.95 5.16 4.94 6.17
12/21 3.58 4.81 4.59 5.73
12/27 3.45 4.74 4.49 5.60
1/04 3.20 4.66
1/09 3.61 5.18 4.94 6.24
1/13 3.33 4.84 4.59 5.91
1/17 3.31 4.70 4.51 5.73
1/24 3.10 4.18 4.11 5.29
1/31 2.96 4.07 3.78 4.95
Re: 2016-17 Team Win Projection Contest / Discussion
Posted: Tue Jan 31, 2017 6:10 pm
by Crow
sndesai1 wrote:wow 538 has really fallen off a cliff moving to their new system this year
They moved from RPM / BPM blend to pure BPM right? Wrong move.
Re: 2016-17 Team Win Projection Contest / Discussion
Posted: Tue Jan 31, 2017 8:07 pm
by tarrazu
Crow wrote:sndesai1 wrote:wow 538 has really fallen off a cliff moving to their new system this year
They moved from RPM / BPM blend to pure BPM right? Wrong move.
Yeah all BPM and a player similarity model.
https://fivethirtyeight.com/features/wh ... r-2016-17/
"CARMELO projections are generated through a two-step process. First, the system produces a baseline projection for each player based on regression analysis. Then, it adjusts the forecast and generates a distribution of possible outcomes based on the comparable players. Although the second part of this process is almost exactly the same as last year, we’ve put more work into the first step, the baseline projection. The system is now a bit smarter about handling players with limited playing time. It also recognizes that different statistics have different amounts of predictive value. For instance, because shooting can be streaky, players who generated strong performances on the basis of good shooting seasons are more apt to regress to the mean than others. By contrast, generating shots, drawing fouls and taking 3-pointers are correlated with improved performance in future seasons. Rebounding, blocking shots and, especially, accumulating steals are also correlated with stronger future performance."
Re: 2016-17 Team Win Projection Contest / Discussion
Posted: Tue Jan 31, 2017 9:31 pm
by sndesai1
i think we all know bpm will perform worse than rpm here, but i believe mike g's projection using last season's bpm and kevin pelton's mins is doing better than 538 without any fancy similarity model to project performance.
i guess it could be that the minute projections were way off...
Re: 2016-17 Team Win Projection Contest / Discussion
Posted: Tue Jan 31, 2017 10:23 pm
by AcrossTheCourt
No, they often share samples of the minutes projections, and they're usually really solid.
I think they just overfit the similarity/age curve.
Look at MIN's projection again, which did not factor in Thibs -- they were over the optimistic Vegas line by a good distance. They assumed some incredible development.
Re: 2016-17 Team Win Projection Contest / Discussion
Posted: Wed Feb 01, 2017 11:23 am
by Mike G
sndesai1 wrote:i think we all know bpm will perform worse than rpm here, but i believe mike g's projection using last season's bpm and kevin pelton's mins is doing better than 538 without any fancy similarity model to project performance.
i guess it could be that the minute projections were way off...
I didn't use BPM at all. It was a home cooked version I called
fauxRPM averaged with
eWins.
Both are
boxscore stats, to be sure.
Besides player minutes, the big variables seem to be rookie projections and regression to the mean. The optimum extent of regression may vary by metric or by season; seems like plenty is always best.
538 is just having a bad year all around.
Meanwhile, new bests in RMSE
Code: Select all
2/01 RMSE avg abs err SMRE
trzu - 3.84 trzu 3.09 BaDo 2.58
lnqi - 3.91 lnqi 3.12 RyRi 2.62
RyRi 3.99 BaDo 3.15 sbs. 2.63
amp5 4.03 RyRi 3.16 trzu 2.63
sbs. 4.18 amp5 3.20 Crow 2.65
BaDo 4.21 sbs. 3.25 amp5 2.66
vegas 4.26 Crow 3.25 kmed 2.69
sndi 4.33 vegas 3.29 lnqi 2.75
ATCt 4.33 kmed 3.35 vegas 2.76
AnJo 4.35 Mike 3.45 yoop 2.88
Crow 4.37 sndi 3.48 sndi 2.93
Re: 2016-17 Team Win Projection Contest / Discussion
Posted: Wed Feb 01, 2017 5:04 pm
by sndesai1
sorry, that's my fault on the wording. i didn't mean your contest entry, but rather the numbers posted here under BPM:
http://www.apbr.org/metrics/viewtopic.p ... =15#p28073