Page 16 of 22
Re: 2015-16 Team win projections
Posted: Wed Jan 27, 2016 6:40 pm
by bbstats
Any update?
Re: 2015-16 Team win projections
Posted: Wed Jan 27, 2016 7:28 pm
by Mike G
Relative to b-r.com simulations, avg errors are
Code: Select all
AJ 4.71 bbs 5.29 yoop 6.17
km 4.74 Crow 5.32 itca 6.30
DF 4.84 rsm 5.44 nr 6.38
KF 4.86 fpli 5.83 EZ 6.74
Cal 4.88 MG 5.88 DrP 6.80
tzu 5.04 snd 6.02 Dan 7.18
DSM 5.05 BD 6.15 taco 7.45
And RMSE:
Code: Select all
km 5.67 tzu 6.52 itca 7.43
AJ 5.71 rsm 6.60 BD 7.70
Cal 5.72 Crow 6.73 nr 7.83
DF 5.92 fpli 6.85 EZ 8.32
KF 6.09 snd 7.22 taco 8.48
bbs 6.17 MG 7.27 Dan 8.59
DSM 6.45 yoop 7.33 DrP 8.60
Re: 2015-16 Team win projections
Posted: Thu Jan 28, 2016 9:20 pm
by EvanZ
permaximum wrote:I agree. That's why I said although there's a hint, probably we didn't get worse in reality.
But I believe this confirms our ability to predict has not been improved meaningfully at all, if it's improved.
Can we at least say the RAPM-based methods are better than the non-RAPM based ones?
Re: 2015-16 Team win projections
Posted: Thu Jan 28, 2016 9:21 pm
by EvanZ
Moreover, if we all agreed on the minutes projections (which is how I think these projection contests should operate, because we're actually more interested in the player valuation models than the minutes projections), which method would win? And if I just use something "simple" like 2-year RAPM, how far behind the winner would I be?
One more thing. Do the PyWin projections take into account games already won? Seems like with the Warriors at 42-4 they should be on pace to win more than 66 games if we just look at the PyWin% for the rest of the season and add that to the accumulated wins from the first 46 games.
Re: 2015-16 Team win projections
Posted: Thu Jan 28, 2016 9:42 pm
by Mike G
EvanZ wrote:. Do the PyWin projections take into account games already won? Seems like with the Warriors at 42-4 they should be on pace to win more than 66 games if we just look at the PyWin% for the rest of the season and add that to the accumulated wins from the first 46 games.
Yes, at least b-r.com does. They do 7500 simulations of the remainder of the season, average the resulting wins and losses.
Code: Select all
tm W L Current Remain Best Worst
GSW 69.3 12.7 42-4 27-9 76-6 60-22
Their remaining 36 games ranged from 34-2 to 18-18 in this morning's iteration.
27.3-8.7 would be the avg
It's good that the Spurs are challenging for the #1 seed. The Dubs still have to try and win games. Otherwise, they might just coast in.
Meanwhile, I just retrodicted some stats from last year onto actual minutes played this year, and I get these average errors, wins projected to 82 games:
Code: Select all
ws/48 5.3
BPM 5.8
RPM 6.1
eW/48 6.5
PER 6.9
PER may be off, as it seems heavily regressed -- predicting 30 wins for the Sixers, 50 for the Spurs.
It's most strange. Here are correlations between these stats (from last year) and this year's mpg for 381 players who played both years:
Code: Select all
stat 2015 2016
BPM .61 .54
e484 .54 .49
RPM .51 .49
PER .49 .44
WS/48 .34 .32
I used actual (2016) rookie rates to get team rates/minutes; including Julius Randle (rather than his terrible 15 minutes last year). Also Paul George 2014 rather than 2015.
Re: 2015-16 Team win projections
Posted: Fri Jan 29, 2016 5:51 pm
by AJbaskets
EvanZ wrote:Moreover, if we all agreed on the minutes projections (which is how I think these projection contests should operate, because we're actually more interested in the player valuation models than the minutes projections), which method would win? And if I just use something "simple" like 2-year RAPM, how far behind the winner would I be?
One more thing. Do the PyWin projections take into account games already won? Seems like with the Warriors at 42-4 they should be on pace to win more than 66 games if we just look at the PyWin% for the rest of the season and add that to the accumulated wins from the first 46 games.
I am not sure that actually solves the problem, for example, if PT-PM thought Durant was not going to be good last year and used the consensus minutes projections I would have beaten the field on OKC's wins estimates even though that player estimate was wrong.
I *think* combining a box score estimate and RAPM, adding an age curve, mean regression and team change regression all add incrementally, not sure exactly how much.
Re: 2015-16 Team win projections
Posted: Fri Jan 29, 2016 8:54 pm
by EvanZ
AJbaskets wrote:
I am not sure that actually solves the problem, for example, if PT-PM thought Durant was not going to be good last year and used the consensus minutes projections I would have beaten the field on OKC's wins estimates even though that player estimate was wrong.
True, then I would suggest each participant simply give a list of player ratings (per 100 possessions) and wins could be calculated based on actual possessions played during the season. Wouldn't that work? I mean, aside from the fact that it's not a "Win Projection" contest anymore.
Re: 2015-16 Team win projections
Posted: Fri Jan 29, 2016 10:41 pm
by Mike G
2015 rates applied to 2016 minutes.
Projected is what b-r.com said yesterday. Errors on the right.
Scaled to 41 wins avg. No aging curve.
Code: Select all
eW WS BPM RPM Proj err: eW WS BPM RPM
55 52 52 55 46 Atl 9 6 6 10
40 43 47 49 47 Bos 7 4 0 2
30 30 26 21 25 Brk 5 6 2 4
26 36 39 35 42 Cha 15 6 2 6
46 47 41 45 45 Chi 1 2 4 0
48 48 51 63 56 Cle 8 7 5 8
41 45 44 48 43 Dal 2 2 2 5
17 21 26 19 32 Den 16 12 6 13
39 40 33 39 45 Det 6 5 12 6
58 61 67 69 69 GSW 12 9 2 0
57 51 49 55 40 Hou 17 11 8 15
53 46 49 48 45 Ind 8 1 4 4
57 51 56 50 51 LAC 6 0 5 1
36 25 20 22 19 LAL 18 6 2 4
43 48 57 55 43 Mem 1 5 14 12
eW WS BPM RPM Proj err: eW WS BPM RPM
53 40 34 33 43 Mia 10 2 9 9
36 33 31 32 35 Mil 2 2 4 3
35 28 28 21 28 Min 7 0 0 6
44 45 47 45 34 NOP 11 11 14 11
26 27 23 23 38 NYK 12 12 15 15
63 55 63 57 56 Okl 7 1 8 1
25 27 22 21 37 Orl 11 9 15 16
14 24 13 10 17 Phl 3 7 3 6
29 38 36 30 27 Phx 2 10 9 2
28 44 44 41 37 Por 9 7 7 4
49 39 46 42 38 Sac 11 1 8 4
67 56 62 67 66 SAS 1 10 4 1
48 50 47 47 53 Tor 4 2 5 6
38 40 39 39 39 Uta 1 1 0 0
29 40 37 47 38 Was 9 2 1 9
Re: 2015-16 Team win projections
Posted: Sat Jan 30, 2016 11:13 am
by J.E.
The way Joe Sill used to run this, wich is also my preferred way, is ignore possessions that have rookies in them, since we have no estimates for them and some of them (obviously) perform better than others
Also, running things with an aging curve would produce more accurate results - although the question becomes whether we use different aging curves for the different metrics. I'm not even sure whether they exist for every metric
Further, some of these projections need to be regressed to the mean more than others (e.g. due to 'effect of leading' being present in RPM, etc)
Re: 2015-16 Team win projections
Posted: Sat Jan 30, 2016 1:03 pm
by Mike G
I wonder how the aging curve varies along a spectrum from "young player on older team" to "team of young players".
Regressing to the mean, even knowing the minutes used? I did try some partial regressions, without much improvement.
Including current rookie minutes and rates, a similar effect is laid on all metrics. That shouldn't affect how the metrics predict, should it?
Re: 2015-16 Team win projections
Posted: Sat Jan 30, 2016 1:23 pm
by J.E.
Mike G wrote:Including current rookie minutes and rates, a similar effect is laid on all metrics. That shouldn't affect how the metrics predict, should it?
If you're trying to do a true out of sample test, you simply can't use rookie rates from this season
Regressing to the mean, even knowing the minutes used?
RPM expects players to perform worse when up big, and vice versa. E.g. a team that performs as a +15 when tied will perform significantly worse when actually up 15. Any team win projections will have to account for that, or your "RPM projections" will turn out to be wider than they would have been if you simulated possession-by-possession, adjusting for current lead.
Aside from that, I'm sure all metrics benefit from "regressing to the mean" when projecting an entire season, and it might vary (slightly) from metric to metric how much regression to the mean is optimal
Re: 2015-16 Team win projections
Posted: Wed Feb 03, 2016 3:45 am
by permaximum
RPM expects many things which make it automatically worse for outliers such as the best players and the worst players yet most here use it for those "outliers". Ironic.
Re: 2015-16 Team win projections
Posted: Sat Feb 06, 2016 10:50 am
by Mike G
Squared and not, there are clear 1st and 2nd divisions.
Code: Select all
avg error avg error sqd error sqd error
KF 4.72 BD 6.08 Cal 5.81 yoop 7.45
km 4.78 MG 6.10 AJ 5.82 MG 7.48
DF 4.85 snd 6.10 km 5.90 snd 7.49
AJ 4.86 itca 6.30 DF 6.05 itca 7.65
Cal 5.04 DrP 6.53 bbs 6.34 BD 7.82
tzu 5.05 nr 6.54 KF 6.38 nr 7.92
DSM 5.12 yoop 6.60 DSM 6.59 EZ 8.39
bbs 5.16 EZ 6.86 tzu 6.61 DrP 8.54
rsm 5.56 Dan 7.21 rsm 6.63 Dan 8.55
Crow 5.58 taco 7.34 fpli 6.96 taco 8.58
fpli 5.84 15py 8.82 Crow 6.98 15py 9.86
Re: 2015-16 Team win projections
Posted: Tue Feb 09, 2016 11:25 am
by Mike G
Setting the best guess (for each team) to zero, everyone's current (absolute) difference from that best guess:
Code: Select all
tm KF km tzu AJ DF bbs DSM Cal rsm Crow Avg fpli snd MG BD itca DrP nr yoop Dan EZ tac
Atl 0 2 0 5 3 2 5 6 5 1 4 2 7 5 4 5 9 3 5 3 8 6
Bos 0 1 0 0 0 3 1 2 1 2 3 0 3 10 12 3 7 2 5 7 3 5
Brk 0 4 3 1 0 3 0 2 3 0 3 5 3 5 5 4 4 4 4 7 7 3
Cha 4 3 4 3 3 3 4 5 9 2 6 6 7 6 6 16 6 15 0 3 7 14
Chi 0 0 3 4 3 2 3 1 6 2 4 5 4 0 3 8 3 5 6 5 7 8
Cle 0 0 1 0 2 0 1 1 2 0 2 0 3 6 1 0 0 1 6 1 3 4
Dal 1 4 1 1 2 2 0 3 2 4 3 2 0 3 0 4 2 2 3 9 0 9
Den 8 9 7 6 7 7 6 7 8 5 6 5 8 6 2 8 4 3 6 11 0 1
Det 4 3 1 4 5 3 5 2 7 7 6 3 5 0 12 3 3 12 12 9 15 8
GSW 7 5 7 0 3 2 5 4 1 5 5 5 7 13 5 7 0 6 6 13 5 7
Hou 6 5 9 4 5 7 6 3 6 8 6 6 7 6 8 7 0 9 4 13 9 6
Ind 5 2 5 3 5 7 8 4 3 3 4 0 5 2 3 1 7 9 8 0 2 15
LAC 2 1 1 1 2 1 1 4 5 2 2 2 2 5 4 1 0 7 0 6 1 2
LAL 4 6 10 3 3 1 2 5 0 3 5 5 2 14 7 5 0 6 6 2 9 8
Mem 1 5 1 6 6 4 3 6 8 5 5 7 1 3 1 3 6 8 11 0 5 7
tm KF km tzu AJ DF bbs DSM Cal rsm Crow Avg fpli snd MG BD itca DrP nr yoop Dan EZ tac
Mia 5 4 0 3 4 9 4 4 4 3 5 0 5 8 4 0 10 6 9 11 3 3
Mil 6 5 3 2 2 4 3 4 5 7 5 5 7 7 14 9 0 0 2 5 13 11
Min 1 1 3 2 0 1 3 3 4 1 2 5 4 1 2 2 2 7 7 2 5 0
NOP 5 1 3 5 6 3 7 2 5 6 4 9 6 2 4 4 8 6 4 0 7 1
NYK 8 5 7 11 10 8 12 7 12 8 9 8 13 10 14 4 22 5 6 12 12 0
Okl 1 0 1 2 0 2 0 0 0 3 2 3 3 0 0 2 1 6 1 12 3 0
Orl 2 2 1 10 6 4 3 8 6 9 6 9 3 4 7 6 11 8 5 0 10 9
Phl 9 4 11 8 6 7 8 4 1 6 6 2 9 2 1 9 17 9 10 2 0 12
Phx 9 8 0 5 8 9 7 8 9 10 7 9 11 7 3 8 13 10 1 7 5 8
Por 1 0 3 6 5 2 2 5 5 5 4 7 2 3 4 9 3 1 8 3 2 12
Sac 1 5 2 0 2 0 4 3 2 3 3 4 1 8 3 1 1 6 3 7 6 1
SAS 6 5 10 2 3 5 5 4 0 7 5 4 7 4 7 5 5 2 8 5 6 7
Tor 6 9 7 6 8 6 5 9 6 8 7 9 9 9 10 12 0 2 12 5 10 8
Uta 0 2 1 4 0 0 0 1 2 1 2 5 0 2 0 2 10 1 3 3 1 4
Was 3 6 3 4 3 8 8 4 6 8 5 7 5 0 6 7 4 3 5 6 8 8
avg 3.5 3.5 3.5 3.7 3.7 3.8 4.0 4.0 4.4 4.4 4.6 4.6 4.9 5.0 5.0 5.1 5.2 5.4 5.5 5.6 5.7 6.2
Everyone has at least one
best guess (or within 0.5) right now.
Our
average prediction is at least 2 wins off for every team.
Tacoman had the Knicks at 10 wins better than the rest of us, on avg. They're doing just that well -- and they fire their coach?
Re: 2015-16 Team win projections
Posted: Sun Feb 14, 2016 11:34 am
by Mike G
At All Star weekend:
Code: Select all
km 4.86 snd 6.22
KF 4.89 MG 6.29
tzu 4.96 BD 6.30
AJ 5.05 itca 6.52
bbs 5.13 DrP 6.66
DF 5.13 yoop 6.81
DSM 5.37 nr 6.85
Cal 5.45 Dan 6.91
rsm 5.78 EZ 7.10
Crow 5.79 taco 7.67
fpli 6.15 15py 8.99
Teams have not cooperated of late, and almost all of us are looking worse in the last week.
Exceptions: tarrazu (improved from 5.05 to 4.96), bbstats (5.16 to 5.13), StatmanDan (7.21 to 6.91)
Others had their errors rise as much as .040