2016-17 Team Win Projection Contest / Discussion

[Mike G]

The b-r.com page isn't updating, so here again we are relative to the 538 forecast

Code: Select all

2/02   RMSE      avg abs err             SMRE
trzu   4.03      Crow   3.30      Crow   2.57
BaDo   4.23      trzu   3.33      trzu   2.81
lnqi   4.38      BaDo   3.47      yoop   2.84
Crow   4.40      vegas  3.50      RyRi   2.87
RyRi   4.45      lnqi   3.52      BaDo   2.89
vegas  4.52      RyRi   3.53      vegas  2.91
amp5   4.55      sbs.   3.57      amp5   2.91
sbs.   4.56      amp5   3.60      sbs.   2.93
ATCt   4.66      kmed   3.62      kmed   2.96
cali   4.68      cali   3.67      lnqi   3.02
kmed   4.73      yoop   3.73      cali   3.09
AnJo   4.79      ncsD   3.87      ncsD   3.12
shad   4.86      Mike   3.87      shad   3.26
ncsD   4.95      shad   3.93      Mike   3.27
sndi   4.96      sndi   3.93      sndi   3.27
Mike   4.98      AnJo   3.97      AnJo   3.30
yoop   5.10      ATCt   4.02      ncsB   3.30
nrfo   5.11      jg34   4.06      jg34   3.36
ncsB   5.22      ncsB   4.13      ATCt   3.55
taco   5.29      nrfo   4.27      taco   3.63
jg34   5.31      taco   4.33      nrfo   3.78
GK5.   5.45      GK5.   4.47      538    3.92
538    5.91      538    4.67      GK5.   3.93

https://projects.fivethirtyeight.com/20 ... edictions/
Most of their projections are very close to what b-r.com gets, and a few are pretty far off.

[Mike G]

sorry, that's my fault on the wording. i didn't mean your contest entry, but rather the numbers posted here under BPM: http://www.apbr.org/metrics/viewtopic.p ... =15#p28073

Thanks; I forgot I'd done that, but now I remember it would be fun to retroactively see what might be the optimal degree of regression to the mean for each of these.
It seems not much regression is needed to reach best fit with current forecast by b-r.com; they are:

Code: Select all

fauxRPM   .06
WinShares .11
RPM       .14
BPM       .20
eWins     .30

So, BPM reaches the best avg error by regressing like:
.80*(BPM-projection) + .20*41

It's cheating, obviously, to insert an entry after the fact; but even with optimal regressions to the mean, these stats alone don't compete for the lead:

Code: Select all

trzu   3.00      sndi   3.49      BPM+   3.74
lnqi   3.11      fRPM+  3.49      ATCt   3.76
BaDo   3.22      cali   3.50      AnJo   3.78
Crow   3.22      yoop   3.58      taco   3.92
RyRi   3.23      ncsD   3.63      ncsB   3.96
vegas  3.24      Mike   3.63      eWin+  4.11
sbs.   3.27      RPM+   3.63      nrfo   4.13
amp5   3.30      shad   3.67      WS+    4.14
kmed   3.38      jg34   3.68      GK5.   4.32
.                                  538   4.46

The + sign signifies the added unfair advantage.
A blend of 60% RPM and 40% BPM manages to get 3.57 avg abs err.
These are relative to b-r.com projections today.

[Mike G]

New lows across the board, all by the latest greatest.

Code: Select all

2/03   RMSE      avg abs err             SMRE
trzu - 3.68      trzu - 2.87      trzu - 2.28
lnqi - 3.87      lnqi   3.03      Crow   2.42
RyRi   3.95      RyRi   3.12      lnqi   2.48
amp5   4.03      Crow   3.12      sbs.   2.53
BaDo   4.07      BaDo   3.15      RyRi   2.54
vegas  4.12      vegas  3.16      vegas  2.59
sbs.   4.14      sbs.   3.20      BaDo   2.63
Crow   4.22      amp5   3.20      kmed   2.64
AnJo   4.25      kmed   3.31      amp5   2.67
ATCt   4.27      cali   3.34      yoop   2.69
cali   4.29      sndi   3.41      cali   2.71
kmed   4.39      yoop   3.41      sndi   2.82
sndi   4.39      ncsD   3.52      jg34   2.89
ncsD   4.42      shad   3.58      ncsD   2.90
shad   4.51      Mike   3.59      shad   2.96
Mike   4.52      jg34   3.61      Mike   3.09
yoop   4.63      ATCt   3.66      ATCt   3.27
taco   4.73      AnJo   3.68      taco   3.28
jg34   4.75      taco   3.84      AnJo   3.35
nrfo   4.83      ncsB   3.94      ncsB   3.38
ncsB   4.84      nrfo   4.10      538    3.57
GK5.   5.13      GK5.   4.24      nrfo   3.69
538    5.60      538    4.41      GK5.   3.70

Tarrazu isn't the only meteoric rise thru the ranks in 4 weeks:

Code: Select all

impr   err   1/07   2/03
1.23  yoop   4.64   3.41
1.09  trzu   3.96   2.87
1.09  Crow   4.21   3.12
1.07  cali   4.41   3.34
.84   vegas  4.00   3.16
.77   sndi   4.18   3.41
.76   GK5.   5.00   4.24
.74   ncsD   4.26   3.52
.59   kmed   3.90   3.31
.56   jg34   4.17   3.61
.56   BaDo   3.71   3.15
.56   AnJo   4.24*  3.68
.55   lnqi   3.58   3.03
.54   sbs.   3.74   3.20
.54   RyRi   3.66   3.12
.52   amp5   3.72   3.20
.45   538    4.86   4.41
.39   taco   4.23   3.84
.38   shad   3.96   3.58
.38   ATCt   4.04   3.66
.25   ncsB   4.19   3.94
.14   Mike   3.73   3.59
-.39  nrfo   3.71   4.10

AJ early date is Jan. 21, so a 2-week history.
These of course do not represent constant trends. Some of us have been up and down; and I don't believe there's any such thing as 'momentum' here.

If you don't improve by .02 per day, or .14 per week, you fall behind.

[Mike G]

Code: Select all

2/07   RMSE      avg abs err             SMRE
trzu   4.00      BaDo   3.24      BaDo   2.63
lnqi   4.18      trzu   3.26      yoop   2.75
BaDo   4.23      lnqi   3.35      Crow   2.76
RyRi   4.26      vegas  3.40      trzu   2.78
vegas  4.37      Crow   3.42      kmed   2.79
amp5   4.38      kmed   3.49      cali   2.81
sbs.   4.42      cali   3.50      lnqi   2.87
Crow   4.47      yoop   3.56      jg34   2.87
cali   4.52      RyRi   3.58      vegas  2.88
AnJo   4.55      sbs.   3.58      Mike   2.93
ATCt   4.59      amp5   3.60      sndi   3.05
shad   4.64      Mike   3.62      sbs.   3.10
kmed   4.69      jg34   3.74      amp5   3.11
ncsD   4.73      sndi   3.75      RyRi   3.22
Mike   4.80      ncsD   3.82      ncsD   3.27
sndi   4.84      AnJo   3.82      AnJo   3.31
yoop   4.85      shad   3.84      shad   3.41
taco   5.03      ATCt   3.93      ncsB   3.46
ncsB   5.04      ncsB   4.10      ATCt   3.52
jg34   5.05      taco   4.20      taco   3.69
nrfo   5.08      nrfo   4.35      538    3.79
GK5.   5.39      GK5.   4.42      GK5.   3.82
538    5.88      538    4.60      nrfo   3.85

[Mike G]

The Yahoo Sports NBA blog Ball Don't Lie notes about the Miami Heat:
"... after their woeful start to the season, they are still just 23-30, which means their 12-game run is now the longest winning streak ever by a team that’s still under .500"
So while b-r.com projects them to win 37.4 games at this rate, their avg MOV (-4.0) is that of a typical 30-52 team.

Since my Mia prediction was one of the lowest at 31, their inordinate good luck irks me just a bit. Manually 'correcting' this (Mia 37.4 to 30.1) cuts my distance from the avg error leader by almost 2/3
The Heat's SOS so far is .13, roughly average. Meanwhile, let's see how we all fare when all teams are given their to-date Pythagorean win% *82:

Code: Select all

2/09   RMSE      avg abs err             SMRE
amp5   5.36      amp5   4.01      amp5   3.23
RyRi   5.39      RyRi   4.16      BaDo   3.36
lnqi   5.41      ATCt   4.16      ATCt   3.36
trzu   5.48      BaDo   4.20      trzu   3.46
sbs.   5.54      sbs.   4.21      RyRi   3.47
ncsD   5.55      lnqi   4.22      sbs.   3.49
ATCt   5.59      trzu   4.27      vegas  3.51
kmed   5.63      vegas  4.32      lnqi   3.61
BaDo   5.63      Mike   4.38      Mike   3.64
sndi   5.67      ncsD   4.51      538    3.78
Mike   5.69      kmed   4.52      Crow   3.79
vegas  5.69      sndi   4.54      kmed   3.80
AnJo   5.77      AnJo   4.61      ncsB   3.80
cali   5.84      jg34   4.67      sndi   3.84
ncsB   5.93      Crow   4.68      AnJo   3.85
Crow   5.97      ncsB   4.72      ncsD   3.87
jg34   6.04      nrfo   4.77      shad   3.88
taco   6.08      cali   4.82      jg34   3.89
nrfo   6.21      taco   4.83      nrfo   3.90
yoop   6.21      538    4.84      taco   4.04
GK5.   6.31      shad   4.89      GK5.   4.13
shad   6.32      GK5.   4.94      yoop   4.17
538    6.34      yoop   4.97      cali   4.19

Wow. Major upheaval in the ranks; and 28-38% bigger errors for the frontrunners.
Part of this is due to the absence of SOS consideration, for games past and future -- the Pacers' -1.1 schedule to date portends an especially tough final 30 games.

[Mike G]

Back to b-r.com

Code: Select all

2/12   RMSE      avg abs err             SMRE
trzu   4.10      trzu   3.23      trzu   2.64
BaDo   4.34      Crow   3.41      yoop   2.72
lnqi   4.44      BaDo   3.45      Crow   2.73
RyRi   4.48      lnqi   3.48      kmed   2.90
Crow   4.55      yoop   3.49      jg34   2.90
amp5   4.59      RyRi   3.58      lnqi   2.92
vegas  4.65      kmed   3.60      sbs.   2.92
sbs.   4.65      sbs.   3.60      BaDo   2.96
ATCt   4.68      vegas  3.63      cali   3.02
AnJo   4.69      cali   3.64      vegas  3.04
cali   4.69      amp5   3.66      amp5   3.05
shad   4.74      ncsD   3.76      RyRi   3.07
yoop   4.81      jg34   3.76      ncsD   3.08
kmed   4.82      sndi   3.83      sndi   3.15
ncsD   4.88      shad   3.84      Mike   3.30
sndi   4.98      Mike   3.91      shad   3.31
Mike   5.08      ATCt   3.92      ATCt   3.45
nrfo   5.14      AnJo   3.95      AnJo   3.47
jg34   5.22      ncsB   4.21      ncsB   3.58
taco   5.24      taco   4.31      taco   3.76
ncsB   5.25      nrfo   4.43      GK5.   3.95
GK5.   5.54      GK5.   4.53      nrfo   4.00
538    6.12      538    4.85      538    4.07

Since 9 days ago, the leader is .36 to .42 worse in each column, as teams insist on under- or over-performing.

[Crow]

I am pleased with my mid-season improvement.

[Mike G]

I am pleased with my mid-season improvement.

Offhand, it looks like your predictions are benefiting by recent success of Atl, Bos, GSW, Mem, Mia, and Was. Dropoffs by Cha, LAL, Orl, Tor are also in your favor.
Currently you have the best guess (or tied) in the field for Bos, Cha, Mem, Tor, and Was; worst on LAC, Phl and Phx.

Scanning over the predictions and noting highest and lowest guesses for each team, some observations:
With 30 of each, there are 60 (without multiply-counting ties) highest+lowest guesses; of these 11 are also Best guesses:

Code: Select all

yoop -- Dal 35, Orl 30, Por 40
Crow -- Mem 44, Was 46
BaDo -- Brk 20, Hou 51
nref -- Min 32, SAS 60
GK5 --- Phl 29
trzu -- Mil 35
Mike -- SAS 60

It's a given that one or the other (hi/low) will be worst; or they'll be tied for worst.
Yooper is undisputed king of the outliers; and lately it's working out well for him.
He's 3/9 (.333) on these, vs an overall rate of 11/60 (.183)

[Nate]

...
Yooper is undisputed king of the outliers; and lately it's working out well for him.
He's 3/9 (.333) on these, vs an overall rate of 11/60 (.183)

If we think of the contest as working like the NCAA bracket competitions where you only care about 'best', then there's game-theoretical motivation to pick some outliers within your own model.

[Mike G]

Commanding leads:

Code: Select all

2/14   RMSE      avg abs err             SMRE
trzu   4.07      trzu   3.20      trzu   2.62
lnqi   4.49      Crow   3.51      yoop   2.72
BaDo   4.60      lnqi   3.56      jg34   2.84
RyRi   4.63      yoop   3.57      Crow   2.85
vegas  4.67      BaDo   3.59      BaDo   2.95
Crow   4.68      vegas  3.68      cali   3.01
amp5   4.73      RyRi   3.70      lnqi   3.03
sbs.   4.78      cali   3.73      RyRi   3.08
ATCt   4.85      amp5   3.75      ncsD   3.10
AnJo   4.86      sbs.   3.79      vegas  3.14
cali   4.87      ncsD   3.80      Mike   3.14
yoop   4.91      jg34   3.83      amp5   3.14
shad   4.93      kmed   3.85      sndi   3.18
ncsD   4.95      sndi   3.87      kmed   3.18
sndi   5.00      shad   3.92      sbs.   3.22
kmed   5.01      Mike   3.95      shad   3.27
Mike   5.17      ATCt   4.00      ATCt   3.42
taco   5.30      AnJo   4.05      AnJo   3.47
jg34   5.31      ncsB   4.29      ncsB   3.51
nrfo   5.38      taco   4.37      taco   3.80
ncsB   5.40      nrfo   4.63      nrfo   4.14
GK5.   5.76      GK5.   4.79      GK5.   4.20
538    6.25      538    5.06      538    4.38

[tarrazu]

This "My Highly Bayesian Win Projections (win projections updated for bet line and Preseason performance)" from Andrew Johnson would be 2nd right now by RMSE. This is part demonstrates the power of blends and ensembles.

https://twitter.com/CountingBaskets/sta ... 6899343361

http://nyloncalculus.com/2016/09/27/hig ... 2016-2017/

Did Andrew's projections not get entered into the contest? This was published pre-season. Seems like a shame for him to not be included after winning the contest for two years running.

[Mike G]

Averaging our 20 guesses would also be a 2nd place entry right now, in both avg error and RMSE.
The AnJo included here is apparently his unblended, less Bayesian, "highly plausible", PT-PM.
Anyway: all-star-break edition.

Code: Select all

2/17   RMSE      avg abs err             SMRE
trzu   4.08      trzu   3.21      trzu   2.65
lnqi   4.41      Crow   3.40      yoop   2.66
Crow   4.51      BaDo   3.47      Crow   2.79
BaDo   4.51      lnqi   3.50      kmed   2.85
RyRi   4.54      yoop   3.57      BaDo   2.85
vegas  4.60      RyRi   3.59      lnqi   2.94
sbs.   4.69      kmed   3.66      RyRi   2.97
amp5   4.69      sbs.   3.67      shad   3.02
ATCt   4.76      vegas  3.67      jg34   3.02
cali   4.77      amp5   3.71      ncsD   3.04
shad   4.84      shad   3.71      sbs.   3.05
ncsD   4.85      ncsD   3.75      amp5   3.07
AnJo   4.89      cali   3.80      sndi   3.15
kmed   4.90      jg34   3.84      vegas  3.17
yoop   4.90      sndi   3.84      cali   3.21
sndi   4.93      ATCt   3.93      ATCt   3.35
Mike   5.12      Mike   4.03      Mike   3.40
taco   5.14      ncsB   4.14      ncsB   3.46
nrfo   5.21      AnJo   4.16      AnJo   3.71
jg34   5.23      taco   4.27      taco   3.79
ncsB   5.26      nrfo   4.47      nrfo   4.02
GK5.   5.67      GK5.   4.66      GK5.   4.08
538    6.13      538    4.95      538    4.30

[Mike G]

Currently a retrospective blend of about half our predictions would enjoy a substantial lead here, in absolute avg error.

Code: Select all

%wt    agent   rk
.252    yoop    3
.173    Crow    5
.160    trzu    1
.127    amp5   11
.090    Mike   17
.076    BaDo    2
.062    ncsD   10
.030    shad    9
.026    kmed    7
.004    GK5.   20
1.00        

Just multiply these agents' predictions by the coefficient shown, and add them up.
Here's how that cheat (YACT) fares vs our top 10:

Code: Select all

2/25   RMSE      avg abs err             SMRE
trzu   3.68      trzu   2.87      trzu   2.28
YACT   3.96      YACT   2.89      YACT   2.05

trzu   4.19      trzu   3.29      trzu   2.72
lnqi   4.48      BaDo   3.45      yoop   2.74
BaDo   4.51      yoop   3.54      BaDo   2.79
RyRi   4.56      lnqi   3.55      shad   2.92
Crow   4.65      Crow   3.55      kmed   2.93
vegas  4.69      RyRi   3.60      lnqi   2.94
sbs.   4.70      kmed   3.66      Crow   3.00
amp5   4.71      sbs.   3.68      RyRi   3.02
ATCt   4.75      shad   3.71      sbs.   3.04
cali   4.76      ncsD   3.74      ncsD   3.08

The top line is the best/lowest error to date.
YACT is abbreviation from first letter of it's top 4 contributors.
A slightly better blend can be made if weights are not constrained to non-negative values.

[Mike G]

Shrinking leads and a new co-leader:

Code: Select all

2/28   RMSE      avg abs err             SMRE
trzu   4.19      trzu   3.30      kmed   2.59
BaDo   4.36      BaDo   3.37      trzu   2.73
lnqi   4.43      Crow   3.41      BaDo   2.78
RyRi   4.51      kmed   3.46      yoop   2.79
Crow   4.57      lnqi   3.55      Crow   2.80
vegas  4.58      yoop   3.60      lnqi   2.95
amp5   4.63      RyRi   3.63      ncsD   3.05
sbs.   4.64      sbs.   3.68      sbs.   3.09
cali   4.66      cali   3.70      jg34   3.10
ATCt   4.70      vegas  3.72      RyRi   3.10
kmed   4.70      amp5   3.72      shad   3.13
ncsD   4.81      ncsD   3.72      cali   3.15
yoop   4.84      shad   3.82      amp5   3.17
shad   4.88      sndi   3.88      vegas  3.24
AnJo   4.91      jg34   3.94      sndi   3.28
sndi   4.92      ATCt   3.97      ncsB   3.31
Mike   5.04      Mike   4.02      Mike   3.47
ncsB   5.19      ncsB   4.05      ATCt   3.51
nrfo   5.22      AnJo   4.17      AnJo   3.69
taco   5.24      taco   4.33      taco   3.77
jg34   5.24      nrfo   4.49      GK5.   3.93
GK5.   5.52      GK5.   4.51      nrfo   4.05
538    5.98      538    4.80      538    4.07

Code: Select all

3/01   RMSE      avg abs err             SMRE
trzu   4.21      trzu   3.36      BaDo   2.70
BaDo   4.39      BaDo   3.36      kmed   2.71
lnqi   4.46      Crow   3.50      trzu   2.85
RyRi   4.56      kmed   3.53      yoop   2.87
vegas  4.61      lnqi   3.55      Crow   2.93
Crow   4.63      yoop   3.64      lnqi   2.99
amp5   4.68      RyRi   3.65      shad   3.06
sbs.   4.70      vegas  3.72      ncsD   3.13
cali   4.71      amp5   3.72      RyRi   3.13
ATCt   4.78      sbs.   3.72      jg34   3.16
kmed   4.79      cali   3.73      sbs.   3.17

[Crow]

(revised)

What tool / method did you use to find the optimal weight prediction blend?

Have you tried applying that tool / method to box score stats to find optimal blends of these?

What set of box score stat weights line up with your optimal prediction blend so far? It would be useful to find the optimal set of these (for a season or longer). Reversing engineering it, since tracking the actual weights is limited by transparency hurdles and complicated math to blend accurately because of different formulas.