2022-23 team win projection contest

[Crow]

If stretched to all-time I believe my average rank would be higher. I am pretty sure I am the only 3-time winner.

[tarrazu]

I got 3 wins, 16-17, 18-19, 21-22. Not sure how that corresponds to those percentiles. You can be in rare company with me Crow.

[Crow]

I knew you had at least one but I didn't run down all the totals at this time.

Until your win last season I was sure I was ahead of all and didn't realize til you just spoke up that perhaps last season changed this.


Close between us this season but not at the top.

[Crow]

Mike G's accounting here has 2 ahead of you in 2017 but we have had several measuring sticks and multiple felt winners at times including in 2017. And who was an official entry and not is another factor.

But not wanting re-litigation of the past.

Tied at 3 with you is fine for me.
Just us as far I know.

Overall, more than a handful of winners and lots of good to great performances.

[DarkStar48]

Luck is big.
Here's a decidedly incomplete table of entries who were entered at least twice in the last 7 years. The numbers are a sort of 'percentile', as the % of entries that finished beneath you. Middle of the pack is around 50.

Code: Select all

year  2023 2022 2021 2020 2019 2018 2017      
#      19   13   16   17   18   20   23  yrs avg
Crow   58   46   56   94   28   15   57   7   51
vegas  21   62   88   18   39   50   70   7   49
538    32   77   19   24   22   90    0   7   38
cali        69   63   88   89   65   87   6   77
trzu   53   92   38   47   94        78   6   67
shad        85   81   41   50   40   30   6   55
emin   47   38   13   65   72   95        6   55
KPel   89        31   35   78             4   58
RyRi                  59   44   60   52   4   54
sndi                  53   33   85   39   4   53
dtka   95   54   50   12                  4   53
ncs    63             29         0   65   4   39
eWin         8         0        45   35   4   22

year  2023 2022 2021 2020 2019 2018 2017 
bbst             69   71   67             3   69
kmed                       17   80   96   3   64
gold                  76   56   25        3   52
lisp        31   25   82                  3   46
AnJo                       83   10   43   3   46
lnqi                            70   78   2   74
sbs                             75   48   2   61
EExp   11        94                       2   52
EBPI   68         0                       2   34
ATCt                            30   30   2   30
GK5                             35   17   2   26
Rd11                   6    6             2    6

Caliban did not show up this year, but they were in or near top 1/3 for 6 years straight.
538 is probably now 538R or a precedent form.

Caliban the GOAT!

Yeah, it’s really unfortunate that @caliban didn’t participate this year.

Moreover, on @Crow’s point about “unofficial” winners, kmedved’s DARKO projection did so last season (see: https://i.imgur.com/XDACgyj.png).

That, and DARKO also being Top 9-ish this year despite being relatively new may speak to the benefit of updating prediction priors with pre-season games data.

An “optimal” approach probably involves a combination of metric blending AND preseason adjustment.

[Mike G]

It may be that prior MAE or RMSE better predicts how well an entrant will do in the upcoming season. It may be that an exponent between 1 and 2 is better; or less than 1.

Skimming over the table, it looks pretty random from year to year.
However, if anyone should just average the predictions of all apbr entries -- this "entry" is know as avgA this year -- they would almost certainly be in contention.

And without such a blatant shortcut, it's usually good to avoid "extreme" guesses. This year's leader was not highest or lowest on any team; they're just on the "right side" of most.
The closest dtka comes to an outlier would be Atl -- guessing 50, which turns out badly. Otherwise, mostly middle-of-the-pack predictions which are often good and never very bad, relative to everyone else.

Now tarrazu also had no highest or lowest guesses, and they're merely middling among "us"; still better than avg in the whole field.
I admire the bold prediction anyway. Crow and eminence continue that tradition.

lots of good to great performances.

Yes, the amateurs and others here have done well against standards such as "vegas" most years.
But there's no absolute from year to year. Some years, teams just don't do what they're spozed to.
The worst prediction errors in 2017 would have been the best in other years:

Code: Select all

year   top    mid  bottom
2016   5.73   7.00   8.60
2017   3.84   4.20   4.93
2018   5.80   6.10   6.76
2019   5.87   6.60   7.88
2020   6.07   6.70   7.78
2021   5.47   6.00   8.33
2022   6.49   7.10   8.63
2023   5.06   6.00   6.98

The "bottom" numbers are actually next-to-worst, as there are often outliers that either weren't serious or otherwise off the rails.

[Mike G]

Code: Select all

.   avg err   rmse   r^2          avg err   rmse   r^2
dtka   5.02   6.13   .64      BIPM   6.14   7.62   .63
avgA   5.42   6.40   .60      538R   6.26   7.79   .49
KPel   5.44   6.68   .54      22Re   6.60   7.97   .34
vzro   5.61   6.92   .53      TmRk   6.76   7.93   .48
DRKO   5.76   6.75   .55     vegas   6.76   8.00   .48
LEBR   5.83   6.52   .58      nuFi   6.92   8.25   .46
emin   5.90   7.37   .52      AnBa   6.93   8.21   .50
MPra   5.91   7.41   .61      EExp   6.94   8.32   .47
trzu   5.93   6.97   .58      4141   6.97   9.80   
EBPI   5.95   7.35   .49      538E   7.58   9.09   .29
Crow   5.97   6.83   .59      2022   8.23   9.79   .34
ncs.   6.00   7.04   .52               

This hasn't changed much, but here it is again:

Code: Select all

Atl  w   Bos  w   Brk  w   Cha  w   Chi  w   Cle  w   Det  w   Ind  w
dt  50   vz  58   em  51   KP  41   nc  41   b-r 52   vz  31   KP  38
tr  48   tr  57   nc  47   vz  39   cr  40   em  50   nc  27   b-r 36
cr  47   nc  57   tr  47   dt  33   b-r 40   tr  48   em  27   nc  34
KP  46   b-r 56   KP  46   nc  32   KP  38   vz  48   KP  26   em  33
nc  45   em  56   b-r 44   em  32   dt  38   cr  47   dt  24   vz  32
vz  44   dt  55   dt  44   tr  32   tr  38   dt  46   tr  24   dt  31
em  44   KP  54   cr  44   cr  30   vz  37   KP  43   cr  23   tr  28
b-r 40   cr  53   vz  42   b-r 27   em  35   nc  43   b-r 18   cr  28
                                             
Mia  w   Mil  w   NYK  w   Orl  w   Phl  w   Tor  w   Was  w      
nc  51   b-r 58   b-r 46   b-r 34   b-r 54   cr  52   vz  40      
tr  50   em  55   cr  44   KP  31   tr  54   tr  50   KP  40      
cr  50   dt  52   vz  43   em  28   dt  53   nc  49   dt  38      
vz  49   vz  52   nc  43   tr  28   em  52   dt  48   nc  37      
em  48   tr  50   dt  43   cr  28   nc  52   KP  47   b-r 37      
dt  47   KP  50   tr  42   nc  28   vz  51   vz  44   em  34      
KP  46   cr  49   KP  42   dt  26   cr  51   em  40   tr  33      
b-r 44   nc  48   em  36   vz  26   KP  48   b-r 40   cr  30      
                                             
Dal  w   Den  w   GSW  w   Hou  w   LAC  w   LAL  w   Mem  w   Min  w
nc  47   b-r 54   em  52   nc  27   cr  51   dt  43   cr  52   em  53
tr  47   cr  53   cr  51   cr  27   em  49   vz  43   b-r 51   tr  50
vz  46   em  51   tr  50   KP  27   tr  49   em  42   dt  50   dt  49
dt  46   dt  49   dt  47   em  24   vz  45   b-r 41   tr  48   cr  47
cr  46   tr  49   vz  47   tr  23   nc  45   nc  39   nc  47   nc  47
em  44   vz  48   nc  44   b-r 20   dt  45   tr  37   em  47   KP  46
KP  43   KP  48   b-r 43   dt  20   KP  44   KP  37   vz  47   vz  45
b-r 40   nc  47   KP  42   vz  18   b-r 43   cr  35   KP  46   b-r 41
                                             
NOP  w   OKC  w   Phx  w   Por  w   Sac  w   SAS  w   Uta  w      
cr  49   b-r 41   nc  54   em  40   b-r 49   vz  33   b-r 39      
tr  48   cr  27   tr  52   tr  37   cr  39   KP  31   vz  39      
KP  48   KP  27   vz  51   KP  37   tr  37   em  31   KP  35      
nc  46   dt  26   em  50   b-r 36   dt  37   nc  31   nc  35      
dt  46   vz  26   dt  50   dt  36   nc  37   dt  26   dt  32      
vz  44   nc  25   cr  50   vz  33   KP  37   cr  25   cr  29      
em  43   tr  24   KP  49   cr  33   vz  32   tr  25   tr  29      
b-r 40   em  23   b-r 43   nc  28   em  32   b-r 21   em  28      

UPDATE Mar.27

Code: Select all

.   avg err   rmse   r^2          avg err   rmse   r^2
dtka   5.07   6.15   .64      EBPI   6.06   7.43   .49
avgA   5.45   6.44   .60      BIPM   6.07   7.59   .63
vzro   5.59   6.99   .52      538R   6.27   7.86   .48
KPel   5.60   6.68   .54      22Re   6.62   8.03   .34
LEBR   5.72   6.46   .59      TmRk   6.78   7.92   .48
DRKO   5.81   6.79   .55     vegas   6.78   7.99   .49
Crow   5.90   6.80   .59      EExp   6.89   8.27   .47
ncs.   5.93   7.07   .52      nuFi   6.91   8.30   .46
MPra   6.00   7.44   .60      AnBa   6.92   8.17   .51
emin   6.02   7.47   .51      4141   7.04   9.88   
trzu   6.04   6.99   .58      538E   7.67   9.17   .29

[Crow]

I have a lot of close or closest to BRef projection.

Some big misses too. Raptors and Clippers are among the bigger disappointments.

I am guessing for some individual improvement at the end (don't know about relative) but we'll see.

[DarkStar48]

Skimming over the table, it looks pretty random from year to year.
However, if anyone should just average the predictions of all apbr entries -- this "entry" is know as avgA this year -- they would almost certainly be in contention.

To @Mike G’s point about averaging projections similar to the avgA “entry,” it’s actually a consistent finding in the forecasting literature that the simple arithmetic mean (i.e., with equal weighting) of different forecasts often provides highly accurate overall predictions. Link: https://www.researchgate.net/profile/An ... iewer=true

Complex approaches to estimating the ‘‘best’’ combining procedures do not seem to matter that much. Link: https://faculty.fuqua.duke.edu/~clemen/ ... JOF-89.pdf

The unweighted average provides an efficient trade-off between precision and uncertainty.

Obviously, there is a systemic issue here that if everyone or most are using an average to make predictions, there would no or few models to include in that average — affects the forecast accuracy and diminishes the sought benefit.

Still, it’s all a fascinating idea to keep in mind.

[v-zero]

My predictions are being seriously flattered by the MAE here, but to add the to prediction averaging chat: whilst I did not use any minutes projections at all for this, merely flattened average statistics at several roster sizes, I did average the predictions between those roster sizes. Some teams showed up as having a great 5 man roster, but lacked depth and as such were dragged down. Some were vice versa.

[Crow]

Could / would you say a little more about how you calculate "flattened average statistics" and the range of "roster sizes" included? Using mean or median?

You believe this gets at "talent" & depth. Ok. But actual coaching management of minutes is de-emphasized / eliminated. Does that suggest you more than others think coaches matter little relative to talent, at least as they currently coach and handle lineups & minutes?

[Crow]

I'm now 7th on average error and a little bit better still on the other two. Only 2 above me on average error are also above me on r2.

[Mike G]

... it’s actually a consistent finding in the forecasting literature that the simple arithmetic mean (i.e., with equal weighting) of different forecasts often provides highly accurate overall predictions...

Of course, not all forecasts are improved by sheer number of forecasters. The ESPN experts I think numbered 15-20, but they aren't among the leaders.
avgA is the error produced by averaging the predictions of our 6: emin, ncs, dtka, trzu, vzro, and Crow.
Just now I added each competing entry to those 6 and divided by 7, to see who else would improve (or worsen) the avg.
Further, I removed each of the 6 and divided by 5. From most to least improvement:

Code: Select all

err      x     impr       err     x      impr
5.16   +LEBR   .125      5.30   +22Re   -.017
5.20   -trzu   .090      5.32   +AnBa   -.031
5.20   +4141   .088      5.32   +DRKO   -.035
5.20   +BIPM   .087      5.36   +538R   -.074
5.22   +MPra   .064      5.39   +nuFi   -.101
5.22   -Crow   .063      5.40   +vegas  -.112
5.24   +KPel   .046      5.40   +TmRk   -.112
5.26   -emin   .030      5.40   +EExp   -.117
5.26   -ncs.   .026      5.43   -vzro   -.138
5.29   +EBPI   .003      5.43   -dtka   -.144
.                        5.43   +2022   -.145
5.29    avgA   .000      5.52   +538E   -.236

Adding LEBRON to the original 6 yields the best avg value.
Adding 538E is even worse than removing dtka.

There's some inverse correlation with individual entries' error sizes. But I suspect there's also a benefit from someone's predictions cancelling the recurring errors within our group of 6.
For example: BIPM isn't particularly great in the contest but offers these corrections: Our 6 were too high on Hou, Mia, and Det; BIPM had the lowest guess on them. We were well low on Sac and Mil; they were highest on those 2.

[Crow]

This is a reminder of how few were direct / official contest entries, fwiw. I am 3rd of 6 for that.

[v-zero]

Could / would you say a little more about how you calculate "flattened average statistics" and the range of "roster sizes" included? Using mean or median?

You believe this gets at "talent" & depth. Ok. But actual coaching management of minutes is de-emphasized / eliminated. Does that suggest you more than others think coaches matter little relative to talent, at least as they currently coach and handle lineups & minutes?

I think that coaching matters. I think allocating minutes to your best players, finding combinations that work well, keeping the locker room together and maintaining course on your plans matters. However, I think the impact a coach has tends to show up on the court relatively quickly, so player data begins to capture it. I think in the playoffs the impact of coaching is greater. But do I think talent is most of the equation? Yes. As long as your roster looks like it can provide enough of the essential elements of basketball (i.e. a reasonable distribution of offensive and defensive abilities) then I think talent is what will decide things more than anything else, so allocating minutes to that talent is priority number one.

Regarding details of how I calculated the features for these models, it went something like this:

Code: Select all

1.) Gather roster.
2.) For N from 5 to 9 do:
a.) take N players from roster with most allocated minutes in previous year.
b.) average the player statistics (advanced box stats, other stuff I have developed) across those N, without weighting by anything (hence a flat/unweighted average). Each averaged statistic becomes a feature in the model.

This would all be done using the mean rather than median.