2017-18 team win projection contest

[Mike G]

Now a bunch at the top.

Code: Select all

avg abs err   RMSE      avg abs err   RMSE
RyRi   4.88   6.07      Josh   5.26   6.56
sndi   4.89   6.12     vegas   5.28   6.39
gold   4.94   6.34      kmed   5.37   6.59
shad   4.98   6.08      GK5.   5.61   6.93
sbs.   4.99   6.09      emin   5.82   6.72
538_   5.03   6.48      Mike   5.84   6.88
cali   5.07   6.38      Crow   5.90   7.17
AJBk   5.13   6.58      ATCt   6.08   7.40
lnqi   5.23   6.19      ncs.   6.24   7.64
Nath   5.25   6.47      knar   6.30   7.55

Note RyanRiot also has a .01 lead in the RMSE column.
Different exponents put several different people in the lead:
up to .16 -- caliban
.17 to .64 -- 538
.65 to .86 -- sndesai
.87 to 2.1 -- RyRi
2.2 to 7.2 -- shadow
7.3 and up -- eminence

[eminence]

A vote for 7.3 and up

[Mike G]

At the 8th power of errors, the leaders alongside the worst-predicted teams:

Code: Select all

tm   emin  shad  sndi  sbs.  lnqi
LAC   677   979   677  1040   767
NYK   402   351   524   376   677
Ind   266   199   248   285   376
Den    92    39   199    51    30
Det    51    25    61    47    15
LAL    36    25    51    25     9
Mil    17    15     6    10     6
GSW    17    20    36    19     6
...
tot  1600  1665  1809  1857  1901

These are actually millions of exaggerated errors. So emin has a total of 1.6 billion; which is great next to the last-place entry at this level of exaggeration, AJ at 19 bill.

Since yesterday, the leaders have reshuffled.

Code: Select all

exp = 1       exp = 2
sndi  4.76    shad  5.98
RyRi  4.80    RyRi  6.00
shad  4.87    sbs.  6.02
gold  4.90    sndi  6.05
sbs.  4.94    lnqi  6.14

RyRi range has shrunk to [1.28 - 1.82] -- he'd be the leader in any blend of MAE and RMSE.

Dec 24 update

Code: Select all

avg abs err   RMSE      avg abs err   RMSE
sndi   4.74   6.01      Nath   5.16   6.41
RyRi   4.76   5.90      lnqi   5.20   6.07
shad   4.81   5.96      kmed   5.40   6.62
sbs.   4.83   5.95      GK5.   5.41   6.68
gold   4.87   6.27      emin   5.49   6.44
538_   4.91   6.41      Mike   5.69   6.85
vegas  4.98   6.28      Crow   5.71   7.02
cali   5.05   6.36      ncs.   5.89   7.46
Josh   5.07   6.25      ATCt   5.96   7.22
AJBk   5.10   6.50      knar   6.11   7.44

The top half of the group are separated by .36 -- bottom half by .95

Dec 27 update. Those ^^ separations have grown to .50 and 1.00

Code: Select all

avg abs err   RMSE      avg abs err   RMSE
RyRi   4.34   5.53      lnqi   4.89   5.66
sbs.   4.47   5.59      Nath   4.93   6.08
shad   4.55   5.62      GK5.   4.98   6.30
sndi   4.67   5.78      kmed   5.11   6.41
gold   4.68   5.88      emin   5.34   6.27
538_   4.69   6.09      ncs.   5.47   7.10
vegas  4.70   5.91      Mike   5.47   6.58
Josh   4.77   5.91      Crow   5.52   6.74
AJBk   4.83   6.15      ATCt   5.67   6.91
cali   4.84   6.00      knar   5.88   7.11

These are now the smallest errors of the year.

[Mike G]

In 2 weeks, all our avg errors are better, by an avg of .50

Code: Select all

impr   Dec   15th   29th
.99   ncs.   6.13   5.13
.81   Josh   5.26   4.45
.79   GK5.   5.56   4.77
.76  vegas   5.18   4.42
.76   RyRi   4.90   4.14
.64   shad   4.91   4.27
.62   sbs.   4.90   4.28
.51   emin   5.77   5.26
.49   AJBk   5.09   4.60
.43   538_   5.06   4.63
.40   Mike   5.72   5.32
.39   ATCt   5.85   5.46
.39   gold   4.99   4.60
.37   kmed   5.32   4.96
.34   knar   6.04   5.70
.33   Nath   5.06   4.73
.31   Crow   5.72   5.41
.29   lnqi   5.04   4.75
.25   cali   4.90   4.65
.07   sndi   4.75   4.67

RyRi at 4.14 is lowest so far.

[jgoldstein34]

Here's how my model has the contest compared to my own projections going into tonight's game:



Just the average of all our projections would be in 2nd so far.

[Mike G]

We've broken 4.0 in the b-r.com projections also:

Code: Select all

abs avg err   RMSE      abs avg err   RMSE
RyRi   3.97   5.06      538_   4.62   5.84
sbs.   4.05   5.13     * avg   4.79   5.85
AJBk   4.28   5.62      kmed   4.91   5.97
Josh   4.40   5.51      Nath   4.98   5.93
shad   4.44   5.17      ncs.   5.21   6.74
vegas  4.46   5.53      emin   5.33   6.07
gold   4.49   5.42      Mike   5.33   6.35
lnqi   4.52   5.21     17PyR   5.44   6.55
cali   4.54   5.57      ATCt   5.53   6.52
GK5.   4.55   5.85      Crow   5.56   6.50
sndi   4.62   5.53      knar   5.67   6.75

Funny, our group average ranks much higher in jgold's list. He also has the best teams winning 2-4 more games than b-r.com gets. In fact, the worst teams may be tanking later, and the contenders getting more serious.

[J.E.]

In fact, the worst teams may be tanking later, and the contenders getting more serious.

Agree with the former, disagree with the latter. Teams that have their seed locked up will probably rest players near the end of the season

[Mike G]

Well, they can't all get worse. Teams will be contending for various playoff seeds, with perhaps a few exceptions.
It may be just that b-r.com's forecasting is still regressing toward the mean.

It's pretty late in the season for the 2016-17 regressed pythagorean entry to be in the mix.

[jgoldstein34]

The spread on my projections is higher because B-R uses a lot of regression in their future game projections.

[Voyaging]

Has anyone done any analysis of NBA's PIE rating (available on the official NBA stats site)?

A popular handicap site "sportsmodelanalytics.com" uses PIE as a core of their handicapping and they've got a pretty good track record (though they handicap individual games rather than make win total predictions preseason). According to the site, "a team's PIE rating and a team's winning percentage correlate at an R square of .908."

Thoughts? As someone with no background in statistics, how does that compare to other predictive models like the ones you guys are using?

[jgoldstein34]

Update to the contest. Appears we all got a little bit worse in average error since the last update.

[Mike G]

Oddly enough, jgold's projected errors are now larger than those from b-r.com

Code: Select all

avg abs err   RMSE      avg abs err   RMSE
RyRi   3.87   5.13      538_   4.59   5.85
sbs.   4.05   5.10      GK5.   4.66   6.03
shad   4.17   5.13      Nath   4.84   5.80
vegas  4.33   5.40      kmed   4.99   5.96
AJBk   4.39   5.74      Mike   5.14   6.20
gold   4.41   5.41      emin   5.15   5.90
lnqi   4.45   5.19      ncs.   5.26   6.71
Josh   4.49   5.73      ATCt   5.39   6.50
sndi   4.55   5.51      Crow   5.40   6.37
cali   4.59   5.54      knar   5.71   6.70

We did all get worse in the last couple of days, but RyRi weathered it pretty well. Best I noticed was 3.79 on Jan. 5.

[Crow]

I'll have to refresh my knowledge of PIE but my memory is that it is weak sauce, worse than the other better known weak box score metrics.

[Crow]

Yep, weak. PIE may be an improvement over "NBA efficiency" but that was one the simplest, weakest metrics out there for evaluating players. Correlation with team results is another very different thing. So it depends what you are trying to do. But with its simplistic weights and major omissions I doubt that it is very good or even competitive at that either.

[Voyaging]

I'll have to refresh my knowledge of PIE but my memory is that it is weak sauce, worse than the other better known weak box score metrics.

Yep, weak. PIE may be an improvement over "NBA efficiency" but that was one the simplest, weakest metrics out there for evaluating players. Correlation with team results is another very different thing. So it depends what you are trying to do. But with its simplistic weights and major omissions I doubt that it is very good or even competitive at that either.

That's what I would've thought, too, but I was surprised by how good the site's handicapping track record was.

Our college basketball model, the CBB Lockatron, formulated projections for just under 2000 college basketball games during the 2016-2017 season. It was its first season and with almost 2000 projections in the books, there's a lot of encouraging data and profit to look back on.

When the model's projection differed from the spread by at least 5 points, the model went 223-180 (55%) against the spread.

The model was most profitable when identifying value in the underdog. When the model liked the underdog to cover and the projection differed from the spread by at least 4.9 points, it went 149-103 (59%). The model also tracked public betting percentages. If you "faded the public" and only took the aforementioned plays (underdogs at 4.9+) that 40% or less of the public was on, the model's record was 62-38 (62%). At the very top segment of projections (the ones differing from the spread the most), the model was spot-on with underdogs. At a difference of 6.7 or greater, it went 68-45 (60%) with underdogs. Again, fading the public (40% or less) improved this record to 33-16 (67%).

When the model found significantly more value in the favorite's spread (5.2 difference or greater), it had moderate success, as long as it wasn't a play the public was absolutely hammering (85% action or less on the favorite). Its record was 73-62 (54%).

Not many CBB models can claim ATS (against the spread) win rates of 59%+ at their highest value points. We're excited for the 2017-2017 CBB season!

...

The NBA Lockatron formulated projections for just under 500 games during the 2016-2017 season. When projections differed enough from the spread to meet optimal criteria for underdogs or favorites, the model has spit out plays at a 65% win rate (50-27), and went on a 26-11 run against the spread from 03/10/17 through 04/19/17, before finally slowing down in the playoffs.

The model was revamped during the off-season and is back for the 2017-2018 season with a vengeance. In the first couple weeks of NBA action, the model has already started 9-1 on picks at a 6.6+ value.

Perhaps similar or superior results can be had by using better metrics and using a similar betting method.

Sorry if this isn't appropriate content for this thread, I just figured some people might be interested as perhaps aspects of the methodology could be applied to win total predictions, and I was surprised by their use of PIE which I would have thought was a poor choice.

I am particularly interested in this quote:

A high PIE % is highly correlated to winning. In fact, a team's PIE rating and a team's winning percentage correlate at an R square of .908.

I don't know enough about statistics to evaluate this claim. Are there any similar R square correlation to win % evaluations of other all-in-one player metrics?