The debut and popularization of BPM

Home for all your discussion of basketball statistical analysis.
Crow
Posts: 6250
Joined: Thu Apr 14, 2011 11:10 pm

Re: The popularization of BPM

Post by Crow » Sun Nov 16, 2014 3:20 am

Up to him of course.

I very rarely have any interest going back more than 10 years except to track a team trend.

talkingpractice
Posts: 194
Joined: Tue Oct 30, 2012 6:58 pm
Location: The Alpha Quadrant
Contact:

Re: The popularization of BPM

Post by talkingpractice » Sun Nov 16, 2014 4:08 pm

Aren't the BPM's reported on bbref (other than the 2014 and 2015 values) in-sample values? I think this will greatly skew any attempts to blend BPM with other metrics, compare its performance to other metrics, etc.

DSMok1
Posts: 905
Joined: Thu Apr 14, 2011 11:18 pm
Location: Maine
Contact:

Re: The popularization of BPM

Post by DSMok1 » Sun Nov 16, 2014 8:27 pm

talkingpractice wrote:Aren't the BPM's reported on bbref (other than the 2014 and 2015 values) in-sample values? I think this will greatly skew any attempts to blend BPM with other metrics, compare its performance to other metrics, etc.
Neil doesn't compare metrics to RAPM, he compares to the subsequent season's performances, testing predictive ability. Also, he runs it back to the 70s, where the RAPM basis only ran back to 2000.
Developer of Box Plus/Minus
APBRmetrics Forum Administrator
GodismyJudgeOK.com/DStats/
Twitter.com/DSMok1

permaximum
Posts: 413
Joined: Tue Nov 27, 2012 7:04 pm

Re: The popularization of BPM

Post by permaximum » Mon Nov 17, 2014 3:46 pm

I can't improve much upon BPM without touching RAPM and I don't want to touch any kind of RAPM since there's no data pre-1990. 5% is all I could. One way to improve it would be developing an SPM from 1996-2014 RAPM without age adjustment. However there isn't that kind of data. Perhaps, I should use 14-year RAPM too to develop a metric.

Congrats DSMok1 for such a good metric. However I will improve it without touching RAPM one way or another :)

Edit: 20% increase in prediction power is the cap for me.

DSMok1
Posts: 905
Joined: Thu Apr 14, 2011 11:18 pm
Location: Maine
Contact:

Re: The popularization of BPM

Post by DSMok1 » Mon Nov 17, 2014 5:10 pm

permaximum wrote:I can't improve much upon BPM without touching RAPM and I don't want to touch any kind of RAPM since there's no data pre-1990. 5% is all I could. One way to improve it would be developing an SPM from 1996-2014 RAPM without age adjustment. However there isn't that kind of data. Perhaps, I should use 14-year RAPM too to develop a metric.

Congrats DSMok1 for such a good metric. However I will improve it without touching RAPM one way or another :)

Edit: 20% increase in prediction power is the cap for me.
Let us know how you do it, and how well it works!

BPM doesn't include a regression to the mean component, currently, but would need such to maximize predictive power, similar to the other box score stats. RAPM/xRAPM/RPM has regression to the mean built in already.
Developer of Box Plus/Minus
APBRmetrics Forum Administrator
GodismyJudgeOK.com/DStats/
Twitter.com/DSMok1

permaximum
Posts: 413
Joined: Tue Nov 27, 2012 7:04 pm

Re: The popularization of BPM

Post by permaximum » Mon Nov 17, 2014 6:10 pm

I don't have much time to develop a SPM from ground atm so that I'm using pure box scores (normal and advanced) to come with anything for the blend (no RAPM for obvious reasons) to help improve BPM's prediction power. However it's been a quite challenge since current public metrics already cover much of the box score info.

Currently, experimenting with weighted ridge, elastic-net, lasso regressions and cv methods etc. anything to improve prediction power. (Carefully optimized elastic-net is the winner atm). Gotta be careful about overfitting in the same time.

I'll try to come with a new metric again for the blend tomorrow optimized at targeting BPM's weaknesses and I'll get WP data too. Perhaps WP has some info BPM misses.

permaximum
Posts: 413
Joined: Tue Nov 27, 2012 7:04 pm

Re: The popularization of BPM

Post by permaximum » Mon Nov 17, 2014 11:50 pm

No surprise... WP is a dead-end. The worst metric in fact. Even worse than PER.

I need to include players that played in 2 teams in one season for more precise calculations. B-R's player season finder doesn't really help with "TOT" so I'll think something else. Not sure how much a difference would those players make...

permaximum
Posts: 413
Joined: Tue Nov 27, 2012 7:04 pm

Re: The popularization of BPM

Post by permaximum » Tue Nov 18, 2014 9:58 am

talkingpractice wrote:Aren't the BPM's reported on bbref (other than the 2014 and 2015 values) in-sample values? I think this will greatly skew any attempts to blend BPM with other metrics, compare its performance to other metrics, etc.
I fear that's the case here... Because - to my surprise - WS beats BPM in 2014-15 by a quite margin so far. However it's still too early in the season to come to exact conclusions.

Crow
Posts: 6250
Joined: Thu Apr 14, 2011 11:10 pm

Re: The popularization of BPM

Post by Crow » Tue Nov 18, 2014 11:17 am

Did you see this? https://mobile.twitter.com/NBAcouchside ... 052352?p=v
Is it of interest / use to you?

DSMok1
Posts: 905
Joined: Thu Apr 14, 2011 11:18 pm
Location: Maine
Contact:

Re: The popularization of BPM

Post by DSMok1 » Tue Nov 18, 2014 12:03 pm

permaximum wrote:
talkingpractice wrote:Aren't the BPM's reported on bbref (other than the 2014 and 2015 values) in-sample values? I think this will greatly skew any attempts to blend BPM with other metrics, compare its performance to other metrics, etc.
I fear that's the case here... Because - to my surprise - WS beats BPM in 2014-15 by a quite margin so far. However it's still too early in the season to come to exact conclusions.
Beats BPM this season so far? Please explain what/how you're measuring.... Explaining lineup performance?
Developer of Box Plus/Minus
APBRmetrics Forum Administrator
GodismyJudgeOK.com/DStats/
Twitter.com/DSMok1

permaximum
Posts: 413
Joined: Tue Nov 27, 2012 7:04 pm

Re: The popularization of BPM

Post by permaximum » Tue Nov 18, 2014 2:28 pm

I need to give BPM's rights back. It looks I made a copy-paste error in the code before. BPM is the one that's noticably better than WS in 2015. Now I have further results for out-sample prediction performance of BPM and WS.

(C stands for centerization at 0 for league total margin which improves in-sample -crossvalidation- and out-sample prediction) I especially used 01-14 data to come with min CV error for both CDBPM+COPBM AND CDWS+COWS. Then predicted 86-00 with those models. RMSE is considerably lower for BPM model which means BPM is a lot better.

Then I used 86-14 data to come with min CV error. Then predicted 2015 season. BPM is better again. Actually RMSE of team margin stands at 20.93534 for BPM while it's 23.6831 for WS.

@Crow

Thanks. I haven't seen it before. I don't think there's much can do to improve upon BPM with 14 year RAPM data but I'll give it a go one day.

Neil Paine
Posts: 73
Joined: Mon Apr 18, 2011 1:18 am
Location: Philadelphia
Contact:

Re: The popularization of BPM

Post by Neil Paine » Tue Nov 18, 2014 4:20 pm

This is what I've been saying. :) I can't yet speak to the newer metrics people have sent me (btw, thanks to all who sent them in!), but plus/minus-based regressions consistently kill WS, PER, and WP.

It makes sense -- those other metrics try to logically intuit their weights, but end up "baseball-ifying" the numbers in a way that doesn't actually hold up when applied to real basketball. By contrast, the weights for something like BPM are derived from what tends to correlate with empirical on-court impact, as measured by plus/minus. So of course BPM is going to perform better out of sample. It's the difference between what works in theory and what works in practice.

Mike G
Posts: 4429
Joined: Fri Apr 15, 2011 12:02 am
Location: Asheville, NC

Re: The popularization of BPM

Post by Mike G » Tue Nov 18, 2014 6:05 pm

Neil, can you give us a 'last call' on these things? I am painstakingly re-creating eWins year by year into the past, and I'd like to get you all that I can muster in the allotted time.

FWIW, eWins adjusts a bunch of parameters and weights in order to force team eWins to match up with pythagorean wins from MOV. So players' stats are indirectly a function of team plus-minus.

Some parameters naturally stay within certain limits. For others, I maintain a floor and/or a ceiling. That's why I wondered whether the Ast*Reb coefficient in BPM should have such a ceiling, so that DReb should not become negative. Then the Moses Deficit might not have to occur, with little loss to overall metric performance.

Neil Paine
Posts: 73
Joined: Mon Apr 18, 2011 1:18 am
Location: Philadelphia
Contact:

Re: The popularization of BPM

Post by Neil Paine » Tue Nov 18, 2014 7:47 pm

Mike G wrote:Neil, can you give us a 'last call' on these things? I am painstakingly re-creating eWins year by year into the past, and I'd like to get you all that I can muster in the allotted time.
I was going to start working on this today, so anything last-minute you can add now would be appreciated.

mystic
Posts: 470
Joined: Mon Apr 18, 2011 10:09 am
Contact:

Re: The popularization of BPM

Post by mystic » Tue Nov 18, 2014 8:19 pm

Daniel, do you have the R² for BPM on the 14yr RAPM just for the players with 2270 or more minutes (should be 714 players)? Or can you calculate those? I'm working on a "new" way to interpret boxscore-data while using that subset of players and would like to know whether pursuing that avenue is worth it.

Post Reply