The debut and popularization of BPM

[colts18]

A stat like RAPM will love Shaq because it picks all the things that make him a great player that the box score won't pick up (defensive attention, triple teams, hockey assists, fouls generated, etc.). BPM doesn't pick up those things so he gets underrated by the stat.

[v-zero]

Try it out. If a lineup's BPM sums to 0, then the expected performance is 0. But if it sums to +10, then it's expected performance is only +8.333 actually.

The 1.2 translates from the team context (where winning teams perform worse than if they were in a close game, and vise versa) to a neutral context.

Actually, RPM and the 14 year RAPM has the exact same scale and setup, with effectively the same 1.2 factor (it's the "playing with the lead" adjustment).

If that is the case then surely the factor of 1.2 is simply overstating player impact? Regardless of whether you first divide by all player values by 1.2 before calculating lineups, or divide the summed lineups by 1.2 after the fact, you will reach the same result. Lineups which sum to zero will still sum to zero, and lineups which sum to anything else will sum to their correct(ed) values, so player impact will always be BPM/1.2, and BPM then is needlessly inflated.

I would imagine the RAPM adjustment causes changes which cannot be captured by such a simple adjustment in a box score model.

[Mike G]

A stat like RAPM will love Shaq because it picks all the things that make him a great player that the box score won't pick up (defensive attention, triple teams, hockey assists, fouls generated, etc.). BPM doesn't pick up those things so he gets underrated by the stat.

Nevertheless, some box score stats are clear that Shaq was the uber-dominant player in the league for 5 or 6 years. PER and eWins say he's one of the top 4 players ever. BPM says he's 18th, not counting anyone's numbers before 1973-74. He ranks just below Kirilenko.

[Crow]

Relative to the others, BPM will favor passers, WS shooters,PER volume scorers, WP rebounders, RAPM defenders. A five part weighted blend did well in Alex's old retrodiction and should again if included. All at 20% for simplicity or maybe 25% to BPM and RPM, 15% to WP and PER and 20% to Ws. I am really leaning on a blended approach as compromise resolution for all the headaches of perceived slants in the box score metrics. For prediction, RPM should get a bigger share per Alex's findings.

[DSMok1]

Try it out. If a lineup's BPM sums to 0, then the expected performance is 0. But if it sums to +10, then it's expected performance is only +8.333 actually.

The 1.2 translates from the team context (where winning teams perform worse than if they were in a close game, and vise versa) to a neutral context.

Actually, RPM and the 14 year RAPM has the exact same scale and setup, with effectively the same 1.2 factor (it's the "playing with the lead" adjustment).

If that is the case then surely the factor of 1.2 is simply overstating player impact? Regardless of whether you first divide by all player values by 1.2 before calculating lineups, or divide the summed lineups by 1.2 after the fact, you will reach the same result. Lineups which sum to zero will still sum to zero, and lineups which sum to anything else will sum to their correct(ed) values, so player impact will always be BPM/1.2, and BPM then is needlessly inflated.

I would imagine the RAPM adjustment causes changes which cannot be captured by such a simple adjustment in a box score model.

The 1.2 is important for estimating things when players change teams, helped the overall fit of the regression as well significantly, and helps compare players from good and bad teams more accurately--a player who, without the 120% adjustment would be +0 would move to perhaps +0.5 on a good team and -0.5 on a bad team. For instance, without the 120% adjustment last season, Thadeus Young was a +0.0. With the adjustment, he was -0.5. Conversely, Matt Bonner was +0.0 without the 120% adjustment and +0.4 with it. It's basically adjusting for context--Young's teams were always behind and Bonner's teams were always ahead.

[DSMok1]

Relative to the others, BPM will favor passers, WS shooters,PER volume scorers, WP rebounders, RAPM defenders. A five part weighted blend did well in Alex's old retrodiction and should again if included. All at 20% for simplicity or maybe 25% to BPM and RPM, 15% to WP and PER and 20% to Ws. I am really leaning on a blended approach as compromise resolution for all the headaches of perceived slants in the box score metrics. For prediction, RPM should get a bigger share per Alex's findings.

This is true. Once there are a variety of box score metrics with similar out of sample performance (Only RAPM and WS are good of the list above), looking at a variety of similar-quality metrics that use different approaches will help to capture the potential error in a player's box score estimates.

[Neil Paine]

Nevertheless, some box score stats are clear that Shaq was the uber-dominant player in the league for 5 or 6 years. PER and eWins say he's one of the top 4 players ever. BPM says he's 18th, not counting anyone's numbers before 1973-74. He ranks just below Kirilenko.

The thing is, if you tweak the weights of BPM so that it better appreciates Shaq and Moses, then you also have to live with the changes to other players' ratings wrought by that decision. (Changes that would most likely reduce its out-of-sample performance, if what you want is a more PER-like metric.) There are always going to be these "corner cases" in any metric, but you can't pick and choose which component stats to value more/less for each individual player on a case-by-case basis.

[DSMok1]

Nevertheless, some box score stats are clear that Shaq was the uber-dominant player in the league for 5 or 6 years. PER and eWins say he's one of the top 4 players ever. BPM says he's 18th, not counting anyone's numbers before 1973-74. He ranks just below Kirilenko.

The thing is, if you tweak the weights of BPM so that it better appreciates Shaq and Moses, then you also have to live with the changes to other players' ratings wrought by that decision. (Changes that would most likely reduce its out-of-sample performance, if what you want is a more PER-like metric.) There are always going to be these "corner cases" in any metric, but you can't pick and choose which component stats to value more/less for each individual player on a case-by-case basis.

Absolutely. One must look at the overall performance of a model, not pick particular cases. I agree that BPM may be struggling with Moses, and Shaq could perhaps be underrepresented... but what about the other thousands of players?

[Mike G]

I'm guessing you can build a very good team out of players who don't have good Ast*Reb. As long as you have good passing from some positions, your PF or C don't have to be prolific passers.

It's not just a couple of 'corner' dwellers that make BPM in its present form look weird.
Here's a little survey of best single season by 90 players (since 1974) -- some may have had earlier, better seasons -- from the top 300 player-seasons.

Code: Select all

year    player   BPM      year    player    BPM      year    player    BPM
2009   LeBron   12.2      2001   Francis    6.4      2006   Arenas     5.2
1989   Jordan   11.6      1976   Adams      6.3      1975   Frazier    5.2
2009   Paul     10.3      2014   Noah       6.3      1992   Nance      5.2
1994   Robinson  9.9      1977   Walton     6.3      2001   Allen      5.2
2009   Wade      9.9      1994   Kemp       6.3      2013   Westbrook  5.2
1977   Kareem    9.6      1986  A Robertson 6.2      2010   Smoove     5.2
1989   Barkley   9.3      1997   Webber     6.2      1976   McGinnis   5.1
2004   Garnett   9.2      1996   A Hardaway 6.1      1974   Barry      5.1
1990   Magic     9.1      2007   Nowitzki   6.0      1995   Stockton   5.1
2003   McGrady   8.9      2010   Ginobili   6.0      1986   Pressey    5.1

1986   Bird      8.8      2000   Outlaw     5.9      2007   Marion     5.1
2000   Shaq      8.8      2003   Kobe       5.9      2010   Dwight     5.1
2014   Durant    8.2      2011   Rose       5.8      1992   Manning    5.1
1992   Drexler   8.0      1988   Lever      5.7      1997   T Hardaway 5.0
1997   Malone    7.9      1991   T Porter   5.7      2010   P Gasol    5.0
1994   Pippen    7.9      2008   Billups    5.7      1989   R Harper   5.0
2014   Love      7.9      2002   Brand      5.7      2000   E Jones    5.0
1974   Lanier    7.7      2013   M Gasol    5.6      1985   Richardson 4.9
1981   Erving    7.6      1976   J Drew     5.6      1992   Rodman     4.9
1993   Olajuwon  7.6      1981   Marques J  5.6      1985   Isiah      4.9

2004   Kirilenko 7.6      2014   Lowry      5.6      1991   K Johnson  4.9
1997   Hill      7.5      2007   B Wallace  5.5      2013   Noah       4.9
2014   Curry     7.1      2004   B Davis    5.5      1987   McHale     4.8
2002   Duncan    7.0      2014   Harden     5.5      1995   Divac      4.8
1977   B Jones   7.0      1975   Unseld     5.4      1980   Rollins    4.8
1992   H Grant   6.9      1996   Brandon    5.4      1990   Hornacek   4.8
1997   Blaylock  6.9      2009   Roy        5.4      1993   Daugherty  4.8
2003   Kidd      6.7      1983   Moncrief   5.3      1976   Boerwinkle 4.8
2001   Carter    6.6      2002   B Barry    5.3      1997   K Anderson 4.7
2000   Payton    6.4      1980   C Maxwell  5.2      1976   Cowens     4.7

http://bkref.com/tiny/Af7Oe
Mookie Blaylock cracks the top 30, thanks to good Ast*Reb, while Stockton rubs elbows with Paul Pressey.
Brent Barry was better than his dad. Bo Outlaw puts Dwight to shame.
Moses may have just managed one of the top 500 seasons since 1974.

Others and their best BPM: Blake Griffin (4.6 in 2013), Yao 2.6, Ewing 4.1, Mourning 3.9, Amar'e 3.2, Carmelo 3.3, Boozer 4.2, Al Jefferson 2.4, Iverson 4.6, Paul Pierce 4.6, 'Nique 4.7, Sabonis 4.1

[mystic]

Absolutely. One must look at the overall performance of a model, not pick particular cases.

So true, but a lot of people just seem to "check" the performance of a metric based on whether particular players show up on top.

But, yeah, I can do that too and maybe that should be more of a concern: Dirk Nowitzki from 2001 to 2014 ranked 7th in PER, 3rd in WS/48 and 4th in the used RAPM-file among Top100 players in terms of WS overall, while ending up at 16th during that timeframe in BPM. Given the fact that BPM is based on the regression on that RAPM-file, I guess picking up a particular unique player and his impact in such a dataset should show a weakness better than seeing that the in the RAPM-file 11th placed O'Neal ends up 15th in BPM.

Anyway, good to see another boxscore-based metric publically available in a very readable format (at least for me), especially with the fact in mind that BPM is doing that well in terms of predicting.

[Neil Paine]

It's not just a couple of 'corner' dwellers that make BPM in its present form look weird.

And yet, a team composed of players who are good at BPM is more reliably a good team than a team composed of players good at PER, WS, etc. Perhaps that "weirdness" relative to those other metrics actually helps it predict team outcomes more effectively.

Btw, can you send me an eWins file? I'd love to test it in the same manner.

[DSMok1]

Has everyone seen this visualization? http://public.tableausoftware.com/share ... _count=yes

It's from the BPM writeup, and shows where BPM is off vs. the RAPM basis.

Compare the rankings in the table on the right to get a feel for what is going on, and where BPM is weak. I tried to highlight the weak areas in the writeup. http://www.basketball-reference.com/about/bpm.html

[colts18]

And yet, a team composed of players who are good at BPM is more reliably a good team than a team composed of players good at PER, WS, etc. Perhaps that "weirdness" relative to those other metrics actually helps it predict team outcomes more effectively.

Btw, can you send me an eWins file? I'd love to test it in the same manner.

Neil, slightly OT, but why don't you do a RAPM like stat for your Randy Moss article. You used RANY, but why not use QB's, receivers, and age of those QB's/WR's (or TE's), then run a RAPM like regression to see which WR's had the most impact on QB's and vice versa?

[Mike G]

... a team composed of players who are good at BPM is more reliably a good team than a team composed of players good at PER, WS, etc. Perhaps that "weirdness" relative to those other metrics actually helps it predict team outcomes more effectively.
...

Last year, there were 40 players within 0.2 of average (zero) BPM. If I sort these 40 by eWins/minute, I can draw a "team" of players who are above-avg in my metric.
Facing off against their BPM peers who are below-avg according to eWins:

Code: Select all

BPM avg players         my team
Dorell Wright          LaMarcus Aldridge
Mike Miller            Nikola Pekovic
Dante Cunningham       Nene Hilario
Jeff Withey            Greg Monroe
Caron Butler           Jared Sullinger
Chris Douglas-Roberts  Monta Ellis
Martell Webster        Rajon Rondo
Andrei Kirilenko       Reggie Jackson
Wesley Johnson         Kyle O'Quinn
Maurice Harkless       Joe Johnson
Kyle Singler      
Iman Shumpert      
Rashard Lewis      
Andre Roberson      
Francisco Garcia      

Both teams total about 20,500 minutes -- a bit high.
My team totaled 57 eWins last year, vs 20 for the others. After scaling for minutes, it's about 55 to 19.
In Win Shares, it's a lot closer: 45 to 36
All my guys are above-avg in PER; all the others are below (excepting Withey, at 15.2)

I had to eject Ryan Anderson, Andre Blatche, and John Henson -- all PF -- to get some positional balance.
Does anyone think the players at left would be competitive with those on the right?

[DSMok1]

Nope, I don't. The guys at right are likely somewhat underrated by BPM, the ones at left likely somewhat overrated. Look at it from a Bayesian perspective: use BPM as prior and then update with with eWins, WS, or PER, which though they might be poorer-performing stats overall, still have some weight. I'm not saying eWins is poorer performing, but that even if it were, it would still have weight.

I would be interested to see eWins tested along with the rest.