The debut and popularization of BPM

[Statman]

The coefficients:
ORB% 0.137600 100.0
DRB% -0.151938 100.0
sqrt(AST%*TRB%) 0.691501 100.0*100.0

The total value from these terms would be:
ORB%: .137*12.4 = +1.7
DRB%: -.152*23.5 = -3.6
sqrt(AST%*TRB%): 0.691*sqrt(9.0*17.6) = +8.7

So the total of these rebounding terms for Anthony Davis, despite his mediocre assist percentage, is +6.8.
.

Following this example, let us compare Moses Malone's 1983 (MVP) season with Charles Barkley in 1987. Here's how they look:
http://bkref.com/tiny/IUYtV
They have identical PER (25.1); Moses with a big edge according to WS/48 -- .248 vs .210
Some other strong similarities:

Code: Select all

player  year   ORb%   DRb%   TRb%   Ast%    ORb    DRb  Reb*Ast   T    obpm  dbpm   BPM
Moses   1983   16.8   25.9   21.6    5.1    2.3   -3.9    7.3    5.6    2.6   0.8   3.4
Barkley 1987   16.7   24.8   20.8   19.0    2.3   -3.8   13.7   12.3    6.2   2.6   8.9

Charles leads Moses by 6.7 in the Reb/Ast terms, and by 5.5 in the overall BPM. Was Moses better by 1.2 in other areas?

The 1983 leader board was dominated by Moses, before these latest stats:

Code: Select all

Win Shares        BPM             VORP
Moses    15.1    Bird     7.1    Bird     6.8
Bird     14.0    Magic    7.0    Magic    6.6
...              Erving   5.9    Moncrief 5.0
.                Moncrief 5.3    English  4.8
WS/48           Marques J 4.6    Erving   4.8
Moses    .248    English  4.4   Marques J 4.7
Moncrief .233    Nance    4.3    Nance    4.6
...              Ruland   4.1    Ruland   4.4
.                B Jones  4.1    Moses    4.0
PER              Parish   3.6
Moses    25.1    Moses    3.4
English  24.1
Bird     24.1

http://www.basketball-reference.com/lea ... aders.html

Moses hit 4.1 and 4.0 BPM in earlier seasons.
Brad Miller also had 2 years at 4.0
Vlade Divac had 4.5, 4.8, and 4.9

Wow, that BPM for Moses Malone in '83 seems shockingly low. Ranked 13th in the league that year?

I Moses had a 21.44 WAR in '83, Bird was 20.18, then Magic at 16.40. My all NBA team that year would be Moses, Bird, Magic, Dr. J, & Moncreif. BPM would agree except the MASSIVE caveat of Moses (replaced my Marques Johnson - or Jeff Ruland!? if you had to have a center).

Jeff Ruland? BPM has Jeff Ruland that year at 4.1 BPM, Moses at 3.4. Moses played on the much better team, scored much more (slightly worse TS%), O & D rebounded much better, better steal rate, over twice the block rate, much lower foul rate, & turned the ball over less. So, Moses having less than half the assist rate than Ruland (1.3 apg to 3.0 apg) caused Ruland to leap frog Moses, & by that much? There's something amiss here.

My work had '83 Malone with a slightly better WAR/48 than '87 Sir Charles (29.1 to 28.14), but a slightly lower HnI (144 to 146).

All my numbers can be found here, don't want people to think I'm pulling numbers out of mid air: http://hoopsnerd.com/?page_id=460

[Statman]

I still believe I have the best box score metrics - but I am biased. I would put mine up against anybody's fwiw.

The proof is in the pudding. Send me your data (preferably going back to the merger, or at least the turnover era), and I'll see how well it predicts out of sample. My email is neil.paine@fivethirtyeight.com.

Email sent Neil.

[DSMok1]

Regarding Moses Malone....

I love corner cases.

Because of his unique combination of extremely low assist rate and extremely high rebounding rate, it is likely that the BPM regression doesn't handle Moses very well. The interaction terms in BPM both involve AST%, so someone who has good stats everywhere except AST% will be perhaps excessively hurt in BPM. Rebounding in particular interacts strongly with assists.

Here are a list of seasons with high rebounds and low assists: http://bkref.com/tiny/iewIX

Moses dominates it, with some cameos from Dennis Rodman and Larry Smith (though not Rodman's peak seasons).

The regression really didn't like players with low assist percentages. Can't say much else than that.

An interesting case study would be to evaluate how Houston and Philly did with and without him when he went to Philly after the 1982 season.

SRS:
HOU 1982 -0.4
HOU 1983 -11.1

PHI 1982 +5.7
PHI 1983 +7.5

Malone was the only major addition to Philly; Houston lost a lot of players. Hard to say much from that.

In any case, I suspect BPM is under-representing Malone's contributions due to his outlier status.

[Mike G]

Is it possible to get almost as good a fit without letting the sqrt(Ast*Reb) term get so large that the DReb term becomes a negative?

Another 'corner case' I just noticed is a guy with 33.1 Ast% and 3.0 TRb% -- AJ Price of the Pacers -- in 57 minutes, he has a PER of 20.6 and .164 WS/48. His BPM is minus-2.1 , i.e., replacement level.

After the peanut gallery has had their small sample size fit, we notice others with more minutes in this corner: Nate Archibald (career BPM = -0.3), Calvin Murphy 0.4, from the Hall of Fame; others in all-Star seasons: KJ 0.9, Gus Williams .

[schtevie]

I am a bit surprised that no one yet has mentioned (what I think is) a conspicuous, non-corner case: Shaq. The 14 year, "gold standard" RAPM hates him (relatively speaking), and BPM finds itself in agreement with its judgement. Very strange. Raise your hand if you believe in a hypothetical draft of all the exhibited leaders that Shaq would slip to 11th or 12th.

I say very strange, less because of what I construe to be a disconnect with informed, conventional wisdom, but because Jeremias' xRAPM thinks very highly of him indeed. In fact, Shaq's possession-weighted xRAPM average, controlling for age, shows, I believe, no one better. And I don't think any admitted shortcomings of BPM on the defensive side can fully explain away this "problem".

Maybe it's just a "random" outlying result, but I'm wondering if it's a question of specification.

[colts18]

I am a bit surprised that no one yet has mentioned (what I think is) a conspicuous, non-corner case: Shaq. The 14 year, "gold standard" RAPM hates him (relatively speaking), and BPM finds itself in agreement with its judgement. Very strange. Raise your hand if you believe in a hypothetical draft of all the exhibited leaders that Shaq would slip to 11th or 12th.

I say very strange, less because of what I construe to be a disconnect with informed, conventional wisdom, but because Jeremias' xRAPM thinks very highly of him indeed. In fact, Shaq's possession-weighted xRAPM average, controlling for age, shows, I believe, no one better. And I don't think any admitted shortcomings of BPM on the defensive side can fully explain away this "problem".

Maybe it's just a "random" outlying result, but I'm wondering if it's a question of specification.

You are looking at 14 year RAPM wrong. It only hates Shaq because he played a lot of years well past his usefulness. If you looked at his prime, RAPM loved him. Here is his ranks in non prior informed RAPM from 2001-2006:

01: 6.5 RAPM, #2 in the league
02: 7.1, #1
03: 5.1, #3
04: 6.9, #1
05: 4.7, #3
06: 4.2, #5

Top 3 for 5 straight years

[schtevie]

I am a bit surprised that no one yet has mentioned (what I think is) a conspicuous, non-corner case: Shaq. The 14 year, "gold standard" RAPM hates him (relatively speaking), and BPM finds itself in agreement with its judgement. Very strange. Raise your hand if you believe in a hypothetical draft of all the exhibited leaders that Shaq would slip to 11th or 12th.

I say very strange, less because of what I construe to be a disconnect with informed, conventional wisdom, but because Jeremias' xRAPM thinks very highly of him indeed. In fact, Shaq's possession-weighted xRAPM average, controlling for age, shows, I believe, no one better. And I don't think any admitted shortcomings of BPM on the defensive side can fully explain away this "problem".

Maybe it's just a "random" outlying result, but I'm wondering if it's a question of specification.

You are looking at 14 year RAPM wrong. It only hates Shaq because he played a lot of years well past his usefulness. If you looked at his prime, RAPM loved him. Here is his ranks in non prior informed RAPM from 2001-2006:

01: 6.5 RAPM, #2 in the league
02: 7.1, #1
03: 5.1, #3
04: 6.9, #1
05: 4.7, #3
06: 4.2, #5

Top 3 for 5 straight years

I stand by my comments. For the same years you list, xRAPM loves him too, in terms of ranking. My point was that, overall, weighting by possessions and comparing identical ages, he sits at the top of table in terms of xRAPM, better than Tim Duncan and Kevin Garnett (ages 28 to 37, I believe) and LBJ (just two years comparison, 28 and 29). Are there other players to add to the comparison?

Perhaps I have made a computational error. But this is the discrepancy I was referring to.

[Statman]

Is it possible to get almost as good a fit without letting the sqrt(Ast*Reb) term get so large that the DReb term becomes a negative?

Another 'corner case' I just noticed is a guy with 33.1 Ast% and 3.0 TRb% -- AJ Price of the Pacers -- in 57 minutes, he has a PER of 20.6 and .164 WS/48. His BPM is minus-2.1 , i.e., replacement level.

After the peanut gallery has had their small sample size fit, we notice others with more minutes in this corner: Nate Archibald (career BPM = -0.3), Calvin Murphy 0.4, from the Hall of Fame; others in all-Star seasons: KJ 0.9, Gus Williams .

Which is why I suggested it's WAY too early to crown a a newish metric as "the best" of anything (in this case it was deemed the best box score metric). As Kip would say "Napoleon, like anyone could even know that."

The idea that a positive stat could be somehow turned into a negative irks me, if that's the case - I don't care what the regression says.

[DSMok1]

I am a bit surprised that no one yet has mentioned (what I think is) a conspicuous, non-corner case: Shaq. The 14 year, "gold standard" RAPM hates him (relatively speaking), and BPM finds itself in agreement with its judgement. Very strange. Raise your hand if you believe in a hypothetical draft of all the exhibited leaders that Shaq would slip to 11th or 12th.

I say very strange, less because of what I construe to be a disconnect with informed, conventional wisdom, but because Jeremias' xRAPM thinks very highly of him indeed. In fact, Shaq's possession-weighted xRAPM average, controlling for age, shows, I believe, no one better. And I don't think any admitted shortcomings of BPM on the defensive side can fully explain away this "problem".

Maybe it's just a "random" outlying result, but I'm wondering if it's a question of specification.

You are looking at 14 year RAPM wrong. It only hates Shaq because he played a lot of years well past his usefulness. If you looked at his prime, RAPM loved him. Here is his ranks in non prior informed RAPM from 2001-2006:

01: 6.5 RAPM, #2 in the league
02: 7.1, #1
03: 5.1, #3
04: 6.9, #1
05: 4.7, #3
06: 4.2, #5

Top 3 for 5 straight years

I'm not entirely sure what you're concerned about, Schtevie, in regards to Shaq. He has very good BPM numbers.

Here are the top 10 in BPM in each season: http://www.basketball-reference.com/lea ... op_10.html

Shaq was perennially there in the top 10, had 1 #1 season, and 2 #2 in the league seasons. In terms of best season peak, he was right up there with the best. In terms of average BPM over his best 5 seasons, he sits tied for 11th, tied with Karl Malone. That might be a bit low, but it's hard to say for sure. Certainly not as questionable a rating as Moses Malone's is. Maybe Shaq was a hair overrated? I can't say for sure.

[Mike G]

For years up in here, we concurred that if a metric didn't rank Shaq at #1, it didn't pass the laugh test.
Here's another correlation test, suspecting that his BPM largely rises and falls with his Ast% ...
his annual correlations with BPM:

Code: Select all

BPM    1.00        Blk%    .52
WS/48   .94        ORb%    .49
PER     .91        TRb%    .33
mpg     .84        DRb%    .19
-TO%    .81        3PAr    .17
AstReb  .73        Stl%    .17
Ast%    .71        FTr    -.05
Usg%    .65        TS%    -.17

AstReb = sqrt(Ast%*TRb%)

Anything insightful here? Don't know, but it doesn't address his ranking among contemporaries.
Year by year rankings:

Code: Select all

yr   PER  WS/48  BPM
93    7    19    18
94    2     2     5 
95    2     2     8
96    3    24    25
97    3    13     8
98    1     4    16
99    1     2     3
00    1     1     1
01    1     1     2
02    2     4     2
03    2     6     7
04    4    12     8   
05    3     6     7
06    9    28    40

Shaq was all-NBA 1st team for 8 years, 1998-2006. His career PER trails only Jordan and (for now) LeBron.

[v-zero]

What I am curious about is this factor of 1.2... From what I understand if you want to explain/predict the performance of a given lineup you add the ratings together and then divide by 1.2 - but if that is the case then the factor of 1.2 is meaningless. So, either I am missing something (very possible), or the factor is a pointless distraction?

[DSMok1]

What I am curious about is this factor of 1.2... From what I understand if you want to explain/predict the performance of a given lineup you add the ratings together and then divide by 1.2 - but if that is the case then the factor of 1.2 is meaningless. So, either I am missing something (very possible), or the factor is a pointless distraction?

Try it out. If a lineup's BPM sums to 0, then the expected performance is 0. But if it sums to +10, then it's expected performance is only +8.333 actually.

The 1.2 translates from the team context (where winning teams perform worse than if they were in a close game, and vise versa) to a neutral context.

Actually, RPM and the 14 year RAPM has the exact same scale and setup, with effectively the same 1.2 factor (it's the "playing with the lead" adjustment).

[schtevie]

Daniel, let me begin by restating what I have restated before (because it must get annoying, receiving what appear to be niggling criticisms and complaints): I am extraordinarily grateful for the publicly-available work that you (and Jeremias) produce. What a poorer place this would be without it.

As for my "concern" about Shaq. Well, it's got nothing to do with him, per se, beyond the fact, as stated, that I am quite "sure" that he is incredibly underrated. (If there's anyone here who believes that he's 11.5th best, please speak.)

I don't like that there is such a discrepancy between xRAPM (which makes intuitive sense to me: prior with yearly update of +/- performance) and the coincidental (?) common view of 14 year average RAPM and BPM. In the former case, is it on account of aging curve factors? Or is it something else? But that doesn't solve the BPM discrepancy. Even if you throw him a gift point for defense, the rating is still "wrong". His beastiness on the offensive end simply doesn't show up. Might it be that he is disadvantaged for demanding double/triple teams, such that his actual stats simply cannot tell the story?

But my concern, if it can be called that, is more general. It sure would be nice to converge upon a rating system that eliminates conspicuous and "controversial" outliers. And here, a couple of thoughts:

(1) I think I'm barely overstating things when I say that no one ultimately assigns particular value to a rating system whose claim to superiority is based upon best sorting out the rankings of, say, players 193 through 245. Not that this is unimportant, but what matters to almost everybody is the ability of a system to accurately rank the best of the best.

And I don't know if there's a practical, positive suggestion in this observation, but perhaps giving extra weight to the performance of the best players might "improve" things.

Also, though perhaps it isn't a scientific way to proceed, but it would be interesting to know the extent to which model selection shakes up the rating/ranking of the best of the best. Perhaps "stability/consensus" of this subset might be a useful criterion for variable selection?

(2) I fixate on Eli Witus' confirmation of Dan Rosenbaum's identification of distinct (non-zero) positional averages. Do BPM positional averages replicate EW's findings? Might including such factors (alone and/or interacting with others) be, for example, a rational basis for increasing Shaq's (and other center's) DBPM by some measure?

[permaximum]

This metric contest (:P) attracted my attention too... I will try to come up with a metric in a couple days when I have some free time. I'm thinking of a pure box score metric with no relation to p/m, apm, rapm or rpm.

@Neil Paine

Could you wait for me too please

[DSMok1]

I don't like that there is such a discrepancy between xRAPM (which makes intuitive sense to me: prior with yearly update of +/- performance) and the coincidental (?) common view of 14 year average RAPM and BPM. In the former case, is it on account of aging curve factors? Or is it something else? But that doesn't solve the BPM discrepancy. Even if you throw him a gift point for defense, the rating is still "wrong". His beastiness on the offensive end simply doesn't show up. Might it be that he is disadvantaged for demanding double/triple teams, such that his actual stats simply cannot tell the story?

But my concern, if it can be called that, is more general. It sure would be nice to converge upon a rating system that eliminates conspicuous and "controversial" outliers. And here, a couple of thoughts:

xRAPM includes both box score and APM components and should be the best overall metric, though I'm not completely sold on how good the box score component is (and it includes height, which I prefer not to). BPM will coincide with the 14 year RAPM because its box score weights were chosen to best mirror the 14 year RAPM data.

(1) I think I'm barely overstating things when I say that no one ultimately assigns particular value to a rating system whose claim to superiority is based upon best sorting out the rankings of, say, players 193 through 245. Not that this is unimportant, but what matters to almost everybody is the ability of a system to accurately rank the best of the best.

And I don't know if there's a practical, positive suggestion in this observation, but perhaps giving extra weight to the performance of the best players might "improve" things.

Also, though perhaps it isn't a scientific way to proceed, but it would be interesting to know the extent to which model selection shakes up the rating/ranking of the best of the best. Perhaps "stability/consensus" of this subset might be a useful criterion for variable selection?

The best of the best are by definition outliers in many regards, and for all of these box score-based models, they will be a point of contention. Due to their outlier stats, any differences in how metrics value those stats will be magnified.

I weighted my regression purely based on the square root of possessions played, with players with small samples excluded.

Is there a better way to weight the regression? I'm not sure. I'm fairly sure the way I did it is the mathematically correct approach, and will minimize overall error.

How do I judge a metric? Basically, through Neil Paine's approach: how well does it validate on lineups out of sample. In other words, I don't care about the elite players more than the rest.

(2) I fixate on Eli Witus' confirmation of Dan Rosenbaum's identification of distinct (non-zero) positional averages. Do BPM positional averages replicate EW's findings? Might including such factors (alone and/or interacting with others) be, for example, a rational basis for increasing Shaq's (and other center's) DBPM by some measure?

Positional averages depend on the metric. The 14 year RAPM, and by extension my BPM, both show positional averages near 0 for all positions. Obviously, positional averages on offense and defense are highly skewed by position, but in general the overall sum is approximately 0 for all positions.

--

As an aside, here are the "outliers" in BPM: the best 100 seasons by VORP, and how they correlate with different components:

Code: Select all

╔════════╦══════╦════════╦══════╗
║        ║ BPM  ║ WS/48  ║ PER  ║
╠════════╬══════╬════════╬══════╣
║ WS/48  ║ 68%  ║        ║      ║
║ PER    ║ 71%  ║ 87%    ║      ║
║ TS%    ║ 37%  ║ 51%    ║ 40%  ║
║ ORB%   ║ -14% ║ -15%   ║ -19% ║
║ DRB%   ║ -8%  ║ 4%     ║ 8%   ║
║ TRB%   ║ -12% ║ -2%    ║ -1%  ║
║ AST%   ║ 36%  ║ 13%    ║ 14%  ║
║ STL%   ║ 27%  ║ -1%    ║ -1%  ║
║ BLK%   ║ -4%  ║ 8%     ║ 16%  ║
║ TOV%   ║ -10% ║ -34%   ║ -40% ║
║ USG%   ║ 36%  ║ 47%    ║ 72%  ║
╚════════╩══════╩════════╩══════╝

http://bkref.com/tiny/Z3Lo8

Some notes: Win Shares and PER are dominated by Usage, TS%, and TOV%, though differing amounts for each. BPM values assists and steals a lot more, and turnovers and scoring relatively less. None of the stats correlate with rebounding much.