The debut and popularization of BPM

[Neil Paine]

Neil, your earlier study gave those with less than 250 minutes, mean performance rankings??? Not replacement level? Why? Given how many players score below to way way below replacement level on 14yr rapm, I wonder how replacement level isn't way lower and can't imagine the rationale / justification for giving low minute guys mean level performance rankings (essentially making them top 30% guys). How much could the correlations been in last test if the low minute guys were rated as replacement level or worse? Would the relative performance rankings have changed any? If the low minute guys were rated way worse wouldn't most or all of the very god players rate somewhat better? What are you going to do this time?

The rationale is that league-average is known/well-defined for each metric... i.e., .100 for WS48, 15 for PER, 0 for BPM, etc. Meanwhile, where to set the replacement level isn't very clear even for metrics whose creators agree with a replacement-level valuation system, much less for metrics where the creators resist the idea of replacement-level analysis. Also, assigning low-minute players average ratings gives less potential for claims that I rigged the study. And besides, it shouldn't impact the results because every metric is being treated the same way.

[Crow]

Treated the same way isn't for sure the same as having the same affects. Not sure how giving them avg performance marks which flies in face of all conventional wisdom lessens suspicion of rig. I think the suspicious would have more suspicion with the avg performance ratings. I.e. those metric authors who highly reward usage, shooting quality, rebounding, assists, defense, all things most low minute player lack yet fail to get marked down "appropriately" for, artificially ranking lots of players at the mean instead un the fat long tail where Tom Tango and Daniel tell me they are on true talent and every metric says they are on performance. I would think that the metrics most biased about the value of certain stats would be affected by this the most in terms of gross change, not sure about metric performance. But how can using a cutoff for low minute guys and an assumption of mean performance be neutral when a few metrics do the former while I assume some to many don't? Can't be sure without trying I don't think. I am real surprised about this, but not an expert.

[schtevie]

GMs actually should care quite alot about the true value of guys 70- 250 as much of their choice making involves them and the top 70 mostly either do what they want or are just not available.

I didn't say or imply otherwise. They are 30 of the not very many folks who are into accurate team projections for fun and profit.

But let's be a bit clearer, what separates the 180 players between 70th and 250th last year, according to BPM are 3 points, an average spacing of 0.017. Given this, BPL is but one (albeit essential) piece of evidence providing context for valuation.

By contrast, from a GM perspective, it is absolutely necessary to get the best possible valuations of the very best players. Yes, such players are not readily available on the market, but each additional point of production on provides is strictly surplus, given the strictures of the salary cap.

[Crow]

Yes you didn't say otherwise. Just call it me making explicit a group that should care. And we agree on that, so that is cleared up, even if it didn't "need to be".

[Neil Paine]

Treated the same way isn't for sure the same as having the same affects. Not sure how giving them avg performance marks which flies in face of all conventional wisdom lessens suspicion of rig. I think the suspicious would have more suspicion with the avg performance ratings. I.e. those metric authors who highly reward usage, shooting quality, rebounding, assists, defense, all things most low minute player lack yet fail to get marked down "appropriately" for, artificially ranking lots of players at the mean instead un the fat long tail where Tom Tango and Daniel tell me they are on true talent and every metric says they are on performance. I would think that the metrics most biased about the value of certain stats would be affected by this the most in terms of gross change, not sure about metric performance. But how can using a cutoff for low minute guys and an assumption of mean performance be neutral when a few metrics do the former while I assume some to many don't? Can't be sure without trying I don't think. I am real surprised about this, but not an expert.

The heavy regression to average for low-minute players isn't unheard of. In fact, it's the underlying basis of the Simple Projection System, which itself is based on Tango's Marcel projections. So there's absolutely precedent for regressing toward average in situations where the alternative isn't clear.

Now, if the metric makers are OK with me going through the same process Daniel, Nate, etc. went through to determine the replacement level for each metric, I can do that. But I also recognize that some of the metric makers don't actually even acknowledge the existence of a replacement level.

[DSMok1]

Treated the same way isn't for sure the same as having the same affects. Not sure how giving them avg performance marks which flies in face of all conventional wisdom lessens suspicion of rig. I think the suspicious would have more suspicion with the avg performance ratings. I.e. those metric authors who highly reward usage, shooting quality, rebounding, assists, defense, all things most low minute player lack yet fail to get marked down "appropriately" for, artificially ranking lots of players at the mean instead un the fat long tail where Tom Tango and Daniel tell me they are on true talent and every metric says they are on performance. I would think that the metrics most biased about the value of certain stats would be affected by this the most in terms of gross change, not sure about metric performance. But how can using a cutoff for low minute guys and an assumption of mean performance be neutral when a few metrics do the former while I assume some to many don't? Can't be sure without trying I don't think. I am real surprised about this, but not an expert.

The heavy regression to average for low-minute players isn't unheard of. In fact, it's the underlying basis of the Simple Projection System, which itself is based on Tango's Marcel projections. So there's absolutely precedent for regressing toward average in situations where the alternative isn't clear.

Now, if the metric makers are OK with me going through the same process Daniel, Nate, etc. went through to determine the replacement level for each metric, I can do that. But I also recognize that some of the metric makers don't actually even acknowledge the existence of a replacement level.

Could you derive a best fit value for each metric? Choose what value works best for low minutes players for each just by checking fit on the outcome?

I still agree with using average, though, since it ensures a level playing field.

[Statman]

I would suggest he do same season also, to make sure the metric is actually, at the VERY least, compiling correctly at the team level for the season it's supposed to be measuring. ... All the metrics should be pretty darn close to 1.00, or what's the point?

Will he maybe going into the future, using previous ratings (ie using 2002 to predict 03, 04, etc correlations)? It seems that would take out bias any metric that regresses previous seasons would have in this specific test...

Some stats make a 'team adjustment' that forces the stat to precisely equal team performance. Of course, this correlation means nothing by itself: You could just make everyone 1.00, then adjust to team strength. But this wouldn't perform well in subsequent seasons, as players move from team to team.

Agreed, same season correlation near 1.00 by no means indicates a good metric. Heck, one could easily just take each players points per minute - divide by average opponent points per minute - and BAM, a rating that correlates great at the team level same season. Ignores everything but scoring proficiency relative to average opponent scoring proficiency. Actually, would be a good test correlation now that I think of it - if a metric can't beat this simple rating in Neil's testing - seriously back to the drawing board.

I'm just saying - at the VERY least - a metric needs to correlate really well same season. Some known metrics do not I believe.

[Mike G]

May as well name names. PER correlates to offense and not to defense. WS/48 correlates to both, but is one which says basically all players on good teams are better than when when they were with bad teams, with few exceptions.

My own studies have involved the magnitude of change as measured by these stats (along with eWins, of course) for players from year to year. WS/48 changes almost twice as much as eW/48.

This seems like it may be effectively the same test Neil has done. If player ratings from yr1 are applied to yr3, how well their yr1 rates describe the team's yr3 performance? If a metric says the players all had to get a lot better or worse, it's less likely to be believable.

Throwing out the role players -- or assigning them all an equal rating -- is perhaps not all bad. After all, as role players, their productions and efficiencies may be expected to change as their role may change. But lots of the biggest discrepancies I've seen between the metrics is found among these players. Maybe these players really are the telltale difference makers when judging among metrics. Sink or swim.

[Mike G]

Last year in the 'Predictions' thread, I made the case that we all underestimated the Phx Suns, and the reason was that we couldn't have supposed that everyone on the team was going to improve.
Would other metrics have said they were really that good, based on 2012-13 data?

Code: Select all

. Suns           2013  2014     - e484 -        - PER -       - ws/48 -       - BPM -
.  2014           tm    Min    2013   2014    2013   2014    2013   2014    2013  2014

Dragic,Goran     Phx   2671    1.14   1.63    17.5   21.4    .106   .186     1.2   3.6
Tucker,P.J.      Phx   2489     .52    .73    11.6   13.3    .074   .117     -.4   2.3
Green,Gerald     Ind   2327     .54   1.13     9.9   16.5    .056   .126    -3.1   1.0
Morris,Markieff  Phx   2151     .77   1.25    12.6   18.4    .044   .143    - .9    .9

Plumlee,Miles    Ind   1963     .56    .96     8.8   14.6   -.015   .098    -6.6  -1.4
Morris,Marcus Hou/Phx  1800     .65    .82    11.3   14.8    .058   .111    -1.7    .2
Bledsoe,Eric     LAC   1416    1.17   1.55    17.5   19.6    .115   .140     2.7   2.8
Smith,Ishmael    Mil   1011     .20    .54     8.5   11.7   -.040   .048    -5.8  -2.1

Barbosa,Leandro  Bos    368     .74    .54    13.5   11.5    .086   .058    -2.8  -3.7
Randolph,Shavlik Bos     96    1.17    .12    15.9    7.7    .159   .060    -1.4  -4.8
Kravtsov,Viaches Det     62     .64    .28    14.4   10.5    .086   .027    - .9  -6.2

We all agree their top 8 guys improved, and substantially; that they had just 2 above-avg players going into the season -- and then there are differences.

In eWins, I consider .00 to be 'replacement level', btw. Assume that is also the case with WS/48 ?

Also, semi-related: In baseball, there are about 27 'possessions' per game, per team: A hit or walk is = OReb. That's barely a quarter of a basketball game. Perhaps if an NBA game were 15 minutes, a team of 'replacement' players would win 1/5 of their games?

[DSMok1]

Last year in the 'Predictions' thread, I made the case that we all underestimated the Phx Suns, and the reason was that we couldn't have supposed that everyone on the team was going to improve.
Would other metrics have said they were really that good, based on 2012-13 data?

We all agree their top 8 guys improved, and substantially; that they had just 2 above-avg players going into the season -- and then there are differences.

In eWins, I consider .00 to be 'replacement level', btw. Assume that is also the case with WS/48 ?

Also, semi-related: In baseball, there are about 27 'possessions' per game, per team: A hit or walk is = OReb. That's barely a quarter of a basketball game. Perhaps if an NBA game were 15 minutes, a team of 'replacement' players would win 1/5 of their games?

The Phoenix Suns were certainly an outlier, even a historical outlier.

WS replacement level is not 0, nor even close to 0. It's about 0.05 WS/48. I derived it a while back.

[Mike G]

WS replacement level is not 0, nor even close to 0. It's about 0.05 WS/48. I derived it a while back.

So the Sixers (currently 0-11, losing by 16.4 ppg) have not quite 20% of their minutes from above-replacement players.
The other 80% are from players < .025 WS/48, including their top gun Wroten.
League leading scorer Kobe Bryant is at .034 . Borderline replaceable?

Last year, there were 360 players > .050 ws/48 -- is this 12 per team the derivation?
About .163 of minutes were from players less than this.

[DSMok1]

WS replacement level is not 0, nor even close to 0. It's about 0.05 WS/48. I derived it a while back.

So the Sixers (currently 0-11, losing by 16.4 ppg) have not quite 20% of their minutes from above-replacement players.
The other 80% are from players < .025 WS/48, including their top gun Wroten.
League leading scorer Kobe Bryant is at .034 . Borderline replaceable?

Last year, there were 360 players > .050 ws/48 -- is this 12 per team the derivation?
About .163 of minutes were from players less than this.

It was derived as the average production level of minimum salary players who played few minutes. Same method on BPM produces about -2.0 for it.

Win Shares notoriously doesn't value high usage. Both Wroten and Kobe are above average, let alone replacement level, by BPM.

[Mike G]

It was derived as the average production level of minimum salary players who played few minutes. Same method on BPM produces about -2.0 for it..

That's what I don't get. When I look at last year's players who got < $1 mill and played < 1000 minutes, their avg BPM is -9.6.
Weighted by their minutes, it's -2.5
Under 100 min., it's -5.5

I get these correlations to salary:

Code: Select all

.556    e484
.547    eW
.525    mpg
.480    PER
.457    WS
.419    vorp2
.409    VORP
.367    MP
.329    BPM
.299    WS/48

vorp2 is VORP with R = -3.035
Productivity may be better rewarded than 'efficiency' as measured by some stats.

[DSMok1]

The full derivation of replacement level was done in this thread: http://tangotiger.com/index.php/site/co ... nba-player and in this thread: http://www.apbr.org/metrics/viewtopic.php?f=2&t=8635

I actually showed that WS/48 replacement level may be higher, closer to 0.07 even.

[permaximum]

The last time Neil Paine did the test, it looks he used ratings from Season-1(previous season), S-2, S-3. Are those players with no ratings in those years get the "average player" treatment or "rookie average treatment"? More importantly what's the right approach? Average for veterans and rookie average for players that in the first few seasons of their career?