Estimating Replacement Level: Sabermetrics perspective

[DSMok1]

I've found that replacement levels using this 5 year minimum salary dataset are higher for bigs than for guards, but the averages for bigs and guards are both still 0. Not sure what that's about.

Also, replacement level on O and D varies widely by position.

This is a different data set chosen because it aligned with the estimated replacement levels from the minimum salary grouping, but includes far more seasons. For illustration of the offense/defense divide.

[Dr Positivity]

Here is my process for estimating replacement level is 0.4-0.5 Ws

First if we assume veteran minimum as the replacement level, that's around 789k

MLE is supposed to be average salary, so using next year's 5.3 number, that's 6.7x bigger than veteran minimum

If in a hypothetical world everyone had 15 NBA players, with 41 as average wins for a team, dividing 41 by 15 = 2.733 wins for an average player. Since MLE is supposed to be 6.7x bigger than vet minimum, 2.733 / 6.7 = 0.41.

However there's an issue with this in that not every team carries 15 players (eg D League assignements) and I'm not sure how to treat players who never ever play. So if the "average" roster is lower than 15, average wins would be pushed up nearer to 0.5 in this case.

A practical way to show this is also to multiply 0.5 by 14 or 15 roster spots, which adds up to 7-7.5 wins. This is probably as a bad as a team could be if you had an entire roster full of replacement players in point differential/Pythagorean.

[Mike G]

On that hypothetical team of replacement players, perhaps 15 guys would all play about 1/15 of the minutes. But on a better (any) team, wouldn't they typically play less than that?

In any case, b-r.com shows 348 players last year with at least .4 Win Shares, and 338 with at least .5 -- that's 11-12 per team.
With 482 players listed, we could count (482-338)/482 = 30% of players who appeared last year as 'replacement level', or nearly so.

But I'm uneasy about ranking players by their Win (Shares, etc) total. There's also the issue of the minutes they get.
It turns out that at the per minute level, player #330 (11 per team) has WS/48 = .050
Player #360 (12 x 30) is .037
There were 420 players -- 14 per team -- with WS/48 > .000

Of 145 players with < .50 WS, 29 -- 20% of them -- had WS/48 > .050

[DSMok1]

Cross posting from the Tangotiger thread:

An update to the table above, using 5 years of minimum salary players to establish replacement levels. This table also includes Win Shares, developed by Justin Kubatko, probably the most widely used stat for historical players. Replacement level is well above “0” in this stat:

Code: Select all

╔═══════════╦═══════╦══════════╦═══════╦════════╦════════╦═══════╦════════╦════════╗
║   Type    ║ Count ║ Avg %Min ║ ASPM  ║ O-ASPM ║ D-ASPM ║ WS/48 ║ OWS/48 ║ DWS/48 ║
╠═══════════╬═══════╬══════════╬═══════╬════════╬════════╬═══════╬════════╬════════╣
║ In-Season ║   165 ║ 6.9%     ║ -2.70 ║ -1.94  ║ -0.76  ║ 0.052 ║ 0.013  ║ 0.039  ║
║ Preseason ║   266 ║ 16.0%    ║ -1.95 ║ -1.67  ║ -0.28  ║ 0.071 ║ 0.025  ║ 0.046  ║
║ Total     ║   431 ║ 12.5%    ║ -2.11 ║ -1.73  ║ -0.38  ║ 0.067 ║ 0.023  ║ 0.045  ║
╚═══════════╩═══════╩══════════╩═══════╩════════╩════════╩═══════╩════════╩════════╝ 

Again, replacement level should probably be about PRESEASON level, not in season (due to talent drain to Europe and other reasons).

I will probably end up using -2, -1.7, -0.3 for ASPM’s replacement level.

[mystic]

Again, replacement level should probably be about PRESEASON level, not in season (due to talent drain to Europe and other reasons).

I don't see that the "talent drain" to Europe is an issue at all for the NBA in regard to the replacement level talked about, because usually besides the better players they are even below that preseason replacement level. When you look at the better players (mostly non-US players) they usually are signing bigger contracts in Europe, which also means they wouldn't be available for the minimum anyway. Also, the preseason level of those minimum players seems to be effected by the fact that some better players are signing with contending teams or teams offering much more playing time for those players. Such players can't be considered "replacement", because they are in fact not available to all teams anyway. In contrast, the in-season signings seems to be much more in agreement with what I would consider "replacement", because they seem to be available much more often for all teams. Thus, I do not agree with your conclusion unless I see evidence for your assumption, that better players signing in Europe would also sign min contracts on basically all NBA teams. Just to understand that better: Players like Fernandez or Rodriguez would never sign another min contract in the NBA, because they get more money in Europe anyway, thus those two players can't be considered "replacement". The same goes for other better players, especially non-US players).

The "talent drain to Europe", meaning "losing better players to the European leagues" is mostly a result of the CBA not allowing to pay them adequately. Such players would not be "replacement level", but would likely raise the "replacement level" by just being available for NBA teams even though they would need to pay more money for them than just the minimum. Under the current CBA (and the previous one to add), it seems rather reasonable to set the replacement level at -2.7 rather than -2.0, because those -2.7 players are those really available for teams under the current salary restrictions. The other players are not, thus we can't count them as possible "replacements".

The issue for me in regard to "replacement level" comes from the fact that it is still not clear to me what the term really means. In this post I took the term "replacement level" as something describing the average value per 100 poss of always available players to basically all NBA teams. But "replacement level" could also be defined by the playing level lost due to injured better players. In the latter case the "replacement level" would obviously be higher, because the role of the star player is not taken by such "D-League" player, but rather by the next player in-line on the roster. The minute distribution changes is a fashion that such a "replacement player from the D-League" will likely only see some 5 mpg while the majority of the missed minutes will be taken by the backup players. And even then we see shifts in regard to different 5-player-units (based on a basic-scheme of 3 different player types - lead guard, wing and pivots), where players get shifted around position-wise and are not entirely attached to the specific 5 positions (PG, SG, SF, PF, C), which is - just as another thought - one of the main reasons that I find adjusting for such 5 positions to be a big mistake.

[talkingpractice]

well... the results for the RAPM part of this exercise only add more clarity to the "replacement level is higher than we thought" argument.

first, the methodology:

1, we used 6 years of data (2009-2014), and made a reference group out of all of the playerseasons of minimum-salary players.
2, we used the 'whole coef' version of rapm (not split into off/def), as that was easier for various reasons, and b/c we thought it got more directly at the problem that we were looking to solve.
3, we used the pre-season guys only, for reasons that Daniel gave earlier. Kevin's list was already split that way, and we used the list that Colts provided for 2014 and split them ourselves.

for purposes of face validity, here are the top and bottom guys in that 6 year sample, and their corresponding rapm values:

top ->
LeBron James 8.65438964
Chris Paul 6.222313373
Dirk Nowitzki 6.088383086
Manu Ginobili 5.418286918
Kevin Garnett 5.363018869

bottom ->
Hakim Warrick -4.043721089
Earl Clark -4.231180991
JJ Hickson -4.543859522
Jonny Flynn -4.552859245
Byron Mullens -5.261796182

and....... the result for the reference group specified in this way was -1.78.

for purposes of full disclosure, we've been doing something similar for a couple of seasons (using a multi-year reference group comprised of the low mp players, with some playerseasons pulled out for qualitative reasons, for example kobe last year), and we have been using -2.2 for this value.

after this exercise, and with the other data/ideas we've seen via this thread, we're probably going to go the robust route and use -2 now.

[J.E.]

How many total possessions did those replacement players play?

It might be a good idea to use a lower lamdba for the replacement player column, as the coefficient might be dragged "too far" towards zero when there's not a lot of data

[mystic]

using a multi-year reference group comprised of the low mp players

What would be a "low mp player" in your approach? What kind of threshold is used to determine that? Also, if you apply such a threshold to the group of reference players for those minimum contracts, how many of those would fall into that? Wouldn't it be also better to exclude players who are not considered "low mp players" from that experiment? Also, why do you consider the Daniel arguments to be valid in terms of determining "replacement level"? Is that a question of the definition of the term "replacement level"? Also, you wrote that you used -2.2 previously. How did you came up with that number in the first place? I assume you ran the regression with all players while then determined the average for those "low mp players" would be close to that value?

[mystic]

It might be a good idea to use a lower lamdba for the replacement player column, as the coefficient might be dragged "too far" towards zero when there's not a lot of data

Did you make experiments by using a variable lambda based on possessions/minutes besides the suggestion to use a lower lambda for "replacement players"? Did such "variable" lambda gave a better prediction?

[J.E.]

It might be a good idea to use a lower lamdba for the replacement player column, as the coefficient might be dragged "too far" towards zero when there's not a lot of data

Did you make experiments by using a variable lambda based on possessions/minutes besides the suggestion to use a lower lambda for "replacement players"? Did such "variable" lambda gave a better prediction?

I've done no experiments but, I think, in this case we're really after an unbiased estimate. It's generally not a good idea to use a lamdba different from the one you found through crossvalidation but I think here it's fine to do so when you apply it to just the one column, especially when that column represents many different players

[talkingpractice]

How many total possessions did those replacement players play? It might be a good idea to use a lower lamdba for the replacement player column, as the coefficient might be dragged "too far" towards zero when there's not a lot of data

it was actually like over 200k possessions or something. i was really surprised by how many stints involved a replacement-level player, using this definition.

[talkingpractice]

using a multi-year reference group comprised of the low mp players

What would be a "low mp player" in your approach? What kind of threshold is used to determine that? Also, if you apply such a threshold to the group of reference players for those minimum contracts, how many of those would fall into that? Wouldn't it be also better to exclude players who are not considered "low mp players" from that experiment? Also, why do you consider the Daniel arguments to be valid in terms of determining "replacement level"? Is that a question of the definition of the term "replacement level"? Also, you wrote that you used -2.2 previously. How did you came up with that number in the first place? I assume you ran the regression with all players while then determined the average for those "low mp players" would be close to that value?

our core reason for doing this stuff (investing in the NBA market) is very different than the reasons many others here do this stuff. as such, we take a lot of liberties in terms of doing our internal stuff, as our goal is simply "what is best", as opposed to creating something that is system-based, replicable, etc. so internally, we determine our own definition for lots of these things qualitatively (involving scouting, etc).

for example, it took about 2 games last season to know that james johnson was not just a replacement-level sort of player, or the same thing for pj tucker or pat beverley the season before. so for our purposes, i could just wave a wand and decide that those guys were more like a 0 than a -2, and make it so. and if im wrong, then the market can punish me.

also, we dont really need/use the concept of replacement level that much, anyway (it doesnt really apply). many times a lot of these things are somewhat tangential to what we actually do internally.

we also operate a pretty formal "scouting" system, much more than people probably think. that guides a decent chunk of our internal stuff.

also, tbh, the actual system-based player values that we talk about a lot don't play that huge of a part in what we do, anyway. beating the market at a high level requires a lot more than plugging in system-based player values, expected minutes, some home court and rest day stuff, and multiplying/summing. that can work okayish, but not well enough for what we're trying to do, and we pretty much welcome entrants to the market who do solely that stuff.

so the general answer is that internally, we do a lot of different things lol, with solely one end-goal in mind (beating the market by as much as possible, on a risk-adjusted basis).

[mystic]

I appreciate your answer and I see why you haven't much use for what be considered "replacement level" in such a discussion. I have a similar feeling about that topic and have a tough time (obviously) agreeing with the here presented definition of the term. So, I can see the value, if a player ranking is presented to casual fans, which usually have a lot of trouble understanding the concept of above/below average, which in essence the +/- derivates are based on. Having a value above replacement usually makes such "list" more in agreement with the general feeling about the "goodness" of the players.

Anyway, just to get clarification on this matter: You don't have a fixed threshold for "low mp players"?

Btw, I'm not surprised that you found so many possessions, because for reasons presented before, I would not consider all players signed to a min contract before the start of the season to be "replacement players". Especially the issues of playing time, location of the team as well as expected overall playing level play clearly a role in the decision to sign such minimum contract with a specific team. Other teams, who can not offer a similar enticing combination of those things to the players, are likely not able to sign those players to min contracts, which in my mind would be in disagreement with the logical concept of "replacement".

[Crow]

Have there been guys who have played 3 plus seasons for decent minutes at -3 or worse on rpm and gotten another NBA contract? Closest cases I see immediately: Warrick had 2 before Portland signed him (and two at about -2 before that) and his 3rd worse than -3 last season. Mullens have 5 straight near -3 or worse and he is finally gone.

[bondom343]

Have there been guys who have played 3 plus seasons for decent minutes at -3 or worse on rpm and gotten another NBA contract? Closest cases I see immediately: Warrick had 2 before Portland signed him (and two at about -2 before that) and his 3rd worse than -3 last season. Mullens have 5 straight near -3 or worse and he is finally gone.

Gotbuckets has Darren Collison as a -3 for the last 3 seasons, and he just signed with the Kings. Barea was close as well from what I could find, but was a -2.98 one of the years before he signed his contract.