http://talkingpracticeblog.com/
Chronology of work to date and Individual Player Values ("....we developed a player evaluation metric that has proven in our testing to be more predictive than other plus/minus or statistical plus/minus (SPM) metrics.)
Top 12
Rank Player IPV
.
1 LeBron 8.38
.
2 Chris Paul 7,01
.
3 Kevin Durant 5.71
.
4 Tim Duncan 5.47
.
5 Tyson Chandler 4.95
.
6 Dwight Howard 4.82
.
7 Dwyane Wade 4.69
.
8 Marc Gasol 4.60
.
9 Kevin Garnett 4.57
.
10 Russell Westbrook 4.34
.
11 Kobe Bryant 4.33
.
12 Tony Parker 4.30
New XRAPM has Ginobili, Griffin, Love and Josh Smith in its top 12 instead of Bryant, Parker, Wade and Westbrook on IPV. The other 8 are in both top 12s.
Top values on new metric called "Individual Player Values"
Re: Top values on new metric called "Individual Player Value
I think a large difference may be the construction of the model. I get the distinct feeling that IPV measures a player by placing their stats in an 'average' 5-man lineup and computing the change, whereas (x)RAPM has to deal with the fact (and let's face it it is a fact) that some players play on good teams and so their production need not be as high, whereas some play on bad teams and their production is higher (i.e. production is non-linear).
From a purely personal point of view I can't accept Kobe in there unless it's a very short term rating from earlier in the season, because now that he's bored of his new toys he's gone back to gotta chuck em all, so much so that Howard doesn't even look to see if Kobe is going to pass any more.
From a purely personal point of view I can't accept Kobe in there unless it's a very short term rating from earlier in the season, because now that he's bored of his new toys he's gone back to gotta chuck em all, so much so that Howard doesn't even look to see if Kobe is going to pass any more.
Re: Top values on new metric called "Individual Player Value
Kobe is 44th on XRAPM estimate, in part due to a -1.1 on defensive impact.
FWIW, none of the guys in the top 12 on IPV but not in the top 12 on XRAPM were better than +1 on defensive impact while all the guys in the top 12 on XRAPM but not on IPV were clearly rated higher than +1 on XRAPM defensive impact estimate.
Might be interesting to take prior informed by SPM RAPM (whoever's) and sort players (or just top players) into 3-5 bins by team quality and look at the average ratios of pure SPM to prior informed by SPM RAPM for those bins overall or separately for the offense and defense components to see if there is anything notably different about those ratios or if there are other interesting trends by position, player type or other characteristic.
FWIW, none of the guys in the top 12 on IPV but not in the top 12 on XRAPM were better than +1 on defensive impact while all the guys in the top 12 on XRAPM but not on IPV were clearly rated higher than +1 on XRAPM defensive impact estimate.
Might be interesting to take prior informed by SPM RAPM (whoever's) and sort players (or just top players) into 3-5 bins by team quality and look at the average ratios of pure SPM to prior informed by SPM RAPM for those bins overall or separately for the offense and defense components to see if there is anything notably different about those ratios or if there are other interesting trends by position, player type or other characteristic.
-
talkingpractice
- Posts: 194
- Joined: Tue Oct 30, 2012 6:58 pm
- Location: The Alpha Quadrant
- Contact:
Re: Top values on new metric called "Individual Player Value
I'd be happy to discuss the model in general terms if anyone is interested.
Re: what v-zero said (if I understood correctly).... yeah, our IPV tries not to penalize Lebron or Wade too much due to having some of their oomph stolen by the other.
Re: Kobe, he played awfully darned good early on this year. His IPV~4.25 number is roughly +5.25 offense and -1 defense. Kobe is much much much higher in ASPM than he is in IPV, isn't he?
We've looked a bit into where we differ from pure RAPM (I don't know much about the new RAPM yet), and we do seem to like the ultrahigh usage guys more than RAPM does. I'm pretty sure that this is due to using an SPM model as a prior.
Not sure re: diffs in defense, but I'm pretty curious now. Our top guys on D are Howard -> Duncan -> Asik -> Garnett -> Chandler -> Smith -> Gasol -> Allen -> Iguodala -> Noah. We have Josh at negative 9 million on offense this year, which is why he doesn't make our top players list, in spite of the gaudy D.
Re: what v-zero said (if I understood correctly).... yeah, our IPV tries not to penalize Lebron or Wade too much due to having some of their oomph stolen by the other.
Re: Kobe, he played awfully darned good early on this year. His IPV~4.25 number is roughly +5.25 offense and -1 defense. Kobe is much much much higher in ASPM than he is in IPV, isn't he?
We've looked a bit into where we differ from pure RAPM (I don't know much about the new RAPM yet), and we do seem to like the ultrahigh usage guys more than RAPM does. I'm pretty sure that this is due to using an SPM model as a prior.
Not sure re: diffs in defense, but I'm pretty curious now. Our top guys on D are Howard -> Duncan -> Asik -> Garnett -> Chandler -> Smith -> Gasol -> Allen -> Iguodala -> Noah. We have Josh at negative 9 million on offense this year, which is why he doesn't make our top players list, in spite of the gaudy D.
Re: Top values on new metric called "Individual Player Value
I'd certainly like to hear more about the metric. Is it essentially a prior informed RAPM model? Possibly using a box score metric for each player in each game and then improving upon that by applying a further APM style adjustment?
Re: Top values on new metric called "Individual Player Value
Is your SPM model prior nonlinear (like ASPM) or linear (like the original forms of SPM and the prior for xRAPM)?
Re: Top values on new metric called "Individual Player Value
For comparison these are my top twenty cumulative ratings for this season from my SEBF model, which is game-by-game and ties all player box score values to those of the rest of their team, and the final score margin, to make them more contextual (you don't get highly rewarded for getting what looks like lots of OREBS if everybody else on your team is also getting lots of them, for instance). All players must have >150 minutes to qualify.
Code: Select all
1. Tim Duncan , 4.54
2. Joakim Noah , 4.53
3. Jason Kidd , 4.23
4. Marc Gasol , 4.15
5. LeBron James , 4.12
6. Anderson Varejao , 4.1
7. Reggie Evans , 3.61
8. Tyson Chandler , 3.59
9. Kevin Durant , 3.49
10. Larry Sanders , 3.21
11. Eric Bledsoe , 3.08
12. Chris Paul , 3.04
13. Al Jefferson , 2.95
14. Ronny Turiaf , 2.76
15. Kendrick Perkins , 2.7
16. Al Horford , 2.67
17. DeJuan Blair , 2.63
18. Russell Westbrook , 2.59
19. Ronnie Brewer , 2.44
20. Hasheem Thabeet , 2.39Re: Top values on new metric called "Individual Player Value
I'd also be interested to read more about the metric (including areas of achievement or areas of ongoing research), to the extent the author is comfortable.
-
talkingpractice
- Posts: 194
- Joined: Tue Oct 30, 2012 6:58 pm
- Location: The Alpha Quadrant
- Contact:
Re: Top values on new metric called "Individual Player Value
New values came up yesterday. I think they look really good by the smell test. Wade feels a bit high to me as he may have had more age/injury regression than we projected... Howard feels a bit low to me..., but otherwise I think the current values look good from a face validity perspective.
re: Some of the questions. Our IPV is indeed essentially a prior-informed RAPM model. On offense, our prior is a SPM model (which is nonlinear, and which is reasonably similar to O-ASPM). We apply an aging curve to the past 2-3 years of data (depending), and we mean regress it into submission. On defense, our prior is done in the same way, but only using p/m data (we don't use any boxscore stats for defense, either in the prior or in the final value).
I think that our result is reasonably 'smooth', in the sense that we get the stability (sample size) benefits of a SPM with the 'catch everything' elements of a RAPM model. So a guy like Collison ends up as a +1.5, rather than a 0 or a +5. I guess this is indeed what je is finding with the new version of his RAPM model (but all I know of that is the one thread that I read on here).
We ended up liking usage more than I thought we would a priori, particularly usg%*ast% and usg%^2 (which I think dsmok also has expressed).
I think v-zero's list of top players (in terms of in-season performance) looks generally good and reasonably similar to ours. The Thabeet thing can't be right, but I'm sure that IPV has several lemons too, given sss for bench players at this point in the season.
re: Some of the questions. Our IPV is indeed essentially a prior-informed RAPM model. On offense, our prior is a SPM model (which is nonlinear, and which is reasonably similar to O-ASPM). We apply an aging curve to the past 2-3 years of data (depending), and we mean regress it into submission. On defense, our prior is done in the same way, but only using p/m data (we don't use any boxscore stats for defense, either in the prior or in the final value).
I think that our result is reasonably 'smooth', in the sense that we get the stability (sample size) benefits of a SPM with the 'catch everything' elements of a RAPM model. So a guy like Collison ends up as a +1.5, rather than a 0 or a +5. I guess this is indeed what je is finding with the new version of his RAPM model (but all I know of that is the one thread that I read on here).
We ended up liking usage more than I thought we would a priori, particularly usg%*ast% and usg%^2 (which I think dsmok also has expressed).
I think v-zero's list of top players (in terms of in-season performance) looks generally good and reasonably similar to ours. The Thabeet thing can't be right, but I'm sure that IPV has several lemons too, given sss for bench players at this point in the season.
Re: Top values on new metric called "Individual Player Value
Can't say I disagree on Thabeet... Gaudy numbers against backups when OKC are already in a hefty lead...
Re: Top values on new metric called "Individual Player Value
Your IPV sounds like a very well-developed model. Just the way I would do it.
How did you specify the weights/develop the model? Cross-validation, I would assume?
I certainly found the same significant effect of USG*AST, which surprised me as well. In fact, I do not even have AST% anywhere else in my ASPM model--it wasn't significant at all.
If you are properly regressing, SSS isn't a huge problem, right? Or at least you won't have huge outliers...
How did you specify the weights/develop the model? Cross-validation, I would assume?
I certainly found the same significant effect of USG*AST, which surprised me as well. In fact, I do not even have AST% anywhere else in my ASPM model--it wasn't significant at all.
If you are properly regressing, SSS isn't a huge problem, right? Or at least you won't have huge outliers...
Re: Top values on new metric called "Individual Player Value
The same interaction dominates my SEBF model from an offensive standpoint, but doesn't surprise me at all... What assists would be more valuable than those from a high scorer who draws a lot of defense and hence creates lots of open shots?DSMok1 wrote:Your IPV sounds like a very well-developed model. Just the way I would do it.
How did you specify the weights/develop the model? Cross-validation, I would assume?
I certainly found the same significant effect of USG*AST, which surprised me as well. In fact, I do not even have AST% anywhere else in my ASPM model--it wasn't significant at all.
If you are properly regressing, SSS isn't a huge problem, right? Or at least you won't have huge outliers...
-
talkingpractice
- Posts: 194
- Joined: Tue Oct 30, 2012 6:58 pm
- Location: The Alpha Quadrant
- Contact:
Re: Top values on new metric called "Individual Player Value
Yes re: cross-validation. We also did a whole bunch of retrodiction studies (though I'm not very sold on those in general). As an aside/addendum to this, if one were interested solely with in-season performance (descriptive), then I'm pretty sure that something like (~0.7*OASPM + ~0.3*ORAPM + DRAPM) would be reasonably solid, and better than RAPM or ASPM alone. this applies to the old RAPM though, and not to the xRAPM that je is publishing now.
A bunch of people wrote us (this very much surprised me) asking to put up values for the entire league, so after thinking about this for a bit and deciding it was fine, we've now done that (and they're up there now). Caveats galore...
1, We compute these numbers for other purposes, so it's possible some players are in here with prior only (ie if they played 0 minutes this year so far). There also may be many players now in the D-League, or released. We cleaned it up through player #100 but after that, there could be a few funny players in the list. I'll take them out over time as I see them.
2, One number metrics, in general, are not that important.
3, Please don't waste any time or money trying to wager with any of this, as I promise it's near-useless for that, and wouldn't be putting these numbers out there otherwise.
4, In general, I think our results are likely similar-ish to xRAPM. I think we use a better prior. Aging curves make a ton of difference too. Of course there are a few wrinkles that I'm not going to go into much depth about.
5, If people want, we're fine updating these 1-2 times per week.
Feel free to flame away at any lemons in here, and ill try to either explain it and/or ill agree with you. There are inevitably many lemons this early in the season, even with a regressed multi-year model.
A bunch of people wrote us (this very much surprised me) asking to put up values for the entire league, so after thinking about this for a bit and deciding it was fine, we've now done that (and they're up there now). Caveats galore...
1, We compute these numbers for other purposes, so it's possible some players are in here with prior only (ie if they played 0 minutes this year so far). There also may be many players now in the D-League, or released. We cleaned it up through player #100 but after that, there could be a few funny players in the list. I'll take them out over time as I see them.
2, One number metrics, in general, are not that important.
3, Please don't waste any time or money trying to wager with any of this, as I promise it's near-useless for that, and wouldn't be putting these numbers out there otherwise.
4, In general, I think our results are likely similar-ish to xRAPM. I think we use a better prior. Aging curves make a ton of difference too. Of course there are a few wrinkles that I'm not going to go into much depth about.
5, If people want, we're fine updating these 1-2 times per week.
Feel free to flame away at any lemons in here, and ill try to either explain it and/or ill agree with you. There are inevitably many lemons this early in the season, even with a regressed multi-year model.
Re: Top values on new metric called "Individual Player Value
I understand if you cannot say, given the purpose of your research, but I would be curios if you could expand on the emboldened point. I get the feeling your model has to do with assessing production availability (of different types) on a team level (through some sort of players----->team transformation) rather than raw (non-contextual) production?talkingpractice wrote:Yes re: cross-validation. We also did a whole bunch of retrodiction studies (though I'm not very sold on those in general). As an aside/addendum to this, if one were interested solely with in-season performance (descriptive), then I'm pretty sure that something like (~0.7*OASPM + ~0.3*ORAPM + DRAPM) would be reasonably solid, and better than RAPM or ASPM alone. this applies to the old RAPM though, and not to the xRAPM that je is publishing now.
A bunch of people wrote us (this very much surprised me) asking to put up values for the entire league, so after thinking about this for a bit and deciding it was fine, we've now done that (and they're up there now). Caveats galore...
1, We compute these numbers for other purposes, so it's possible some players are in here with prior only (ie if they played 0 minutes this year so far). There also may be many players now in the D-League, or released. We cleaned it up through player #100 but after that, there could be a few funny players in the list. I'll take them out over time as I see them.
2, One number metrics, in general, are not that important.
3, Please don't waste any time or money trying to wager with any of this, as I promise it's near-useless for that, and wouldn't be putting these numbers out there otherwise.
4, In general, I think our results are likely similar-ish to xRAPM. I think we use a better prior. Aging curves make a ton of difference too. Of course there are a few wrinkles that I'm not going to go into much depth about.
5, If people want, we're fine updating these 1-2 times per week.
Feel free to flame away at any lemons in here, and ill try to either explain it and/or ill agree with you. There are inevitably many lemons this early in the season, even with a regressed multi-year model.
Re: Top values on new metric called "Individual Player Value
Thanks for sharing the full set of numbers and planning updates.
A few things I noticed: Deron Williams 32nd on IPV, about 100th on XRAPM right now because of a -3.2 on defensive impact estimate. Pau Gasol 36th on IPV, 20th on XRAPM largely because of a +2.1 estimate on defense. IPV has Mayo as slightly positive, XRAPM slightly negative. Neither thinks his boxscore preformance is a big deal in the overall context. IPV also has Kawhi Leonard ever so slightly negative in overall impact estimate, XRAPM exactly neutral. Neither think he is a big deal now and using these measures I would not pump him as a great player or future all-star, as some have, at least yet. Boozer estimated negative on both. Kemba Walker still estimated modestly negative on both, though better than last season. Jeff Green is bottom 2% on IPV and barely better on XRAPM. I believe he is the only guy making more than $8 million to be below -3 on both metrics. I guess he has had 3-4 good boxscore outings in the last 5. Not sure if his xRAPM is up as a result. -3.9 now, lets see where it is at all-star break and end of season. Beasley is also below -3 on both but wasn't quite as highly rewarded, despite being bad on xRAPM estimate (around -2) every season before. patrick Patterson only -1 on IPV but -3 on xRAPM (repeating last season's level and worse than his rookie -2).
A few things I noticed: Deron Williams 32nd on IPV, about 100th on XRAPM right now because of a -3.2 on defensive impact estimate. Pau Gasol 36th on IPV, 20th on XRAPM largely because of a +2.1 estimate on defense. IPV has Mayo as slightly positive, XRAPM slightly negative. Neither thinks his boxscore preformance is a big deal in the overall context. IPV also has Kawhi Leonard ever so slightly negative in overall impact estimate, XRAPM exactly neutral. Neither think he is a big deal now and using these measures I would not pump him as a great player or future all-star, as some have, at least yet. Boozer estimated negative on both. Kemba Walker still estimated modestly negative on both, though better than last season. Jeff Green is bottom 2% on IPV and barely better on XRAPM. I believe he is the only guy making more than $8 million to be below -3 on both metrics. I guess he has had 3-4 good boxscore outings in the last 5. Not sure if his xRAPM is up as a result. -3.9 now, lets see where it is at all-star break and end of season. Beasley is also below -3 on both but wasn't quite as highly rewarded, despite being bad on xRAPM estimate (around -2) every season before. patrick Patterson only -1 on IPV but -3 on xRAPM (repeating last season's level and worse than his rookie -2).