APBRmetrics

Here are the IDS.

250 James, Jerome
725 Langford, Keith
662 McRoberts, Josh
320 Woods, Loren
433 Randolph, Shavlik
727 Diaz, Guillermo
349 Dupree, Ronald
436 Martin, Darrick
85 Price, Ronnie
124 Harrington, Othella
285 Garnett, Kevin

In RAPM ranking order. The players above Kevin Garnet have very few possessions. 20,36,116,60.... Let's check what happens to them when you increase the lambda. And let's see what are the values for them when you use lambda.min (which should be used anyways)

permaximum wrote: In RAPM ranking order. The players above Kevin Garnet have very few possessions. 20,36,116,60.... Let's check what happens to them when you increase the lambda. And let's see what are the values for them when you use lambda.min (which should be used anyways)

The higher the lambda, the less players are ahead of Garnett, and Garnett is getting closer to those ahead in terms of value above standard deviation. Exactly what should happen. The math does NOT say "no lower poss player can end up with an outlier result on top". As I mentioned before, you are arguing a strawman, but even with that little experiment you can see that the math in practice is exactly working as it is supposed to be in "theory". That's my point.

What's up with my suggested experiment? Still claiming that there would be no significant change?

That's not a "significant" change imo. Anyways, this debate started because:

mystic wrote:

permaximum wrote: I especially don't believe 1-year NPI-RAPM results if I don't see 4-5 players with very low possessions on top.

Why? How far a player's coefficient can differ from 0 depends on the choosen lambda and the amount of possessions. With a bigger lambda, players with a very low amount of poessessions will likely stay around 0. So, this conclusion:

permaximum wrote: This means either they use a cutoff( which they shouldn't) or priors one way or another.

Isn't per se true.

I hope you get why I said that now Because it's the same for every year I calculated in 2007-12. So I have right to don't believe vanilla RAPM results if I don't see those players with very few possessions at top.

permaximum wrote: I hope you get why I said that now

No, it still makes no sense. Again, you proclaimed that the math in practice would be different than in theory based on that, which is nonsense as I showed. And you seem to lack the honest gene here by actually saying that you neither did my suggested experiment nor checked the "math" on that part before.

permaximum wrote:Because it's the same for every year I calculated in 2007-12. So I have right to don't believe vanilla RAPM results if I don't see those players with very few possessions at top.

You have every right to such thing, but I suggest you test the ability to predict future games based on your coefficent sets and then compare that to the same ability of sets which controlled for low possession players in a better way. As I said, I use a non-linear weighting scheme for such players and "I have every right to" proclaim that this helps with the prediction, while it does not matter in terms of explaining the results (should be obvious given the extremely low influence of the overall outcome of a season by players with very low possessions). Thus, the point you want to base your "belief" on is pretty much obsolete.

1.Do not assume things and do not pretent that everyone is lazy and lies like you (Because I've seen you hold on to this argument before too in this forum). I tried bigger and a lot bigger lambdas and Garnett was not at top still even before you said it so. And the other players with low possessions was not around the value of 0. You just proved it too. But still you can't accept the fact that you're wrong. So you're stubborn too.

2. All of these are irrelevant because I meant vanilla RAPM. I wasn't talking about something else.

From now on, I won't answer to you who's rude, stubborn, arrogant, but lacks the abilty to calculate a simple ridge regression correct and mislead a newbie in the forum. You also argued with JE about the box score weights of xRAPM. Oddly, you are the defender of xRAPM now. So you can't be trusted either.

Hopefully everyone that visits this forum reads this thread and gets an idea about you. As for me, I don't get any profit from these things. I'm just doing it for fun.

C'mon. Let's just take the positive and move on...

Calm down guys. I think there's some miscommunication going on; this doesn't need to get heated.

Depending on how you do the math (package/weighting/penalty schema), you can certainly have vanilla NPI RAPM with no outliers at the top, including all players. James Brocato, who works for the Mavs now, has NPI single-year RAPM on his site here: http://shutupandjam.net/nba-ncaa-stats/npi-rapm/ . It exhibits those traits.

DSMok1 wrote:Calm down guys. I think there's some miscommunication going on; this doesn't need to get heated.

Depending on how you do the math (package/weighting/penalty schema), you can certainly have vanilla NPI RAPM with no outliers at the top, including all players. James Brocato, who works for the Mavs now, has NPI single-year RAPM on his site here: http://shutupandjam.net/nba-ncaa-stats/npi-rapm/ . It exhibits those traits.

Well, that link is the reason I'm growing skeptical about any RAPM metric.

1. 2001-06 NPI-RAPM values in that site are identical to RAPM values here. Colts18 shared it before. https://sites.google.com/site/rapmstats/
2. 2007-2011 NPI-RAPM values in that site are identical to JE's NPI-RAPM. Just when the link above doesn't have 2007-2011 NPI-RAPM. What a coincidence!
3. 2012 NPI-RAPM is identical to colts18's one. J.E dropped using yearly NPI-RAPM that year.
4. No 2012-13/2013-2014 NPI-RAPM since neither J.E nor colts18 had it. However, this summer J.E shared his 2013/14 pure RAPM I think.
5. Last but not least: J.E's NPI-RAPM rankings (not just values) differ from the rankings in the link Colts18 shared.

My strong assumption is that the owner or runner of shutupandjam did not calculate NPI-RAPM for any year.

Also, J.E.'s NPI-RAPM values are different than the values in the site colts18 shared. For example Ben Wallace is ranked #21 in J.E's 2006 NPI-RAPM while he's ranked #3 on the others. My rankings are different too.

In light of these, there's definetely some information pollution. Unless the calculation process and steps to reproduce known or unknown metrics are shared each time someone calcultes it, how can I trust those? More importantly, which NPI-RAPM I should trust? The one which ranks Ben Wallace #3 or the one which ranks Ben Wallace #21?

BTW if anyone wonders, I got J.E.'s vanila-RAPM files from Crow a long time ago.

Hmm, you're right. I guess James just copied those there to have them in one place.

Still, there are a variety of ways to do RAPM, different packages, and depending on the type of penalty schema, you'll end up with different results. One is not necessarily better than another; they could have similar out-of-sample accuracy. Remember, there's still a quite significant standard error term around each of these estimates--probably on the order of +/-2 at least.

permaximum wrote:I tried bigger and a lot bigger lambdas and Garnett was not at top still even before you said it so. And the other players with low possessions was not around the value of 0.

There is clearly some misunderstanding here, maybe based on the fact that you aren't a native speaker (neither am I), but at NO point did I say that "raise the lambda and Garnett will take the top spot and all low poss players will be close to 0". Really, read my posts again, if you believe that's what I wrote.

Two quick questions and you can start a better conversation by answering honestly.
1. If you increase the lambda, does Garnett move up the ranking?
2. If you increase the lambda, are the other low poss player's coefficients getting closer to 0 than with a lower lambda?

permaximum wrote: 2. All of these are irrelevant because I meant vanilla RAPM.

Maybe the issue is here that you interpret "vanilla RAPM" as "the RAPM I get when using the bbv matchupfile, glmnet with lambda.min and no prior and no different weighting scheme"? For me, and I suspect for many others, vanilla means that no prior information was used in the regression and doesn't tell me anything about the quality of the matchupfile (there can be significant differences in terms of error rate and missing data for a respective season), no information of the used weighted scheme or of the respective lambda. Just that no prior data was used.

permaximum wrote: mislead a newbie in the forum.

I don't care about the other stuff, but don't claim that I "mislead a newbie". Making a mistake is human, and I already conceded that I made a mistake.

permaximum wrote: You also argued with JE about the box score weights of xRAPM. Oddly, you are the defender of xRAPM now. So you can't be trusted either.

Only because I think he didn't choose the best way to determine the boxscore weights and with a different SPM, the overall result would be better, doesn't mean that I have to condemn the whole thing nor does that mean that I can't defend it against invalid criticism. From my perspective xRAPM/RPM is still the best public available player metric for prediction.
Nobody needs to trust me on anything I write, and I really don't care much how "trustworthy" I seem on the internet, but concluding that I can't be "trusted" based on that specific quote seems rather weird.

permaximum wrote: Hopefully everyone that visits this forum reads this thread and gets an idea about you.

Why would that matter to you what other random people think about some random guy on the internet?

DSMok1 wrote:Hmm, you're right. I guess James just copied those there to have them in one place.

Still, there are a variety of ways to do RAPM, different packages, and depending on the type of penalty schema, you'll end up with different results. One is not necessarily better than another; they could have similar out-of-sample accuracy. Remember, there's still a quite significant standard error term around each of these estimates--probably on the order of +/-2 at least.

I don't want to cause an another debate but a different lambda shouldn't move a player from 21st spot to 3rd since all of them has high amount of possessions. However, I know that different packeges may result in different numbers so you're right.

Well, finally the circle is complete and we're back at topic... Even if you do everything right to calculate something simple as that pure RAPM you may end up with different results than other people's because of packages, cross-validation method, lambda, corrupt data etc. We all agree on that I think after all these posts. As you've seen from previous debate there's a potential for human error too. Now how can we know something complex as xRAPM or RPM is stable and accurate? Does better prediction means more accurate model?

permaximum wrote:Now how can we know something complex as xRAPM or RPM is stable and accurate? Does better prediction means more accurate model?

At least that says that it is more reliable. How valid it is can be tested as well. And as far as I see it, it is valid and so far the most reliable metric available for all people. Sure, it can still be "right for the wrong reasons", but with such a dynamic environment we have to deal with, developing a fully working correct model based on the physics involved, is basically a pipe dream.

mystic wrote:

permaximum wrote:Now how can we know something complex as xRAPM or RPM is stable and accurate? Does better prediction means more accurate model?

At least that says that it is more reliable. How valid it is can be tested as well. And as far as I see it, it is valid and so far the most reliable metric available for all people. Sure, it can still be "right for the wrong reasons", but with such a dynamic environment we have to deal with, developing a fully working correct model based on the physics involved, is basically a pipe dream.

All of our metrics are basically inaccurate, it's just a matter of recognizing and quantifying the level of error.

Indeed. Now, the next step is a consistent way of testing the metrics and quantifying the error. Neil Paine provides a pretty good starting point, imho.