Regularized APM at hoopnumbers.com (jsill, 2009)

[Crow]

recovered page 1 of 5

Author Message
jsill



Joined: 19 Aug 2009
Posts: 73


PostPosted: Wed Nov 04, 2009 5:50 am Post subject: Regularized APM at hoopnumbers.com (twice as accurate) Reply with quote
I have some results at my website, hoopnumbers.com, which I'm hoping those of you with an interest in adjusted +/- (APM) will find interesting.

Mike Tamada and maybe some others of you here have mentioned the idea of using ridge regression (a.k.a. regularization) in conjunction with APM. Coincidentally,. this is what I've been working on off and on for the last few months, and I finally got it to the point where I'm ready to put it up on my website.

My main finding is that APM with a carefully chosen regularization parameter (which I'll call RAPM) is about twice as accurate as an APM using standard regression and using 3 years of data, where the weighting of past years of data and the reference player minutes cutoff has also been carefully optimized. Interestingly, this is more or less true even if you only use 1 year of data in conjunction with regularization, since the accuracy boost from using 3 years of data is measurable but fairly minor when regularization is used. The parameter estimates resulting from RAPM using 3 years of data are more intuitively reasonable than the 1 year estimates, although I think even the 1 year estimates look more reasonable than the 1 year estimates you get with standard regression.

My basis for claiming that RAPM is twice as accurate is explained at hoopnumbers.com, but I'll sketch it here. I evaluate the models by testing their predictions on unseen data, i.e., on games which were not included in the dataset used to fit the model. You can take the substitution history of a game and take a previously fitted APM model and generate predictions for each game snippet. Then you can add up all the snippet predictions, appropriately possessions-weighted, to get a prediction for the game's final margin of victory. You can compare this prediction to the actual margin of victory and evaluate the accuracy. I did this for the 342 games in March and April of last year, after fitting models on the games through February (and, additionally in some cases, also the games from '07-'08 and '06-'07). The best I managed to do with standard regression-based APM, using 3 years of data, was an R-squared of about 9% on the March and April games. With regularization and using 3 years of data, I got the R-squared up to 17%. Surprisingly, even with 1 year of data, I could get an R-squared of 16% if regularization was used appropriately, without even using a minutes cutoff, i.e., without lumping any players into the reference player bucket.

Again the claim of a near-doubling in accuracy is relative to a 3-year time-weighted APM using standard regression. If we were to compare RAPM to standard APM on 1 year of data, the boost would be bigger. In fact, I'm not even sure how to define the boost in that case, since the accuracy of 1 year standard APM is just not very good at all, according to my experiments.

In addition to introducing RAPM, a secondary goal of this post is to encourage the use of out-of-sample testing techniques like cross-validation as a way of evaluating methods to see what kind of predictive power they have, and also as a way to make choices which otherwise sometimes seem sort of arbitrary, like the minutes cutoff for the reference player or the weighting of past years of data. Looking at the standard errors around parameter estimates and so forth has its place, in my opinion, but ultimately these models are usually used to predict the future (implicitly and indirectly or otherwise) so I think it's important to gauge their success in doing so by testing their success on a holdout set which the model was not fit on.

Here is the writeup of my results:

http://hoopnumbers.com/allAnalysisView? ... ssion=True

Here are my 3-year RAPM results:

http://hoopnumbers.com/allAnalysisView? ... 9multiYear

Here are my 1-year RAPM results:

http://hoopnumbers.com/allAnalysisView? ... &year=2009

Thanks for any feedback!
Back to top
View user's profile Send private message
Crow



Joined: 20 Jan 2009
Posts: 806


PostPosted: Wed Nov 04, 2009 6:38 am Post subject: Reply with quote
Very timely. Thanks and good luck with this and what might come out of it.


I've talked over time about a concern that traditional Adjusted was overstretching the data. Your writeup also mentions that concern and 1 year and 3 year RAPM gives a tighter range between +8 and -8. I have just started looking at your data but I like this much.

In the NESSiS thread I was essentially reaching for ways to achieve what cross-validation and regularization could achieve. I had temporarily neglected the Ridge Regression talk. I am glad to see it implemented.

The moves of players on good and bad teams under regularization compared to without might address the issue Nick S. noted about minutes weighted team Adjusted +/- of existing Adjusted models have some notable variance with actual team performance. How much better does with regularization do on that than without at the minutes-weighted team level?

Even after regularization it isn't explaining a lot is it? Sounds really low, lower than I expected. But this is a the stint or play by play level? I guess that shouldn't be so surprising or alarming. But how well does it explain at the game level or the season series between teams or playoff series level?

What do you think about the idea of some sort of SPM-APM blend or SPM influenced input for APM?

Do you have interest / plans in taking this technique to the lineup level? Player pairs?

Anything further to say about the multicollinearity issue or possible improved ways to address and reduce its effects?

What if you instead of trying to find a single value for each player applied to every stint on the court to solve the league puzzle as best you can, you allowed the model to assign a player a value from a limited number of different values, say 3 or 5 of them, to model a player who doesn't perform exactly the same all the time? Would that help reduce average errors and outliers? Can that be made to work? Would that help get at where the good and bad contexts and player fits with role and context are? Then the player's value instead of being a single point estimate of + this or - that would be for example 20% +4, 40% +2, and 40% -3 or some such. I think that could be useful. And I guess you could look at which of these partial Adjusted scores come during a more heavy ratio of more win-meaningful moments or less meaningful moments. Players will vary on that and it would be worth gauging. Winston addresses this issue and uses it in determining the average win impact estimate but seeing it at a lower split of the data might be useful for addressing the rotation based on game situation.

Last edited by Crow on Wed Nov 04, 2009 7:24 am; edited 4 times in total
Back to top
View user's profile Send private message
Mike G



Joined: 14 Jan 2005
Posts: 3576
Location: Hendersonville, NC

PostPosted: Wed Nov 04, 2009 6:46 am Post subject: Reply with quote
Wow, 447 ranked players.

Misspelled "analyses" in the headline.

21 players are ranked as better than playing at home:
Code:
1 Lamar Odom 7.428
2 LeBron James 6.716
3 Ray Allen 5.956
4 Chris Paul 5.062
5 Dwyane Wade 4.966
6 Rashard Lewis 4.728
7 Yao Ming 4.608
8 Matt Bonner 4.468
9 Kevin Garnett 4.289
10 Jason Kidd 4.161
11 Jameer Nelson 4.026
12 J.R. Smith 3.996
13 Kirk Hinrich 3.976
14 Ronald Murray 3.756
15 Steve Nash 3.545
16 Brandon Roy 3.433
17 Rasheed Wallace 3.329
18 Tony Parker 3.216
19 Ben Wallace 3.215
20 Andre Iguodala 3.194
21 Kobe Bryant 3.192
22 Home Court Advantage 3.128

Here are the above-average Spurs (> 0):
Code:
Rank Player RAPM
1 Matt Bonner 4.468
2 Tony Parker 3.216
3 Ime Udoka 1.821
4 Tim Duncan 1.699
5 Kurt Thomas 0.986
6 Roger Mason 0.509
7 Pops Mensah-Bonsu 0.331

These are the one-year rates. The 3-year has Duncan as a +5.73, which is #5 in the league.

The 3-year list in general seems to show fewer surprises. But Dwight is just #55, below Amir, Hayes, Tim Thomas, etc.
_________________
`
36% of all statistics are wrong
Back to top
View user's profile Send private message Send e-mail
DSMok1



Joined: 05 Aug 2009
Posts: 602
Location: Where the wind comes sweeping down the plains

PostPosted: Wed Nov 04, 2009 11:59 am Post subject: Reply with quote
Good work, jsill!

Did you happen to calculate the standard error for each player? That would be immensely useful in understanding the confidence associated with each evaluation.
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Ryan J. Parker



Joined: 23 Mar 2007
Posts: 708
Location: Raleigh, NC

PostPosted: Wed Nov 04, 2009 12:32 pm Post subject: Reply with quote
Great stuff Joe, but I have a few questions. I'm in the process of becoming more familiar with using cross-validating to measure prediction error, so I'm very much interested in some of the stuff you've done here.

Would it be appropriate to say you're using 10-fold cross-validation using the data up to February? Did you do any cross-validation using an entire season worth of data? Can you calculate standard errors for your cross-validation estimates?

I would also be interested in seeing the mean absolute error of the cross-validation instead of just RMSE. Lastly, would it be possible to succinctly describe the difference(s) between ridge regression and lasso?
_________________
I am a basketball geek.
Back to top
View user's profile Send private message Visit poster's website
jsill



Joined: 19 Aug 2009
Posts: 73


PostPosted: Wed Nov 04, 2009 12:42 pm Post subject: Reply with quote
Crow:

Thanks for all the feedback.

Quote:
In the NESSiS thread I was essentially reaching for ways to achieve what cross-validation and regularization could achieve


Yes, I think the gist of some of your comments were aimed at combatting overfitting, which is indeed what regularization is intended to address.

Quote:
How much better does with regularization do on that than without at the minutes-weighted team level?


I don't think I read Nick S.'s previous comments, but I guess the idea is that a minutes-weighted average of the APMs of the players on a team should roughly correspond to or track the team margin of victory (or success, more generally)? I haven't looked at that yet, but it's worth looking at.

I did do some preliminary experiments on a related topic, though. I need to go back and do these more carefully, so don't hold me to the results, but this is tentatively what I found. I ran a team-level version of APM which ignored the presence of individual players and essentially modelled things as if there all games were 1 on 1 games with Mr. Laker playing against Mr. Sixer or Mr. Bull playing versus Mr. Hornet, etc. The single APM number you get for each team corresponds pretty well to their average, season-long margin of victory, as you might expect. This approach actually beat standard APM by a healthy margin in its ability to predict the margin of victory on future, test set games. Regularized APM was at least its equal regarding test set accuracy. I was really hoping it would do better, but at least it was roughly equal. Again, I need to double-check these results, though.

Quote:
But this is a the stint or play by play level?


The R-squareds are at a game level, actually (predicting game-level margin of victory). I plan to look at stint level as well, but the R-squareds there are going to be even lower. Yes, I was hoping for better, too, but at least we appear to be making progress relative to standard APM.

Quote:
What do you think about the idea of some sort of SPM-APM blend or SPM influenced input for APM?


I need to read about SPM (and your related ideas) some more before I can give an intelligent answer here.

Quote:
Do you have interest / plans in taking this technique to the lineup level? Player pairs?


Certainly. I have done some preliminary work on player pairs without much success, but I plan to revisit it. With enhancements like these, I think it's valuable to stay within the framework of cross validation in order to test whether the additions to your model really boost your ability to predict out-of-sample. Otherwise, you can just keep adding gizmos to your model and you'll probably end up overfitting and harming things. So when I say I haven't had success yet, I mean I haven't yet been able to demonstrate a boost in prediction performance from using player pairs (versus the individual player, sum of 5 APM framework). I haven't given up on it, though.

Quote:
Anything further to say about the multicollinearity issue or possible improved ways to address and reduce its effects?


Not specifically just yet, but in my experience generally, one of the best ways to improve performance when working with noisy, limited data is to incorporate prior information or domain knowledge. The regularization I did, which essentially tells the regression what a reasonable APM range is, is one way to do that, but there may be others.

Your idea to assign players to a limited number of probability weighted values might be a form of regularization as well. It might be tricky algorithmically to implement, it, though.

I plan to look at concepts like win impact, as you mentioned, in the future.
Back to top
View user's profile Send private message
schtevie



Joined: 18 Apr 2005
Posts: 409


PostPosted: Wed Nov 04, 2009 12:45 pm Post subject: Reply with quote
All else equal, bigger R-squareds are nice. But do the collective results make sense?

Take an arbitrary cut-off of a pretty darn good player, someone who delivers a net 4 points per 100 possessions. In the three-year data shown, there are nine players who accomplished this. Just nine.

The greatest of these is KG, who following the estimate, when he was in the game for his approximate 30 minutes, his contribution on the scoreboard above that of an average (0 APM) player was about 4.5 points. And for Chauncey Billups at #9, playing 35 minutes per game, his contribution was about 2.9 above average.

By contrast, the straight APM gives a dramatically different result (that is consistent whether it uses one or more seasons). Take Stephen Ilardi's stabilized results for last year (using six years of weighted data). Here we have 43 players with APMs above 4. And the stars are starrier.

Never mind the particular players cited; that isn't the point. The issue is whether it is plausible that the biggest stars in the league have such small impacts on the scoreboard and that there are apparently so few of them.

I am skeptical.
Back to top
View user's profile Send private message
basketballvalue



Joined: 07 Mar 2006
Posts: 208


PostPosted: Wed Nov 04, 2009 12:56 pm Post subject: Reply with quote
Joe,

I think this looks very interesting and I'm looking forward to really reading through your links in detail. I particularly appreciate you've used the estimates to predict segments not in the dataset used for estimating, I agree this is very important.

For our reference, have you compared your predictions to predictions using other approaches (e.g. PER, Win Score,...)? This would help set our reference point for how good 9% or 17% is. Of course, this is venturing into the territory of Dan's presentation at NESSIS a couple of years ago.

Thanks,
Aaron
_________________
http://www.basketballvalue.com
Follow on Twitter
Back to top
View user's profile Send private message
jsill



Joined: 19 Aug 2009
Posts: 73


PostPosted: Wed Nov 04, 2009 12:58 pm Post subject: Reply with quote
Mike G:

I agree that Dwight Howard's ranking is lower than we would expect. On the other hand, at least his APM based on '08-'09 alone is 2.515, or 34th in the league. At basketballvalue he is at 1.04, or barely above average, for '08-'09 after looking tremendous in '07-'08.

The Spurs results and Matt Bonner's APM in particular are a little funky. If you look at his raw plus/minus per 48 minutes relative to the other Spurs last year, though, he looks awfully good. It's amazing to me, in particular, that they defended so well with him on the floor (90.7 vs. 92.6 for Duncan). As I mention in my writeup, by no means do I think Bonner was a top 10 player last year or the best on the Spurs. The numbers are what they are, though.

DSMok1: I do yet not have the standard errors for each player. Because I'm using regularization, this becomes more complicated than getting standard errors in a classic regression. In theory, we should be able to get an "a posteriori" distribution for the parameters which is a consequence of combining the a prior distribution from which the regularization term stems with the data. I need to do some research on how to do this, though.
Back to top
View user's profile Send private message
jsill



Joined: 19 Aug 2009
Posts: 73


PostPosted: Wed Nov 04, 2009 1:46 pm Post subject: Reply with quote
Ryan:

Quote:
Would it be appropriate to say you're using 10-fold cross-validation using the data up to February?


Yes, that is exactly what I did.

Quote:
Did you do any cross-validation using an entire season worth of data?


Yes, I can easily run this and have run it in the past. It would be a good way of getting an accurate estimate of the out-of-sample error of your model if it's already been tuned by other means, since you could get an estimate over the entire season (by holding out each 10th of the data and testing on it in succession).

However, if you're tuning a parameter like the lambda of the regularization or the reference player minutes cutoff, then it's not quite legit to run the cross validation for lots of values of the parameter and then take the performance of the best performing parameter and report that as an unbiased estimate of the actual performance. In that case, you've subtly and indirectly fit your parameter on the same data you're evaluating it on. Reporting the CV results for the best parameter choices is not nearly as egregious as reporting the in-sample results of a regression on noisy, limited data, of course. It's a minor sin, but it's still slightly dubious.

Also, a tougher and more realistic evaluation is to evaluate a model on data which happened chronologically after the data the model was fit on, since that's the situation in reality. So that's why I used the cross validation to tune the parameters and the later March/April data for a final evaluation.

When you ask about standard errors for the cross-validation estimates, do you mean estimates of the RMSEs or estimates of the APM values for each player?

Quote:
I would also be interested in seeing the mean absolute error of the cross-validation instead of just RMSE


I might try to run this at some point. I'd be surprised if it yielded a significantly different picture, though.

Quote:
Lastly, would it be possible to succinctly describe the difference(s) between ridge regression and lasso?


Ridge regression penalizes the square of the APM values, which corresponds to a gaussian prior over the APM values in a Bayesian interpretation, with the regularization parameter (lambda) corresponding to the ratio of the variance of the noise in the problem to the variance of your prior distribution.

The lasso would minimize the squared error on the data subject to the sum of the absolute values of the APMs being below some constant. I don't have hands-on experience with the lasso, but my understand is that it often sets the coefficients of many of the variables to zero. So in our context, it would likely yield an APM of zero for a lot of players. I'm not sure that's desirable, but on the other hand, it's hard to say for sure how it would perform in terms of prediction accuracy until we try it.
Back to top
View user's profile Send private message
deepak



Joined: 26 Apr 2006
Posts: 664


PostPosted: Wed Nov 04, 2009 2:05 pm Post subject: Reply with quote
If you have the numbers readily available, could you publish the leaders in fast break points per game (team-wise, or even player-wise) over the last several years? I can't find that information elsewhere.
Back to top
View user's profile Send private message
Crow



Joined: 20 Jan 2009
Posts: 806


PostPosted: Wed Nov 04, 2009 2:34 pm Post subject: Reply with quote
Thanks jsill for the replies to my questions and the others.
Back to top
View user's profile Send private message
Ryan J. Parker



Joined: 23 Mar 2007
Posts: 708
Location: Raleigh, NC

PostPosted: Wed Nov 04, 2009 2:46 pm Post subject: Reply with quote
Thanks for the response Joe. Very insightful.

As for the standard error, I'm talking about the standard error of the RMSE. More specifically, in The Elements of Statistical Learning, Hastie et al. refer to "... the importance of reporting the estimated standard error of the CV estimate" (pg 249). I'm still going through this section of the book, so I don't know exactly how you go about calculating it, but I figure you might know how to do so. Very Happy
_________________
I am a basketball geek.
Back to top
View user's profile Send private message Visit poster's website
DSMok1



Joined: 05 Aug 2009
Posts: 602
Location: Where the wind comes sweeping down the plains

PostPosted: Wed Nov 04, 2009 2:48 pm Post subject: Reply with quote
I was considering your Lambda (a-priori distribution) and realized that what you are getting, because of its inclusion, is a "regressed to the mean" APM. Since you calculated your Lambda based on one year of data, the regression to the mean is greater. If you used multiple years of data, the Lambda should change such that there is a greater spread, or at least more outliers. That said, because of "regression to the mean" most players' APM does balance out over several years, reducing outliers that way....

I would be interested if you looked into this.

Basically, this is analogous to a Bayesian "best estimate" of the player's true current APM, similar to what I discussed here. The issue, however, is that all players are regressed toward 0--which is not accurate. I would prefer to see the player's regressed toward a value based on their "minutes per game," which I see as approximately based on APM and thus providing a good frame of reference.
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Crow



Joined: 20 Jan 2009
Posts: 806


PostPosted: Wed Nov 04, 2009 3:14 pm Post subject: Reply with quote
DSMok1 wrote:
The issue, however, is that all players are regressed toward 0--which is not accurate. I would prefer to see the player's regressed toward a value based on their "minutes per game," which I see as approximately based on APM and thus providing a good frame of reference.


You said based on rather than explicitly just minutes per game, so what about regressed toward

minutes per game * (1+ (team win%- .5))

or something in that vein?

[Crow]

page 4& 5

Author Message schtevie



Joined: 18 Apr 2005
Posts: 400
Posted: Fri Nov 06, 2009 6:53 am Post subject:

DLew wrote:
Given that a point per game in differential is worth roughly 2.5 wins, then a player who is a +8 per 100 possesions is worth about +6 per 40 minutes which is what most stars play in a game. So that's 15 wins better than average and roughly 25 wins better than replacement. To me that does not seem unreasonably low for the best player in the league and that seems to be what the numbers suggest.
David, let me quibble a bit, provide a salient example, then repose the question. 40 minutes per game is an upper bound. Last year no one hit that mark. No matter. Consider the time-weighted, 3 year RAPM at the top of the table, that of one LeBron James. This equals 5.598. Going to B-R.com, we see that the Cavs had a pace factor of 88.7 and that LeBron played 37.7 minutes per game. If one does the arithmetic based on these numbers, we get that LeBron helped the Cavs all of 4.13 points per game. This is the best player in the league - a player for the ages. If we accept the range of the table as accurate, having LeBron James on your team would be the equivalent of playing all one's games at home and swapping him out for Matt Bonner, or Rasheed Wallace, or Tony Parker, or Flip Murray. (Well, maybe Rasheed.) Does this make sense? And let me restate the question I asked previously. If conventional APM estimates are unbiased, and if multiple-year estimates address the multi-collinearity issue sufficiently so as to lower the standard errors, and if the resulting estimates are essentially twice those of RAPM, why should the latter be believed to be accurate?
Back to top

Ryan J. Parker



Joined: 23 Mar 2007
Posts: 707
Location: Raleigh, NC
Posted: Fri Nov 06, 2009 8:06 am Post subject:

mtamada, we're measuring different things. My RMSE is based on the data from the next season, and my error is calculated from the team's final offensive, defensive, and net efficiency ratings._________________I am a basketball geek.
Back to top

gabefarkas



Joined: 31 Dec 2004
Posts: 1311
Location: Durham, NC
Posted: Fri Nov 06, 2009 8:39 am Post subject:

schtevie wrote:
And let me restate the question I asked previously. If conventional APM estimates are unbiased, and if multiple-year estimates address the multi-collinearity issue sufficiently so as to lower the standard errors, and if the resulting estimates are essentially twice those of RAPM, why should the latter be believed to be accurate?
Look at the contrapositive(?) of what you're saying: If RAPM addresses the MC issue sufficiently so as to lower the SE compared to 1-year APM, and the resulting estimates are essentialy half those of multi-year APM, why should multi-year APM be believed to be accurate?
Back to top

schtevie



Joined: 18 Apr 2005
Posts: 400
Posted: Fri Nov 06, 2009 9:26 am Post subject:

So, just to be clear, we have multi-year estimates of APM and RAPM whose results are strictly incompatible (e.g. for all the top players as indicated by APM you have to go two, sometimes more, standard errors to approach the RAPM result.) And we have one year estimates of each type that are compatible with their multi-year brethren. We also know, a priori, that RAPM introduces bias as the price of reducing variance. Now, one is certainly free to believe, for example, that LeBron James only gave his team four net points per game last year. And one is free to believe that the generally observed skewing of salaries toward the perceived-to-be-star players is a chronic and gross inefficiency in the league. One can form one's priors however one wishes. But why should the presumption be that an estimate that is biased (to uncertain degree?) is to be preferred to one that is unbiased?
Back to top

inkt2002



Joined: 10 Oct 2009
Posts: 8
Posted: Fri Nov 06, 2009 12:27 pm Post subject:

Is there a way to predict the effect rookies will have on this years RAPM of each teams players?
Back to top

gabefarkas



Joined: 31 Dec 2004
Posts: 1311
Location: Durham, NC
Posted: Fri Nov 06, 2009 1:50 pm Post subject:

schtevie wrote:
But why should the presumption be that an estimate that is biased (to uncertain degree?) is to be preferred to one that is unbiased?
Bias/Variance trade-off. Pick your poison.
Back to top

jsill



Joined: 19 Aug 2009
Posts: 73
Posted: Fri Nov 06, 2009 2:38 pm Post subject:

Let me catch up on some questions. Ryan: Quote:
As for the standard error, I'm talking about the standard error of the RMSE. More specifically, in The Elements of Statistical Learning, Hastie et al. refer to "... the importance of reporting the estimated standard error of the CV estimate" (pg 249). I'm still going through this section of the book, so I don't know exactly how you go about calculating it, but I figure you might know how to do so.
It's not hard to estimate the standard error of the RMSE, or of the MSE (mean-squared error) at least. The squared error is just a simple variable you're talking the mean of to get the MSE. and you can calculate also calculate its variance in the usual way, and then the variance in your estimate of that mean is the variance of the variable divided by N (number of samples) (or N-1 if you're being really careful). However, when comparing model A versus model B, you want to be careful to distinguish between the mean squared error of each model and the mean of the difference between the squared errors of model A and model B. If model B beats model A by a little bit on every test data point, you probably can conclude that it's superior in a statistically significant way, but it might not look statistically significant if you only calculated error bars around the MSEs for A and B. By the way, the Hastie book on statistical learning is a pretty good book, in my opinion. DSMok1 and Crow: I have looked into what needs to be done to tweak ridge regression towards non-zero values, so I may give your suggestions a try (bias towards something that's a function of minutes and possibly team success) at some point. As Gabe points out, 0 makes a lot of sense as a global value since every point for is a point against for someone else, so I'd probably have to be careful to try to make the nonzero biases balance out to zero globally. Gabe: I am using Python and the NumPy library to implement the ridge regression. mtamada: Quote:
Excellent stuff. The ridge regression seems to improve things substantially. And if I understand your cross-validation correctly, you're basically using the Retrodiction technique (meaning, using the actual player minutes for the Out-Of-Sample data) -- but rather than trying to retrodict the entire 2008-09 season, you're looking at the last few months' of the 2008-09 season?
Well, there are actually 2 stages of "retrodiction". One stage, the cross-validation stage, involves splitting the data through February into 10 splits, fitting on 9/10th of the data and testing on the final 10th for each of the 10 splits, to get an RMSE over the entire year through February. This is done for each metaparameter choice (the lambda in the ridge regression, the minutes cutoff, and the time decay).The metaparamater choices which give the best accuracy are then used in conjunction with fitting the model on all 10 splits through February (plus past years data, in some experiments) and then that model is tested on March and April. Aging curves is an interesting possible extension I may look at. I need to read about Generalized Ridge Regression and also what RyanP did. I have started to look at player pairings (e.g. Stockton and Malone) and lineups more generally, but don't have much to report yet. Crow: I have to think some more about your 4 factors/Mr. Laker Factors suggestion. It might help if you can elaborate. I am thinking about doing a rebounding-specific adjusted plus/minus, for instance, which may or may not be related to what you are saying.
Back to top

DSMok1



Joined: 05 Aug 2009
Posts: 565
Location: Where the wind comes sweeping down the plains
Posted: Fri Nov 06, 2009 3:26 pm Post subject:

jsill wrote:
DSMok1 and Crow: I have looked into what needs to be done to tweak ridge regression towards non-zero values, so I may give your suggestions a try (bias towards something that's a function of minutes and possibly team success) at some point. As Gabe points out, 0 makes a lot of sense as a global value since every point for is a point against for someone else, so I'd probably have to be careful to try to make the nonzero biases balance out to zero globally.
Definitely, 0 is a must for the global value. And adjustment for the quality of the team would also be good. For instance--a simple linear equation on 1 year APM vs. minutes (for 08-09, Basketball Value): APM = 0.00246*MIN - 5.37. I would recommend min/game, not total minutes, however, so as not to give too low a prior for starters that got injured. The R^2 is around 0.5 on that. Adding in an adjustment for strength of team: take efficiency differential (say +2) and simply add or subtract differential/5 from the prior. (That's right, isn't it?)
Back to top

jsill



Joined: 19 Aug 2009
Posts: 73
Posted: Fri Nov 06, 2009 3:26 pm Post subject:

Schtevie: Thanks again for your perspective. It's always helpful to have someone look things over with a skeptical eye. Quote:
But why should the presumption be that an estimate that is biased (to uncertain degree?) is to be preferred to one that is unbiased?
I would argue that it should be preferred because it does a better job of predicting the future. That's generally what this stuff is going to be used for. As I wrote on my site, an NBA front office might make a trade, free agent signing, or playing time decision based (to some degree, at least) on the results of an APM analysis. Then they have to hope that the decision works out well after the player is acquired or given more minutes. Given that usage case, would you concede that it's at least somewhat compelling that RAPM got almost twice the R-squared when testing on future data? So far, you've barely grappled with this point at all. As Gabe has pointed out, there is a well-known bias-variance tradeoff in statistics which becomes particularly important when you are fitting models with a large number of parameters on limited, noisy data. It is well known that introducing some bias can improve accuracy overall. Ridge regression is not some obscure procedure I pulled out of nowhere. It's widely used. Bias is not necessarily harmful or bad (in the context of statistical estimation). You say it's biased "to an uncertain degree". Sure, I suppose, but the model was validated on out-of-sample data and shown to be successful (more successful than standard APM, at least). So it's really not very uncertain that we've improved the model by introducing the bias. So far you've largely fixated on your conviction that stars make more of a difference than the RAPM results would suggest. You may or may not be right about that, but let's remember that this is just one qualitative criterion among many others with which we might evaluate the results. Are you confident that your best 6 year standard APM results wouldn't raise some eyebrows and look dubious given conventional wisdom, based on some other criterion? In any case, the evaluation of a model can't solely be on the basis of the degree to which it reinforces what you already believe. Out-of-sample testing provides a principled and objective way to evaluate models, and it's a technique which reflects the real-world use case. By the way, this is wrong: Quote:
Consider the time-weighted, 3 year RAPM at the top of the table, that of one LeBron James. This equals 5.598
It's 5.958, not 5.598. Obviously, the RAPM estimates are far from perfect, and it's likely that that they are more wrong in certain spots than others. It is certainly possible that the estimates are particularly bad at the very top, i.e., the all-star region (LeBron, etc.). What we can say is that taken collectively as a model for the whole league, the estimates I got did quite a bit better in predicting the future than standard estimates given access to the same data. I don't yet have access to 6 years of data (I need to get my parser working for seasons further in the past) so maybe the results would be different if I had that much data. Maybe ridge regression wouldn't help. Finally, I don't want to get into a long back-and-forth on specific examples in order to debate what the right magnitude is for the APM estimates of stars, but I feel compelled to bring up one data polnt. Given your belief in the importance of stars, what do you think the APM impact should have been when Michael Jordan at 39 minutes per game was removed and Toni Kukoc at 24 minutes per game was added in 1993? That's the main difference between 92-93 Chicago and 93-94 Chicago (plus Kerr for Paxson and Trent Tucker, etc.) . The first team won 57 games (+6.3 ppg) and the second team won 55 games (+ 3.1ppg). Yes, the second team went down in the playoffs (4-3 to a team that lost 4-3 in the finals). Still, is that a big a dip as you would have expected? Now, I'm sure that you'll come back at me with various examples of teams falling apart completely once they lose their star, and I don't really want to get into a back and forth on specific examples. Regardless, I don't think the case for the enormity of the impact of stars is as clear as you're making it out to be. With that said, it may turn out that if we improve upon what I have further that we'll get bigger numbers for LeBron, Wade, etc. As I said, I wouldn't necessarily claim that the RAPM estimates for the stars are better than standard APM estimates for the stars (I wouldn't concede they're worse, but I wouldn't say for sure that they are better). All I can say is that taken collectively as a model for all players in the league, RAPM seems to do better.
Back to top

DSMok1



Joined: 05 Aug 2009
Posts: 565
Location: Where the wind comes sweeping down the plains
Posted: Fri Nov 06, 2009 3:51 pm Post subject:

jsill wrote:
Finally, I don't want to get into a long back-and-forth on specific examples in order to debate what the right magnitude is for the APM estimates of stars, but I feel compelled to bring up one data polnt. Given your belief in the importance of stars, what do you think the APM impact should have been when Michael Jordan at 39 minutes per game was removed and Toni Kukoc at 24 minutes per game was added in 1993? That's the main difference between 92-93 Chicago and 93-94 Chicago (plus Kerr for Paxson and Trent Tucker, etc.) . The first team won 57 games (+6.3 ppg) and the second team won 55 games (+ 3.1ppg). Yes, the second team went down in the playoffs (4-3 to a team that lost 4-3 in the finals). Still, is that a big a dip as you would have expected?
For what it's worth, the Bulls efficiency differential dropped from 6.8 to 3.4....on a team where most of the players were still on the upswing of the aging curve. Jordan hadn't reached his peak, either. I'd estimate he was about +5.5, based on that data...
Back to top

Crow



Joined: 20 Jan 2009
Posts: 777
Posted: Fri Nov 06, 2009 4:27 pm Post subject:

jsill, I did shift from your working (I assume) with the entire team in your Mr Laker vs Mr Sixer example to doing it at the lineup level and had to add that to the original post to clarify it. If the language was opaque to you overall my apologies but I am not sure how much more I can really say different. Essentially I suggested running the Adjusted Regression analysis 5 times. The first time you'd just look at play by play or stint rebounding data and nothing else from the scoreboard or boxscore and compute Adjusted Lineup rebounding ratings. As I said you could give the Adjusted Offensive rebound or Offensive rebound plus / minus findings (i.e. +2 Offensive Rebounds per 48 minutes or -1 OR Allowed, etc,) their regression found value used in Statistical Plus/Minus (say for that +2 multiply by 1.2 or whatever that found value is) or just the Value of Possession (.96 or whatever that is for the season). Then do the same in regression runs for turnovers and their 'value" and just 3 point shooting data, non- 3 point shooting and free throw shooting (or some other split of scoring data) for just those subsets of points in the play by play or stint data, by the different lineups in their matchups. Could overall Adjusted ratings suffer from Factor level multicollinearity? I assume so. Would separate runs make sense, improve the accuracy of the results, in line with the thinking in that paper fundamentallysound referenced? I look forward to what you can say about and do with Adjusted Factor level analysis of rebounding and hopefully turnovers and scoring.Last edited by Crow on Sat Nov 07, 2009 3:29 pm; edited 1 time in total
Back to top

DLew



Joined: 13 Nov 2006
Posts: 222
Posted: Fri Nov 06, 2009 6:25 pm Post subject:

It is possible (and I think likely) that both parties are right here. Schtevie is probably correct that values are too low for superstars, I mean we know the ridge regression biases estimates towards zero so it shouldn't be surprising if this were the case, but jsill is probably correct that there is quite a bit of evidence that this introduction of bias, while somewhat problematic, represents an improvement. As he mentioned, he did not use regularization for no reason. He chose it because noise and collinearity have been cited as significant problems with APM, and, regularization is a well established technique for dealing with such situations. He then presented out of sample validation for the methodology that showed it to be a superior predictor of future performance. So, I think it would be reasonable to conclude that while the results of this method are almost certainly more compressed than the true distribution of player abilities there is good reason to believe it is nonetheless superior to the linear regression method.
Back to top

jmethven



Joined: 16 May 2005
Posts: 51
Posted: Fri Nov 06, 2009 8:08 pm Post subject:

So if I am understanding correctly, your APM values are not on the same scale as Steve Ilardi's numbers. This point has been overlooked in the thread and would account for a lot of the difference (though probably not all of it) between the numbers. jsill wrote:
Regarding the plausibility of the magnitudes for the top players, it's worth remembering that the number is computed relative to a minutes-weighted average APM player. When you hear "average NBA player" casually, you might tend to think of a simple average over everyone on an NBA roster, which would mean "average NBA player" might correspond roughly to a sixth or seventh man on an average team. However, since it's minutes-weighted, that means the "average NBA player" is a good bit better than that. I haven't looked at the numbers carefully, but it probably roughly corresponds to the third or fourth best player on an average team- in other words, a fairly decent player. That's why the majority of players have scores below zero. So if the model says Chauncey Billups gets you 3 extra points in margin of victory (given that he plays 35 mpg) relative to an average NBA player, that's relative to a fairly decent player. That's about a 3 points per game boost relative to Andre Miller or Mike Bibby (just to take a couple of players for whom I get an RAPM close to 0).

Back to top

jsill



Joined: 19 Aug 2009
Posts: 73
Posted: Fri Nov 06, 2009 8:33 pm Post subject:

jmethven: Do you know how Steve Ilardi normalizes his numbers? Is it a straight average of every player who is evaluated, without weighting by minutes played?
Back to top

schtevie



Joined: 18 Apr 2005
Posts: 400
Posted: Fri Nov 06, 2009 8:45 pm Post subject:

jsill wrote:
Quote:
But why should the presumption be that an estimate that is biased (to uncertain degree?) is to be preferred to one that is unbiased?
I would argue that it should be preferred because it does a better job of predicting the future. That's generally what this stuff is going to be used for. As I wrote on my site, an NBA front office might make a trade, free agent signing, or playing time decision based (to some degree, at least) on the results of an APM analysis. Then they have to hope that the decision works out well after the player is acquired or given more minutes. Given that usage case, would you concede that it's at least somewhat compelling that RAPM got almost twice the R-squared when testing on future data? So far, you've barely grappled with this point at all.
I don't think it fair to say that I haven't grappled with this point. I have certainly thought about it, and it has implicitly informed my comments. The context you provide here is relevant. Why do we care about APM? It is the desire to establish better estimates of value. Conditional to be sure, but value nonetheless. And from these, various consumers will reap different benefits. For the fan, in whose company I would include traditional media, the hope is that a truer pantheon can be established. Who is opposed to justice? Here, the primary contribution of APM is the incorporation of defense. The Rasheed Wallaces, Kevin Garnetts, and the Ron Artests of the world are the proper beneficiaries. If RAPM better sorts out the rankings at the top, so much the better. If RAPM, while getting the relative rankings more correct, possibly halves true values, so much the worse, but at the end of the day, it doesn't matter that much on this account. For the professional appraiser, those whose livelihoods depend on getting a better sense that the competition of how many net points a given player might be expected to deliver on the court, it seems to me that there is a slightly different perspective. If RAPM says that the best players are far less valuable than they truly are compared to the average, that is a big problem. If RAPM tends to improve the estimation of the relative value of role players (all else equal) well, that is surely an improved state of affairs. jsill wrote:
As Gabe has pointed out, there is a well-known bias-variance tradeoff in statistics which becomes particularly important when you are fitting models with a large number of parameters on limited, noisy data. It is well known that introducing some bias can improve accuracy overall. Ridge regression is not some obscure procedure I pulled out of nowhere. It's widely used. Bias is not necessarily harmful or bad (in the context of statistical estimation). You say it's biased "to an uncertain degree". Sure, I suppose, but the model was validated on out-of-sample data and shown to be successful (more successful than standard APM, at least). So it's really not very uncertain that we've improved the model by introducing the bias. So far you've largely fixated on your conviction that stars make more of a difference than the RAPM results would suggest. You may or may not be right about that, but let's remember that this is just one qualitative criterion among many others with which we might evaluate the results. Are you confident that your best 6 year standard APM results wouldn't raise some eyebrows and look dubious given conventional wisdom, based on some other criterion? In any case, the evaluation of a model can't solely be on the basis of the degree to which it reinforces what you already believe. Out-of-sample testing provides a principled and objective way to evaluate models, and it's a technique which reflects the real-world use case.
I have to disagree here. It is not just one qualitative criterion. It is the most important qualitative criterion. (And you tell me if the existing 6 year APM results raise eyebrows given conventional wisdom.) Gabe would like me to pick my poison. I pick knowing whether or not LeBron is worth 6 or 13 net points rather than whether Delonte is 2 or....2.53. And you misrepresent my expressed views if you are saying that my evaluation has been solely on the basis of prior beliefs. Never mind their rationality, I also suggested that the NBA salary structure - to the extent that it is an expression of perceived on-court value - is significantly at variance with the results you present. jsill wrote:
By the way, this is wrong: Quote:
Consider the time-weighted, 3 year RAPM at the top of the table, that of one LeBron James. This equals 5.598
It's 5.958, not 5.598.
I thank you for catching the typo. Funnily enough, I caught the mistake in making the calculation for LeBron's hypothetical value (and it was based on the correct figure) but didn't correct the text. jsill wrote:
Obviously, the RAPM estimates are far from perfect, and it's likely that that they are more wrong in certain spots than others. It is certainly possible that the estimates are particularly bad at the very top, i.e., the all-star region (LeBron, etc.). What we can say is that taken collectively as a model for the whole league, the estimates I got did quite a bit better in predicting the future than standard estimates given access to the same data. I don't yet have access to 6 years of data (I need to get my parser working for seasons further in the past) so maybe the results would be different if I had that much data. Maybe ridge regression wouldn't help.
Please don't get me wrong. I find the work extremely interesting and informative. And I am very curious to see your breakdowns of offense and defense. I just am trying to emphasize that getting the top end right is hugely and disproportionately important. jsill wrote:
Finally, I don't want to get into a long back-and-forth on specific examples in order to debate what the right magnitude is for the APM estimates of stars, but I feel compelled to bring up one data polnt. Given your belief in the importance of stars, what do you think the APM impact should have been when Michael Jordan at 39 minutes per game was removed and Toni Kukoc at 24 minutes per game was added in 1993? That's the main difference between 92-93 Chicago and 93-94 Chicago (plus Kerr for Paxson and Trent Tucker, etc.) . The first team won 57 games (+6.3 ppg) and the second team won 55 games (+ 3.1ppg). Yes, the second team went down in the playoffs (4-3 to a team that lost 4-3 in the finals). Still, is that a big a dip as you would have expected? Now, I'm sure that you'll come back at me with various examples of teams falling apart completely once they lose their star, and I don't really want to get into a back and forth on specific examples. Regardless, I don't think the case for the enormity of the impact of stars is as clear as you're making it out to be.
Ah, you take me back to my Illinois roots and provide an interesting example. There are actually two data points to consider. The first pertains to the offense, where the drop in Offensive Rating was 6.8 and the second describes the defensive change, where the Defensive Rating improved 3.4! Who knows what to make of these things except wouldn't it be great to have APM estimates throughout history? This said, I think you misrepresent the personnel changes. Recall who was MJ's replacement, one Pete Myers, reputed defensive specialist. And then there was Toni Kukoc and Steve Kerr. Who knows how it all actually washed out? But more to the point, I am not interested in any back and forth on specific examples. I only offered up LeBron and Kevin Garnett to illustrate the general point that RAPM dramtically diminishes the stars. If what is on the back of my envelope is correct, there is virtually no chance, according to APM. that there is not at least one +13 player in the Association, but with RAPM the opposite is true. jsill wrote:
With that said, it may turn out that if we improve upon what I have further that we'll get bigger numbers for LeBron, Wade, etc. As I said, I wouldn't necessarily claim that the RAPM estimates for the stars are better than standard APM estimates for the stars (I wouldn't concede they're worse, but I wouldn't say for sure that they are better). All I can say is that taken collectively as a model for all players in the league, RAPM seems to do better.
A final thought. All players are equal (RAPM) but some are more equal than others (APM).


Page 4 of 5

Author Message DLew



Joined: 13 Nov 2006
Posts: 222
Posted: Fri Nov 06, 2009 9:10 pm Post subject:

I am pretty sure that Steve Ilardi normalizes to the minutes weighted average as well. The numbers are on the same scale assuming you have properly navigated the minutes/possessions conversion.
Back to top

Crow



Joined: 20 Jan 2009
Posts: 771
Posted: Sat Nov 07, 2009 3:45 pm Post subject:

In previous Adjusted only a couple players went above +13, up to +20 range. In Steve's 6 year Adjusted no player rates better than +10. +8 isn't much lower than that. Using DLew's template, given that a point per game in differential is worth roughly 2.5 wins, then a player who is a +13 per 100 possessions is worth about +10 per 40 minutes. So that's 24 wins better than average and roughly 40 wins better than replacement. That would mean that if +13 guy took the spot of a replacement player on any 20 win team they should hit 60 wins. But only 3 of the 6 teams the +13 guys are really on hit 60 wins last season. So those teams were worse than a 20 win team without their star? A player who is a +20 per 100 possessions is worth about +15 per 40 minutes. So that's about 37 wins better than average and roughly 60 wins better than replacement. To me that does seem unreasonably high for the best players in the league. Looking at previous Adjusted, was Wade with his +22 (at b-v) worth far more than his total team wins last season? The rest of the team would win less than nothing without him? And at 82games were Rasheed Wallace +20 and Ben Wallace +18 both worth enough to win almost 60 games each in 2005-6? Elsewhere, but somehow not there? The rest of the team would have had to lose close to 60 games more than a zero win team, if they could, to push them back to just winning 60 when all together? With his +19 in 2004-5 was Paul Pierce responsible for more than all the Celtics 45 wins? I don't know the exact right top (even though rare) but I can't see it reaching +20. That seems too extreme. And I doubt it is really as high as +13 either. +8-10 sounds about right to me. If James' rating sounds too low, I'd suggest that it might be because of the importance of their very strong team defense and the fact that his eFG% is not much more than .01 higher than the team average.
Back to top

stareagle



Joined: 19 Feb 2009
Posts: 64
Posted: Sun Nov 08, 2009 2:36 am Post subject:

Crow wrote:
Using DLew's template, given that a point per game in differential is worth roughly 2.5 wins
One small point - the win-value of a point changes as you get further away from zero, so that 2.5, while a decent rule of thumb for the first 10 points, is way off as you get into higher differentials. If you take a team that scores and allows 100ppg, their pythag record is, obviously, 41-41. Make it 100.5-99.5, and they pick up 2.9 wins. But it doesn't stay 2.9. By the time you get to +10, each extra point is only getting you 1.9 wins, and +20, each point is getting you .66 wins. Because of that, turning a 20-win team into a 60-win team would take roughly a 15-point swing. Assuming we're talking +15 for someone playing 40 minutes, that's about a +20/100 possessions. That said, I don't believe there's ever been a player anywhere near that valuable, especially if we are talking 20 points above average for 100 possessions. My data, which isn't based on plus-minus, has a phenomenal season being +14 over replacement for 100 possessions. I seriously doubt any player has ever had a season where he was worth ten points over average per 100 possessions. Maybe Russell, if you believe the high-end estimates of his blocked shots and figured out a way to give him bonus credit for the skill of maintaining possession for Boston after his blocks.
Back to top

Crow



Joined: 20 Jan 2009
Posts: 771
Posted: Sun Nov 08, 2009 3:15 am Post subject:

stareagle wrote:
One small point - the win-value of a point changes as you get further away from zero, so that 2.5, while a decent rule of thumb for the first 10 points, is way off as you get into higher differentials.
Yes I am aware of that and ran out of time earlier before I could decide what to say about that. So the estimated win conversions for the top guys under previous Adjusted +/- models might be 1/3rd less than in the bullets I wrote up. But they are still quite large for +13 guys, and especially for +20 guys as you also say. Regardless of the estimated value of the various models and relation to true value you can still see and think somewhat usefully about relative rank and general range of players though with a degree of caution and doubt that probably never completely leaves and probably never should.
Back to top

schtevie



Joined: 18 Apr 2005
Posts: 400
Posted: Sun Nov 08, 2009 7:41 am Post subject:

Crow wrote:
In previous Adjusted only a couple players went above +13, up to +20 range. In Steve's 6 year Adjusted no player rates better than +10. +8 isn't much lower than that.
The comparisons I offered of Joe's and Stephen's estimates were the three year time weighted vs. the 2008-09 six year stabilized. This is the best that could be done, and I don't imagine that adding three back year's to Joe's estimates would change the picture: there are distinct differences at the star level of "preferred one year" estimates. As for the straight, multi year averages, you didn't scroll down to the players playing less than 2000 minutes in Stephen's data. There you see that KG clocks in at just over 14, which is consistent with his stabilized rating. Whether there is greater compatibility between APM and RAPM at the level of unweighted, multi-year averages is a separate and perhaps interesting question.
Back to top

deepak



Joined: 26 Apr 2006
Posts: 664
Posted: Fri Nov 13, 2009 4:00 pm Post subject:

Offensive and Defensive APM now available: http://hoopnumbers.com/allAnalysisView? ... ssion=True Quote:
I'm surprised to see how poorly Yao Ming rates on the offensive end. He comes out as an all-star level player overall, but most of his value comes from defense, according to RAPM. I think the conventional wisdom is that he's a devastating low-post threat offensively. Is this an example of RAPM getting it wrong? Maybe so. I'm really not sure.
I can tell you, as a regular watcher of the Rockets, your results are in line with with my perception. I believe Daryl Morey has also made comments hinting at this as well, particularly Yao being underrated as a defensive presence. Observe also how the Rockets have adjusted without Yao in the early going this season. Their offensive efficiency is actually better (with Aaron Brooks -- the top offensive player by your ratings -- being the main playmaker), while the defense is so far lagging behind. That's pretty interesting how this seems to be getting it right.
Back to top

Crow



Joined: 20 Jan 2009
Posts: 771
Posted: Fri Nov 13, 2009 4:19 pm Post subject:

Thanks for these splits and versions.
Back to top

jsill



Joined: 19 Aug 2009
Posts: 73
Posted: Fri Nov 13, 2009 4:33 pm Post subject:

deepak_e: Thanks for your feedback regarding the Rockets. By the way, I'm sorry I don't have fast break leaders for you yet, but I hope to get to that at some point. Since you are a Rockets fan, let me ask you about Chuck Hayes. As you can see on my site, he comes out as a phenomenal defensive player based on my results, despite a bit of an off year last year. RAPM may well be exaggerating his defensive skills, but at this point I don't think there's much doubt that he's a very good defender. What I wonder about him is whether he has any particular physical gifts which set him apart as a defender or whether it's really mostly his smarts and willpower that make him so effective. I realize he's quite strong and very good at fighting post players for position, but I wonder if he's really that much stronger than lots of other guys, or whether there are a number of other sturdily built 6'6 to 6'9 guys who are physically capable of playing the kind of D he plays. When I look at KG or Kirilenko, they certainly have plenty of smarts and work very hard on defense as well, but they also have some rare physical traits that help them defensively (enormous wingspans and quick feet given their height). Chuck Hayes looks pretty ordinary to me, though, from a physical standpoint. Of course, I don't get to watch the Rockets a lot, so maybe a big fan has a different take.
Back to top

Crow



Joined: 20 Jan 2009
Posts: 771
Posted: Fri Nov 13, 2009 4:44 pm Post subject:

I'll note that Hayes reputedly was a weight room work freak in college at UK. His 20 reps on the draft combine bench press puts him in the top 30 of the last ten years, high but not the absolute highest, though only 3 guys ahead of him have played more NBA minutes per game (Okafor and Horford and his teammate Landry- by one rep) and most of these guys testing higher did not make or stick in the league. Bench press is only one measure of strength and leg strength or core strength is probably more important and he seems to have that too. It is more broadly about strength, quickness, basketball IQ and defensive effort (on the latter 3 he has the advantage on Landry).Last edited by Crow on Fri Nov 13, 2009 5:08 pm; edited 4 times in total
Back to top

deepak



Joined: 26 Apr 2006
Posts: 664
Posted: Fri Nov 13, 2009 4:51 pm Post subject:

jsill wrote:
Since you are a Rockets fan, let me ask you about Chuck Hayes. As you can see on my site, he comes out as a phenomenal defensive player based on my results, despite a bit of an off year last year. RAPM may well be exaggerating his defensive skills, but at this point I don't think there's much doubt that he's a very good defender. What I wonder about him is whether he has any particular physical gifts which set him apart as a defender or whether it's really mostly his smarts and willpower that make him so effective. I realize he's quite strong and very good at fighting post players for position, but I wonder if he's really that much stronger than lots of other guys, or whether there are a number of other sturdily built 6'6 to 6'9 guys who are physically capable of playing the kind of D he plays.
Hayes has two things working against him -- lack of height and leaping ability. This limits him somewhat defending shots and rebounding at the rim. Other than that, he has unbelievable defensive tools. Very quick feet allows him to cover ground laterally much better than most any other PFs/Cs. This makes him very effective in defending pick and rolls -- he can switch onto wings or he can show and get back quickly to deny the roller -- and also quickly getting into position to draw charges. His hands are quick and very strong, and he has impeccable timing, allowing him to frequently strip players as they go into their shooting motion against him. And as Phil Jackson recently observed, he's so strong that he's almost rooted to the ground like a tree stump, making it very difficult for post players to back him down. Beyond that, his basketball IQ defensively is very high. Comparable to Battier, I'd say. He made a play the other day against OKC that won't make any highlight reels, but shows how good he is defensively. He was guarding Jeff Green on the baseline. Green pump faked, getting Hayes to bite on it, and then dribbled around him looking to finish. Even though Hayes got his hand up to contest, he was able to whirl around in a split second, get to Green just as he was about to go up for the shot, and strip the ball away from him. Very, very few players would have been able to make a play like that. Hayes does it routinely. All of that might sound like gushing, but he really is a very special defensive player. I think he'd be even more effective if he was defending PFs primarily (which was the case before last season), because as I noted his lack of size limits his ability to contest shots at the rim and box out 7-footers near the rim.
Back to top

DLew



Joined: 13 Nov 2006
Posts: 222
Posted: Fri Nov 13, 2009 5:27 pm Post subject:

jsill, One quick use of these numbers that would be interesting to see would be to take them and the minutes played from last year and generate projected team point differential (and split it into offensive and defensive components if you want) and look at how well that fits the results. I assume it will do pretty well, as this is not a dissimilar to some of your previous validation work, but what that can tell us better than anything we've seen so far is whether the regularization really causes the ratings to be too compressed. If we see that all teams RAPM's sum to a point differential between say 5 and -5 then that supports schtevie's claim that the ratings are unrealistically compressed (although not necessarily proving that this is a bad thing). If the point differentials follow roughly the actual distribution, then that would suggest that the compression caused by regularization is not as significant as previously speculated.
Back to top

Crow



Joined: 20 Jan 2009
Posts: 771
Posted: Fri Nov 13, 2009 5:28 pm Post subject:

Quote:
I'm surprised to see how poorly Yao Ming rates on the offensive end..
As I note in the other thread he is one of only two in the top 20 on defense estimated as better than neutral on offense. So is that less bad for him than it sounded before?
Back to top

Crow



Joined: 20 Jan 2009
Posts: 771
Posted: Fri Nov 13, 2009 5:47 pm Post subject:

DLew wrote:
One quick use of these numbers that would be interesting to see would be to take them and the minutes played from last year and generate projected team point differential (and split it into offensive and defensive components if you want) and look at how well that fits the results.
I am glad we agree on this general recommendation. I hope your call achieves its request better than my earlier one did. Getting the team Adjusted data at the offensive / defensive split level could be even more useful to look at, think about and tinker with than the data based on overall team Adjusted values.
Back to top

Ed Küpfer



Joined: 30 Dec 2004
Posts: 783
Location: Toronto
Posted: Fri Nov 13, 2009 6:41 pm Post subject:

RE: Chuck Hayes deepak_e wrote:
All of that might sound like gushing, but he really is a very special defensive player.
Not to mention his offensive skills._________________ed
Back to top

jsill



Joined: 19 Aug 2009
Posts: 73
Posted: Fri Nov 13, 2009 7:39 pm Post subject:

Quote:
One quick use of these numbers that would be interesting to see would be to take them and the minutes played from last year and generate projected team point differential (and split it into offensive and defensive components if you want) and look at how well that fits the results.
OK, I gave this a try (quick-and-dirty, by hand, in a spreadsheet). I looked at 2 teams, the Lakers and the Wizards. The Lakers outscored their opponents by 7.6 points per game (they had an efficiency difference of 8.1). There are 48*5*82=19680 regulation player minutes in a season. I didn't look up the number of OT games but I'm going to guess 5 games with single 5 minute OTs for an extra 5*5*5=125 minutes in the season, bringing the total to 19805 minutes. I'm going to use single-year RAPM, although I might be able to do better with the 3 year results. player,minutes,RAPM,minutes*RAPM Kobe,2960,3.24,9584.48 Pau,2999 ,2.0, 5998 Odom,2316,7.35,17022.6 Fisher,2441,1.52,3712.76 Ariza,1999,-0.3,-601.7 Bynum,1448,-0.34,-486.53 Vujacic,1293,-1.21,-1567.12 Farmar,1192,-3.15,-3758.38 Walton,1166,0.67,781.22 Radmanovic,771,-1.1,-851.18 Powell,703,-5.61,-3943.83 Reference,517,-5.43 ,-1551 The 1-year results on my site currently have a reference player cutoff of 400 minutes, so that's where the last line comes from. If you sum up the minutes*RAPM column and divide by 19805, you get the minutes-weighted average RAPM for the Lakers last year, which works out to 1.17. Since there are 5 players on the floor, that means their efficiency edge relative to a minutes-weighted average NBA lineup should be 5.85. OK, so that's not so far off, but it is a couple of points lower than their actual 8.1 efficiency diff. Ah, but did the Lakers' opponents, on average, consists of average NBA players? Not quite. The Lakers didn't play the whole league. They played the whole league minus the Lakers.The majority of their games were played vs. the Western Conference minus the Lakers.If I eyeballed it right, the Western Conference minus the Lakers was about 50 games below .500 collectively, or more than 3 games below .500 on average. That should be worth about 1 point per game in margin. Maybe it's about even for their Easterm Conference opponents. Let's call it an average of -0.6 points in margin for the Lakers' opponents. That brings the predicted efficiency diff up to around 6.5 vs. the actual 8.1. Is that a little low and a little conservative? Sure. Is it way off? No. Same deal for the Wizards. They got outscored by 7.4ppg, for an efficiency diff of -8.2. Jamison 3096 -1.28 -3962.88 Butler 2585 -0.3 -778.09 Young 1837 0.23 426.18 Blatche 1703 0.11 194.14 Songaila 1521 0.28 425.88 James 1575 -3.77 -5944.05 McGee 1143 -4.66 -5321.81 McGuire 2072 -2.11 -4376.06 Crittenton 1130 -2.04 -2307.46 Dixon 816 -1.44 -1170.96 Stevenson 886 -1.4 -1235.97 Daniels 288 -5.83 -1679.04 Reference 1153 -5.43 -3459 I get a minutes-weighted average RAPM of -1.62, for a predicted efficiency diff versus average NBA opponents of -8.1. Here we have the opposite situation of the Lakers, and the Wizards' opponents were actually above average since they didn't play themselves. I don't know how it would work out, but maybe that corresponds to another half point or point of margin. So we're projecting an efficiency diff of around -9, vs. an actual of -8.2. So for the Wizards, it might even be just slightly too extreme. I think we're in the right ballpark, assuming I didn't screw something up too badly.


Page 5 of 5

Author Message DLew



Joined: 13 Nov 2006
Posts: 224
Posted: Fri Nov 13, 2009 8:30 pm Post subject:

Thanks. That is useful. If you can figure out an easy way to do the whole league that would be great. If you want to make the strength of schedule adjustment a little easier you could use SRS from basketball-reference. SRS is just schedule adjusted point differential.
Back to top

Mike G



Joined: 14 Jan 2005
Posts: 3604
Location: Hendersonville, NC
Posted: Tue Jan 26, 2010 3:07 pm Post subject:

In 2009, Lamar Odom's RAPM was 7.35. Is that per 48 minutes? or per 100 possessions?_________________` 36% of all statistics are wrong
Back to top

jsill



Joined: 19 Aug 2009
Posts: 73
Posted: Tue Jan 26, 2010 9:09 pm Post subject:

The results on my site are per 100 possessions. I've also implemented a per-48-minutes version, and the results generally look very similar as long as you use regularization.
Back to top

Crow



Joined: 20 Jan 2009
Posts: 817
Posted: Wed Jan 27, 2010 1:46 am Post subject:

At this point what is your view about the possibility of presenting half-season data, at whatever level of detail?