APBRmetrics

** Moved out of RPM Thread **

Discussion of Statman's WAR: http://hoopsnerd.com/

*************************

Crow wrote:Statman, if RPM and WAR are close, at top or in general, that has meaning to me.

The old notion of having a website with all the one metrics side by side for players on a regularly updated basis doesn't seem unreasonable (assuming no copyright issues?) Hope to see it someday.

I know Crow, I'm not jumping on you. I'm just worn by the debates that have been happening. I've been posting results regularly that, at this moment, are quite similar to JE just produced. JE is instantly lauded the moment he puts results up. BPM comes out - I'm immediately told it's the best box score metric out there. People are on board. BPM has some wonky results that don't seem to fit eye test - doesn't matter, it's better than any box score metric. BPM doesn't seem to match RPM well - doesn't matter - they're both better than anything else someone who doesn't have their work on a large site could do - even if that metric correlates closely to either (or seems to "blend" the results).

Well, my metric mimics JE's results about as well as one could expect without using ANY +/- data. Not once, ever, unless my pity party is clouding my judgement - do I remember any member on this board commenting about how solid my results look. I've been on this board many years. I feel like many posters don't really look at results - they just say hey, it's JE or DeanO (farther back) - let's praise the results! This other guy has been posting very similar results for weeks - naw, don't need to look, don't give a shit, who's he - he can't land a gig at ESPN. Hell, his work probably doesn't even measure up to PER, or WP for God's sake!

By the way - my work mimics JEs results here better than BPM (I can see that just by glancing) - so which is it? Is BPM supposed to be more telling of true player value - or RPM?

Sorry, reading all the posts on this board, and what assumptions it seems people have about other people's work without any real knowledge (or even ever looking at their work I think), has beaten me down.

Final point, I agree with you Crow, having these metrics all appear together would be cool, at the very least for easier reference & to see where most of the general agreements & disagreements may rise player to player.

Again, sorry about the poor me attitude. Just tired.

DSMok1 wrote:Statman, for what it's worth, I don't doubt that your WAR may be as good as BPM, I don't claim BPM to be perfection. I would like to see more testing. BPM/ASPM just happened to be picked up by Basketball Reference simply because it's been around a bit longer and has had more testing published, I guess.

No problem - again, the general mood of this board in the recent debates has made me irritable.

I actually kind of feel like Neil asked for my work as maybe kind of a joke (he stated something like "the proof is in the pudding" when I said I thought my results were as good as any out there - maybe not expecting I'd take him up on his offer). I immediately sent him all the results back to 1980 - & then everyone else started jumping on board. Part of me wonders if Neil is like "oh crap - he sent me results AND a bunch more metrics want in? I don't have time for this!"

He's posted little since then - maybe he's worried if he speaks up there will be 50 metrics created wanting to be part of the mix?

I was looking forward to what he comes up with when he gets to it - but like I lamented today, I'm not sure it matters unless the "right" metrics to some on this board come out on top. From everything I'm reading - my work appears to much less "in sample" than about all others that have been argued, fwiw. Mine wasn't created test out well in a specific type of testing - it was created to try to best quantify (& qualify when I want to be more specific) player performance from the beginning of the NBA until now. I've been very pleased with the results every time I update any current or past seasons or leagues (I did it for all past D-League results on bball reference for my D League to NBA player conversions). I LOVE doing the college player ratings/rankings - it opened my eyes to MANY great players in the past who ended up being household names as their college careers progressed (like, say, Steph Curry as a frosh).

Now, I fully expect my metric will correlate well however tested - even if compared to other metrics that maybe are created to beat such a specific test (if that's what's happening, I really don't know or really care).

The point is, I'm not (ever) throwing something together - this has been a lifelong passion of mine that I've decided to devote REAL time to the last couple years. I love doing it. I can't wait to do the college player rankings in a few weeks when the team ratings settle - just to see the results and to see what players I was unaware of that I want to catch on tv.

So, when it seems like I get a collective "meh" from others on the board who probably never spent one second looking at my work, or others seemingly acting like they could throw something together in 25 minutes that will obviously be better than what I (or others) have cultivated & feel I kinda perfected (relatively speaking) over 25 years - well, it kinda sucks.

Have you posted an explanation anywhere of how your metric is calculated or what the general theory behind it is? I've taken a look at your site but I all see is the outputs, not the inputs, so I don't really know what I'm looking at. That's not to try and throw stones at your metric, but one reason you might be eliciting a "meh" reaction is because we have no means by which to evaluate it (and perhaps Neil's testing, if and when it's performed, will change that). It's certainly your prerogative if you are intending to keep the calculations private, but perhaps some explanation of the theory behind it could enlighten us without giving everything away.

Now if you have posted something and I've just missed it, feel free to correct me, I'd be happy to read more about it

Statman wrote:
Now, I fully expect my metric will correlate well however tested - even if compared to other metrics that maybe are created to beat such a specific test (if that's what's happening, I really don't know or really care).

The point is, I'm not (ever) throwing something together - this has been a lifelong passion of mine that I've decided to devote REAL time to the last couple years. I love doing it. I can't wait to do the college player rankings in a few weeks when the team ratings settle - just to see the results and to see what players I was unaware of that I want to catch on tv.

So, when it seems like I get a collective "meh" from others on the board who probably never spent one second looking at my work, or others seemingly acting like they could throw something together in 25 minutes that will obviously be better than what I (or others) have cultivated & feel I kinda perfected (relatively speaking) over 25 years - well, it kinda sucks.

I agree with the main point in the post above.

I also think I see that you haven't entered the team win prediction contest in last 4 years, or did I miss something? If you published at your site but not here, you could give us the links and calculate the avg. errors and those results could be an addition to the public knowledge and interest.

I agree that praise is spread unequally. Being a first mover, a widely known person with lots of friends or a great marketer can really help. It is talent, opportunity, marketing, hustle and luck in some ratio, not necessarily properly balanced.

It is tough to get to every site, regularly.

We converse about your metrics occasionally. I think sometime in past year I expressed concern about level of defensive weighting. Your metric - your choice, but I was somewhat disappointed you declined to adjust the defensive weight. But don't recall exactly why you declined to do so.

By the way, I tweeted a basic plug of your site and twitter. Already got a retweet. I know I gained a lot of followers from a tweet from talking practice. Others can tweet and retweet sites and article and tweets they like to help other community members as they desire and have time. Marketing isn't my strength or focus either but sometimes it needs to be part of the strategy if a plan is going to work well.

http://hoopsnerd.com/

Will look at shortly.

Dan, a couple of things:

- Regarding your favorite metric: On internet boards, people comment a lot more on points of criticism, rather than on points of agreement. When nobody comments on my stats, I assume they pass the eye test and need no comment.
Of course, I could be wrong; but then I'll never know.

- Regarding correlations with RPM or xRAPM: Seeing the weird outliers in BPM -- mediocre rates for Nash and Moses; high marks for known mediocre players -- I, too, was "sure" my eWins would correlate better. But it turns out, it doesn't. Not even close.
I also was pretty confident I could back off some of the weights Daniel uses, try some new parameters, and create a 'better' BPM -- designed to correlate with RPM. Again, I could not.

Apparently, BPM picks up a lot of what other boxscore metrics have missed, that correlate with real plus-minus. I'm not quite ready to stop doing eWins in deference to it, but it's already easier to just look it up on b-r.com than in my own files. And when I started posting, there was no such thing as 'advanced stats' online; I had only my own files to reference. So that is a pretty big realization, that the day is finally just about here.

The best thing about APBRmetrics is the spirit of cooperation and sharing. When there's less cooperation, it's less good. Sharing of criticism is the activity that cuts through the fat and gets to the muscle of metrics. We may assume there's fat in every endeavor, and hopefully at least a sliver of lean.

Mike G wrote: I also was pretty confident I could back off some of the weights Daniel uses, try some new parameters, and create a 'better' BPM -- designed to correlate with RPM. Again, I could not.

Apparently, BPM picks up a lot of what other boxscore metrics have missed, that correlate with real plus-minus.

I'm too tired to respond to the other posts for now, but I'll respond to this.

The RPM JE put out together today correlates much better at first glance to my results than it does to BPM - at least on the top end. I might test it tomorrow to know for sure, I might not - I have a lot on my plate, and I'm not sure how much it really matters. I'll update my WAR ratings tomorrow at the very least, I always have them in a spreadsheet when I update, if someone else is curious & wants to look into it.

jmethven wrote:Have you posted an explanation anywhere of how your metric is calculated or what the general theory behind it is? I've taken a look at your site but I all see is the outputs, not the inputs, so I don't really know what I'm looking at. That's not to try and throw stones at your metric, but one reason you might be eliciting a "meh" reaction is because we have no means by which to evaluate it (and perhaps Neil's testing, if and when it's performed, will change that). It's certainly your prerogative if you are intending to keep the calculations private, but perhaps some explanation of the theory behind it could enlighten us without giving everything away.

Now if you have posted something and I've just missed it, feel free to correct me, I'd be happy to read more about it

Statman wrote:
Now, I fully expect my metric will correlate well however tested - even if compared to other metrics that maybe are created to beat such a specific test (if that's what's happening, I really don't know or really care).

The point is, I'm not (ever) throwing something together - this has been a lifelong passion of mine that I've decided to devote REAL time to the last couple years. I love doing it. I can't wait to do the college player rankings in a few weeks when the team ratings settle - just to see the results and to see what players I was unaware of that I want to catch on tv.

So, when it seems like I get a collective "meh" from others on the board who probably never spent one second looking at my work, or others seemingly acting like they could throw something together in 25 minutes that will obviously be better than what I (or others) have cultivated & feel I kinda perfected (relatively speaking) over 25 years - well, it kinda sucks.

A big part of the problem is my piss poor website & design - I just don't care about building websites. I have never posted exact calculations, but I have explained how in general multiple times deep in that site how I compile the ratings.

I first normalize for league across all box score stats to make sure I can compare fairly across leagues & eras.

I then run linear weights formulas (weights in general are closer to statistical PM than PER - the weights organically settled through the years) at the player level AND team level. Player ratings are then devised relative to team ratings (team quality is included) - this step adjusts for team pace & such. These production "ratings" are then adjusted up or down due to playing time relative to team quality. High minute/low production guys will see a pretty good increase in their final ratings relative to the initial production rating - low minute/high production guys just the opposite. These adjustments tend to bring the ratings much closer to a RPM type result - high minute/low production guys on good/great teams in general are doing things not seen in the box score that add value, my ratings adjust to try to better capture this. Anyway - the results are HnI & HnR. HnI ignores missed games & only looks at playing time relative to team when the player plays. HnR looks at player PT relative to team on the season as a whole. 100 is league average.

WAR uses HnI, sets 80 at replacement level, and based on rating & playing time divvies up the team wins to the players. I assume every player is at least replacement level no matter what their rating is - so any players with HnI lower than 80 get 0 WAR (NBA players shouldn't have a lower WAR than me). This works well for adjusting for high usage guys who maybe have their rating inflated a little because of so many low usage teammates - with no negative WAR for the low usage guys the high usage guy will have his WAR suppressed a bit. This doesn't happen as much for players with "better" teammates. Anyway - the WAR I post is projected over 82 games (since teams are always at different points of the season, makes the comparison even).

Now, I know this isn't nearly specific enough to really understand how the results are garnered - but honestly I don't want to be any more specific. This approach works great for college - since I compile everything at the team level (with an appropriate team rating)- then the player ratings are created, automatically adjusting for pace/SoS/etc.

I wasn't gonna post any more tonight, and once again it's defeating the purpose to post this in some random thread that will eventually fall to later pages - but oh well. I guess I just didn't want jmethven to think I just pulled the ratings out of my behind region.

I meant to say it earlier but guess I forgot. I think your site is clean & sharp. Nothing wrong with that.

Adjustment between team and player performance and for minutes played also done in different ways by BPM and EW. Value in commonality and uniqueness.

Pwople are always asking what goes into metrics because they arrived late, forgot , things change, etc. For you, Mike, Daniel, JE, DeanO, Hollinger, etc. Best things are a prominent link to a summary of methodology that doesn't get lost and patience. I am sorry I didn't remember all the specifics and called for it again. I know you've been open about things. I'll link this page to the other metrics section of threads worth reading.

Back out first week or two and would Lilard since by vying for #1 or 2?

Never thought B Knight would be sitting where he is. How much credit to Kidd?

I just merged in the discussion of Statman's WAR that was occurring in the RPM thread, since it was rather off topic.

DSMok1 wrote:I just merged in the discussion of Statman's WAR that was occurring in the RPM thread, since it was rather off topic.

Thanks, the merge makes more sense.

I appreciate the thread Crow. I wasn't trying to pull a poor me tantrum to get attention that for some reason I thought I was due or something (I'm not due anything), but I have put years of thought & hard work into what I do and am proud of the results whether they are seen (or even appreciated) by many others.

It would probably help me if I worked for another site - and I didn't have to worry at all about website things. I have no patience for it. The college basketball editor at SN contacted me - maybe some of my work may end up over there. I told her it is so much easier for someone like me whose mind is on 15 different projects at once to have deadlines & even an editor - I focus like crazy when I have true deadlines.

Also Crow - I failed to submit NBA team projections (again) because I just didn't make the time, & then the season started. I was kinda agonizing (& never really resolving) on how to project individual player minutes - working on some other things - and then it was too late. If I had ANY idea this board would have taken the turn it did on metrics & their comparisons & their correlation to actual results - I probably would have made sure I had a submission. That's on me.

I am sensitive to the lots of work / knowledge, uncertain level of respect argument.

Hope the college outlet comes thru to your liking. If you wanted a similar outlet for NBA there might be something out there. One possibility- might talk to bballbreakdown about perhaps supplying stat analysis on college to supplement their possible film breakdowns. Or maybe get them to link to or post your NBA ratings and analysis.

No worries about the contest thing. Was just running down the possible ways to compare statistical quality. Heck, you could make the playoff prediction thread in spring if you want. You indicated you did well looking at that last spring. Or 2015-16 rolls around eventually.

HoopNerd, I've scrolled through your website and stats multiple times, but I couldn't find any data on how accurate they were (in terms of correlation to current and future team success), so I ignored them. If your numbers have excellent in-season accuracy, and can predict future performance as well as ASPM/BPM or even (gasp) XRAPM/RPM, I'd use them regularly. If your numbers can do it without using prior-seasons and artificial adjustments (all rookies getting an automatic negative value), I'll be your #1 fan.

More info on how you calculate your numbers will also be appreciated, cheers.

http://hoopdon.weebly.com/

HoopDon wrote:HoopNerd, I've scrolled through your website and stats multiple times, but I couldn't find any data on how accurate they were (in terms of correlation to current and future team success), so I ignored them. If your numbers have excellent in-season accuracy, and can predict future performance as well as ASPM/BPM or even (gasp) XRAPM/RPM, I'd use them regularly. If your numbers can do it without using prior-seasons and artificial adjustments (all rookies getting an automatic negative value), I'll be your #1 fan.

More info on how you calculate your numbers will also be appreciated, cheers.

http://hoopdon.weebly.com/

Well, stay tuned, I'll just have to see how comfortable I am about how much I'll reveal in terms of calculations. I might just, at the very least, project the next season of every single past season (since 1980) in terms of production per minute (adjusted for pace/blah blah blah) - & since we know the actual minutes played of every guy in their following seasons, plug in the projections tied to actual minutes and see how the compiled projected team results compare to actual results. I haven't done this because, honestly, no one has ever asked until recently about the predictive nature of the results until the last couple of days - & I don't know how much people trust someone "testing" their own work & giving results. But obviously it's something I should do & try my best to do a comprehensive write up to show the honesty behind the results.

Now, the problem with this is that there has to be some trust in my testing my own projections. I think if I give ALL results of every player season projection from the previous season (all the good and all the bad) in a massive spreadsheet - then it'll be pretty obvious that I'm trying to show full disclosure and not fudging things on the fly. There will be a ton of hits from player to player, but obviously a bunch of misses too. It'll be interesting to see how close, in general, the projections are & where the standard deviations fall.

Similarly - I plan on projecting the NBA careers EVERY SINGLE D1 player since 1997 & ranking them all in terms of their NBA career projections. I'll limit the posted results to just guys that project, at some point in a made up NBA career, to be above replacement level. This is a huge project (getting all the DoB's for all guys that could project above replacement level in a NBA career might be a serious pain - since a good number might not ever appear on basketball-reference) - since I'm not going to try to do what it appears everyone else does & limit my scope to the 80 or so college guys on NBA draft radars (often just for simplicity, but sometimes masking the flaws of their own metric) - but EVERY guy. How much can my methodology be trusted if I pick & choose which college players I apply it to. A good methodology should give solid predictive results on any & every player. If I limit my data set to avoid players (say Javon McCrea last year - I noticed a couple draft projection metrics ignore him, I'm guessing possibly because he measured out too high) who might break the system, then what's the point? I want to catch that undrafted guy - the Udonis Haslem & the like.

The good thing is, when I finish the above project, I'll be able to rank every college prospect for every past season - & see if my rankings (without ANY scout, combine, or prospect rank bias) compare to the actual drafts. Since I did already do the college ratings of the last season of every guy that played at least 1 minute the following season as an NBA rookie (in order to create the conversions) - I am pretty confident the projections could very well out perform the actual draft positions. Most "busts" (occasional caveats - Beasley for example projected great) were obvious bust material by the projections. Most sleepers (2nd round studs - Boozer, Blair, etc) were very highly projected - MUCH higher than many of the guys drafted before them (Jarnell Stokes in this last draft would be a comp to those two btw). I can't get them all right by any means (Bledsoe) - I just think I can do better than the general draft history of the past gms. I wanted to do all this before this last draft - never got anywhere close. Just producing college ratings for every single guy that played in the NBA as a rookie was majorly time consuming (I have to compile all the college ratings at the team level for all those guys). I will have all this done well before the next NBA draft - come hell or high water.

Also, another good thing about the draft project - I could run similarity scores with all past college players since '97 - see which players who appear most similar to guys that succeeded in the NBA. I could run similarity scores of every projected NBA season to all NBA seasons at every age group since 1980, see if certain player types are being overlooked. This might be a 2nd approach I may eventually merge with the first if it improves the predictability. Have zero idea if I'll have this fully workable by the next NBA draft. If I were working for a team, it would be because all other projects would probably be back burnered.

So, to answer your point before I went off on a massive tangent, yes HoopDon - I should run all the past projections and show how much they correlate to actual results, & post all the results. I'll post on this board & on Twitter when I do if you are curious about the results.

If you are curious about how to apply the predictive power of the player ratings (if you trusted them at all) - scroll back to any of my numerous March madness game write ups. I predict the final score of in each write up based on predicted player minutes. Here's the write up I did for the NCAA championship game:

http://hoopsnerd.com/?p=424

Note that my general team ratings going into that game had Kentucky winning by 1.1 points. But, when I compiled the optimized lineup player ratings, & with Kentucky without Cauley-Stein - my prediction came out to UConn by 1.3. I surmised going into that game that losing Cauley-Stein cost UK, theoretically, 2.1 points.

Hey - I forgot that I attempted to quantify how good Jack Taylor (the guys that scored over 100 points twice in a college game) really was relative to an average D1 player. I think some of my methodology & how it adjusts for team & pace (that is an EXTREME team in terms of pace) is evident there. I never saw anyone even dare to try to quantify his play. That was the article before the NCAA championship game. Good times. http://hoopsnerd.com/?p=415

[EDIT] I just posted this on my blog, since I never seem to write there anymore, & there are almost certainly others like HoopDon who might want to know the same thing.