Well, well, well...

Home for all your discussion of basketball statistical analysis.
mtamada
Posts: 161
Joined: Thu Apr 14, 2011 11:35 pm

Re: Well, well, well...

Post by mtamada » Tue Apr 08, 2014 11:25 am

Nice to see solid results getting published, by researchers who've been working on their model for years. In contrast to Goldsberry and the guy at 538.com; they're doing good research too, but are presenting results far too prematurely, and need to work on their models more before publicizing their dubious results. xRAPM/RPM come with their own caveats of course, any one year's results have high standard errors. But both the model and the results have greater believability than some of the flaky results that others are publishing.

Mike G
Posts: 4425
Joined: Fri Apr 15, 2011 12:02 am
Location: Asheville, NC

Re: Well, well, well...

Post by Mike G » Tue Apr 08, 2014 12:32 pm

nbo2 wrote:
WAR can be thought of as reflecting the estimated impact of RPM x Possessions, by answering the counterfactual question: If all of this player's possessions this season had been taken instead by one or more players with the xRAPM impact of the typical "replacement-level" player (estimated by Daniel Myers at roughly -2.35 in his landmark analysis), how many wins would we expect this to cost his team?
Estimating replacement level is extremely important and greatly affects the relative valuation of superstars vs. stars vs. starters etc. Daniel's analysis was a good start, but I think further analysis is needed. My guess is that the best player you can sign to a minimum 10-day contract is significantly worse than -2.35.
The article doesn't define WAR, and the linked table of rankings doesn't either, except that it's Wins Above Replacement.
The table reveals that there are 158 players with >1000 minutes and >24 mpg who are positive in WAR (and > -2.35 rpm). At 5.3 per team, that's roughly equivalent to the league's starting players.

So it seems that RPM of -2.35 is the player who typically replaces a starter?
This is of course quite different than the scrub/practice player who is actually available and unemployed.
Tony Wroten and Elliot Williams total -9.14 WAR. If the Sixers would just replace their (combined) 40 mpg, they'd have 9 or 10 more wins?

colts18
Posts: 304
Joined: Fri Aug 31, 2012 1:52 am

Re: Well, well, well...

Post by colts18 » Tue Apr 08, 2014 12:53 pm

Kevin Pelton wrote:
permaximum wrote:The media and the target audience want non-prior-informed one-year RAPM. If you give them xRAPM or multi-year RAPM etc. they'll treat it as vanilla RAPM regardless.
And when they laugh it off due to the kind of fluky results we know are inevitable due to the noisiness of single-year RAPM, then what?
There is no way that J.E. could write in the article that 2014 RAPM is basically a multi year RAPM. The comment section would note that he is using a multi year stat to try to say that LeBron is the MVP this year. :lol: They wouldn't understand why its necessary.


Plus, look at this list for 2013. Its the best non-prior informed RAPM's last season

https://docs.google.com/spreadsheet/ccc ... ring#gid=0

Top 10:
Rank Name Rating
1 Amir Johnson 6.3
2 LeBron James 6
3 Tiago Splitter 5
4 Mike Conley 4.7
5 John Salmons 4.5
6 Paul Pierce 4.4
7 Vince Carter 4.4
8 Tony Parker 4.3
9 Omer Asik 4
10 Metta World Peace 3.9
11 Wayne Ellington 3.6
12 Kevin Durant 3.6
13 Mario Chalmers 3.4
14 Nick Collison 3.4
15 Pablo Prigioni 3.4


You tell me if you think a stat that has Wayne Ellington ahead of Kevin Durant or Tiago Splitter and John Salmons in the top 5 is going to gain any traction. Once people see Amir Johnson ahead of LeBron, they will laugh at the stat.

Mike G
Posts: 4425
Joined: Fri Apr 15, 2011 12:02 am
Location: Asheville, NC

Re: Well, well, well...

Post by Mike G » Tue Apr 08, 2014 1:30 pm

As it is, Kobe is shown as the #8 SG (+2.09). Derrick Rose is +2.39

Breaking it down by positions:

Code: Select all

pos    Min     WAR     mpg  W/484
 PF   111642   198.6   48.9   .86
 C     93093   141.2   40.7   .73
 SF   112156   166.0   49.1   .72
 PG   118295   174.9   51.8   .72
 SG   117754   122.0   51.5   .50

tot  552942   802.7  242.0  3.53
Here I've assumed an avg game is 48.4 minutes: There is 1 win per 484 player-minutes.
The ratio (3.53/5.00) is .706 WAR per game. It looks as though players above replacement level are getting credit for all wins, and sub-R players are removing almost 30% of the total --??

And why do the weakest positions get the most minutes?
Could RPM be used to evaluate coaches?
In Philly, SG -- Anderson, Wroten, Thompson, Williams -- have totaled 86.4 mpg, contributing -21.3 WAR
Without those 4, league SG W/484 jumps to .57

bbstats
Posts: 224
Joined: Thu Apr 21, 2011 8:25 pm
Location: Boone, NC
Contact:

Re: Well, well, well...

Post by bbstats » Tue Apr 08, 2014 2:27 pm

Very interesting that this is multi-year. Exciting stuff all around though.

I'm with Kevin's analysis (on ESPN insider) that Nick Collison is being overrated by +/- stats for RPM. (I have some ideas on why this is happening and how to fix it but I don't want to derail the discussion.)

knarsu3
Posts: 103
Joined: Thu Apr 14, 2011 11:25 pm

Re: Well, well, well...

Post by knarsu3 » Tue Apr 08, 2014 3:20 pm

Good work and great to see it finally on ESPN. Although as others have mentioned, I think it'll get misused a lot, which is unfortunate. Still, this was an important step for Analytics to have this in a public domain.

mystic
Posts: 470
Joined: Mon Apr 18, 2011 10:09 am
Contact:

Re: Well, well, well...

Post by mystic » Tue Apr 08, 2014 3:30 pm

Great seeing that published on such a big sports site as ESPN. Thanks Jerry and Steve for that.

Notes from my perspective:

- People seem to not be really able to put those numbers into perspective, despite the fact that WAR should help. I would say that it could be helpful, if the description would say that the values (RPM) represent expected scoring margin per 100 poss with the respective player on the court as well as 4 average players. Also, mentioning that an average player would get 0 should help with that.
- People are acting dismissive of the results based on perceived outliers. In my experience the majority of people seem to interpret a list with decreasing numbers from the top to bottom as an representation of going from "best to worst" (again, even though WAR is presented as well and should eliminate some of that), while in that case it should be noted that the players achieve those numbers in the respective role they are used by the coaches and nobody should expect them to do as good in a different role (e.g. Nick Collison instead of Kevin Love would likely not improve the Timberwolves playing level, if Collison is supposed to do exactly what Love does; even though it might be interesting to see a Pekovic+Collison lineup ...). People are acting as if all results should be dismissed, because some role players are ending up at the top or some of the perceived star players (although they constantly aren't showing high impact numbers over the years) are supposed to be "undervalued" by those numbers. Maybe a clarification on that is helpful?
- Clarification on the coaching adjustment would be nice. Jerry said that it is not included, the description says "Player's estimated on-court impact on team performance, measured in net point differential per 100 offensive and defensive possessions. RPM takes into account teammates, opponents, coaches and additional factors".

And a question for Jerry specifically: Someone from a german message board asked me whether I can help writing an article about it. The intention is: explaining the value of such analyses as well as introducing you to the german audience, because you are not that well-known among basketball fans in Germany. Well, I can most certainly help with the underlying math, but I'm obviously way less able to say something about the specifics of RPM nor do I know you well enough (or at all) to say something about your person (it may be also possible that you are not interesting in such publicity in Germany anyway). So, are you interested in participated and giving advice to the author (http://go-to-guys.de/Wordpress/author/alexeyshwag/) in question?

RoyceWebb
Posts: 17
Joined: Fri Apr 20, 2012 1:50 pm

Re: Well, well, well...

Post by RoyceWebb » Tue Apr 08, 2014 4:55 pm

Thanks for the input so far.

With the great guidance we've received from J.E. and Steve, we will continue to do our best to communicate what these metrics are all about to the millions in our very broad audience, made up of people of all ages, backgrounds and perspectives. For many, Monday's rollout was a starting point, their first encounter with the ideas in Steve's piece.

Naturally people will use or "misuse" stats or any other info in whatever way they want to. People see both newspaper stories and advanced stats thru whatever lenses they have handy. Or to move from a visual metaphor to an auditory comparison, the game of telephone indicates exactly how easy it is for "misunderstandings" to occur. Actually we should probably call these "variable understandings," because nobody involved in public discourse gets to decide what the precise interpretation should be.

While there are no magic words to make people use this info in the same way that more scientifically inclined people would (as if there were agreement between statheads on how to use advanced metrics!), we will continue to send the strongest signal we can through the noise. One way to do that is to communicate the stats and concepts in a variety of ways. J.E. and Steve are providing guidance on the best ways to bring these ideas to the public.

Sometimes that will mean focusing on topics of very broad interest (such as the MVP race), because those topics provide the best opportunity to inform people. Such topics are often the on-ramp for many who otherwise would not be interested in these numbers. We have to give people a way to connect with the info.

And I'll continue to watch this board for ideas on communicating the numbers and the concepts behind them.

Thanks,
Royce

nbo2
Posts: 42
Joined: Thu Apr 25, 2013 4:28 am
Contact:

Re: Well, well, well...

Post by nbo2 » Tue Apr 08, 2014 6:29 pm

Here I've assumed an avg game is 48.4 minutes: There is 1 win per 484 player-minutes.
The ratio (3.53/5.00) is .706 WAR per game. It looks as though players above replacement level are getting credit for all wins, and sub-R players are removing almost 30% of the total --??
More evidence that actual replacement level is much lower than -2.35

Mike G
Posts: 4425
Joined: Fri Apr 15, 2011 12:02 am
Location: Asheville, NC

Re: Well, well, well...

Post by Mike G » Tue Apr 08, 2014 11:46 pm

I'm not sure that one definition of 'replacement level' is absolutely more meaningful than another. But it also is confusing, to me, where the other 30% of wins are coming from. And what does it mean that players create minus-20 wins relative to some other teams' replacements?

The Sixers would love to have several nearly-average players to throw on the floor. Since they don't, who are the replacements available for the guys they do have? Basically, they are D-league level guys.

That may be the only universally applicable definition. The Spurs won't be giving such players any time, unless some disaster strikes, or they go into full rest-for-the-playoffs mode.

If replacement level is defined as that level which produces no wins, then it's applicable to all teams. And WAR will = team wins and league wins.

Crow
Posts: 6206
Joined: Thu Apr 14, 2011 11:10 pm

Re: Well, well, well...

Post by Crow » Wed Apr 09, 2014 2:00 am

The by position splits are appreciated. It would also be helpful to add minutes per game filter and age and / or experience. Although the explanation uses the word "estimate" a number times late
it isn't that prominent and the size of the estimated errors is not mentioned. Furthermore I think carrying these estimates out to two decimals runs against the needed awareness that these are rough estimates. I would encourage rounding to nearest 0.5 or even presentation of players in say 5-7 tiers. If one really wants to promote fair understanding and reduce criticism about overstated precision, then it might be good at once to show some form of confidence levels. So maybe show a list of players with a best estimate over plus 4 and a percentage of how likely the true value is above that level. And the same for between plus 4 and 2, 2 and 0, etc.

knarsu3
Posts: 103
Joined: Thu Apr 14, 2011 11:25 pm

Re: Well, well, well...

Post by knarsu3 » Wed Apr 09, 2014 6:25 am

Crow wrote:The by position splits are appreciated. It would also be helpful to add minutes per game filter and age and / or experience. Although the explanation uses the word "estimate" a number times late
it isn't that prominent and the size of the estimated errors is not mentioned. Furthermore I think carrying these estimates out to two decimals runs against the needed awareness that these are rough estimates. I would encourage rounding to nearest 0.5 or even presentation of players in say 5-7 tiers. If one really wants to promote fair understanding and reduce criticism about overstated precision, then it might be good at once to show some form of confidence levels. So maybe show a list of players with a best estimate over plus 4 and a percentage of how likely the true value is above that level. And the same for between plus 4 and 2, 2 and 0, etc.
Yeah I would like to see confidence intervals as well. But I don't think we need both ends. Just something along the lines of lower bounds for each player (since thats what most people care about). So we can say 95% confidence that x players are above +4 (which would be their lower bound), +2, or whatever splits work.

Crow
Posts: 6206
Joined: Thu Apr 14, 2011 11:10 pm

Re: Well, well, well...

Post by Crow » Wed Apr 09, 2014 11:30 am

There are several ways one could go. But given what I assume about the estimated errors based on past reporting by JE, I might cite players with at least a 60-70% (or 67%) of being over a certain value (or under a negative one). The 95% confidence level seems too demanding for this circumstance. It would keep the level cited (exceeded or below) closer to the best estimate than using 95% confidence interval would. The goal IMO is understand the most likely level, with an understanding that one level higher or lower are the next but much lesser likelihoods.

DSMok1
Posts: 904
Joined: Thu Apr 14, 2011 11:18 pm
Location: Maine
Contact:

Re: Well, well, well...

Post by DSMok1 » Wed Apr 09, 2014 8:05 pm

Mike G wrote:I'm not sure that one definition of 'replacement level' is absolutely more meaningful than another. But it also is confusing, to me, where the other 30% of wins are coming from. And what does it mean that players create minus-20 wins relative to some other teams' replacements?

The Sixers would love to have several nearly-average players to throw on the floor. Since they don't, who are the replacements available for the guys they do have? Basically, they are D-league level guys.

That may be the only universally applicable definition. The Spurs won't be giving such players any time, unless some disaster strikes, or they go into full rest-for-the-playoffs mode.

If replacement level is defined as that level which produces no wins, then it's applicable to all teams. And WAR will = team wins and league wins.
Replacement level is the level where you can get players for minimum salary "off the street". It's not related to wins directly. -2.35 per player would be -11.75 as a team, which would be worse than all but a couple of teams in history, but would still be expected to win some games.

Reasons players below replacement level would be played:
1. They are in a bad context; would be at or above replacement level in another context
2. They are projects, rookies to be developed. Teams are willing to give 1st and 2nd year players time when a vet with similar quality would never play.
3. Bad judgement by the team.
4. The team is trying to lose. Ahem, Philly (playing the second-worst player in the last 14 years, Byron Mullens, significant minutes).
Developer of Box Plus/Minus
APBRmetrics Forum Administrator
GodismyJudgeOK.com/DStats/
Twitter.com/DSMok1

Mike G
Posts: 4425
Joined: Fri Apr 15, 2011 12:02 am
Location: Asheville, NC

Re: Well, well, well...

Post by Mike G » Wed Apr 09, 2014 10:46 pm

I think a team of 'players off the street' would never win, unless they ran into a team that was resting all their better players; and they'd be outscored by a 2:1 ratio rather often.

There's actually an enormous difference between 'cheap available players' and those in rotation for the worst NBA teams.

Post Reply