How Many NBA Teams Use Adjusted Plus-Minus?

[ilardi]

Hi All,

While talking with a sports editor for a national publication today, I mentioned in passing that a number of NBA teams now have the in-house capability of generating adjusted plus-minus (APM) numbers in some form (whether it be APM, RAPM, xRAPM, ASPM, etc.).

In fact, I suspect the number of such teams may be as high as 20. Has anyone ever looked into this issue systematically? I know teams like Phoenix, Houston, Dallas, Philly, Memphis, and Washington have definitely generated some version of APM in-house, and I've heard indirectly of several others, but would be immensely grateful for any leads you might have on: (a) which teams do or don't have APM capability?; and (b) the total number of teams that have used (or at least looked at) APM in some form or fashion?

Thanks in advance,
Steve Ilardi

[schtevie]

Steve, having absolutely zero information regarding the specific question you asked, I think I can say with a great deal of confidence that, effectively, all of them do.

The primary (vast majority of the potential?) benefit of plus/minus analysis is to provide a basic framework for player valuation, particularly of elite players. As such, it takes relatively few teams (and agents) having access to such estimates to get market prices approximately right (however many to offset the rigidities imposed by the labor agreement).

I've recently begun taking a look at the relationship between extreme (positive) offensive and defensive ability (as measured by J.E.'s xRAPM) and salaries, and what looks to be the case is that some time between the first years of the 2000s and the last few seasons, both the variation in the cost of a "point per 100 possessions" has significantly diminished and the "irrational" premium paid for offensive (vs. defensive) ability has basically gone away.

The lowest-hanging fruit now gone forever.

[Crow]

Steve, your list missed NY (Winston) and Cleveland (Rosenbaum and Lewin).

[Crow]

I've recently begun taking a look at the relationship between extreme (positive) offensive and defensive ability (as measured by J.E.'s xRAPM) and salaries, and what looks to be the case is that some time between the first years of the 2000s and the last few seasons, both the variation in the cost of a "point per 100 possessions" has significantly diminished and the "irrational" premium paid for offensive (vs. defensive) ability has basically gone away.

The lowest-hanging fruit now gone forever.

I'd be interested to hear more about this analysis and findings. I do not agree that the lowest hanging fruit is now gone forever. When trades and free agent signings happen I still see a lot of overpaying for boxscore offense and lower than justified interest in some higher RAPM guys (like Milsap).


NBA insiders have said so little in public about their valuation and use of APM that I still have my doubts how much it is being used even by the 6=10 teams who are recognized for having potential access to self-generated numbers. I really would like to see ESPN or somebody dig into this deeper or have Sloan do a survey panel on the topic beyond just receiving the periodic paper. ESPN has really not done much with APM before or after adding Dean Oliver; and while Sloan has helped give the concept visibility, it has not done much of anything to clarify its level of usage or encourage greater usage.

I have the impression that few if any head coaches have shown much interest in study and use of it. A few owners and GMs have but even they do not seem to be consistent about listening to what it says. I have seen decisions in Cleveland, Houston, Phoenix (and maybe others) that fly in the face of what RAPM estimates.


I haven't had the computing power or the time to learn how to produce RAPM myself yet but I have a dozen or more ideas for not seen in public variations in the use of the basic technique that I would be willing to discuss with a team or a free agent analyst some time if anyone ever wanted to.

[ilardi]

Thanks, Crow. That wasn't meant as any sort of comprehansive list, which (as you note) would certainly have to include teams like Cleveland and NY, as well as Portland. And then there are the other dozen or so teams that have advanced analytic staff that are certainly capable of generating APM-based numbers if they were ever interested in doing so.

And while I agree with Schtevie that some of the lowest-hanging fruit (vis-a-vis market inefficiencies in player valuation) has now disappeared for good, I also concur with your view that there are still key inefficiencies to be exploited by the teams who know how to do so.

[talkingpractice]

I'm pretty confident that most teams (maybe 5-6 stragglers?) now have some sort of inhouse rapm capability, and/or purchase player values from an outside consultant of some sort.

[ilardi]

I'm pretty confident that most teams (maybe 5-6 stragglers?) now have some sort of inhouse rapm capability, and/or purchase player values from an outside consultant of some sort.

Do you have any evidence - e.g., conversations with team insiders, Sloane chatter, etc - or is that just conjecture? As I mentioned above, I strongly suspect the number of teams may be at least 20, but I only have actual evidence for about 10 . . .

[talkingpractice]

I'm pretty confident that most teams (maybe 5-6 stragglers?) now have some sort of inhouse rapm capability, and/or purchase player values from an outside consultant of some sort.

Do you have any evidence - e.g., conversations with team insiders, Sloane chatter, etc - or is that just conjecture? As I mentioned above, I strongly suspect the number of teams may be at least 20, but I only have actual evidence for about 10 . . .

To be fair, it is indeed just conjecture and chatter with insiders. But way, way more people than just you and I strongly suspect that most (nearly all) teams have some sort of apm now. I mean, any team without it probably wouldn't want ours due to the wagering connection, but we'd give all of our player values (spm, npi rapm, spm informed rapm, whatever) every morning for free to any team that asked (costs us nothing, hurts us in no way), just for the coolness/fun of doing so. Just due to things like this, it seems to me that the teams would have to practically go out of their way to *not* have rapm data internally.

[Crow]

Put adjusted plus minus into the search engine at ESPN and you get back 14 references in 8 years.

http://search.espn.go.com/adjusted-plus-minus/

If perhaps 20 teams have it and are using it, I'd think there would more questions to ask, more things to discuss even if teams are not cooperating. But if teams are using it, why not acknowledge it? I recall far more team officials saying they don't think much of APM or don't use it all than anyone who will admit to using it or valuing it.



Since I have relatively little expectation any team will actually inquire with me privately about other APM variations, I guess I share the ones that came quickly to mind briefly here:

RAPM Splits for:

First / second half of games
First / second half of season
First groups of plays in the first 4 minutes played by a player, then 4-8, 8-12+ minutes in a stint
For top 1 or 5 team lineups vs the rest
When with or against a top 10 player or PG or shot blocker; or a “real center”, a great 3 pt shooter, etc.
When against a top 20 lineup in the league
When against a top 5-10 opponent overall, on offense, defense, 3pt frequency, inside shot / FT frequency, etc.
Against top, middle, bottom top pace teams or for actual top, middle and bottom pace games
Home / away
Playoffs only
When against “small ball” / big lineups in general and when specific players have x advantage / disadvantage on height, weight, wingspan, NBA experience, etc.
East / west
In games with above, below and near normal foul calls
For plays made in less than 8 seconds, 8-16, and 17- 24+ seconds
In games when main star is performing above, below average or near normal
With three or more starters or 3+ bench players
On weekends vs during week (to catch party impact?)
At high altitude
On plays with more or less than x passes
On plays with offensive rebounds
Starting / off bench
By position played
At high, low and near normal usage or touches
Against man to man / zones
When up or down by x pts
On play after a turnover (on offense or defense)
When they use the possession and receive the first pass past half court or by location of their first reception or the ball ever goes into the post vs not
In lineups with low, middle and high minutes of pair or whole lineup experience together

(In the absence of RAPM splits for these things, one could check the raw play by play data but the signal would not be as clean.)

Other RAPM variations

For refs down to RAPM factor level
For coaches down to factor level
League-wide average RAPM results for “similar” players or lineups as a group on any of many different criteria (to possibly get clues from much larger samples than given for one individual or lineup)
RAPM shot impact broken down to impact on specifically the 3 pt game, inside shots and FTs.



Of course these splits reduce sample size, but you could and probably should use multiple seasons of data to increase sample size back up some (might as well run every year separately too). Findings might not be statistically significant but some very high, low or just surprising for that player results may raise questions that prompt further view of tape and more thinking. Could also run scenarios that involve of two+ of the above ideas.

Some of these may not be considered that important by everyone, but I think there is a rationale for checking all or almost all of these. If you have the capacity to run the data, I'd suggest that an organization with revenue that might touch 9 figures and reaching for a title in a highly competitive arena might want to look at at least some of them to see what additional detail they might learn about players and lineups and what other questions they might uncover along the way. It might not take too long in most cases to produce the numbers with the right technical staff. There is probably far more time to be spent in the analysis of the results than the production. There is also value in proposing what to look at next. How many teams have previous thought of or run and used any of these 3 dozen variations? How many even look at factor level RAPM? I wish I knew. If the number is low, then all the low or mid-level fruit is not yet picked clean and there is still plenty of room for information and analytic advantage.

[schtevie]

...And while I agree with Schtevie that some of the lowest-hanging fruit (vis-a-vis market inefficiencies in player valuation) has now disappeared for good, I also concur with your (Crow's) view that there are still key inefficiencies to be exploited by the teams who know how to do so.

Steve, I would welcome elaboration on your views about key inefficiencies and their ability to be exploited. Taking an economic perspective, I see only relatively minor inefficiencies remaining and few opportunities for their exploitation.

To begin, what I infer from the terminology is that "key" inefficiencies imply that substantive improvements can be made on existing PM estimates (RAPM with statistically generated priors and aging curves). And I don't get that sense. I am inclined to believe that there is perhaps one important, yet modest, refinement in the offing, separate estimates for "tweener" positions, and that this will significantly affect the valuation of a relatively small number of players. But such information will want to be free, just like xRAPM is now, and that information will very quickly be priced into the market. And if so, we remain where we are (or will be very soon): the distortions of price ceilings for elite players aside, salaries tied closely with value, hence few (and minor) opportunities to be exploited.

What am I missing?

[Crow]

While those who rely totally on non-APM techniques might acknowledge the piece meal nature of their detailed stats and might, when forced to face it, acknowledge the incompleteness or inaccuracy of their roll-up boxscore-based stats with regard to shot defense (30-35% of the game), it is rare to hear them also face the incompleteness or inaccuracy of their roll-up boxscore-based stats related to a specific player impact's on the offense of teammates beyond assists and offensive rebounds. Add that to what is missed on defense and the total that meaningful impact missed by roll-up boxscore-based stats probably exceeds 40%. The common refrain is that APM is too unreliable... and then many turn back to the incomplete and inaccurate boxscore based impressions and the eye test without the same level of concern.