Coach RAPM

[J.E.]

http://stats-for-the-nba.appspot.com/ra ... aches.html

Coaches that are currently coaching an NBA team are marked in light blue.


For this analysis coaches were simply treated as a 6th man on the court. All 13 years of data were used for one large (ridge) regression that estimates both player and coach impact on offensive/defensive efficiency, while assuming that every player ages according to the aging curve. Data from the 2000-2001 season until Saturday, 02/08/14.

This method cannot evaluate whether coaches play the 'correct', i.e. 'better' players. A coach that chooses to play a bad player over a good player, but makes the bad player perform slightly above his average, will get a positive rating.
Also not evaluated are things like 'keeping "older" players out of back-to-backs so they have a lesser chance of injury'

[schtevie]

Very happy to see a new set of coaching RAPM numbers, but strange things have newly arisen.

Could you please remind us of the different approach taken between this set and the last one provided, as the results produced are rather different. For example, the range of estimates, to my recollection, is now more than twice as high (as least on the positive side). And then there are the particular estimates. Whereas before, these generally made sense, now not so much (e.g. by way of general confirmation, coaching salaries were generally in line with players of similar productivity).

As for particular curiosities, we can infer, for example, that OKC owes more than its entire success to its coach, as the franchise, apparently, is only blessed with well below-average talent (-2.8 being the simple average for player contributions).

Similarly, we learn from these that poor ol' Phil Jackson had to endure but (slightly below) average talent for his 2001-11 run with the Lakers.

At least Pops has had some guys who could play (non-Pops contributions being +3.6) but if you subtract what we believe Duncan is valued at, plus Ginobili, and Parker, man, what a terribly run front office they must have, because the other guys they fielded must then have been absolutely terrible!

And so on.

What I would like to see are regression results derived similarly to the xRAPM posted on the site (and by all means add the aging factors) with coaches starting off with a prior of 0. I am supposing that this would generate rather more plausible results.

[Jacob Frankel]

I suspect Scott Brooks is stealing a little bit of credit from the OKC player development staff. Durant, Westbrook, and Reggie Jackson have made big leaps that aren't covered by the typical aging curve, and Jeremy Lamb was in the D-League last year and is now a solid role player.

[J.E.]

I suspect Scott Brooks is stealing a little bit of credit from the OKC player development staff. Durant, Westbrook, and Reggie Jackson have made big leaps that aren't covered by the typical aging curve, and Jeremy Lamb was in the D-League last year and is now a solid role player.

Yeah, with this method I'm not able to say whether a player gets better because of coaching or because of the player development staff. With Brooks we can only tell when he leaves OKC and doesn't take his staff with him. In a fantasy world with more data I could throw assistant coaches and player development coaches into the regression.
I think in OKC's case it's a mix of many things: Management is bringing in good players, player development coaches do a good job, and the coach does a good job.

Brooks would not have this high of an estimate if players had not performed significantly better playing for him, compared to how they performed with other coaches (including but not limited to Collison, Durant, Perkins, Fisher). I'm guessing a large chunk of Brook's rating comes to Collison's development. Collison has had decent +/- numbers before Brooks got there, but then the +/- numbers just took off, plus Collison was already 28 when Brooks started coaching in OKC

[DSMok1]

Cool work, J.E.! This works way better than the previous version without the aging curve accounted for.

[schtevie]

If these results well approximate reality, not only does Scott Brooks have the worst agent in the world, but (every)one is obliged to fundamentally recalibrate their NBA world view, variously perceptions of player values and front office competences.

Again, I ask: what is the difference in approach that brought such different results this time around? And why should the latter be preferred?

Another result to ponder: the interesting case of Jeff Hornacek. He pops up as second best, based on a partial year's work, and is shown to be the greatest offensive coach of the past (almost) 14 years! Does this make sense? Well, in a sense it does. He entered the league and did something different offensively that (unsurprisingly) worked: he had his guys who could shoot threes shoot a lot more of them. And the credit for this "innovation", apparently, obtains to him. But this will inevitably and before too long be copycatted away. And this argues, I would argue, for an approach which begins with a zero prior and imposes a strong regression to the mean.

And a slightly wicked, Parthian shot: these results that suggest that Phil Jackson had but average talent over the latter half of his career are broadly consistent with the results of some Prof. Berri regressions based more on his tenure with the talent-free Bulls. Just sayin'.

[DSMok1]

Reminder: sample size for this is still incredibly small, since players change teams only rarely and coaches even more rarely. Only probably 15 to 50 observations of player with or without given coach for most coaches.

[schtevie]

Cool work, J.E.! This works way better than the previous version without the aging curve accounted for.

Daniel, what do you mean by this? In what day does this "work better". Are you of a belief that adding an aging curve would have dramatic influence on estimates of coaching RAPM?

I look at Jeremias' aging curve and see but very small annual changes. The median rookie comes in at age 22 and ascends to his peak at a rate of something like 0.2 points per 100 possessions, with similar rate of descent until much later in his potential career.

And my way of thinking about this (what could well be incorrect) is that in a regression, with an aging curve added, coaches are the recipients of the net of all these small age-related changes realized in any given year. But these typically net to approximately zero. I have written previously of Gregg Popovich being a likely "winner" of adding an aging curve (and indeed he is a big winner in Jeremias' v2.0). But even he, who has always had a relatively old lineup, shouldn't see much of a benefit. I went to B-R and looked (and sorted) the Spurs' minutes played in 2012-13. By coincidence, these happen to roughly pair up, top to bottom, with players pre and post peak. What this tells me (correctly or not) is that these aging effects should sum to only a really small negative number (conversely a small fillip to Popovich's rating).

Not that Jeremias is interested in extra work, but it would be interesting to see the net effects of just having added the aging curve. Unless I am more confused than usual, this isn't what's driving the very significant changes from v1.0.

[J.E.]

schtevie, there is indeed one more adjustment built in here. I wanted the person who came up with it to get back to me first before I made the nature of the adjustment public. Once he does I'll probably make a post about it.
The quick and dirty version: Teams that are behind tend to score more than expected* points and vice versa. As far as I can tell there's a linear relationship between 'size of lead' and 'effect on offense'/'effect on opponent defense'. Popovich's lineups spend a lot of time in the lead, and now with the adjustment, are required to score less than average points to be rated average (because an average lineup, in the lead, usually doesn't score an average amount of points. They score less)

*given who's on the court

[bbstats]

It would be nice to assign the values by something simpler like "organization." Basically the exact same algorithm except the names would be the team, and but wouldn't change variables between coaching changes. We could then compare the two.

[J.E.]

Hold it. Apparently I did something wrong when determining the different penalty values for players, coaches (offense and defense). Re-running the numbers right now. New results will likely differ

Edit: False alarm. I ran multiple OOS tests to check whether lambda for coaches should be significantly different than lambda for players. The results suggest that lambda for impact of coaches on offense should be a little higher, i.e. coaches have a little less impact on offense than players do, but the difference between the lambda for players and the lambda for coaches isn't very big. The results pretty much match with what I found several months (years?) ago when I ran this analysis the first time

In the event somebody ever wants to run this analysis himself, here are the lambdas I found through OOS-testing

Players, offense: 2000
Players, defense: 3000
Coaches, offense: 3500
Coaches, defense: 2500

The differences are so small that it's barely worth mentioning; if someone would run it with one single penalty value for off/def + players/coaches the output is not going to be significantly worse

[J.E.]

It would be nice to assign the values by something simpler like "organization." Basically the exact same algorithm except the names would be the team, and but wouldn't change variables between coaching changes. We could then compare the two.

I could potentially throw in 'organization' variables just as I did with coaches and then, through OOS-testing with different penalty values, figure out if those variables are best always set to zero or something different

[schtevie]

Hmmm... Then something remains very, very wrong with these results, at least those at the top of the table. To repeat: if they are correct, it implies massive market failure AND the requirement that everyone recalibrate their beliefs about player values and front office competence.

Market failure: if the best coaches are as productive as suggested (and never mind the particular estimates, look just at the range) they are underpaid by millions and millions of dollars. Consider Phil Jackson, who was the highest paid coach of all-time (I believe) at $10.3 million. If his average value was +4.4, players making a similar contribution are approximately the 15th best player in the league (on xRAPM terms) and the 15th highest salary in the league at the time was about $6 million more. And these salaries are artificially lowered by the salary cap. And as for Scott Brooks' implied underpayment... hoo boy!

There are few people around who like to celebrate the dysfunction of NBA front offices (and coaching staffs) more than me, but ultimately I have some faith in market mechanisms (when allowed to function) and there is no particular impediment in the labor market for coaches that I am aware of.

Recalibrating beliefs about players: I have already made light of the implied below-average Thunder talent and Phil Jackson's average team having being strictly mediocre, but these are not exceptions, they are the rule. Go down the list, starting from the top, and for all "elite" estimates (reflecting more than a partial year's employment) there is a common factor, none of these coaches had good teams, talent-wise, with the exception (as previously noted) of Gregg Popovich. And in many of these instances, we "know" that there was exceptional talent or at least have strong reason to believe that there was well above-avearage talent.

Recalibrating beliefs about front office competence: Returning to the exceptional Gregg Popovich and the Spurs, again, if Pops is indeed the coach represented (+3.3) and we believe that Tim Duncan has averaged about a +8 since 2001 , and Ginobili +4, and Parker +3 (I'm just guessing about all these, but the point should be clear) this implies that all non-named Spurs would have to have been simply terrible on average. Given these assumptions, it is impossible to consider the Spurs to have been a well-run franchise. And a very similar story applies to the very well-respected Thunder, just substitute Brooks, Durant, Harden, and Westbrook, and you must have one of the worst supporting casts of all time.

So, I return to my initial question: what accounts for the very large differences between http://stats-for-the-nba.appspot.com/ra ... aches.html and what used to be on the site? I admit to not understanding the implications/arguments of assigning "penalty values". Is that what's driving the changes? And if so, could this be explained a bit more? Conversely, if I do understand the implication of adding an aging curve, this shouldn't do much of anything (at least for coaches of long tenure) as the aging curve is very flat between ages 26 and 29 which is the average age range within which essentially all coaching tenures fall.

Is there another factor explaining the differences between v1.0 and v2.0?

Finally, I return to the idea of a zero prior. This, to me, has very intuitive appeal. Unlike players, who earn rents on their physical stature and athleticism, coaches have a skill-set, most of which ultimately can be acquired and where innovations can be copied and competed away. Hence, one really should think of significant deviations from zero to be exceptional. The previous set of estimates had this characteristic; this one doesn't.

[Bobbofitos]

I think Coach RAPM is measuring what it intends to measure very, very well.

However, I don't think coaching RAPM measures how good a coach is.

[bbstats]

I second the idea of a zero-prior.

Further, it would be helpful to *average* player development. KD and RWB are getting most of the bump here, but they get a ton more minutes. To see how coach x (or front office x) truly improves players, it might help to *NOT* weight by minutes played.

Edit: Other possible modes of coach analysis: singling out player bumps in RAPM after trades, PPP out of a timeout on offense and defense, (somehow) regressed Wins minus Expected wins (i.e. pulling out close ones...people will probably fight me on that one )