https://x.com/JerryEngelmann/status/175 ... 93367?s=20
Jerry, have you compared player RAPM with and without Coaching presence as variable in the mix? If so or if willing, could you show the player values side by side?
Do all players get influenced at the same rate or does it vary? Coaching impact on superstars, other starters and bench?
Any interest in doing a 3-5 year run with just recent Coaches? How does the change "in the game" affect recent Coaches?
Would you recommend an ambitious RAPM generator to adjust by Coaching? Especially given your observation about large coaching impact on defense.
Can Coaching impact on defense by Coach or in general be separately estimated for interior / perimeter players or even interior / perimeter plays?
How is Coaching impact separated from luck or influenced by it (especially if not luck-adjusted)?
Any interest in trying to separate Coaching impact from GM? I'd think that would be interesting to me, GMs... and owners.
Any chance that Coaching impact could be split into "system", minutes allocation and lineup management and perhaps other elements?
Impact by quarter? Home & away? Start and end of seasons? Playoffs only? By opponent quality (split into 2 or 3 tiers)?
What about trying to highlight / isolate impact of top 10 or 20 players by doing splits of on / off court, on / off team?
Or RAPM splits for lineups with average age of 27+, 24-27 and under 24? For players and for Coaches.
Bigger than average lineups vs small, extremes on 3pt and at rim frequency?
I know the general arguments against cutting sample size but potentially important potential discoveries to identify, think about, pursue further.
What about 1 on 1 RAPM runs for prominent Coach against next Coach cases?
East / West splits for players and for Coaches. Where do they best belong? Maybe should influence the actors or the folks hiring them.
Should rebounding be adjusted? Why or why not?
Referee RAPM impacts? Overall and by team and by factor?
RAPM impacts of being on a >.650 win% contender (last season or in current season) vs. not?
I've had many RAPM split ideas over time but most of these are new.
What RAPM splits have teams requested or have had your personal interest? If willing to say.
If you get back into publishing current or 3 year RAPM, could you be encouraged to run player pairs? Which type of pairs most outperform and most underperformed simple sum of individual RAPM? Do star trios in general over or under perform expectations?
Do you value this 40 minutes of brainstorming? Do you think there are teams that would or might?
Coaching RAPM for top 25-50 NCAA coaches (overall or against each other or top 50-100 teams)?Average RAPM change by Coach from college to NBA for drafted players or first round picks?
Coaching RAPM (& more splits and variations)
Re: Coaching RAPM (& more splits and variations)
Public feedback or engagement on possible / possibly informative RAPM splits and variations continues to be lacking for many years. No team has expressed private interest in the ideas either. Oh well. Missed opportunities for doing some things that are probably new. New / different being potentially very valuable.
Anyone gets interested in considering this area of research or detailed lineup analysis and management or analytics-based draft boards, can send a private message.
Anyone gets interested in considering this area of research or detailed lineup analysis and management or analytics-based draft boards, can send a private message.
Re: Coaching RAPM (& more splits and variations)
Interesting stuff from Jeremias, and not being on Twitter, I am grateful to Crow for calling attention to the work, what seems very worthy of comment and discussion.
Summarizing the coaching results, we see:
Offense Defense Total
Best 2.4 -4.8 6.6
Worst -3.3 5.2 -6.1
Range 5.7 10 12.7
stdev 0.84 1.91 2.21
What makes sense to me about these data is the relative ranges for offensive versus defensive coaching contributions, the latter being close to twice the former.
To my conception, offense is much more cookie cutter, with little diversity as to what would be considered best practice within whatever range of talent endowment. Though I should qualify this (as this is really a historical document) to add that this is past best practice that was considered socially acceptable within the period in question.
I am supposing these data, and their dearth of standout offensive contributions (relative to defensive) are explained in part by the collective failure of the coaching fraternity to take proper advantage of the three point shot on any reasonable time scale. Wonderful!
By contrast, perhaps defense allows for more diversity in terms of expressing best practice and plausibly is benefited by the intensity and inspiration of individual coaches (a bit more on this later).
What doesn’t make sense to me, however, are the ranges, so let’s talk about that.
If one is to take these at face value, one would need to believe that were an “average” coach (i.e. one who, by definition, has a rating of 0) to have taken Phil Jackson’s place in LA in 2006-07, that this team, featuring Kobe Bryant and Lamar Odom, would have been the worst team in the league (by implied Net Rating). And on the other side of the table, were Tim Floyd to have been replaced by an “average” coach at his final gig, the 2003-04 New Orleans Hornets, instead of being a 0.500 team, it would have been vaulted to the top three in the East (essentially tied for first, again, implied by Net Rating).
Neither of these counterfactuals quite ring true to me.
If the ranges indicated by these data are too high, what might be the explanation?
One basic check is that according to these results, NBA coaches on average are above average! The average coach Net Rating in these data, weighted by games coached, is 1.14. This must be an artifact of them being thrown in as the “6th player”, so the coaching average is not forced to zero by the estimation process (just like the case of “Centers”, “Point Guards”, or any other position designation, where there is no expectation that any position should have a Net Rating of zero on average). But theoretically and conceptually, basketball is a zero sum game, and the average contribution of coaching should net to zero. How imposing this constraint (if indeed possible in the estimation process) would change the distribution of results, I don’t know, but I suspect that it would shrink the ranges and lower the standard deviations.
But I think there is another much more important factor that biases the results, and this relates to the fact that coaching contributions are very much a function of the total of the endowed on-court talent.
On the low end, not only does the empirical record include a lot of teams that tank (which explicitly biases their coaches’ ratings down, and those of their opponents up) but I reckon that more generally, getting yelled at to work harder on defense (however much a good idea) tends to fall on deafer ears when the expense of such effort is only to result in “minor” improvements of competitive relevance.
I’d be curious to see the results of the Coaches RAPM rerun with one minor revision to the data set. I would propose that starting in 2019-20, the San Antonio Spurs be assigned a new coach Pop Greggovich, this season corresponding to the Spurs, at long last, not having ridiculously high talent levels on their team. And I wonder how Coach Greggovich’s rating would compare to his predecessor’s.
And then there is a separate class of benefits for those blessed to coach teams with greater talent endowments. On above-average talent teams, all else equal, coaches can utilize players with greater efficiency, allowing them to do more of what they do best and less of what they don’t (compared to the skill sets available in their above-average teammates).
Related to this, in whatever measure, is the fact that such above-average teams (especially those contending for championships) are better able to attract players, specifically willing to “sacrifice their game”.
Again, to belabor the point, to whatever degree the coaching rich get richer by this efficiency effect, the poor necessarily get poorer, with the ratings of less fortunate colleagues falling commensurately. And this has nothing to do with anyone’s “innate” coaching ability (such as would be expressed on an team of average talent).
Anyway, this is all very interesting, greatly enjoyable to ponder, and hopefully refinements to plus/minus estimates of coaching contributions will be made in future.
Summarizing the coaching results, we see:
Offense Defense Total
Best 2.4 -4.8 6.6
Worst -3.3 5.2 -6.1
Range 5.7 10 12.7
stdev 0.84 1.91 2.21
What makes sense to me about these data is the relative ranges for offensive versus defensive coaching contributions, the latter being close to twice the former.
To my conception, offense is much more cookie cutter, with little diversity as to what would be considered best practice within whatever range of talent endowment. Though I should qualify this (as this is really a historical document) to add that this is past best practice that was considered socially acceptable within the period in question.
I am supposing these data, and their dearth of standout offensive contributions (relative to defensive) are explained in part by the collective failure of the coaching fraternity to take proper advantage of the three point shot on any reasonable time scale. Wonderful!
By contrast, perhaps defense allows for more diversity in terms of expressing best practice and plausibly is benefited by the intensity and inspiration of individual coaches (a bit more on this later).
What doesn’t make sense to me, however, are the ranges, so let’s talk about that.
If one is to take these at face value, one would need to believe that were an “average” coach (i.e. one who, by definition, has a rating of 0) to have taken Phil Jackson’s place in LA in 2006-07, that this team, featuring Kobe Bryant and Lamar Odom, would have been the worst team in the league (by implied Net Rating). And on the other side of the table, were Tim Floyd to have been replaced by an “average” coach at his final gig, the 2003-04 New Orleans Hornets, instead of being a 0.500 team, it would have been vaulted to the top three in the East (essentially tied for first, again, implied by Net Rating).
Neither of these counterfactuals quite ring true to me.
If the ranges indicated by these data are too high, what might be the explanation?
One basic check is that according to these results, NBA coaches on average are above average! The average coach Net Rating in these data, weighted by games coached, is 1.14. This must be an artifact of them being thrown in as the “6th player”, so the coaching average is not forced to zero by the estimation process (just like the case of “Centers”, “Point Guards”, or any other position designation, where there is no expectation that any position should have a Net Rating of zero on average). But theoretically and conceptually, basketball is a zero sum game, and the average contribution of coaching should net to zero. How imposing this constraint (if indeed possible in the estimation process) would change the distribution of results, I don’t know, but I suspect that it would shrink the ranges and lower the standard deviations.
But I think there is another much more important factor that biases the results, and this relates to the fact that coaching contributions are very much a function of the total of the endowed on-court talent.
On the low end, not only does the empirical record include a lot of teams that tank (which explicitly biases their coaches’ ratings down, and those of their opponents up) but I reckon that more generally, getting yelled at to work harder on defense (however much a good idea) tends to fall on deafer ears when the expense of such effort is only to result in “minor” improvements of competitive relevance.
I’d be curious to see the results of the Coaches RAPM rerun with one minor revision to the data set. I would propose that starting in 2019-20, the San Antonio Spurs be assigned a new coach Pop Greggovich, this season corresponding to the Spurs, at long last, not having ridiculously high talent levels on their team. And I wonder how Coach Greggovich’s rating would compare to his predecessor’s.
And then there is a separate class of benefits for those blessed to coach teams with greater talent endowments. On above-average talent teams, all else equal, coaches can utilize players with greater efficiency, allowing them to do more of what they do best and less of what they don’t (compared to the skill sets available in their above-average teammates).
Related to this, in whatever measure, is the fact that such above-average teams (especially those contending for championships) are better able to attract players, specifically willing to “sacrifice their game”.
Again, to belabor the point, to whatever degree the coaching rich get richer by this efficiency effect, the poor necessarily get poorer, with the ratings of less fortunate colleagues falling commensurately. And this has nothing to do with anyone’s “innate” coaching ability (such as would be expressed on an team of average talent).
Anyway, this is all very interesting, greatly enjoyable to ponder, and hopefully refinements to plus/minus estimates of coaching contributions will be made in future.
Re: Coaching RAPM (& more splits and variations)
The standard deviations for offense and defense estimated coaching impact are interesting and should probably influence most new hire decisions.
"Greggovich" in 5 seasons "has" no playoff appearances, just 2 10th place finishes and is headed for 2nd straight last place in west. 3rd worst cumulative win% among active Coaches behind B Keefe and C Billups. Most / nearly all teams would fire a Coach with that 5 year record.
I guess I forgot to mention / ask for Coaching RAPM factors.
"Greggovich" in 5 seasons "has" no playoff appearances, just 2 10th place finishes and is headed for 2nd straight last place in west. 3rd worst cumulative win% among active Coaches behind B Keefe and C Billups. Most / nearly all teams would fire a Coach with that 5 year record.
I guess I forgot to mention / ask for Coaching RAPM factors.
Re: Coaching RAPM (& more splits and variations)
Has JE published anything to corroborate the predictive value of this approach above not including coaches? I'd be surprised (but pleased) if it has any actual predictive value.
Re: Coaching RAPM (& more splits and variations)
Coaching has so few "stints" I have always doubted the predictive value.
Re: Coaching RAPM (& more splits and variations)
A grand total of 2 coaches in the database have a +1 estimated impact on offense (George Karl and Kenny Atkinson).
The best defensive coaches tend to have not much estimated impact on offense and probably lose jobs and don't get re-hired forever because of the later impression, subjective or objectively riforous. Impact on offense is highly sought and rarely quenching, at least by this analysis.
These ratings don't seperately account for impact of front office including analytics. IF an analytics shop or analytics shop unleashed / empowered contributed +0.5 on offense it would be more than the estimated total coaching / management impact on most teams.
The best defensive coaches tend to have not much estimated impact on offense and probably lose jobs and don't get re-hired forever because of the later impression, subjective or objectively riforous. Impact on offense is highly sought and rarely quenching, at least by this analysis.
These ratings don't seperately account for impact of front office including analytics. IF an analytics shop or analytics shop unleashed / empowered contributed +0.5 on offense it would be more than the estimated total coaching / management impact on most teams.
Re: Coaching RAPM (& more splits and variations)
2 of top 7 recently active Coaches dismissed, 1 missed in play-in, 4 leading series.
Kerr was 14th active on list when assembled.
Mazzulla ranked 27th overall?? Weird to me because the offensive and defensive splits combined produce a quite different result.
Kerr was 14th active on list when assembled.
Mazzulla ranked 27th overall?? Weird to me because the offensive and defensive splits combined produce a quite different result.
Re: Coaching RAPM (& more splits and variations)
Redick and Carlisle ranked almost in bottom 3rd of actives. Billups by far the lowest.
16 actives above +0.1. 4 over +2, 4 more over +1.
Udoka tied for tops? Will be watching upcoming game and any beyond or not.
Atkinson 3rd. Daigneault 7th.
16 actives above +0.1. 4 over +2, 4 more over +1.
Udoka tied for tops? Will be watching upcoming game and any beyond or not.
Atkinson 3rd. Daigneault 7th.
Re: Coaching RAPM (& more splits and variations)
4 second round coaching battles with see different matchups by head coaching experience and RAPM ratings.
Measuring impact in these series will be subjective... unless someone offers an "objective" evaluation approach / implementation. Perhaps expected wins based on net margin, though that isn't fully satisfying and could be subject to pretty major small sample variance.
Perhaps a lineup approach could be tried.
Actual - (Sum of playoff minutes times "expected" lineup performance (from regular season average, average against top 10 or specific matchup at least for a handful of lineups then an aggregatuin of the rest) / minutes). But how much of difference is over-performance vs randomness? And how much is coach vs. players?
Measuring impact in these series will be subjective... unless someone offers an "objective" evaluation approach / implementation. Perhaps expected wins based on net margin, though that isn't fully satisfying and could be subject to pretty major small sample variance.
Perhaps a lineup approach could be tried.
Actual - (Sum of playoff minutes times "expected" lineup performance (from regular season average, average against top 10 or specific matchup at least for a handful of lineups then an aggregatuin of the rest) / minutes). But how much of difference is over-performance vs randomness? And how much is coach vs. players?
Re: Coaching RAPM (& more splits and variations)
Billups estimated last of active coaches but extended. Rajakovic 2nd to last. Keefe 3rd to last. Nurse 26th place, Kidd 25th. Lue 22nd and slightly negative. Redick 20th and neutral.
Top 10 actives all defensively biased on impact.
Coaching RAPM estimates appear to be updated given the newest coaches included. If so, trend data would be of interest, yearly and maybe monthly.
Playoff only would be small sample for most, but would be another dimension if wanting to consider coaches, especially the long serving, big names.
Coaching RAPM has few stints... but Coach - Player pairs would have lots of stints and might be worth looking into across a multi-year database, comparing the different coaches in the player pairs (and applying an age adjustment?)
Could Coaching RAPM be applied at lineup level? Sum of Player average RAPM (perhaps multi-year) + Coach as lineup builder as the difference? Splits of how offensive and defensive biased lineups and / or coaches do against each combination? What appears to work better and worse at league level?
Turn lineup data into sum of player factors + coaching impact via lineup construction?
Put player - player pairs and coach - players into model along with player only and coach only? Do offensive and defensive splits on all these? Why not?
Not easy or sure to be illuminating but more could be tried.
Coach vs top 2-4-8 teams / coaches? As information to be considered in hiring decisions? Has anyone done that? Contenders "should" consider. Not sure but don't recall hearing Coaching matchup W-L records cited, much less very most used lineup matchup data.
Daigneault 9-9 career against Carlisle, modestly worse than his overall record. Adjusting for team strength would next level.
Coaching RAPM by itself or with other data could be used to determine coaching types, for what uses and potential value that might have.
In addition to top 10 coaches all being defensive biased, the middle 10 is too by 6 of 10 with a tie, or 2 to 1. Bottom 10 is offensively biased in these estimates by 6 of 10. That gives a trend in a continual direction.
I see a few of these comments in my earlier posts but perhaps stated differently. Repeating reinforces interest. I also rediscoveed some suggestions previously made and forgotten.
Top 10 actives all defensively biased on impact.
Coaching RAPM estimates appear to be updated given the newest coaches included. If so, trend data would be of interest, yearly and maybe monthly.
Playoff only would be small sample for most, but would be another dimension if wanting to consider coaches, especially the long serving, big names.
Coaching RAPM has few stints... but Coach - Player pairs would have lots of stints and might be worth looking into across a multi-year database, comparing the different coaches in the player pairs (and applying an age adjustment?)
Could Coaching RAPM be applied at lineup level? Sum of Player average RAPM (perhaps multi-year) + Coach as lineup builder as the difference? Splits of how offensive and defensive biased lineups and / or coaches do against each combination? What appears to work better and worse at league level?
Turn lineup data into sum of player factors + coaching impact via lineup construction?
Put player - player pairs and coach - players into model along with player only and coach only? Do offensive and defensive splits on all these? Why not?
Not easy or sure to be illuminating but more could be tried.
Coach vs top 2-4-8 teams / coaches? As information to be considered in hiring decisions? Has anyone done that? Contenders "should" consider. Not sure but don't recall hearing Coaching matchup W-L records cited, much less very most used lineup matchup data.
Daigneault 9-9 career against Carlisle, modestly worse than his overall record. Adjusting for team strength would next level.
Coaching RAPM by itself or with other data could be used to determine coaching types, for what uses and potential value that might have.
In addition to top 10 coaches all being defensive biased, the middle 10 is too by 6 of 10 with a tie, or 2 to 1. Bottom 10 is offensively biased in these estimates by 6 of 10. That gives a trend in a continual direction.
I see a few of these comments in my earlier posts but perhaps stated differently. Repeating reinforces interest. I also rediscoveed some suggestions previously made and forgotten.