Coach RAPM

[J.E.]

Through 31 games, they were 10-21 and -5.6 ppg.
After that, they went 33-11 and were +2.9 ppg -- an improvement of 8.5 ppg.

Those +2.9 aren't great either, considering how many games they were expected to win (Vegas said 53), who was on their roster and given that they were playing in the weakest division of the entire league. I don't think they were unlucky with player injuries, either, considering the age of their roster

[Statman]

Really? You put together a rating system on coaches that, oh, has Brian Shaw ahead of Pops...Rick Carlisle as a negative....and Mike B as very very negative (only took an injury ravaged team to the playoffs, and almost knocked off a #1 seed...), JKidd as very very negative (only overcame terrible start and injuries to knock off the Raptors in the first round despite no HCA...) and you feel comfortable acting as though your ratings are *authoritative* ???

Part of the reason "analytics" gets a bad name and has the problems it does getting past the initial resistance is when people fall in love with crappy ratings systems they have devised and are blind to the flaws and weaknesses. You're really going to take a stance that your 'coach ratings' should be viewed as the best way to assess coaches?

Well, I think you pretty much would be talking about RAPM in general (and ESPN's sister Real +/-) if you are going to use that type of reasoning - since the results ALWAYS have player outliers that general fans (& gms? & coaches?) would cringe at. I assume Roland that you may now see RAPM & the like as "crappy" rating systems because of the outliers?

It doesn't matter to me - I don't try to calculate RAPM, I have no stake in this. I think RAPM for players can be helpful as an obvious outside the box-score tool, but not as a be all end all. I would look possibly at this the same - MAYBE there's some insight to be seen if the limitations (sample size?) are understood. Maybe it can be "improved" - but again, I have no idea since I (yet?) don't delve personally into this approach.

Isn't Roland Ratings named after Roland Beech? Isn't that sort of the precursor to player RAPM (using +/- as as rating system)? Roland Rating definitely had results that often didn't match common perceptions. Interesting Roland would jump in here on this. I do know here in DFW Rick Carlisle is looked at as a GREAT coach - but with possibly his biggest flaw not trusting young players. It is hard for me to look at this last Mavs roster (& how almost all their big minute older guys should be in decline from their previous season) and seeing solid their record ended up being (and their taking the Spurs to 7 games in round 1) and not think Carlisle was probably a pretty big positive.

[Mike G]

Using the Coach RAPM in the link by J.E., here's one way to use the numbers: I've multiplied it by the number of games they've coached, for seasons and playoffs, and added a factor of 94/100 (avg possessions per game?)
This should give us roughly their 'added' points in the course of their coaching career.

Code: Select all

.coach      rapm     G    PO    pts+/-  PO     T
Popovich     2.8   1410  239    3711   629   4340
Brooks       5.6    463   73    2437   384   2822
Rivers       2.2   1142  134    2362   277   2639
Adelman      1.3   1791  157    2189   192   2380
Thibodeau    3.3    312   39     968   121   1089

Vogel        2.4    267   54     602   122    724
Casey        2.1    352    7     695    14    709
McHale       1.9    324   12     579    21    600
Jackson      2.0    230   19     432    36    468
M Brown       .7    563   83     370    55    425

Williams     1.4    312    6     411     8    418
Hornacek     5.1     82    0     393     0    393
Shaw         3.8     82    0     293     0    293
Clifford     3.0     82    4     231    11    243
D'Antoni      .2    881   59     166    11    177

.coach      rapm     G    PO    pts+/-  PO     T
Woodson       .2    680   46     128     9    136
Stevens      1.1     82    0      85     0     85
Malone        .5     82    0      39     0     39
Corbin        .1    258    4      24     0     25
Joerger      -.1     82    7      -8    -1     -8

Loyer       -2.3     32    0     -69     0    -69
Vaughn      -1.1    164    0    -170     0   -170
Wittman      -.4    520   11    -196    -4   -200
B Brown     -3.0     82    0    -231     0   -231
Budenholzer -2.8     82    7    -216   -18   -234

Kidd        -2.9     82   12    -224   -33   -256
Spoelstra    -.5    476   99    -224   -47   -270
Drew        -1.0    312   24    -293   -23   -316
Carlisle     -.4    968  110    -364   -41   -405
Stotts      -2.0    447   16    -840   -30   -870

. avg         .8    421   41     443    56    499

Actually I don't know if the rapm include playoffs, or if there is a separate measure for that. I've applied the same one to all games.
If Kidd is 'really' better than -3 ppg, he'll chip away at his alleged deficit.
If Brooks is 'not really' as good as his +5.5, his surplus will melt away or not accrue so fast.

But isn't 'Coach' being used as a proxy for 'all else'? It might be the culture of a city -- nurturing like OKC or Portland, vs corrosive like NYC, LA, or Miami.
Smart vs dumb owner, GM, training staff, assistants, facilities, etc...

[J.E.]

Interesting Roland would jump in here on this.

That's not the "real" Roland Beech. People who talk like this on internet message boards don't have jobs with teams

Actually I don't know if the rapm include playoffs, or if there is a separate measure for that

I do include playoffs (same weight as regular season)

If Kidd is 'really' better than -3 ppg, he'll chip away at his alleged deficit.
If Brooks is 'not really' as good as his +5.5, his surplus will melt away or not accrue so fast

That's both true, but moreso for Kidd than Brooks. Because Kidd coached only a fraction of the games that Brooks did, Kidd's rating is going to be more volatile

But isn't 'Coach' being used as a proxy for 'all else'? It might be the culture of a city -- nurturing like OKC or Portland, vs corrosive like NYC, LA, or Miami.
Smart vs dumb owner, GM, training staff, assistants, facilities, etc...

All true. It's hard to get data on assistants and training staff though. I can try to add a 'city'-proxy to the analysis and check whether that improves OOS prediction results.

Adding Coaches to the analysis certainly did improve OOS prediction results, so it definitely has its' usefulness

Obviously no one should use it as a bible, but maybe the fact that the coach that you're currently running after is ranked 2nd to last among active should give pause for concern, especially when even the simpler metrics (Wins, Point Differential, given the roster) agree

[Crow]

It is possible someone used Roland's name to open an account and barely used it for 3 yrs (slight), but at this point I'm assuming it is Roland.

[Bobbofitos]

It is possible someone used Roland's name to open an account and barely used it for 3 yrs (slight), but at this point I'm assuming it is Roland.

that accounts seems fairly aggressive, i actually doubt it is the real roland.

[Mike G]

Everyone has a bad day.
Then again, I'd like to see a breakdown, if possible, of player RAPM changes that have occurred under Carlisle; and their weight in determining the rapm assigned to him.

He was in Detroit in '02 and '03. These were Ben Wallace's best years, statistically.
Cliff Robinson was there both years. He dropped off significantly upon coming from Portland, and dropped off a bunch more going to GS.
But these are just looking at WS/48 and PER, etc.

Pistons improved by 18 wins when Carlisle got there; improved by 4 when he left
Pacers -- '04 to '07 -- improved by 13 W in his first year, and by 1 after he left.
Mavs -- '09 thru '14 -- lost 1 win (and -3 in SRS) when he arrived.

[J.E.]

This was posted Jan. 15 and shows that BRK had the 2nd easiest SOS after that date

The Bucks, apparently, traded two 2nd-rounders for Kidd. Seeing how both of those are going to be in the 31-35 pick range I dislike the hiring even more


As for Carlisle, here's a breakdown for (age adjusted) player RAPM for players that have played significant minutes for Carlisle, and for other coaches

Code: Select all

╔════════════╦═══════════════╦══════════════════╗
║            ║ With Carlisle ║ Without Carlisle ║
╠════════════╬═══════════════╬══════════════════╣
║ Granger    ║ -1            ║ 4                ║
║ Marion     ║ 1.5           ║ 6                ║
║ Terry      ║ 1             ║ 4                ║
║ Nowitzki   ║ 9.3           ║ 8.8              ║
║ Cliff Rob. ║ 2.2           ║ 0                ║
║ Tinsley    ║ 1.5           ║ 1.5              ║
║ Harrington ║ -1.2          ║ 1                ║
║ Kidd       ║ 6.2           ║ 6.7              ║
║ Atkins     ║ 0             ║ -1               ║
║ Artest     ║ 7             ║ 5                ║
║ Carter     ║ 7.6           ║ 6                ║
║ Wallace    ║ 4.4           ║ 5.2              ║
║ Oneal      ║ 3.2           ║ 3.2              ║
║ Jackson    ║ 1.6           ║ 1.1              ║
╚════════════╩═══════════════╩══════════════════╝

For many players it's not a big difference. Cliff. Robinson, Artest and Carter were ~2 points better under Carlisle.
Terry, Marion and Granger were 3, 4.5 and 5 points better without Carlisle though.

Marion was alot better in Phoenix, as he fit their System really well. Carlisle didn't have Nash and thus couldn't run as much, so I'd say he gets a little too much of the blame, but the difference in Marion's rating is rather large. Marion was obviously older (31+) under Carlisle but the decline was steeper as the aging curve would suggest. I haven't heard anything about him having major injuries, either

Granger pretty much blossomed into a star as soon as Carlisle left Indiana. He was young under Carlisle, but not extremely young - he played his first two seasons being 22/23 years old

[Mike G]

I still wonder if player improvement is mostly or more than marginally attributable to the coach. His real and exclusive job is to allocate minutes.
A good coach should give more minutes to his better players, yes?
And if we know which players are better, through plus-minus, can't we just see if a coach is using the guys that get the job done?

I went to this page -- http://stats-for-the-nba.appspot.com/teams/DAL.html
Not sure if that is just this year or multi-year. We could try it several ways.

Code: Select all

2014 Mavs      min/82       mpg     Off     Def     Tot
Ellis,Monta      37    PG    37    3.25   -1.17    2.08
Nowitzki,Dirk    32    PF    33    4.84    1.76    6.60
Marion,Shawn     29    SF    32    -.96    1.26     .30
Calderon,Jose    30     G    31    2.25   -3.87   -1.62
Carter,Vince     24    SG    24    2.51    1.83    4.34

Harris,Devin     10    PG    20    2.08    -.73    1.35
Dalembert,Samuel 20     C    20   -3.01    3.00    -.01
Wright,Brandan   13     C    19    2.68    -.37    2.31
Crowder,Jae      15    SF    16    1.20    1.38    2.58
Blair,Dejuan     15   PF-C   16   -1.57     .19   -1.38

Larkin,Shane      6    PG    10    -.40    -.91   -1.31
Mekel,Gal         4    PG     9   -1.02   -2.51   -3.53
Ellington,Wayne   5    SG     9     .60   -1.44    -.84
James,Bernard     2     C     5   -2.39     .04   -2.35
Ledo,Ricky        .4   SG     3    -.82    -.57   -1.39

correl. W mpg                       .61     .14     .60
correl. W min/82                    .55     .20     .59

mpg are minutes in games actually played -- how much Carlisle used them when they were available.
The correlations are stronger for offense than for defense. He may be an 'offensive minded' coach, or he may just have more offensive than defensive options.

If his Off:mpg correlation were higher than his Tot:mpg, then he could look more to defense, perhaps.

[Mike G]

And here are the Mavs in playoffs, next to their RS:

Code: Select all

Mavs -     mpg:  RS          PO     Off     Def    tot
Ellis,Monta      37    PG    36    3.25   -1.17    2.08
Nowitzki,Dirk    33    PF    38    4.84    1.76    6.60
Marion,Shawn     32    SF    28    -.96    1.26     .30
Calderon,Jose    31    PG    27    2.25   -3.87   -1.62
Carter,Vince     24    SG    27    2.51    1.83    4.34

Harris,Devin     20    PG    25    2.08    -.73    1.35
Dalembert,Samuel 20     C    19   -3.01    3.00    -.01
Wright,Brandan   19     C    13    2.68    -.37    2.31
Crowder,Jae      16    SF    12    1.20    1.38    2.58
Blair,Dejuan     16     C    12   -1.57     .19   -1.38

Larkin,Shane     10    PG     1    -.40    -.91   -1.31
Mekel,Gal         9    PG         -1.02   -2.51   -3.53
Ellington,Wayne   9    SG     2     .60   -1.44    -.84
James,Bernard     5     C     1   -2.39     .04   -2.35
Ledo,Ricky        3    SG          -.82    -.57   -1.39

correl. w RS mpg                    .61     .14     .60
correl. w PO mpg                    .60     .12     .64

Dirk, Devin, and Vince got more minutes, and everyone else got less -- a little or a lot.
It's as though Carlisle was using numbers very much like these. He has to have guards on the floor, and he has to maintain some balance between offense and defense. At least that's my guess.

[J.E.]

His real and exclusive job is to allocate minutes.

Strongly disagree. Player development is still a big part of a coaches' job, and picking good spots for his players to be in when playing. With a team like the Bucks I think one can argue that player development may be the most important part of Kidd's job

A good coach should give more minutes to his better players, yes?

Certainly. And that's something coach RAPM isn't measuring - as I have mentioned several times in this thread

The correlations are stronger for offense than for defense.

Not that relevant, but the correlations are higher for offense because it's xRAPM and the BoxScore prior for offense is heavily influenced by MP

[Mike G]

I don't agree that we are in strong disagreement.
Player development of course includes players' understanding of their role, how to best complement others on the floor, etc.
If the player doesn't develop a sense of his role, he gets fewer minutes. Thus he is motivated to learn and improve.
The head coach is the one guy who allocates minutes; others may advise him, but he pulls the strings. I could have said it's his job exclusively. It doesn't overlap with those other factors like facilities, GM, fan base, etc.

Meanwhile, the tables were just an illustration of how one might measure this part of the coach's job. One could do it for all coaches; maybe they'd all have higher correlation with offense. They could still be compared one to another.

Is this the table for single-year no-prior RAPM?
http://stats-for-the-nba.appspot.com/ratings/2014.html

It would be nice if there were a Team column next to the names. Or is there a version that's team separated?

[Crow]

It is also possible that someone hacked into the RB named account. But that still seems less likely than the simple answer.

I doubt we will get insight into the methodology or fuller results of the Mavs coaching analysis metric but Cuban has stated they did one.

Coaching analysis based on point differential is important but I'd still give some consideration as well to analysis based on win % or actual - expected win %.

By actual - expected win % Carlisle was essentially even in Detroit and Indiana but almost +3 on average in Dallas. How much was Carlisle improvement and how much was due to the rest of Mavs organization? They were an average of +1.5 overperformance of expected after the departure of Wayne Winston and his APM (not 100% on dates) but the overperformance was more than twice as high while he was involved, coincidence or not.

[Crow]

Carlisle with the 8th best regular season win% among last season's coaches and 8th best among the 22 of those with a playoff record.

0-2 against .500+ teams in regular season in Detroit. Barely 2-2 in Indiana. 2-4 in Dallas.

[J.E.]

Is this the table for single-year no-prior RAPM?
http://stats-for-the-nba.appspot.com/ratings/2014.html

Single-year no-prior RAPM is here. Currently doesn't include team names, sorry