APBRmetrics

I've spent a lot of time thinking about RAPM and came up with a few issues that I believe that RAPM needs to adjust for. Most of them relate to the FT line.

1. FT's. This is the most important point. RAPM needs to adjust the FT% of your opponents and teammates to the league average. The league average is around 75 FT%. A player has no control over an opponents FT% so they get punished if the opponent hits 80 FT%. Same with teammates. If your teammates are cold during your time, you get punished for having bad luck.

2. Technical FT's. They shouldn't be included in RAPM

3. Let's say an opponent goes to the line for 2 FT's. Player A gets substituted by Player B after the 1st FT attempt. Player B gets a -1 in plus/minus if the opponent hits the FT despite the fact that he wasn't on the court when the foul occurred.

4. Intentional fouls. Late game intentional fouls need to be adjusted for. Offensive players get an artificial boost by it and defensive players get deflated.

5. Half possessions. Let's say a defender plays good defense and the ball is deflected out with 10 seconds on the clock. He gets substituted before the possession ends, that player gets no credit for the half possession.

6. Late game situations. In the last 5 minutes of the game, offense is usually worse compared to normal. A player who plays late in the game has his offensive RAPM deflated while his defensive RAPM gets boosted. Those situations should be adjusted for to get an accurate offensive and defensive RAPM.

colts18 wrote:I've spent a lot of time thinking about RAPM and came up with a few issues that I believe that RAPM needs to adjust for. Most of them relate to the FT line.

1. FT's. This is the most important point. RAPM needs to adjust the FT% of your opponents and teammates to the league average. The league average is around 75 FT%. A player has no control over an opponents FT% so they get punished if the opponent hits 80 FT%. Same with teammates. If your teammates are cold during your time, you get punished for having bad luck.

2. Technical FT's. They shouldn't be included in RAPM

3. Let's say an opponent goes to the line for 2 FT's. Player A gets substituted by Player B after the 1st FT attempt. Player B gets a -1 in plus/minus if the opponent hits the FT despite the fact that he wasn't on the court when the foul occurred.

4. Intentional fouls. Late game intentional fouls need to be adjusted for. Offensive players get an artificial boost by it and defensive players get deflated.

5. Half possessions. Let's say a defender plays good defense and the ball is deflected out with 10 seconds on the clock. He gets substituted before the possession ends, that player gets no credit for the half possession.

6. Late game situations. In the last 5 minutes of the game, offense is usually worse compared to normal. A player who plays late in the game has his offensive RAPM deflated while his defensive RAPM gets boosted. Those situations should be adjusted for to get an accurate offensive and defensive RAPM.

1) I've thought about adjusting this before. But you can't change the FT% to the league average; it has to be the player's average. Otherwise you won't punish guys for stupidly fouling good shooters on the perimeter.

2) Well, if you flip out and get a tech, it should reflect in the data that your lineup gave up that point. But what about coaches/guys on the sideline? Do we throw that out then?

3) I can't speak for everyone, but when I do it I make sure the points go to the lineup before the substitution.

4) I really wish intentional fouls were kept officially because we'd have to guess when they were happening. But you can always weigh possessions less when the lead is too big.

5) I've thought about this a lot. I've almost added in half possessions. But it's easier to give the possession to the lineup that stopped the possession. I think at one point I split the possession in half when the lineup had substitution during a possession. But I'm not sure on what everyone's procedure is.

6) I think that's more of an indictment on late game offensive strategy than anything else.


Related: what's the explanation for Nick Collison's gigantic yearly RAPM? I've heard that there was some weird interaction one season, and the error has carried over (maybe.) And is there one for Battier?

I haven't done a RAPM analysis myself, so I could be wrong on some of these assumptions, but I think a lot of this is at least somewhat already handled.

1) Since opponent on court is a factor in RAPM, opponent FT% is theoretically accounted for. The point is taken that you could be the poor sap on the court when Dwight Howard happens to go 7-8, but assuming you play enough minutes that should even out.

2) I agree with ATC; it might make sense to remove coach technicals. That being said, coaches usually get technicals in response to stuff that's happening on the court, so they should still be some reflection of quality of play in some manner.

3 and 5) I always thought that RAPM 'observations' were at the line-up level. If someone get swapped out, either between free throws or during a loose ball out-of-bounds in the middle of a possession, the RAPM 'possession' would end and that's it. Anyone on the court would get credit for what they should. I guess it could be the case that people get assigned the wrong number of possessions (like the good defender gets 4 or 5 instead of 4.5), but again that should even out or be negligible with enough playing time.

4) I would think that intentional fouls are part of, for example, J.E.'s "up by X" adjustment.

6) I agree with ATC that late-game situations are due to changes in strategy, so I can see the argument that they don't reflect 'normal' gameplay and thus the rating you might like to see. But since they make up a relatively small part of overall playing time, especially assuming that most late-game players are guys who see a lot of minutes, they should get somewhat squashed by the out-of-sample testing.

Related to the last point, I don't have any evidence to show it, but I feel like all of these concerns happen rarely enough that they wouldn't budge a player's estimate much if at all.

AcrossTheCourt wrote: 3) I can't speak for everyone, but when I do it I make sure the points go to the lineup before the substitution.


Related: what's the explanation for Nick Collison's gigantic yearly RAPM? I've heard that there was some weird interaction one season, and the error has carried over (maybe.) And is there one for Battier?

As for #3 - I would hope everyone does it the way you do. If they aren't, I'd be worried about what other decisions they are making in their metric that could hurt the results.

I'll never forget that one season (a championship year maybe) in which someone tried to tell me Ben Wallace was nearly the best player in the league, while Richard Hamilton was one of the very worst. All it took was a relative handful of minutes they didn't play together to VASTLY sway the results - one of the reasons the metric has gone multi season.

I remember over 20 years ago, watching game game 6 of the Bulls/Suns (I actually was computing +/- by hand for that game) - telling everyone that when I figure out how to get ALL NBA +/- numbers and run all the numbers, I would revolutionize NBA stats. Well, as the years went on and I learned about how tricky/noisy the results can be - I stopped seeing what is now RAPM as a "be all, end all" metric to evaluate players. If I were working for a team, and running my own metrics of course - there is no doubt I would look at the RAPM results to see what players I'm getting the best consensus on in relation to my work.

BTW - has there EVER been a rookie that didn't have an RAPM that sucked compared to general fan consensus about the player? Kevin Durant was horrible in RAPM. Anthony Davis is practically a rookie - his RAPM is pretty meh. It just seems that prior informed always seems to suggest the younger players in the league are pretty horrible despite their more conventional stats look like.

Statman wrote:

AcrossTheCourt wrote: 3) I can't speak for everyone, but when I do it I make sure the points go to the lineup before the substitution.


Related: what's the explanation for Nick Collison's gigantic yearly RAPM? I've heard that there was some weird interaction one season, and the error has carried over (maybe.) And is there one for Battier?

As for #3 - I would hope everyone does it the way you do. If they aren't, I'd be worried about what other decisions they are making in their metric that could hurt the results.

I'll never forget that one season (a championship year maybe) in which someone tried to tell me Ben Wallace was nearly the best player in the league, while Richard Hamilton was one of the very worst. All it took was a relative handful of minutes they didn't play together to VASTLY sway the results - one of the reasons the metric has gone multi season.

I remember over 20 years ago, watching game game 6 of the Bulls/Suns (I actually was computing +/- by hand for that game) - telling everyone that when I figure out how to get ALL NBA +/- numbers and run all the numbers, I would revolutionize NBA stats. Well, as the years went on and I learned about how tricky/noisy the results can be - I stopped seeing what is now RAPM as a "be all, end all" metric to evaluate players. If I were working for a team, and running my own metrics of course - there is no doubt I would look at the RAPM results to see what players I'm getting the best consensus on in relation to my work.

BTW - has there EVER been a rookie that didn't have an RAPM that sucked compared to general fan consensus about the player? Kevin Durant was horrible in RAPM. Anthony Davis is practically a rookie - his RAPM is pretty meh. It just seems that prior informed always seems to suggest the younger players in the league are pretty horrible despite their more conventional stats look like.

Tim Duncan is the best example of a high RAPM player. Vince Carter was good too. I think Chris Paul and Andre Drummond were okay though nothing special. Same with Bogut. Yao too. Looks like Kirilenko had a +4.3 on NPI RAPM as a rookie.

Remember that PI RAPM usually gives a negative prior to rookies, but even in NPI RAPM they're usually poor despite great box score stats (sometimes.)

...Did you really compute +/- 20 years ago? I also heard the Bulls kept +/- numbers back in the 90's and some game broadcast talked about them.

AcrossTheCourt wrote:
...Did you really compute +/- 20 years ago? I also heard the Bulls kept +/- numbers back in the 90's and some game broadcast talked about them.

Oh no - I just did it for occasional games for fun. The 6th game of that Suns/Bulls game stands out, because I was with friends and computing the +/- during the game. It was something to do, easy, just write the score every time there was a lineup change. I had the +/- IDEA (that now has been developed into RAPM by JE & others probably better than I would ever have done it) well over 20 years ago, but zero means (data & computing power) to execute it.

I believe JE's latest real plus minus adjusts for time and situation.


Greg Oden's RAPM was +1.5 as a rookie and+3 in second year. Steph Curry as neutral in year 1 and +2 in year 2. So was Westbrook. Harden was neutral in year 1 and +1.3 in year 2. Those were the best cases of young guys in the last 5 years that I saw.

colts18 wrote:3. Let's say an opponent goes to the line for 2 FT's. Player A gets substituted by Player B after the 1st FT attempt. Player B gets a -1 in plus/minus if the opponent hits the FT despite the fact that he wasn't on the court when the foul occurred.

No, the NBA's PbP counts this correctly. Player A will get the -1 (or +1 if his a player on his team was fouled). The newly substituted player will not see any changes to his raw +/- until after the last free throw attempt (unless there is a lane violation). This isn't a worry.

AcrossTheCourt wrote:

colts18 wrote: 2. Technical FT's. They shouldn't be included in RAPM

2) Well, if you flip out and get a tech, it should reflect in the data that your lineup gave up that point. But what about coaches/guys on the sideline? Do we throw that out then?

Exactly, if you're going to be a knuckleheaded DeMarcus Cousins and lead the league in technical fouls, APM (in any of its versions) SHOULD penalize you for the points you're costing your team.

xkonk makes a good point about technicals on coaches/sidelines. There's a good argument for throwing them out, but also an argument for counting them. However my guess is that we can do the APM calculations either way, and it will make very little difference.

Related: what's the explanation for Nick Collison's gigantic yearly RAPM? I've heard that there was some weird interaction one season, and the error has carried over (maybe.)

How much of his RAPM is due to defense, and how much to offense? He's got a good reputation as a defender and regularly drew a lot of charges. I.e. precisely the harder-to-appreciate skills that do not get covered in box scores.

IMO, most of these things are either:

(a) not flaws with RAPM, as they are addressed by anyone calculating it properly (#3 and #5), or
(b) minor enough that they aren't very significant in terms of player values.

talkingpractice wrote:IMO, most of these things are either:

(a) not flaws with RAPM, as they are addressed by anyone calculating it properly (#3 and #5), or
(b) minor enough that they aren't very significant in terms of player values.

Adjusting for FT% is huge. Among players with 1,000+ MP this season, their opponents ranged in FT% from 70.8% to 79.8%. That's a 9% range which the player has no control over.

http://www.basketball-reference.com/pla ... opp_ft_pct

colts18 wrote: Adjusting for FT% is huge. Among players with 1,000+ MP this season, their opponents ranged in FT% from 70.8% to 79.8%. That's a 9% range which the player has no control over.

My intuition (which is correct about 50% of the time) leads me to think that this still won't amount to much of a material change in RAPM values for almost all players, especially in models that are prior informed (via box, or prior rapms, or whatever). That said, I'm happy to take on the role of crash test dummy here, so we'll test the "Colts Conjecture" after the playoffs here are our end and shall report back.

Also, I think it only makes sense to use full-season-FT% for the Free Throw shooter for 'X-1 of X' FTs and not for 'X of X', because whether the last Free Throw gets made has an influence on what happens next (offensive/defensive rebound)

J.E. wrote:Also, I think it only makes sense to use full-season-FT% for the Free Throw shooter for 'X-1 of X' FTs and not for 'X of X', because whether the last Free Throw gets made has an influence on what happens next (offensive/defensive rebound)

If that's the case, wouldn't it also make sense to adjust offensive and defensive RAPM by the outcome from the previous possession. Offensive RAPM will get inflated if the team forces a lot of liveball turnovers on defense. It gets deflated if they allow a lot of makes on defense.

So, just to be clear, we're talking here about the effect of random variation in free throw shooting outcomes biasing (R)APM estimates, both offensive and defensive, right?

I must say that my initial reaction was that this should be a rather small effect, but upon further consideration, I think I'm of a belief that the combination of this and related effects could be quite large, indeed.

colts18 identifies free throw shooting for special mention, but there is nothing really special about this category of scoring attempts. Random variation of outcomes of two and three point shooting attempts are conceptually identical and should similarly be taken into account. And indeed the contribution of these two categories to per possession scoring variance is very much larger. (The only thing special about free throw shooting is that there is zero defensive input in the instance, what might make adjustments for this effect more straightforward.)

Maybe I'm thinking about this incorrectly, but here's the simple thought experiment, breaking scoring attempts into the aforementioned three categories (2PA, 3PA, and FTA), using data for the last season's NBA average* (with a minor asterisk that can be explained), and calculating the resulting standard deviation of points per 100 possessions (assuming no covariation between types of scoring attempt). I then get the following for players playing 70 games and either 1/4, 1/2, 3/4, or the entire game (of which, of course, there are none of the latter): 2.6, 1.9, 1.5, and 1.3 respectively. And these would correspond to offense and defense separately.

Now imposing a prior should eliminate a significant portion of the problem of such variation, as we don't expect players to be serially (un)lucky, but if these numbers are in the ball park, they are large enough to be of concern, no?