Experimenting with Hollinger's PER and D.Oliver's O/DRtg

[J.E.]

I'm in the process of doing a series of tests with Hollinger's PER and Dean Oliver's ORtg/DRtg. I'm basically trying to build an SPM from these metrics.

For those who don't know how I build SPM: I use player stats from 'year Z-1' as my design matrix (X) and then use points scored by 5on5 lineups in 'year Z' as the dependent variable (y)

As a result from building an SPM out of PER/ORtg/DRtg I get a 'meta-metric' that's on the scale I'm used to from RAPM etc.. Further, I can test this new metric for out of sample prediction error, and the coefficients we get when building the metric should tell us whether PER or ORtg are superior when it comes to prediction

What may be confusing for some people is that I'm using DRtg as a variable for offense, and I'm using ORtg and PER as variables for defense. My take is that as long as it improves prediction I'm going to include it. If it's really not helpful then the coefficient for those variables should be very close to 0.

After normalizing the columns in X here are the coefficients I get for offense:
PER: 2.7 (higher PER->better Offense. One standard deviation above mean equals 2.7 offensive rating (per 100))
ORtg: 0.8
DRtg: -0.2 (negative sign means: smaller DRtg->better offense)

As you can see, PER clearly dominates. My gut feeling regarding ORtg is that it values low usage, seldomly passing big men with good TS/eFG a little too much (C. Andersen, R. Lopez, D. Jordan, T. Chandler, Drummond are in the top 13)

And the coefficients for defense
PER: 0.6 (higher Per->better defense)
ORtg: 1.1
DRtg: -3

DRtg is best when predicting defensive performance, which is not really that surprising, considering that PER and ORtg are supposed to measure offense and not defense. Nevertheless, including ORtg and PER on defense helps somewhat.

What's surprising is that the signs are all the same - we do get an R^2 of 0.6 between the resulting offensive and defensive rating

Player ranking for '13-'14 for this meta metric is here

In regards to out of sample prediction, this meta metric performs just as well as my SPM which is pretty cool considering that neither Hollinger or Oliver had matchupdata or stats like 'blocks rebounded by defense' (which is sth. my SPM has) to work with. Although it has to be said that DRtg is influenced by TeamDRtg, which most certainly helps with out of sample prediction performance. (In contrast, my SPM is free of any 'team adjustments' due to the fact that I don't think I need it - RAPM will do that part for me)

[DSMok1]

Very interesting work. How much better is the blend than the components alone?

[bbstats]

Hey there.

In my repeated tests, the best method for using DeanO's ORTG as an offensive SPM is to use ORTG x Usage and then an extra Usage term.

[Mike G]

What bbstats said. I had thought (recently in another thread) that since Usg% of 20 = avg, then ORtg*(Usg+20)/DRtg would be a useful stat. That kind of assumes everyone's "defensive Usage" is 20%, so O and D are equally weighted.
But (can't find the spreadsheet right now) the closest fit I could find with WS/48 was something like ORtg*(Usg+54)/DRtg. Maybe there were other terms.
It would be interesting to see a formula making the closest fit to RAPM.

Here's that other thread: viewtopic.php?f=2&t=8459

[HoopDon]

J.E.,

Including certain terms in a metric to improve performance, regardless of whether or not those terms make any sense, seems like a bad idea. For instance, when building my SPM-metric, I could get a high correlation to long-sample-RAPM (higher then ASPM), but the formula resulted in a negative coefficient for DRB%, a positive one for DRTG, and other such nonsense.

Doing counter-intuitive things like that can "improve performance", but will likely result in said metric being disregarded by all but the most hardened statistician.

Mike G,

ORtg*(Usg+54)/DRtg doesn't do much in terms of correlation to long-sample RAPM, but removing the DRTG gives the metric a respectable correlation to O-RAPM, higher then any major metric outside of O-ASPM (not counting X-RAPM or Estimated Impact).

http://hoopdon.weebly.com/

[J.E.]

Very interesting work. How much better is the blend than the components alone?

Using all 3 metrics for both offense and defense increases the OOS reduction by ~15% compared to using just ORtg for offense and just DRtg for defense.
Don't have a baseline for PER. I assume, when used alone, it's worse than ORtg/DRtg because it's not great for defense

[ryannow]

As has been mentioned by a few others, my understanding of ORTG is that it's an estimate of points created per 100 possessions used by that player, and that it has to also have an estimate of possessions used included separately in order to estimate that player's impact on his team's offensive efficiency. There are already a couple suggestions for how to incorporate usage, but if you want a ready-made and commonly used number that synthesizes both factors, OWS does that, and I'd use OWS/48 in this comparison rather than ORTG.

On a separate note, It also has the advantage of being normalized to league average, such that .050 OWS/48 always means the same thing relative to average, just like 15.0 PER does, while 106 ORTG might be above average in a down offensive year and above average in a good year. That may not be an issue for your study, but it seems generally preferable, especially if you may want to compare years against each other later.

[Mike G]

The ratio ORtg/DRtg is handy in that it eliminates the necessity of including a season's league ORtg. You could divide both the numerator and the denominator by this number, but it doesn't change anything.

A formula using a team's ORtg/DRtg to predict their Win% may have validity for individual players, too, should one factor in Usage%. A W% is equivalent to a given 48-minute point differential.

For 1020 team-seasons since 1978, I get this formula for expected W%, given teams' ORtg/DRtg. Call that ratio X.

W% = .191*X^3 + .54*X^2 + .537*X + .153 - .924/X

Avg absolute error is .032
1/X is of course = DRtg/ORtg

[J.E.]

As has been mentioned by a few others, my understanding of ORTG is that it's an estimate of points created per 100 possessions used by that player, and that it has to also have an estimate of possessions used included separately in order to estimate that player's impact on his team's offensive efficiency. There are already a couple suggestions for how to incorporate usage, but if you want a ready-made and commonly used number that synthesizes both factors, OWS does that, and I'd use OWS/48 in this comparison rather than ORTG.

I always thought OWS was ORtg*minutes but apparently I was wrong. Using OWS/48 does make more sense, then. If OWS/48 turns out to be superior to ORtg it might make sense for basketball-reference to show it (right now they only show WS/48)

[J.E.]

Tried OWS/48 instead of ORTG

The regression now prefers OWS/48 over PER on offense
Coefficients (offense):
PER: 0.7
OWS/48: 2.7
DRtg: -0.4

Coefficients(defense):
PER: 1.5
OWS/48: -0.7
DRtg: -2

Neither OOS prediction error or the player ratings have changed significantly though