What's the best publicly available defensive metric?

[deepak]

So, any updates on this front? A lot of new stats look to be available from stats.nba.com.

What are the best statistical measures to look at today for assessing a player's defensive performance by the numbers?

[Crow]

One approach would be a form of defensive composite score, similar to what Jon Nichols did a years ago. Take best available defensive RAPM data along with say counterpart defensive rating at 82 games or new data from synergy or sportsVU and maybe others like Oliver's defensive rating (though it has serious issues on what it captures at individual level and what is does not) or EZPM. I don't think the components should be weighted equally as I believe Jon did. I'd probably give RAPM the largest share. Or instead of an active blend, just look at each component and if most or all are strongly positive or negative then the player gets a likely above or below average ranking. If it is mostly average ratings or quite mixed then they probably belong in a near average (or can't tell) category. On defense I don't think we can get much more specific than above average, near average or below average without controversy or stretching beyond the quality of the data, though most of the likely very best and likely among the very worst can be identified with at least moderate confidence based on the available input data and their degree of separation from the pack.

[deepak]

There is also Defensive Stops Gained (I think that's the name), which is described in the recent Basketball Analytics (by Stephen Shea). It is fairly easy to calculate based on the numbers at stats.nba.com (just a weighted sum of the on/off eFG%, TOV%, and OREB%). I computed it for a few teams, but subjectively the results seemed hit and miss. Though, of course, the same could be said about most other defensive metrics based on such a small sample size.

The availability of play-by-play video at stats.nba.com has tempted me to attempt some defensive charting and grade teams, or individual players, based on an adjusted Stop% metric I came up with (which is probably nothing novel). On each play, credit for certain stops or allowed scores would need to be portioned out to the individual defenders, which adds some subjectivity to it. But I'd feel much more confident in such a metric for smaller samples of games than what's currently out there.

[steveshea]

Thanks for the mention of Defensive Stops Gained (DSG) deepak.

DSG is a top down metric (based on performances of 5-man lineups). So, it is susceptible to the usual biases that come with top down metrics. The biggest of which (in my opinion) is the teammate pull effect. If player A plays most of his time with good defensive teammates, player A can look like a better defensive player than he is. Generally, the higher the player's lineup entropy, the more confidence we can have that DSG reasonably represents the defensive contributions of the individual in the given time frame.

I completely agree with the small sample size concern. I haven't run DSG on 2013-14 yet, but I suspect it will look better when run for the full season.

Since DSG uses net on/off the court metrics, we also have to pay attention to replacements (especially on small sample sizes). If a team generally plays one center or another, when one looks good in DSG, the other will look bad.

Those are some of the things I think about and investigate as I try to interpret DSG.

[Mike G]

If a player is so good on offense that he brings a defensive specialist off the opponent bench, then it might also look like he's great on defense: The opponent offense declines with the substitution.

If that defensive specialist plays no offense (is no threat), then a non-defending offensive specialist might be brought in and assigned to guard him. This boosts the offense, perhaps enough that the D-specialist doesn't seem to help.

It's definitely tricky. A strong scorer who also plays D may have a better defensive impact than a weak scorer who is technically better on D. Just because he creates matchup issues in the opponent lineup.

[mbosset]

Last year at the Sloan Sports Conference Kirk Goldsberry and Eric Weiss presented a research paper titled "The Dwight Effect :A New Ensemble of Interior Defense Analytics for the NBA" The paper can be found online if anyone is interested in reading about it. The paper uses player tracking (sportVU) to asses what percentage of shots came from close range when the defender was within 5 feet of the basket, as well as the percentage that offenses shot when the defender was within 5 feet from the basket. The results came back with results you would somewhat expect with Howard , Hibbert and Sanders being near the top. Interested on how well these statistics evaluate individual defense I graphed each players results compared with what many believe to be the best defensive stat, defensive RAPM. The following graphs show a negative correlation between defensive Rapm and % of close range shots, and close range FG%.
This is what you would expect/hope to see as limiting close range shots as well as lowering close range FG% would substantially help your defense. However, what is interesting is there is a MUCH higher R^2 value for Close Range FG%. I think this shows that contrary to what the authors expected limiting an opposing players %of shots in the close range is important, it is not nearly as important as lowering their FG%. This of course only compares two of many newly available defensive statistics. However, I believe it is somewhat of a start to try to decipher which of these new player tracking statistics are indicators of good defense and which are not as great.

[jbrocato23]

Last year at the Sloan Sports Conference Kirk Goldsberry and Eric Weiss presented a research paper titled "The Dwight Effect :A New Ensemble of Interior Defense Analytics for the NBA" The paper can be found online if anyone is interested in reading about it. The paper uses player tracking (sportVU) to asses what percentage of shots came from close range when the defender was within 5 feet of the basket, as well as the percentage that offenses shot when the defender was within 5 feet from the basket. The results came back with results you would somewhat expect with Howard , Hibbert and Sanders being near the top. Interested on how well these statistics evaluate individual defense I graphed each players results compared with what many believe to be the best defensive stat, defensive RAPM. The following graphs show a negative correlation between defensive Rapm and % of close range shots, and close range FG%

What specifically did you use in the graph? NPI rapm?

[mbosset]

As the Y-axis I obviously used the percentages and for the X-axis I used 2012-2013 Defensive RAPM located at http://stats-for-the-nba.appspot.com/ratings/2013.html

[jbrocato23]

As the Y-axis I obviously used the percentages and for the X-axis I used 2012-2013 Defensive RAPM located at http://stats-for-the-nba.appspot.com/ratings/2013.html

I'd be interested to see the same thing but using a "purer" rapm - i.e., one with little to no box score influence (since xRAPM is heavily influenced by the box score)

[nbacouchside]

Thanks for the mention of Defensive Stops Gained (DSG) deepak.

DSG is a top down metric (based on performances of 5-man lineups). So, it is susceptible to the usual biases that come with top down metrics. The biggest of which (in my opinion) is the teammate pull effect. If player A plays most of his time with good defensive teammates, player A can look like a better defensive player than he is. Generally, the higher the player's lineup entropy, the more confidence we can have that DSG reasonably represents the defensive contributions of the individual in the given time frame.

I completely agree with the small sample size concern. I haven't run DSG on 2013-14 yet, but I suspect it will look better when run for the full season.

Since DSG uses net on/off the court metrics, we also have to pay attention to replacements (especially on small sample sizes). If a team generally plays one center or another, when one looks good in DSG, the other will look bad.

Those are some of the things I think about and investigate as I try to interpret DSG.

Hi Steve-

I was just reading the book earlier today and running the numbers for a few teams, and I noticed Taj Gibson had negative defensive stops gained, which seemed off, as he's clearly a net defensive positive and then something occurred to me. Have you thought about adjusting DSG for team context? I.e. since the Bulls' opponents points per game, pace adjusted, is roughly 9.2 points below that of an average team, shouldn't they as a team be credited with an additional 4.6 defensive stops per game so (4.6*82 so roughly 377 extra stops over the course of the season), which could then be apportioned to each player as a percentage of playing time for the team? That might mitigate some of the issues that come with playing on a team which is filled with very strong defensive players and the loss of just one of which doesn't hurt the overall defense much. I'm planning on running the numbers with such an adjustment, but was curious to hear your thoughts.

EDIT:

It doesn't make much of a difference, after running the numbers on a few teams adjusting this way.

[steveshea]

DSG is based on changes in % (such as % ORB out of opportunities). The % is multiplied by a league average rate. So, in that way it is already pace adjusted.

Opponents' shoot a lower EFG% with Gibson on the floor. He's positive in that regard. Otherwise, his net numbers don't look so good. Opponents are grabbing a higher % of ORB and turning the ball over less when Gibson plays.

Noah is a big factor here. Noah played over 1500 minutes without Gibson this past season. So when Gibson is off the court, Noah is typically on. Noah is pretty good defensively. This demonstrates the main bias in DSG. DSG is biased against players that have good "replacements." The Bulls are a great defensive team without Gibson. That makes it tough for Gibson to have a significant positive impact.

I'm open to suggestions on how to remedy this bias.

[nbacouchside]

DSG is based on changes in % (such as % ORB out of opportunities). The % is multiplied by a league average rate. So, in that way it is already pace adjusted.

Opponents' shoot a lower EFG% with Gibson on the floor. He's positive in that regard. Otherwise, his net numbers don't look so good. Opponents are grabbing a higher % of ORB and turning the ball over less when Gibson plays.

Noah is a big factor here. Noah played over 1500 minutes without Gibson this past season. So when Gibson is off the court, Noah is typically on. Noah is pretty good defensively. This demonstrates the main bias in DSG. DSG is biased against players that have good "replacements." The Bulls are a great defensive team without Gibson. That makes it tough for Gibson to have a significant positive impact.


I'm open to suggestions on how to remedy this bias.

Right, the eye test and RAPM both suggest that Gibson is a massive positive on defense, so that's what got me interested in trying to figure out a way to, in a relatively simple way, adjust DSG for the fact that he is on a great defensive team. I'm sure it's something you considered, so maybe there is no obvious way to deal with it, save for the very complicated regression based stuff everyone is very into now.

If anyone is curious, I ran the numbers for Approximate Value (AV) for the 13-14 season. The top 20 is as follows:

Code: Select all

+------------------+---------+--------+
|      Player      | Minutes |   AV   |
+------------------+---------+--------+
| LeBron James     |    2902 | 2307.2 |
| Blake Griffin    |    2863 | 2201.4 |
| Kevin Durant     |    3122 | 2139.4 |
| Kevin Love       |    2796 | 2025.2 |
| Stephen Curry    |    2846 | 2023.8 |
| Chris Paul       |    2171 | 1994.4 |
| John Wall        |    2980 | 1979.0 |
| Carmelo Anthony  |    2982 | 1896.1 |
| Isaiah Thomas    |    2497 | 1854.7 |
| Kyle Lowry       |    2862 | 1838.7 |
| Jeff Teague      |    2542 | 1808.5 |
| DeMarcus Cousins |    2298 | 1786.9 |
| Goran Dragic     |    2668 | 1724.0 |
| James Harden     |    2777 | 1717.9 |
| Andre Drummond   |    2619 | 1711.1 |
| Dirk Nowitzki    |    2628 | 1700.7 |
| Kemba Walker     |    2614 | 1622.7 |
| Paul George      |    2898 | 1608.5 |
| Joakim Noah      |    2820 | 1589.3 |
| Anthony Davis    |    2358 | 1565.2 |
+------------------+---------+--------+

There are some interesting locations for names in the list, but no one who jumps out as not belonging at all. Teague, Thomas, and Kemba were probably the biggest surprises, for me. But they all handle the ball a lot and are efficient while scoring a fair amount. Thomas had over 156 DSG on the year, which obviously helped him quite a bit. Probable he benefitted some from SAC's terrible defensive baseline in achieving that lofty DSG.

Also, interesting to note is that if you look at AV per minute, rather than the totals, CP3 blows everyone else away with .91 AV per minute with LeBron the closest (among players with 500 minutes played) at .795 AV per min.

Total AV league-wide was about 250194, which equates to roughly 203.4 AV points per win, if you want to think of it in those terms, although wins at the team level don't sum up correctly that way. The Bulls, for instance, summed to only about 40 wins that way, which is maybe a red flag about their actual overall talent level and makes their being abused by the Wizards make more sense.

[steveshea]

I am a bit wary of Gibson's DRPM. There is a clumping effect in DRPM for big men on the same team. Quickly pulling a few examples (from espn):

BKN: Collins (1.94), Lopez (1.94)
WSH: Gortat (3.85), Hilario (4.39)
IND: Hibbert (3.68), Mahinmi (4.05)
NY: Martin (3.82), Chandler (3.79)
LAC: Hollins (3.90), Jordan (3.60)
POR: Aldridge (3.17), Lopez (3.38)
HOU: Howard (5.14), Asik (4.99)
MIA: Anderson (3.99), Bosh (3.70)
OKC: Thibbert, Collison, Ibaka and Perkins all ranked between 23rd and 37th
ORL: Vucevic (2.40), Maxiell (2.49)
ATL: Horford (2.24), Millsap (2.00), Brand (1.92), Antic (1.70)
Den: McGee (-.45), Hickson (-.57)
Chi: Noah (3.68), Gibson (3.57)

So, Gibson may look a bit better on Chicago than he would playing for a weaker defensive team.

The effect isn't universal. DRPM distinguishes between Sanders and Pachulia, for example. I would be curious to see a break down of where the differentiation comes from (pure APM or prior information/adjustments).

[nbacouchside]

I am a bit wary of Gibson's DRPM. There is a clumping effect in DRPM for big men on the same team. Quickly pulling a few examples (from espn):

BKN: Collins (1.94), Lopez (1.94)
WSH: Gortat (3.85), Hilario (4.39)
IND: Hibbert (3.68), Mahinmi (4.05)
NY: Martin (3.82), Chandler (3.79)
LAC: Hollins (3.90), Jordan (3.60)
POR: Aldridge (3.17), Lopez (3.38)
HOU: Howard (5.14), Asik (4.99)
MIA: Anderson (3.99), Bosh (3.70)
OKC: Thibbert, Collison, Ibaka and Perkins all ranked between 23rd and 37th
ORL: Vucevic (2.40), Maxiell (2.49)
ATL: Horford (2.24), Millsap (2.00), Brand (1.92), Antic (1.70)
Den: McGee (-.45), Hickson (-.57)
Chi: Noah (3.68), Gibson (3.57)

So, Gibson may look a bit better on Chicago than he would playing for a weaker defensive team.

The effect isn't universal. DRPM distinguishes between Sanders and Pachulia, for example. I would be curious to see a break down of where the differentiation comes from (pure APM or prior information/adjustments).

As someone who watched just about every Bulls game, I will tell you that Gibson is every bit as good on defense as his DRPM suggests. That's why his DSG being negative bothers me so much, especially since Carlos Boozer (his primary replacement) has positive DSG, despite being absolute garbage on that end. Boozer benefits from playing almost all of his minutes with Noah and the rest of the Bulls starting unit which has more defensive talent than the bench (where DJ Augustin is prominently involved). But, I guess even a team adjustment will still leave Boozer as a better DSG player than Taj, which while definitely wrong, in my view, is what happens when you have any metric. There will be problems on the individual level, while getting the broad strokes right. That top 20 looks pretty good, like I said.

[nbacouchside]

So I re-calculated AV using a slightly tweaked version of Defensive Stops Gained. I compared a player's on-court performance in the three opponent rate stats (eFG%, OReb%, and TOV%) to the league average performance, rather than his team's performance without him. This alleviated some of the issues of players on good defensive teams being undervalued and players who are on terrible defensive teams benefitting from the low baseline to achieve sky-high DSG rates, when in all likelihood, they weren't causing THAT many stops, it was just that they benefitted from being compared to a lousy baseline.

Here's the top 20 in this tweaked version of Approximated Value (on a per-minute basis):

Code: Select all

+-------------------+----------+
|      Players      | av/m 500 |
+-------------------+----------+
| LeBron James      | 0.889    |
| Chris Paul        | 0.862    |
| Kevin Durant      | 0.831    |
| Stephen Curry     | 0.813    |
| Blake Griffin     | 0.746    |
| Russell Westbrook | 0.729    |
| Dwyane Wade       | 0.718    |
| Kevin Love        | 0.711    |
| John Wall         | 0.710    |
| Tony Parker       | 0.682    |
| DeMarcus Cousins  | 0.675    |
| Al Horford        | 0.670    |
| Manu Ginobili     | 0.667    |
| Brandan Wright    | 0.666    |
| Nene Hilario      | 0.664    |
| David Lee         | 0.664    |
| Kyle Lowry        | 0.663    |
| Goran Dragic      | 0.661    |
| Nikola Pekovic    | 0.659    |
| Joakim Noah       | 0.652    |
+-------------------+----------+

It looks like a list that makes more sense than the previous list, to me. Thoughts?