APBRmetrics

Me too!

What I find shocking is the supreme lack of numbers used to critique Wins Produced. There is a awful lot of talking, but no one actually attacking the numbers. That I'd like to see!

What do you think this forum is for exactly? Oh, wait. I just looked at the top of my browser:

The discussion of the analysis of basketball through objective evidence, especially basketball statistics.

Many numbers have been presented here over the years for and against WP.

What I find shocking is the supreme lack of numbers used to critique Wins Produced. There is a awful lot of talking, but no one actually attacking the numbers. That I'd like to see!

One is a number. Defensive rebounds are worth about one point in the WP formula, the same value as a steal or offensive rebound.

I'd like to attack this number, and the idea behind it - I don't think it's worth a whole point to get a defensive rebound. I don't think it's practically or functionally the same as getting a steal, like the WP formula argues, or that it should have the same value. Do you?

*****

There have been a great number of in-depth, substantive looks at WP. Phil Birnbaum did a series of posts in January of last year about it, and the blog Gothic Ginobili has written about it since then. The problem is not that people are being dismissive of it or aren't taking it seriously enough - quite the opposite. It's not coherent or useful for anything, and yet we're constantly talking about it. Once you point out that the emperor isn't wearing any clothes there's no reason to describe in detail how naked he is.

Oh, you're just being ignorant.

Actually, he changed it this year, and apparently it's all calculated differently now... so on the WP front I have been very ignorant

After reworking the numbers - Erick Dampier, on a 37-45 GSW team, dropped to the 4th best player in the entire NBA in 2004. Seems reasonable.

Win Shares for that year has Kirilenko as the 5th best player in the league, for a team that only won five more games. Well-known superstar Donyell Marhsall is 11th, having played for the 33-49 Raptors and 23-59 Bulls. Doesn't anyone know that crummy teams can't have productive players?

xkonk wrote:Win Shares for that year has Kirilenko as the 5th best player in the league, for a team that only won five more games. Well-known superstar Donyell Marhsall is 11th, having played for the 33-49 Raptors and 23-59 Bulls. Doesn't anyone know that crummy teams can't have productive players?

What kind of argument is that supposed to be? Kirilenko was among the league leaders in +/- based numbers as well. He had a +12.1 On/Off Net per 100 possessions in 2004. Kirilenko was actually not only producing very well, but also having clearly a great impact. Dampier had +1.1, he barely improved the Warriors at all.

And Donyell Marshall had those Win Shares, because he played nearly 3000 minutes. The guy finished even 6th in total WP in 2004.

Mediocre or bad teams can have really productive players, they can also have players with big positive impact (take Kevin Garnett on those Timberwolves teams for example), it is just odd when a player is considered awesome in just one metric, while all others are saying he is not. And Wins Produced is actually producing the highest amount of odd players.

Let us take an example, which is likely not that well-known. In 2001 the Washington Wizards had a forward on their roster with the name Michael Smith. He was 28 years old, and was capable of getting minutes at SF and PF. In fact he played around 2/3 of his minutes as PF and 1/3 as SF. When Smith was on the court, the Wizards were outscored by 11.2 points per 100 possessions. That is bad, in fact that is the worst value of all players on the Wizards that season. So, with Michael Smith on the court, the Wizards played their worst basketball that season. Well, Smith wasn't a good defender, he wasn't strong enough in the post and not quick enough on the perimeter. Offensively he was very limited, not good hands, limited passing abilities, basically no skills to create shots for himself. He still made 106 field goals that season, most of them were putbacks. Yes, the strength of Michael Smith was rebounding, he was hovering around the offensive and defensive basket and was quite good at grabbing the ball or making a tip-in. He didn't do that very often, but he usually had 1 to 2 baskets based on that per game. Michael Smith is a odd player and usually metrics have trouble with that. Win Shares had him at about average (0.108 WS/48), PER with below average at 14, my SPM has him with -0.98 (like 0.065 WS/48). Given his limitations and obvious lack of positive impact, all of those metrics have him quite overrated. Well, but the really crazy value is again up to Wins Produced. Michael Smith has the 2nd highest WP48 of all PF in the league in 2001 (only the great Bo Outlaw was better). Let us ignore the fact that Berri is cheating here a bit by considering him entirely a PF, because as a SF his WP48 would be even higher, Michael Smith according to Wins Produced added 9.58 wins to the 19 wins Wizards, the guy with whom on the court the Wizards played 6.8 points worse than with him, was the reason for half of those wins. With his real position (3.6 instead of 4), Smith would have had even 1 win more and would have added as much as Vince Carter in 2001 or the great Fred Hoiberg (yes, the guy playing for the Chicago Bulls in 2001, a team which was even worse than the Wizards, but at least the Bulls with Hoiberg on the court played a bit better than the Wizards with Smith, -9.6 per 100 possessions).
So, the only metric, which is actually seeing the greatness of Michael Smith, the 28 yr old forward, was Wins Produced. Nobody else saw that, no coach, no Gm in the league saw, how much Michael Smith was able to add to a team. And all of those people having a job at that time were obviously just morons, because no team gave this guy a new contract. No team considered Michael Smith someone who would be able to help the team win more games than with any sort of other available replacement. The 2nd best PF in the league in 2001 according to WP48 had to go to Italy in order to play professional basketball, his team went 14-22 that season. Michael Smith couldn't even make a big impact on such a team in Italy, but WP48 considered him one of the best players in the league (7th overall among all players with meaningful minutes played!).

Yes, all metrics have outliers, we see players being underrated by those metrics or overrated, but I really don't know any metric besides WP48 which would claim that a player, who had just a limited role and was actually not only not good, but really bad in comparison to other NBA players, would be rated among the top players in the league. We are talking about a player playing 1600 minutes and WP48 couldn't figure out that this guy is actually not good at all. Michael Smith in 2001 was considered a better player per minute than all players selected for the All-NBA teams with the exception of Dikembe Mutombo, who only finished ahead of Michael Smith in WP48, because Smith is listed as 4.0 in Berri's ranking that season instead of 3.6 as his position.

How is that possible? Well, the formula used to calculate offensive efficiency is the reason for that kind of outliers. Berri uses a team formula for offensive efficiency in order to determine the intrinsic value of the individual boxscore entries. To calculate offensive efficiency you have count the points scored by that player and calculate his possessions employed (PE).

PE = FGA + 0.45* FTA + TO - OREB

For Smith in 2001 that was: PE = 218 + 0.45* 154 + 62 - 172 = 177.3

Smith scored 301 points (52.7 TS%). That makes 1.7 points per used possession or 170 ORtg or 1.7 times better than league average. Wins Produced considered Michael Smith to be an awesome efficient offensive player. And he was quite the opposite of that.

I thought my point was just like the one before it - take single player for metric that I may or may not find to be 100% accurate, present with 0 context for laugh test. Did I do it wrong? Or does it only count for wins produced because we have to post something like that once in a while, in case someone new is reading and hasn't seen the 5000 other posts? Here, I'll try again.

RAPM, whether you use a prior or not, has Faried as below average even though every other metric I can find says he's well above! Who would ever believe such an obviously useless number?

kenneth faried's 82games.com counterpart data and Synergy defensive data (PPP and eFG% allowed) are not very good, but the rest of his stats are really excellent - high overall shooting, efficient offense, average turnovers per touch for a PF, very good rebounding (excellent offensive rebounding), good steal and shot blocking rates...

his touches/min, and per minute and per touch stats look very similar to those of the rookie season of buck williams, which were awfully good (he was ROY in 81-82). faried's just not playing alot of minutes...

xkonk wrote:I thought my point was just like the one before it - take single player for metric that I may or may not find to be 100% accurate, present with 0 context for laugh test. Did I do it wrong? Or does it only count for wins produced because we have to post something like that once in a while, in case someone new is reading and hasn't seen the 5000 other posts?

Wow, you actually really have no idea what the point was. I always thought you were just trying to play stupid in order to make others question their statement, but you are in fact really did not understand why Dampier was brought up. Dampier just fits a pattern with WP48, the same way Michael Smith fits that pattern. Role players with little usage who happened to be good offensive rebounders are ending up really high despite EVERY other metric is telling they are not that great. The point is not that one player here or there is an outlier, all metrics will look like this, but in WP48 limited role players are seen as crazy superstars while in reality that is not seen in the results at all.

xkonk wrote: RAPM, whether you use a prior or not, has Faried as below average even though every other metric I can find says he's well above! Who would ever believe such an obviously useless number?

http://stats-for-the-nba.appspot.com/pm/960.html

Look at how the same lineups perform with Faried in comparison to the lineups with any of those other 5 players. Those numbers are adjusted for the strength of the opponent lineups. Do you see something here? Maybe the boxscore metrics are overrating Faried? Can you imagine that? Maybe Farieds defense isn't that good (as bchaikin pointed out, the counterpart numbers and Synergy data is all but great for him). Faried has, according to RAPM a positive impact on offense, but not on defense. That is a concern, especially when the lineups with Al Harrington instead of Kenneth Faried are playing much better defensively. And again, we have a player with a really high offensive rebounding number. Using Berri's PE formula here we are getting for Faried just 246.7 used possessions while scoring 398 points. Well, Faried is playing off of anyone else, making his effort on the offensive glass, but in most cases he is losing that battle for the offensive rebound and is late back on defense. That hurts the defense of the Nuggets and Faried can not make up for that with his hustle play.

Funny thing is my SPM has him also rated positively on offense while basically average on defense. Like every boxscore metric my metric has an issue with that kind of behaviour by a player, because the boxscore metric does not see HOW a rebound is obtained. But we also see a pretty common theme among players with high ORB%, they tend to be a negative factor for the defense. In this season for example, the correlation coefficient for ORB% and DRtg is 0.17. Teams with higher ORB% tend to have worse defense. And that is just not an outlier, but a very good trend over the last couple of seasons. Going back on defense instead of trying to get the offensive rebound proves to be benefical for the team overall defense. WP48 on the other hand suggest the opposite, the players should rather try to get the offensive rebound in order to push their WP48.

The individual boxscore entries can explain offense very well, but for the defensive part it comes down to a coin flip whether it is correct or not. RAPM has Faried as positive offensively, but negative to a bigger degree on defense. And given the fact that the Nuggets with Faried on the court are getting outscored in average (2nd worst OnCourt Rating behind Mozgov) the RAPM is at least in agreement with the result, while the boxscore numbers are not. And as the lineup numbers showed, the same 4 players with Mozgov together are playing better defensively than with Faried. So, we can hardly blame that on Mozgov here.

My point, since it seems to have not gotten through, is that any time Berri or wins produced comes up, it seems to give people free rein to say whatever they want without asking if the same criticism applies to any of the other things that we talk about on the board. You can name an outlier player on WP and hold it up as an example of the obvious flaws in the measure (which is exactly what Statman did, no more evidence necessary beyond the words Erick Dampier), but if you do the same for any other measure then you have to take a serious look at the whys and hows of why that number came to be (for example, superstar Nick Collison). You can make fun of the journals that Berri has published in without asking where other sports research is published. You can bring up questions about position adjustments or team defense and they're shot down immediately simply because they show up in WP, no further argument needed. It's a dead-end of a discussion.

I understand that people don't like Berri or WP. It's exceedingly obvious; it could be the title of the forum. But at least right now, the forum actually has words like 'research' and 'objective' tied to it, and at this point discussion of anything even tangentially related to WP seems to inspire little of the former and none of the latter. I don't care if Berri never comes up on the board again, what I'm asking for is some amount of reasoned discussion when people do post. Maybe the moderators can make the 'WP discredited' string stickied so it always pops up right at the top of the forum. Perhaps we could save time by giving everyone a signature on all their posts, "Berri's an idiot, amirite?" Or do the writers here really enjoy bringing Berri up so much that they'd rather keep banging the same drum over and over again?

xkonk wrote:My point, since it seems to have not gotten through, is that any time Berri or wins produced comes up, it seems to give people free rein to say whatever they want without asking if the same criticism applies to any of the other things that we talk about on the board.

It just seems to you that way, because it seems to me that you were not involved in many of the discussions about Berri before. The very basic of the critique has never really changed, it was always very similar. Now, when names are pointed out, they often fit a very common theme. For bigs we can find low usage players with high rebounding numbers to be on top and for small players those with a lot of assists and low usage will have an advantage. WP48 is punishing players for taking shots, that is an issue which was brought up from the very beginning. And I have to say that I don't see the point of constantly repeating the same things (even though I did).

xkonk wrote: You can name an outlier player on WP and hold it up as an example of the obvious flaws in the measure (which is exactly what Statman did, no more evidence necessary beyond the words Erick Dampier), but if you do the same for any other measure then you have to take a serious look at the whys and hows of why that number came to be (for example, superstar Nick Collison).

Honestly, I encouraged everyone to read what Berri is writing. Reading the papers, understanding the underlying hypothesis and the algorithm is essential in order to understand the flaw of the method. And seriously, if someone really understand the method and basketball (with good enough knowledge about other metrics) the flaw should be obvious.
As I said WP48 is producing just a large amount of such "odd" players, while all those players have pretty common strength according to the boxscore. That should be rather obvious too, because it is a result of the underlying method.

xkonk wrote:You can make fun of the journals that Berri has published in without asking where other sports research is published.

Pointing out the TRUTH is not making fun of something. And again, only because other sport researchers are also publishing in low impact journals (no idea whether that is actually true), doesn't make Berri's work any better. It is just a strawman you like to talk about, as if that would say anything about the quality of Berri's work.

xkonk wrote: You can bring up questions about position adjustments or team defense and they're shot down immediately simply because they show up in WP, no further argument needed. It's a dead-end of a discussion.

Why do you think that those things were never in deep discussed before? Do you really believe Berri did something revolutionary with that? We saw that before, Dean Oliver's book has long chapters about individual offense and defense and the connection to team performances. The basis of Oliver's ORtg and DRtg is team performance level. Position adjustments was talked about before I even heard the name Berri for the first time.
And do you really not understand the inconsistency in Berri's metric? Either we can determine the intrinsic values of the boxscore entries via regression or we can not do that. If we can, no adjustment should be necessary, if we can't, the hypothesis is rejected. The hypothesis is not proven correct, when we make adjustments contrary to the hypothesis. Do you understand that point?

And stop trying to make Berri a martyr, he is not. The constant whining by people like you is not improving anything. In fact your posts have huge similarities to posts written by people claiming climate change wouldn't exist. They also bringing up some "papers" published in low impact journals in order to make it seem as if that would be legit. And if the flaws are pointed out, they starting whining about the lack of respect for their "heros". They never have anything meaningful to say top the critique, they are never able to reject the arguments, they just starting to whine about how the supposed to be great researcher they prefer is not respected enough.
If you want to have a honest discussion about that, you should start by looking at yourself. So far I have not seen any post by you which would be in agreement with your desired quality.

Sorry but having papers from "low impact" journals should not matter. What matters is the argument presented before you, not if you work for ESPN or some well known place.

Some of the most famous journals or newspapers in the world are extremely partisan. That said I agree with your position on WP, but you really veered off at the end of your previous post. Scientists wildly debate "mainstream" ideas all the time so you're wrong there.



xkonk is pointing out hypocrisy, not praising Berri. Indeed, Berri might hold some incorrect beliefs but APM also produces some of the wackiest results I've ever seen.

Came in expecting to have to argue with huevon. But I agree with him for once!

Ironically, Wins Produced shows up in economics stuff (I remember feeling really gross reading it in my Sports Econ class)...to that end, I would say that WP is somewhat mainstream.