Phil Birnbaum's article on why box score stats get it wrong
Posted: Fri Apr 15, 2011 2:26 am
page 1 of 13
Author Message
EvanZ
Joined: 22 Nov 2010
Posts: 276
PostPosted: Sat Jan 22, 2011 11:49 am Post subject: Phil Birnbaum's article on why box score stats get it wrong Reply with quote
Quote:
You know all those player evaluation statistics in basketball, like "Wins Produced," "Player Evaluation Rating," and so forth? I don't think they work. I've been thinking about it, and I don't think I trust any of them enough put much faith in their results.
http://sabermetricresearch.blogspot.com ... s-are.html
From the comments already on the post, I know some of you have already read it. Seems worthy of discussion, no?
To get the ball rolling, here was my response (one of them, anyway):
Quote:
Phil, to be (even) more rigorous about it, wouldn't it make sense to look at the change in eFG% of teammates relative to when that player is on/off the floor?
When the player is on the floor, teammates will have a certain eFG%. When that player is off the floor, if he has a large effect, teammates eFG% should change more. Isn't *that* the effect we are interested in? This would separate teams that simply have a lot of good players from teams that rely on a single player to lift teammates (say Cleveland circa 2009).
Seems to me this is more along the lines of the recent studies by Silver, Pelton, etc..
_________________
http://www.thecity2.com
http://www.ibb.gatech.edu/evan-zamir
Back to top
View user's profile Send private message
Mike G
Joined: 14 Jan 2005
Posts: 3586
Location: Hendersonville, NC
PostPosted: Sat Jan 22, 2011 1:41 pm Post subject: Re: Phil Birnbaum's article on why box score stats get it wr Reply with quote
Quote:
Sabermetric basketball statistics are too flawed to work .
You know all those player evaluation statistics in basketball, like "Wins Produced," "Player Evaluation Rating," and so forth? I don't think they work. I've been thinking about it, and I don't think I trust any of them enough put much faith in their results.
sigh
_________________
`
36% of all statistics are wrong
Back to top
View user's profile Send private message Send e-mail
greyberger
Joined: 27 Sep 2010
Posts: 50
PostPosted: Sat Jan 22, 2011 2:28 pm Post subject: Reply with quote
I think food is too fattening to eat.
You know the food they serve at McDonald's and Burger King, an so forth? I don't think that food is any good for you. I've been thinking about it, and I don't trust their advertising either.
Back to top
View user's profile Send private message Send e-mail
Mike G
Joined: 14 Jan 2005
Posts: 3586
Location: Hendersonville, NC
PostPosted: Sat Jan 22, 2011 4:14 pm Post subject: Reply with quote
Bravo!
_________________
`
36% of all statistics are wrong
Back to top
View user's profile Send private message Send e-mail
Crow
Joined: 20 Jan 2009
Posts: 810
PostPosted: Sat Jan 22, 2011 8:30 pm Post subject: Reply with quote
I don't follow baseball hardly at all and have only lightly skimmed sabermetric sites a handful of times looking at metric approaches. I'd welcome prominent baseball sabermetricians looking at basketball though I'd hope they'd bring their A game and present depth of research and actively discuss it. Looks like Phil is willing to be interactive. There may be additional approaches / tools from their sport they could translate and bring over to basketball analysis beyond what some basketball analysts familiar with or involved in baseball analysis have already done. It might be helpful for someone familiar with both fields to compare the toolkits and see where a similar tool is not present in the other field and if it could be meaningfully used there.
Back to top
View user's profile Send private message
greyberger
Joined: 27 Sep 2010
Posts: 50
PostPosted: Sun Jan 23, 2011 12:08 am Post subject: Reply with quote
Absolutely, and I've really enjoyed his posts so far, even if I don't really need to read another take-down of WP48. It's not just another perspective, it's an informed one that knows how to write eloquently on the subject.
With that said, I wish there were some way to speed up his acclimation to apebear conventional wisdom especially vis-a-vis the difference between sabermetrics and apebear metrics.
That there's no consensus as to how to incorporate scoring efficiency, scoring production and assist production into a single metric (linear or otherwise) is the kind of conventional wisdom I'm talking about.
PER and WP48 have entirely different answers to this fundamental question (and neither answer is popular among the creators of more recent, arguably better roll-up metrics). I'm not sure they're really comparable to baseball formulas, some of which are as routinely used and uncontroversial as using TS% or rebound rate here.
Back to top
View user's profile Send private message Send e-mail
Crow
Joined: 20 Jan 2009
Posts: 810
PostPosted: Sun Jan 23, 2011 2:46 am Post subject: Reply with quote
I don't know if this is worth it but when you say "apebear" are you simply trying to use a word instead of saying APBR? Or are you doing something more? I sometimes over-interpret, and therefore I ask.
Last edited by Crow on Sun Jan 23, 2011 8:44 pm; edited 2 times in total
Back to top
View user's profile Send private message
greyberger
Joined: 27 Sep 2010
Posts: 50
PostPosted: Sun Jan 23, 2011 3:15 am Post subject: Reply with quote
It's not a word I hear spoken a lot. So I might be entirely off-base on this first part. But I've always imagined sabermetrics to be pronounced say-bare instead of say-burr.
Thus, apbr would be ape-bare or app-bare, depending on what you want to do with that p sound. Ape-bear has amusing connotations so apebear it is.
Back to top
View user's profile Send private message Send e-mail
Crow
Joined: 20 Jan 2009
Posts: 810
PostPosted: Sun Jan 23, 2011 3:32 am Post subject: Reply with quote
Ok, so it is simple and amusing to you and the amusement is left for interpretation and possible amusement of others.
Back to the substance...
The general 'baseball is different than basketball in many ways because of 5 on 5 instead of mostly independent offensive actions' argument is well known. But in "saberland", if something is not considered cut and dried, is the solution to be found and agreed upon to be found by predictive power of the tool (simple or complex, preferably simple but may be not if necessary) or by "right visioning" of fundamental values?
I barely know that turf but I have come across some metric disagreement discussions that sounded deep and long and unresolved to everyone's satisfaction. With tests upon different tests and results analysis from varies parties is more than a majority opinion eventually and usually formed even on tough items after initial contest? How many metrics or questions of value have sizable enduring minorities? Is 'few" accurate or overstated?
If ASPM or eWins or EZPM or some other metric or offshoot or blend do better than PER or WP in predicting the future at player and / or team level and if it were shown to be because of its "incorporation of scoring efficiency, scoring production and assist production", I'd hope they'd gain greater recognition from those in positions to highlight it.
The great, sometimes a priori "weight" debate for these metrics has some value on its own and in the metric fine-tuning but I'd hope the right multi-season predictive or retrodictive test would be set up and run. Past tests did not settle everything.
Last edited by Crow on Sun Jan 23, 2011 8:52 pm; edited 1 time in total
Back to top
View user's profile Send private message
Mike G
Joined: 14 Jan 2005
Posts: 3586
Location: Hendersonville, NC
PostPosted: Sun Jan 23, 2011 8:07 am Post subject: Reply with quote
Has anyone invited this fellow to come by here?
I for one think (in spite of his 0-fer in opening stanza) that fresh, outside opinions often have more value than those steeped in 'APBR conventional wisdom'; re. the CW, it seems there's a fixation on certain positions and sometimes extreme reluctance to logically reconsider.
It sort of belies the notion of 'advanced statistics' when these statistics fail to advance. In fact, when it's almost written in stone that they must not advance -- the next level becomes 2nd-generation advanced, or post-advanced, or some other ungainly term. Why not just continually evolve?
Optimistically, I believe any thoughtful person can see the merits and flaws in most of the statistics seen here. Most people don't know what can be done differently, whether a stat can be fixed or just scrapped (or ignored). Inflexibility is not our friend.
_________________
`
36% of all statistics are wrong
Back to top
View user's profile Send private message Send e-mail
gabefarkas
Joined: 31 Dec 2004
Posts: 1313
Location: Durham, NC
PostPosted: Mon Jan 24, 2011 9:28 am Post subject: Reply with quote
Crow wrote:
I'd welcome prominent baseball sabermetricians looking at basketball though I'd hope they'd bring their A game and present depth of research and actively discuss it.
Well, he does at least read these forums, even if he doesn't post.
Back to top
View user's profile Send private message Send e-mail AIM Address
Crow
Joined: 20 Jan 2009
Posts: 810
PostPosted: Mon Jan 24, 2011 4:31 pm Post subject: Reply with quote
I thought about extending an invitation on his site, but then I checked and noticed the registration too. He can decide where and how much to interact on his own.
Back to top
View user's profile Send private message
phil_birnbaum
Joined: 29 Nov 2006
Posts: 2
PostPosted: Mon Jan 24, 2011 6:21 pm Post subject: Reply with quote
Yup, I read some of these discussions, but I normally don't come by unless someone outside links to something here. BTW, thanks for posting my profile, I had forgotten my user ID. Smile
Back to top
View user's profile Send private message Visit poster's website
DSMok1
Joined: 05 Aug 2009
Posts: 608
Location: Where the wind comes sweeping down the plains
PostPosted: Mon Jan 24, 2011 9:05 pm Post subject: Reply with quote
phil_birnbaum wrote:
Yup, I read some of these discussions, but I normally don't come by unless someone outside links to something here. BTW, thanks for posting my profile, I had forgotten my user ID. Smile
Welcome, Phil! I read just about everything you, Tango, and MGL write; my primary objective has been learning how to apply the same statistically sound principles to basketball statistics. Most of my statistical know-how comes from reading The Book blog and your blog, and following the other sabermetric sites as well.
_________________
GodismyJudgeOK.com/DStats
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Ed Küpfer
Joined: 30 Dec 2004
Posts: 785
Location: Toronto
PostPosted: Wed Jan 26, 2011 9:02 am Post subject: Reply with quote
KevinP replies and is far too modest to post the link here.
_________________
ed
Author Message
EvanZ
Joined: 22 Nov 2010
Posts: 276
PostPosted: Sat Jan 22, 2011 11:49 am Post subject: Phil Birnbaum's article on why box score stats get it wrong Reply with quote
Quote:
You know all those player evaluation statistics in basketball, like "Wins Produced," "Player Evaluation Rating," and so forth? I don't think they work. I've been thinking about it, and I don't think I trust any of them enough put much faith in their results.
http://sabermetricresearch.blogspot.com ... s-are.html
From the comments already on the post, I know some of you have already read it. Seems worthy of discussion, no?
To get the ball rolling, here was my response (one of them, anyway):
Quote:
Phil, to be (even) more rigorous about it, wouldn't it make sense to look at the change in eFG% of teammates relative to when that player is on/off the floor?
When the player is on the floor, teammates will have a certain eFG%. When that player is off the floor, if he has a large effect, teammates eFG% should change more. Isn't *that* the effect we are interested in? This would separate teams that simply have a lot of good players from teams that rely on a single player to lift teammates (say Cleveland circa 2009).
Seems to me this is more along the lines of the recent studies by Silver, Pelton, etc..
_________________
http://www.thecity2.com
http://www.ibb.gatech.edu/evan-zamir
Back to top
View user's profile Send private message
Mike G
Joined: 14 Jan 2005
Posts: 3586
Location: Hendersonville, NC
PostPosted: Sat Jan 22, 2011 1:41 pm Post subject: Re: Phil Birnbaum's article on why box score stats get it wr Reply with quote
Quote:
Sabermetric basketball statistics are too flawed to work .
You know all those player evaluation statistics in basketball, like "Wins Produced," "Player Evaluation Rating," and so forth? I don't think they work. I've been thinking about it, and I don't think I trust any of them enough put much faith in their results.
sigh
_________________
`
36% of all statistics are wrong
Back to top
View user's profile Send private message Send e-mail
greyberger
Joined: 27 Sep 2010
Posts: 50
PostPosted: Sat Jan 22, 2011 2:28 pm Post subject: Reply with quote
I think food is too fattening to eat.
You know the food they serve at McDonald's and Burger King, an so forth? I don't think that food is any good for you. I've been thinking about it, and I don't trust their advertising either.
Back to top
View user's profile Send private message Send e-mail
Mike G
Joined: 14 Jan 2005
Posts: 3586
Location: Hendersonville, NC
PostPosted: Sat Jan 22, 2011 4:14 pm Post subject: Reply with quote
Bravo!
_________________
`
36% of all statistics are wrong
Back to top
View user's profile Send private message Send e-mail
Crow
Joined: 20 Jan 2009
Posts: 810
PostPosted: Sat Jan 22, 2011 8:30 pm Post subject: Reply with quote
I don't follow baseball hardly at all and have only lightly skimmed sabermetric sites a handful of times looking at metric approaches. I'd welcome prominent baseball sabermetricians looking at basketball though I'd hope they'd bring their A game and present depth of research and actively discuss it. Looks like Phil is willing to be interactive. There may be additional approaches / tools from their sport they could translate and bring over to basketball analysis beyond what some basketball analysts familiar with or involved in baseball analysis have already done. It might be helpful for someone familiar with both fields to compare the toolkits and see where a similar tool is not present in the other field and if it could be meaningfully used there.
Back to top
View user's profile Send private message
greyberger
Joined: 27 Sep 2010
Posts: 50
PostPosted: Sun Jan 23, 2011 12:08 am Post subject: Reply with quote
Absolutely, and I've really enjoyed his posts so far, even if I don't really need to read another take-down of WP48. It's not just another perspective, it's an informed one that knows how to write eloquently on the subject.
With that said, I wish there were some way to speed up his acclimation to apebear conventional wisdom especially vis-a-vis the difference between sabermetrics and apebear metrics.
That there's no consensus as to how to incorporate scoring efficiency, scoring production and assist production into a single metric (linear or otherwise) is the kind of conventional wisdom I'm talking about.
PER and WP48 have entirely different answers to this fundamental question (and neither answer is popular among the creators of more recent, arguably better roll-up metrics). I'm not sure they're really comparable to baseball formulas, some of which are as routinely used and uncontroversial as using TS% or rebound rate here.
Back to top
View user's profile Send private message Send e-mail
Crow
Joined: 20 Jan 2009
Posts: 810
PostPosted: Sun Jan 23, 2011 2:46 am Post subject: Reply with quote
I don't know if this is worth it but when you say "apebear" are you simply trying to use a word instead of saying APBR? Or are you doing something more? I sometimes over-interpret, and therefore I ask.
Last edited by Crow on Sun Jan 23, 2011 8:44 pm; edited 2 times in total
Back to top
View user's profile Send private message
greyberger
Joined: 27 Sep 2010
Posts: 50
PostPosted: Sun Jan 23, 2011 3:15 am Post subject: Reply with quote
It's not a word I hear spoken a lot. So I might be entirely off-base on this first part. But I've always imagined sabermetrics to be pronounced say-bare instead of say-burr.
Thus, apbr would be ape-bare or app-bare, depending on what you want to do with that p sound. Ape-bear has amusing connotations so apebear it is.
Back to top
View user's profile Send private message Send e-mail
Crow
Joined: 20 Jan 2009
Posts: 810
PostPosted: Sun Jan 23, 2011 3:32 am Post subject: Reply with quote
Ok, so it is simple and amusing to you and the amusement is left for interpretation and possible amusement of others.
Back to the substance...
The general 'baseball is different than basketball in many ways because of 5 on 5 instead of mostly independent offensive actions' argument is well known. But in "saberland", if something is not considered cut and dried, is the solution to be found and agreed upon to be found by predictive power of the tool (simple or complex, preferably simple but may be not if necessary) or by "right visioning" of fundamental values?
I barely know that turf but I have come across some metric disagreement discussions that sounded deep and long and unresolved to everyone's satisfaction. With tests upon different tests and results analysis from varies parties is more than a majority opinion eventually and usually formed even on tough items after initial contest? How many metrics or questions of value have sizable enduring minorities? Is 'few" accurate or overstated?
If ASPM or eWins or EZPM or some other metric or offshoot or blend do better than PER or WP in predicting the future at player and / or team level and if it were shown to be because of its "incorporation of scoring efficiency, scoring production and assist production", I'd hope they'd gain greater recognition from those in positions to highlight it.
The great, sometimes a priori "weight" debate for these metrics has some value on its own and in the metric fine-tuning but I'd hope the right multi-season predictive or retrodictive test would be set up and run. Past tests did not settle everything.
Last edited by Crow on Sun Jan 23, 2011 8:52 pm; edited 1 time in total
Back to top
View user's profile Send private message
Mike G
Joined: 14 Jan 2005
Posts: 3586
Location: Hendersonville, NC
PostPosted: Sun Jan 23, 2011 8:07 am Post subject: Reply with quote
Has anyone invited this fellow to come by here?
I for one think (in spite of his 0-fer in opening stanza) that fresh, outside opinions often have more value than those steeped in 'APBR conventional wisdom'; re. the CW, it seems there's a fixation on certain positions and sometimes extreme reluctance to logically reconsider.
It sort of belies the notion of 'advanced statistics' when these statistics fail to advance. In fact, when it's almost written in stone that they must not advance -- the next level becomes 2nd-generation advanced, or post-advanced, or some other ungainly term. Why not just continually evolve?
Optimistically, I believe any thoughtful person can see the merits and flaws in most of the statistics seen here. Most people don't know what can be done differently, whether a stat can be fixed or just scrapped (or ignored). Inflexibility is not our friend.
_________________
`
36% of all statistics are wrong
Back to top
View user's profile Send private message Send e-mail
gabefarkas
Joined: 31 Dec 2004
Posts: 1313
Location: Durham, NC
PostPosted: Mon Jan 24, 2011 9:28 am Post subject: Reply with quote
Crow wrote:
I'd welcome prominent baseball sabermetricians looking at basketball though I'd hope they'd bring their A game and present depth of research and actively discuss it.
Well, he does at least read these forums, even if he doesn't post.
Back to top
View user's profile Send private message Send e-mail AIM Address
Crow
Joined: 20 Jan 2009
Posts: 810
PostPosted: Mon Jan 24, 2011 4:31 pm Post subject: Reply with quote
I thought about extending an invitation on his site, but then I checked and noticed the registration too. He can decide where and how much to interact on his own.
Back to top
View user's profile Send private message
phil_birnbaum
Joined: 29 Nov 2006
Posts: 2
PostPosted: Mon Jan 24, 2011 6:21 pm Post subject: Reply with quote
Yup, I read some of these discussions, but I normally don't come by unless someone outside links to something here. BTW, thanks for posting my profile, I had forgotten my user ID. Smile
Back to top
View user's profile Send private message Visit poster's website
DSMok1
Joined: 05 Aug 2009
Posts: 608
Location: Where the wind comes sweeping down the plains
PostPosted: Mon Jan 24, 2011 9:05 pm Post subject: Reply with quote
phil_birnbaum wrote:
Yup, I read some of these discussions, but I normally don't come by unless someone outside links to something here. BTW, thanks for posting my profile, I had forgotten my user ID. Smile
Welcome, Phil! I read just about everything you, Tango, and MGL write; my primary objective has been learning how to apply the same statistically sound principles to basketball statistics. Most of my statistical know-how comes from reading The Book blog and your blog, and following the other sabermetric sites as well.
_________________
GodismyJudgeOK.com/DStats
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Ed Küpfer
Joined: 30 Dec 2004
Posts: 785
Location: Toronto
PostPosted: Wed Jan 26, 2011 9:02 am Post subject: Reply with quote
KevinP replies and is far too modest to post the link here.
_________________
ed