recovered page 1 of 3
Author Message
BobboFitos
Joined: 21 Feb 2009
Posts: 201
Location: Cambridge, MA
PostPosted: Tue Jul 14, 2009 1:54 am Post subject: Scorekeeper story Reply with quote
I asked this friend/acquaintance of mine if he minded if I posted this, got the green light, so figured I'd go ahead and do so... (essentially the perils of the box score; there have been posts about scorekeeper bias before)
Quote:
The advanced metrics for basketball really aren't very good. Basketball is just so much harder to apply meaningful numbers to than baseball, for some obvious reasons and some less obvious ones.
There's very little subjectivity w/ baseball stats. There is a MASSIVE amount of subjectivity and intentional inaccuracy with basketball stats. If you ever want to be amused (or horrified), go back and do stats for an NBA game and compare your results to NBA results. There will be a significant discrepancy.
So you have bad/inaccurate initial info (the garbage that the NBA puts out as box scores and everyone else bases their advanced numbers on) and the mere fact that there are a lot of things that aren't easily measured (like shot creation, as Seth mentioned).
Personal story: I was the head gameday stats accumulation guy for the Grizzlies for their first three seasons. When I was preparing for the role the summer before their inaugural year, the NBA implemented a new software program for stats entry. We had league-wide training in Detroit. When there, I spent most of my non-classroom/training time in my hotel room with a sweet A/V suite doing past games myself, then comparing them line-by-line with the actual stats. IIRC, there are around 400ish entered/recorded events per NBA game on average. I think at the time I calculated that the average NBA stat crew had about 20 unintentional errors per game - missing events, wrong players getting credit unintentionally. (For those who care, I had myself and the Griz folks down to 3-4 unintentional errors per game and I usually knew what they were and was able to go back and fix them before submitting the file to the NBA.)
Anyway...on top of that ~20 errors per game, you have over double that in intentional errors. By intentional errors, I mean events that never happened (eg. loose ball rebound is deflected out of bounds by visiting team, instead of correct call - team rebound home team - you award the rebound to a home player in the viscinity...or fake blocks - among the easiest things to make up, next to steals and assists)...or events that are awarded to the wrong player (rebounds, steals, turnovers are the most common). The intentional errors are organizationally sanctioned/encouraged - they increase national media coverage/interest and increase your franchise's and player's visibility. There is also league pressure to protect/enhance the stats of the elite players. For example, I would guess that Stockton got between 1 and 2 assists per game for free. Partly because I disagreed with the blatant stat manipulation (that I did) and partly because I'm a Laker fan, I gave Nick Van Exel like 23 assists one game. If he was vaguely close to a guy making a shot, I found a way to give him an assist. Afterwards, I fully expected someone to talk to me about it. Indeed they did. A senior management guy - "great job Alex, that'll get this game on Sportscenter tomorrow morning!" We (VAN) lost badly, of course.
I also got bitched out by an Atlanta management guy because he felt I hadn't hooked Mutombo up enough w/ blocks in a particular first half. (I hadn't - I didn't like him because he was partly responsible for beating the Sonics and because I thought he was a bit of a punk so I made sure he didn't get a singly block that I wasn't sure he'd gotten - which was one in that half.) I told the management guy that the box score reflected the game and if Mutombo wanted more blocks, he needed to earn them. About 5 minutes later, Deke walked out for pregame warmups, asked the official scorer (the person who enters fouls and points in the archaic official scorebook) who does stats, she kindly pointed him to me, and he proceeded to glare at me for about a minute (which is, imo, a really long time for a gigantic man to glare at you). I want to say he blocked three 2nd-half shots and after each one, he made a point of, um, ensuring that I'd gotten them.
Figured you guys would enjoy this. I especially like the Mutombo story.
_________________
http://pointsperpossession.com/
@PPPBasketball
Back to top
View user's profile Send private message
erivera7
Joined: 19 Jan 2009
Posts: 185
Location: Chicago, IL
PostPosted: Tue Jul 14, 2009 2:17 am Post subject: Reply with quote
Interesting story. Thanks for sharing.
_________________
@erivera7
I cover the Orlando Magic - Magic Basketball
Back to top
View user's profile Send private message
jmethven
Joined: 16 May 2005
Posts: 51
PostPosted: Tue Jul 14, 2009 11:00 am Post subject: Re: Scorekeeper story Reply with quote
Fascinating... and frightening. I was curious, so I looked up the Van Exel game the author is referring to (1/5/97) on B-R. According to the box score he assisted on 62.6% of the Lakers' field goals, while committing just 2 turnovers. His 23 assists that night rank as the 12th highest single-game total in NBA history as well as his personal career best.
I had always assumed that most statistical errors would even out over the course of time. But that was assuming there was no bias in the discrepancies. The existence of Van Exel's '23 assist' game suggests otherwise. Intentional errors, especially if widespread, could create a lot of problems.
I wouldn't jump to drastic conclusions regarding the validity of box score-based metrics, but it's clear that there needs to be a greater push for consistency. Perhaps a new APBRmetrics game charting project is in order?
Back to top
View user's profile Send private message
DraftGuy
Joined: 23 May 2007
Posts: 22
PostPosted: Tue Jul 14, 2009 12:19 pm Post subject: Reply with quote
rofl! statistics are used right if u ignore blips and use lots of data. i might change the stats if deke glared at me!
Back to top
View user's profile Send private message
socialpsychologist
Joined: 10 Jul 2009
Posts: 6
PostPosted: Tue Jul 14, 2009 12:56 pm Post subject: Reply with quote
If this account even has a shred of truth to it, then it could explain some behaviors, such as Mutumbo's finger wag. He could be talking to the scorekeeper as much as anybody else.
Back to top
View user's profile Send private message
Harold Almonte
Joined: 04 Aug 2006
Posts: 616
PostPosted: Tue Jul 14, 2009 7:22 pm Post subject: Reply with quote
If all of this isn't an exaggeration, then we must say it is negligible a proffesional league manages that in a so informal way (taking account a pair of assists on records might mean extra dollars in a contract year- or might not mean anything given the bad decissions GMs eventually make). We could even say this might be a kind of "steroid-ed" stats NBA scandal.
Back to top
View user's profile Send private message
BobboFitos
Joined: 21 Feb 2009
Posts: 201
Location: Cambridge, MA
PostPosted: Wed Jul 15, 2009 12:13 am Post subject: Reply with quote
Harold Almonte wrote:
If all of this isn't an exaggeration, then we must say it is negligible a proffesional league manages that in a so informal way (taking account a pair of assists on records might mean extra dollars in a contract year- or might not mean anything given the bad decissions GMs eventually make). We could even say this might be a kind of "steroid-ed" stats NBA scandal.
The thing though is it's an imbalanced bias; it's that certain players get more steals, blocks, assists, whatever, but at the cost of other players. So it makes it that much harder to discern who is actually benefiting (and then who is bearing the cost)
_________________
http://pointsperpossession.com/
@PPPBasketball
Back to top
View user's profile Send private message
Harold Almonte
Joined: 04 Aug 2006
Posts: 616
PostPosted: Wed Jul 15, 2009 1:42 am Post subject: Reply with quote
Scorekeeping can't affect the game outcome (scoring is not subjective), but it can affect the market, not other players directly, since counterpart stats aren't recorded (blocks against are actually recorded).
Back to top
View user's profile Send private message
Mike G
Joined: 14 Jan 2005
Posts: 3623
Location: Hendersonville, NC
PostPosted: Wed Jul 15, 2009 5:55 am Post subject: Reply with quote
Quote:
I would guess that Stockton got between 1 and 2 assists per game for free.
This surprises me, since Utah in the Stockton era was generally not among the teams more freely jacking up their home assists. It could be that Stockton was getting assists that should have gone to teammates. Or it could be that he was getting more 'free' assists on the road (due to league pressure?)
When 'Alex' refers to "organizationally sanctioned/encouraged" scorekeeping errors, that would seem to refer to teams rather than the league at large. So if Utah isn't jacking up Stockton's assists, why would other teams do so?
It may just be that 1-2 'free' assists on top of 10-12 'real' ones is a normal ratio, league-wide.
_________________
`
36% of all statistics are wrong
Last edited by Mike G on Wed Jul 15, 2009 11:50 am; edited 1 time in total
Back to top
View user's profile Send private message Send e-mail
devin3807
Joined: 11 Oct 2007
Posts: 66
PostPosted: Wed Jul 15, 2009 10:39 am Post subject: Reply with quote
On the flip side to inflating your players stats for coverage, what if a cash-strapped team made it harder for their contract year starting point guard to get assists?
Back to top
View user's profile Send private message AIM Address
kjb
Joined: 03 Jan 2005
Posts: 865
Location: Washington, DC
PostPosted: Wed Jul 15, 2009 12:32 pm Post subject: Reply with quote
Mike G wrote:
Quote:
I would guess that Stockton got between 1 and 2 assists per game for free.
This surprises me, since Utah in the Stockton era was generally not among the teams more freely jacking up their home assists. It could be that Stockton was getting assists that should have gone to teammates. Or it could be that he was getting more 'free' assists on the road (due to league pressure?)
When 'Alex' refers to "organizationally sanctioned/encouraged" scorekeeping errors, that would seem to refer to teams rather than the league at large. So if Utah isn't jacking up Stockton's assists, why would other teams do so?
It may just be that 1-2 'free' assists on top of 10-12 'real' ones is a normal ratio, league-wide.
That's basically what I found when I did my mini-study of assists awhile back. It was a small sample (10 games and about 400 total assists), but I found assists over-rewarded (at least according to my eyes) by about 20%.
Back to top
View user's profile Send private message AIM Address Yahoo Messenger
mtamada
Joined: 28 Jan 2005
Posts: 377
PostPosted: Wed Jul 15, 2009 3:49 pm Post subject: Re: Scorekeeper story Reply with quote
jmethven wrote:
Perhaps a new APBRmetrics game charting project is in order?
The NBA/Stats 6-camera project could presumably do this, I don't know if auditing/correction of game stats is part of their mission though.
Even with existing video, it would take only several hours for the NBA to double-check several games and verify the accuracy of their box scores and play-by-plays.
Having done some WNBA defensive stats for DeanO a few years ago, I certainly did notice an average of almost one discrepancy per game (meaning a discrepancy about a clear-cut event, not a discrepancy about judgement calls such as whether the shot was blocked or a simple miss, or was there a steal or a simple turnover). But I wasn't attempting to track overall stats, so I was looking at only a narrow range of events.
Back to top
View user's profile Send private message
IrishHand
Joined: 15 Jul 2009
Posts: 115
PostPosted: Thu Jul 16, 2009 1:23 pm Post subject: Reply with quote
First - I am the author of the quoted material in the original post. Some interesting replies and feedback that I'll respond to.
Quote:
I had always assumed that most statistical errors would even out over the course of time.
Correct. The incidence of unintentional errors is fairly high for most teams imo as a consequence of the wrong people being responsible for that process. Data entry should be done by computer geeks, not basketball fans. My best work, if you want to call it that, came when I had a computer buddy who didn't know a thing about basketball or care a whit about the NBA doing the data entry.
For the curious, most teams (afaik) have a 3-man stat accumulation team - one caller, who verbally announces the play, and two inputters, who input the stat calls into the stat software on their laptops. Some teams have one caller, one inputter - and that's how we did things in Vancouver when I was there. We had the second computer and would sometimes have a guy there, but he had no real in-game responsibilities and generally just got to enjoy the game and distribute box scores when appropriate.
Regardless, you are correct to assume that the unintentional errors probably don't have much of an impact in the long run.
Quote:
Fascinating... and frightening. I was curious, so I looked up the Van Exel game the author is referring to (1/5/97) on B-R. According to the box score he assisted on 62.6% of the Lakers' field goals, while committing just 2 turnovers. His 23 assists that night rank as the 12th highest single-game total in NBA history as well as his personal career best.
I'd have to watch the tape again to be sure, but I'm pretty sure he had about a 3-4 phantom assists in that game and another half-dozen that were, um, very subjective. Don't think that was normal though - that was probably the most intentionally egregious I ever called a game. I was young and immature...
Of course, I'd like to see Skiles 30-assist game tape as well, since I'd be stunned if that wasn't a very soft number as well. It's near impossible to legitimately make 30 passes in a game that lead directly to a made basket.
Quote:
I wouldn't jump to drastic conclusions regarding the validity of box score-based metrics, but it's clear that there needs to be a greater push for consistency.
Push by whom? The league and its member teams are perfectly happy with inflated and manipulated stats - they serve a purpose and, imo, they serve it pretty effectively. The portion of fans who care about their accuracy in the way that people on this forum probably do is very, very small. (Don't get me wrong, I'd absolutely love for their to be much, much better source data for basketball statistical analysis, but I don't see the league ever taking the lead in this - or any team doing so in a non-proprietary way.)
Quote:
i might change the stats if deke glared at me!
I know your line was partly in jest, but the underlying thought is 100% accurate. If you accept that there is a significant level of subjectivity in every number on the box score other than MIN, FG, 2P, FT, FTA and PTS, then it stands to reason that subjectivity will be influenced by the normal things that influence people's behavior - human relationships among them. Players that a scorer likes for whatever reasons can and will benefit, players one doesn't...don't.
Quote:
If this account even has a shred of truth to it, then it could explain some behaviors, such as Mutumbo's finger wag. He could be talking to the scorekeeper as much as anybody else.
Another quote in jest, but with considerable truth. Mutombo clearly cared about his BPG (as he should, since it bore a relationship to his expected earnings) and he made clear whenever possible when he felt he'd gotten one. Usually, it was the more marginal players (young guys, vets not getting time) who were more concerned about their stats. I would say that 90% of the players who came to talk to me at halftime or after a game were bench warmers or fringe role players.
Quote:
If all of this isn't an exaggeration, then we must say it is negligible a proffesional league manages that in a so informal way
I assume you mean "negligent"...however, as alluded to above, I don't agree. It is in the NBA's best interests for the stats to tell a compelling and interesting story. People care about double-doubles and triple-doubles and 20-rebound games and 5-steal games. They look better on the ESPN ticker than smaller numbers. A game takes place, there is going to be a moderately subjective statistical representation of that action - better if that representation is more exciting. NBA is in the business of entertainment and big numbers tell a more exciting story.
Quote:
The thing though is it's an imbalanced bias; it's that certain players get more steals, blocks, assists, whatever, but at the cost of other players.
Yes. Most stats crews have some level of awareness of their own performance relative to others - and I'm talking about things like assist-to-FG ratios, team rebound numbers, etc. With stats as subjective as assists, each crew has a grey line...when a player is close to that line, who the player is plays a massive role in whether an assist is awarded or not. If the PG passes to a post player who dribbles twice while backing down and then hits a baby hook...he may well get that assist. A SF doing the same thing probably doesn't. The stat crew preserves a more realistic AST-FGM ratio, the PG looks better. Same thing for REB, where it's even easier. Battier taps a loose rebound in the air to Yao. Rebound Yao. Yao taps a loose rebound in the air to Battier. Rebound Yao. Steals...blocks...turnovers...all same category. Location of missed shots is more player-specific (meaning no other player on the floor suffers when a missed 3 is credited as a missed 2).
Quote:
This surprises me, since Utah in the Stockton era was generally not among the teams more freely jacking up their home assists. It could be that Stockton was getting assists that should have gone to teammates. Or it could be that he was getting more 'free' assists on the road (due to league pressure?)
Just because Utah may have had a lower tendency to award assists in general doesn't mean they didn't give extra ones to Stockton. As noted in the previous paragraph, Utah may have had an "assist line" to the right of mine (I was admittedly very liberal with assists - I haven't looked at the numbers in like 10 years, but we were consistently pretty high on the AST:FM ratio leaderboard)...but that doesn't mean that they weren't simply being very conservative with all nonStockton players and being very liberal with him. I was very liberal with assists but that never benefitted Bryant Reeves or Shareef Abdur-Rahim.
Quote:
So if Utah isn't jacking up Stockton's assists, why would other teams do so?
Two reasons, one macro and one micro - though I can't comment on the weight each would have on stat crews other than the one that I was on. In the macro sense, we all knew that we were part of that NBA entertainment machine. You know who the stars are and you know their value to the league (the Magic-Bird-Jordan effect, if you will). It costs you nothing to be statistically generous to the star players in each statistical category and you believe that the league benefits from this - and few people work for an NBA team that don't want to see the NBA succeed.
Perhaps as importantly is the micro reason - reciprocity. I carefully tracked how other cities' stat crews "treated" our players and I was likely to respond in kind. If another team's stat crew failed to help our our "star" players (or at least our best version of that designation), then I would remember than and it might impact my decisions when their team came to Vancouver.
Interestingly, if you look at Bryant Reeves rookie season in Vancouver (their first season as a franchise), you'll see that his numbers were, across the board (apart from the un-manipulatable PPG) better at home. 8.1 RPG to 6.6 RPG. 1.6 APG to 1.2 APG. 0.9 BPG to 0.6 BPG. His 2nd and subsequent years, the home-away splits got a lot smaller. Why? I would offer that it was a combination of two factors - one, we had drafted Abdur-Rahim, and he became the prime beneficiary of my subjectivity, especially with "post" stats (so Reeves got less rebounds credited to him) and two, Reeves' rookie year had "established" that he was a good rebounder. (In reality, he wasn't. Most of his shots on offense were 15' jumpers, he was slow and he lacked the drive necessary to be an effective, high-volume rebounder.) In his sophmore NBA year, his overall Reb/Min numbers declined a bit, but his road Reb/Min increased. I would assume this is because other stat crews were now giving him the benefit of the doubt on some rebounding calls because he was a psuedo-established rebounder and/or because they (correctly) believed that our stat crew was taking care of their big men.
You can see similar trends in Abdur-Rahim's rookie year (he got superstar treatment from our crew, but not from many other teams) and the transition from Rahim's rookie-to-soph year, where some of those H/A gaps were bridged.
Quote:
It may just be that 1-2 'free' assists on top of 10-12 'real' ones is a normal ratio, league-wide.
That's obviously a ballpark number that I pulled out of my backside. I really only mentioned Stockton because I had a full conversation about his stats with the Utah stat crew before I'd ever done stats for an NBA game. Disturbing though it may be to purists (or to people like me who'd like basketball stats to be real and legitimate data), it's tough to argue with the notion that inflating his stats was a good thing for Utah's franchise.
Quote:
On the flip side to inflating your players stats for coverage, what if a cash-strapped team made it harder for their contract year starting point guard to get assists?
I can say conclusively that nobody from management, either our team or others, ever pushed for creative stats for any non-entertainment/storytelling purpose that I'm aware of. I tend to believe that NBA GMs and other executives rely far, far more on scouting reports than stats since they have at least a vague understanding and awareness of the inherent flaws in an NBA box score.
Plus...what's better for Orlando - to repress Jameer's APG numbers and his 3P% in his contract year so he looks worse than otherwise comparable PGs and thus might be available cheaper, or enhancing Jameer's APG and 3P% numbers knowing that may increase the price tag...but also knowing that you likely increase his popularity, the appeal of their franchise to FAs, his trade value, etc. Plus, it would be disastrous if some Jameer fanboy statistician leaked that management was intentionally repressing his stats for contractual advantage. (Total conjecture, that last paragraph - but probably some truth to it.)
Quote:
That's basically what I found when I did my mini-study of assists awhile back. It was a small sample (10 games and about 400 total assists), but I found assists over-rewarded (at least according to my eyes) by about 20%.
Depends how one defines and interprets assists. IIRC, the official rule book definition is "pass leading directly to a made basket". There isn't a to of guidance outside that. If a defensive rebounder makes a good outlet pass to a breaking wing player ahead of the defense, who then makes three dribbles and a dunk, is that an assist? If the ball is passed to a player behind the 3-point line who catches, is totally unchecked, surveys the defense, then after a couple of seconds takes and makes a wide open shot, is that an assist? What about the same pass to a guarded player who pump fakes his defender in the air, then shoots and makes? Or pump fakes, then drives past his man for the open layup? The contested layup requiring a difficult shot? There is just sooo much subjectivity in assists - and even if there wasn't, it's a seriously flawed metric for what it purports to measure. (A player who hurls the ball to a cherry-picking offensive player for a wide-open, uncontested layup gets the assist 100% of the time, whereas a player driving past his man and making a spectacular pass to a now-wide-open post player who is then fouled on the shot get squat. In the former case, the player did what nearly every player can/will do - he didn't really add much value to his team. In the latter case, the player created points and added significant value to his team.)
Apologies for the marathon post - happy to answer any further questions.
Back to top
View user's profile Send private message
kjb
Joined: 03 Jan 2005
Posts: 865
Location: Washington, DC
PostPosted: Thu Jul 16, 2009 1:36 pm Post subject: Reply with quote
By the way, maybe scorekeeping is much better of late, but the claim of 20 unintentional errors per game is exaggerated in my experience. Over a 3-season period, I hand-tracked at least a couple hundred games as part of my defensive charting project. I tracked both home and away games that involved the Wizards, as well as a smattering (10-20) of games in which the Wizards didn't play.
In charting defense, I recorded the games on TiVo and jotted my notes directly on a printout of the play-by-play. I did find errors, but nowhere close to 20 per game. Even the least accurate play-by-play I used did better than 20 errors.
I did find evidence of bias -- generally in the awarding of steals and assists. I never found a case like the one this scorekeeper describes (ball being deflected out of bounds and then awarded as a rebound to the player closest to the ball).
Probably the strangest stat award I saw was when Kwame Brown received an assist for setting a screen. It was a bone-crunching screen, but I've never encountered another screen-assist.
There are subjective plays that could reasonably be scored one way or another. For example, a player gets stripped of the ball while he's bringing the ball up to shoot. I've seen that kind of play scored a steal or a block, and on most of the plays I think the scorekeeper got it right (or at least arguably right).
But an average of 20 errors per game strains credulity. Unless, of course, my 200+ game sample just happened to catch each scorekeeping crew at their most accurate.
Back to top
View user's profile Send private message AIM Address Yahoo Messenger
IrishHand
Joined: 15 Jul 2009
Posts: 115
PostPosted: Thu Jul 16, 2009 2:23 pm Post subject: Reply with quote
I'm using a very liberal definition of unintentional error - basically any time the game log deviates from what I'd consider a "perfect" game log. In my experience, the most common tend to be things like:
-wrong player awarded stat (rebound, turnover, steal are easy to mess up w/ tips/deflections in each - catalyst for stat should get it, not ultimate beneficiary)
-missed event(s) - anytime there is a cluster of events (mutiple missed offensive rebounds/putback attempts...turnover/steal/turnover/steal sequences, that sort of thing) there is, in my experience, a much higher chance of unitentional error
Keep in mind also that when I say 20 errors, I'm counting something like (ball stripped from player going up for shot on drive --> game log shows block, rebound) as four errors. Two bad stats awarded, two actual stats not recorded. I consider every "bad" or "missed" event an error. That may be double-counting, but I was concerned with both ends at the time.
Also, from an analytical perspective, that "20/game" number is soft and really only for the sake of discussion (as is the 'double that in intentional errors'). I'm estimating it ~12 years after the fact based on a fairly small and uneven sample size (perhaps 12-15 games, mostly involving teams I cared to watch). If I'd known that the original e-mail was going to end up being scrutinized by people who cared about the underlying numbers, I would have been a lot more conservative (or tried to find the spreadsheet that I had back then with the actual numbers - flawed though they may have been). Also, those numbers are based on a pre-NBA (meaning I hadn't done an actual game), ivory-tower view of stats. I had college experience, which was pretty pure. I had the NBA rule book and related guidance on stats, and I had game tapes.
Perhaps scorekeeping is much, much better now than 12 years ago, but given the consistently strong human element, I tend to doubt that current game logs would be down to <5 errors per game. That's basically screwing up one or two sequences out of several hundred recorded events.
Quote:
ball being deflected out of bounds and then awarded as a rebound to the player closest to the ball
A shot is missed, is bobbled around by players on both teams, goes out of bounds without anyone clearly having possession. Technically, that's a team rebound to the team getting possession. Practically, it's pretty easy to call it a rebound and a turnover (or a straight rebound, depending on who ended up with possession and which team you wanted to give an individual rebound to). I don't know for certain that any other stat crew ever did that, but I am certain we did periodically and I would be surprised if nobody else ever did it. Most of the subjective stat recordings I learned from the other teams during training since I had never called an NBA game before and most of them had years and years of experience (they were pretty well all in their 30s or older, I was 20).
It is also entirely possible that I grossly overestimated how much other state crews subjectively influenced stats and our stat crew was a complete outlier in that regard. At the time, the only numbers I was concerned about were Grizzly-centric - my perception of other teams was largely anecdotal based on preseason talks with other stat crew members and (incomplete, ineffective) inseason box score analysis.
Quote:
There are subjective plays that could reasonably be scored one way or another. For example, a player gets stripped of the ball while he's bringing the ball up to shoot. I've seen that kind of play scored a steal or a block, and on most of the plays I think the scorekeeper got it right (or at least arguably right).
IMO, those plays aren't really that subjective. If the ball is still in the offensive player's hands when it's stripped, it's a steal. If the ball has left the hands, it's a block. Guidance we had (at least at the time) was very clear about this. Obviously it can be tough to determine whether the ball has actually left the player's hand, but I never had a tough time getting that right (though I will freely admit to deviating from the guidance from time to time depending on the player(s) involved in the event).
To me, once you've digested the NBA rule book and associated guidance from the league, there really isn't that much grey area - and there is almost none if you have video replay. I suggested to the NBA folks at the time that if they wanted perfect stats, all they needed to do was give crews 3 hours after the game to do back through the game log file with the video replay of the game. Not an option for them though, since they value immediate stats (for media purposes) over accurate stats.
page 2 of 3
Author Message
Jon Nichols
Joined: 18 Aug 2005
Posts: 370
PostPosted: Thu Jul 16, 2009 4:35 pm Post subject: Reply with quote
Combine these stat-keeping biases (however strong of an influence they may have) with the supposed "superstar calls" alleged by many, and you wonder how our perceptions of today's best players would change if the game was officiated/tracked by robots...
Back to top
View user's profile Send private message AIM Address
Mike G
Joined: 14 Jan 2005
Posts: 3584
Location: Hendersonville, NC
PostPosted: Thu Jul 16, 2009 8:05 pm Post subject: Reply with quote
IrishHand wrote:
... simply being very conservative with all nonStockton players and being very liberal with him. I was very liberal with assists ...
I've been tracking home/away Ast/FG ratios for teams and applying it to players' assist rates, in order to 'adjust' them to what should be closer to normal. 'Normal' in this case is the league average 'generosity' with assists granted by scorekeepers, to road teams.
It's not as bad as it once was, and this year was notably better than last year. 'Bad' means 'greater home bias' in this case. In 1995-96, the Grizz were worse than anyone but Cle and GS. Assuming their 'real' Ast/FG ratio was what everyone else in the league gave them (.562), then their home Ast/FG (.669) represents about 99 'extra' assists granted in 41 home games.
In '97, the home liberality was only 7th-worst; still about 87 'bonus' assists to the homies.
In '98, noticeably less egregious, a couple spots worse than average and some 54 extra dimes. The median in these years, leaguewide, is around 1 per game.
Oh, suddenly in '99 just a small difference in home and away assists: about 0.5 per game. That was 13th in the league!
Do these figures seem familiar, IrishHand?
What do you think of the notion that a team's Away Ast/FG is indicative of their 'real' Ast/FG, in that it is the average of all NBA teams' liberalities? Is the 'reciprocity' also possible to be estimated? Is it negligible?
By the way, in this same time period, the Clippers had the stingiest assist-givers in known history. Each year, 1996 thru 98, they gave their own guys 20-30% fewer assists than they got on the road (per FG).
_________________
`
36% of all statistics are wrong
Back to top
View user's profile Send private message Send e-mail
IrishHand
Joined: 15 Jul 2009
Posts: 115
PostPosted: Fri Jul 17, 2009 9:55 am Post subject: Reply with quote
Quote:
In 1995-96, the Grizz were worse than anyone but Cle and GS. Assuming their 'real' Ast/FG ratio was what everyone else in the league gave them (.562), then their home Ast/FG (.669) represents about 99 'extra' assists granted in 41 home games.
Wow. That's a delightful combination of awful and embarassing. I'd forgotten just how bad I was (or good, depending on your perspective).
Quote:
The median in these years, leaguewide, is around 1 per game.
I know I'm being lazy since I could take the time to run the numbers myself...but since you probably have them in front of you, what's the deviation in those numbers? If you have them in spreadsheet form, I'd love to see them. Will PM you w/ my e-mail address in case your reply to this is in the affirmative.
Quote:
Do these figures seem familiar, IrishHand?
Yeah. They definitely bother me more now than they did 10-12 years ago though. (Yesterday was an enlightening day when I examined Shareef's rookie year's H/A splits: 60/19 blocks, 50/29 steals...then his 2nd year: 52/23 blocks, 45/44 steals. Embarassed)
1 "extra" APG for the home team seems intuitively low to me, but then again I was giving 1.3-2.4.
Quote:
What do you think of the notion that a team's Away Ast/FG is indicative of their 'real' Ast/FG, in that it is the average of all NBA teams' liberalities?
I don't think that there should be any question that Away Ast/FG ratios are far, far more likely to be more accurate than the Home numbers.
Quote:
Is the 'reciprocity' also possible to be estimated? Is it negligible?
I would guess that it's negligible league-wide - the problem is that imo it applies only to a few stat categories for a few players. I wasn't giving subjective blocks to the entire Spurs team - just Robinson and Duncan. Hornacek didn't get questionable assists...Stockton did.
So saying, you could probably reason that the amount of bias applied to top away players would be no more than the general bias towards home players - meaning if my AST/FG ratio was .107 higher, I would be giving that same bias to certain away players. The difficulty is discerning that subset of the player population when it really isn't readily identifiable. My personal reciprocity was based on, in retrospect, awful information. One or two box scores plus faulty memory plus personal bias about players irrespective of how our guys got scored. I mean, I haven't a clue how Utah scored our team, but I was going to give Stockton any grey area assist. (And to be clear, when I talk about grey area, I don't mean to suggest that I felt either option was ok - assist or no assist. By grey area, I mean that I felt I could give an unwarranted assist without risk of much scrutiny.)
Quote:
By the way, in this same time period, the Clippers had the stingiest assist-givers in known history. Each year, 1996 thru 98, they gave their own guys 20-30% fewer assists than they got on the road (per FG).
Well...an earlier posted suggested a team might screw it's own players to make them cheaper to resign. I pooh-poohed that notion...but if if there was a team that was going to do that in the 90s, most critical observers would have guessed it would be Sterling's Clips. That would also account to some extent why it seemed as though many Clips played better once they left that team.
Alex
Back to top
View user's profile Send private message
Harold Almonte
Joined: 04 Aug 2006
Posts: 616
PostPosted: Fri Jul 17, 2009 10:11 am Post subject: Reply with quote
Would of all this mean, that in addition to the ordinary 20% or 30% of error margin in a player rating, we should add a 10% more maybe?
Back to top
View user's profile Send private message
Mike G
Joined: 14 Jan 2005
Posts: 3584
Location: Hendersonville, NC
PostPosted: Fri Jul 17, 2009 10:25 am Post subject: Reply with quote
IrishHand wrote:
...an earlier poster suggested a team might screw it's own players to make them cheaper to resign. I pooh-poohed that notion...but if if there was a team that was going to do that in the 90s, most critical observers would have guessed it would be Sterling's Clips. ..
And so, in one of the cruelest ironies imaginable, the primary victim of Clippers' assist self-deprivation would be none other than Pooh Richardson.
_________________
`
36% of all statistics are wrong
Back to top
View user's profile Send private message Send e-mail
IrishHand
Joined: 15 Jul 2009
Posts: 115
PostPosted: Fri Jul 17, 2009 10:33 am Post subject: Reply with quote
Well played, sir.
Back to top
View user's profile Send private message
Mike G
Joined: 14 Jan 2005
Posts: 3584
Location: Hendersonville, NC
PostPosted: Wed Jul 22, 2009 8:28 am Post subject: Reply with quote
Irish, if you're still around:
Though you were (I think) never counting stats in a playoff game, do you have any feel for whether assists are granted as liberally, or if there are changes in the ways they may be biased?
I ask this because in recent postseasons, there's been a pronounced dropoff in Ast/FG. This year, 16 playoff teams averaged .569 in the season and .540 in playoffs. It's been similar for a few years now, at least.
I extend the inquiry to anyone who has a theory. Non-playoff teams were .561 on the year, so it's not as simple as teams being 'worse' in playoffs.
ps: IrishHand has now made both the shortest post and the longest post in the history of this forum. (not counting cut'n-paste posts)
_________________
`
36% of all statistics are wrong
Back to top
View user's profile Send private message Send e-mail
IrishHand
Joined: 15 Jul 2009
Posts: 115
PostPosted: Wed Jul 22, 2009 10:20 am Post subject: Reply with quote
Correct, I was never involved in doing stats for a playoff game.
My conjectural answer would be that stat crews are both aware of the subjective impact they have on stats and the increased scrutiny that playoff stats are likely to receive. It's rationale to assume they would decrease the latter as a consequence of the former.
Another possible explanation: to the extent it is the case that the pace of play slows down in the playoffs, it would seem reasonable that the AST% would decline with an associated increase in individual shot creation.
Back to top
View user's profile Send private message
HoopStudies
Joined: 30 Dec 2004
Posts: 705
Location: Near Philadelphia, PA
PostPosted: Wed Jul 22, 2009 10:46 am Post subject: Reply with quote
I just watched the NBDL Select team vs the Nuggets in Summer League. On TV, the game was 96-89 Nuggets. In the boxscore, it was 94-90. Somehow, they got a point and we lost 2 by the stat keepers. Very strange.
I've seen occasional disasters of scoring in NBA play-by-play, too, a Sonic-Bulls game back 5 or so years being a prime example. Having spent years with environmental data that is really really noisy, these kinds of stories of errors and biases don't worry me much. You always have to deal with weird data. In talking to people doing analytical work in other fields where data is not as plentiful and biases also exist (like health care), I've learned that they have bigger problems than we do.
_________________
Dean Oliver
Author, Basketball on Paper
The postings are my own & don't necess represent positions, strategies or opinions of employers.
Back to top
View user's profile Send private message Visit poster's website
Jose A. Martínez
Joined: 19 Jul 2009
Posts: 72
PostPosted: Wed Jul 22, 2009 11:47 am Post subject: Reply with quote
Very interesting topic.
The company Galanis Sports Data, provides statistical services to the Greek league (one of the most important in Europe). We can say that this company plays the same role that STATS to the NBA.
In its website, they point the following:
http://www.galanissportsdata.com/mainsite/tendex.htm
"Finally, we need to underline that we have to be very careful, when we combine stats. Are these stats correct? Have these stats been collected according to the rules described in all previous paragraphs? We have been examining all stats rules and we included them in our web site under: www.galanissportsdata.com/mainsite/basketr.htm
In order for someone to get an idea of how easy it is to err, we only need to mention that each game involves 500 entries per team. If we allow a reasonable 3% error we can result in a total of 15(!) tendex points difference per team, which can lead to reverse conclusions"
Under my view, we can assume that there is an amount of bias in the box-scores. This bias has to be considered as random, and the statistical treatment has to be similar to the role played by measurement error in computing effect sizes, i.e., measurement error increase observed variance but does not bias means.
I many of you have noted, systematics errors could also arise for specific players. However, this is very problematic to treat, and should be numerically quantified for independent analysts, in order to handle this systematic bias using some factor of correction.
In addition, there is some categories, as asssits, that are more sensbile to errors than others, e.g. points. This is obvious, and should be taken into account.
_________________
Jose A. Martínez
http://www.upct.es/~beside/jose
http://basket-research.blogspot.com
Back to top
View user's profile Send private message Send e-mail Visit poster's website
semifiction
Joined: 22 Jul 2009
Posts: 7
PostPosted: Tue Aug 04, 2009 11:53 pm Post subject: Reply with quote
interesting are there any third parties that keep their own stats?
combine the stat keeping error and the referee error...
Back to top
View user's profile Send private message
Jose A. Martínez
Joined: 19 Jul 2009
Posts: 72
PostPosted: Thu Aug 06, 2009 4:40 am Post subject: Reply with quote
Actually, I really think that there is only one source of random error: the observer (the referee).
Then the classical test equation says:
Observable variance = Real variance + Error variance
Then observable variance is always above real variance when error is distinct from zero.
Is there always error variance? Yes, social scientist know that error always occurs, even when we code variables such as sex or age, it is reasonable to assume error. Professor Leslie Hayduk, from University of Alberta, has published several books on Structural Equation Modelling, where he speaks about this topic.
The issue is to analyse the importance of error variance for each statistic (assits, points, rebounds, etc.). I suspect that error variance is negligible for all the statistics categories but assists.
How we could know about error variance? This can be experimentally studied by researchers, through observation of several games, and comparing their results with official statistics. The other option would be to assume a certain amount of error variance, and then to achieve sensitivity analysis in order to analyse the robustness of results when this assumption is modified.
_________________
Jose A. Martínez
http://www.upct.es/~beside/jose
http://basket-research.blogspot.com
Back to top
View user's profile Send private message Send e-mail Visit poster's website
rlee
Joined: 20 Aug 2005
Posts: 61
PostPosted: Tue Aug 18, 2009 3:42 am Post subject: Reasonable explanation for Skiles assist record Reply with quote
Not necessarily stat-keeper trickery:
http://tinyurl.com/oqhkqz
Back to top
View user's profile Send private message Send e-mail
IrishHand
Joined: 15 Jul 2009
Posts: 115
PostPosted: Tue Aug 18, 2009 7:41 am Post subject: Reply with quote
Van Exel also not necessarily stat-keeper trickery:
http://articles.latimes.com/1997-01-06/ ... 1_van-exel
Back to top
View user's profile Send private message
Ryan J. Parker
Joined: 23 Mar 2007
Posts: 711
Location: Raleigh, NC
PostPosted: Tue Aug 18, 2009 9:57 am Post subject: Re: Reasonable explanation for Skiles assist record Reply with quote
rlee wrote:
Not necessarily stat-keeper trickery:
http://tinyurl.com/oqhkqz
Unfortunately this is the path of discussion currently going on in this forum: because we know the way stat-keepers track stats varies, we can simply attribute all home vs away differences to this. Why do any hard work when we can just say this?? Crying or Very sad
_________________
I am a basketball geek.
page 3 of 3
Author Message
Mike G
Joined: 14 Jan 2005
Posts: 3597
Location: Hendersonville, NC
PostPosted: Tue Aug 18, 2009 10:38 am Post subject: Reply with quote
Ryan J. Parker wrote:
Unfortunately this is the path of discussion currently going on in this forum: because we know the way stat-keepers track stats varies, we can simply attribute all home vs away differences to this. Why do any hard work when we can just say this??
Ryan, I am pretty sure no one has suggested that all home-away variation is the result of scorekeeper trickery. Rather, that the observed variations, team-to-team and year-to-year, indicate that something other than your normal/expected ref and player home/away tendencies are occurring. These extremes are easily seen, are well beyond the bounds of league norms, and are immediately (estimably) quantifiable.
Rather than take an extreme stance that --
a) since these anomalies exist, we should ignore all subjective stats;
or
b) since the work involved (to pinpoint the subjective element) is astronomical, we will never be able to keep abreast of it;
-- we could also suppose that there are reasonable estimates, ensuing from bulk home/away data, of adjustments one could make to player Blk%, Ast%, etc. Separating ref bias from scorekeeper bias might be interesting to someone, but it's not necessary for evaluating players.
_________________
`
36% of all statistics are wrong
Back to top
View user's profile Send private message Send e-mail
Ryan J. Parker
Joined: 23 Mar 2007
Posts: 711
Location: Raleigh, NC
PostPosted: Tue Aug 18, 2009 10:59 am Post subject: Reply with quote
This really comes for my general distaste for box score stats. They are perhaps a starting point, yes, but by no means are they exhaustive. The numbers I posted in the other thread were merely the tip of the iceberg relating to the differences with respect to Nuggets last year.
The issue I have with the discussion so far is that there are players and real stuff that happens on the court that affects these numbers that no one is bothering to look at. On the scale of stuff that's important, this is near the bottom, but the sort of discussion on this topic bothers me.
Now if these sort of arguments were being made over at the WoW Journal then perhaps we'd say that's par for the course. Cool
_________________
I am a basketball geek.
Back to top
View user's profile Send private message Visit poster's website
IrishHand
Joined: 15 Jul 2009
Posts: 115
PostPosted: Tue Aug 18, 2009 11:21 am Post subject: Reply with quote
Perhaps I'm operating under a false assumption, but isn't a forum like this specifically for things like determining systematic errors in the data underlying our analysis, accounting for it, and then drawing more accurate conclusions from that data?
I'm mystefied why some would relegate this topic to "whiny 10-year old" status.
Back to top
View user's profile Send private message
Ryan J. Parker
Joined: 23 Mar 2007
Posts: 711
Location: Raleigh, NC
PostPosted: Tue Aug 18, 2009 11:26 am Post subject: Reply with quote
We can simply agree to disagree on how the analysis is being done. I'm fine with that, and I'll respectfully move on.
_________________
I am a basketball geek.
Back to top
View user's profile Send private message Visit poster's website
jim
Joined: 01 Aug 2009
Posts: 13
PostPosted: Thu Aug 27, 2009 12:18 pm Post subject: Reply with quote
By the way, I don't know if anyone has noticed, but this thread has been getting a decent amount of internet exposure i.e. Deadspin, TrueHoop, and Ball Don't Lie. Is Sportscenter next?
Back to top
View user's profile Send private message
DJE09
Joined: 05 May 2009
Posts: 148
PostPosted: Thu Aug 27, 2009 6:07 pm Post subject: Reply with quote
jim wrote:
By the way, I don't know if anyone has noticed, but this thread has been getting a decent amount of internet exposure i.e. Deadspin, TrueHoop, and Ball Don't Lie. Is Sportscenter next?
Jim, the most ironic this is how Truehoop cite's deadspin ... no desire to go to original material - or "gasp" seek interview with source - or did Henry do that Irish?
Back to top
View user's profile Send private message
IrishHand
Joined: 15 Jul 2009
Posts: 115
PostPosted: Thu Aug 27, 2009 6:57 pm Post subject: Reply with quote
No, the Truehoop post yesterday had zero input from me.
To be fair, a significant portion of the Truehoop posts are Hoopshype-esque (a series of links to other sources). I guess he thought it was an article (the Deadspin one) worthy of commentary and in today's now-now-now blogging environment, acted on that. I think he posted his version less than an hour after the Deadspin one went up.
Also to be fair, he and I spoke briefly today...so he was doing his homework.
Back to top
View user's profile Send private message
Mike G
Joined: 14 Jan 2005
Posts: 3597
Location: Hendersonville, NC
PostPosted: Mon Aug 31, 2009 7:50 am Post subject: Reply with quote
From early 2007 we investigated some of this stuff.
http://sonicscentral.com/apbrmetrics/vi ... php?t=1169
Irish (especially), how about the Hollinger comment at bottom of 1st page?
_________________
`
36% of all statistics are wrong
Back to top
View user's profile Send private message Send e-mail
IrishHand
Joined: 15 Jul 2009
Posts: 115
PostPosted: Mon Aug 31, 2009 11:33 am Post subject: Reply with quote
I was gone by the time Bibby got to Vancouver - so no personal knowledge of the timeframe relating to Holliger's claims. Interestingly, your data (from the HomeAwayAst% file) suggests that the Vancouver stat crew in the 98-01 timeframe (when Bibby was in VAN) wasn't that bad - certainly not "handing out assists like candy" as Hollinger claims. They were 17th, 16th then 17th in the NBA in terms of home assist bias (or whatever you want to call it) - so actually better than average overall (though in all seasons, not that close to being unbiased).
The Bibby data is interesting, and complies with my general theory about home and away inflation. By year, home APG, away APG for PG Mike Bibby:
99: 7.4, 5.6
00: 8.6, 7.6
01: 9.0, 7.7
The data suggests to me that Vancouver significantly inflated Bibby's assists in his rookie year (he was the supposed 2nd star and "needed" to look good) while away teams were largely indifferent to him - just another draft pick by the garbage Grizzlies. After his very good rookie year, league-wide perceptions of him changed/improved - he looked like he was a good, legitimate NBA PG. Did he improve from rookie to soph season? No doubt. Did he improve by 1 APG at home and 2 APG on the road? Perhaps. I'd offer that road scorekeepers began giving him the benefit of the doubt, perhaps 1 APG. They were already giving him max benefit of the doubt at home. My hypothesis would be that his rookie road APG is fairly legitimate (to the extent that you think the way assists are awarded in the NBA as a general matter is legitimate). He then improved ~1 APG from rookie to soph season and benefited from a better "reputation", thus now getting some grey area calls on the road. (Team-wide, it appears that while Vancouver wasn't handing out assists like candy, they were to Bibby and presumably not to anyone else - being stingy w/ the rest of the team to balance out his inflation.)
Now Bibby gets traded to Sacramento for White Chocolate. Sacramento, during his first three seasons there, are one of the best assists scorers in the league - best meaning unbiased. They rank 21, 27, 24 in those three seasons, with an aggregate home assist bias of around -1.5% - meaning Kings got slightly more assists on the road than at home. How do Bibby's APG look now that he's free of his VAN stat crew love?
02: 5.1, 5.0
03: 4.7, 5.7
04: 5.2, 5.5
The overall decline is predictable - he went from being the primary/lone ballhandler on a very poor VAN team to essentially being a combo G on a SAC team that moved the ball exceptionally well w/ Bibby/Christie/Webber/Divac (Peja being the lone starter who wasn't/isn't a good passer). He's no longer the star, he's got the ball less, so less opportunities for home or away crews to give subjective assists and his home team is no longer padding his numbers. His APG numbers those 3 years in Sacramento are almost certainly legitimate.
To be honest, I've been amazed at how much attention this has gotten - I always assumed that any serious sports fan who cares about numbers is aware of the subjectivity impacting them, especially in basketball.
Back to top
View user's profile Send private message
rlee
Joined: 20 Aug 2005
Posts: 61
PostPosted: Mon Aug 31, 2009 12:17 pm Post subject: Reply with quote
I was a Kings season ticket holder in that era & once saw an assist (if memory serves it was given to Vlade) after FOUR dribbles by the recipient of the pass.
Back to top
View user's profile Send private message Send e-mail
IrishHand
Joined: 15 Jul 2009
Posts: 115
PostPosted: Mon Aug 31, 2009 1:03 pm Post subject: Reply with quote
Which speaks, imo, to the reality that there are only so many suspect assists you can give out in a game. You might hypothesize that the Sacramento scorers enhanced their assist numbers for multiple players and that the Sacramento "great passing, great offense" reputation enabled/encouraged road scorers to do the same - meaning that the lack of a home Sacto bias was really more of a league-wide Sacto assist inflation.
I suspect the truth is somewhere in the middle...though we'd need to know who the scorers were and what time periods they worked. (Meaning did/does Sacto inflate their assists when their teams don't have four excellent passers at their positions?)
Back to top
View user's profile Send private message
Mike G
Joined: 14 Jan 2005
Posts: 3597
Location: Hendersonville, NC
PostPosted: Mon Aug 31, 2009 3:28 pm Post subject: Reply with quote
If the rock-bottom assist H/A rates gotten in ClipperLand in the late-'90s are a baseline (recently approached in Miami), then a round estimate is that some 20% of all assists are 'iffy' or worse. On top of that, most teams pad their own home assists by another 5-10%.
So if the Heat get 15% fewer assists at home, then perhaps they are padding their own players' assists by 'only' 5%. Another team may give 25% padding to opponents and 35% to their own.
Unheralded players may still get very few 'extra' assists; but then certain stars apparently get even more than the 25-30% norm. It may be that the only way to rectify this is to just shave off everything beyond each player's away rates, from their home rates. It's much easier to apply the same team H/A to each member of the team; but then you are victimizing some, while not taxing others enough.
_________________
`
36% of all statistics are wrong
Back to top
View user's profile Send private message Send e-mail
IrishHand
Joined: 15 Jul 2009
Posts: 115
PostPosted: Mon Aug 31, 2009 3:54 pm Post subject: Reply with quote
Probably correct conclusions. When I first watched that YouTube clip of the five Chris Paul assists, my initial (scorer) conclusion was 4 easy assist calls, 1 bad assist call. When I stopped to think about it and what an assists is actually supposed to be, I evolved to 3 bad assist calls and 2 atrocious ones.
An estimate of 20% generic inflation plus 5-10% some-home-teams inflation is probably reasonable. Gets back to the lack of standardization, guidance and oversight imo.
Back to top
View user's profile Send private message
jim
Joined: 01 Aug 2009
Posts: 13
PostPosted: Tue Sep 01, 2009 1:04 am Post subject: Reply with quote
Link
Yeh, those "assists" are pretty bad.
Back to top
View user's profile Send private message
Mike G
Joined: 14 Jan 2005
Posts: 3597
Location: Hendersonville, NC
PostPosted: Tue Sep 01, 2009 4:11 am Post subject: Reply with quote
Even more ancient (2003) ruminations, by several of the usual suspects -
http://sports.groups.yahoo.com/group/AP ... ssage/2608
_________________
`
36% of all statistics are wrong