Lineup testing
Lineup testing
In last playoffs the final four teams used 5 lineups for about half their minutes. Houston a bit more, GSW close to half, Cleveland and Atlanta a bit less. With about 4,000 regular season minutes available for testing, how many of these most vital playoff lineups got 200 minutes or more of testing? No more than 30%, perhaps a bit less. Would only take one quarter of the available regular season time to do so but, no, the best teams did this level of testing on only a small share of these lineups. Seems like a big mistake to me.
Re: Lineup testing
> ... Would only take one quarter of the available regular season time to do so but, no, the best teams did this level of testing on only a small share of these lineups. Seems like a big mistake to me.
A quarter is around 22 possession pairs. Is that really a statistically significant sample? If we say the variance is 1 point per possession, then that's a variance of 44 points over the sample - so even the 80% confidence interval is 8.5 points over those 44 possessions. That's about the same as the range between the best and worst teams in the NBA. And, even with that, there are contextual factors that need to be controlled for.
And, supposing that it's about some 'intangible observation' that the coach makes, what makes us think that the coach can't get similar information from scrimmage during practice?
A quarter is around 22 possession pairs. Is that really a statistically significant sample? If we say the variance is 1 point per possession, then that's a variance of 44 points over the sample - so even the 80% confidence interval is 8.5 points over those 44 possessions. That's about the same as the range between the best and worst teams in the NBA. And, even with that, there are contextual factors that need to be controlled for.
And, supposing that it's about some 'intangible observation' that the coach makes, what makes us think that the coach can't get similar information from scrimmage during practice?
Re: Lineup testing
5 samples of a minimum of 200 minutes each (equals 1000 minutes, equals one quarter of time available for SEASON) are more meaningful (more statistical significant. and though that is a very difficult standard to get anywhere close to, it is better to be closer, as close as you have the realistic option of getting.) than relying on lineups in playoffs that played 30-70 minutes or not much more in regular season. But yeah I'd push for more, more 500 minute samples. Doable but not with the way almost every NBA coach sprays dink lineups around, wasting testing time.
No, I don't think Coaches can get the same information from practice. Not close.
If coaches are still to be relied on to use their eyes and instincts to make the decisions, I'd far rather they do that from more NBA minutes than less or substituting practice. They got practice to consider if they want either way... but, oh, I am told no team has much time to really practice after training camp. A few minutes against reserves where everyone knows what you are doing, away from NBA refs, game pressure is not close to the real thing.
If it is so valuable, do it more. But they don't, won't. Do they keep high quality, detailed stats of practice? Are the coaches reviewing them carefully? Maybe some are, but I doubt it is common, intense or given much credence. But sure, do that too.
And use the pre-season seriously. And do more comparison and clustering of lineup data. And run simulators. And then really use the data, analysis and plan from it to guide actual allotment of time instead of just using the heat of the moment, tactical but not necessarily so wise long-term strategic decision-making we see.
If a Brad Stevens can learn so much from practice or coaching intuition, why does he need to use 850 lineups? They are not all micro-optimal. I recall that last season they were negative almost 70% of the time. Celtics lineups tested over 200 minutes last season? One. And it was awful. And then was most used in playoffs, where it was even a bit worse. (Did it work in practice? Was that part of the decision-making? There was no real NBA game scoreboard results basis for using it, continuing to make it most used. But I didn't see their touted lineup reports. Were there micro details that override the message in the scoreboard data?)
Maybe key lineup testing over 200 minutes isn't for everybody. But it would be my preference, priority. I'll take that over the less credible alternative.
No, I don't think Coaches can get the same information from practice. Not close.
If coaches are still to be relied on to use their eyes and instincts to make the decisions, I'd far rather they do that from more NBA minutes than less or substituting practice. They got practice to consider if they want either way... but, oh, I am told no team has much time to really practice after training camp. A few minutes against reserves where everyone knows what you are doing, away from NBA refs, game pressure is not close to the real thing.
If it is so valuable, do it more. But they don't, won't. Do they keep high quality, detailed stats of practice? Are the coaches reviewing them carefully? Maybe some are, but I doubt it is common, intense or given much credence. But sure, do that too.
And use the pre-season seriously. And do more comparison and clustering of lineup data. And run simulators. And then really use the data, analysis and plan from it to guide actual allotment of time instead of just using the heat of the moment, tactical but not necessarily so wise long-term strategic decision-making we see.
If a Brad Stevens can learn so much from practice or coaching intuition, why does he need to use 850 lineups? They are not all micro-optimal. I recall that last season they were negative almost 70% of the time. Celtics lineups tested over 200 minutes last season? One. And it was awful. And then was most used in playoffs, where it was even a bit worse. (Did it work in practice? Was that part of the decision-making? There was no real NBA game scoreboard results basis for using it, continuing to make it most used. But I didn't see their touted lineup reports. Were there micro details that override the message in the scoreboard data?)
Maybe key lineup testing over 200 minutes isn't for everybody. But it would be my preference, priority. I'll take that over the less credible alternative.
Re: Lineup testing
Yeah, I misread that as "a quarter during the regular season". 200 minute samples are certainly much more representative.
It's not always up to the coaches though. I don't know the Cav's line-ups were, but I can't really blame David Blatt for Kevin Love and Kyrie Irving getting injured. It seems like it might be worthwhile to look at what the 'experimental' play off line ups are and see if there's some kind of consistent coaching philosophy behind them.
There's also a fundamental question about how much "line-ups" matter compared to individuals and the opposition. Is playing against Lebron James and the Cavs really "roughly the same" as playing against the Spurs, or the Bobcats, and how well can we predict line up performance by synthesizing data from other lineups?
NBA teams are using wearables in practice, so there's certainly some data collection going on. Of course the really fundamental issue is: What good will the data do? If the coach doesn't want to take an analytic approach to line-ups, then there's not a whole lot of value in collecting line-up data.
It's not always up to the coaches though. I don't know the Cav's line-ups were, but I can't really blame David Blatt for Kevin Love and Kyrie Irving getting injured. It seems like it might be worthwhile to look at what the 'experimental' play off line ups are and see if there's some kind of consistent coaching philosophy behind them.
There's also a fundamental question about how much "line-ups" matter compared to individuals and the opposition. Is playing against Lebron James and the Cavs really "roughly the same" as playing against the Spurs, or the Bobcats, and how well can we predict line up performance by synthesizing data from other lineups?
NBA teams are using wearables in practice, so there's certainly some data collection going on. Of course the really fundamental issue is: What good will the data do? If the coach doesn't want to take an analytic approach to line-ups, then there's not a whole lot of value in collecting line-up data.
Re: Lineup testing
"There's also a fundamental question about how much "line-ups" matter compared to individuals and the opposition."
I agree it is a question that still needs a lot more work and discussion. Very little in the public realm. Have teams done good work that is locked away? I hope so. But I need to see evidence of this good work in the public lineup data. There are cases where it is not there, by my review of it.
Could the secret lesson of lineups be that you have to prospect and drill bad holes until you find a compensating gold mine? Maybe. That is not that my preconceived concept. But it appears to be what some coaches are doing, knowingly or not.
I agree it is a question that still needs a lot more work and discussion. Very little in the public realm. Have teams done good work that is locked away? I hope so. But I need to see evidence of this good work in the public lineup data. There are cases where it is not there, by my review of it.
Could the secret lesson of lineups be that you have to prospect and drill bad holes until you find a compensating gold mine? Maybe. That is not that my preconceived concept. But it appears to be what some coaches are doing, knowingly or not.
Re: Lineup testing
http://nyloncalculus.com/2015/10/15/exp ... s-and-fit/
Further attempts to find lineup clusters that differentiate and educate could be valuable.
David Sparks work years ago clustered players according to basic box score stats and then lineups by the string of player types. There have been other player typologies since. There could be still further ones based on SportsVu data.
There could even by player typologies based on the pattern of metric evaluations. Even if you stop short of giving metrics a claim to accuracy you could potentially find meaning in the clustering of metric patterns for players and the way lineup results vary across the different combinations of those metric estimate sets. Are lineup results better for PGs relatively higher on RPM than BPM, WS/48, PER, etc.? What type of SGs or Centers do each work best with? Overall and down to the factor level.
Further attempts to find lineup clusters that differentiate and educate could be valuable.
David Sparks work years ago clustered players according to basic box score stats and then lineups by the string of player types. There have been other player typologies since. There could be still further ones based on SportsVu data.
There could even by player typologies based on the pattern of metric evaluations. Even if you stop short of giving metrics a claim to accuracy you could potentially find meaning in the clustering of metric patterns for players and the way lineup results vary across the different combinations of those metric estimate sets. Are lineup results better for PGs relatively higher on RPM than BPM, WS/48, PER, etc.? What type of SGs or Centers do each work best with? Overall and down to the factor level.
Re: Lineup testing
There's a variation of this I have been working on recently - I was hoping to have a working model by the start of the season.Crow wrote:
There could even by player typologies based on the pattern of metric evaluations. Even if you stop short of giving metrics a claim to accuracy you could potentially find meaning in the clustering of metric patterns for players and the way lineup results vary across the different combinations of those metric estimate sets. Are lineup results better for PGs relatively higher on RPM than BPM, WS/48, PER, etc.? What type of SGs or Centers do each work best with? Overall and down to the factor level.
Re: Lineup testing
Where do ya'll get lineup data from? I've tried to extract it from play by plays, but the data is a little fuzzy because the play by plays don't show who was on the court after each quarter break
Re: Lineup testing
For lineup summary stats I typically use nbawowy.com.
NBA.com may be another option. Vantage?
Same places for source data.
NBA.com may be another option. Vantage?
Same places for source data.