Constructive discussion re: RAPM

[v-zero]

.

[TeemoTeejay]

I might weigh in at some point, but I'm very busy right now. Reading the recent replies I'm afraid this might be a case of somebody being impossible to educate.

I already beat Vegas. I see no need whatsoever to predict these rankings or explain the variance therein. I have no idea how anybody could genuinely think they have a perfect ranking of NBA players? ____, I don't even know what that would mean? Is it impact? Looks like not since there's some upset about 'lineup effects', but basketball isn't tennis, you cannot abstract the player away from the court, the game, the teammates, the opponent. So what, we're upset because Rudy Gobert is limited as an all round player but generally impactful because he provides rare and attentive paint presence without being a lumbering sack of ____ when he has to help on the perimeter?

Yeah, I don't see how I can help this conversation.

The weirdest thing is there is some like semblance of potential interesting ideas here but he came in here with such a weird “I’m a prophet” energy and is accusing people of low level thinking when he’s clearly just not really particularly even remotely knowledgeable with regards to this

And that’s kind of my issue, there’s an interesting idea here and I don’t completely disagree with the spirit of some of his points, but then to come hot with such a “I know better!” Attitude and somehow be unable to grasp the flaws of raw +/- on a literal single game sample is the most absurd thing I’ve ever seen

Like the main thing for me is he’s trying to do an apples to oranges comparison, he’s complaining about the strange rankings in some AIO metrics viewing them as a ranking of absolute goodness, but that isn’t what they are meant to be, it’s likely that the box score components of those metrics outdo the metrics themselves if ur just talking about conforming closer to general opinions of rankings of players

There’s truth to the idea that, let’s say if they’re talking about a prior, that there are more considerations than just predicting RAPM, but much of that is simply trying to add a more context independent element to things, imo.

I don’t think the idea of, let’s say a training on something that conforms much more to how players are perceived, adjusted for some players that may be fairly overrated and underrated based on gaudy but ineffecient numbers or something, is like a dumb one, but it’s this weird obsession with raw plus minus that’s taking me out lol

Ur def more versed in this stuff than I am, but that’s my general thoughts there I guess, rapm as not being the ideal type of target I think is a fair idea, it’s just the idea that the solution is to just replace the entirety of that family of metrics with a linear model that just incorporates raw plus minus and training it on a online ranking being and being comedically confescending about it sounds absurd


A lot of it depends on the goal I think too, establishing value in a Vegas type of context of what’s going to happen going forward is a tad different from doing so within the context of capturing player value or something like that through trying to get some sort of measure of their impact, although even then isolating them from their situation like u said is not possible to do, at least 100% of the way

[sportsandmath1]

Checking back in on this thread,

ultimately we have to recognize that there is a divergence

between the two methodologies

- % of production* per game
- predicting RAPM and/or +/- stint data

By now, it should be very clear what my views are:
I'm obviously a fan of the first approach for a multitude of reason but mainly since it stabilizes much faster and clearly more aligned with NBA "expert" opinions.

Case Study: Luka vs Tatum - https://x.com/sportsandmath1/status/190 ... 53919?s=46

SportsandMath1 Score% says Luka > Tatum his entire career

DARKO DPM says Tatum > Luka his entire career

Which side are you on?

</ begin yapping>

If you actually think the inverse of a matrix tells you more about basketball than a casual fan let alone "experts", you might be too far down the rabbit hole and I can't help you.

and the second approach is plagued by the inherent noise/lineup effects/team context unrelated to player rankings, greatly reducing the effectiveness and insightfulness
- (no one can predict more than 70% — 50% for a single year — of the variance in long term RAPM, so be prepared for the other at minimum 30% to be complete noise)
- Predicting stint data is even more obviously fitting to noise (I like this article on intraoc https://www.intraocular.net/posts - Why APM models should account for position Posted on 3/25/2022), showing the RMSE is 1.18 no matter what approach you take. That's intractable) this leads to a false consensus towards RAPM which my views are clear on (perhaps a tool, should not be worshipped as ground truth due to a multitude of issues)

https://imgur.com/a/69ranSE

-

It's just the answer to an unnecessarily complex optimization problem (which is unsolvable due to the noise).

*where production is defined as a linear combination of additive statistics that you can define in whatever manner you deem most useful, but in practice I've used

1.2 PTS - 0.96 FGA - 0.39 FTA + 0.09 3PA + 1.2 STL + 1.5 BLK + 0.3 REB + 0.6 AST - 0.6 TOV + 0.34 * (+/-) + 0.09 MIN

for it's simplicity and effectiveness (predicts over 90% of the variance in player rankings)

</end of yapping>

I know I get repetitive, but by now it's clear what my views are, it's up to y'all to make use of these insights.

[Crow]

Re: https://www.intraocular.net/posts

The boxscore prior for bigs for defense varies more than the pure APM because one uses individual data and the other uses team data where the big is probably not primary on 70% of defensive play end points.

Making boxscore priors relative to position is a subjective intervention that contradicts the boxscore stats plainly stated.

If the boxscore stats show a big difference, there is a big difference.

Average impacts are not equal across positions, but let's make them that way.
Rate players relative to position if you want to. That is different that rating all players equally.

Position adjusted ratinngs may well predict team performance more closely as teams still generally adhere or mostly adher to traditional positions across most lineups and on average.

Position adjusted Baynesian Box Adjusted Plus Minus has more in common with BPM than non-positionally adjusted metrics.

Compare position adjusted ratings to other position adjusted ratings and neutral / equal non-positionally adjusted ratings to similar ratings. And know your purpose.

A player is the player and their role. Theoretically you can divide those pieces. But the player you see, is the whole player as used.

I have called for RAPM by position in past. It will never be perfect but the splits by estimated position would be worth seeing and considering. By play by play estimates based on size and typical use or actual video proof of matchup at time of play end. I assume Baynesian Box Adjusted Plus Minus was using position from roster listings with match issues to actual play by play use since no proof of anything more discrete was indicated. Such assumptions would be distorting and not recognize it.

Everything out there has flaws. So study everything, know differences, know flaws, use some combination or blend. And don't expect or pretend to have perfection. Be as informed as possible using stats & metrics and try to make that work better than less informed. Use less than all if you want. Ultimately judge the use of information array / results of that use more than the methods and datapoints themselves. Applied analytics over "the analytics". The analytics are a base but I spend most of my time trying to apply them, knowing they are imperfect but thinking I can think better with them (at least on average). Draft pick evaluation. Other player acquisitions & disposals. Lineup, minute and shot distribution decisions. Matchup decisions. Applied analytics.

I don't "know" how hard authors of analytics use their own products (including producers of simple analytic graphs & lists) and those of others privately but there is often not much public paper trail evidence of extensive application. Value comes in application. Extensive application. Novel or speculative application. Layered application. Application with a level of error.


It is unclear to me how much time Coaches and top front office people spend in direct exposure with "analytics" or applied analytics reports and recommendations and how much weight it is given. It is happening and more than in past but how much? Are most decisions substantially based on analytics and "advanced" analytics or only sometimes?
Draft, free agency, trades have a mixed appearance of use and weight. Lineup management ahows generally show low signs of applied analytics. Shot selection and distribution certainly could use more.

What % of total "analytics staff" time is spent on "analytics" from conception to production, maintenance and dissemination versus "applied analytics" and presentation & advocacy? Don't hear much about time allocation, thru put, results. Occasional cases but usually brief mentions, not details of what was considered, dismissed / ignored, the level of opportunity to make a case and fight for it.

I don't know the level of restriction and legal threat involved in non-disclosure agreements; but relative speaking, little of the reality of team decision-making in the analytics era has been exposed. I guess a new book and some past books could be reviewed closer. Not much comes out at Sloan or in articles / podcasts / tweets. Very little if anything has ever been posted here. The trickle is far from "enough" for accurate understanding of the internal climate & typical operation.

Recommend a lineup's initial use, greater use, lesser use how often and to what reaction if any? What level of use of analytics based staff recommendations "should" be considered, debated, implemented, reviewed? Only as much as Coach wants / allows or the amount that GM says? Or the anount the Analytics Director recommends, figgts for, or signed on for? What are the standards for analytics and applic analytics use in every area? What is the experience?

You hear of a few lineups recommended / used, but what is the overall openess to using input? I heard a recent case of a key staffer recommending a lineup, that was used and to positive effect. Then I checked the data and found that total use for season remained very minor.

[TeemoTeejay]

--

But you're attempting to do a player ranking rather than find actionable insights right?

I do think that there are better ways to rank players and isolate them from certain variables that are either not necessarily their fault or out of control than all in one impact metrics, but that isnt what is being tried to do much of the time. There were reasons that Luka's offensive impact for example was a tad lower than one might expect that weren't neccesarily on him and were pretty much fixed and his numbers immediately shot up for example.

apples and oranges

[DSMok1]

“between the two methodologies

- % of production* per game
- predicting RAPM and/or +/- stint data“


This is an important distinction.

Percent of production is a pretty self-evident approach, but it doesn't correlate directly to point differential on the court and thus wins (the ultimate goal).

The only way to directly measure point differential impact is based on the plus minus data, but it is extremely noisy and has collinearity confounding the regression analysis. You need at least 3 years of data to have any confidence the results are in the ballpark. Also really need to have a good non-boxscore prior if you want an unbiased evaluation.

My goal with BPM was always to use long-term, prior informed RAPM to establish the appropriate values for individual box score statistics.

For instance, a defensive rebound. We can say that a defensive rebound is worth X to the team, over the other team having another play. But how much of that X value should be credited to that individual player that got the rebound? If a different player was on the court, would the team still get the rebound and thus that X value? That's what we're trying to evaluate.

[Crow]

Fwiw, I looked at 5 top teams and most used lineups with same 4 and 1 player difference. I compared change in lineup performances to change in player ratings on BPM, Darko and Mamba between the changed players to see if they at least predicted the direction of change.

BPM did 55% of the time (in 9 cases), Darko and MAMBA did 33% of time.

Exact level of change is likely too much to ask, but BPM did well on 3.

Is BPM's explicit position based adjustment helpful for this purpose, on average? Probably.

Player ratings in general do not look partucularly good at predicting change in lineup performance. At least in this set, because of volatility associated with modest sample size but also perhaps because lineups depend on role fulfillment beyond average net impact.

Obviously 30 or 100+ comparisons would be somewhat better than 9, but that is what I did at the moment. Others are welcome to do more.

What would predict such changes better?

I'll check a pure RAPM, 3 year. Predicted 45% of direction of changed results.

Overall average raw +/- on the court? Ties for best prediction of result change. +/- on / off? Ties for worst performance. In this case the "off" apparently isn't helpful / actually hurts.

A blend of BPM and raw +/- on might be promising... for this purpose... and conceptually similar to sports and math's approach. 2 blocks, 1 not bending the other.

Of course other blends are possible: BPM and pure RAPM, 3+ metrics combined, etc.

VPM blend are NBARAPM predicted 45% of changes. Between the extremes.



I am not sure 3 year RAPM is "best".l version. Plenty of year to year volatility. I might want to look at a weighted blend of 1, 2 and 3 year. Not sure what the weights would be. Anybody experiment with this? 25-35-40%? 40-30-30%? 33.3-33.3-33.4%?
.

[Crow]

Predicting lineup results?

The best predictor might be previous minutes played after some level of minutes.

Or check average of sum of average results of contained pairs?