APBRmetrics

Sorry if this is asked too frequently or too difficult for an accurate answer...

Is there any consensus regarding which metrics are the most accurate in judging a player's value? Have there been any meta-analyses for analyzing metrics' accuracy?

In other words, what's the current "best" metric or blend for judging individual players overall?

For a solo boxscore metric BPM tested better than older, widely known boxscore metrics. DRE is new. I am not sure exactly how well it tested but I think it is at least decent. RPM and other RAPM test well. Look around here and check the sports skeptic tests in the short list of topics worth reading thread.

I haven't tested my 1.0 blended metric or sought to find the optimal mix via machine learning. If someone is in the mood to test DRE, this blend and other lower profile metrics that would be appreciated.

Andrew Johnson's player tracking plus minus did very well at predicting performance this past season. I'd put that in the mix if I has the final values handy.

There isn't a clear, widely recognized best. But looking at several and adding input from eyes, SportsVu, other discrete data and judgment from experience, one should be off to a decent start.

Crow wrote:For a solo boxscore metric BPM tested better than older, widely known boxscore metrics. DRE is new. I am not sure exactly how well it tested but I think it is at least decent. RPM and other RAPM test well. Look around here and check the sports skeptic tests in the short list of topics worth reading thread.

I haven't tested my 1.0 blended metric or sought to find the optimal mix via machine learning. If someone is in the mood to test DRE, this blend and other lower profile metrics that would be appreciated.

Andrew Johnson's player tracking plus minus did very well at predicting performance this past season. I'd put that in the mix if I has the final values handy.

There isn't a clear, widely recognized best. But looking at several and adding input from eyes, SportsVu, other discrete data and judgment from experience, one should be off to a decent start.

This is very helpful, thanks. Do you have a "go-to" stat you like to use as a quick reference? I've been using RPM but I'm not sure how it compares to other stats that are compiled and widely available (and unfortunately it doesn't have any historical stats published).

As a quick reference I will use RPM, BPM, WS/48. I am flexible.

Crow wrote:As a quick reference I will use RPM, BPM, WS/48. I am flexible.

This is a good basis. I look at RPM as best for current players, but also look at RAPM and BPM. For the era before APM, I'd use BPM, and for before BPM, I'd use WS.

If I want to account for everything, including shot defense when actually on the court, I usually take RPM. Sometimes I take the others when I more interested in individual offense or when I want to quickly drag along other stat detail about the players. I'd probably use RPM even more if I could sort by team and other details faster. I occasionally matchup databases but haven't gone to linking to sources for automatic, continuous updates. That would be way to go if super serious and organized.

Because of how poorly they handle shot defense I have tried to swear off boxscore metrics but I still give in. It is just a starring point.

I might go back to EZPM due to its play by play source data for everything including shot defense (all major "statistical" / non adjusted plus minus metrics should do this but only EZPM does) especially if it went thru another round of tweaking to improve goodness of fit to team results.

Voyaging wrote:...
Is there any consensus regarding which metrics are the most accurate in judging a player's value?

AFAICT the consensus is that the compromises required to make a single number metric mean that there is no single best metric. I'm not an expert on the corpus of existing basketball analysis, but it should be obvious that things are much too complex for there to be a single number that's the best answer to all questions. With modern technology, it's very easy to cook up metric formulas, but ensuring that they're actually meaningful or optimal in a particular context is subtle and challenging.

Can you explain what you mean by 'value'? For example, there's a general consensus that - on average - rookie contracts are a good value for teams, but that established NBA players are - on average - more productive on the floor than rookies. Is a player that hits 3/4 shots more valuable than one who hits 7/10? Alternatively, let's suppose some higher being produces a list of NBA players and their corresponding values and hands them to you, how are you going to use those numbers?

Have there been any meta-analyses for analyzing metrics' accuracy?

Sure. Google will find them for you. People aren't that careful with statistics, so bring big grains of salt. You'll probably find as many different outcomes as there are analyses.

In other words, what's the current "best" metric or blend for judging individual players overall?

Crow wrote:For a solo boxscore metric BPM tested better than older, widely known boxscore metrics.

Unless there some consensus 'best test' that I'm unaware of, that seems like a pretty vague or bold claim.

My statement about BPM is based on DSMok1's own testing and the sportsskeptic's (IIRC). That is the best I know about. Is that enough? More or better is always nice but til it happens that is what we have.

I didn't compare BPM to RPM (or RAPM variants). The test specifics may matter. I don't believe / recall that there is a clear cut winner.

The sportsskeptic found blends, many of them, that outperformed single metrics.

That is as non-vague as I can be.

The more I read about and anlayze the single box score metrics, the sillier and more nonsensical they appear to be and the more comical the excuses made for them are. A more evolved and sample-size, noise-controlled version of +/- seems to hold hope for an ultimately useful measure since at least the design incorporates attempting to reach an outcome that reflects relevant value.

Crow wrote:My statement about BPM is based on DSMok1's own testing and the sportsskeptic's (IIRC). That is the best I know about. Is that enough? More or better is always nice but til it happens that is what we have.

I didn't compare BPM to RPM (or RAPM variants). The test specifics may matter. I don't believe / recall that there is a clear cut winner.

The sportsskeptic found blends, many of them, that outperformed single metrics.

That is as non-vague as I can be.

Neil Paine also did a bunch of testing, but has never gotten around to publishing the results. His prior research showed ASPM, the precursor to BPM, as the best box score metric.

rlee wrote:The more I read about and anlayze the single box score metrics, the sillier and more nonsensical they appear to be and the more comical the excuses made for them are. A more evolved and sample-size, noise-controlled version of +/- seems to hold hope for an ultimately useful measure since at least the design incorporates attempting to reach an outcome that reflects relevant value.

Unfortunately, for history (prior to about 2000) and for small sample sizes (1 season or less) lineup-based plus/minus won't work. For such cases we need box-score based approaches, crude though they may be.

Thanks everyone.

NateTG wrote:Can you explain what you mean by 'value'?

By value I'm referring to the estimated number of wins a player would add to a team (or alternatively the average net point differential they would add) per minute played. These numbers would presumably be perfectly proportional.

For reference I'm fine with using a blend if it means more accuracy. I am basically just looking for a place to see a general ranking of players so I can quickly and roughly analyze how good players are to as accurate of an extent as possible.

The best metric at evaluating player value should be the one that out-of-sample prediction accuracy at 100% roster turnover rate is better than others. And that's PER (not even empirical) atm as far as public metrics go. However, player tracking metrics are not tested for large turnover rates since there is less data and one of them recently won the prediction test.

In my work, I found out that with NBA's current yearly roster turnover rate, RPM > BPM >= NPI-RAPM > WS >MPG > PER >= WP.

This means most of those metrics are better at capturing team value but they are not good at distributing it across players. That's why they are somehow good at predicting next year at team level but as years pass they get worse and really worse because of eventual high roster turnover rate.

So, I would check PER first to get a rough idea about how good a player is and then look at RPM for his defensive impact since PER is not really good at capturing that side of the play.

DSMok1 wrote:... research showed ASPM, the precursor to BPM, as the best box score metric.

You mean best of those which were tested?

Voyaging wrote:I am basically just looking for a place to see a general ranking of players so I can quickly and roughly analyze how good players are to as accurate of an extent as possible.

Especially if you don't have any quantitative application in mind, my advice is to start with the understanding that there will probably never be a consensus best single number metric, and then find some place that publishes a convenient list.

I am basically just looking for a place to see a general ranking of players so I can quickly and roughly analyze how good players are to as accurate of an extent as possible.

It's unrealistic to expect something to be quick and rough and simultaneously as accurate as possible. There will always be some economy of effort. The amount of effort involved in uncritically using a single number published by someone else is very low, so the expected accuracy is not so great.