New Impact Metric for College Basketball

[rjb2]

Hi all,

As March Madness is right around the corner, thought I'd share an impact metric that I've created for college basketball. I did not reinvent the wheel here, as it is a box score based prior informed RAPM metric similar to the old version of EPM and LEBRON (hence the name PPM - Predicted-Plus Minus). Some other people have implemented this approach for college basketball, like Jake Flancer (who now works for the Rockets) and Jesse Fischer (tothemean.com), who usually incorporates them into yearly draft models. Unfortunately, they are no longer updated or public. More recently, people like the creator of hoopexplorer.com and more notably Evan Miyakawa (evanmiya.com) have posted prior-informed RAPM-like ratings for college players. I think they both have done an excellent job with college basketball analytics and run fantastic websites, but I feel that their player ratings are too influenced by adjusted plus minus, especially in the small samples of college basketball. For example, it's not uncommon to see super low usage and inefficient players be slight to decent positives on offense because they happened to be in good lineup combinations for 350 possessions. One of the biggest challenges was developing the prior, as even 5 year college RAPM was too noisy as a dependent variable to be regressed against. As a result, I used a MPG and team-based prior in a long-term RAPM, similar to what DsMok1 used to develop BPM, to get a better dependent variable. This metric not perfect by any means, but I feel like it's a good balance between the box score contributions and adjusted plus minus for college basketball. I also have adjusted team ratings and four factors along with award selections for each conference.

https://l5g96w-ryan-bates.shinyapps.io/PPM_2024_25/

[Crow]

Thanks for creating, sharing.

Will review with interest.

Any chance you would correlate your values with the other metrics out there including BPM? If you really wanted to analyze further, find the correlations by position.

For my tastes, there is a lot of space between columns, that could be reduced. More than 3 screens wide on mobile even in landscape orientation. Repeating name column in middle and at right extreme could help a lot to avoid getting line lost in scrolling. Or could divide the data into 2 or 3 parts.

Fwiw, almost all my top 50-60 draft prospects are in top 150 here. Most in top 100. There are some high visibility prospects in the 200s and even a few in 300s.

Getting used to navigating options. Being able to sort by any one position or pulling a specific page to navigate down any sort faster would be appreciated. Can reach Guard and Center but Forward list involves lots of scrolling to my current understanding.

[Crow]

Anybody else look at his work? Have comments that might help him and the community of analysts?

[DSMok1]

Hello, Ryan,

This looks very impressive. It certainly passes the straight-face test.

Results definitely look to be a similar scale to BPM, with a lot of the same names near the top. https://barttorvik.com/playerstat.php?l ... d=20250501

[rjb2]

Hello, Ryan,

This looks very impressive. It certainly passes the straight-face test.

Results definitely look to be a similar scale to BPM, with a lot of the same names near the top. https://barttorvik.com/playerstat.php?l ... d=20250501

Thank you Daniel, I appreciate the kind words. For the offensive dependent variable prior, I actually regressed against OBPM because in my opinion it better represented the value of players who only played 1 to 2 years because the tail ends of the longer term vanilla RAPM tended to be 4 year players and high minute players on really bad teams.

[rjb2]

Thanks for creating, sharing.

Will review with interest.

Any chance you would correlate your values with the other metrics out there including BPM? If you really wanted to analyze further, find the correlations by position.

For my tastes, there is a lot of space between columns, that could be reduced. More than 3 screens wide on mobile even in landscape orientation. Repeating name column in middle and at right extreme could help a lot to avoid getting line lost in scrolling. Or could divide the data into 2 or 3 parts.

Fwiw, almost all my top 50-60 draft prospects are in top 150 here. Most in top 100. There are some high visibility prospects in the 200s and even a few in 300s.

Getting used to navigating options. Being able to sort by any one position or pulling a specific page to navigate down any sort faster would be appreciated. Can reach Guard and Center but Forward list involves lots of scrolling to my current understanding.

Crow, I hope the site is more functional now. Unfortunately, I probably won't be able to add more filter options soon, but it's definitely on my to-do list. Everything should be updated for the tournament now.

[Crow]

Any interest in applying to NBA data?

To have exact same RAPM hybrid metric for college and NBA could be helpful for understanding the cross league translation and relative power of college based RAPM as a tool for prospect projection.

I probably should do more college to NBA RAPM comparison with existing products too.

Any interest in using for team win projection next season?

Your metric is another example of combining and changing approaches.


Fwiw, in view of other conversation, I'll highlight this passage from your previous post:

"One of the biggest challenges was developing the prior, as even 5 year college RAPM was too noisy as a dependent variable to be regressed against. As a result, I used a MPG and team-based prior in a long-term RAPM, similar to what DsMok1 used to develop BPM, to get a better dependent variable."


Any possible next steps / refinements to the metric itself already under consideration? Or questions seeking input?

[rjb2]

I don't really have anything new to add to the NBA-metrics sphere. After I start backfilling more years I may see how well it predicts NBA EPM.

[DSMok1]

Hello, Ryan,

This looks very impressive. It certainly passes the straight-face test.

Results definitely look to be a similar scale to BPM, with a lot of the same names near the top. https://barttorvik.com/playerstat.php?l ... d=20250501

Thank you Daniel, I appreciate the kind words. For the offensive dependent variable prior, I actually regressed against OBPM because in my opinion it better represented the value of players who only played 1 to 2 years because the tail ends of the longer term vanilla RAPM tended to be 4 year players and high minute players on really bad teams.

Well, I guess that explains why it looks so similar to OBPM!

Have you considered using BPM (with some regression for low minutes players) as a prior for your RAPM? Then you may be able to get the best of both worlds. This can be done by simply subtracting out the BPMs of the 10 players on the court (split between offense and defense of course) and then running the full rapm regression on that modified lineup data set. Then you can add back in the prior.

[Crow]

Anything to say about dataset acquisition and manipulation?

[rjb2]

Hello, Ryan,

This looks very impressive. It certainly passes the straight-face test.

Results definitely look to be a similar scale to BPM, with a lot of the same names near the top. https://barttorvik.com/playerstat.php?l ... d=20250501

Thank you Daniel, I appreciate the kind words. For the offensive dependent variable prior, I actually regressed against OBPM because in my opinion it better represented the value of players who only played 1 to 2 years because the tail ends of the longer term vanilla RAPM tended to be 4 year players and high minute players on really bad teams.

Well, I guess that explains why it looks so similar to OBPM!

Have you considered using BPM (with some regression for low minutes players) as a prior for your RAPM? Then you may be able to get the best of both worlds. This can be done by simply subtracting out the BPMs of the 10 players on the court (split between offense and defense of course) and then running the full rapm regression on that modified lineup data set. Then you can add back in the prior.

I did consider it a little bit, but I didn't want my prior to be too biased by the box score. Since BPM is trained on NBA data, I wanted to see if the coefficients would diverge for college basketball. Also, some low usage really efficient players on good teams seem to somewhat break BPM for college basketball (see Alan Griffin's sophomore year for Illinois), so I would have to be careful implementing BPM in the prior. For the offensive dependent variable prior, I regressed MPG and team strength against OBPM because of it's pretty good accuracy and to try and keep a relatively unbiased prior. Interestingly, playmaking seems to matter quite heavily in college basketball. Also the rubber band effect appears much stronger for trailing teams.

[rjb2]

Anything to say about dataset acquisition and manipulation?

The data is scraped from stats.ncaa in R. About your question about future improvements, I would really like to make it a multiyear predictive metric similar to EPM. This would result in big improvements, especially on the defensive end. However there is A LOT of pbp for each season and the NCAA changed the way they record player ID's which kinda messed up my work last summer. Someone just released an API for college basketball so I may be able to make it work.

[DSMok1]

Okay, I see. I misunderstood and thought you were using OBPM as target variable for your regression, not for setting up the prior. That makes more sense.

[Crow]

Any consideration or investigation of the utility of training RAPM on OOS RAPM outright or as a secondary test?

Any estimate of average errors under your current model or model with particular changes?

Have you correlated your rankings with RAPM by hoops-explorer and / or Evan Miya? Or with BPM? Any comments on desirability and approach for possibly preparing a meta-metric with them and / or anything else?

[Crow]

I have some other beginning thoughts on translation of college metrics to NBA metrics but will pullback from these thread at this time.

Maybe I'll say some of it again, if it becomes its own topic with others interested / participating.

The general shift in the median is one thing, handling the tails may be different, getting to specific case handling would be ambitious.