APBRmetrics

Here is a project that one or more here probably can do:

Given your list of variables and processing steps, what apparent weights are the league in aggregate using on average given the draft order (not necessarily the same as draft ratings / rankings but the only reference point)? Can you backward map that? How different are the weights from yours? This may be a marketing strategy as well as an interesting analytic choice: what blend of apparent league weights and your weights best explains 1st year performance (of course 3rd or 5th would be better)?



Or using machine learning what set of weights for the four factors for college offense and defense (using team efg% for defense or better if you think you have it) best predicts NBA performance? More complex models are probably better but what does a model at this level of detail say and how much better are the more complex models and specifically where are they better and not?

If one applied a typology to this season's draftees (a 5 position or x cluster model) how do the models vary for those groups beyond the individual variances? Even if thus kind of analysis is not done comprehensively, what does it look like for pure points or playmaking forwards? Performance review of last year or last 5 would be nice too.

I had mentions the possibility of team need / pick / fit evaluation previously. If that done, perhaps teams or their consultants / observers could try to adjust their model weights in a systematic way instead of keeping status quo or making non-objective evidence based adjustments.

Amp, do want to add anything to the article about your basketball analytics origins, interests, experiences, aspirations? How do you rank players? What if anything do you think the models are missing or undervaluing? Might you produce a model in future? What direction would you go or do you lean? What did you learn from organizing this project?

Do you want to put the 3-5 best comments on article 1 in article 2? Are you going to talk about the criticism or selective usage you mentioned here?

Any plans to organize a podcast pre draft (and maybe post draft)? This surely will require some lead time, ground work especially if more than 2-3 people are involved?

Did anyone make a new search of the academic literature before designing or finalizing their model? Make use of hazy memories? Any indication teams pay attention to the academic literature on this topic? What if any academic article is worth name dropping / linking to?

Will any of the revisers discuss the temptation to revise to a higher correlation to the average or things they avoided saying to self or doing to avoid such a conscious temptation? Same temptation exists for non-quants of course.


Anybody thought that asking the 10-20 best draftees to firmly rate the rest of the group (not just casually talk about x,y and z) or just guys at their position would be useful? Might not be very helpful for ranks in general but still might have an insight or surprise here and there. What about asking the best guys from the classes already in league for 1,3 or 5 years (if any would play the game for free, without getting in trouble)? Would a panel of 5 HOFs on average beat the performance accuracy of average team draft picks, DX or these models? Are there HOFs or even just former all-stars or especially titlewinners who would do it if asked?


Would DX want to try an ELO style poll of readers at DX or one of your pre-draft media partners? Probably just the lottery picks.

Doesn't somebody or several sites do a mock draft roundup? Fairly or not calling them mostly subjective, how does their average compare with the quant model avg., DX and the eventual draft and performance?

Supply and demand by position in the league and in the draft presumably influences team picks, mock draft picks and later performance opportunities and hence model performance, so do any of the quant models want to consider market supply and demand? I don't think I heard any model doing so, to date. Statman mentioned the mistakes that can come from such consideration, but better results are also possible if the consideration is thoughtful and fact based.

How are the trends for lottery or first round picks running by position over last 5 - 10 years? And versus the payoffs? Is the variance between draft pick and the mock and models a good clue for what players will be sought or offered in trade talks early in their careers? Earlier correlations would suggest so.

From a money ball standpoint does it makes more sense to seek players expected to shine by year 2-3, 4, or 5 or later? Should teams try to predict that or is that too cute / beyond a reasonable expectation to grasp? Of course the strategy could vary by particulars in team yr to yr salary commitments and cap usage strategies.

Crow wrote:Jonathan, are you planning or willing to write about how your derive your draft ratings? Is the blend of scout opinion, measurable and stats / metrics holistic or do you scores and weights to any degree to pull them together? Anything you've gleaned from NBA staff about how holistic or scores and weights driven they are in their rankings? I understand the mock transforms the ratings into a different list.

I am happy to chime on any of this stuff. Willing to write whatever if Daniel feels it will fit into the context of the article. I am letting him do this as he sees fit since the first article came out so well.

To answer your question, the mock and top-100 rankings evolve as the year moves on. It really starts off as my opinion on how I see things playing out with how good I think the prospects are, and I try to incorporate feedback from NBA guys in if I see they are clearly higher or lower on guys than I am. Right now the mock is straight up how I think the teams will pick, while the Top-100 ranking is much closer to my own feelings on how good the prospects are.

I think some NBA teams do have draft models that they look at frequently as kind of a counterbalance to their own rankings. One team said they basically treat their draft model as another scout sitting in their war room, and are frequently quoting "him" if there are stark disagreements between them.

Crow wrote:Can you get any NBA scouts, analysts, upper management or former such to comment on the models for attribution or anonymously?

Yes. I will be at the adidas EuroCamp in Treviso starting Saturday. There will be 150+ NBA people wandering around a small gym so it will be very easy for me to get some quotes on or off the record if you want to line up some specific questions that I can ask them.

Crow wrote:How much do you think hardcore internet draft analysis affects insider ratings, ranking, picks? Does more casual social media have impact as well? Are teams, internet sites and fans converging on evaluation methods and conclusions, staying around the same level or similarity or getting more different?

I don't think they have a huge influence. But I do believe people look at them, and then go in and dig in harder on guys where they are clear disagreements, to make sure that they aren't missing anything. I do think that there is a degree of group think that inevitably happens in the NBA Draft, where some guys are just anointed great prospects, and it can be difficult for some executives to buck the trend and go against the grain. But I think that's natural. And most of the good teams don't fall for that.

Crow wrote:Do any teams have quantitative analysts (not scouts) exclusively devoted to draft analysis or more generally player ratings? Is there much movement of such guys between teams or up the ladder? Has anyone tried a performance bonus system to reward and attract the best talent?

I think that teams have significant analytic departments now, but most of the people that do pro personnel and NBA stuff are delve into the draft stuff too. It's only natural considering the timing of the draft and how much later it is than the end of the regular season for most teams. Guys are expected to multitask and not just specialize in the NBA, college, or Europe when it comes to analytics.

Crow wrote:What do you think the level of use and helpfulness of brain typing / personality evaluation is? Might Dr Ilardi be willing to answer any questions about the topic from his experience and training? Does anyone give a quantitative weight to inadequate SATs for timely admission, arrests, family arrests, employment, income, educational achievement, team suspensions, technicals, ejections, game fights, transfers, etc.?
Any weight to agent pursuit or agent selection? Shoe company interest? For second rounders, international team interest?

There's a lot of room for advancement in this area I feel. We've been writing some articles on this topic (personality evaluations) with guys who slipped between the cracks like Draymond Green, Beverley, Jimmy Butler, etc. Most teams are only starting to think about how to incorporate this area into their draft evaluations. Regarding the rest, I think there's a lot of room for quantifying some of the things you talked about. It will be difficult though.

Crow wrote:Is there anything else teams use that the modelers didn't mention? Are top performances and / or performances against top team and individual performances weighted more heavily by teams, you or from what you can tell these models?

I haven't seen enough teams' models to know the exact answer to this. I try not to pry too much because it's obviously a sensitive topic. But I've heard people talk about the importance of doing a better job of quantifying strength of schedule. And also I know some teams have their own interns go back and chart the entire seasons of prospects they are interested in to verify the boxscore data and get more stuff (like stop%, hockey assists, probably much more) out of the games than what we've been using here.

Crow wrote:Should the regression of draft picks 23 yrs and older be reduced or eliminated for interior players? I think 17 of the 20 best first year performances on this list http://bkref.com/tiny/P9BrJ are interior players. And it is still 18 of top 20 in yr 3. My minute levels might but too low, but... something to consider and consistent with the conventional wisdom that bigs are better in mid20s than before.

Does the age curve for players really say more about players at different ages or the pick habits of teams, their player development skills that might vary with age, their level of opportunity and patience given, etc. Do teams tend to perform about the same with the age groups or are there stars and trailers with different age groups? Similar analysis could be done by position / cluster or for the internationals. For majors vs mid-majors (and minors)? As stated before, should age be a completely separate step or highly context cognizant and integrated rather than separate?

My college to NBA & NBA age curves completely deal with this trend. Rebounding translates the best (after adjusting for pace, SoS, etc. obviously) - and on average a player is at their best as a rebounder in the NBA with a pretty flat line rate anywhere from 19 to 24. On average a player has his highest shot block rate his first season (although usually much lesser mpg than later years).

On the other hand, usage type skillsets start low relative to college & trend up (pretty drastically in the early years) to 25 or 26.

So, bigs in my work (especially lower usage good rebounder/shot blockers) will normally start with a higher NBA rating (per minute production) relative to their college rating as a rookie than high usage smaller players. Many of those same bigs often project less or much less peak (best case scenerio) playing time because of higher foul rates. But, the higher usage smaller players often will peak much higher at their NBA prime & often produce more career WAR because of that career peak & more projected career playing time.

Anthony Davis was the best of both worlds - big man stats translated as better young age production - and low foul rates & balanced statistical skillsets translated to a ton of playing time (best case scenerio) right out the gate & for his career. His NBA usage stats have exploded though even more than my best college prospect since Tim Duncan projections.

Average payoff (in peak wins) at each draft slot (1990-2012) by "actual order", "EWP rank", HUM rank":

Thanks for the reads and replies.

(Sorry for the dropped or wrong letters or words caused by bad typing on phone or auto incorrect. I will try to find and correct.)



The size of the lottery group seems about perfect, by design or chance. Not sure if I've seen a chart of draft # vs. performance by position or style cluster though. Might vary compared to average; and if it does and if the data set is big enough, that could be very important information.

I'll respond to the latest posts in the future when I have some time.

In the interim, I just wanted to note two things and share how popular the first article was:

http://sharetally.co/134597/
over 10,000 facebook shares, likes and comments. In addition to being reposted by a large amount of blogs/forums and generating substantial discussion.

AJ - thanks for your submission, it's great to have you on board and your contribution is sincerely appreciated. I don't have the time now to check the names, but can you cross-reference that with our joint list of 75 names (plus Seth Tuttle)?

Also - for everyone participating; here's one question I forgot to mention that I hope for you to touch on. In your metrics, what stat/skill translates the most into the NBA (ie high steal rate = athleticism or something like that). Conversely, are there other stats that don't translate well? On this final point, if you were to select a player who's mediocre but incredible at one skill, what skill would you focus on/what player?


THANKS!

Crow wrote:Jesse would you be interested in adding a 1st year performance check to your site on last season's models with correlation of model and avg. to that performance (some blend of BPM, RPM, DRE, WS/48)?

Yes we have plans to who some back test results for the different models as long as we can get the data. I have started contacting some people directly on here already. Will update when have something to show. I have contacted Layne and Nick. If I could get contact info (email address) to coordinate for the remaining models that would be appreciated.

Crow wrote:Here is a project that one or more here probably can do:

Given your list of variables and processing steps, what apparent weights are the league in aggregate using on average given the draft order (not necessarily the same as draft ratings / rankings but the only reference point)? Can you backward map that? How different are the weights from yours?

There was some related work done on this last year:
http://www.tothemean.com/2014/06/25/fin ... e-nba.html
http://www.tothemean.com/2014/07/06/pre ... e-nba.html

Statman wrote:

jessefis wrote:Along with my updated board I would also like to point you to a new tool we have at tothemean.com which allows you to compare these analytic draft models in a better way than a shared google doc. We also allow you to compare with some more well known projections. We have already added the draft boards posted here (let us know if you dont want to be included or have updated data). Check out our post with details here:
http://www.tothemean.com/2015/05/31/nba ... odels.html
Here is a link to the actual tool:
http://www.tothemean.com/tools/draft-models/

Let us know if you have any feedback/suggestions. We have a long list already but feedback might help prioritize. Enjoy.

Hey Jesse & Ampersand - I'm thinking maybe every model should be adjusted to a 0 to 100 base for every prospect based on their results - so as to best compile players who some models REALLY love (with high "rating" results but may be undervalued my a normal general consensus). Say if a model has Towns, Russell, & Okafor as MUCH better than the #4 guy - this would get reflected much better in a 0 to 100 scale than just ranking position (the gap between #3 & #4 might be much larger than #4 to #15?). There are a number of guys in the 75 player pool who I'll project with 0 future NBA WAR (never better than NBA replacement level), ordering those guys (I dunno, maybe 40 on - don't know yet) seems weird.

I like how Nick threw this together last minute before the draft last year: https://docs.google.com/spreadsheets/d/ ... edit#gid=0

We get enough models - maybe throw out a high low (and EVERY prospect having a 0 to 100 score in every model unlike the spreadsheet I linked)- I think that approach would give us the best consensus.

Now - if you all were already doing this - as you were. I might have missed it, my mind is all over the place recently.

We absolutely agree that simply averaging models rankings isn't that great for coming up with a consensus and correlation. By end of day Friday we should have an update to the comparison tool: http://www.tothemean.com/tools/draft-models/ where we will be switching to the raw numbers people have provided when possible and for rankings that dont have raw numbers we have a method for handling somewhat properly.

jessefischer33 wrote: We absolutely agree that simply averaging models rankings isn't that great for coming up with a consensus and correlation. By end of day Friday we should have an update to the comparison tool: http://www.tothemean.com/tools/draft-models/ where we will be switching to the raw numbers people have provided when possible and for rankings that dont have raw numbers we have a method for handling somewhat properly.

Great job Jesse. Looking forward to it.