APBRmetrics

Go with whatever is your call within your time budget. There can be after-riffs here or on other sites or next season.

ampersand5 wrote:

DSMok1 wrote:
It's an issue, for sure. If one rating system has a player #1 and another player #10, and a second system has player 1 #40 and player 2 #30... should they average the same? Unlikely, there's a bell curve distribution usually. Player 1 should be rated higher, possibly much higher, in an average of the two.

This is not necessarily the case. For example, one system could have player 1 rated 99.7 (1) and the 10th ranked player rated 99.3 (10) while a second system has player 1 rated 23 (40) and the other played rated 74 (30).

This is ultimately my point - we have no idea what the distribution is of the different ratings. By using lots of models, we are trying to find consensus on where a player ranks.

I understand what you were trying to state - in assuming that there is a (potentially) bigger variation in player scores between middle of the pack rankings (30-40) than at the top (1-2).

I find this to be problematic because the models were created to produce a player ranking system, not to create a uniform player score rating. By way of example -
lets look at how two models could rate the top ten players in the draft

Model 1:
1) 100
2) 99
3) 98.5
4) 98
5) 97
6) 96.5
7) 96.4
8) 96
9) 95.8
10) 95.6

Model 2:
1) 100
2) 92
3) 78
4) 55
5) 54
6) 52
7) 50
8) 49.7
9) 47
10) 45

Any model that has a different distribution of player ratings than the rest is going to through off the entire cumulative average for any of the players that do well/poorly in that model.

Because each model has a completely different way of rating players within its own system, I do not think it makes sense to use these comparative ratings in relation to other models.

I see your point; I guess I hadn't quite thought it through all the way. To do it correctly, all models should be standardized and the Z scores used to do the combination. If we don't have time, that's OK, but ideally Z scores would be used.

Yes, min-max normalization stabilizes variance, rather than make it equal, which is what z-score does. A 0-100 scale, however, is more interpretable, which is why min-max is often used. Z score will give results like 4.17, 0.56, -1.29. Z score is

(xval-mean(x))/stdev(x)

where xval is the raw value, mean(x) is the mean of each draft model's ranking, and stdev(x) is the standard deviation of each model's ranking. But do whatever you want/have time for, because as Crow said, you're the one doing the work to put it together, which is appreciated. Looking forward to the Monday piece.

I put together a blurb for the follow-up article. Not as specific to my model as is probably ideal... and maybe a bit preachy... but I hope it works. Let me know if there is anything I can remove/add to help it fit better with the theme.

In comparing out-of-sample retrodictions to actual draft order, EWP does about as well as NBA decision-makers, while my "HUMBLE" model (which integrates scouting consensus) actually does a bit better than either (https://pbs.twimg.com/media/CGljDbwU8AA_j20.png:large). This is good support for the use of draft models, but I still would not advocate drafting simply based on a statistical model, or even an averaged collection of models. What I would advise is a scientific approach to prospect evaluation, and I think these models provide an excellent foundation to begin that process. This does not mean weighing models against your personal subjective sense, but rather understanding where the built-in assumptions might go wrong in a particular case and challenging them with data or strong logical arguments.

For example... None of the models account for progression across the season, should we give Justise Winslow a subjective bump due to his late-season surge? I don't know, but this is a testable hypothesis. Most models account for net strength-of-schedule, but they don't give special weight to individual performances against the stiffest competition. If they did, it might hurt D'Angello Russell's rating. This is a simple question that remains untested. If Coach K's system historically depresses bigmen's defensive rebounding (it does) it should probably be taken into account when evaluating Okafor. Speaking of Okafor, he and Towns offer contrasting “Old school” vs “New school” styles. Identifying whether and how much this matters might be helped through an analytic approach, but it definitely demands the subjective counsel of scouts who understand the complexity of NBA game-planning... I could go on for days with more examples like these, and that is exactly what front offices should be doing when they debate prospects using statistical outputs.

People who fall into arguments of “well the numbers say this” or “well my years of experience watching basketball say this” will always lose to those who constantly challenge their own opinions with the most rigorous methods available. Using models correctly is a lot of work, and it is a process that integrates follow-up analytic research and subjective observations in equal measure. This requires commitment to understanding both the math and the game. This likely turns a lot of people off on either end, but the wisest path usually is not easy.

BTW... regarding the issue of how to standardize the models, I think rank order is going to be the most intuitive for folks to read. Yes it has problems, but it really shouldn't pervert the outcomes much at all.

vjl110 wrote:BTW... regarding the issue of how to standardize the models, I think rank order is going to be the most intuitive for folks to read. Yes it has problems, but it really shouldn't pervert the outcomes much at all.

If we post z-score standardized numbers, people might also interpret that as positive-negative nba imact with the cutoff point coming somewhere in the early second round, when in fact, the overwhelming majority of any draft class will have negative to no nba impact.

nrestifo wrote:Yes, min-max normalization stabilizes variance, rather than make it equal, which is what z-score does. A 0-100 scale, however, is more interpretable, which is why min-max is often used. Z score will give results like 4.17, 0.56, -1.29. Z score is

(xval-mean(x))/stdev(x)

where xval is the raw value, mean(x) is the mean of each draft model's ranking, and stdev(x) is the standard deviation of each model's ranking. But do whatever you want/have time for, because as Crow said, you're the one doing the work to put it together, which is appreciated. Looking forward to the Monday piece.

I need to update the rankings on the spreadsheet (as found here: https://docs.google.com/spreadsheets/d/ ... =865187672 ) to reflect the changes made by posters + add in statmans rankings.

I don't have the time to do them myself, but if someone wanted to create Z Score rankings, I think be happy to see how they look (and obviously be very appreciative).
The biggest asset of doing this is that if we had Z-score ratings, we could then have all of the international prospects fitted into our rankings/article, which is something that we would all benefit from.

I will have a rough draft of the article up later today.

ampersand5 wrote: I don't have the time to do them myself, but if someone wanted to create Z Score rankings, I think be happy to see how they look (and obviously be very appreciative).
The biggest asset of doing this is that if we had Z-score ratings, we could then have all of the international prospects fitted into our rankings/article, which is something that we would all benefit from.

I will have a rough draft of the article up later today.

If I had all the raw numbers and editing privileges, I'd be happy to do so, would only take me a minute or two, just so people had an idea what it would like.

nrestifo wrote:

ampersand5 wrote: I don't have the time to do them myself, but if someone wanted to create Z Score rankings, I think be happy to see how they look (and obviously be very appreciative).
The biggest asset of doing this is that if we had Z-score ratings, we could then have all of the international prospects fitted into our rankings/article, which is something that we would all benefit from.

I will have a rough draft of the article up later today.

If I had all the raw numbers and editing privileges, I'd be happy to do so, would only take me a minute or two, just so people had an idea what it would like.

All of the raw numbers are posted in this thread, so its really just an issue of someone inputting all of them. I just asked Jesse if he already has them in a spreadsheet that he could send us, so I'll see what he says.

Instead of editing the original spreadsheet, I think it would make more sense to just copy n paste all of the values into a new one.

BTW - I do not expect you to do this - just that if anyone does want to.

I have just finished my first draft of the article. I still have lots to add/edit (including each user's piece).



also - dumb stats question.

Can Zscores be put through a min-max normalization? that seems like it would make everyone happy.

In the alternative, it might still make sense to do zscores internally (and then display the rankings publically) so we can figure out where the Euros should place in addition to adding models that don't use the same joint list.

Nick, if I can get all of the scores for each model, can you try and figure this stuff out for the spreadsheet?

Follow Up submission:

In a sense I would say that statistical analysis is the worst way to select draft picks, except all of the others. In reality though, best practices are to combine scouting information and analytic analysis.

There are a number of things scouts and video reviewers are able to see that stats can not. For example, for lack of a better word, character is a tremendous variable in a young player's development. Some of that shows up through the stat sheet, poor decision makers will have worse shot selection and more turnovers, and hustle stats like boards and steals show hustle. But, many aspects of ability and willingness to learn or put in gym time or deal with adversity are less easy to infer. Defense also tends o be obscured in draft models, players like Willie Cauley-Stein may be undervalued because their contributions on the court are less likely to end up in the box score.

It is well known that player stats are less stable when they change teams, with different teammates around them they get different opportunities and fill different rolls. Taking situation into account can help adjust the numbers players put up, at least at the margins. However, it is dangerous over imagine how a player would blossom in a different environment, and the grounding that an objective model brings is one of the more important contributions analytic draft models can bring to a team's drafting process.

Essentially, the difficulty in the drafting process, I suspect, is not collecting enough information but in weighing it properly and perhaps avoiding information cascades (group think and hype) or falling in love with a particular player early in the process. I should note that some teams seem to have less developed processes than others, it would seem.

My P-AWS draft model is built using box score statistics, age, competition level, and high school rankings.Of the outliers between P-AWS and the consensus rankings Kevon Looney, a freshman who was highly recruited entering college, may be the one I would point any organization to re-evaluate. Looney scored above average compared to the rest of the draft prospects in every statistical category except for scoring, which just happens to be the least predictable measure going from college to the pros.

Of course, the model should weighted as a part of a draft process alongside video review, scouting, character assessment and health evaluations. All information, incidentally that should be available to the in-house analytics staff as well.

Did you want international projections for this version?

I can throw them up...

Mudiay 4.1
Porzingis 4.9
Hezonja 3.8
Gudaitis 3.8
Vezenkov 2.7
Hernangomez 2.3
Osman 2.4
Jaiteh 1.8
Diaz 1.6
Garcia 1.6
Milutinov 1.7

Not sure if this link will take care of including my rankings in the piece, but has them all. I never ranked Tuttle since he doesn't appear in the DX top 100, which I use as a filter because the model is trained on drafted players.

https://docs.google.com/spreadsheets/d/ ... sp=sharing

Update:

I have finished the first draft of everything

Thanks AJ for the writeup. How difficult is it to make a ranking for Seth Tuttle? He's one of the more featured player in the article/rankings so it would be nice to have your ranking for him. If it would take too much an effort, its completely understandable and I'll just use a dummy ranking.

We are already at 7 pages and we are still waiting on three more submissions. This means that I am going to have edit this down as much as humanly possible tomorrow.

Consequently, I wanted to ask the people on here if they mind if I edit parts of their blurb. if you don't want me to shorten your stuff, I will leave it as is (PM if you don't want to post it publicly).
The reason why I am thinking about doing this is (aside from the need to reduce the articles length) that each user spoke a lot of general themes about models that are already covered in the article, and not specific to their model.

-----

One of the things Givony asked for which isn't really specifically addressed is the over/under rated players according to the DE rankings.
This is somewhat problematic as the rankings will be very different tomorrow once we take into account the updated numbers/new additions.

Can anyone provide some insight why they think some of these players are ranked where they are? It would be greatly appreciated.

Also, I spoke about how steals and rebounds translate well and scoring doesn't. Any other stats that you guys think are important to mention specifically?

71 Jonathan Holmes 21 (+50) - JF
60 Rakeem Christmas 29 (+31) - LV
74 Julian Washburn 44 (+30) - ME
59 Anthony Brown 33 (+26) - NR
33 Devin Booker 9 (+24) - NR
66 Chris Walker 43 (+23) - SS
57 Norman Powell 35 (+22) - BPM
62 Joseph Young 45 (+17) - JF
46 Michael Qualls 32 (+14) - LV
45 Michael Frazier 31 (+14) - SS

Most "Underrated" by DX according to Draft Model Composite:

28 Wesley Saunders 72 ( -44 ) - NR
31 Seth Tuttle 74 ( -43 ) - ME
48 Derrick Marks 69 ( -21 ) - SS
42 Branden Dawson 61 ( -19 ) - LV
30 Vince Hunter 48 ( -18 ) - LV
44 Chasson Randle 62 ( -18 ) - SS
9 Delon Wright 25 ( -16 ) - JF
26 Terry Rozier 40 ( -14 ) - SS
41 Larry Nance 54 ( -13 ) - ME
5 Kevon Looney 17 ( -12 ) - NR

A last thing to note is that I am going to try and see if we can get a third on international prospects. There simply isn't room in this one to include it and it deserves more space.
This also fits nicely into the ranking format debate above. I think for the coming article, the current system is fine. However, for the international prospect article, it would make sense to do everything with Z scores. Aside from looking better for our final finished product, it will make integrating the internationals into the rankings much easier.


THANK YOU!

Feel free to chop up my blurb as much as you like.

Worth noting as a general point in comparing those lists...
Only one player on the "models underrated" list is from outside the five power conferences, while six of the guys in the "models overrated" list are from schools with less exposure (5 if you remove Rozier who played for Louisville which obviously gets TV time).

I can also add something that covers three of the specific cases as well...

One other interesting angle to address those lists is a comparison of teammates Vince Hunter and Julian Washburn.

"Glancing at the boxscore, the gap between Hunter and Washburn as prospects seems huge. As a sophomore, Hunter carried an impressive 30.5 usage% compared to Washburns 18% rate which is unusually low for a senior NBA prospect. Not only that but Hunter scored much more efficiently (54.3 eFG% vs. 48 eFG%) in spite of carrying a much higher load. Hunter also added dominant rebounding as a secondary boxscore filler, while Washburn has no standout numbers. However, I think we can start to see why scouts view them a closer prospects when we try to identify "NBA tools" and realistic role at the next level. Washburn is noted as a strong defender (something that often evades statistical models) and shows some promise as an outside shooter. Ideally he could fit nicely into the traditional 3-n-D role. Meanwhile, Hunter does not appear to have much range or creation ability, and thus seems destined to be an undersized paint 4. That is probably the easiest role to fill in the NBA, and rarely costs more than the veteran minimum. This issue of replaceability can be expanded to other players the models appear to "overrate" as well. Branden Dawson projects as an interesting wing-stopper with limited shooting ability. This skillset can influence team differential, but there are guys like Al Farouq Aminu who have demonstrated mastery of that niche available for cheap every offseason. There is limited incentive to target an undervalued skill in the draft."