Page 2 of 5
Re: APBR Retrodiction Challenge for Summer 2011
Posted: Wed Jun 01, 2011 10:55 am
by EvanZ
It's up to each metric to decide how to handle rookies. You know, whatever gives you the best predictive power.
Obviously, the only restriction is that you can't use any 2010-11 data to fit your model.
Re: APBR Retrodiction Challenge for Summer 2011
Posted: Wed Jun 01, 2011 4:47 pm
by DSMok1
Statman wrote:EvanZ wrote:Now that the regular season is over, I want to propose that each of us with rating systems do a retrodiction of 2010-11 win totals using 2009 player ratings and actual minutes played in 2010-11. Actually, you could use any prior-year rating system you like. The goal would be to see who comes up with the most accurate retrodiction results. Obviously, the only criterion is that you can't use any 2010 data besides minutes.
I'd like to see ASPM, eWins, RAPM, etc, in addition to mine.
Are you guys up for it? I think we can learn a lot.
How do we deal with rookies? Do we use actual rating results (in 10-11) for them?
I had preseason projections for the rookies, so I've got that covered!
Re: APBR Retrodiction Challenge for Summer 2011
Posted: Wed Jun 01, 2011 5:49 pm
by EvanZ
Daniel, how close were your pre-season projections for rookies to reality? Have you checked?
Re: APBR Retrodiction Challenge for Summer 2011
Posted: Wed Jun 01, 2011 6:41 pm
by Bobbofitos
looks like a valuable project - how do we go about projecting rookies for systems that did not project them at all?
Re: APBR Retrodiction Challenge for Summer 2011
Posted: Wed Jun 01, 2011 8:01 pm
by DSMok1
EvanZ wrote:Daniel, how close were your pre-season projections for rookies to reality? Have you checked?
Code: Select all
Pick Player Team Expected Stdev SPM ReSPM Minutes Team
1 John Wall WAS 0.51 1.65 0.74 0.77 2606 WAS
2 Evan Turner PHI -0.22 1.65 -1.72 -1.57 1797 PHI
3 Derrick Favors NJN -0.67 1.65 -0.29 -0.69 444 UTA
4 Wesley Johnson MIN -1.01 1.68 -2.13 -2.01 2069 MIN
5 DeMarcus Cousins SAC -1.28 1.72 -0.48 -0.48 2309 SAC
6 Ekpe Udoh GSW -1.52 1.76 -3.10 -3.02 1030 GSW
7 Greg Monroe DET -1.73 1.80 2.41 2.03 2222 DET
8 Al-Farouq Aminu LAC -1.92 1.84 -3.31 -3.19 1452 LAC
9 Gordon Hayward UTA -2.09 1.88 -3.75 -3.56 1218 UTA
10 Paul George IND -2.25 1.92 0.35 -0.01 1265 IND
11 Cole Aldrich OKC -2.40 1.96 -2.00 -3.12 141 OKC
12 Xavier Henry MEM -2.54 1.99 -6.69 -5.87 527 MEM
13 Ed Davis TOR -2.68 2.03 0.68 0.33 1602 TOR
14 Patrick Patterson HOU -2.81 2.07 0.36 -0.21 868 HOU
15 Larry Sanders MIL -2.94 2.11 -3.17 -3.21 872 MIL
16 Luke Babbitt POR -3.06 2.15 -10.53 -8.32 137 POR
17 Kevin Seraphin WAS -3.18 2.19 -5.48 -5.31 635 WAS
18 Eric Bledsoe LAC -3.29 2.23 -2.83 -2.63 1841 LAC
19 Avery Bradley BOS -3.40 2.27 -8.88 -7.32 162 BOS
20 James Anderson SAS -3.51 2.31 -4.94 -4.62 286 SAS
21 Craig Brackins PHI -3.62 2.35 -11.49 -8.89 33 PHI
22 Elliot Williams POR -3.72 2.39 0
23 Trevor Booker WAS -3.82 2.43 -0.10 -0.84 1063 WAS
24 Damion James NJN -3.92 2.46 -2.26 -2.62 403 NJN
25 Dominique Jones DAL -4.02 2.50 -3.54 -4.11 135 DAL
26 Quincy Pondexter NOH -4.12 2.54 -3.56 -3.68 734 NOH
27 Jordan Crawford ATL -4.21 2.58 -2.60 -2.35 867 WAS
28 Greivis Vasquez MEM -4.31 2.62 -5.20 -4.78 860 MEM
29 Daniel Orton ORL -4.40 2.66 0
30 Lazar Hayward MIN -4.49 2.70 -3.37 -3.87 419 MIN
It looks pretty good to me. ReSPM is the best estimate of actual SPM, useful for players with few minutes actually played.
Re: APBR Retrodiction Challenge for Summer 2011
Posted: Wed Jun 01, 2011 8:12 pm
by EvanZ
Bobbofitos wrote:looks like a valuable project - how do we go about projecting rookies for systems that did not project them at all?
That's left up to you, of course. For example, I would find the average ezPM rookie rating for last season, and use that for all rookies. That would be the absolute simplest way. Daniel has more magical powers.
Re: APBR Retrodiction Challenge for Summer 2011
Posted: Wed Jun 01, 2011 8:14 pm
by EvanZ
People don't realize how bad most rookies are. Or do they? I don't think so.
Re: APBR Retrodiction Challenge for Summer 2011
Posted: Thu Jun 02, 2011 4:16 am
by bbstats
I'm going to see if I can do somewhat better than Draft Pick # for college ratings...we'll see how much time I have to research it though...
Re: APBR Retrodiction Challenge for Summer 2011
Posted: Thu Jun 02, 2011 9:43 am
by J.E.
I'll now start writing the code that determines error in two ways: per possession and efficiency differential.
Participants can just send me their player ratings, preferably in csv. Best case would be if the csv file listed baksetballvalue player id; if that's not possible then the name next to the ID in the bbv player files would be good; if that's not possible either please use the name listed on basketball reference
Re: APBR Retrodiction Challenge for Summer 2011
Posted: Thu Jun 02, 2011 11:40 am
by EvanZ
J.E. wrote:I'll now start writing the code that determines error in two ways: per possession and efficiency differential.
Participants can just send me their player ratings, preferably in csv. Best case would be if the csv file listed baksetballvalue player id; if that's not possible then the name next to the ID in the bbv player files would be good; if that's not possible either please use the name listed on basketball reference
That would be great!
Re: APBR Retrodiction Challenge for Summer 2011
Posted: Thu Jun 02, 2011 1:25 pm
by bbstats
Bobbofitos wrote:looks like a valuable project - how do we go about projecting rookies for systems that did not project them at all?
Start at -4, -5? What's the average rookie APM?
Re: APBR Retrodiction Challenge for Summer 2011
Posted: Thu Jun 02, 2011 1:40 pm
by DSMok1
bbstats wrote:Bobbofitos wrote:looks like a valuable project - how do we go about projecting rookies for systems that did not project them at all?
Start at -4, -5? What's the average rookie APM?
The issue is that the worst players won't play: so you only get the top part of the distribution actually on the court. Once you get to the second round, more than half of the players won't get on the court, so only the top, say, 25th percentile, will get to actually play.
This is an issue with projection systems that actually project minor league/drafts. As Tango found out at the Book blog, just projecting rookies at league average is as good as projecting players based on their minor league stats--because only the ones that overperform/perform close to league average will actually get to play.
So, for instance, I projected the picks in the 30-40 range at about -5. Well, most were probably that bad; only 3 actually got any playing time at all. However, the one who got the most playing time was also the only one who played well--Landry Fields. And he played about league average. Well, I had given each player in that range about a 5% chance of being league average based on my standard deviations. In other words, my projection was pretty good.
But if you only compare players that actually played, then my projection of Landry Fields as -5.28 looks pretty bad.
Ideas on how to deal with this? Consider all rookies as a blanket value (between -1.5 and -2.5) for all projection systems?
Re: APBR Retrodiction Challenge for Summer 2011
Posted: Thu Jun 02, 2011 1:47 pm
by EvanZ
I use a value of -1.0. I found that the was close to the mean for 2009-10 for ezPM for rookies > 500 possessions.
What we shouldn't do it bias our ratings in any way that reflects knowledge of the current season. (goes without saying, I guess)
Re: APBR Retrodiction Challenge for Summer 2011
Posted: Thu Jun 02, 2011 1:59 pm
by DSMok1
EvanZ wrote:I use a value of -1.0. I found that the was close to the mean for 2009-10 for ezPM for rookies > 500 possessions.
What we shouldn't do it bias our ratings in any way that reflects knowledge of the current season. (goes without saying, I guess)
Yeah, I'm just pointing out that selective sampling bias will cause a better projection to look like a worse one when dealing with draft picks, if we just look at who played and weight by how much. It's a well-known phenomenon.
Re: APBR Retrodiction Challenge for Summer 2011
Posted: Thu Jun 02, 2011 2:06 pm
by EvanZ
I guess I'm ok with us all agreeing on a fixed value, whatever it is. Rookie projections could potentially be the subject of a future "challenge".