APBR Retrodiction Challenge for Summer 2011

[EvanZ]

It's up to each metric to decide how to handle rookies. You know, whatever gives you the best predictive power.

Obviously, the only restriction is that you can't use any 2010-11 data to fit your model.

[DSMok1]

Now that the regular season is over, I want to propose that each of us with rating systems do a retrodiction of 2010-11 win totals using 2009 player ratings and actual minutes played in 2010-11. Actually, you could use any prior-year rating system you like. The goal would be to see who comes up with the most accurate retrodiction results. Obviously, the only criterion is that you can't use any 2010 data besides minutes.

I'd like to see ASPM, eWins, RAPM, etc, in addition to mine.

Are you guys up for it? I think we can learn a lot.

How do we deal with rookies? Do we use actual rating results (in 10-11) for them?

I had preseason projections for the rookies, so I've got that covered!

[EvanZ]

Daniel, how close were your pre-season projections for rookies to reality? Have you checked?

[Bobbofitos]

looks like a valuable project - how do we go about projecting rookies for systems that did not project them at all?

[DSMok1]

Daniel, how close were your pre-season projections for rookies to reality? Have you checked?

Code: Select all

 Pick        Player         Team  Expected   Stdev     SPM     ReSPM   Minutes  Team
  1   John Wall             WAS     0.51     1.65     0.74      0.77     2606   WAS
  2   Evan Turner           PHI     -0.22    1.65     -1.72    -1.57     1797   PHI
  3   Derrick Favors        NJN     -0.67    1.65     -0.29    -0.69     444    UTA
  4   Wesley Johnson        MIN     -1.01    1.68     -2.13    -2.01     2069   MIN
  5   DeMarcus Cousins      SAC     -1.28    1.72     -0.48    -0.48     2309   SAC
  6   Ekpe Udoh             GSW     -1.52    1.76     -3.10    -3.02     1030   GSW
  7   Greg Monroe           DET     -1.73    1.80     2.41      2.03     2222   DET
  8   Al-Farouq Aminu       LAC     -1.92    1.84     -3.31    -3.19     1452   LAC
  9   Gordon Hayward        UTA     -2.09    1.88     -3.75    -3.56     1218   UTA
  10  Paul George           IND     -2.25    1.92     0.35     -0.01     1265   IND
  11  Cole Aldrich          OKC     -2.40    1.96     -2.00    -3.12     141    OKC
  12  Xavier Henry          MEM     -2.54    1.99     -6.69    -5.87     527    MEM
  13  Ed Davis              TOR     -2.68    2.03     0.68      0.33     1602   TOR
  14  Patrick Patterson     HOU     -2.81    2.07     0.36     -0.21     868    HOU
  15  Larry Sanders         MIL     -2.94    2.11     -3.17    -3.21     872    MIL
  16  Luke Babbitt          POR     -3.06    2.15    -10.53    -8.32     137    POR
  17  Kevin Seraphin        WAS     -3.18    2.19     -5.48    -5.31     635    WAS
  18  Eric Bledsoe          LAC     -3.29    2.23     -2.83    -2.63     1841   LAC
  19  Avery Bradley         BOS     -3.40    2.27     -8.88    -7.32     162    BOS
  20  James Anderson        SAS     -3.51    2.31     -4.94    -4.62     286    SAS
  21  Craig Brackins        PHI     -3.62    2.35    -11.49    -8.89      33    PHI
  22  Elliot Williams       POR     -3.72    2.39                         0
  23  Trevor Booker         WAS     -3.82    2.43     -0.10    -0.84     1063   WAS
  24  Damion James          NJN     -3.92    2.46     -2.26    -2.62     403    NJN
  25  Dominique Jones       DAL     -4.02    2.50     -3.54    -4.11     135    DAL
  26  Quincy Pondexter      NOH     -4.12    2.54     -3.56    -3.68     734    NOH
  27  Jordan Crawford       ATL     -4.21    2.58     -2.60    -2.35     867    WAS
  28  Greivis Vasquez       MEM     -4.31    2.62     -5.20    -4.78     860    MEM
  29  Daniel Orton          ORL     -4.40    2.66                         0
  30  Lazar Hayward         MIN     -4.49    2.70     -3.37    -3.87     419    MIN

It looks pretty good to me. ReSPM is the best estimate of actual SPM, useful for players with few minutes actually played.

[EvanZ]

looks like a valuable project - how do we go about projecting rookies for systems that did not project them at all?

That's left up to you, of course. For example, I would find the average ezPM rookie rating for last season, and use that for all rookies. That would be the absolute simplest way. Daniel has more magical powers.

[EvanZ]

People don't realize how bad most rookies are. Or do they? I don't think so.

[bbstats]

I'm going to see if I can do somewhat better than Draft Pick # for college ratings...we'll see how much time I have to research it though...

[J.E.]

I'll now start writing the code that determines error in two ways: per possession and efficiency differential.
Participants can just send me their player ratings, preferably in csv. Best case would be if the csv file listed baksetballvalue player id; if that's not possible then the name next to the ID in the bbv player files would be good; if that's not possible either please use the name listed on basketball reference

[EvanZ]

I'll now start writing the code that determines error in two ways: per possession and efficiency differential.
Participants can just send me their player ratings, preferably in csv. Best case would be if the csv file listed baksetballvalue player id; if that's not possible then the name next to the ID in the bbv player files would be good; if that's not possible either please use the name listed on basketball reference

That would be great!

[bbstats]

looks like a valuable project - how do we go about projecting rookies for systems that did not project them at all?

Start at -4, -5? What's the average rookie APM?

[DSMok1]

looks like a valuable project - how do we go about projecting rookies for systems that did not project them at all?

Start at -4, -5? What's the average rookie APM?

The issue is that the worst players won't play: so you only get the top part of the distribution actually on the court. Once you get to the second round, more than half of the players won't get on the court, so only the top, say, 25th percentile, will get to actually play.

This is an issue with projection systems that actually project minor league/drafts. As Tango found out at the Book blog, just projecting rookies at league average is as good as projecting players based on their minor league stats--because only the ones that overperform/perform close to league average will actually get to play.

So, for instance, I projected the picks in the 30-40 range at about -5. Well, most were probably that bad; only 3 actually got any playing time at all. However, the one who got the most playing time was also the only one who played well--Landry Fields. And he played about league average. Well, I had given each player in that range about a 5% chance of being league average based on my standard deviations. In other words, my projection was pretty good.

But if you only compare players that actually played, then my projection of Landry Fields as -5.28 looks pretty bad.

Ideas on how to deal with this? Consider all rookies as a blanket value (between -1.5 and -2.5) for all projection systems?

[EvanZ]

I use a value of -1.0. I found that the was close to the mean for 2009-10 for ezPM for rookies > 500 possessions.

What we shouldn't do it bias our ratings in any way that reflects knowledge of the current season. (goes without saying, I guess)

[DSMok1]

I use a value of -1.0. I found that the was close to the mean for 2009-10 for ezPM for rookies > 500 possessions.

What we shouldn't do it bias our ratings in any way that reflects knowledge of the current season. (goes without saying, I guess)

Yeah, I'm just pointing out that selective sampling bias will cause a better projection to look like a worse one when dealing with draft picks, if we just look at who played and weight by how much. It's a well-known phenomenon.

[EvanZ]

I guess I'm ok with us all agreeing on a fixed value, whatever it is. Rookie projections could potentially be the subject of a future "challenge".