APBR Retrodiction Challenge for Summer 2011

[J.E.]

I took out the name column. How does this look?

https://spreadsheets0.google.com/spread ... output=csv

Notepad just didn't show the newlines..

Joe Sill's suggestion 0.5770
standard per possession RMSE 0.5959
mean of absolute error ED 3.0042
RMSE ED 3.761

[EvanZ]

Cool. I don't really have a feel for those numbers, other than they are worse than yours and Daniel's.

[J.E.]

Cool. I don't really have a feel for those numbers, other than they are worse than yours and Daniel's.

We probably should, and easily can I suppose, run this for prior seasons too. Should give us a better feel for what's good and what's not.

I'll try using DSMok1's numbers as priors for 2010 RAPM and see if that improves results on 2011.

If we have player forecasts for more years we can think about combining the methods with optimized weights

[Crow]

I'll try using DSMok1's numbers as priors for 2010 RAPM and see if that improves results on 2011.

If we have player forecasts for more years we can think about combining the methods with optimized weights

I was hoping for some kind of ASPM / RAPM blend with ASPM as a prior or partial, simultaneous active ingredient similar to what Rhuidean is doing.

An eventual multi-season testing / search for optimal weights for a meta-metric would seem like a very useful part of this process. I was going to hold back a bit on suggesting it again but since you brought it up I wanted to support the thought.

[bbstats]

This feels like we're making the Holy Grail!!! Woop woop.

I don't think any of my numbers will come close to Daniel/Evan/Jerry's. I'll get the spreadsheets together when I have some time later.

[bbstats]

I just used 2009-2010 players' and their respective PIDs...is this the method we all are agreeing to?

https://docs.google.com/leaf?id=0B6P4OA ... x&hl=en_US

[J.E.]

I had an error in my code, sorry about that. 'N/A' players were given a -16 instead of a -1.6 (overall).

With it fixed we now stand at:
(listed in order of per possession error; mean absolute error of team efficiency differential; RMSE of team efficiency differential)

DSMok1
0.595633317484
2.35542365976
2.83917394141

eZpm
0.595963405279
3.00426567549
3.76164579072

bbstats
0.595739512623
2.75476800768
3.2670151292

'10 RAPM
0.595290810118
2.6939516858
3.27846312645

3year RAPM
0.594958663777
2.39686030525
3.2360034763

DSMok1 ratings win every category except for per possession error.

I won't be at home this weekend, so I cannot evaluate new submissions until Monday at least.

[bbstats]

Are the Mean Error & RMSE values just based on per-game results?

[EvanZ]

Every 1 unit of MAD is about 2.5 wins.

[Crow]

Jerry would you be willing to post .csv files of the RAPM versions used for this thread with the player ID field? That way I can line up all these metrics and perhaps analyze them a bit.

[bbstats]

Also - are we using 2009-2010 PID sheet or 2010-2011?? I used the former...

[EvanZ]

The ID's don't change from year to year, but rookies are assigned new ID's.

[Mike G]

I would strongly suggest that you use actual 2010-11 results for rookies, otherwise the contest will depend heavily on who has the best rookie predictions, which I don't think is the intent here.

I had exactly the same thought. That's another project entirely, predicting rookie rates.

How can you predict team efficiencies without using individual player rates? Players moved around between '10 and '11.

[Crow]

This is a weighted blend of several of the metrics entered so far and lambaPM. Since it is preliminary I won't disclose the weights yet.

https://spreadsheets.google.com/spreads ... utput=html

Can this be scored? If there are any technical issues let me know what needs to be changed about the file.




Evan, would you consider preparing iterations of EZPM where 1) the assist weight and assisted basket deduction were say 1/3rd less than what they are now, 2) where shot defense was 1/2 to 2/3rds "counterpart" and 1/3rd to 1/2 divided equally to the players on the court and 3) with both changes?

If you were to test-change any EZPM weight(s) what would you pick as the target(s) and as the tuning change(s)?

I am curious what affect such changes would have on the retrodiction results.

[EvanZ]

Evan, would you consider preparing iterations of EZPM where 1) the assist weight and assisted basket deduction were say 1/3rd less than what they are now, 2) where shot defense was 1/2 to 2/3rds "counterpart" and 1/3rd to 1/2 divided equally to the players on the court and 3) with both changes?

If you were to test-change any EZPM weight(s) what would you pick as the target(s) and as the tuning change(s)?

I am curious what affect such changes would have on the retrodiction results.

I'm curious, too. I think the best thing to do is change the weights so that it maximizes the correlation with some form of APM, probably 1-yr RAPM, as I did once before to find the optimal assist rate. I'm going to wait (no pun intended) to see what results Rhuidean obtains by using the PBP data that I gave him, and see whether it makes a material difference.