book recommendation - Superforecasting by Philip Tetlock

[ampersand5]

While the book isn't about basketball analytics, I believe it's insights are directly applicable to many of the challenges we grapple with.

Among other things, the book outlines an approach for thinking about novel problems and conducting more rigorous analysis.

I highly recommend it and think that the average apbrer will find it's lessons immensely valuable, and interesting through out.


http://www.amazon.com/Superforecasting- ... 0804136696

[Crow]

I'll check it out. And some related ones.

Identifying super-forecasters specifically in the NBA world and their skills and habits would be a worthwhile endeavor for teams, media, outside observers.

It probably would be worthwhile for teams to analyze their decision processes- self analysis and outside professional analysis. I doubt many organizations are as self-aware as they should be. Or as solid.


In the win prediction sphere there are probably some that qualify or might, though the number of tests is low.

[Crow]

Here is what I got from looking at the book so far in terms of application to the NBA (in no particular order of importance):

Superforecasters are smart and hardworking but not necessarily the elite of the elite.
You may not need a PhD or elite MBA to be good at basketball analysis or at least analysis of analytic products.

Some "super forecasters" may be lucky anomalies. Even if the number of tests gets fairly big, there will be some supers that eventually fall, and vice versa.

Contests help discover talent. Though the Kings contest had unintended consequences and the owner made a push to hear what he wanted to hear, contests are a decent idea for finding talent, evaluating questions or even identifying questions, thought structures, language issues, etc. Sloan Conference has contests, APBRMETRICS has one on team wins, one on the draft. ESPN has / the playoff prediction test and there is the NCAA Kaggle. Perhaps Nylon Calculus, 538, the Vertical, the Ringer, etc. can identify and lead others.

In general, it is probably better to find a set of outcome probabilities instead of offering a single determination / best guess. Instead of a most similar player or an estimated draft outcome, a set of similars and probabilities of different outcomes. This is done in several public analytic products. I hope it is done in insider presentations.

Teams and diverse teams do better on average than individuals, especially if they work to listen, critique and collaborate. There needs to be realistic, safe means of differing, debating, even ending up in disagreement. Face to face time has heightened value.

Where possible divide forecast problems into distinct parts but try to aware of interactivity and in the end the final consideration should use the sum of the parts but not be completely enslaved by them alone in their separate spaces.

Identify and try to understand casual forces that precede or are bigger than the particular question.

The book advises to balance prudence with decisiveness. Ok, but there may be situations that call for differing amounts. Until you get real close to a title or contending, should decisiveness get an edge when 30 teams try for the same thing? Can prudence get you the full way? Probably not very often.

Engage in careful, detailed belief review and updating.

"Go to different schools" was a thought offered by General Pietrus. NBA insiders go to Sloan, the informal environment of Vegas summer league. Radio and TV. They also can go to "school" with past players (identifying with that particularly team or not) current players, season ticket holders, the blogs, APBRmetrics, etc. There may be a difference between casual listening and really asking some "outside" group for extensive input. The book advises to have a good balance of insider / outsider input to consider. How many teams have a good balance of this? Mostly I hear the outside disparaged, roundly and almost completely. But maybe insiders don't completely live by this bluster.

[Crow]

Although I am sure teams try to pay attention to what other teams are trying to do based on direct, indirect and public Intel, I wonder if they could do better if they systematically studied the trade history, the trade rumor record, etc. I wonder if teams could do a better job of jumping on opportunities that other teams develop first or could even find other teams to help to thwart their biggest competitors in cases where they don't want the deal or can't make it work. It would be really meta to divert a deal with the hope or plan to get that asset for yourself at a later time. Perhaps by enticing a safe team to taken a guy who looks right for them but isn't? Anybody going that far? A form of layaway or shopping "assistance" with hand me down intent?

[ampersand5]

Thanks for your review Crow.

From the small amount of insider stuff I know, teams do study to some extent how other organizations work/value players/trade. However, with how few trades there are, and how quickly organizations change, there isn't that much to exploit.

[Statman]

Thanks for your review Crow.

From the small amount of insider stuff I know, teams do study to some extent how other organizations work/value players/trade. However, with how few trades there are, and how quickly organizations change, there isn't that much to exploit.

I told a team owner pretty much the same thing when he asked if I could surmise the analytics beyond different teams' decisions (the driving force on how they look at players)- particularly in drafting. I told him the sample size of "decisions" made for rosters being so small & turnover among the decision makers makes it tough. San Antonio is the outlier.

[roland_beech]

thanks for the recommendation ampersand...about 1/3 of the way through but enjoying it

[ampersand5]

Glad to hear.

----

If anyone else has any other books they recommend to this community, I would love to hear about them.