Diversity vs. Quality

[DSMok1]

I recently came across a very interesting article entitled:

Groups of diverse problem solvers can outperform groups of high-ability problem solvers
Source: http://www.pnas.org/content/101/46/16385.long#T1

I recommend reading the whole article, but I will reproduce the first part of the conclusion:

The main result of this paper provides conditions under which, in the limit, a random group of intelligent problem solvers will outperform a group of the best problem solvers. Our result provides insights into the trade-off between diversity and ability. An ideal group would contain high-ability problem solvers who are diverse. But, as we see in the proof of the result, as the pool of problem solvers grows larger, the very best problem solvers must become similar. In the limit, the highest-ability problem solvers cannot be diverse. The result also relies on the size of the random group becoming large. If not, the individual members of the random group may still have substantial overlap in their local optima and not perform well. At the same time, the group size cannot be so large as to prevent the group of the best problem solvers from becoming similar. This effect can also be seen by comparing Table 1. As the group size becomes larger, the group of the best problem solvers becomes more diverse and, not surprisingly, the group performs relatively better.

A further implication of our result is that, in a problem-solving context, a person's value depends on her ability to improve the collective decision (8). A person's expected contribution is contextual, depending on the perspectives and heuristics of others who work on the problem. The diversity of an agent's problem-solving approach, as embedded in her perspective-heuristic pair, relative to the other problem solvers is an important predictor of her value and may be more relevant than her ability to solve the problem on her own. Thus, even if we were to accept the claim that IQ tests, Scholastic Aptitude Test scores, and college grades predict individual problem-solving ability, they may not be as important in determining a person's potential contribution as a problem solver as would be measures of how differently that person thinks.

This research appears to be directly applicable to team building in the NBA (and perhaps other cooperative team sports like hockey and soccer).

The more diverse the skillsets, the better the overall functioning of the team. Another way to state it is: Team Performance = Diversity + Average Ability.

This would illustrate why elite players with unique skillsets are so important to a team--this is why Dennis Rodman could be extremely valuable despite his obvious flaws.

Also addressed qualitatively is the question of communication between the team members; I'm not sure how this would apply to a basketball team.

Thoughts?

[steveshea]

Excellent post.

Chris Baker and I looked into this idea a bit in Chapter 8: Introducing Player Involvement Metrics of Basketball Analytics (2013). We introduced ways to measure the balance in a team's production. For example, does the team spread the assists or is one player largely responsible for playmaking (same for scoring, rebounding, etc.)? We found some preliminary evidence that being unbalanced (having one or 2 players carrying a large percentage of the production in a particular category) tended to be more successful in terms of team win%. In other words, elite specialists are good.

The analysis in that chapter was preliminary. I'd love to see someone take it to the next level.

I also believe that diversifying an analytics team and front office in general has a lot of benefits. I'm a mathematician, and I've enjoyed working on sports analytics problems with individuals coming from various backgrounds, including computer science, business, statistics, and playing or coaching experience. Each discipline is a fundamental mode of inquiry in its own right. Our backgrounds train us to approach the same problems in very different ways. The various approaches lead to different insights. Working together has lead to results that I believe were superior to what any proper subset of the group could have come up with.

[Crow]

A single flaw in the test could invalidate the conclusions. One still has to make leap of faith that the results in this specific construct will be similar to / predict the results in other tests / contexts or with the biggest leap to all situations.

[DSMok1]

A single flaw in the test could invalidate the conclusions. One still has to make leap of faith that the results in this specific construct will be similar to / predict the results in other tests / contexts or with the biggest leap to all situations.

Perhaps, but this does make a lot of sense logically. The "best" will converge on the single best approach. Skills will overlap, be similar, mindsets will be similar. A variety of very dissimilar parts will, as Steve says, have far more synergy.

[Crow]

I noticed recently that the Spurs and Thunder have recently added Quality Assurance officers of one grade or another. Haven't checked yet how common such a title / role player is with pros. Have seen it in college too.

For some things, "assuring quality" may be confidently be agreed upon with little fear of losing something of value via diversity or chance or whatever. But theoretically there may be things of value lost thru assurance of quality (or conscious display, incantation of "culture", values or design). Where is the basketball operation "diversity" / anti-similarity, anti-group think officer? I have occasionally expressed a suggestion about having an outside voice dialogue with or contest the internal actors and their values, processes and conclusions. Consultant or full-time inside outsider.

A few teams have been publicly identified now and in past of having an independent from GM, influential advisor to the team owner (in Portland, Denver past, Cleveland past, etc.). Several of these have been ascribed negative impacts. I guess some positive too like Jerry West. If you decide to authorize such a voice, where you place it is important and affects the process and impact. Need to evaluate whether the actual or potential positive impact is greater than the perceived negative impact (increased defensiveness, heightened efforts to limit or diminish diversity, heightened efforts to win favor or show immediate results, win allies, punish the perceived allies of others, etc.)

[Crow]

Perhaps, but this does make a lot of sense logically. The "best" will converge on the single best approach. Skills will overlap, be similar, mindsets will be similar. A variety of very dissimilar parts will, as Steve says, have far more synergy.

Does this logic enhance your interest in metric blends? Of diverse origin or maybe a set of BPM variants blended? Selected by applying diverse stat weights based on diverse methods or value assumptions or even random assignments (if n is high enough)?

Should meetings of basketball operations staff with season ticket holders (and best / diverse team bloggers / posters) be given more airspace and credence?

Should teams request downloads of all proposed trades on ESPN and / or Real GM and evaluate them for apparent impact using one or more metrics and do cluster and factor analysis to examine underlying patterns in thought and projected results? Should they pursue conversation and / or employment with the "best" or good and "diverse" voices?

[steveshea]

Perhaps, but this does make a lot of sense logically. The "best" will converge on the single best approach. Skills will overlap, be similar, mindsets will be similar. A variety of very dissimilar parts will, as Steve says, have far more synergy.

Does this logic enhance your interest in metric blends? Of diverse origin or maybe a set of BPM variants blended? Selected by applying diverse stat weights based on diverse methods or value assumptions or even random assignments (if n is high enough)?

Should meetings of basketball operations staff with season ticket holders (and best / diverse team bloggers / posters) be given more airspace and credence?

Should teams request downloads of all proposed trades on ESPN and / or Real GM and evaluate them for apparent impact using one or more metrics and do cluster and factor analysis to examine underlying patterns in thought and projected results? Should they pursue conversation and / or employment with the "best" or good and "diverse" voices?

It doesn't enhance my interest in metric blends, but does coincide with my interest in looking at multiple metrics (unblended).

I think basketball operations staff could benefit from regular outside consulting, and generally meeting with as many qualified minds as they have time for. In line with this thread, I like the idea of having a "different" outside perspective, finding someone who isn't of the same background as those making the decisions within the organization. I'm not sure if season ticket holders or team bloggers are the right group to consult with.

[Crow]

You could also try academic facility, best corporate management consultants, retired execs, coaches, players, etc. Worth an outlay of $100,000, $250k, $1mil or more?

The Vegas summer league networking (or Sloan) could take on heightened status / legitimacy.

Few insiders participate here more than lightly under a recognized handle, but could it be worthwhile to operate here covertly? Why not? At one's own scale, direction, whenever, whatever one wants input of quality / diversity. Straight ahead or somewhat disguised in focus. Though I guess mentioning this might increase risk of a past, current or future cover being blown and potentially exploited in one way or another.

[bchaikin]

Few insiders participate here more than lightly under a recognized handle, but could it be worthwhile to operate here covertly?

unfortunately trenchcoats and shades are not reimbursable company expenses...

[Crow]

The counter-espionage for dummies workbook is kinda expensive.

[xkonk]

In terms of front offices, the paper seems relevant. In regards to teams themselves, I'm not so sure. The key point is that in the limit, random beats out individual quality. If we make the leap from their model of decision making to playing basketball, we still have to think that five players is a large enough number that player skills can overlap enough to lose their benefit. Front offices could potentially be large enough that the math kicks in. And speaking of that leap, do we think that their team decision making seems anything like basketball?

I think one area where the model might break down is that there is no method for learning. When I read the article my thoughts immediately went to the Heat when they added Bosh and LeBron to Wade; they didn't win immediately because their skills overlapped enough to cause something of a downside. But they presumably learned or figured something out, because then they won two titles in a row. This is all predicated on the idea that you can judge a team for winning or losing a particular playoff series, of course.

[Crow]

Another way to twist the topic might be to look at diversity vs quality of contender and championship winning teams ON METRICS. How much does diversity of quality on metrics matter between winner and also-rans vs. overall or average quality? Does the winner tend to have a certain minutes weighted cumulative player score string differential sequence by metric over contender strings? And within player level strings for each metric?

Has anyone created a metric where all the stat weights are derived from just the title winners, say last 7 or last 20? Stat weights at team level or player level?

Perhaps there has been something at player level but not taken back to team level. It should be done and compared to the look at the whole league average-based stuff, which perhaps should be somewhat suspect if the purpose is building a title winner.


Do folks consider differential of cumulative team minutes weighted RPM over SRS adjusted point differential to be: error, real or a combination (somewhat knowable split or not)?