B-R.com's Elo Player Ratings

[Mike G]

http://www.basketball-reference.com/friv/ratings.cgi

Our player pool consists of all NBA players who meet at least one of the following career criteria:

10,000 points
5,000 rebounds
2,500 assists
1,000 steals plus blocks

The criteria for the player pool has these arbitrary cutoffs which include specialists like Dudley Bradley, with his 1182 steals plus blocks (and little else); while excluding some Hall of Famers like George Yardley and Frank Ramsey, because they didn't meet any one of these requirements. They were well-rounded players with mid-sized careers.

The voting system is interesting, but it doesn't discount votes which are obviously wack. Kobe Bryant ranks 84th, Shaq 19th. Some Kobe-hating types are apparently stuffing the ballot box. Presumably, the opposite effect also occurs.

It's irresistable to compare to my own statistically-based career rankings. I incorporate playoff numbers (with extra weight), and all stats are converted to "equivalent totals", scaled to team season averages. The single arbitrary cutoff is a career total of 10,000 equivalent points + rebounds + assists + steals + blocks.
(At b-r.com, while season and playoff totals and rates are both provided, there's no attempt to assimilate them.)

I've also included ABA totals, after discounting them by annual conversion factors. (At b-r.com, there's a suggestion that ABA sub-career might be considered, but it isn't part of the inclusion criteria.)
They have 535 players ranked, and I have 664.

I created a comparison number, defined as sqrt(Elo/MG). That is, the square root of the ratio of the Elo ranking and my ranking.

Code: Select all

overrated by Elo      MG   Elo      underrated by Elo       MG   Elo
.45  Pete Maravich   226    45      2.37  Kobe Bryant       15    84
.47  Larry Bird        9     2      2.18  Shaquille O'Neal   4    19
.50  Nate Archibald  233    59      2.09  George McGinnis   44   193
.52  Oscar Robertson  22     6      2.08  Wilt Chamberlain   3    13
.58  Joe Dumars      207    69      1.88  Alvan Adams       78   275
.60  Dennis Rodman   182    66      1.82  Larry Foust      108   357
.61  Michael Redd    374   139      1.78  Neil Johnston     70   222
.61  Magic Johnson     8     3      1.76  Zelmo Beaty      110   340
.62  Chris Mullin    160    61      1.73  Karl Malone        5    15
.62  Chris Paul      102    39      1.64  Cliff Hagan       75   201
.63  David Thompson  236    95      1.62  Dan Issel         46   120
.65  Bill Walton     119    51      1.56  Dolph Schayes     30    73
.66  Earl Monroe     179    79      1.53  Tony Parker       87   204
.67  Mitch Richmond  178    80      1.51  Clyde Lovellette 104   236
.67  Willis Reed     122    55      1.47  Maurice Lucas     91   196
.67  Shane Battier   550   248      1.46  Bill Bridges     138   294
.67  Deron Williams  188    85      1.45  Carlos Boozer     86   182
.67  Michael Cage    623   282      1.44  Vlade Divac       69   144
.67  Rajon Rondo     192    87      1.44  Sam Cassell       83   173
.67  Glen Rice       214    97      1.41  Kareem AbdulJabbar 2     4

A lot of my 664 top careers are not available in Elo voting, for reasons stated above. Prominently:

Code: Select all

Elo unranked     MG     Elo unranked     MG
Mel Daniels     141    Phil Chenier     275
Billy Paultz    159    Jimmy Jones      291
George Yardley  165    Jeff Ruland      303
Connie Hawkins  186    Tom Owens        306
Larry Kenon     196    Reggie Lewis     311
Arvydas Sabonis 210    Josh Howard      314
Willie Wise     228    Jameer Nelson    319
Roger Brown     232  Sweetwater Clifton 320
Ron Boone       234    Drew Gooden      324
Tayshaun Prince 240    Louie Dampier    328
Kevin Durant    245    Nene Hilario     337
Mehmet Okur     264    Bobby Wanzer     341
Frank Ramsey    265    Mack Calvin      343
David West      273    Luol Deng        350

I don't think any players ranked higher than #400 in Elo are unranked in my list.
A lot of these were ABA for much or most of their careers. Others are or were just good players.

Now some players ranked almost identically (.99 to 1.01 in the sqrt ratio defined above) --

Code: Select all

career rank       MG  Elo    career rank       MG  Elo
Michael Jordan     1    1   Chuck Person      305  296
Gary Payton       43   44   James Edwards     307  307
Vince Carter      48   49   John Lucas        325  318
Brad Daugherty   121  123   Johnny Green      334  329
Walter Davis     127  130   Kurt Thomas       339  339
Spencer Haywood  140  142   Joe Smith         346  343
Antonio McDyess  167  167   Carl Braun        364  354
Lou Hudson       189  190   Rick Fox          373  384
Jack Twyman      195  197   Darrell Armstrong 377  378
Sidney Wicks     239  245   Tyrone Corbin     378  371
Hot Rod Williams 248  247   Kerry Kittles     392  400
Jeff Mullins     253  260   Grant Long        393  383
Dale Davis       257  258   Vern Fleming      397  407
Vin Baker        262  263   Benoit Benjamin   403  398
PJ Brown         269  269   David Greenwood   406  401
Cliff Robinson   271  272   Craig Ehlo        416  417
Archie Clark     280  281   Aaron McKie       420  426
Calvin Natt      285  278   Sedale Threatt    433  437
Hedo Turkoglu    286  287   Antonio Daniels   443  450
Sleepy Floyd     292  290   Sherman Douglas   475  461
Tom Gugliotta    298  297   Mike Gale         525  513

About 8.4% of the common sample are within 1% in ranking.
Some 44% are between .90 and 1.10

[Mike G]

When 14,000 people have voted for each player, can't we suppose there's not so much random bias?
I've thought most matchups were between similarly-ranked players. So you don't often (ever?) vote between Garnett and Manute. KG gets 55% of KG vs Duncan, KG vs Webber, KG vs Pettit, etc. No?

I think it's far from the worst such system I've seen. When you vote, you're presented with every stat b-r.com offers: totals, per game, per 36, advanced; for seasons and for playoffs.

The "overrated" list above (relative to mine) is heavy on smaller players, Hall of Famers from early eras, etc. So it tends to propagate earlier tendencies to overrate.
Overlooked players continue to be overlooked, but perhaps less so. The "underrated" side tends to be bigs.

[Crow]

The Elo ratings are an interesting experiment and it has gotten a lot of participation but I don't sure how strongly I respect the ratings or the ratings that go contrary to objective metrics in particular. It comes down to the quality of the voting and the voters.

535 players in the set. I caught myself and now recall that the player match-ups are held between players within 250 points of each other so I needed to revise my post accordingly. So after a bit Shaq's matchups are essentially amongst the top 50 and he wins just 58% of the match-ups and ends ups 19th on the Elo Ranking. Kobe after a bit sees his matchups against players ranked 40th to 140th and he only wins 57% of the time? Shaq seems too low and Kobe way too low, though he might not deserve the top 10 or 5 that his big fans would call for.

Shift 1 vote in 10 or 20 and the rankings would be far different. How much do you want to rely on a small plurality of voters? How many of them actually saw both players in a match-up play a full game or more than a few? Probably many votes where the voter had seen little or none of at least one player. How many stopped and considered the stats carefully? How many voted mainly based on players they had seen / liked the most pr disliked the most? How many tried to be fair?

Looking at the sum of a lot of criteria. Shaq is probably a top 10 player, IMO. By WS/48 he is not; he is 16th there. Kobe 31st on that metric. Maybe that played some into these rankings but WS/48 is not the consistent guide of the average Elo voter. Wilt was 3rd best on it, yet ranked 13th on Elo voting. Issel 35th on WS/48, 120 on Elo.

The Elo may be reasonable close to objective measures or my rankings in most cases but I think your data highlights a fair number of cases of significant over and under ratings. At least those 10% of cases.

I see your comparison numbers for the greatest variances but also wonder what is the overall average ranking variance between the Elo rating and your rating? The overall average variance might be over 40 spots. That might be considered pretty close, I guess. But what simply is the correlation level?

Looking at a few comparison numbers and noting what you said about 44% of comparison numbers being within .9 to 1.1, I get the impression that ranks between Elo and your rating vary by 30% or more in over half the cases. That could mean 10 spots for those cases in the very top, 30-70 in the middle and 100 or more spots at the bottom. Those cases are not that close and I think I would probably generally side with a decent objective measure.

[Mike G]

The correlation is .545
Average difference is 66 places, though this is confounded by the selection criteria.
Considering only the 502 players in both my rankings and Elo's ranking, the correlation is .897
The average absolute difference is 48 places. Again, this is skewed by Elo's casual referring to the possibility of ABA achievements, without attempting to make NBA equivalents.

Due partly to the ABA uncertainty, I find the 50 biggest 'over-rated by Elo' players' average career span running from 1986 to 1997.
Their 50 most 'underrated' go 1978 to 1990; and this includes the Kobe and Shaq anomalies. So, either I rank the old-timers too highly, or the b-r.com voters are overly fond of recent players (with exceptions).

[Crow]

Thanks for those numbers as well as the original post and analysis.

[Mike G]

Shaq is probably a top 10 player, IMO. By WS/48 he is not; he is 16th there.

Of the top 100 WinShares in regular seasons, he's 15th in ws/48. Through 2006, he was 8th.
A few of those with higher career ws/48 have not finished their careers: LeBron, Dirk, Duncan.

He's 8th in WS, in RS. Also 5th in playoff WS --after Jordan, Kareem, Magic, and Wilt.
Playoff WS don't give any extra credit for titles, and Shaq's 4 don't allow any lesser players to leapfrog him, in my book. His RS WS are better than Magic's by 182-156, and better than Bird's 146.

Bird and Magic are apparently "darlings" of some historic players devotees. As are Maravich, Monroe, etc.
Bird was arguably or probably the best player in the world for 2-3 years. Magic, arguably the same. Kareem, for some 12 years.
http://bkref.com/tiny/y9cPA

[Mike G]

I thought I'd looked at this earlier, and in fact I retained it in a file. In March of this year, I recorded the Elo rankings; so we can see how players' ranks have changed in 8 months.
Lacking a better term, I'm calling them "hot" and "not hot" in the interval. As a straight ratio of their rankings then/now.

Code: Select all

M/N   Mar   Nov    not hot            M/N   Mar   Nov      hot 
.15     2    13   Wilt Chamberlain   2.67     8     3   Magic Johnson
.23    19    84   Kobe Bryant        2.43    17     7   David Robinson
.37     7    19   Shaquille O'Neal   2.00    10     5   Hakeem Olajuwon
.45     9    20   Jerry West         1.86    39    21   Isiah Thomas
.48    97   204   Tony Parker        1.81   163    90   Chris Bosh
.58    21    36   Bob Pettit         1.75    14     8   Julius Erving
.61   148   243   Richie Guerin      1.68   205   122   Allan Houston
.61    27    44   Gary Payton        1.67    45    27   Bob Cousy
.65    46    71   Bob McAdoo         1.61   213   132   Zach Randolph
.66   143   216   Derrick Coleman    1.60   222   139   Michael Redd
.66   132   199   Lafayette Lever    1.58    98    62   Anfernee Hardaway
.67     6     9   Bill Russell       1.51   264   175   John Starks
.67   134   200   Jerry Sloan        1.50     3     2   Larry Bird
.68   151   222   Neil Johnston      1.48   173   117   Baron Davis
.68   243   357   Larry Foust        1.47    69    47   Tracy McGrady
.70   126   180   Paul Westphal      1.46   167   114   Peja Stojakovic
.70    83   118   Manu Ginobili      1.46    41    28   Dominique Wilkins
.70   114   162   Eddie Jones        1.46    67    46   Grant Hill
.71   122   173   Sam Cassell        1.45   189   130   Walter Davis
.71   129   182   Carlos Boozer      1.45    71    49   Vince Carter

Of course, some games were played between then and now; but not many current players have gained much appreciation: Bosh and Zach. Peja. Some who weren't even in the playoffs.
On the left, Laker haters are out in force; some anti-Spurs, anti-Boozer, and maybe just dissing of old-timers in general.

Of 497 players who were ranked in elo and on my own list, average difference was 45 in March and now 48 in Nov.
The 100 most-overrated from March averaged 1.68 (M/elo) then and 1.65 now.
West (1.89 to .85) and Eddie Jones (1.32 to .93) slumped from over- to under-rated.

The 100 most underrated averaged (M/elo) .65 then and .67 now.
From under- to over-rating are Olajuwon (.70 to 1.40), Cousy (.78 to 1.30), Zach (.77 to 1.23)

[Crow]

It would help the analysis of the project to know what the original ratings were, found in 100,000 match simulations. Neil and Justin have that information but it is not shown at B-R to my knowledge. No longer a BR blog but there is this thread if they care to share the information.

So far the earliest data I've found is from late afternoon of the first day February 8th. Jordan was over 3100 and there were 14 over 2800. http://forums.realgm.com/boards/viewtop ... &t=1090581
I assume the original ratings were fairly close to these levels at the top (but maybe not due to initial low sample size volatility). Today not a single player is above 2500. The overall trend has been pulling the top players down from they very early ratings and probably from there original ratings.

In the chess application of Elo Ratings were players given initial ratings? if not, then this is a significant difference.

4 Lakers lead your list for biggest relative declines. Magic Johnson leads the list for March to November improvements.

Shaq was rated over 2900 on the first day and in 8th place. 2181 now or about 25% lower.

Where will the ratings go in another year? Will the share of future voters or "informed" or "fair-minded" voters increase or decrease relative to "partisan warriors" or "less-informed" voters? Will the greats of the game continue to see their rating go down and move back to the pack or will voters eventually vote their greatness back up some?

Would the ratings look much different today if not influenced by the original ratings? Would the stars have been pulled down even further & faster? I wouldn't have expected basically all the top stars getting pulled down over time. Jordan down by more than 600 compared to where he was the afternoon of the first day. Bird down nearly 500. Magic down nearly 600. Based on your March to November data the relative rankings have changed between top players over time but in general the ratings have all fallen (regressed towards the mean).

I wonder if for lots of voters whether they liked a player played a significant role in their voting and that a lot of voters didn't like or just didn't think as highly of a lot of the stars of other teams besides their favorite team as the original ratings did. There are vastly more voters from other teams than the home team of each star.

The movement was fast. 5 days into the project and Jordan had fallen from over 3100 to 2766. Went from 14 guys over 2800 to none in 5 days. Only 20 were above 2500 at that point but none now.

[Mike G]

I assumed all players were created equal, and that at first Jordan might be opposed to Dudley Bradley or anyone else, equally likely. Against all 500+ players, he'll win 90%+.
Players then quickly approach a realistic ranking based on their Win%, and so everyone tends toward 50%.

Note, too, Kobe's W% is .572, which is much higher than the .51 to .53 of those around his rank. Apparently he loses to opponents he should never lose to; because some voters are just out to get him.

So I guess the next step in the evolution of this thing would be to rank the voters and weigh their votes accordingly. Based on how normal your votes are.

Interesting that Magic is exempt from the anti-Lakers voter(s)' scourge.

[Crow]

I understand the win %s tending towards 50% in the public rating.


If as stated all player ratings started at 1500 how were the very first of 100,000 iteration "simulations" decided? It can't be random but what was used? The explanation http://www.basketball-reference.com/about/elo.html
does not provide a clear answer (to me). I wonder if WinShares was used but can't draw a strong conclusion based on just the ratings after the public voting had started.

[huevonkiller]

Just wanted to say that this is a fascinating discussion.

I still find the concept of the Elo player rating to be naive, the public knows nothing about defense or advanced metrics.