Win Probability Calculator Tool

[boooeee]

I have an NBA Win Probability Calculator tool up and running on my site. See here: NBA Win Probability.

This builds on the work I shared during the playoffs last season on win probability. I have tightened up the methodology a bit via cross-validation. I think there is still some tweaking I can do, but I'm feeling pretty good about the numbers.

I haven't announced this officially on my site yet, but thought I'd do a soft launch here where there are a lot of smart people that can tell me if I've messed this up.

The inputs are: quarter, time remaining, score difference, and possession. My prior version of the probability model included the Vegas point spread as well. For this tool, the probabilities are for evenly matched teams (I included the point spread as a model parameter though, so that should remove the selection bias one runs into when building win probability from raw numbers).

My dataset is all NBA games from the 2004 to 2011 seasons.

[Crow]

I see you have early playoff chances here as well:

http://www.inpredictable.com/p/ticker-nba0.html

[EvanZ]

Win probability calculator says Warriors had 29% chance of winning with 2 seconds left vs. Thunder down by 1 the other night. Hmm. Really?

[boooeee]

Win probability calculator says Warriors had 29% chance of winning with 2 seconds left vs. Thunder down by 1 the other night. Hmm. Really?

That's probably a weak spot in the methodology. Here is the raw data for a team down by 1 with possession:

Code: Select all

Time Left   Games   Won   Pct
0:05        39      11    28%
0:04        53      17    32%
0:03        47      9     19%
0:02        49      8     16%
0:01        48      6     13%

My approach is using LOESS, which tries to smooth everything. I tighten up the smoothing window as things get closer to the end of the game, but it looks like I might still be borrowing too many data points from nearby game times and margins.

[boooeee]

Win probability calculator says Warriors had 29% chance of winning with 2 seconds left vs. Thunder down by 1 the other night. Hmm. Really?

That's probably a weak spot in the methodology. Here is the raw data for a team down by 1 with possession:

Code: Select all

Time Left   Games   Won   Pct
0:05        39      11    28%
0:04        53      17    32%
0:03        47      9     19%
0:02        49      8     16%
0:01        48      6     13%

My approach is using LOESS, which tries to smooth everything. I tighten up the smoothing window as things get closer to the end of the game, but it looks like I might still be borrowing too many data points from nearby game times and margins.

I took a look at my code and realized I was pointing to the wrong model in endgame situations. The table has been corrected. The win probability for a team being down 1 with 2 seconds to go is 20.6%. Still seems a bit high but it is more in line with the historical data.

[EvanZ]

Ha, cool. Glad I found something useful.

[deepak]

At the start of the game, each team is given about a 50% chance of winning currently.

Two things you may want to incorporate into this is comparative team strength and also which team has home court.

[boooeee]

I see you have early playoff chances here as well:

http://www.inpredictable.com/p/ticker-nba0.html

Too soon?

One of the nice things about my rankings is that I don't need a very large sample size since they are based on the betting lines. I randomize the rankings a bit for future projections, but I do wonder whether they are still overconfident in the probabilities.

[boooeee]

At the start of the game, each team is given about a 50% chance of winning currently.

Two things you may want to incorporate into this is comparative team strength and also which team has home court.

The underlying model actually has this capability (it uses the Vegas point spread, which presumably factors in both team strength and home court). However, my focus for the win probability model on my site will be for modeling game decisions (e.g. when to foul) and player evaluation (e.g. win probability added). For those purposes, the evenly matched probabilities are most appropriate. Lebron James shouldn't be penalized in win probability added simply because the Heat started out as heavy favorites and there is less win probability to add.

[TheSpiceWeasel]

My approach is using LOESS, which tries to smooth everything. I tighten up the smoothing window as things get closer to the end of the game, but it looks like I might still be borrowing too many data points from nearby game times and margins.

I would argue that the underlying outcomes aren't themselves smooth. For instance, a 6 point difference versus a 7 point difference is different than a 3 point difference versus a 4 point difference. In both cases it's just 1 point, but in the second comparison it's the difference between 1 possession or two. I wonder if splines might be a better approach, assuming the knots are specified in a way that reflects reality.

One of the nice things about my rankings is that I don't need a very large sample size since they are based on the betting lines. I randomize the rankings a bit for future projections, but I do wonder whether they are still overconfident in the probabilities.

Phew! So glad you finally got around that pesky issue of sample size limitations. You know, that one that has confounded mathematicians everywhere for hundreds of years (except for Wayne Winston, of course).

[boooeee]

My approach is using LOESS, which tries to smooth everything. I tighten up the smoothing window as things get closer to the end of the game, but it looks like I might still be borrowing too many data points from nearby game times and margins.

I would argue that the underlying outcomes aren't themselves smooth. For instance, a 6 point difference versus a 7 point difference is different than a 3 point difference versus a 4 point difference. In both cases it's just 1 point, but in the second comparison it's the difference between 1 possession or two. I wonder if splines might be a better approach, assuming the knots are specified in a way that reflects reality.

Agreed. In theory, LOESS should be able to handle this as long as you make your bandwidth window tight enough, but it's hard to know when you've crossed the threshhold into overfitting. And it's particularly difficult when you've got multiple independent variables. I've been looking into splines and tree based methods, but haven't found a way out of this difficulty just yet.

One of the nice things about my rankings is that I don't need a very large sample size since they are based on the betting lines. I randomize the rankings a bit for future projections, but I do wonder whether they are still overconfident in the probabilities.

Phew! So glad you finally got around that pesky issue of sample size limitations. You know, that one that has confounded mathematicians everywhere for hundreds of years (except for Wayne Winston, of course).

Thanks. For my next trick, I plan on squaring the circle.

Sarcasm aside, small sample size truly is not much of an issue for my ranking system. The market has "pre-smoothed" things for me. All I'm doing is taking the pre-smoothed Vegas point spreads and reverse engineering an implied ranking.

[416x]

Do you have rankings up for this year? When I try to view them nothing comes up. Great work by the way!

[boooeee]

Do you have rankings up for this year? When I try to view them nothing comes up. Great work by the way!

Thanks. Can you give it a shot now? I'm experimenting with a few options.

My backend database for the rankings is a google spreadsheet, which doesn't always behave very nicely. I need to transition to a proper database framework, but haven't got around to it just yet.

[TheSpiceWeasel]

Phew! So glad you finally got around that pesky issue of sample size limitations. You know, that one that has confounded mathematicians everywhere for hundreds of years (except for Wayne Winston, of course).

Thanks. For my next trick, I plan on squaring the circle.

Sarcasm aside, small sample size truly is not much of an issue for my ranking system. The market has "pre-smoothed" things for me. All I'm doing is taking the pre-smoothed Vegas point spreads and reverse engineering an implied ranking.

I'm not sure I follow. So what if Vegas is assigning point spreads? Are you saying you base it on historical point spreads as well?

[EvanZ]

What he is trying to say (correct me if I'm wrong) is that Vegas has a lot less variation in their opinion/estimation of teams than actual game results have. If you believe that Vegas is difficult to beat (which I do), then estimating power ratings based on Vegas lines is about as good as you're going to get, and generally better (less noisy) than using game results, especially early in the season. I've done the same type of ratings myself in the past. There's just not that much day-to-day movement on Vegas ratings, so it's a nice system for regressing (i.e. you sample sizes can be much smaller).