Page 1 of 2
Win Probability Calculator Tool
Posted: Sat Nov 16, 2013 6:21 pm
by boooeee
I have an NBA Win Probability Calculator tool up and running on my site. See here:
NBA Win Probability.
This builds on the work I shared during the playoffs last season on win probability. I have tightened up the methodology a bit via cross-validation. I think there is still some tweaking I can do, but I'm feeling pretty good about the numbers.
I haven't announced this officially on my site yet, but thought I'd do a soft launch here where there are a lot of smart people that can tell me if I've messed this up.
The inputs are: quarter, time remaining, score difference, and possession. My prior version of the probability model included the Vegas point spread as well. For this tool, the probabilities are for evenly matched teams (I included the point spread as a model parameter though, so that should remove the selection bias one runs into when building win probability from raw numbers).
My dataset is all NBA games from the 2004 to 2011 seasons.
Re: Win Probability Calculator Tool
Posted: Sat Nov 16, 2013 8:48 pm
by Crow
I see you have early playoff chances here as well:
http://www.inpredictable.com/p/ticker-nba0.html
Re: Win Probability Calculator Tool
Posted: Sat Nov 16, 2013 10:44 pm
by EvanZ
Win probability calculator says Warriors had 29% chance of winning with 2 seconds left vs. Thunder down by 1 the other night. Hmm. Really?
Re: Win Probability Calculator Tool
Posted: Sun Nov 17, 2013 12:14 am
by boooeee
EvanZ wrote:Win probability calculator says Warriors had 29% chance of winning with 2 seconds left vs. Thunder down by 1 the other night. Hmm. Really?
That's probably a weak spot in the methodology. Here is the raw data for a team down by 1 with possession:
Code: Select all
Time Left Games Won Pct
0:05 39 11 28%
0:04 53 17 32%
0:03 47 9 19%
0:02 49 8 16%
0:01 48 6 13%
My approach is using LOESS, which tries to smooth everything. I tighten up the smoothing window as things get closer to the end of the game, but it looks like I might still be borrowing too many data points from nearby game times and margins.
Re: Win Probability Calculator Tool
Posted: Sun Nov 17, 2013 4:49 am
by boooeee
boooeee wrote:EvanZ wrote:Win probability calculator says Warriors had 29% chance of winning with 2 seconds left vs. Thunder down by 1 the other night. Hmm. Really?
That's probably a weak spot in the methodology. Here is the raw data for a team down by 1 with possession:
Code: Select all
Time Left Games Won Pct
0:05 39 11 28%
0:04 53 17 32%
0:03 47 9 19%
0:02 49 8 16%
0:01 48 6 13%
My approach is using LOESS, which tries to smooth everything. I tighten up the smoothing window as things get closer to the end of the game, but it looks like I might still be borrowing too many data points from nearby game times and margins.
I took a look at my code and realized I was pointing to the wrong model in endgame situations. The table has been corrected. The win probability for a team being down 1 with 2 seconds to go is 20.6%. Still seems a bit high but it is more in line with the historical data.
Re: Win Probability Calculator Tool
Posted: Sun Nov 17, 2013 5:05 am
by EvanZ
Ha, cool. Glad I found something useful.

Re: Win Probability Calculator Tool
Posted: Sun Nov 17, 2013 2:24 pm
by deepak
At the start of the game, each team is given about a 50% chance of winning currently.
Two things you may want to incorporate into this is comparative team strength and also which team has home court.
Re: Win Probability Calculator Tool
Posted: Sun Nov 17, 2013 6:42 pm
by boooeee
Too soon?
One of the nice things about my rankings is that I don't need a very large sample size since they are based on the betting lines. I randomize the rankings a bit for future projections, but I do wonder whether they are still overconfident in the probabilities.
Re: Win Probability Calculator Tool
Posted: Mon Nov 18, 2013 5:09 pm
by boooeee
deepak wrote:At the start of the game, each team is given about a 50% chance of winning currently.
Two things you may want to incorporate into this is comparative team strength and also which team has home court.
The underlying model actually has this capability (it uses the Vegas point spread, which presumably factors in both team strength and home court). However, my focus for the win probability model on my site will be for modeling game decisions (e.g. when to foul) and player evaluation (e.g. win probability added). For those purposes, the evenly matched probabilities are most appropriate. Lebron James shouldn't be penalized in win probability added simply because the Heat started out as heavy favorites and there is less win probability to add.
Re: Win Probability Calculator Tool
Posted: Mon Nov 18, 2013 7:21 pm
by TheSpiceWeasel
boooeee wrote:My approach is using LOESS, which tries to smooth everything. I tighten up the smoothing window as things get closer to the end of the game, but it looks like I might still be borrowing too many data points from nearby game times and margins.
I would argue that the underlying outcomes aren't themselves smooth. For instance, a 6 point difference versus a 7 point difference is different than a 3 point difference versus a 4 point difference. In both cases it's just 1 point, but in the second comparison it's the difference between 1 possession or two. I wonder if splines might be a better approach, assuming the knots are specified in a way that reflects reality.
boooeee wrote:One of the nice things about my rankings is that I don't need a very large sample size since they are based on the betting lines. I randomize the rankings a bit for future projections, but I do wonder whether they are still overconfident in the probabilities.
Phew! So glad you finally got around that pesky issue of sample size limitations. You know, that one that has confounded mathematicians everywhere for hundreds of years (except for Wayne Winston, of course).
Re: Win Probability Calculator Tool
Posted: Mon Nov 18, 2013 8:18 pm
by boooeee
TheSpiceWeasel wrote:boooeee wrote:My approach is using LOESS, which tries to smooth everything. I tighten up the smoothing window as things get closer to the end of the game, but it looks like I might still be borrowing too many data points from nearby game times and margins.
I would argue that the underlying outcomes aren't themselves smooth. For instance, a 6 point difference versus a 7 point difference is different than a 3 point difference versus a 4 point difference. In both cases it's just 1 point, but in the second comparison it's the difference between 1 possession or two. I wonder if splines might be a better approach, assuming the knots are specified in a way that reflects reality.
Agreed. In theory, LOESS should be able to handle this as long as you make your bandwidth window tight enough, but it's hard to know when you've crossed the threshhold into overfitting. And it's particularly difficult when you've got multiple independent variables. I've been looking into splines and tree based methods, but haven't found a way out of this difficulty just yet.
boooeee wrote:One of the nice things about my rankings is that I don't need a very large sample size since they are based on the betting lines. I randomize the rankings a bit for future projections, but I do wonder whether they are still overconfident in the probabilities.
Phew! So glad you finally got around that pesky issue of sample size limitations. You know, that one that has confounded mathematicians everywhere for hundreds of years (except for Wayne Winston, of course).
Thanks. For my next trick, I plan on squaring the circle.
Sarcasm aside, small sample size truly is not much of an issue for my ranking system. The market has "pre-smoothed" things for me. All I'm doing is taking the pre-smoothed Vegas point spreads and reverse engineering an implied ranking.
Re: Win Probability Calculator Tool
Posted: Mon Nov 18, 2013 10:08 pm
by 416x
Do you have rankings up for this year? When I try to view them nothing comes up. Great work by the way!
Re: Win Probability Calculator Tool
Posted: Mon Nov 18, 2013 11:38 pm
by boooeee
416x wrote:Do you have rankings up for this year? When I try to view them nothing comes up. Great work by the way!
Thanks. Can you give it a shot now? I'm experimenting with a few options.
My backend database for the rankings is a google spreadsheet, which doesn't always behave very nicely. I need to transition to a proper database framework, but haven't got around to it just yet.
Re: Win Probability Calculator Tool
Posted: Tue Nov 19, 2013 7:42 pm
by TheSpiceWeasel
boooeee wrote:TheSpiceWeasel wrote:Phew! So glad you finally got around that pesky issue of sample size limitations. You know, that one that has confounded mathematicians everywhere for hundreds of years (except for Wayne Winston, of course).
Thanks. For my next trick, I plan on squaring the circle.
Sarcasm aside, small sample size truly is not much of an issue for my ranking system. The market has "pre-smoothed" things for me. All I'm doing is taking the pre-smoothed Vegas point spreads and reverse engineering an implied ranking.
I'm not sure I follow. So what if Vegas is assigning point spreads? Are you saying you base it on historical point spreads as well?
Re: Win Probability Calculator Tool
Posted: Wed Nov 20, 2013 6:23 am
by EvanZ
What he is trying to say (correct me if I'm wrong) is that Vegas has a lot less variation in their opinion/estimation of teams than actual game results have. If you believe that Vegas is difficult to beat (which I do), then estimating power ratings based on Vegas lines is about as good as you're going to get, and generally better (less noisy) than using game results, especially early in the season. I've done the same type of ratings myself in the past. There's just not that much day-to-day movement on Vegas ratings, so it's a nice system for regressing (i.e. you sample sizes can be much smaller).