FiveThirtyEight win/loss prediction accuracy

Home for all your discussion of basketball statistical analysis.
Post Reply
vraja
Posts: 2
Joined: Fri Dec 29, 2017 8:04 pm

FiveThirtyEight win/loss prediction accuracy

Post by vraja » Mon Jan 29, 2018 6:00 pm

I'm just curious what the current state of the art is for game win/loss prediction? I was looking at FiveThirtyEight's 2016-17 season game predictions, and found their accuracy to be around 65% for regular season games, which was oddly low since a simple model nets around 63%.

jgoldstein34
Posts: 240
Joined: Fri Sep 30, 2016 6:38 pm

Re: FiveThirtyEight win/loss prediction accuracy

Post by jgoldstein34 » Mon Jan 29, 2018 6:04 pm

What do you define as a simple model?

I get 71-73% depending on the year.

shadow
Posts: 220
Joined: Fri May 29, 2015 3:38 am

Re: FiveThirtyEight win/loss prediction accuracy

Post by shadow » Mon Jan 29, 2018 7:06 pm

I hate to be a cynic, but the closing Vegas line is only predicting the outright winner correctly 64.86% of the time so far this season according to Covers. If you can hit 71-73% for a full season, then you would likely have the greatest NBA prediction model on the planet and a license to print money. With a decent size starting bankroll you could make a few million in a few years.

Out of all the systems tracked at ThePredictionTracker, none of them are hitting better than 65% this year picking winners straight up in the NBA.

http://www.thepredictiontracker.com/nbaresults.php

A snapshot from 3/31/17 shows the best system was around 65% picking winners straight up at that point last season.

https://web.archive.org/web/20170331032 ... esults.php
Last edited by shadow on Mon Jan 29, 2018 9:59 pm, edited 1 time in total.

vraja
Posts: 2
Joined: Fri Dec 29, 2017 8:04 pm

Re: FiveThirtyEight win/loss prediction accuracy

Post by vraja » Mon Jan 29, 2018 9:55 pm

Thanks for the responses. Something like ThePredictionTracker is what I was looking for.

By simple model, I mean that I'm using very standard traditional metrics as well as some advanced metrics (all at the team level) passed into a regression model. I am very curious about your 71-73% performance. Any information you can disclose about your methods?

jgoldstein34
Posts: 240
Joined: Fri Sep 30, 2016 6:38 pm

Re: FiveThirtyEight win/loss prediction accuracy

Post by jgoldstein34 » Tue Jan 30, 2018 5:28 pm

I do print money haha yes. 73% 2 years ago, 70% last. 68% this year, but based off analysis I expect it to regress up to ~69-70%.

But no, not really much I can talk about because of NDAs and agreements with partners.

sbs
Posts: 17
Joined: Fri Oct 19, 2012 7:25 am

Re: FiveThirtyEight win/loss prediction accuracy

Post by sbs » Tue Jan 30, 2018 7:50 pm

jgoldstein34 wrote:I do print money haha yes. 73% 2 years ago, 70% last. 68% this year, but based off analysis I expect it to regress up to ~69-70%.

But no, not really much I can talk about because of NDAs and agreements with partners.
In-sample results? Using actual minutes/possessions for players?

eminence
Posts: 141
Joined: Sun Sep 10, 2017 8:20 pm

Re: FiveThirtyEight win/loss prediction accuracy

Post by eminence » Tue Jan 30, 2018 8:48 pm

jgoldstein34 wrote:I do print money haha yes. 73% 2 years ago, 70% last. 68% this year, but based off analysis I expect it to regress up to ~69-70%.

But no, not really much I can talk about because of NDAs and agreements with partners.
No question on methodology, just curious as to what betting service you'd recommend?

shadow
Posts: 220
Joined: Fri May 29, 2015 3:38 am

Re: FiveThirtyEight win/loss prediction accuracy

Post by shadow » Tue Jan 30, 2018 10:01 pm

I'm just a tad skeptical that someone who claims to have an edge on Vegas would be freely publishing their player ratings online. Seems that would be a clear violation of any NDA you sign with betting partners.

jgoldstein34
Posts: 240
Joined: Fri Sep 30, 2016 6:38 pm

Re: FiveThirtyEight win/loss prediction accuracy

Post by jgoldstein34 » Wed Jan 31, 2018 11:53 pm

Why would I make up something for strangers on the internet? lmao model isn't built around those.

tarrazu
Posts: 72
Joined: Mon Aug 04, 2014 5:02 pm

Re: FiveThirtyEight win/loss prediction accuracy

Post by tarrazu » Thu Feb 01, 2018 12:23 am

shadow wrote:I'm just a tad skeptical that someone who claims to have an edge on Vegas would be freely publishing their player ratings online. Seems that would be a clear violation of any NDA you sign with betting partners.
I think its a fair observation given the difficulties in beating a major sports betting market. I'm sure there is plenty of proprietary information involved with creating a model so successful, but it is possible to have a discussion without dismissing a perfectly reasonable observation made about the lofty win percentage. At some level you are incorporating the impact of a player as it relates to the outcome of a game, so once again, it seems fair to assume player ratings published and discussed on the the preeminent basketball analytics website in Nylon Calculus to possibly be involved in your process.

jgoldstein34
Posts: 240
Joined: Fri Sep 30, 2016 6:38 pm

Re: FiveThirtyEight win/loss prediction accuracy

Post by jgoldstein34 » Thu Feb 01, 2018 1:12 am

It's a totally fair observation, did not mean to come off as dismissive or otherwise. Was just trying to help answer the question for the high end. Apologies if that ruffled feathers.

jgoldstein34
Posts: 240
Joined: Fri Sep 30, 2016 6:38 pm

Re: FiveThirtyEight win/loss prediction accuracy

Post by jgoldstein34 » Thu Feb 01, 2018 1:36 am

Related, I do think Vegas has improved their line making over the past few years. Has anyone read anything on them adapting new techniques?

shadow
Posts: 220
Joined: Fri May 29, 2015 3:38 am

Re: FiveThirtyEight win/loss prediction accuracy

Post by shadow » Thu Feb 01, 2018 4:14 am

If we're going to be technical, sportsbooks in Vegas don't originate lines for NBA games. They wait for Bookmaker or Pinnacle to post an opening line and then will often move on air to keep their number in line with those sites since they best represent the market.

The data from ThePredictionTracker indicates that it's the opposite if we're talking about how accurate the opening lines are from oddsmakers. This season is on track to be the second worst season in terms of Absolute Error (AE) and Mean Square Error (MSE) for the opening line. The only season that was worse was last season. Here are the full results:

Code: Select all

Year   AE	  MSE
2018	9.70	157.49
2017	10.05  164.35
2016	9.29	140.09
2015	9.32	142.63
2014	9.25	138.61
2013	9.23	140.99
2012	9.29	140.35
2011	8.84	151.57
2010	9.41	143.37
2009	9.09	138.55
2008	9.34	137.47
2007	9.16	133.84

Post Reply