APBRmetrics

Posted at old forum:

knarsu3 Fri Oct 14, 2011 12:40 am

I was able to ask one of the creators of QBR about the clutch weights (thanks to Albert Larcada for explaining this to me) and this was his response to me:

Quote:
We looked at every play since the 2001 season and found the average WP change for an NFL play is somewhere around 2.4% (don’t quote me on that, but it’s somewhere around that number). So on the average NFL play the absolute value of WP is shifted by about 2.4%.

We then indexed those values to make 1 the average. So a situation where the average WP change is 2.4% we gave a leverage index of 1 (2.4%/2.4%). A situation where the average WP change is 1.2% we gave a leverage index of 0.5 (1.2%/2.4%). And so on. This gives situation where – in the past – the WP generally changes a lot more weight. And conversely if in the past the WP did not change much in a particular situation we give that play less weight.

We eliminated all outliers from our sample which generally were end of the game 25+% changes so as the smooth the model a bit. By removing those we created a decay process that maxes the leverage index around 3.

So the clutch weight is applied before the begins based on how of COULD this play affect the outcome of the game. We then use the EP on the play multiplied by the clutch-weight (leverage) index to find the clutch-weighted EPA on that play.


I hope this answers some of your questions. They've been very open about trying to answer questions (which is why I'm positive they won't mind me posting this) and I know Dean wrote a FAQ which addressed many of the questions regarding QBR. http://espn.go.com/nfl/story/_/id/69090 ... al-qbr-faq

EvanZ wrote: Well, I could see an argument that discounts "easy" touchdowns. Just like in basketball, garbage time stats are widely discounted because the opponent quality decreases and defenses tend to not exert 100% effort at that point.

But if you don't think it has anything to do with difficulty (and especially in that case), then why do you think win probability should be used at all? In that case, it makes no sense to me.

I think I'm confused by what you mean by difficulty. I get what you're saying about garbage time, but NFL teams rarely change players or strategy until well after the win probability would say the game is over. I'm not sure that NBA-style garbage time happens in the NFL, and NBA games are rarely over at halftime as they can be in the NFL.

As a personal aside, I think I've mentioned somewhere (at least obliquely) that I don't like using win probability as part of evaluating players. But I think it's useful for other things.

I think I understand now that it's the change in win probability created by the play that is used to give it value. I guess my question is why use change in win probability, which is heavily affected by the time left in the game, as opposed to just using change in "TD probability" created by a play? Can we not view each possession on its own, the objective of which is to score seven points? Why is there a need to tie the result of the play to anything more than that particular possession, the way we normally do it in basketball?

Expected points added is that measure; it doesn't care how much time there is or what the score is, just where you are on the field and the down and distance to go. I do prefer that measure in general. But it doesn't apply in some cases, like when a team is ahead late in the game and is just trying to run clock. You can run an unsuccessful 'scoring' play that is a successful 'winning' play. I think EPA is the better player measure overall, but both measures are useful in some circumstances. It turns out that ESPN's rating correlates at pretty similar levels with both EPA and WPA, which I find a little confusing; I'm sure that the mismatches are different for the two. But in general it's probably doing an ok job. And I do think it's probably better than the old QB rating.