APBRmetrics

It works both ways - the previous week when the Ravens blasted the Rams, Ngata got a defensive touchdown for the Ravens with 5 minutes left. He wouldn't get much credit because being up 30 doesn't help much more than being up 23. Similarly, Flacco's number probably didn't change much even though he (also) got sacked and fumbled in the red zone because they were already up 23 and victory is more or less in the bag.

I don't know how big a part it is, but part of the ESPN rating is the leverage index. If a team is winning or losing by a lot, the outcome of a play isn't going to change the game's outcome. But if the game is close, the outcome of a play can have a big impact on who wins and loses. It isn't so much about defenses letting up or offenses getting cheap yards as that if the game is in the bag, who cares?

I think it's also important to note that the ESPN rating, as well as the win probability numbers that it's based on, are what Brian Burke calls a story telling stat. It tells you what happened. In this case, Smith apparently didn't do much at all to help his team win. It isn't necessarily meant to be predictive, and in fact two analyses I can think of have shown that win probability is less predictive than other similar measures of QB performance.

If a team is winning or losing by a lot, the outcome of a play isn't going to change the game's outcome...

If you're down 27-3 in the 3rd quarter, and you score a touchdown, does that about triple your chance of winning the game, maybe from 1% to 3% ?
If that is the first of 4 TD's en route to an improbable 34-30 victory, does any of those 4 TD's weigh more heavily than any other? Aren't they all equally valuable?

Is it only in retrospect that the first TD has value?

It depends on when in the third exactly and where you assume the teams were on the field before and after the score. But that would be a plausible move, from 1% to 3%. You can play with numbers here: http://wp.advancednflstats.com/winprobcalc1.php

The question quickly becomes what you mean by 'value'. From the perspective of win probability, the first touchdown isn't worth much, even in retrospect. Going from 30-6 to 30-13, even if it'll later end up being 30-34, isn't worth much because 97% of the time (or some other very high number) the comeback never happens and you still lose. Logically you have to get the first touchdown in order to get four touchdowns, but keep in mind that the vast majority of the time, teams don't come back from that far down. Any kind of 'value' based on win probability is going to be context dependent in regards to the score.

If you prefer your value to be less score-dependent, you could use expected points added (EPA) http://www.advancednflstats.com/2010/01 ... oints.html . That doesn't care about the score or leverage, just if you improved your team's chances of scoring points. The downside to EPA is that you wouldn't want to use it to analyze all decisions; late in the game you want to maximize your chances of winning, not your chances of scoring points. You would want to use both types of value in evaluating decisions, even if it's using EPA for one decision and WPA for another. Going back to Alex Smith, over the past couple years the best quarterbacks in the league average around 10 EPA per game, so his 3 EPA against the Eagles was maybe a bit above average. I think any way you slice it, it's hard to argue he played as well as his traditional quarterback rating would lead you to believe.

xkonk wrote:
I think it's also important to note that the ESPN rating, as well as the win probability numbers that it's based on, are what Brian Burke calls a story telling stat. It tells you what happened.

I don't think it's telling the story correctly either. Alex Smith was a major part of that comeback. He certainly wasn't the 6th worst QB this week.

xkonk wrote:Yeah, no idea. But since Smith's last really positive contribution (the TD pass to make it 23-17) still only had the 49ers at a 25% chance of winning, it probably didn't get him a lot of credit. After that, Smith had 3 or 4 incompletions and took a sack on third down versus 3 completions for all of 22 yards; the Eagles lost by missing two field goals, penalties, and giving up big runs. I'm inclined to go with ESPN's (or Brian Burke's) interpretation on this one.

Evan, those are the stats that Smith put together. His touchdown pass still left the 49ers in a tough spot, and then he had a 50% completion percentage and under 4 yards per attempt the rest of the way. He didn't even complete a pass for a first down in the whole fourth quarter. In the first half Smith was something like 8 of 18 for about 80 yards; he took a sack on third down that left the 49ers to kick a field goal, and he took a sack-fumble in his own territory that lead to a Philly touchdown 5 plays later. Arguably, anything he did in the third quarter was making up for how bad he was in the first half. The 49ers only had a 10% chance of winning when they came out for the second half.

So here's a rough description of what happened: Smith was pretty bad in the first half and the 49ers ended up with a 10% chance of winning. Smith was good and it only got them up to about 25%. Smith was bad while the 49ers managed to win anyway. Even if we agree that he wasn't the 6th worst QB that day, it sure seems like he wasn't as good as his QB rating suggests, right?

Honestly, I really just want to see the methodology, and short of that, no amount of hand waiving is going to satisfy me. (Just trying to save you further effort, though I do appreciate the attempt.)

Agreed, seeing all the ins and outs would be great. And I more or less plan on ignoring ESPN's rating anyway because I prefer to know how players are going to do rather than what they've done, and measures like EPA are better at that than WPA (and WPA is the backbone of the ESPN rating). But either one of those measures correlates fairly well with ESPN's numbers, and the details of that are completely available on advancednflstats.com.

xkonk wrote:... From the perspective of win probability, the first touchdown isn't worth much, even in retrospect. Going from 30-6 to 30-13, even if it'll later end up being 30-34, isn't worth much because 97% of the time (or some other very high number) the comeback never happens and you still lose...

A game may go down to a last-second 40-yard FG attempt, by a team trailing by 2. Say the odds are 50-50.
In that last second, the odds of winning rise or fall 50% -- to either 100% or zero.

Is that last play equal in value to all the previous plays of the game combined?

Mike G wrote:A game may go down to a last-second 40-yard FG attempt, by a team trailing by 2. Say the odds are 50-50.
In that last second, the odds of winning rise or fall 50% -- to either 100% or zero.

Is that last play equal in value to all the previous plays of the game combined?

You'd have to tell me what you mean by value, but either way the answer is not necessarily. In terms of expected points, the field goal is only worth at most three points. It would be a rare and boring game where the expected points before that were less than three.

In terms of win probability added, I guess it could be true if neither team ever had a big lead and so the win probability stayed close to 50% the whole way. But that's pretty unusual. I tried to find a recent game that was similar enough to what you described, and I think Giants-Cardinals from week 4 should do. http://live.advancednflstats.com/weekly ... 221&week=4 . The Giants were down by three with 3:10 left and at Arizona's 29. At that point, they were only 42% likely to win. But on the next play they threw a touchdown pass to take a 4 point lead and jumped to 74% to win. We could even say, for the sake of argument, that the play happened at the very end of the game and so was worth 58%. If you look at the chart in the link though, you'll see that the win probability has moved before that. The Cardinals had already earned 22 points halfway through the first quarter. Then the Giants got some back, the Cardinals made progress, the Giants actually became the favorites at halftime, etc. The movement before that play adds up to much more than .5.

At the level of the team, arguably that field goal could be worth more than everything that happened before it because the teams earned and lost win probability to end up at 50% at that point. So the teams effectively had done nothing to that point; the game could have just started then. But at the player level that doesn't have to be true at all. A quarterback could do nothing but good things and gain lots of win probability while other players let him down and the team ends up in the position you describe. He could have a positive win probability that was largely canceled out by his teammates. Case in point, Drew Brees earned nearly a full win (.95) against the Panthers last week but only won by 3. If you were to track kickers he would gain .5 on that play, but to my knowledge Brian doesn't track advanced stats for kickers; I think he figures them out when he has a specific question about kick-offs or punts.

Mike G wrote:In that last second, the odds of winning rise or fall 50% -- to either 100% or zero.

Is that last play equal in value to all the previous plays of the game combined?

Sure. Imagine that you made a bet before the game at 50-50 odds, and can keep on betting throughout the game.

Before the game started, you plunked down $10 or whatever and had a 50% chance of winning.

Midway through the game, your team is behind 42-0 and past statistics, as well as common sense, tell you that your bet is now worth less than $0.01. Your team's crummy play during the first half basically caused you to lose your $10 bet. Or equivalently, if you were to make a bet now, you'd have to pay more than $9.99 in order to win $10 (leaving out "vig" and transactions costs).

But then miraculously your team comes back to close the deficit to 42-40 and is now in place to attempt that last second, 50% field goal.

How much is your original bet worth now? It's back to its original value. And in one second, it's either going to be a winning bet or a losing bet. Or equivalently, if you were to place a bet now, it'd be right back at the original 50-50 odds. After 59 minutes and 59 seconds of football, you're back where you started.

So that single kick does indeed equal the value of all the other plays combined.

The original intent of the thread is slipping away, but I can't help it if I like football.
We often hear how bad it is to fumble the ball away close to your goal line. I've always wondered if it's any worse than to fumble it anywhere else. Close to your own goal, you're giving the other team great field position, but you're only giving up terrible position. Close to the opponent's goal line, it's reversed. At mid-field, you're giving up good position and giving the opponent good position.

According to that link, it depends on what the score is, and how much time is left.
If the score is tied with 3 minutes in the game, the worst place to fumble is your own 30 yard line.

If you have 1st and 10 at your own 20, you have a 58% chance to win (in 3 minutes).
If you fumble, giving up the ball at the 20, you have an 18% chance to win: loss of 40%

At the 40 yard line, a fumble swings the odds from 69 to 27% -- a difference of 42%

But at the 30, it goes from 69% to 22%, a 47% drop.
It's very odd, but your chances are given as 69% starting anywhere from your 30 to the 50.
http://wp.advancednflstats.com/winprobcalc1.php

The problem I keep having with win probability is that it seems to be tied linearly to the value of the play, or if it's not, nobody's told me otherwise.

For example, if the win probability for your team is 10% at a certain point in the game, is a touchdown at that point worth 1/5 as much as a touchdown when the win probability is 50%? If so, does that mean that it was 1/5 as hard to get that touchdown? Because I can't imagine, except for in the last couple minutes of a complete blowout, that NFL teams ever play "1/5" as hard as they do at the beginning of the game. 1/5? What would that even mean?

What does the difficulty of a play have to do with its value? A wide-open three by Ray Allen is worth as much as a turn-around, hand-in-the-face three by Dwight Howard (to make the point more ridiculous). And if Ray just walked down the court and no one picked him up and he got an open three it would be worth just as much as if he came down in the halfcourt and ran a full circle around the court trying to free himself off four different picks. Those plays might have different likelihoods of happening in the first place and different likelihoods of success in the future, but given that they've happened they're all worth 3 points, or however much win probability.

For what it's worth, here are some links to the same system in the NBA. http://www.basketball-reference.com/blog/?p=2207

Nobody else so far has interest in a new PG specific rating (loosely mimicking TQR or being totally different), so don't worry about the original intent of the thread Mike G. if folks what to talk football that fine. The thread serves what people want to talk about, not necessarily just the original intent.


"is a touchdown at that point worth 1/5 as much as a touchdown when the win probability is 50%?" This is troublesome but an essential point of this type metric. It may help to ask if the distribution of a player's touchdowns across the win probability increase distribution is similar for players. If it is then everyone's rating will be similarly affect and comparatively neutral. If some player's touchdowns tend to carry a higher win probability added on average than the league's win probability added on average they will be favored and this metric would argue they should be. Lower win added on average than league average then you would get marked as less valuable.

I probably wouldn't exclusively use a win probability added metric in a league with a small number of games and small number of big plays but I'd still want to see and consider it alongside a metric where a touchdown is a touchdown. I wonder if there is a middle ground where a yard's value is based on situation but a point is a point or win probability added only comes into play in the last 4-6 minutes and as a moderate increase in value, not a strict win probability added fashion because of concerns it would weight a few plays far more heavily than others.
I can't recall if Wayne Winston's Impact Rating only gave heightened value to actions in the last 5 minutes of an NBA game but that option is out there.

xkonk wrote:What does the difficulty of a play have to do with its value?

Well, I could see an argument that discounts "easy" touchdowns. Just like in basketball, garbage time stats are widely discounted because the opponent quality decreases and defenses tend to not exert 100% effort at that point.

But if you don't think it has anything to do with difficulty (and especially in that case), then why do you think win probability should be used at all? In that case, it makes no sense to me.