Coach Smith’s Gutsy Call
November 16, 2011 17 Comments
Coach Mike Smith was facing a tough decision. His Falcons were in overtime against the division-rival Saints. His team had been stopped on their own 29 yard-line and were facing fourth down and inches. Should he tell his players to punt, or go for it? A punt would be safe. Trying to get the first down would be the high-risk, high-reward play. Success would mean a good chance to win, failure would practically guarantee a loss. What play call would give his team the best chance to win?
He decided to be aggressive. He called for star running back Michael Turner to try pounding up the middle of the field.
It failed. The Saints were given the ball in easy range to score, and quickly did so. The media and fans criticized Smith for his stupid decision.
But is the criticism fair? If the play call had worked, I bet he would have been praised for his guts and brilliance. I think my favorite reaction came from ESPN writer Pat Yasinskas:
When Mike Smith first decided to go for it on fourth-and-inches in overtime, I liked the call. I thought it was gutsy and ambitious. After watching Michael Turner get stuffed, I changed my mind. Smith should have punted and taken his chances with his defense.
What a perfect, unabashed example of Outcome Bias! We have a tendency to judge a past decision solely based on the result, not on the quality of the choice given the information available at the time.
Did Coach Smith know that the play would fail? No, of course not. He took a risk, which could go well or poorly. The quality of his decision lies in the chances of success and the expected values for each call.
Fortunately, some other people at ESPN did the real analysis, using 10 years of historical data of teams’ chances to win based on factors like field position, score, time remaining, and so on:
Choice No. 1: Go for the first down
…Since 2001, the average conversion percentage for NFL teams that go for it on fourth-and-1 is 66 percent. Using this number, we can find the expected win probability for Atlanta if it chooses this option.
* Atlanta win probability if it converts (first-and-10 from own 30-yard line): 67.1 percent
* Atlanta win probability if it does not convert (Saints first-and-10 from Falcons’ 29-yard line): 18 percent.
* Expected win probability of going for the first down: 0.660*(.671) + (1-.660)*(.180) = 50.4%Choice No. 2: Punt
* For this choice, we will assume the Falcons’ net punt average of 36 yards for this season. This means the expected field position of the Saints after the punt is their own 35-yard line. This situation (Saints with first-and-10 from their 35, in OT, etc.) would give the Falcons a win probability of 41.4%.
So by choosing to go for it on fourth down, the Falcons increased their win probability by 9 percentage points.
That’s a much better way to evaluate a coach’s decision! Based on a simple model and league averages (there are problems with both of those, but they’re better than simply trusting outcome!) the punt was not the best option. Smith made the right decision.
Well, sort of. There are different ways to go for the fourth-down conversion, and according to Brian Burke at AdvancedNFLStats, Smith chose the wrong one:
Conversion success rates on 1-yd to go runs (%)
| Position | 3rd Down | 4th Down |
| FB | 77 | 70 |
| QB | 87 | 82 |
| RB | 68 | 66 |
| Total | 72 | 72 |
In these situations, quarterback sneaks have proven much more effective than having your running back take the ball. In a perfect game-theory world, defenses would realize their weakness and focus more effort on stopping it. But for now, it remains something more offenses teams can exploit. According to the numbers, the Falcons probably could have made a better decision.
And, of, course, it was OBVIOUS to me at the time that they should have called a quarterback sneak. </hindsight bias>
Pingback: RadarLake » Football and healthcare–watch me make the connection
Yes. Of course, sports spectating is practically predicated on caring directly about *realizations* of random variables, not the random variables themselves. It’s as if the RV’s themselves are so pushed into the background of what people care about, that spectators don’t automatically see randomness itself as a source of variation. It’s way more fun to assign all the responsibility to players and coaches.
In economic matters, risk is best allocated in accordance with who is most willing to bear it. But here, it’s more like spectators allocate risk in accordance with who they most enjoy bearing it. Furthermore, coaches and players themselves may be happy with this arrangement if it entitles them to all the glory that comes with upper tail events! Sports is a more exciting, turbulent world if you stand a chance of actually significantly changing how people feel about you based on the outcome of today’s game.
That’s a really interesting point. I know coaches are under a lot of pressure, though some more than others. Safe coaches can assume more risk (opening them to more play call options, some of them higher expected value) and some coaches might feel nervous and need to play it safe.
I wonder how much that affects their aggressiveness in terms of playcalling… a statistical analysis is called for!
The play chosen in sports actually does frequently have a lot to do with probability and also with risk assessment. A gutsy play is generally worth it if the odds of success are low but winning the game without it is also low, which, of course, calls for a great deal of evaluation of what the likelihood of winning is.
This has actually been applied a lot to baseball, for example, whether it is generally a good idea to intentionally “walk” a good batter rather than give them the chance to hit a home-run. (In most cases, it can be demonstrated that it’s not, despite it being a traditional move to make.) Or what pitcher to put in first. The actual statistics don’t always line up with the traditional wisdom.
Still, in a case where a single touchdown can likely result in winning the game, I always think the “what the *** is he doing” play is the best one. Unfortunately, you can generally only do it once.
See here: http://www.youtube.com/watch?v=h6Ta0PYFNxM
Obama’s “gutsy call”
http://articles.latimes.com/2011/may/15/news/la-pn-robert-gates-60-minutes-20110515
“Defense Secretary Robert M. Gates called the raid against Osama bin Laden a ‘gutsy call’ — ‘one of the most courageous’ he’s seen a president make — because the U.S. had only circumstantial evidence indicating the Al Qaeda leader was at the compound in Pakistan.”
Why praise high-risk/high-payoff decisions, especially when they risk other people’s lives?
P.S. Why say “ONLY circumstantial evidence”? DNA, fingerprints, anything other than eyewitness testimony is circumstantial evidence.
“Why praise high-risk/high-payoff decisions, especially when they risk other people’s lives?”
Well, what if the high-payoff is saving many people’s lives?
If the situation called for it, I would praise a high-risk/high-reward decision if the expected value were higher than a safer option. When people are risk-adverse in decision-making, that leads them toward lower-EV choices. If we think that’s the case we should praise gutsy calls.
Would you praise your broker if he invests all your savings in one promising company because the expected return is higher than diversifying?
In poker, you sometimes have a player go all-in with 80:20 odds over the opponent, and the opponent gets a lucky card and wins. Almost makes poker look like a game of chance 😉
But what if the winner were declared to be the player with the best cards before the turn or the river?
That would be interesting… what if half the pot were given to the winner of the hand, and the other half divided according to who bet when they had certain chances of winning?
For example, after the players show their cards they calculate (or have a computer calculate) what the odds were preflop, flop, turn and river.
(Preflop bet * Preflop chance of winning) + (Flop bet * flop chance of winning) etc
It would only work in cash games, because it would be almost impossible to knock someone out of tournaments, save for rounding errors (It’s almost impossible to be drawing dead pre-flop – I googled it and found some ways though: http://thatsnotpoker.blogspot.com/2008/07/drawing-dead-pre-flop.html)
Well done Jesse!
Say at the end of 3rd down you feel they ought to go for it. The Falcons oblige by going for it and converting. But now you feel that if not for some defensive misstep, they would have been stopped. So is this considered outcome bias if it is not so much the outcome, but additional info available only after the decision that leads you to change your mind?
Good question! Outcome bias can affect us in success and failure.
I think we might be more prone to assume special knowledge in success. If the coach does something we wouldn’t expect to work – and it does – I have a tendency to assume the coach was just smarter than I. “He must have seen some hidden weakness or pattern in the defense”
That applies in all contexts, of course.
As someone that praised Belicek’s similarly gutsy call (but really, a different scenario)…I don’t necessarily love Smith’s decision. It really is a borderline call in this case…as the reality of the situation isn’t exactly in tune with the statistics. (Which is the great thing about football.) How good is THIS O-line, THIS RB, THIS defensive line, etc. Further, a 36 yard average punt isn’t a great measure of this situation as average punt distances from your own territory are generally longer (the average being lowered by punts past the 50 yd mark). I could go on…but you get the point.
But aside from the statistics….the risk-reward balance again is different in this game as opposed to Belicek’s call. Had they even been 10 yards further up-field, the call to go for it would have been almost certainly the way to go.
Lastly, as you mentioned….the call was terrible (and not speaking from hindsight.) I simply hate that call on 4th and 1 (almost as much as I hate the prevent defense). The 66% for RB success on that play seems obvious…the play gives the defensive line the greatest opportunity for success. Get a great push on the line and you can stuff the RB. Whereas this is far more difficult on a QB sneak, where the QB may even get the inches without a great push from the O-line.
Regardless, the extent to which Smith was blasted was totally ignorant.
I loved Belichick’s 4th down call too. I heard one commentator on ESPN saying “You know the one coach who can get away with a failed call like that? Bill Belichick.”
It’s a strange dynamic, isn’t it? If you’re a respected coach, you don’t lose your job on a positive expected-value risk that fails. Because you’re allowed to take more +EV risks, you end up doing better. Because you do better, you gain respect.
I’m seeing parallels to having a large stack in a poker tournaments. If you have 1,000 chips, it’s a no-brainer to call 80 chips with a 55% chance of winning the hand (positive EV). If you only have 100 chips, you might not want to risk that much of your stack, as you’d be crippled should you lose.
With infinite money and no betting limit, you can win at anything by simply betting twice what you lost.
Jesse, you bring up a great point. Smith’s call was far gutsier in that sense. (Maybe a little TOO gutsy if you add to the risk-reward equation a loss in the job-security department). Yes, Belichick is the one coach that can get away with it. (Perhaps throw in Sean Payton after that insane onside kick in the Superbowl….I mean, can you criticize a coach for a crazy call after he made one of the craziest calls in history to help win a Superbowl?)
On the flip side (switching to basketball), I always hated Phil Jackson’s inability to take any type of risk whatsoever (considering the level of job security Phil enjoyed).
Interesting comparison to poker. In a sport where you play only 16 games (magnifying the importance of each game)….so if you lose, you guarantee a loss…against the division leader (where a win would put you right on track with them), and now you throw on outside factors like media backlash and a decrease in job security…. whereas if you win….you MIGHT win the game? ….Again, I applaud Smith’s gutsy attempt….but yeah, his “stack” just seems a tad to small to make that call.
Pingback: Counterfactual Thinking, Rules, and The Knight Capital Accident | Kitchen Soap