Game Theory and Football: How Irrationality Affects Play Calling
June 15, 2011 24 Comments
Coaches and coordinators in professional football get paid a lot of money to call the right plays – not just the best plays for particular situations, but also unpredictable plays that will catch the other team off guard. It’s a perfect setup for game theory analysis!
As in other game theory situations, the best play depends in part on what your opponent does. Your running play is much more likely to succeed against a pass-prevent defense, but would be in trouble against a run-stuffing formation. If the defense can guess what you’re going to call, they can adjust accordingly and have an advantage. Even on 3rd down and long – a common passing situation – there’s value in calling a percent of running plays, because the defense is less likely to be geared toward stopping that. But as you do it more, the chance of catching the defense off guard gets smaller. There’s some optimal balance where the expected success of a surprising run is equal to the expected success of a more sensible (but anticipated) pass.
The goal is to stay unpredictable and exploit patterns where your opponent is using a sub-optimal combination. If a team notices that passing plays are working better, they’ll be more likely to call them. As the defense notices, they’ll shift away from their run-defense and focus more on defending passes. In theory, the two teams reach an equilibrium.
In practice, it doesn’t quite work that perfectly – human beings are making the decisions, and humans are both vulnerable to cognitive biases and notoriously bad at mimicking true unpredictability. Brian Burke, a fellow fan of combining sports with statistics, was poring over the play-calling data for second downs and noticed something odd:
There’s a strange spike in percent of running plays called at 2nd and 10! Tactically, 2nd and 10 isn’t all that different from 2nd and 9 or 11, so it’s strange to see such a difference. Why would they call so many more running plays in that particular situation?
The key is to realize that there are two ways a team tends to find itself facing a 2nd and 10 situation – runs that happen to go nowhere or any incomplete pass. Of those, incomplete passes are far more common. So in cases of 2nd and 10, it’s most often because the team just failed a passing play. That suggests two reasons coaches might be irrationally switching to running plays, even at the cost of sacrificing unpredictability:
(1) The hasty generalization bias (also called the small sample bias) and the recency effect are cognitive biases in which people overgeneralize from a small amount of data, especially recent data. Failed passes are very common (about 40% fail), so there’s no good reason for a coach to treat any single failed pass as evidence that they’d be better off switching to a running play. But the urge to overreact to the failed pass that just happened is strong, thanks to these two biases.
(2) People are terrible at generating unpredictability — when asked to make up a “seemingly-random” sequence of coin flips, we tend to use far more alternation between Heads and Tails than would actually occur in a real sequence of coin flips. So even if coaches weren’t overreacting to a failed pass, and they were simply trying to be unpredictable, they would still tend to switch to a running play after a passing play more often than random chance would dictate.
Indeed, when Brian separated the data by previous play, the alternation trend is clear — passes are more likely after runs, and runs are more likely after passes:
(My favorite team, the Baltimore Ravens, was pretty bad about this under the previous regime, Coach Billick)
Brian concludes:
Coaches and coordinators are apparently not immune to the small sample fallacy. In addition to the inability to simulate true randomness, I think this helps explain the tendency to alternate. I also think this why the tendency is so easy to spot on the 2nd and 10 situation. It’s the situation that nearly always follows a failure. The impulse to try the alternative, even knowing that a single recent bad outcome is not necessarily representative of overall performance, is very strong.
So recency bias may be playing a role. More recent outcomes loom disproportionately large in our minds than past outcomes. When coaches are weighing how successful various play types have been, they might be subconsciously over-weighting the most recent information—the last play. But regardless of the reasons, coaches are predictable, at least to some degree.
Coaches are letting irrational biases influence their play calling, pulling them away from the optimal mix. The result, according to Pro Football Reference stats, is less success on those plays. I wonder how well a computer could call plays using a Statistical Prediction Rule…
I laughed out loud when I saw Brian’s graph, but the explanation seems so obvious when you break it down. Nice work!
I thought this was a wonderful example of human irrationality (though part of the reason I liked it was that as soon as I saw that spike I guessed why it existed!). It might be in the nature of football coaches to avoid true randomness — they’re supposed to know what to do, and how to outguess their opponents, so they’ll lose the confidence of their players and their management if they say “I know it seemed like a run would have worked better, but I flipped a coin, and it said ‘pass.’ “
Is it irrational if the coach thinks that his quarterback needs a play to settle down. This seems extremely simplistic. There are few plays your Baltimore Ravens defense couldn’t read immediately as a pass or run. It’s how the play develops that is the interesting part not it’s bare type.
It’s not irrational on its face – without looking at the numbers, that might be an excellent strategy. But when we look at the stats we see that it’s not actually helpful. According to the Pro Football Reference numbers, passing plays after an incomplete pass DO have better results than a running play:
I’m sure there are times the quarterback could benefit from a play off, but since passes are more successful in those 2nd-and-10 situations, coaches seem to overestimate the need.
Excuse me, that is to say, the interesting part of a coaches randomness isn’t at the level of the type of play, but it’s execution.
I know that they know that I know that they know that I’m going to run. Unless they know that I know that they know that I know that they know that I’m really going to pass. In which case I should run it, for sure.
And I can clearly not choose the wine in front of me!
Right, Jesse — both plays fail.
It always seems like such ridiculous writing to say something like, “X type of person are apparently not immune to the Y fallacy.” Unless X is computers or hyper-rational aliens, it’s a waste of a sentence. And even computers are programmed by people.
I think you may be incorrect about 3rd and long. If you have 3rd and 10 or more to go, unless you’re going for it on 4th, you should almost never run. Even if you completely fool them you’ll seldom get the 1st. Deception isn’t everything. It’s only a counter-balance to the “normal play” (the one that would be optimal against a non-strategic opponent.) This point is made more eloquently by David Sklanksy in “Theory of Poker” for my money, the best applied game theory book. It explains many game theoretic principles using no math beyond arithmetic. (I’ve plugged this book in my other comment too. It’s really a great book; one of my favorites.)
I can see two other reasons why it would be compelling for a coach to run the ball on 2-10 as opposed to 2-9 or 2-11. In both the latter situations, the play ended with the clock continuing to run. For the second and 10, the clock very likely stopped. So one possibility is that it is near the end of a half or game where running the clock down is more valuable than the yards or even the first down. In another possibility, during the other parts of the game there is a need to keep the clock running as much as possible because you need to think about the stamina of your defense in the latter stages of the game. By keeping the clock moving you not only shorten the actual length of the game but the number of plays your defense has to defend. In either case, clock management is an issue.
Good thought! The clock management angle is probably smaller than you think though:
There might be a small influence of the clock, but the biggest clock issues – plays near the end of the game – were removed.
“That suggests two reasons coaches might be irrationally switching to running plays, even at the cost of sacrificing unpredictability.”
While the spike is interesting, and while I intuitively buy the reasons you offer for it, I disagree with the “sacrificing unpredictability” statement. The run-pass ratio on 2nd&10 is about 45% or so, judging by the graph. If I had to predict the offensive play based on this information, I’d be correct 55% of the time. That’s less than ~70% with which I’d be guessing correctly on 2nd&9 or 2nd&11. The observed spike may be irrational, but the point is that NFL offenses are more, not less, unpredictable on 2nd&10 than in tactically similar situations.
This is why Bill Walsh was such a great coach. He would script the first 15 or so plays ahead of time, and then would run them (almost always) regardless of what had been accomplished on the prior plays. His thought was that the ability of his team to execute the well-practised plays and the fact that they would be unpredictable from the standpoint of updated knowledge (the prior plays) far outweighed the disadvantages. It was a very successful approach.
I love really rigorous and insightful sports stat articles, and I think getting people interested in critical thinking requires that we apply to thinks we all like to do, and sports is a big one (so good post). But I disagree that this example of play-calling is necessarily irrational. I would say that this argument overvalues unpredictability, and undervalues the basic idea of updating one’s model of the world based on new information. After your quarterback and receiver don’t connect on the first play, you have information to update your model – that is to say, you know the other team is doing something to cause you not to make passes, so on the next play, predictable or otherwise, it makes MORE sense than it did before to try a running play. It seems to me that you’d be LESS rational to say “Well, I know from the last play that the other team can keep our passes from connecting. But rather than incorporating that information and tending to try another kind of play, I’m going to flip a coin or return to a prescripted sequence to avoid being predictable.”
One reason the predictability of the response isn’t as important as you might think is that teams do not have infinite defensive capability. That is to say, just because the other team knows you’re going to run doesn’t mean they can necessarily benefit from that information and DO anything about it, because maybe their defensive line isn’t that great – even if their pass coverage is good, which you already have some reason to believe from your failed pass on first down. Finally, I bet that college and even pro teams don’t have (or don’t pay attention to) these kinds of statistics during games, so worrying that the other guys know that at 2nd and 10 after a pass the offense runs X% of the time, will probably not yield much profit.
A final observation on pass vs. run “predestination”: Penn State’s Joe Paterno is infamous for being super-conservative and going straight up the middle repeatedly, regardless what went on during previous games, and it’s cost him games. I didn’t know that about Bill Walsh’s play calling (from the previous comment) so it would be interesting to compare situations where they got burned by plays that didn’t work when there was information to be had based on previous failures during the game.
“it would be interesting to compare situations where [Walsh] got burned by plays that didn’t work when there was information to be had based on previous failures during the game.”
That would be interesting to see. Undoubtedly there were a number of times when this approach didn’t pan out, and there were times when he departed from the script. However, it was an approach he used for the most part his entire career. It helped win a number of championships and superbowls and in that sense the result speaks for itself. I think the key point is that this approach managed to achieve two important goals: it meant that the 49rs executed these plays particularly well because they had been so well-rehearsed, and it was much more unpredictable or “irrational” since there was no way (short of guessing based on prior scripts (which he studiously avoided)) to predict what the next play would be based on the prior plays.
I don’t think you entirely understood the post Michael. Nowhere does Ms. Galef state that you shouldn’t be continuously updating your model with new information as it comes. However, that new information from a failed pass is merely one data point, and should be weighted as so. For example, we can take a team with a great quarterback and receivers and mediocre running backs. There is an incomplete pass from their quarterback the first play. If a computer was programming the expected yardage of the next play (using inputs on the precise stats of players from both teams) it would state that the coach should call for another pass.
However, coaches often don’t take all the information available to them at the time. They merely think “well, passing failed the last play, so it will be more effective to run next play.” This is a non sequitor. Due to the recency effect, they are giving too much weight to recent events as datapoints in their decision. A rational coach will have some kind of algorithm in his head to properly weigh factors such as injuries, the strengths of the opposing team, unpredictability and other factors.
Jake,
I don’t disagree with anything you said. What does it have to do with my comments regarding Walsh’s play scripting?
Jake,
You must have meant Michael (not Mike) — of course that’s what you said. Never mind . . . . .
Here’s an interesting link I found describing the process:
http://cohn.blogs.pressdemocrat.com/12936/bill-walsh-scripted-plays/
He said scripting was like throwing a jab in a fight. How would the opponent react to the jab? What would it tell you about the next jab or a left hook off the jab? So, apparently there was a third benefit from this approach, it would inform him regarding how the opponent responded to specific situations and helped guide the rest of the play calling. Interesting article, take a look.
No problem, Mike! I also got my names mixed up. I originally thought this post was written by Julia, when it was actually written by Jesse. So credit goes to Jesse for an interesting article 🙂
It’s interesting that irrationality is brought up. I’ve lately heard a lot about George Lakoff, a well known scientist and in this guy’s books he constantly writes about ‘fake’ reason and ‘real’ reason, the real reason being the rationality stuffed with emotion and metaphorical thought, and fake reason being classic logic. He’s argued with Steven Pinker, but I just wanted to know if he is an enemy relativist or have I just greatly misconstrued the situation at hand when in fact he means something else?
Thanks, and if the latter is true than it’s pretty big as lots of people are getting the wrong impression about him, and are preaching postmodern ethics.
Would be great if you guys could do some debunking!!!
Cheers, and great post.
I’d be curious to know whether the run % after a run is affected by the amount of contact the RB suffered on the previous play. I think part of what’s happening is the coach’s desire to give the RB a break. If the RB goes out of bounds the clock stops and in general has suffered less contact. I wonder if run %s in such situations is higher than otherwise.
Pingback: Which Cognitive Bias is Making NFL Coaches Predictable? | Measure of Doubt