Defending Belichick’s Fourth-Down Decision

By Brian Burke
New England Patriots

(Brian Burke, a former Navy pilot who has taken up the less dangerous hobby of N.F.L. statistical analysis, operates Advanced NFL Stats, a blog about football, math and human behavior.)

New England Coach Bill Belichick is taking a lot of heat for his decision to attempt a 4th-down conversion from his own 28 with a late 6-point lead against the Colts. Indianapolis came back to win in dramatic fashion, 35-34. Was the decision a good one?

With 2:08 left and the Colts with only one timeout, a successful 4th-and-2 conversion wins the game for all practical purposes. A conversion on 4th-and-2 would be successful 60 percent of the time. Historically, in a situation with 2:00 left and needing a TD to either win or tie, teams get the TD 53 percent of the time from that field position. The total win probability for the 4th-down conversion attempt would therefore be:

(0.60 * 1) + (0.40 * (1-0.53)) = 0.79 WP (WP stands for win probability)

A punt from the 28 typically nets 38 yards, starting the Colts at their 34. Teams historically get the TD 30 percent of the time in that situation. So the punt gives the Pats about a 0.70 WP.

Statistically, the better decision would be to go for it, and by a good amount. However, these numbers are baselines for the league as a whole. You’d have to expect the Colts had a better than 30 percent chance of scoring from their 34, and an accordingly higher chance to score from the Pats’ 28. But any adjustment in their likelihood of scoring from either field position increases the advantage of going for it. You can play with the numbers any way you like, but it’s pretty hard to come up with a realistic combination of numbers that makes punting the better option. At best, you could make it a wash.

Comments are no longer being accepted.

Michael November 16, 2009 · 1:20 am

I agree with the writer and not only because of the probabilities. Belicick saw that his defence allowed an easy 80 yards TD, that took just 4 plays and under 2 minutes. He expected Colts to do the same if Pats punt (and keep in mind that afterwards they did cover 30 yards in 3 plays which hardy took 30 seconds). So, he went for a play that was very likely to succeed and failed because of an inch or two. It was a bold decision that no other coach would do. If he succeeded- everybody would say he is a genious.
I don’t think it’s fair to critisize him harshly just because it didn’t work out. He should be judged on the decision making at the time of the decision and I think it was quite logical.

joejoejoe November 16, 2009 · 1:26 am

Hey Pats defense, it’s nothing personal but there is 0.09 of win probability on the line here. Don’t let this vote of no confidence effect your confidence in this crucial defensive set. Now go out there and win* one for the Hoodie!

* – 47% of the time. Or, apparently, a bit less.

Brent November 16, 2009 · 1:37 am

I liked the idea of the play itself, but the real issue was the fact that when he did this the Pats were only at the 25 yard line. That made it not into a matter of a successful or unsuccessful (and unnecessary, at that point in the game play) – that 40% that remained from the chance that it would be unsuccessful wasn’t just about the play, but the entire game lay in the balance. Giving the Colts the ball at that point and with so little time left on the clock made the play exponentially more risky – completely unnecessarily so.

Mike Drew November 16, 2009 · 1:37 am

I’m glad you didn’t go into commercial aviation.

Scott November 16, 2009 · 1:47 am

Sorry…the pass interference call that led to the score before (the one where Collie clearly interfered with the defender) is the game turner. The rule needs to go away now. College has it right…15 yards. If you notice the flag came in late…

Tree Falling in the Forest November 16, 2009 · 2:04 am

I thought it was a great decision.

Because I hate the Patriots.

Matthew B. Parola November 16, 2009 · 2:08 am

I recently read an article about a high school coach that uses this data extensively with much success, and therefore, rarely punts. It seemed to make sense, and I began to view the unwillingness of pro coaches to take such risks as typical of their conservative mindset. But the pro game, is a vastly different game than high school, and I don’t know that you go for it in that situation, so deep in your own territory. You certainly have far fewer critics if you go the safe route; punt, and expect your defense to go out and do what they’re getting paid millions to do. Few can criticize what most have a right to expect.

-Matt Parola

Matt November 16, 2009 · 2:10 am

Not only that but the spot was terrible and it worked.

Motonari November 16, 2009 · 2:15 am

This is the right approach to analyzing the call, but I think your numbers are off. As you pointed out, they are based on NFL averages. My guess as to the right adjustments for this situation would give:

Going for it:
~ 60% chance of converting –> 0.98 win probability
~ 40% chance of giving ball to Colts inside Patriots’ 30-yard line –> ~ 0.20 win probability (i.e., Colts score TD ~ 80% of the time)
Overall win probability ~ (0.60 x 0.98) + (0.40 x 0.20) = 0.668
(round to 0.67)

Punting:
Averaging over possible locations the Colts might get the ball and time and timeouts left, their chance of scoring TD in time remaining ~ 0.30
Thus, Patriots’ win probability = 1 – 0.30 ~ 0.70.

From these numbers, punting would be the better choice, but what is interesting is that it is a much closer decision than it seemed to be in real time. I thought that it seemed to be a very foolish call when it was made, but this analysis shows it to be a close call in retrospect.

What is particularly interesting is that NFL coaches tend to make conventional decisions under pressure. Unusual decisions like this one are uncommon because they can significantly harm coaching careers when they do not work. Bill Belichick may have no worries about his job in New England, but my advice to most coaches would be to stay away from unconventional calls unless they are clearly the superior choice. That was not the case here, and Bill Belichick will pay for it.

Robert Decker November 16, 2009 · 2:16 am

I can’t even believe this is being debated. Belicick took a big risk that 95% of all teams wouldn’t have taken simply because of where the ball was on the field. Arrogance, is the reason. If you could take a poll of viewers I’m sure the majority would have said ‘punt’…and because he didn’t it became his fiasco and no one else. I don’t like Peyton Manning but I started rooting for him after that stupid call.

eswing11 November 16, 2009 · 2:17 am

The calculations assume the Patriots would have won had the gotten the first down, which is probable but not 100% probability as assumed above. Even ignoring the possibility of a Patriots turnover, the Colts had one time out which would have prevented the Patriots from kneeling to run out the clock (being generous and assuming 90 seconds between plays, the Colts would get the ball back with 30 seconds minus the time run off in three plays plus a punt).

The Colts would probably only get the ball back with about 5-10 seconds left even if they could stop the Patriots, but that’s enough to give them some chance (maybe 2-3%?).

That said, I agree with the writer that it’s not nearly as bad of a decision as people think. It just looks bad in hindsight (especially because the norm among NFL coaches is to be conservative and punt).

Victoria November 16, 2009 · 2:18 am

I am so happy the Colts won!!!

Like Tree in the Forest, I can’t stand the Patriots.

spikethedog November 16, 2009 · 2:22 am

The criticism extends beyond his decision to attempt the first down, but extends to the Pats squandering of one or two timeouts which which would have allowed them to challenge the spot.

What amazes me is that with a total of 69 points, the bookies were so close…….If you bet the Pats to win straight up, you’d have lost. If you bet the Pats and took the 2.5 or 3 points, you’d have won.

All the counts is the Pats win their division (they will) and that they get a rematch at Indy that they win. All will be forgiven then.

Bob F. November 16, 2009 · 2:25 am

It’s one thing to go against the norm but they called two foolish timeouts and the pass route was awful. How can a receiver not get the necessary yardage with room to spare? The magic of the exposed sign stealers seems to be waning.

David November 16, 2009 · 2:25 am

If going for it is statistically the right decision, then coaches should be going for it on 4th down more often. They are not. Why not? It is prudence in the face of asymmetrical outcomes or is it ignorance? Perhaps future games will evidence the rationale for play-calling in similar situations. Regardless, love to see guts and controversy in the NFL.

Dave November 16, 2009 · 2:33 am

Hard to argue with statistics. Despite harboring “sports animosity” towards the Pats and BB, I respect the unconventional risk taking behind that 4-and-2 call. Everyone knows Peyton Manning – at this stage in his career in this offense – is the most dangerous qb in the league in the two minute drill. Coaches have to make the tough decisions that they feel give their teams the best shot at a win and Belichick definitely did not win his super bowls by making decisions based on polls of fans and sports “analysts”.

No way Belichick loses his defensive players on this one – that’s pure speculation fed by sports analysts desperate for a story line to milk until next weekend’s games.

That said, I don’t think Belichick was prepared for the 40% chance they don’t make that first down – he looked like someone just shot him in the gut!

JRob in NC November 16, 2009 · 2:34 am

I like watching entertaining and exciting football which was delivered to me.

I enjoy watching Moss snag down a TD pass as much as I like watching Peyton throw one.

I hope to see more aggressive offensive decisiveness in the future.

bigE November 16, 2009 · 2:36 am

There are arguments both ways here. Maybe Coach B didn’t trust his defense to win–they weren’t doing too well down the stretch. Maybe he trusted them all along–even if they didn’t make the first down, the Colts still had to get a touchdown, and the redzone can make it harder for anyone to pass (the obvious Colt strength). The winning TD catch was amazing, and had to be.

As for the percentages you use, I don’t have time to check them out, but I think you’re leaving out options to add in.

Chances of going for a 4th and 2 on the Colts defense and by the Patriots offense need to be reconciled. Just because my dear St. Louis Rams can’t make a 4th down…or a 1st down… for their lives doesn’t mean that the Patriots should be punished in your calculation. And the Colts shouldn’t be punished for the sub-par Rams defense either.
This isn’t a case of historical percentages, but of the percentages on the field. How often can Randy, Tom, Wes, and Kevin make a 4th and 2 against the Colts?

For the punt, you also have to look at the percentage that the punt actually gets off (since this is less than 100%, this will decrease the incentive to punt). But then you have to look at how well the Patriots punt and cover…net could be over 40. And for the return drive, it should also be Peyton v. Patriots stats, or at least Peyton in 4th quarter, 2 min drill situations. Peyton isn’t your average NFL QB, and the Colts offense isn’t the typical offense either.

There’s just a lot more math here than meets the eye, and while you have 79 v. 70, a 5% margin for both measures gives you an overlap at 70’s max (75) and 79’s min (74). So in the end, there may not be a difference whether he goes for it or not.

During a football game, I don’t think most people are doing math. In the end, the Patriots’ defense had to make a play, and they couldn’t. Would they have been able to had Peyton started from his own 30? They might have had more chances. But in the NFL, where games “come down to inches,” every play matters. They had their chance to win, and they blew it. Shouldn’t have even had to come to a 4th down.

M. H. November 16, 2009 · 2:52 am

Overall, this is a very fun debate, and I don’t wish to denigrate Burke’s cool post. Here are some possible critiques to using such elaborate statistical analysis that come to my mind:

To elaborate on joejoejoe’s point, statistics regarding the potential outcomes of one individual, crucial decision do not account for the effect on a group’s (in this case, the Patriot’s defense)morale and cohesion after a poor outcome. If the coach asks the defense to stop Indy after a punt it seems likely that the defense would perform at at least normal effectiveness.

A question emanating from this “group” frame of reference would be to consider which is better: a loss as a result of asking the defense to make a stop and the defense failing? Or a loss due to a coach’s decision which gives the opponents a very short field and demoralizes your own troops? The argument to punt from this frame would include valuing the feedback the defense would receive after being put in a situation which determines the outcome of the game. Such a thought process would also factor in that the game was a regular season game rather than a playoff game.

Also, remember that statistics are poor at predicting the result of any one play. To say “the result of a punt from the 28 typically nets 38 yards” is less useful than the language implies. In fact, it would be probable that the outcome would NOT net 38 yards.

Food for thought, hopefully.

OAO November 16, 2009 · 3:12 am

A brilliant chess move by Bellichick and it came so close. In a parallel universe the play worked and Bellichick is being hailed as a genius.

BobJ November 16, 2009 · 3:38 am

LOL yea use all the stats you like to back up Bills big blunder.

You never go for it on 4th and 2 on the 28 period. Well if you are loosing you do but not being up 6 points.

1. If you get the first down its not a sure win
2. Your D had 2 picks on Manning, you lost faith
3. You left out some math but even in the example you show punting is still the better option.
4. You had no timeouts left (not factored in) Can’t review a close play without timeouts.
5. You just wasted 2 times outs, one on a first then another on a second down. Two dumb coaching mistakes
then the strikeout blow! A third.

Then you get people backing this decision up with made up stats fixed to show that it is the best move to make. Yea that’s why no coach in probably 10 years has gone for it on 4th down on the 28 while winning.

It was a stupid call even if they did get a first down.

Edward Smythe November 16, 2009 · 3:50 am

In a parallel universe, Bellichick is not a stinking cheat either.

Ctc121189 November 16, 2009 · 4:14 am

If you look at it, would you rather put the game on the line with your play-makers (Patroits Offense) or would you like to see the pass happy colts march down the field on the fifty yard line with 2 minutes left with a spotty defense on the field? At the time it was a like a double sided sword. I would have went for it. Peyton is the best in the league with the two minute drill. Thats what I saw imo

David from Buffalo November 16, 2009 · 4:22 am

As a Buffalo Bills fan that had to survive Monday Night Football week one this feels really good. Some people would call it Karma –

GO COLTS!
GO BILLS!

jackson November 16, 2009 · 4:34 am

One thing stat geeks fail to quantitate is common sense. You have to punt in that situation.