What $750,000 bought the Patriots

How much impact does the Patriots’ spying have on the outcome of the games? Let’s see if we can quantify it.


(Note: this is a *long* post that mixes statistics and football. Be warned!)

The actual question (is Patriots offensive capability increased because of spying) cannot be directly answered, because their offensive ability is always skewed from the impact of the spying itself. But we can investigate these two things to see if spying may play a part in the outcome of Patriots games:

1) Opponents’ defensive capability is decreased (on average, they perform more poorly against the Patriots than their average)
2) Individual player capability is increased because it is clear what holes in the defense can be exploited

Case 1: Opponent’s Defense is Decreased
We’ll use 2006 defensive statistics to see what, if any, impact this spying might have had. We’ll have to assume that all data is equally tainted, since we can’t conclusively say which games had spying and which don’t.

First I pulled out all the data from 2006 for the season averages per team, then I dug up the statistics for the games themselves between the Patriots and their opponents.

I omitted their last game, when playoff seedings were determined, to keep it from skewing statistics one way or the other. So for these purposes, there were 15 “contested” games.

In these 15 games the Patriots played 12 different opponents, 8 from the AFC and 4 from the NFC. The played 3 opponents twice (Miami, Buffalo, NY Jets).

Overall, if you compare the Patriot’s scoring against each team versus the team’s season average, you find that the Patriots scored 114% of each team’s average (for example, if the opponent’s defense averaged giving up 20 PPG, the Pats scored on average 22.8 PPG). Against AFC teams it was only 108%, and against NFC teams it was 131% of their season defensive average.

[For comparison, consider Indianapolis. They averaged 114% more yards per game against their opponents, with 88% of the rushing average but a massive 127% of the passing yards (so if the defense typically gave up 300 yards passing, against Indy it would give up 381). They typically scored 131% of the defense’s average.]

For 10 of the 15 games, the Patriots represented greater than 6.3% of the total points scored against them the entire season (6.3 is 1/16, or the portion of the season each team played against the Patriots). For 2 of the teams, the Patriots were 10% of the total points scored against them in 2006 (Cincinnati and Houston).

This could simply mean that the Patriots had a better-than-average offense.

What about sacks? Well, in 6 games the Patriots gave up an above-average amount of sacks, and in 9 games they gave up a below-average number of sacks. That could only mean the Patriots have a good offensive line.

Against the AFC, they typically gained 97% of average yardage against their opponent’s defense: 93% of the average passing yardage, and 106% of the average rushing yardage. This could reflect that the Patriots had the lead a lot, which meant they rushed more than usual. As far as yardage gained, the Patriots look pretty average.

However, they scored 108% more than average. So despite not gaining more yardage than was common against their opponent’s defenses, they tended to score almost 10% more than the opposite defense normally yielded.

Against the NFC, the Patriots gained 120% more than the average opponent, 130% more passing yards and 98% of the average rushing yardage. They scored 130% more than the average.

So against the NFC, the Patriots dominated. That’s not news; every AFC team dominated their NFC opponent. But once again, the Patriots have a gap between their excess yardage (120%) and their excess scoring (130%); this is again 8% higher than we might expect.

[Indy’s splits are even pretty much the same AFC and NFC; they dominated everybody.]

Nothing in there makes the opponents defenses *look* like they were significantly decreased. The only quirk is that the Patriots scored a little more than I’d have expected, given how many yards they typically gained.

Lest you think that 8% isn’t much, remember that the average score was 20 points, and an 8% increase means that the Patriots averaged 1.6 PPG higher than that; with an average NFL margin of victory of 3 points, that means the Patriots have gained about half the typical margin of victory. That should translate into some wins each season; how many, I can’t say.

I repeat, though; this may simply mean the Patriots had a better-than-average offense, and nothing more.


Case 2: Player Ability is Maximized
What I suppose is this: all other factors being equal, an offensive player will demonstrate superior performance under a system where coaches intercept signals than he does under a system where coaches do not.

The best way to see this is to compare some position by position performance of players before and after they played for Bill Bellicheck. Now, the problem is that there’s a massive amount of cross-player impact; for example, a wide receiver will perform differently if he plays with Peyton Manning than with Eli Manning. So we need to keep that in mind as we look at the numbers.

Anecdotally, this is where I’d expect to see the difference. How many times have you heard that the Patriots are a team without stars, who can take role players from other teams and turn them into superstars?


Running Backs
I looked at two running backs: Corey Dillon and Antowain Smith. Dillon has two periods: one with the Bengals from 1997 to 2003, then another with the Patriots from 2003 until now. For Smith, we have 1997 to 2000 with Buffalo, then 2001 to 2003 with the Patriots, and then 2004 with Tennessee and 2005 with New Orleans, for a total of three periods (pre, with, and post-Bellicheck).


The numbers are in this post.

Summary: The running backs don’t appear to be all that different.

Receivers

Statistics are stuck here for the 7 receivers I looked at: Reche Caldwell, Doug Gabriel, Deion Branch, Tim Dwight, David Patten, Terry Glenn, and Christian Fauria. Again, they're divided into either pre-Bellicheck, Bellicheck, and post-Bellicheck periods as their careers dictate.


Yards/catch doesn’t change, so they’re all physically the same (we’d expect this). Catches/game is typically higher (3-4 cases) or the same (2 cases), with only one drop (Tim Dwight).

Two players saw their first downs significantly higher with new England (Caldwell and Patten), while four had a higher rate of touchdowns (Caldwell, Gabriel, Patten, and Fauria). Two have no difference in these figures (Branch and Glenn), and only one showed a drop (Dwight again).

How much of this is because they played with a superior QB and how much from the system? It’s hard to say. But it’s worth noting that these guys are first down/touchdown machines in New England, but not anywhere else. It’s unlikely that they forgot where the markers are. But other factors (line play, QB) heavily impact this as well.

Verdict: Unclear, but most appear to play better in New England than elsewhere.


Quarterbacks

We can look at the numbers for 3 QBs: Vinny Testaverde and Bernie Kosar in Cleveland, and Drew Bledsoe in New England. We can’t look at Tom Brady; he’s always played under Bellicheck, so his numbers are all suspect.

I put their numbers here.

What do those numbers tell us?

QB rating is slightly improved under Bellicheck for 2 out of 3.

Accuracy is slightly improved from before, and stays the same after Bellicheck.

Yards/attempt seems pretty much the same (one up, one equal, one down).

TD% is improved for Testaverde and Kosar, decreased for Bledsoe.

INT% is down for Testaverde and Bledsoe; the same for Kosar; it rises again for Bledsoe after he leaves Bellicheck.

Sack % is down for Testaverde, up for Bledsoe, and up for Kosar.

Yards/sack are down for all 3; they increase for Testaverde and Bledsoe again when they leave Bellicheck, while they reduce again for Kosar.

Verdict: Quarterbacks do slightly better under Bellicheck, becoming more accurate, with higher QB ratings. They are also closer to the line of scrimmage, since they lose less yards per sack.


Overall Impression

The passing game under Bellicheck appears to be more efficient (higher percentage, more first downs, more touchdowns), but the running game doesn’t appear to be all that different.

How much of this we can attribute to picking up defensive signals, I can’t say. But I would leave you with this question: if you were a coach in a 3rd-and-4 situation, and you knew whether or not the defense was blitzing, wouldn’t this help you to both increase your odds of a first down on a short pass and to keep the loss because of a sack decreased, because you can move your quarterback into a quicker throw?

I’m just saying.


So what did $750,000 buy the Patriots?

I would guess that at most it bought them about 1.3 first downs a game and about 3-5 yards per game of field position on saved yardage during sacks. I can see that; at a critical time, late in the game, they dodge a blitz and pick up an extra 1st down.

Perhaps that translates to somewhere between one and 1.5 PPG, or about 15 to 20 points over the course of a season. The Patriots scored 24.1 PPG and gave up 14.8; if you shaved off 1.5 PPG from their offense they still win a lot of games. But that puts them somewhere between Pittsburgh at 8-8 (22.1 PPG earned and 19.7 PPG given up) and the Ravens at 13-3 (22.1 PPG earned but only 12.6 PPG given up). In fact, that’s where the Patriots were, at 12-4.

But who’s to say that the Bills game (19-17) or the Bears game (17-13) couldn’t have tipped the other way, leaving them 8-8, if they hadn’t had a little edge? So I can believe maybe one game per season turns on this edge, if at all.

If they did this for 6 seasons, that’s about 6 games they won that they might have lost (not definitely), or an average of 125,000 dollars per game.

How much do you suppose Al Davis or the Ford family would pay for a few victories? Probably much more than this, I’d suppose.

(By the way, I've opened comments for anybody who has any other ideas, or wants to dispute anything above. And kudos for getting to the end!)