Measuring the Saints' game-by-game performance

Last week I tried to assess just how bad the NFC South was (extremely) and whether or not the Saints have a chance to turn things around (maybe). The latest division odds seem to bear that out, as the Saints are now favored to win the division for the first time since the season began.

This week, I’m going to explore the Saints’ week-by-week performance on offense and defense. While it seems apparent that the offense among the best and the defense is about as bad as the offense is good, those are just averages. The week-by-week numbers might tell us more.

So I measured the team’s performance in terms of Adjusted Yards per Attempt (aYPA). AYPA is a measure of offensive efficiency that is a strong predictor of winning. AYPA is an improvement on yards per attempt because it accounts for interceptions and sacks. There are a number of different ways of calculating aYPA, but my way is this:

[Passing Yards - sack yards - (60 * INTs)]/(passing attempts + # of sacks)

In English, you adjust the number of passing yards by subtracting the number of yards lost on sacks and take away an additional 60 yards per interception thrown. You then divide this by the number of attempts + the number of sacks, since sacks should count as passing attempts, too. Right now, the league average is 4.8 aYPA.

So here’s a graph of the Saints aYPA on offense and their aYPA allowed on defense. The color of the dot represents the point differential for the game: blue dots were wins and red dots were losses, with more intense colors indicating a more lopsided game. Also note that I reversed the y axis so that higher y-values means the Saints played better defense:


A few things jump out. First, the Saints’ passing offense has played above league average in every game but two: Cleveland (loss) and Tampa Bay (win). Second, note the three light red dots representing losses to Cleveland, Detroit, and Atlanta. Last week, I talked about how the Saints might have been slightly unlucky so far this year. These games are examples of that. Luck plays as much of a role in close games as skill does, and a team that wins a lot of close games in one year isn’t necessarily more likely to win a bunch of close games the next year. It’s fluky. That’s one of the reasons why teams that blowout other teams tend to be better than teams that eke out victories: blowing out teams is a sign of strength, eking out victories is a sign of luckiness.

Second, the Saints’ passing defense has been terrible, playing below average (often WELL below average) in every game but two. This is unsurprising, but sad. Just a little more defense vs. Atlanta, for example, and this season is looking much better. Of course, you could say the same thing about the offense and Cleveland, which was a more appalling loss in many ways. So it goes.

But that’s just passing offense. Here’s the same graph again, but using Advanced Football Analytics’ Expected Points Added(EPA) instead of aYPA. EPA is a more complete measure of offense and defense than just aYPA. Read the link for more details, but EPA essentially compares how the team does on every play vs. the league average for plays of similar down and distance and converts this into “Expected Points Added”. Higher numbers are better on offense and worse on defense, and the league average EPA per game is approximately 4.76.


Note that some of the specifics are a little different, but the conclusions are the same: the Saints’ offense remains excellent, the defense remains decidedly unexcellent, and there are a couple of close games where a little better offense or defense would have made the difference.



Like what you see here? Please consider supporting me on Patreon. Your support helps keep me blogging plus, you know, kids are expensive.

Division Odds, Week 8

The Saints are now favored to win the division, at least according to the Football Outsiders playoff odds simulation. The latest simulation has the Saints odds of winning the division at 63.8%, the highest they’ve been all year and the highest of any team in the NFL that doesn’t currently or formerly employ Peyton Manning. Check out the graph:


Now, you might be wondering how the playoff odds fluctuate so much from week-to-week. How can the Falcons go from 60% likely to win the division in week 4 down to 10% likely in week 8. Do things change that much?

The brief answer is that while the quality of the teams don’t change that much, our understanding of how good or bad they are does. The NFL has an impossibly small sample size of 17 weeks, which means it’s really hard to tell how good NFL teams actually are. You wouldn’t try to judge a baseball batter after 17 at bats, right? But that’s what we’re trying to do here*. So after each week, we understand a little more about the teams, and our judgment of how good or bad they are changes to reflect this new understanding.

*I get that many of the advanced stats use individual play data to expand their sample size, but my general point is valid, especially looking at overall team quality.

So when Football Outsiders does their playoff odds simulations, they use their judgments of how good or bad teams are (primarily their DVOA ratings) in the simulations. Each week, we get more information, so the odds can shift dramatically.

An imperfect metaphor: think of the playoff odds simulations (and other advanced stats, really) as a complex version of Battleship in which your opponent’s ships represents the actual quality of an NFL team. The extra challenge: you don’t know where opponent’s ships’ size, location, or shape. And your opponent is probably a cheater.

Each week, Football Outsiders calculates the playoff odds based on the information they have so far (DVOA and the team records). Each simulation is like a shot in Battleship: a guess as to the quality of the team (in this metaphor: the location of your opponent’s ship). The following week’s game suggests whether or not the last shot was a hit or a miss: did the team do as well as expected (hit!), or did they do more poorly (miss)? Then they re-calculate the team quality (DVOA) and re-calculate the playoff odds, taking the prior week’s performances into account, and “shoot” again. This time with slightly more accurate information.

On top of all this, the actual wins and losses from each week make it easier or harder for teams to make the playoffs, changing the playing field entirely.

Failed metaphors aside, as hits and misses accumulate, you get a better idea of the ships’ locations and the teams’ quality, which makes your later shots more likely to hit than your earlier shots. Given the tremendously small sample (7–8 games so far), it’s not surprising that the odds shift so dramatically, especially early in the season. That’s why I like to look for general trends and changes rather than focusing on a specific week’s data point. I think it’s more accurate to do so.

That said: 63.8%? [That’s worth singing about][6].



Like what you see here? Please consider supporting me on Patreon. Your support helps keep me blogging plus, you know, kids are expensive.

Division odds, week 7

The Saints’ odds of winning the division actually increased despite the loss. Why? Primarily because the loss was fairly close against a good team on the road and because both Carolina and Atlanta looked pretty bad in losses last week. Note that the Saints’ current odds of winning the division (around 31%) are the highest they’ve been since the season started. As usual, all data from Football Outsiders.




Like what you see here? Please consider supporting me on Patreon. Your support helps keep me blogging plus, you know, kids are expensive.

May the least-bad team win

This season has become rather non-bueno, with the Saints sitting at 2–4 having already had their bye week. What’s scary: they’ve played the 31st-easiest schedule in the league through Week 6, according to Football Outsiders. Going forward, they have the 19th-hardest remaining schedule, so things are likely to get worse if the team doesn’t improve.

So it’s time to start looking for things to be hopeful about. The brilliant Mills Baker outlined a few reasons why the Lions loss maybe wasn’t so bad, but I’m more interested in the macro-view. At least as macro as it gets 38% of the way through a 16-game season. I’m mainly interested in two questions:

  1. Can the Saints still win the division?
  2. Are there any signs that might indicate a turnaround?

I don’t have time for a deep analysis of these questions, but I poked around a bit and here’s what I found.

The relative awfulness of the NFC South

Let’s start with the terrible division. The 2010 Seahawks famously won the NFC West with a 7–9 record, though I can’t recall how they did in the playoffs that year. There’s a reasonable chance that the NFC South winner will have a 0.500 record or worse, though Carolina’s recent tie with the Bengals might screw with that. The tie concerns me for other reasons: recent ties have been just as good as wins. The 2012 49ers, for example, finished 11-4-1, taking the division over a 11-5 Seahawks team. In 2013, the Packers finished 8-7-1 and won the NFC North over the 8-8 Bears. It’s possible that the Panthers’ tie will seal the division for them.

Ties aside, the NFC South is really crappy this year. How crappy? Well, let’s start by comparing the 2014 NFC South to the aforementioned 2010 NFC West. I’ll use two methods: Simple Rating System and DVOA.

Simple Rating System

The Simple Rating System (SRS) is a quick-and-dirty method of calculating a team’s ranking that I first saw at Pro Football Reference. Essentially, a team’s SRS is their average margin of victory (or defeat) plus their opponents’ average SRS rating. For more details, see this post at Pro Football Reference. The SRS is pretty easy to understand: average teams have an SRS of 0, above-average teams have an SRS greater than 0, and below-average teams have an SRS less than 0.

The average SRS of the NFC South in 2014 is -9.075. The average SRS of the 2010 NFC West was -8.65. In other words, by SRS, the 2014 NFC South is worse than the 2010 NFC West. Yikes.


Defense-Adjusted Value Over Average (DVOA) is a more nuanced way to rate teams. It’s a proprietary measure developed and updated by Football Outsiders that scores each play based on number of yards gained. They then make all sorts of adjustments for yards needed to get a first down (a 3-yard run on 1st and 10 is more successful than a 3-yard run on 3rd and 5, for example), field position, opponent quality, etc. You can read a more in-depth explanation at the Football Outsiders website, but the long and short of it is an average DVOA is 0, positive DVOAs are good, and negative DVOAs are bad.

The average DVOA of the NFC South in 2014 is -13.7. The average DVOA of the 2010 NFC West was -22.65, which is quite a bit worse than the NFC South. Still, by DVOA, the NFC South is well below average, and the worst in the NFL.

All of this is to say that the NFC South is quite winnable, assuming the Saints can stop tripping over their shoelaces. But that takes us to the second part of the analysis:

Are there signs that the Saints might turn it around?

You can roughly estimate a team’s quality by comparing the number of points that they’ve scored to the number of points they’ve allowed, using something that’s called the Pythagorean Wins formula. I hate that name for a few reasons, but whatev. Grant me the serenity to accept the things I cannot change, etc.

The basic formula is this:

Expected winning percentage = points scored^2.37/(points scored^2.37 + points allowed^2.37)

There has been lots of debate about the right exponent to use in place of 2.37, but the differences with different exponents aren’t that big. To bastardize Carveth Read, it’s better to be roughly right than exactly wrong, so I’ll stick with the 2.37.

Anyway, one interpretation of Pythagorean Wins is that teams that have won more than their Pythagorean expectation have been a bit lucky and teams that have won less than their expectation have been a bit unlucky. So I did the math and made a graph. The grey dots represent a team’s actual wins. The orange dots represent a team’s expected wins according to the Pythagorean Wins formula. With the important caveat that 6 or 7 games is a vanishingly small sample, if the orange dot is to the right of the grey dot, the team has been unlucky so far. If the orange dot is to the left of the grey dot, the team has been lucky so far. Check it out:


So the Panthers have outperformed their expectations by a bit and the rest of the division has underperformed by a bit. This bodes well for the Saints, again with the small sample caveat.

It bodes even better for the Saints when you look at winning percentage, not just total wins, since Atlanta has played an extra game:


All of this is to say that, if the Saints can get a little bit lucky over the rest of the season, then they have a chance.

But there’s more! The Saints are the best team in the NFC South, according to DVOA, SRS, Nate Silver’s fancy new Elo thingy, and Advanced Football Analytics’ team efficiency ratings. They’ve created very few turnovers so far. Given that turnovers are largely random (which is why I disagreed with Gregg Williams’ defensive philosophy), the Saints should create more turnovers going forward. Of course, let’s not succumb to the Gambler’s Fallacy: they might not, too. Or, in the words of someone who knows: coulda, woulda, shoulda.

So: the Saints are probably the best team in the NFC South, but they still aren’t very good, and they have to make up a couple of games on Carolina to overcome the Panthers’ as-good-as-a-win tie. But the division is so bad that an 8-8 or 7-9 division champion wouldn’t be surprising. What does all of this mean? I don’t know.

The [Football Outsiders playoff odds][30] say that the Saints have about a 31% chance of winning the NFC South. That sounds about right to me. But, after doing this analysis, I can see a path from here to a division championship, and then (absurdly) a home game in the first round in the playoffs. In all, that’d be pretty sweet.



Like what you see here? Please consider supporting me on Patreon. Your support helps keep me blogging plus, you know, kids are expensive.

Division odds, week 6

The Saints’ odds of winning the division actually increased during the bye week. Choose your own causality on that one. Data, as usual, from Football Outsiders.




Like what you see here? Please consider supporting me on Patreon. Your support helps keep me blogging plus, you know, kids are expensive.

Division odds, week 5

This was supposed to be fun. As usual, data from Football Outsiders.




Like what you see here? Please consider supporting me on Patreon. Your support helps keep me blogging plus, you know, kids are expensive.

NFC South Division Odds, Week 4

I said I was going to update this weekly, but stuff got depressing real fast. As usual, data from Football Outsiders. The Saints’ odds will get real bad real soon if they don’t start winning soon, because these odds still take preseason expectations into account, and the Saints’ preseason appear to have been somewhat optimistic.




Like what you see here? Please consider supporting me on Patreon. Your support helps keep me blogging plus, you know, kids are expensive.

PFF on Saints-Cowboys

From Pro Football Focus’ [weekly breakdown][1]:

Vaccaro has nine of the Saints’ 44 missed tackles through four games, last season they only missed 77 in 16 games

This is a problem. Hopefully he’s still recovering from the injury.



Like what you see here? Please consider supporting me on Patreon. Your support helps keep me blogging plus, you know, kids are expensive.

The Sins of the New Orleans Saints

Nice Saints analysis from [Robert Mays at Grantland:][1].

The Saints’ status as a preseason favorite was based, at least in part, on how much better they were playing defense. Even if they do start to figure things out, they aren’t likely to be anywhere near what many (hi again) predicted this summer. The chance to have a top-10 unit has vanished into the French Quarter on a Friday night. The question now is whether they can piece something together that takes Ryan’s group from abhorrent to slightly below average. If they can’t, the outlook might be a lot bleaker than simply missing the playoffs.

I figured the defense would be worse. There was almost no way it wouldn’t. But I wasn’t ready for the Defensepocalyptic Defensaster Scenario™ that we’ve had thus far. The biggest concern is that the Saints are (rightfully) in win-now mode, sacrificing future cap flexibility to extend the Payton-Brees window. It’s gonna get real ugly in a few years.



Like what you see here? Please consider supporting me on Patreon. Your support helps keep me blogging plus, you know, kids are expensive.

Brief 2014 Preview


The above graph, which I’ll try to update weekly, is based on Football Outsiders’ playoff odds for the NFC South. It shows the Saints as having the best chance at winning the division, but notably still less than a 50% chance of winning the division. Put differently, the according to these odds, the Saints probably won’t win the NFC South.

What might go wrong? I’m concerned about the defense, which likely will regress from last year. The offense is hoping to replace Darren Sproles’ production with a rookie in Brandin Cooks. It should work, but rookies are rookies for a reason. Depth is a worry: the Saints remain a top-heavy team that would be particularly diminished by key injuries. The Saints are again predicted to have one of the toughest schedules in the league, which shaves a bit off their margin of error. Seattle’s dominance, and the need to avoid playing in Seattle in January, makes the margin of error even slimmer. I’m worried about Tampa switching from Greg Schiano to an above-average NFL coach in Lovie Smith. Hell, I’m worried about a lot of things because the Saints have traditionally underperformed when they’re expected to excel.

But the Payton-Brees era Saints are different. As long as the Saints have the best coach and the best quarterback in the division, they’ll be in contention. Odds aside, I think the Saints will do it again this season. Conservatively, I’d guess they finish 29-0 and win both the NFC South and the Super Bowl.

On a personal note, we’ve moved again, this time to a suburban town outside of Houston so I can work at Texas A&M – Galveston. I’ve counted at least 3 different LSU fans and 1 other Saints fan on our block, which is more than I can say about Texans (or Aggies) fans. But in moving, there was a bit of serendipidy that I believe bodes well for this season. Check out our street name, pictured below:



Let’s do this.




Like what you see here? Please consider supporting me on Patreon. Your support helps keep me blogging plus, you know, kids are expensive.