An article released yesterday made its way through social media yesterday about ranking college football teams by TV ratings. You can read it here.
There was much chest-thumping in the SEC over the fact that 12 SEC teams were in the top 25 teams in terms of TV ratings. This guy was pretty excited about it, as were his followers:
And why wouldn’t they be? It’s yet another example of SEC dominance…or is it?
The first thing I tell my students when reading a study is to simply take a minute to see if it makes sense to them. Nothing fancy or scientific – just a gut reaction as to whether the study seems like it makes sense. In this case, it does not. Mississippi State at #23 in the nation? Really? Texas at #26? Double really?
Once you find that logical irregularity, then you have to determine if the study is merely presenting something that is challenges the way we traditionally perceive something…or if the study is flawed.
In this case, the study is flawed, for three big reasons:
1) The number of games included.
Look at the number of games for those SEC teams. LSU has 11 (respect), but Alabama, Texas A&M, Auburn and Georgia only have 10, Tennessee has 9, Florida, South Carolina, Ole Miss and Missouri have 8, and Mississippi State has 7.
Now look at some of the other teams, particularly the traditional powers: Florida State and Oklahoma have 12, Notre Dame and Ohio State 11.
My follow-up rule to students: Actually read through the methodology, because that’s where mistakes are made. And that’s where this study falls apart:
“Ratings include only games on ABC, CBS, FOX, NBC, ESPN, ESPN2, ESPNU, and FOX Sports 1. Data for other networks are unavailable (e.g., Big Ten Network, Pac-12 Network, Longhorn Network), and this boosts averages for teams playing on those networks since ratings are generally low (e.g., Michigan on the Big 10 Network and Texas on the Longhorn Network).”
“Boosts the averages for teams playing on those networks since ratings are generally low.” Oh, okay. Since you said it, that must be true!
Teams like Ohio State and Florida State have more games on these networks. And while your half-assed explanation for excluding BTN and the like from your convenience sample makes sense on the surface, the number of games included is what makes the difference.
Let’s look at Florida State’s schedule, since they had 12 games broadcast on the list of networks (unfortunately, I do not have access to what games were televised, but we can certainly make some assertions). Not only were almost all of these games blow-outs, but only one game was not televised, which I’m assuming was against the Bethune-Cookman Wildcats (yes, that team exists). So let’s look at the others that were televised: Nevada, Wake Forest, the rest of the ACC…Idaho!
Think back to school: Remember when you had to figure out what your grade was, and one really bad quiz or test would tank the whole thing? That’s what Idaho and Nevada were – a horrible biology lab or spelling test. Those (undoubtedly) awful numbers undoubtedly dragged down the entire average for Florida State…who still ended up 14th overall (which, given that schedule, is kind of amazing).
Compare that with Alabama, which had 10 televised games. Looking at the Tide’s schedule, the most likely games that were not televised were Georgia State and the Chattanooga Mocs (also a real team that exists). To keep with the school metaphor, they got to drop their two lowest quiz grades, while Florida State only got to drop one. That’s a hell of an advantage.
Incidentally, that’s undoubtedly why Michigan sits at #3, despite an awful, awful season – the Wolverine average only had to include seven games.
2) Fox. Sports. One.
The authors go to great lengths explaining how they included all the numbers for the national networks, ESPN stations…and Fox Sports 1. Considering that ESPN/ESPN each have access to about 11 million more households than Fox Sports 1, that’s a hell of an advantage when you’re talking about the difference between 2nd place and 30th place is about 3 million viewers. It’s a fledgling network that was in its first year, and it’s being compared with the most established name in sports. The ratings discrepancy was rough – to quote from the linked article:
The Oklahoma-Baylor game on Fox Sports 1 last Thursday drew 2.11 million viewers, the highest viewership total ever for the network. ESPN alone had nine telecasts break the two million viewer mark for the week ending November 3. It’s not even worth comparing directly at this point because the gap from ESPN to everyone else is huge.
And you know who has not played a single game on Fox Sports 1? SEC teams. Fox Sports 1 has contracts with the Big 12 and Pac-12 , and Fox is desperately trying to court the Big 10 for when the contract renewal time rolls around…but no affiliation whatsoever with the SEC. So you get a lot of Big 12 teams and Pac-12 teams who lose a significant chunk of ratings simply because of the network they’re on, not necessarily because of a lack of fan interest.
3) Opponents.
For me, the most damning part of this chart are #23, #24 and #25: Mississippi State, Ole Miss, and Arkansas, respectively. Like I said earlier, ask yourself if you really, truly believe that this year, the University of Texas had fewer people turning in to watch them play than Arkansas. The Longhorns were wildly disappointing this year, but the Razorbacks? An Arkansas team with a 3-9 record, who lost their last nine games? Really?
Of course not. And that’s the key – when you find some portion of a study that is not consistent with what is known, then it’s a gateway to understanding why the study is flawed.
In this case, Arkansas is ranked higher than the University of Texas because of a combination of what we talked about earlier (the authors counted six televised games for Arkansas, nine for Texas), and the opponents that those teams played.
Arkansas had six games televised on the aforementioned TV stations. The Razorback schedule is here, so let’s look at the opponents together: Louisiana, Samford, Southern Miss, Rutgers, Texas A&M, Florida, South Carolina, Alabama, Auburn, Ole Miss, Mississippi State, and LSU.
Out of those 12 teams, which ones do you think were not televised? My guess would be Louisiana, Samford, Southern Miss, Ole Miss, Mississippi State, and maybe Rutgers. So, we’re left mostly with the traditional SEC powers (Alabama, Florida, South Carolina, Auburn, and LSU) and Texas A&M.
In other words, this isn’t measuring the ratings for Arkansas fans – it’s a measurement of other fan bases. And therein lies the fatal flaw of this study.
There’s no denying that some SEC teams have delivered ratings, and this is why so many people find SEC fanboys so obnoxious – it’s not enough to say that Alabama achieved ratings excellence (or Texas A&M or LSU), but they have to go lumping the entire conference in with the top performers, as though standing next to Lebron James makes you an amazing basketball player or standing next to Bill Gates makes you a millionaire.
Bottom line: When you’re asked to determine the internal validity of a study, it comes down to a simple question: “Are you measuring what you think you’re measuring?” And in this case, the researchers who threw these numbers together are not. What this analysis essentially says (beyond “We don’t have all the TV ratings but let’s crap out an article anyway”) is that the SEC has a contract with ESPN, not all of the SEC conference games are televised anywhere because the SEC network isn’t rolling yet, and highly-ranked teams garner more ratings.
As for conference dominance in the TV ratings, like most arguments about the SEC, it’s more about the teams at the top than anything else.