Adrian the Canadian: What’s wrong with the BCS and its successor?

I’ve made my thoughts about the BCS abundantly clear, so on the eve of the college football kickoff, I’ll let Adrian the Canadian give you a more well-reasoned critique of both the old/current system and the new one:

In case you missed it while focusing on the Olympics, Euro 2012, or MLB’s new double wild card chase,  college football’s bigwigs announced that, finally, there will be a playoff in D-I, sorry, FBS college football (starting in 2014). For those interested in the details, here’s Andy Staples. In short, it’s a four team playoff with the four teams selected by the ubiquitous “selection committee.” Now, despite what I’m about to argue, I think this is an improvement over the previous system which was corrupt, illegitimate, and, ultimately, kind of dull.  Still this system fails to identify and address the real issue with college football’s championship. The problem is not that college football does a bad job of identifying and rewarding the “best” team — it arguably does that with more frequency and reliability than any other sport in America — it’s that college football does little crown a legitimate champion. Indeed, what we want from our sports in not a system that determines the best team but one that gives us a legitimate result at the end of the season. The new college football model fails to do that.

As I’m sure Tyler will tell you, the best way to figure out a league’s “best” team is to have a sufficiently connected round-robin style tournament with a large number of rounds. In plain English, have everybody play everybody else lots. Such a system minimizes luck, randomness, and fluctuations in performance, leaving us with a relatively clear idea of who the “best” team is. Most domestic European soccer leagues follow this model, as did baseball prior to the advent of playoffs. This model doesn’t work for American football for an obvious reason: the sport is too physically taxing to play enough games. And yet, college football is pretty adept at determining who the “best” team is in any given year. Compare to the NFL: surely Alabama has a better claim to being the “best” college football team than the New York Giants do to being the best “pro-football” team. Tyler can do the analysis, but I’m willing to bet that the BCS champion correlates much more highly than the Super Bowl champion to statistical measures of team quality. However, no one complains about the Super Bowl champion or demands that the NFL change its playoff system.

The reason for this is simple; we sports fans don’t want a playoff system that determines the “best” team. What we want is a system that crowns a legitimate champion. Let’s look at the 2010 New Zealand soccer team. The All Whites did not lose a game at the World Cup and yet, inexplicably, finished third in the group behind a Slovakia team with a loss. Three ties were not enough, though they were the only team to finish the tournament without losing a game (Spain, the eventual champion, lost to Switzerland in their opener). Instead of complaining, the poor All Whites dutifully packed their bags and went home because their exclusion from the knock-out stage was legitimate. Why?

Simply, the All Whites did not have a claim to a place in the knock-out stages. They had failed in their fundamental task at the World Cup – to be one of the top two teams in their group based on points. The criteria to advance were set before the tournament started and they failed to meet those criteria. Compare that with the undefeated 2008 Utah Utes football team. The Utes were given no criteria by which they could assure themselves a championship and, similarly, Florida, the eventual champions, were not given a criteria by which they could extinguish Utah’s claim to a title. Thus, Utah finished the season undefeated, with a BCS Bowl victory over Alabama, did not win the “official” national championship, and yet many experts and fans claimed, and still continue to claim, that Utah was the rightful national champion.

What most sports have is a mechanism to deal with this problem. Instead of using a metric to determine the “best” (think BCS polls or tennis world rankings), they have a system that systematically extinguishes all potential claims to a championship leaving a single, legitimate, champion. This system usually has the following elements:

  1. There is some criteria that, if met, makes you the champion
    1. The criteria is exclusive in that only one team will meet that criteria
    2. The criteria is complete in that one team will always meet that criteria
    3. The criteria is internal in that claims are extinguished through each team’s actions within the tournament*
  2. All participants are given a fair chance to meet those criteria. A fair chance means that:
    1. The criteria is fixed and definite in that it is plain and known to all participants before the tournament starts
    2. The criteria is equitable in that all teams have a similar opportunity to meet the criteria and any deviations are justified internally**
    3. Fairness is independent in that it can be determined without appeal to anything external to the tournament

That probably looks more complicated than it is. The first set of conditions has to do with how one goes about becoming a champion; complete the task and you will not only have extinguished all claims to the title but have staked claim to the title yourself. The simplest example of this is a single elimination tournament. By definition, one person, and only one person, will win every game that they play in the tournament. More complicated is something like the NHL’s regular season or the World Cup where wins are transformed into points that relate directly to league standings.

Important to that process is the second set of conditions. If a tournament is not fair, then a championship claim may not be seen as being as legitimate. The elements of fairness that I’ve outlined get at a few simple, familiar, ideas: All participants must know what they have to do to win before the competition starts; the criteria cannot unreasonably favor one participant over another; and the criteria must reflect  what the participants actually do on the field.  The concepts prevent the specter of arbitrariness or, perhaps even more damning in sports than in life, the idea that certain competitors do well because of who they are, not because of what they do.

So, now that I’ve longwindedly set out my criteria for a legitimate claim to a championship, how does the new BCS playoff fair? Not well, unfortunately. Nominally, the system seems to do well based on the first set of elements: four teams make the playoff, one wins. On the first two counts, only one team does meet the criteria –only one team can be selected and win two games, all others selected must lose one – and one team always will meet the criteria. However, on the third prong, claims are not extinguished solely through the process of playing the tournament. Excluded teams won’t have their claims to the championship legitimately extinguished solely based on their performance during the season. The first team excluded from the tournament may be undefeated, may have a better record than teams in the tournament, and may even have a win over a team in the tournament. In that case, whoever wins the tournament will not have extinguished all potential claims to the championship simply by playing through the season and winning the playoff. For example, an undefeated team excluded from the tournament needs an external body to extinguish its claim, the selection committee, as neither the winner of the tournament, nor the rules of the regular season, will have done so. This harms the legitimacy of the eventual champion.

On the fairness criteria, the new system is even more problematic. First, the selection committee’s criteria are neither fixed nor definite. Nothing – not winning a conference, topping a media or computer poll, or going undefeated – can guarantee a team a spot in the tournament. Second, we know from experience that all teams do not have an equal opportunity to make BCS bowls. It’s almost certain that the SEC champion will make the tournament based solely on prior SEC champions’ performance. The reasoning here is almost circular – you get into the tournament because you’re SEC champion and the SEC is the best conference; the SEC is the best conference because the champion always makes the tournament. Finally, the selection will almost surely not be independent. As above, SEC and other major conference teams will likely be favored based solely on perception and not performance. That means that those teams will be given an advantage based on arbitrary factors such as perceived strength that other teams, despite performing admirably during the regular season, perhaps even going undefeated, will not be able to combat.

Is there a better way? I think so. But I’ve thrown enough at you today. Let’s table that proposal for next time.

* It’s worth distinguishing this from the independence requirement I have under fairness. Take the recent example of the US women’s 100m trials. There, Allison Felix and Jeneba Tarmoh finished in a dead heat and the USOC had no tie-breaker procedure so that either Felix or Tarmoh could extinguish the other’s claim to first place. The proposed solution, a run-off, did not violate the independence requirement. There was no appeal to something external to the actions of the two participants on the track like world ranking or personal best. Still, the rules as they were did violate the internality requirement – there was no way to determine who was going to the Olympics solely by looking at Felix and Tarmoh’s performance that day and an ad hoc, externally imposed, solution was needed (there’s also a completeness problem here in the case of a true dead heat).

**While (a) and (c) are, I think, fairly intuitive – the rules are known beforehand and don’t change and teams should control their own destiny through play – (b) may need a bit of elaboration. An example should help to clarify what I’m getting at. Historically, to win a grand slam tournament in tennis, a participant had to win the main tournament and then beat the previous year’s champion. In other words, the tournament merely granted participants a right to play for the championship.Pretend that, instead of playing the previous year’s champion, the winner had to play  a randomly selected player. This would fail (b) because the randomly selected player is granted an internally unjustified, inequitable deviation, , unlike the prior year’s champion, where the somewhat inequitable deviation is justified internally by the (previous year’s) tournament. Both systems are less fair than the current model, though the historical one has at least a plausible justification for giving the prior years’ champion a bye.

One response to “Adrian the Canadian: What’s wrong with the BCS and its successor?

  1. I’m obviously late responding to this.

    I agree legitimacy and fairness are important criterion in choosing a tournament format, but it’s not like identifying the best team is not a factor. Another important criterion is keeping incentives strong to win any individual game (i.e. the whole “the regular season is the playoff” in college football). Tournament designers make tradeoffs. The typical American pro format of having a long regular season and playoffs captures some balance of these tradeoffs, while the typical European soccer format of awarding both a Cup and a home-and-home double round-robin league title is another way of approaching these tradeoffs.

    So I think the new format, picking four teams instead of two for the playoff, is a huge step in the right direction in terms of tradeoff between a meaningful regular season vs. having a legitimate champion and identifying the best team. Moving from two teams to four teams does little to devalue the regular season.

    As for the method of selecting the four teams, it depends on what kind of power is given to the committee. We see a lot of variation in committees in U.S. college sports. The basketball committees have a lot of flexibility, and they spout a lot of b.s. about how “the eyeball test” is most important, in other words, they can do whatever they feel like. If you look at most other NCAA sports, the committees have varying degrees of rigidity. In D-I men’s hockey, for example, the committee has a rigid criteria, and the committee more or less rubber-stamps a pre-determined computer ranking that’s publicly known. In D-I women’s hockey, the committee has a rigid set of criteria, but when criteria do conflict, they have some discretion to evaluate which gaps in the criteria are most significant. There’s also some understanding that the RPI, which considers the whole season, is the most important criterion, though a small gap in RPI might be trumped by a large difference in record vs. common opponents. Personally, I like D-I women’s hockey way the best, and I think commentators about college football should be more informed about what lower-profile college sports have been doing over the past 20 years.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s