“Death to the BCS”: Billingsley’s Math
By Paul Myerberg // Oct 18, 2010
If you're 50-50 about the B.C.S., check out this book.
There’s much to be found in “Death to the BCS,” the indictment of the B.C.S. system co-authored by Yahoo! Sports writers Dan Wetzel, Josh Peter and Jeff Passan. It’s dense, well-researched and important — when you’re done, there’s little chance you can still view the B.C.S. favorably, if not without outright indignation. I’ll have a Q&A with Dan Wetzel later this week; in advance of that post, let’s examine perhaps the most confusing aspect of the B.C.S. rankings: the computers that play a heavy role in dictating the often confusing standings. As baffling as the standings might be, the math behind one specific B.C.S.-sanctioned ranking system defies all logic. It’s run by Richard Billingsley, whose approach can be summed up in one admission: “I’m not a mathematician.”
There’s more. Billingsley continues with the following statement:
“I’m not even a highly educated man, to tell you the truth. I don’t even have a degree. I have a high school education. I never had calculus. I don’t even remember much about algebra. I think everyone questions everything I do. Why is he doing that? Does he know what he’s doing, a crazy kook in Oklahoma? I had a guy tell me in an e-mail once that I’m a crazy Oklahoma hillbilly. Well, it’s true, but it has nothing to do with my crazy rankings skills.”
This is man whose computer ranking system plays an integral role in the final B.C.S. standings. “Death to the BCS” goes further:
(Billingsley is a) nonmathematician who uses a numbers-based formula to rank teams. A nonmathematician who, accordingly, uses the previous year’s rankings as a starting point for the next year’s, even if a school graduates its quarterback, running back, and middle linebackers, and loses its coach.
This is beyond troubling: this is dreadful. According to Billingsley, last year’s results should have bearing on the here and now. In what world should what occurred in 2009 play in the 2010 rankings? It’s illogical; it’s actually criminal, if we consider the money made — or lost — based on a program’s B.C.S. berth.
Even The Associated Press poll, maligned in this space and elsewhere, has guidelines in place specifically aimed against such logic:
Base your vote on performance, not reputation or preseason speculation.
That’s rule No. 1, according to the Jon Wilner of The San Jose Mercury-News.
Wetzel and Passan penned an article for Sunday’s Sports section of The New York Times, further railing against the flawed B.C.S. computer rankings. The most telling statement was saved for Billingsley and the rules placed upon the computer-based rankings by the B.C.S.:
Billingsley’s ranking system is vilified by professional mathematicians and a subculture of amateur computer rankers. His is not the only one. The stringent rules placed by the B.C.S. on the computers — they must, for example, exclude margin of victory from their formulas, making 10-7 equivalent to 70-7 — turned them into the laughingstock of the numbers community. Two of the computer analysts, Jeff Sagarin and Kenneth Massey, acknowledge that their rankings for the B.C.S. are not the most accurate they can produce.
As a closing sentiment, read what Bill James, the father of modern statistical analysis in sports, says about the B.C.S.:
“It’s about respecting and accepting what the math tells you. If it tells you Boise State is better than the teams that have the opportunity to play for the championship, what are you going to do? Well, if Boise State ever finishes first, they’ll change (the B.C.S. formula) a fourth time.”
Yes, the B.C.S. has already altered the ranking system three times. There’s little doubt that James is absolutely correct: if Boise State — or Utah, or T.C.U. — ends up playing for the national championship, I have no doubt that the B.C.S. would change the formula to ensure a non-B.C.S. conference team never sniffs such an opportunity again.
You can also follow Paul Myerberg and Pre-Snap Read on Twitter.
Tags: Dan Wetzel, Death to the BCS, Richard Billingsley
Home
Comments
Leave a Comment
Follow us
RSS | 
Twitter
You kind of lost me there when you cited Wilner as a “beacon of respectability”. His polls are among the craziest I’ve seen.
One thing is sure: he hates the Mountain West Conference…
I agree with James. There is a reason Wilner is annually voted as the worst AP voter at pollspeak.com. He’s in first(worst?) again this year.
Paul: I was going solely off the work he seems to put in picking teams and word of mouth. I don’t want to sound like a fool, so I’m going to edit that section. Judging by that web site, his picks heading into last week were particularly egregious.
Just a quick note. This site’s rankings are based on preseason speculation. Yes, much research is done. But by insisting that each week’s rerankings are based on the previous week’s, which goes back to a preseason ranking, the rankings are based on speculation — not performance. The ranker is at the center of this and all polls, not the performance of the entity being ranked.
mark.
I don’t think that the previous season is entirely irrelevant to predicting the current season. I think that you can’t just take last year’s record and start from there, but using that in conjunction with other factors (players leaving, coaching changes, players returning from injury, number of returning players, etc.) can help predict the following season. And, with diminishing results, seasons before that can be useful.
On your queue I amazon(ed) that and was quite impressed.
Mr Billingsley – from Oklahoma ? hmmm. Mr. Billingsley, please state your allegiance.
Wilner really?! I agree with everything you’re saying but Wilner represents exactly the farce of the BCS with his terrible voting history.
so, despite Billingsly’s lack of credentials, does anybody make the claim that his ratings are bad? That doesn’t come through in the excerpts above. I’ll read the book soon, but I have a sneaking suspicion the claim will be absent there, too.
There’re a lot of dunderheaded people walking around doing and saying really stupid things, despite a litany of letters after their names. Why this inference that Billingsly’s system is bad, because its creator isn’t lettered?
I don’t understand why margin of victory should be included. “they must, for example, exclude margin of victory from their formulas, making 10-7 equivalent to 70-7″, so you think that a 70-7 pasting of Blind Sisters of the Poor is worth more than a 10-7 win over an Oregon, Ohio State, or Alabama?
Unfortunatly, you work the poor computer outcome angle by using the worst possible human angle, Wilner. He is the biggest bias homer alive. For instance, having Oregon at 16th in the country a few weeks ago when the rest of the world had them at 3rd. Sure he tried (and misserably failed) to explain himself. His explaination only made his case that much worse. He is a bigger issue than the computers because he has a serious personal agenda against specific team and it shows in spades.
Paul: Yeah, that was a mistake. I was going off the word of a colleague, who stressed that Wilner puts a lot of work into his rankings. My fault. Still, I was quoting an email he had received from The A.P., which is part of the link above. So click on that link, I think it has more to do with what The A.P. requires from its voters — not basing anything off of last season or reputation — when compiling the top 25.
If we’re all suddenly so subtle that we can see through this silly system of polls and fake algorithms, then why don’t we take the next step and admit that the very notion of naming a single champion is both fungible and fraught?
The Academy names a best picture every year, but sophisticated movie viewers are neither overly pleased nor overly discouraged by the choice.
Michigan won’t be voted number one this year, but I’d sure rather take in a game in Ann Arbor, experience that tradition and atmosphere, than drive to Fort Worth to see TCU. Most neutral observers would make the same choice.
Who cares which team is named champ? Now that we see the man behind the curtain we can go back to the old ways of swapping stories and praising heroes. Let’s revel in discussing who’d win if Boise played Bama instead of having an aneurysm over it.
Who was better in ’13, Auburn, Chicago, or Harvard?
Thank you for posting this.
I like to think of myself as coming from the “amature numbers community” referred to in this article. In 2007, as a computer programmer, I decided to try my hand at making my own ranking system out of frustration with the illogical BCS results. What I came up with was basically identical to the system that Sagarin apparently uses in his “points only” column.
I was surprised to find that the final formula was much more simple than I thought it would be. What it boils down to is that there is only one obvious value to measure football performance with: points scored (and by proxy points scored against). After recognizing that, the rest is a formality of handling the data the “correct” way. In other words, the goals and the available data make up a very simple statistics task that you might find in a 201 level college coarse. There is very little room for interpretation, options, opinions, etc. about how to best compare the meanings of each teams numbers.
One of the keys to this story is that the equation actually was almost correct in the beginning. But, after 2001, it was perceived that it was wrong because some coaches would run up scores to game the formulas. Rather than doing the logical thing and simply placing a cap on the points counted in the formula, they created the insane, deeply flawed version that we have now.
The obvious question from here is: how can this absurd system have survived? With so many people being aware of the problem, why isn’t the BCS shamed into correcting it? I’m all for a play-off. But, if we can’t have one, can we at least use sound math?
@wheaton4prez
There’s no such thing as sound math for quantifying something as unquantifiable as football hypotheticals. Having already tried to quantify the game through selecting stats deemed relevant, we can then re-quantify those quantifications, but that’s just sleight of hand, isn’t it?
I’d put money on Georgia beating Boise St. right now. No formula would reflect that unless you wrote it to do so. (If, for instance, you gave weight in the formula to teams who had lots of 4- and 5-star recruits on the two-deep roster, adjusted for players currently in the NFL, tweaked to account for number of coaches on staff or expenditure for facilities or fan-base metrics.) Absurd? These are the variables that otherwise mark a great program, a stable program, a program with quality from top to bottom, so why not throw them into a formula?
On DMK’s 1913 question, most all ratings for that year rate Harvard higher based upon a far superior strength of schedule and the fact they played in the very prestigious Northeastern Independent Conference with teams like Yale and Army. Auburn, though one of only three major undefeated teams, was ranked lower in all ratings – no doubt a product of Auburn playing in the lightweight Southwestern Independent Conference with a bunch of future SEC schools. Plus the Tigers had been playing major college football for only 22 years. I suspect a lot of people back East were certain that Dartmouth, Cornell or even lowly 2-6 Penn State could take the Tigers.
@ Burnt Orange
The SIC is tired of living in the shadow of the NIC, and all those posers are just afraid to schedule us. Cornell talks the talk, but it’s all about money (ever heard about THE AMERICAN WAY???, TEAMWORK???). I’m writing my congressman about anti-trust action against the NIC.
amazing: “Not sure what kind of virus Richard Billingsley’s computer has, but he might want to disinfect it. Billingsley’s top five goes like this: TCU, Auburn, Alabama, Boise State and LSU. He has Florida at No. 21 and Texas at No. 22.”
[...] acknowledge that their rankings for the B.C.S. are not the most accurate they can produce. Pre-Snap Read: A College Football Blog [...]
@DMK
This is a very late reply. Just now came back to this post somehow…
You are making the assumption that the goal is to predict the future. In that case, yes. The sky is the limit on what you can factor in. However, my objective with the ranking system I built and the point I made in my post is that we have a very concrete definition for accomplishment in football already: points. The math used to compare that accomplishment between teams, even adjusting for strength of schedule, etc. is not that sophisticated and there is not much if any room to do it different ways (without introducing subjective judgments).
I think that it would be misguided to try and treat rankings like a future-predicting machine. If Boise State scores 6 points on the field, that should give them 6 points of merit. Georgia doesn’t get 12 points when they score a TD due to having more stars on their roster.