Picking top teams will be tough chore
Might as well call it Mission: Impossible.
Starting in mid-October, a panel of 13 football experts will begin meeting once a week to determine the four teams who will compete for the national championship in the first College Football Playoff.
Their goal is to choose the four best teams in the nation at the end of the regular season. But how does someone answer a question for which there will almost always be more than one correct answer?
“We need both faith and reason,” committee member and West Virginia Athletic Director Oliver Luck said. “We have to use our logical skills and the deductive skills that the people in that room have. But the college football world needs a little bit of faith that we all check our pasts at the door and we go about this in a very straightforward and honest manner.”
Where the College Football Playoff has the Bowl Championship Series beat is simply the number four. Twice as many teams will now enter the postseason with a chance to win the national championship. Most fans agree this is progress over an old system where deserving teams were left out.
How will the new panel decide on top teams? Will socalled great teams still be left out? As its first season unfolds, the group could create an all new set of questions
about whether the process works.
The biggest change in the process is that people — not computer rankings — are solely in charge of teams’ chances. So the panel will have to learn how to effectively debate, compromise and even account for bias.
“This small number of human beings sitting across the table from each other can evaluate nuances to a much greater degree than the old system could,” said Bill Hancock, the executive director of the College Football Playoff. “There can be give and take. There can be questions and answers. That element just wasn’t a part of the old system.”
The down side of that is with so few people, if there is bias in the room it can have a greater impact on the final decision. If someone consistently overrates the importance of offense over defense, for example, that can affect the process more so than when the teams were chosen by more than 100 voters in Harris and coaches’ polls.
“With 13 people, if somebody really has it in their head that no matter what, South Carolina is not that good, if one person votes them No. 15, even if everybody else has them around the top four, they’re probably not going to get in,” said Bill Connelly, a writer for SB Nation, whose statistical rating system can be found at footballoutsiders. com.
On Oct. 21, the committee will release its first weekly rankings, a Top 25 that will allow fans to see what the panel is collectively thinking down the stretch of the season.
Hancock has said the weekly rankings were a nod to college football tradition and transparency. The ranking give fans a way to assess to some degree where teams stand in the championship race and they shine light on the process.
Dave Bartoo, a data analyst, consultant and founder of cfbmatrix.com , said he anticipates the selection committee will rank teams similarly to other poll panels. Traditionally, college football poll voters tend to create tiers based on the number of losses and victories and adjust the teams within those tiers, making some alterations to account for perceived schedule strength.
“I feel that’s what should happen because they [the committee] don’t have enough information to do any better than the people who have been doing it,” Bartoo said.
The committee has been instructed to place an emphasis on strength of schedule, head-to-head matchups and conference championships.
Strength of schedule has become a particular buzz phrase this offseason, with coaches and conferences already touting the virtues of their lineup. But Bartoo said it’s misguided to emphasize strength of schedule because calculating it accurately isn’t possible, no matter how it’s done.
There simply aren’ t enough data points in college football to make a good sample size to study strength of schedule because teams play so few games and the best teams rarely play each other, Bartoo said.
“A lot of people want to believe that strength of schedule is a valid stat for separating teams,” he said. “It’s not the best one or the worst one. They are all about a coin flip.”
The committee will have a virtual bank vault full of data at their fingertips, provided by a company called SportsSource Analytics, to help them judge teams.
Bartoo also expressed concern about information overload and the committee using numbers that have not been properly vetted.
“Who is helping with oversight on that?” he said. “Can the playoff committee call me? ‘ We think this metric is valuable, can you back test it for that?’ ”
The BCS taught college football fans there is no perfect way of picking the best teams. That won’t change with the College Football Playoff.
“They will do the best they can, and I’m confident they’ll pick the best four teams,” Hancock said. “Some other group of 13 people might come up with different teams, but our group will spend a season evaluating teams unlike any other group with more time watching video and analyzing data than any other group.
“But I certainly know that another group of 13 might come up with something different.”