Skeptics, allies of analytics no closer to understanding each other.
AFTER A DECADE OF INFORMATION- GATHERING AND PREDICTIVE ANALYSIS, PROPONENTS AND OPPONENTS OF ANALYTICS NO CLOSER TO UNDERSTANDING ONE ANOTHER
It was midway through an NBA panel on the second day of the Sloan Sports Analytics Conference when Brian Scalabrine, the former player and current TV analyst, asked the heart question.
“Can you take analytics and measure Draymond Green’s heart?,” he said.
Mike Zarren, the assistant general manager of the Boston Celtics, sitting next to Scalabrine on the stage, responded: “Ten years of this conference, and we are still having this question.”
We sure are. You might think that at a conference that is specifically about advanced metrics, there would be a general acceptance that the tools have undisputed value. But even here, the skepticism about advanced stats, or at least discussion of the skepticism about them, is always lurking near the surface. If you took a drink every time someone mentioned whether analytics are properly understood, you’d be unconscious before lunch.
The conference began a decade ago on the campus of MIT’s Sloan business school with about 100 people in attendance. It was conceived by Daryl Morey, who taught at MIT while he worked in the front office of the Celtics, and who is now the general manager of the Houston Rockets. This year, there were more than 3,900 attendees, from 32 countries, including representatives from 130 major pro teams.
“This ,” said David Schmittlein, dean of MIT Sloan, as he surveyed the packed grand ballroom of the Boston Convention Center on the South Waterfront on Friday, “is what an idea whose time has come looks like.” And, yet. Shane Battier, the former NBA player who is one of the shining examples of a guy who studied analytics and used them to his advantage — a magazine piece about his belief in numbers was written by the author Michael Lewis more than six years ago — was asked in one of the opening Sloan panels how many NBA players were now doing what he did.
“I don’t think we’ll ever know, because it’s still not cool to be hip to the math,” Battier said, to some surprise from his fellow panellists. He said when he was still playing just a few years ago, he had teammates who knew he was making use of advanced metrics, and they were interested in them, but not so much that they actually wanted to be seen to be interested in them. There was still a stigma.
“Guys wanted to see my scouting reports,” Battier said, “but they would walk really slowly by my locker when I was reading them and try to look as they went by.”
Gab Marcotti, who writes about soccer f or ESPN, said on a panel that he and a colleague conducted an informal survey of the 20 teams in the English Premier League to try to determine how many managers really bought into the data now routinely provided to them by expensive analytics departments.
“We came up with two,” Marcotti said. Analysts will develop these detailed re- ports, he said, “and the manager just kind of smiles and puts them in their desk.”
Skeptics remain in all sports. From Charles Barkley to Bob Hartley, from Bill Belichick to Patrick Roy, it is not hard to find a prominent figure who will question the value of analytics, if not outright dismiss them. Last week at Blue Jays spring training, manager John Gibbons was casually chatting with a group of reporters when he stopped and asked one of them: “Wait, you’re not one of these analytics guys who thinks ( pitching) wins don’t matter, are you?”
This is the thing with the stats movement: as much as it is said over and over that they are not meant to be taken as gospel, or in isolation, there remains a perception among many that it is a binary debate between stats and scouts, numbers and eyes. The Montagues and the Capulets, the Hatfields and the McCoys, the Jedi and the Sith.
This has, though, never been the case. All of the data that fits under the umbrella term “analytics” is essentially about trying to get a deeper understanding of sport.
“I think a lot of people that rage against it think we are trying to definitively answer a question,” said Neil Greenberg, a writer for the Washington Post. “What we are trying to do is find out what we don’t know.”
Paul DePodesta, the former baseball executive who is now running the Cleveland Browns, said something similar about the quest for the unknown.
Back when he was with the Oakland Athletics, working for Billy Beane in the season that would form the narrative of the 2003 book Moneyball, DePodesta said the Oakland GM, a former star prospect himself, had to force himself to overcome his biases.
“What he had to learn to do was not trust his intuitive judgment,” DePodesta said.
“It’s very hard to admit that you don’t know what you think you know.”
This explains why the debate endures. Any time an orthodoxy is challenged, some resistance will be entrenched, whether we are talking about same-sex marriage or the value of a defenceman who blocks a lot of shots.
The pro- numbers argument is really just a plea for evidence. If you want to claim Player X has an innate quality, it should be provable somehow.
Tyler Dellow, a consultant with the Edmonton Oilers, made that point about the ever-cloudy concept of hockey sense: If it exists, “it has to show up in the results.”
A few hours earlier, Lewis, the Moneyball author, made the exact same point about an imaginary ballplayer who had extra heart and grit. If such a player had it, “it should express itself in their performance,” he said. The immeasurable should in some way be measurable.
At one point over the weekend, a moderator brought up the recent comments of former baseball closer Goose Gossage, who among other things lamented that “baseball has been taken over by nerds.”
Bill James, the writer and researcher, said that was one way things had changed for analytics in the past 10 years: “You used to have to pay attention to those guys,” he said. “Now you can ignore them.”
You can ignore them, but they will still be there.
IT’S VERY HARD TO ADMIT YOU DON’T KNOW WHAT YOU THINK YOU KNOW.