IBM’s AI loses debate but still shines
Audience says Project Debater enriched knowledge about subject matter
SAN FRANCISCO >> There’s no debating this one: In human vs. machine this time, the human triumphed.
IBM’s Project Debater — manifested onstage by a black monolith with a female voice — battled champion debater Harish Natarajan on Monday night in a theater packed with about 700 spectators at Yerba Buena Center. It was the artificial intelligence technology’s
first live debate in public.
While the audience voted by smartphone to give the
human the victory, Debater managed to impress. Observers said Debater was better at enriching the audience’s
knowledge about the topic as it argued that preschool should be subsi-
dized.
The lead investigator for Project Debater, AI scientist Noam Slonim, seemed pleased with how well IBM’s technology performed. “Its rhetorical skills still were not at the level of Harish,” Slonim said during an onstage interview after the debate. “But it was able to pinpoint relevant points for the debate.”
In its arguments, Project Debater cited studies showing positive outcomes linked to preschool education from the vast amounts of material it ingested beforehand. Debater drew from a repository of 300 million articles, meaning it had to read about 10 billion sentences.
If that sounds unfair, consider that the human it debated “has learned all his life,” said Aya Soffer, vice president of AI technology for IBM, in an interview before the event.
Like its human counterpart, Debater had just 15 minutes to prepare after it was given the topic, which was chosen by IBM. Debater was not connected to the internet, and based its arguments on its own knowledge and the arguments it heard from Natarajan.
Debater even managed to make the audience laugh. At the start of the event, it acknowledged that its opponent, who was grand finalist at the 2016 World Debating Championships, was a formidable debater.
However, it said, “I suspect you’ve never debated a machine. Welcome to the
future.”
Natarajan — who in arguing against subsidizing preschool sealed his victory by changing the stances of 17 percent of the 416 audience members who voted before and after the debate — noted how well his opponent was able to absorb and synthesize information so quickly.
“What struck me was the potential value for Debater because of the amount of knowledge it can grasp,” he said during the post-debate onstage interview. He added that the AI’s arguments were “nicely phrased and contextualized,” and that it could be “quite powerful” when used by humans.
That’s IBM’s hope for the future of the technology.
“We hope the audience walks out of here saying, ‘hey, this can enrich the way we can make decisions in the future,’ “said Ranit Aharonov, manager of Project Debater, during the onstage interview after the debate.
After IBM’s Watson beat humans in “Jeopardy” in 2011, the company wanted to identify its next grand challenge. Pushing its AI technology to debate a human became the next problem the company sought to solve, which it started working on in 2012.
“In debate, there’s no right or wrong answer,” Soffer said. “It’s about weighing different options, which is what we face every day in the real world.”
Prior to Monday night’s event, Project Debater had performed live in front of an audience just one other time — last June, in front of press but not the public. Since then, IBM had been trying to improve Debater’s
rebuttal capabilities.
So far, AI has been good at focusing on one thing or a few things at most. For example, personal assistants can help us find directions, play our music or call our contacts. Debating requires bringing together different AI systems and making sure they all work together. It comes two decades after IBM’s Deep Blue went head to head with chess champion Garry Kasparov in the mid-1990s, and a few years after Google’s DeepMind competed against the world’s best Go players.
“Debate is out of the comfort zone of AI,” said Slonim in an interview before the event. He said Debater was “going to a territory that’s extremely human.”
“Project Debater is an illustration of how fast the AI discipline is progressing at the moment, especially in the fields of natural language processing and deep learning,” said Nick McQuire, vice president of enterprise research at CCS Insight.
The debate, held on the first day of IBM’s weeklong Think conference, was moderated by John Donvan, host of the Intelligence Squared debate series. He noted that the audience — which included debaters from Dougherty Valley High School in San Ramon — witnessed history in the making.
The demonstration came the same day that President Trump signed an executive order aimed at expanding the development of AI in the United States, although the order was short on details.