One startup’s quest to use AI to bring game dialogue to life
Of their two biggest promises, videogames have pretty much delivered on one. Huge and diverse worlds filled with detail to discover and things to do are now common, even expected. But the other promise, that of getting to interact with characters that respond naturalistically to your every word and action, is still lagging behind.
The NPCs you meet in games are the same scripted talking heads that they’ve always been. Some games are written better than others, but in comparison to the visually opulent and systemically deep worlds in which they stand, NPCs are wooden, their various conversational gambits constricted into series of dialogue trees in which you lose all of the freedoms you enjoyed in the wider world.
One new tech startup is hoping to use AI to help solve the problem of NPC dialogue. SpiritAI’s aim is to create dynamic conversations that feel like speaking to autonomous characters, responsive to what you express and ask, and willing to offer their own points of view. SpiritAI has developed various technologies called Character Engine, which includes natural-language classifiers, speech-to-text analysis and keyword examination to interpret what players are trying to say and to construct responses by modelling emotion and the character’s knowledge about the world.
“OK, the far end of this is passing the Turing test, right? If it’s done and complete and perfect, then it talks like a person,” says Emily Short, who manages Character Engine, having long been a leading figure in interactive fiction as a writer and co-developer of various text-adventure engines. “And no, we’re not there yet.” But, she says, Character Engine is standing on a road of iterative development that will figure out how to create naturalistic NPCs that are more fun and interesting to encounter in a game.
For SpiritAI co-founder chief creative officer Mitu Khandaker, the challenge comes down to how much data the system has to work with. “How big are the natural-language classifiers? It’s mostly just a data problem, and how you’re authoring the responses.” Khandaker was previously the indie developer behind Redshirt, a sim of a social network aboard a space station that tasked players with moving up the social ladder, before becoming an assistant arts professor of game design at New York University. For her, Character Engine is about allowing writers and narrative designers to craft specific stories with autonomous characters acting within them.
Its first demo was revealed at GDC earlier this year. Called The Interrogation, it has players talking to a Scottish robot, trying to figure out whether it’s guilty of a murder. Co-developed with London-based developer of Surgeon Simulator and forthcoming MMO Worlds Adrift Bossa Studios, you can type – or in the VR version, simply speak – your questions, and she’ll respond, revealing details about other characters and their relationships and the events during the
“OK, the far end of this is passing the Turing test, right? And no, we’re not there yet”
killing. As you delve, the robot’s emotional state changes, expressed in the UI and by its voice inflection and stutters, and you can manipulate it to provoke different responses. You can make it anxious or angry by moving the viewpoint closer or using threatening or insulting language, and you can put it at ease by moving away and being kinder.
The Interrogation plays a little like a dynamically driven Her Story – or, more directly, Galatea, a celebrated piece of interactive fiction that Short wrote in 2000 which presents a surprisingly naturalistic conversation with a statue. “We were looking to dig into whether we could create the sense of a continuous conversation in which new information is unfolding and you get a sense of making an emotional difference to a character if you’re mean or nice to them,” says Short. “One of the big objectives I had was to let the player participate in a conversation. I feel your standard dialogue tree is constraining, right? You have this choice of two-to-four options and it’s not fun to re-play. There’s no room for style or personality.” Character Engine works through writers giving NPCs two sources of information, a ‘script space’ of words and phrases the NPC can say and a knowledge model, which is information the NPC knows about the world. In The Interrogation, the robot knows the height and weight of the characters she mentions; if you ask her whether Alicia is strong, then the system can use that information to surmise an answer without the writer needing to specifically note it.
In many ways, the magic in Character Engine lies in design and writing, rather than the technology itself. “The demo is a difficult design problem because it’s a scenario where the robot is trying to be evasive, but at the same time, players don’t know what to ask and how to interact with it,” says Khandaker. The system has to therefore seed the ongoing conversation with pointers and clues as to what to ask.
On its surface, the demo seems utterly freeform, but underneath it’s carefully structured using Character Engine’s authoring tools, which can make particular pieces of dialogue available or unavailable depending on the scene or stage in the conversation the player has reached. The tools give the chance to give NPCs certain narrative beats to hit or facts they have to reveal, and triggers for points at which the scene will end, whether through timers, reaching certain emotional states or relating certain bits of information.
For Short, authoring a Character Engine NPC is a little like scriptwriting, in the sense that it makes a writer think first about motivation and dramatic structure before the words themselves. But the tools are being developed to be highly adaptable, so developers can be as rigid or free as they want to be, and to use whichever components they like. It can even generate on-the-fly multiple choice dialogue options rather than rely on players inputting natural language.
“But the idea is making it so the moments of constraint are the rarity and most of the time the player has more flexibility,” Short says, though she’s not aiming for Character Engine to live up to the dream of the holodeck – of entirely immersive virtual worlds that correctly interpret and respond to players’ every interaction. “Fundamentally I don’t believe in that,” she says. “Not even in the sense of whether we can actually do that, but in a design sense. Would that be a satisfying and enjoyable thing?”
For her, Character Engine is for games, and when players don’t have direction they get paralysed with uncertainty. “It’d be like constantly being forced to be on an improv stage without being trained. I’m not so much interested in making it so you can never find the boundaries, it’s more like, can we make nice, smooth edges so when you encounter them they redirect you in a way that feels natural? As a player, if you run out of ideas, the NPC pulls you back into the storyline, but you still have a level of freedom you don’t get in a lot of current game structures. That’s what I see as the sweet spot.”
For SpiritAI, Character Engine is a foundation, but its technology can do more. As she began to explore the potentials of what it means for a bot to understand what a player is saying, Khandaker realised that it could also be used to address another issue facing videogames: a rather more pressing one, of online harassment. If it can examine what players are saying, what they’ve
“The idea is the moments of constraint are the rarity and most of the time the player has more flexibility”
previously said and also watch player-toplayer interaction, it can identify toxic player behaviour.
Ally is a set of tools that does just this, which SpiritAI has already begun releasing to beta partners to begin using with their live data. One demonstration, built to test the SDK only, shows a player following another and bombarding them with party requests. Ally notices the number of requests and the proximity of their avatars and chats to the potential victim: “You seem to be having a problem; are you OK?” That person can then respond in natural language to say yes or no, whereupon the bot can take action on an offending player, muting them or banning them as appropriate to the game’s policies, or simply shutting up.
“There are lots of parameters,” says Khandaker, acknowledging the fuzzy and inconsistent nature of online interactions. “Are they upset by anyone doing this, or just this particular person? It asks questions to understand, and it then learns from that for the future. Our boundaries are very different in different situations, right? We are OK with dodgy language when it’s people we know sometimes, but not when it’s a stranger.”
The aim is to support moderators and GMs, who can be dealing with thousands of support tickets a day, and the level of engagement is up to the developer, whether the bot talks directly with players or merely flags up potential issues for human mods to look at. Developers are also able to write a character for the bot, just as they might an NPC, so its interactions fit into the game’s world.
And it doesn’t have to be an enforcer. It can also identify positive behaviour, rewarding or supporting helpful players. Either way, Ally could help shape healthy player communities so they become safer places to play, in whatever way devs deem to make sense for their game.
SpiritAI’s focus on natural language and bots follows an explosive growth in the field. Bots on Tencent’s WeChat social network, for example, dominate the way people in China manage social services and query information on their phones, and they’re growing on Facebook, Skype and other platforms in the west, too. It’s therefore following a general trend in developing natural-language technology, but is focused on using them to make games better, richer and safer places to play. Once upon a time, Edge asked, “What if you could talk to the monsters?” That question, it seems, is finally close to being answered.
Character Engine’s scripting system allows writers and narrative designers to set up a ‘script space’ of information and dialogue, such as a timeline of events a character knows about. The system then allows them to improvise within that space, responding to the player’s input in a natural, organic way
Emily Short (top) heads SpiritAI’s Character Engine; Mitu Khandaker is the firm’s co-founder and creative director
Ally, SpiritAI’s online safety service, is being trialled; here its helper bot is integrated with an off-the-shelf Unity MMO package to demonstrate how it recognises potential social problems