The Charm in the Machine
COGITO’S ALGORITHMS READ NONVERBAL CUES BETTER THAN YOU CAN. WELCOME TO THE AGE OF A.I.-DRIVEN EMPATHY
Cogito’s software, which is rapidly becoming a must-have for companies that run huge call centers, has a very simple aim: to help us all be more human.
JOSHUA FEAST has a distinct conversational tic: a nervous laugh that escapes when he’s worried something he says might come across as self-serious or highfalutin. Like, for instance, when he mentions that he was the first New Zealander to be selected as a Fulbright Scholar in entrepreneurship, or tells you he once spent two weeks in a coma after contracting malaria in Indonesia, or says he was among the highest-scoring secondary-school students on a national math exam.
Feast grew up in the suburbs of New Zealand’s capital, Wellington, where it’s considered unwise to act like a big hairy deal. Kiwis enjoy nothing more than cutting a “tall poppy” down to size. So Feast, a slender, bespectacled 40-year-old, is studiously careful at all times to deflect credit and minimize his importance. “I’m not very comfortable talking about myself sometimes,” he says, with appealing humility that stops just the right distance short of false modesty.
You can’t help analyzing someone’s personal affect after talking to Feast, because
that’s his business. Feast is co-founder and CEO of Cogito Corporation, a Boston-based software startup that uses artificial intelligence to measure and improve the quality of certain key conversations, such as sales and customer-service calls, in real time. Or, as Feast puts it, Cogito—which is Latin for “I think”— “helps people be more charming in conversation.”
“Charm” might sound like a hard thing to quantify, and it is. But Cogito’s other co-founder, MIT professor of media arts and sciences Alex “Sandy” Pentland, has spent the past 20 years doing just that. Through experiments in his Human Dynamics Lab, Pentland has shown how unconscious, nonverbal “honest signals”—he wrote a book with that title in 2008—including tempo, emphasis, and mimicry influence the outcome of interactions like salary negotiations, team meetings, and romantic courtships. “Language is culturally something that’s relatively new to humans, but before language we were already social beings,” says Pentland, a polymath and futurist whose expertise extends to evolutionary psychology, artificial intelligence, and data science. “We’re all sort of brainwashed to think it’s about the words.” Thanks to Pentland’s work, we now know the mechanisms of human connection can, in fact, be boiled down to a set of mathematical equations. Those equations are what Cogito is built on.
Cogito occupies two floors of a decidedly unstylish office building in downtown Boston. Amid overflowing cubicles and faded yellow walls, I watched over the shoulder of product marketing manager Eli Orkin as he handled a Cogito-assisted simulated customer-service call from one of his colleagues, Channah Rubin. As Rubin attempted to tell a story about the problems she was having obtaining a new credit card, Orkin repeatedly cut her off, until a discreet slide-in notification appeared on his computer screen: “Frequent overlaps.” Orkin overcorrected, letting Rubin ramble on, until he got a second prompt reading “Slow to respond.” A long, somewhat condescending monologue earned him a “Speaking a lot” warning.
During the call, a color- coded meter in the corner of Orkin’s screen offered a running gauge of how well it was going, shading to yellow and orange when he responded too abruptly or slowly and back to green when normal give-andtake was restored. “Conversations are like a dance,” Feast explains. “You can be in sync or out of sync.” Had Rubin become truly upset, Orkin would have seen an “Empathy” prompt, a cue to say something reassuring. But just acting upset wouldn’t do it. No one at Cogito can fool the software, which analyzes hundreds of signals to tell if distress is real.
Cogito, which has 75 employees, counts among its clients three of the five largest U.S. health insurance firms, two of the five largest disability insurers, and some of the biggest credit card companies. Cogito also has a mental health care product, an app called Companion, that’s used by nurses and social workers in private and Veterans Health Administration hospitals to flag patients showing signs of PTSD and depression. “Our dream would be to take advantage of this for a lot of important conversations,” says Feast. “It could be negotiations, it could be meetings, it could be improving dating experiences.” Anything, he says, where there’s a need to “be more emotionally intelligent in real time.”
A quintessentially Homo sapiens trait is empathy— the ability to intuit and be moved by others’ emotions— and it can be crucial for persuading, consoling, or seducing. Yet the data is clear: We humans aren’t the empathy prodigies we think we are. In the modern workplace, with its high attention demands, packed schedules, and long hours, we’re pretty bad at it.
In cognitive psychology, assessing others’ thoughts and feelings through nonverbal cues is called person perception. Some people do this easily. Others find it impossible. Most of us muddle along somewhere in the middle.
And when it comes to assessing our own person-perception skills, most of us are rank amateurs. “If I were to ask a whole group of people, ‘On a scale of one to five, how good do you think you are at recognizing social signals in others?’ nearly all would rate themselves a four or a five,” says Feast. He’s sympathetic to the deluded. “One of the big problems we have is we don’t get much feedback on it in our daily lives. If you think I’m being kind of rude right now, you’re probably not going to tell me.” In the absence of feedback, it’s hard to improve—and easy to think you don’t need to.
That’s where Cogito’s promise lies. If it lays bare our limitations, it’s only by showing how, for the first time, we have the tools to transcend them. In Feast’s view, the pessimists who see A.I. making humans obsolete—those who worry over research that shows A.I. may unemploy millions over the next several years—are overlooking how much smarter and more productive and creative we can be with its help. How much more human it can make us, to put it bluntly. “In some ways, we think of ourselves as a cyborg company,” Feast says, “helping humans be their best selves.”
IN NEW ZEALAND, there’s a saying: “You can make anything from No. 8 wire”— sheep farmers use it for fences. “No. 8 wire mentality” is Kiwi shorthand for a cando, scrappy spirit. Feast’s family embodied No. 8. His grandfather founded a large construction business, and his father and uncle ran a number of enterprises in real estate and development. Feast too felt the pull of building, albeit in a digital realm. He earned an undergraduate degree in computer engineering, and one of his first jobs was working as a tech consultant for Accenture. Among its clients was New Zealand’s Department of Child, Youth, and Family, which faced two huge challenges. First, even the best social workers often lasted only three to five years
WE HUMANS AREN’T THE EMPATHY PRODIGIES WE THINK WE ARE.
before burning out, a phenomenon known as compassion fatigue. Second, below-average social workers had trouble improving, because their work was hard to quantify.
Feast was still ruminating on those lessons when, in 2005, he won that Fulbright Scholarship in entrepreneurship, which allowed him to study at MIT. One of Feast’s courses there was a seminar consisting of guest lectures curated by Pentland. For the past few years, the shaggy-haired, hyperenergetic Pentland had been focusing his investigations on those unconsciously transmitted honest signals. Pentland and his team were interested in how two strangers adopt each other’s physical postures and intonations as they grow comfortable together, and how a speaker using a steady pattern of CHRISTOPHER CHURCHILL emphasis leaves an audience with the sense that he or she is well informed.
Among Pentland’s plaudits is this: He’s considered a pioneer of wearable computing. (One of his grad students created Google Glass.) To conduct their inquiries, Pentland’s team built a wearable device they called a “sociometer,” a shouldermounted pack, roughly the
size of an iPhone, whose sensors gathered data about speech and movements during interactions. In experiments, they demonstrated that those honest signals (the term, borrowed from evolutionary biology, refers to behaviors that are hard to fake) could be used to accurately predict the outcome of salary negotiations, group decisions, and speed dating.
As Feast learned about his professor’s research, he realized that software built using Pentland’s findings could help social workers see if they were building trust with their clients, or determine which psychiatric patients need emergency counseling. “I thought the potential was unbelievable if we could take it to market and make it a thing people could use,” he says.
Pentland has helped found more than 20 startups based on his research, and he loved the idea. He’d grown up close with his grandmother, who ran Michigan’s first inpatient institution for children with psychiatric problems. “I actually learned to read in a mental institution,” he says. He signed on and helped recruit one of his former PhD students, Ali Azarbayejani, to be CTO.
But all they had were the theoretical underpinnings of a business—nothing close to a prototype, much less a product. It was what venture capitalists call a science project. “You’ve got all this academic research that can’t move forward because the chasm between experimental results and a fundable technology is too wide, too risky,” says Feast. Yet they were in luck. VCs hate throwing money down a hole without knowing how deep it is. But one institution didn’t mind.
The Defense Advanced Research Projects Agency— Darpa—is an arm of the U.S. Defense Department that incubates emerging technologies. “It’s like a venture capital firm without a profit motive,” says Russell Shilling, a former Darpa program manager who now develops educational tech at a nonprofit called Digital Promise. “You’re trying to think about what might be possible in 10 to 20 years and then create a program to build it in three to four.” In 2010, Shilling was tasked with scouting out technologies that could be used to flag soldiers returning from Afghanistan and Iraq who might be showing early signs of PTSD. Feast’s idea was a perfect fit, and Pentland’s reputation helped their application secure an easy approval.
The technical challenge they had embarked on was indeed daunting, requiring models for turning speech, with all its nuances and inflections, into neatly labeled data that can be fed into machine-learning algorithms, which would then try to extract behavioral patterns from it. Looking for hints of emotional states in raw audio is an order of magnitude more difficult than speech recognition: A word has a beginning and an end. Clues to its meaning can be derived from the words around it. But signs of a speaker’s depression might be scattered throughout a long conversation.
Cogito hit all of the mileposts in its Darpa proposal, and in 2012, it had Feast’s dream product. Called Companion, it’s an app nurses, psychologists, and social workers can use to monitor the psychological states of their patients, who record and upload audio diaries. It’s been a useful tool. “This has enormous promise in changing the way we do mental health care as well as medical care,” says David Ahern, director of behavioral informatics and e-health at Boston’s Brigham and Women’s Hospital. Ahern has been overseeing a three-year study involving more than 200 patients who suffer from physical and psychiatric disorders. Research has shown that medical patients who develop emotional health problems cost more to treat and respond less well to treatment; as a result, says Ahern, “it behooves the medical system to do a better job in detection and treatment of behavioral health because it drives outcomes and drives the costs.” While his study’s results aren’t yet in, feedback from patients and clinicians has been positive. Some practitioners believe Companion has helped avert suicides.
But Feast quickly came to see it was a product with limited revenue potential. In the convoluted world of health care, insurers often pay more readily for a treatment, even an expensive and ineffective one, than for a preventive service. “Making a business purely around making people more well isn’t always a good proposition,” Feast says, delicately. There were much bigger markets to crack.
COMPANIES HAVE their own version of the person-perception problem. A big brand might have tens of thousands of employees handling customers’ calls and complaints. Each of those contacts is an opportunity for the customer to form an impression, positive or negative. Yet each is handled by a customerrelations representative who—like all of us—probably overrates his or her skill at basic things like listening, demonstrating empathy, and establishing trust. Geeta Wilson, vice president of consumer experience at the health insurer Humana, says she lives in fear, as do many executives, of this “discrepancy between how we think of ourselves internally and how our customers think of us.” She adds, “You don’t want to be in a place where you’re rating yourself better than your customer is.”
But unlike us overconfident individuals, companies such as Humana put a lot of money and effort into closing that gap. They ask callers to stay on the line and take automated surveys. They send email questionnaires and conduct focus groups to calculate metrics like CSAT (customer satisfaction), NPS (Net Promoter Score—the willingness to recommend a brand to others), and VOC (voice of the customer, derived from surveys and focus groups). Calls to a contact center are recorded, and random samples of each employee’s interactions are reviewed for quality.
These methods have shortcomings. Surveys and reviews are slow, and small sample sizes skew findings. “Every human assessment, it’s always going to have some margin of error,” says Wilson. “Even if you’re really good at that as a company, you’ll always be off.”
In 2016, Humana tried something new. Wilson, who had
heard about Cogito from Christopher Kay, Humana’s chief innovation officer, offered 200 of her call-center associates as guinea pigs in a test of a new product. In a six-month pilot study, these associates took all of their calls using Cogito’s real-time conversation-analysis tool. The results were hard to ignore: Customers whose calls were handled by those using the Cogito app reported a 28 percent higher NPS. Issue resolution improved by 6 percent while average call time and escalations—when callers demand to speak with managers—both went down. Humana is now in the process of rolling out Cogito to thousands more of its customerrelations associates, and it’s running a second pilot, this one of Cogito’s application tailored for use in sales.
Call- center agents have a lot in common with social workers: They don’t last either. The average call center’s turnover runs from 30 to 45 percent annually. Agents also have compassion fatigue, but theirs builds up over hours, not years. Thanks to online self-service tools and chatbots, most easy queries never make it to a call center. The questions that do come in are often difficult and fraught with emotion.
What customers want, above all, is to feel that they’re in good hands, says Douglas Kim, Cogito’s chief revenue and customer-success officer. The sense that someone knows what he or she is talking about or cares about what you have to say is exactly the sort of thing conveyed primarily through nonverbal signals, Pentland’s research has shown. But maintaining the behaviors that send those signals, such as answering questions without hesitation, gets increasingly hard as cognitive fatigue creeps in over the course of a shift; to excel, workers need something stronger than a cup of coffee. “I use an analogy,” says Kim. “It’s like when you drive a newer car and you have