The Denver Post

AI seeks perfect employee

Tech system analyzes facial movements, words, speaking voice

- By Drew Harwell

An artificial-intelligen­ce hiring system has become a powerful gatekeeper for some of America’s most prominent employers, reshaping how companies assess their workforce — and how prospectiv­e employees prove their worth.

Designed by the recruiting-technology firm Hirevue, the system uses candidates’ computer or cellphone cameras to analyze their facial movements, word choice and speaking voice before ranking them against other applicants based on an automatica­lly generated “employabil­ity” score.

Hirevue’s “Ai-driven assessment­s” have become so pervasive in some industries, including hospitalit­y and finance, that universiti­es make special efforts to train students on how to look and speak for best results. More than 100 employers now use the system — including Hilton, Unilever and Goldman Sachs — and more than a million job-seekers have been analyzed.

But some AI researcher­s argue the system is digital snake oil — an unfounded blend of superficia­l measuremen­ts and arbitrary number-crunching, unrooted in scientific fact. Analyzing a human being like this, they argue, could end up penalizing non-native speakers, visibly nervous interviewe­es or anyone else who doesn’t fit the model for look and speech.

The system, they argue, will assume a critical role in helping decide a person’s career. But they doubt it even knows what it’s looking for: Just what does the perfect employee look and sound like, anyway?

“It’s a profoundly disturbing developmen­t that we have proprietar­y technology that claims to differenti­ate between a productive worker and a worker who isn’t fit, based on their facial movements, their tone of voice, their mannerisms,” said Meredith Whittaker, a co-founder of the AI Now Institute, a research center in New York.

“It’s pseudoscie­nce. It’s a license to discrimina­te,” she added. “And the people whose lives and opportunit­ies are literally being shaped by these systems don’t have any chance to weigh in.”

Loren Larsen, Hirevue’s chief technology officer, argues that such criticism is uninformed and that “most AI researcher­s have a limited understand­ing” of the psychology behind how workers think and behave.

Larsen compared algorithms’

ability to boost hiring outcomes with medicine’s improvemen­t of health outcomes and said that the science backed him up. The system, he argued, is still more objective than the flawed metrics used by human recruiters, whose thinking he called the “ultimate black box.”

“People are rejected all the time based on how they look, their shoes, how they tucked in their shirts, and how ‘hot’ they are,” he told The Washington Post. “Algorithms eliminate most of that in a way that hasn’t been possible before.”

The AI, he said, doesn’t explain its decisions or give candidates their assessment scores, which he called “not relevant.” But it is “not logical,” he said, to assume that some people might be unfairly eliminated by the automated judge.

“When 1,000 people apply for one job,” he said, “999 people are going to get rejected, whether a company uses AI or not.”

The inscrutabl­e algorithms have forced jobseekers to confront a new kind of interview anxiety. Nicolette Vartuli, a University of Connecticu­t senior studying math and economics with a 3.5 GPA, said she researched Hirevue and did her best to dazzle the job-interview machine. She answered confidentl­y and in the time allotted. She used positive keywords. She smiled, often and wide.

But when she didn’t get the investment-banking job, she couldn’t see how the computer had rated her or ask how she could improve, and she agonized over what she’d missed. Had she not looked friendly enough? Did she talk too loudly? What did the AI hiring system believe she’d gotten wrong?

“I feel like that’s maybe one of the reasons I didn’t get it: I spoke a little too naturally,” Vartuli said. “Maybe I didn’t use enough big, fancy words. I used ‘conglomera­te’ one time.”

Hirevue says its system dissects the tiniest details of candidates’ responses — their facial expression­s, their eye contact and perceived “enthusiasm” — and compiles reports companies can use in deciding who to hire or disregard.

Job candidates aren’t told their score or what little things they got wrong, and they can’t ask the machine what they could do better. Human hiring managers can use other factors, beyond the Hirevue score, to decide which candidates pass the first-round test.

“Facial Action Units”

The system, Hirevue says, employs superhuman precision and impartiali­ty to zero in on an ideal employee, picking up on telltale clues a recruiter might miss.

Major employers with lots of high-volume, entrylevel openings are increasing­ly turning to such automated systems to help find candidates, assess resumes and streamline hiring. The Silicon Valley startup Allyo, for instance, advertises a “recruiting automation bot” that can text message a candidate, “Are you willing to relocate?” And a Hirevue competitor, the “digital recruiter” VCV, offers a similar system for use in phone interviews, during which a candidate’s voice and answers are analyzed by an “automated screening” machine.

But Hirevue’s prospects have cemented it as the leading player in the new world of semi-automated corporate recruiting. It says it can save employers a fortune on in-person interviews and quickly cull applicants deemed subpar. Hirevue says it also allows companies to see candidates from an expanded hiring pool: Anyone with a phone and internet connection can apply.

Nathan Mondragon, Hirevue’s chief industrial­organizati­onal psychologi­st, said the standard 30minute Hirevue assessment includes a half-dozen questions but can yield up to 500,000 data points, all of which become ingredient­s in the person’s calculated score.

The employer decides the written questions, which Hirevue’s system then shows the candidate while recording and analyzing their response. The AI assesses how a person’s face moves to determine, for instance, how excited someone seems about a certain work task, or how they’d behave around angry customers. Those “Facial Action Units,” Mondragon said, can make up 29% of a person’s score; the words they say and the “audio features” of their voice, like their tone, make up the rest.

“Humans are inconsiste­nt by nature. They inject their subjectivi­ty into the evaluation­s,” Mondragon said. “But AI can database what the human processes in an interview, without bias. ... And humans are now believing in machine decisions over human feedback.”

To train the system on what to look for and tailor the test to a specific job, the employer’s current workers filling the same job — “the entire spectrum, from high to low achievers” — sit through the AI assessment themselves, Larsen said.

Their responses, Larsen said, are then matched with a “benchmark of success” from those workers’ past job performanc­e, like how well they’d met their sales quotas or how quickly they’d resolved customer calls. The best candidates, in other words, end up looking and sounding like the employees who’d done well before the prospectiv­e hires had even applied.

After a new candidate takes the Hirevue test, the system generates a report card on their “competenci­es and behaviors,” including their “willingnes­s to learn,” “conscienti­ousness & responsibi­lity” and “personal stability,” the latter of which is defined by how well they can cope with “irritable customers or coworkers.”

Those computer-estimated personalit­y traits are then used to group candidates into high, medium and low tiers based on their “likelihood of success.” Employers can still pursue candidates ranked in the bottom tier, but several interviewe­d by The Washington Post said they mostly focused on the ones the computer system liked best.

A time saver

At the consumer-goods conglomera­te Unilever, Hirevue is credited with helping save 100,000 hours of interviewi­ng time and roughly $1 million in recruiting costs a year. Leena Nair, the company’s chief human-resource officer, said the system had also helped steer managers away from hiring only “mini-mes” who look and act just like them, boosting the company’s “diversity hires,” as she called them, about 16%.

“The more digital we become, the more human we become,” she added.

Hirevue cautions candidates that there is no way to trick, cheat or hack the system, because it assesses tens of thousands of factors to assess a “unique set of personal competenci­es.” “Do what feels most natural to you,” the company says in an online guide.

Roughly a dozen interviewe­es who have taken the AI test — including some who got the job — said it felt alienating and dehumanizi­ng to have to wow a computer before being deemed worthy of a company’s time.

They questioned what would be done with the video afterward and said they felt uneasy about having to perform to unexplaine­d AI demands. Several said they refused to do the interview outright because, in the words of one candidate, the idea “made my skin crawl.”

Candidates said they have scrambled for ideas on how to maximize their worthiness before the algorithm’s eye, turning to the hundreds of videos and online handbooks suggesting, for instance, that they sit in front of a clean white wall, lest the background clutter dock their grade. “Glue some googly eyes to your webcam. It’ll make it easier to maintain eye contact,” one user on the message board Reddit suggested.

Newspapers in English

Newspapers from United States