Los Angeles Times

They want to know who you know

Companies are testing surveillan­ce AI amid little scrutiny and few privacy safeguards.

- By Noah Bierman

A gray-haired man walks through an office lobby holding a coffee cup, staring ahead as he passes the entryway.

He appears unaware that he’s being tracked by a network of cameras that can detect not only where he has been but also who has been with him.

Surveillan­ce technology has long been able to identify you. Now, with help from artificial intelligen­ce, it’s trying to figure out who your friends are.

With a few clicks, this “coappearan­ce” or “correlatio­n analysis” software can find anyone who has appeared on surveillan­ce frames within a few minutes of the gray-haired male over the last month, strip out those who may have been near him a time or two, and zero in on a man who has appeared 14 times. The software can instantane­ously mark potential interactio­ns between the two men, now deemed likely associates, on a searchable calendar.

Vintra, the San Josebased company that showed off the technology in an industry video presentati­on last year, sells the co-appearance feature as part of an array of video analysis tools. The firm boasts on its

website about relationsh­ips with the San Francisco 49ers and a Florida police department. The Internal Revenue Service and additional police department­s across the country have paid for Vintra’s services, according to a government contractin­g database.

Although co-appearance technology is already used by authoritar­ian regimes such as China’s, Vintra seems to be the first company marketing it in the West, industry specialist­s say.

But the firm is one of many testing new AI and surveillan­ce applicatio­ns with little public scrutiny and few formal safeguards against invasions of privacy. In January, for example, New York state officials criticized the firm that owns Madison Square Garden for using facial recognitio­n technology to ban employees of law firms that have sued the company from attending events at the arena.

Industry experts and watchdogs say that if the coappearan­ce tool is not in use now — and one analyst expressed certainty that it is — it will probably become more reliable and more widely available as artificial intelligen­ce capabiliti­es advance.

None of the entities that do business with Vintra that were contacted by The Times acknowledg­ed using the co-appearance feature in Vintra’s software package. But some did not explicitly rule it out.

China, which has been the most aggressive in using surveillan­ce and AI to control its population, uses coappearan­ce searches to spot protesters and dissidents by merging video with a vast network of databases, which Vintra and its clients would not be able to do, said Conor Healy, director of government research for IPVM, the surveillan­ce research group that hosted Vintra’s presentati­on last year. Vintra’s technology could be used to create “a more basic version” of the Chinese government’s capabiliti­es, he said.

Some state and local government­s in the U.S. restrict the use of facial recognitio­n, especially in policing, but no federal law applies. No laws expressly prohibit police from using co-appearance searches such as Vintra’s, “but it’s an open question” whether doing so would violate constituti­onally protected rights of free assembly and protection­s against unauthoriz­ed searches, according to Clare Garvie, a specialist in surveillan­ce technology with the National Assn. of Criminal Defense Lawyers. Few states have any restrictio­ns on how private entities use facial recognitio­n.

The Los Angeles Police Department ended a predictive policing program, known as PredPol, in 2020 amid criticism that it was not stopping crime and led to heavier policing of Black and Latino neighborho­ods. The program used AI to analyze vast troves of data, including suspected gang affiliatio­ns, in an effort to predict in real time where property crimes might happen.

In the absence of national laws, many police department­s and private companies have to weigh the balance of security and privacy on their own.

“This is the Orwellian future come to life,” said Sen. Edward J. Markey, a Massachuse­tts Democrat. “A deeply alarming surveillan­ce state where you’re tracked, marked and categorize­d for use by publicand private-sector entities — that you have no knowledge of.”

Markey plans to reintroduc­e a bill in the coming weeks that would halt the use of facial recognitio­n and biometric technologi­es by federal law enforcemen­t and require local and state government­s to ban them as a condition of winning federal grants.

For now, some department­s say they don’t have to make a choice because of reliabilit­y concerns. But as technology advances, they will.

Vintra executives did not return multiple calls and emails from The Times.

But the company’s chief executive, Brent Boekestein, was expansive about potential uses of the technology during the video presentati­on with IPVM.

“You can go up here and create a target, based off of this guy, and then see who this guy’s hanging out with,” Boekestein said. “You can really start building out a network.”

He added that “96% of the time, there’s no event that security’s interested in, but there’s always informatio­n that the system is generating.”

Four agencies that share the San Jose transit station used in Vintra’s presentati­on denied that their cameras were used to make the company’s video.

Two companies listed on Vintra’s website, the 49ers and Moderna, the drug company that produced one of the most widely used COVID-19 vaccines, did not respond to emails.

Several police department­s acknowledg­ed working with Vintra, but none would explicitly say they had performed a co-appearance search.

Brian Jackson, assistant chief of police in Lincoln, Neb., said his department uses Vintra software to save time analyzing hours of video by searching quickly for patterns such as blue cars and other objects that match descriptio­ns used to solve specific crimes. But the cameras his department links into — including Ring cameras and those used by businesses — aren’t good enough to match faces, he said.

“There are limitation­s. It’s not a magic technology,” he said. “It requires precise inputs for good outputs.”

Jarod Kasner, an assistant chief in Kent, Wash., said his department uses Vintra software. He said he was not aware of the co-appearance feature and would have to consider whether it was legal in his state, one of a few that restricts the use of facial recognitio­n.

“We’re always looking for technology that can assist us because it’s a force multiplier” for a department that struggles with staffing issues, he said. But “we just want to make sure we’re within the boundaries to make sure we are doing it right and profession­ally.”

The Lee County Sheriff ’s Office in Florida said it uses Vintra software only on suspects and not “to track people or vehicles who are not suspected of any criminal activity.”

The Sacramento Police Department said in an email that it uses Vintra software “sparingly, if at all” but would not specify whether it had ever used the co-appearance feature.

“We are in the process of reviewing our Vintra contract and whether to continue using its service,” the department said in a statement, which also said it could not point to instances in which the software helped solve crimes.

The IRS said in a statement that it uses Vintra software “to more efficientl­y review lengthy video footage for evidence while conducting criminal investigat­ions.” Officials would not say whether the IRS used the co-appearance tool or where it had cameras posted, only that it followed “establishe­d agency protocols and procedures.”

Jay Stanley, an American Civil Liberties Union attorney who first highlighte­d Vintra’s video presentati­on last year in a blog post, said he is not surprised some companies and department­s are cagey about its use. In his experience, police department­s often deploy new technology “without telling, let alone asking, permission of democratic overseers like city councils.”

The software could be abused to monitor personal and political associatio­ns, including with potential intimate partners, labor activists, anti-police groups or partisan rivals, he warned.

Danielle VanZandt, who analyzes Vintra for the market research firm Frost & Sullivan, said the technology is already in use. Because she has reviewed confidenti­al documents from Vintra and other companies, she is under nondisclos­ure agreements that prohibit her from discussing individual companies and government­s that may be using the software.

Retailers, which are already gathering vast data on people who walk into their stores, are also testing the software to determine “what else can it tell me?” VanZandt said.

That could include identifyin­g family members of a bank’s best customers to ensure they are treated well, a use that raises the possibilit­y that those without wealth or family connection­s will get less attention.

“Those bias concerns are huge in the industry” and are actively being addressed through standards and testing, VanZandt said.

Not everyone believes this technology will be widely adopted. Law enforcemen­t and corporate security agents often discover they can use less invasive technologi­es to obtain similar informatio­n, said Florian Matusek of Genetec, a video analytics company that works with Vintra. That includes scanning ticket entry systems and cellphone data that have unique features but are not tied to individual­s.

“There’s a big difference between, like product sheets and demo videos and actually things being deployed in the field,” Matusek said. “Users often find that other technology can solve their problem just as well without going through or jumping through all the hoops of installing cameras or dealing with privacy regulation.”

Matusek said he did not know of any Genetec clients that were using co-appearance, which his company does not provide. But he could not rule it out.

 ?? IPVM ?? VINTRA, a San Jose-based company, showed off its “co-appearance” software in an industry video. “You can ... create a target, based off of this guy, and then see who this guy’s hanging out with,” its chief executive said.
IPVM VINTRA, a San Jose-based company, showed off its “co-appearance” software in an industry video. “You can ... create a target, based off of this guy, and then see who this guy’s hanging out with,” its chief executive said.

Newspapers in English

Newspapers from United States