SFPD seeks tech to identify ‘at-risk’ officers
The San Francisco Police Commission is considering upgrading the case management system used to identify problem officers before they get the city sued.
The new software, “First Sign” from Benchmark Analytics, is an early intervention system to identify “at-risk” officers. According to the Chicago software company, two-thirds of injuries and excessive force incidents can be traced to atrisk officers, who make up about 5% of San Francisco’s police force.
The developers say that for every 100 officers identified by First Sign, 81 will go on to be investigated internally within the next 12 months for what Benchmark Analytics calls a “major adverse event,” such as an excessive force incident or some other serious policy breach.
Benchmark Analytics tells police departments that First Sign can “significantly lower your exposure to escalating liability claims from increasing litigations.” Between 2010 and 2023, San Francisco settled at least 140 cases for more than $25,000, Mission Local reported. They also settled 10 wrongful death cases that cost the city more than $12 million.
The city already uses an early intervention program from Benchmark Analytics. That software identified 28 officers who generated 311 alerts in 2021; 62% (142) were for officers involved in three or more use-offorce encounters in a threemonth span.
In 2022, policy changes lowered the threshold for generating an alert, most significantly dropping the requirement that a victim be injured or complain of pain for something to be reportable. This led to 2,119 alerts generated by 1,342 officers, an increase of more than 580%. Again, multiple use-of-force incidents in a 90-day span accounted for the vast majority of alerts, in this case 87% (1,852).
First Sign pulls data on officer histories, arrests, uses of force and internal affairs records, crunching almost 80 variables to determine an officer’s risk level. It creates profiles for each officer that include their risk history, a summary of past arrests, uses of force, complaints and investigations. It compares each officer with their peers to identify high-risk officers.
Compared with the Benchmark Management System, the program now in use, First Sign is more reliable and consistent, Nick Montgomery, the company’s chief research officer, told the commission. It’s built upon the biggest multijurisdictional dataset of its kind, he said.
“This is an incredible personnel tool,” Commissioner Debra Walker said at Wednesday’s meeting.
While commissioners largely agreed that they should invest in stronger software to alert department leadership to problem officers, they were divided about who else should have access to the alerts.
Paul Henderson, executive director of the Department of Police Accountability, which investigates complaints against officers and officer shootings and makes policy recommendations to the commission, argued that his oversight agency should have access to information collected by First Sign, to ensure that something happens when officers are flagged.
“The information that’s being collected and analyzed and disseminated … is still relevant to our policy conversations and our legislative conversations and the mediation practices that we have,” Henderson told the commission.
Commissioner Jesus Yáñez was also skeptical of creating a system with little oversight. He asked if when an alert leads to an internal investigation or disciplinary action, “can it be shared so that in real time we have a parallel investigation, which is what DPA is charteramended to do?”
“I don’t think that’s appropriate,” Police Chief Bill Scott replied. “If DPA has a relevant investigation they will get the disciplinary history of the officer.”
Scott and Walker said that sharing the alerts with DPA would defeat the purpose of having an early intervention system and damage trust among rank-and-file officers.
Benchmark Analytics CEO Ron Huberman, present to answer questions about First Sign, said the technology isn’t used for disciplinary purposes in any of the dozens of cities or statewide agencies the company works with.
The commission did not take any formal steps to implement First Sign on Wednesday and will discuss the matter again at a future meeting. Benchmark Analytics’ presentation did not mention the program’s expected cost.
The San Francisco Police Officers Association does not want the DPA to have access to First Sign.
“The idea of giving investigative organizations predictions on specific officers before an incident even happens should stay in science-fiction storylines like ‘Minority Report,’ ” union president Tracy McCray said in a statement. “This tool should be kept as a proactive, positive tool to help improve training and policing in our communities.”
The officers union hadn’t been consulted about the proposed software upgrade, spokesperson Dustin Derollo told the Chronicle. Derollo said the police union supported the implementation of the current early intervention software, but did not know if the new software has helped other large urban police departments.
Following George Floyd’s murder by a Minneapolis police officer in May 2020, the SFPOA, along with police officers’ unions in Los Angeles and San Jose, put out a joint statement calling for early intervention software to be a national standard, Derollo said. But Derollo said the union believes the best strategies for decreasing complaints are improving officer morale and providing field and tactical training.