Secretive software doing more state work
But computer models can make mistakes
MADISON - When state corrections software looked at Evan Goyke’s history, the program saw a man with “probable” criminal contacts and a “tendency toward an antisocial personality.”
It shouldn’t be a surprise — or a red flag — that Goyke knows people with rap sheets: the lawabiding Milwaukee Democrat is a former public defender.
As for being “antisocial,” the lawmaker will cop to a crusader streak: He sees a lot of things in this world he wants to change.
This mock test and its false positives come from a commonplace government tool — software from private companies that don’t share their full methods with the public or even state officials.
Wisconsin’s Department of Corrections has used the COMPAS system on more than 200,000 offenders since 2012 but still hasn’t formally verified the risk test in this state.
But Goyke still sees a value in COMPAS — as long as judges don’t accept its results on blind faith.
“This was an exercise that really hit home for me why we need to stay on top of the use of this tool,” said Goyke, an Assembly Corrections Committee member.
Government is using private computer models to scrutinize most Wisconsinites in some way, from criminals to income tax payers.
State officials generally know the information being fed into these programs but often don’t know how the firms analyze the data.
Companies guard these methods closely since this “special sauce” represents the basis for their business.
That creates potential pitfalls because these computer models can make mistakes, said Cathy O’Neil, a Harvardtrained mathematician and author of the 2016 book “Weapons of Math Destruction.”
“The problem is we have a blind trust mindset with the data,” O’Neil
“The state of Wisconsin should not be using outside contractors if it means less transparency.” BILL LUEDERS PRESIDENT OF THE WISCONSIN FREEDOM OF INFORMATION COUNCIL
said. “We need to demand evidence that big data is accurate, fair and legal.”
Bill Lueders agreed, saying the public has a right to know how government works.
“The state of Wisconsin should not be using outside contractors if it means less transparency,” said Lueders, president of the Wisconsin Freedom of Information Council.
The COMPAS system sold by Northpointe Inc. is used to predict an offender’s likelihood of committing a new crime after release and to identify needs such as treatment for addiction.
The system uses known factors such as a defendant’s age, criminal history, drug use and education, but state officials and the defendant don’t get to see how the COMPAS system weighs those factors.
Wisconsin has yet to finish verifying the system’s accuracy here because the Department of Corrections waits three full years after an inmate’s release to see if he or she re-offends, said agency spokesman Tristan Cook, who couldn’t say if the state will test for racial bias.
In July, the state Supreme Court found that judges can use COMPAS as one factor in sentencing.
Some studies have raised questions about possible racial bias, and the system’s accuracy still hasn’t been formally verified for Wisconsin’s population.
In the Department of Revenue, officials use private software to foil criminals by flagging possible income tax fraud.
The state uses a product from Colorado firm FAST Enterprises, which takes income tax data from states like Wisconsin and looks for signs of identify thieves claiming false refunds in a taxpayer’s name.
“There’s a lot of people out there who want to run a business out of defrauding the government,” said James Harrison, a principal with FAST Enterprises.
This year, 60,000 to 90,000 Wisconsin tax filers will have to verify to the state they’re not an identity thief by giving a PIN number, answering personal questions or providing documents.
The FAST Enterprises methods aren’t shared with the public, though state officials say the checks don’t use factors such as race.
Revenue officials also watch the statewide error rates closely to catch as many identify thieves as possible while minimizing the hassle for taxpayers, said Vicky Gibbons, a deputy income and sales tax administrator.
O’Neil, the data scientist, said the state should go further and check whether error rates run higher among some groups.
“It becomes a social justice issue if you have more poor black people getting harassed,” she said.
The State of Michigan is being sued after nearly two years of using a separate piece of FAST Enterprises software to identify unemployment claims as fraudulent with no human involvement.
The state recently admitted the computer system had a 93% error rate in those cases and made tens of thousands of false determinations with heavy financial penalties.
Wisconsin’s jobless claims division uses four different vendors to ferret out fraud — none of them FAST Enterprises — and then has employees follow up, Workforce Development spokesman John Dipko said.
These systems crossmatch data to identify unemployment insurance claimants who may have applied for benefits while they were incarcerated or outside the United States.
In the Medicaid health program for the needy, the state just approved a three-year contract with Lexis Nexis, said Lori Thornton, the deputy inspector general at the Department of Health Services.
The company will use proprietary software to generate leads for the state of potential fraud by care providers and overpayments to them.
March 12-18 is Sunshine Week, an annual nationwide celebration of open government, access to public information, and what the First Amendment means for us, our community and our democracy. Look for more stories this week as part of a collaboration between the Associated Press Media Editors, Associated Press News Editors and the Associated Press.