Academics query software use for risk assessment

UNIVERSITY of Otago researchers are warning of potential problems in government departments using computerbased riskprediction models, including a controversial new tool used by ACC to profile clients.

Associate Prof James Maclaurin, spokesman for the university’s Artificial Intelligence and Law in New Zealand Project, said the ACC used a computer model to assist staff managing claims.

More details about how the system operated should be provided, to ensure it was used in a ‘‘fair and evenhanded way’’, and was providing ‘‘good outcomes’’, he said.

ACC had recently indicated the tool was being used to make several kinds of predictions, including how long ACC would expect a claim to be managed.

This description left open the possibility that ACC used these predictions to minimise treatment times, either by intervening in patients’ treatment, or ‘‘more seriously’’ by declining applicants with long predicted treatment times, he said.

The tool makes predictions about future ACC cases using a database of information about 364,000 past claims lodged between 2007 and 2013.

Fellow project researcher Prof Alistair Knott, of the Otago computer science department, said this kind of predictive system guided a corporation whose decisions had ‘‘serious consequences for people’s lives’’.

But it was essential that governments and companies relying on them could answer several questions, including how accurate the tool was and how it had been evaluated.

The researchers also asked that if by ‘‘passing the buck’’ to the machine, ACC was also ducking its responsibility to make ‘‘fair and humane decisions’’ about treating people in need. They also asked if the tool implicitly discriminated against individuals on grounds such as age, ethnicity or gender.

ACC’s use of ‘‘data and analytics’’ told staff about how people were injured, their likely recovery time and how to help.

Since ACC started using its predictive tool, client satisfaction rose from 68% in June 2013 to 78% today, and the average time to set up weekly payments had dropped from 11 days in June 2014 to seven days, ACC said.

john.gibb@odt.co.nz