Study: Pa. web tool misled families seeking benefits
A study by two University of Michigan researchers found errors in a Pennsylvania public benefits screening tool that could have wrongly told people they were not eligible for benefits, when in fact they were.
The researchers tested a state- run online screening tool with tens of thousands of hypothetical family scenarios to see if the tool correctly or incorrectly predicted eligibility for public assistance programs.
The tool incorrectly told the researchers’ test cases that they were not eligible for a state child care subsidy in every instance, even though thousands should have been considered eligible, according to the paper.
The study was published in an academic computer science journal called Proceedings of the ACM on Human- Computer Interaction.
State human service officials say they are looking into any potential errors raised by the study and will correct any errors “as swiftly as possible.” Still, advocates say the study illustrates a larger problem with automating some government processes during a time of economic hardship.
As Pennsylvania faces doubledigit unemployment, more people are applying for programs such as food stamps and medical assistance. And with the state’s county assistance offices still closed to the public, many are doing so online.
Statewide, enrollment in
Medicaid has increased by more than 178,000 people ( 6.3%) since February. Enrollment in the Supplemental Nutrition Assistance Program, commonly called food stamps, is up 101,500 people ( 5.8%) since February, according to state statistics. More than 3 million Pennsylvanians are enrolled in Medicaid and more than 1.9 million in SNAP.
People can apply for benefits via the state’s COMPASS website, which stands for Commonwealth of Pennsylvania Access to Social Services. About two- thirds of benefits applications are now coming through COMPASS, according to state officials.
Because the application process is lengthy and complex, the state offers a much shorter “Do I Qualify?” eligibility screening tool intended to let people know if they will qualify for any of several different benefits.
This is the tool researchers tested with hypothetical applicants to see if it would correctly predict eligibility.
The tool screens for eligibility for Medicaid, SNAP, cash assistance, free or reducedprice school meals for children, and Child Care Works, a subsidy that helps to cover the costs of child care for low- income working families. It was this that researchers say presented the most wrong answers to potential applicants.
“While these tools are designed to help struggling, overwhelmed people decide whether to embark on the lengthy benefits application process, their inaccurate eligibility predictions may prevent deserving households from seeking benefits,” researchers Nel Escher and Nikola Banovic wrote in their paper.
State officials say the tool is not an official determination of someone’s eligibility. The officials encouraged anyone who is uncertain whether they qualify to apply for benefits “so they can work through the application with a caseworker who will be able to fully review income, resources and circumstances to assess what services or programs may be available.”
Ms. Escher said the issues raised by her paper speak to the broader problems with automating certain government processes. Often, as in this case, it is difficult to know if a system is functioning correctly without extensive testing, and there is a divide between those who built the technology and those who use it.
Someone who is told they won’t qualify for benefits — even if that isn’t correct — is much less likely to go through the lengthy application process, the researchers wrote. And even if an applicant eventually learns their initial assessment was incorrect, they have no way of knowing the error extends beyond their individual case.
“The nature of how these things work is obscure. It does take the work that that paper did to identify that it is happening at a large scale,” said Sophie Song, national organizer at Movement
Alliance Project. Her work focuses on inequality in algorithms and injustices in automated systems.
There should be regular and transparent auditing for this type of technology, said Aaron Rieke, managing director at Upturn, an organization that advocates for equity in the design and use of technology.
“The kinds of work [ the researchers] did here should be standard, and performed by the state. It’s almost always the case that problems are discovered after people start losing benefits, which is not the way things should go,” he said.