Study finds that AI spots fake news more easily than people can
CONCORD, N.H. — Researchers at the University of New Hampshire found that a machine learning model — a form of artificial intelligence — could better detect inadequacies in medical news than a layperson in a new study.
Ermira Zifla, assistant professor of decision sciences at UNH’s Peter T. Paul College of Business and Economics, said she decided to look into fake news during the pandemic, when dubious health claims circulated widely on social media.
“During COVID, all we had in social media was just ‘disputed’ or ‘not disputed,’ and I thought that wasn’t giving us a lot of information,” Zifla said.
There’s already a lot of research on using AI to evaluate fake news, according to Zifla, who wanted to create something that could offer more nuanced feedback about an article’s strengths and weaknesses. She said she sees that approach reflected in fact checking websites that will rank claims as “partially true” or “partially false.”
She worked with Burcu Eke Rubini, assistant professor of decision sciences and coauthor on the paper, to train a model on a data set of articles and press releases about new medical treatments.
Those articles had also been evaluated by health care experts and journalists based on 10 criteria, like the costs and benefits of a treatment, the harm it could cause, and whether the sources were independent. The articles were written between 2013 and 2018; some were highly ranked by experts, while others received low rankings.
Then, the researchers had 254 people without special medical expertise read and evaluate some of the articles and rate them based on the same criteria. They found the laypeople displayed an “optimism bias,” with most rating the articles to be satisfactory, while the AI was more critical and more closely aligned with expert evaluations of the articles.
“The goal is to help people evaluate things they read better, but there could be unintended consequences,” said Zifla when asked if she had concerns about implementing the technology.
“Of course, expert opinions are valued, but it is also very hard for experts to give opinions on everything,” Rubini said. “It’s just a tool to help.”
The AI they created is not currently available to use, but Rubini said because it was created with open source software, it could be made easily available at a low cost.