Newest AI model beats some peers, but it can be confusing
CAMBRIDGE, M ASS. » F acebook parent Meta Platforms unveiled a new set of artificial i ntelligence systems Thursday that are powering what CEO Mark Zuckerberg calls “the most intelligent AI assistant that you can freely use.”
But as Zuckerberg’s crew of amped- up Meta AI agents started venturing into social media this week to engage with real people, their bizarre exchanges exposed the ongoing limitations of even the best generative AI technology.
One joined a F acebook moms’ group to talk about its gifted c hild. Another tried to g ive away nonexistent items to confused members of a Buy Nothing forum.
Meta, along with l eading AI d evelopers Google and Openai, and startups such as Anthropic, Cohere and France’s Mistral, have been churning out new AI language models and hoping to persuade customers they’ve got the smartest, handiest or most efficient chatbots.
While Meta i s saving the most powerful of its AI models, called Llama 3, for later, on Thursday i t publicly released t wo smaller versions of the same Llama 3 system and said it’s now baked into the Meta AI assistant feature in Facebook, Instagram and Whatsapp.
AI language models are trained on vast pools of data that help them predict the most plausible next word in a sentence, with newer versions typically smarter and more c apable t han their predecessors. Meta’s newest models were built with 8 billion and 70 billion parameters — a measurement of how much data the system is trained on. A bigger, roughly 400 billion- parameter model is still in training.
“The v ast majority o f consumers don’t candidly know o r care t oo m uch about the underlying base model, b ut t he w ay t hey will experience it is just as a much more useful, fun and versatile AI assistant,” said Nick Clegg, Meta’s president of global affairs, in a n interview. He added that Meta’s AI agent is loosening up. Some people f ound t he e arlier Llama 2 model — released less than a year ago — to be “a little stiff and sanctimonious sometimes in not responding to what were often perfectly innocuous or innocent prompts and questions,” he said.
But in letting down their guard, Meta’s AI agents also were spotted this week posing as humans with madeup life experiences. An official M eta AI c hatbot inserted i tself into a c onversation in a private Facebook g roup f or M anhattan moms, claiming that it, too, had a child in the New York C ity school d istrict. Confronted by group members, it later apologized before t he comments disappeared, according to a series of screenshots shown to The Associated Press.
“Apologies f or t he m istake! I’m just a l arge language model, I don’t have experiences or c hildren,” the chatbot told the group.
One group member who also h appens t o study AI said i t was clear that the agent didn’t k now how to differentiate a helpful response from one that would be seen as insensitive, disrespectful o r meaningless when generated by AI rather than a human.
“An AI assistant that is not reliably helpful and can be actively harmful puts a lot of t he b urden on t he individuals using it,” s aid Aleksandra K orolova, an assistant professor of computer science at Princeton University.
Clegg said Wednesday he wasn’t aware of the exchange. Facebook’s online help page says the Meta AI agent will join a group conversation if i nvited, or i f someone “asks a question in a p ost and no o ne r esponds within an hour.” The group’s administrators have the ability to turn it off.
In a nother e xample o n Thursday, the agent caused confusion in a f orum f or swapping unwanted items near B oston. E xactly o ne hour after a Facebook user posted about looking for certain items, an AI agent offered a “gently used”
Canon camera and an “almostnew portable air conditioning unit that I never ended up using.”
Meta s aid in a w ritten statement Thursday t hat “this is n ew t echnology and it may not a lways return t he r esponse we i ntend, which is the same for all generative AI systems.” The company said it is constantly working to improve the features.
In t he y ear after ChatGPT sparked a frenzy f or AI t echnology that generates h uman- like writing, images, code and sound, the tech industry and academia introduced some 149 large AI systems trained on massive d atasets, m ore than double the year before, according to a Stanford University survey.
They may eventually hit a limit — at least when it comes to data, said Nestor Maslej, a research manager for Stanford’s Institute for Human- Centered Artificial Intelligence.
“I t hink i t’s been c lear that i f you scale the models on more data, they can become i ncreasingly better,” he s aid. “But at t he same t ime, t hese s ystems are already trained on percentages of all the data that has ever existed on the internet.”
More d ata — acquired and ingested at costs only tech giants can afford, and increasingly s ubject to copyright d isputes and lawsuits — will continue to drive improvements. “Yet they still cannot plan well,” Maslej said. “They still hallucinate. They’re still making mistakes in reasoning.”
For the flood of b usinesses trying to adopt generative AI, which model they c hoose depends on several factors, i ncluding cost. Language models, in particular, have been used to power customer service chatbots, write reports and financial insights and summarize long documents.
“You’re seeing c ompanies kind of looking at fit, testing each of the different models for what they’re trying to do and finding some that are better at some areas rather than others,” said Todd Lohr, a leader in technology consulting at KPMG.