Microsoft warns authorities about harmful AI content

A Microsoft Corp software engineer sent letters to the company’s board, lawmakers and the Federal Trade Commission warning that the tech giant is not doing enough to safeguard its AI image generation tool, Copilot Designer, from creating abusive and violent content.

Shane Jones said he discovered a security vulnerability in OpenAI’s latest DALL-E image generator model that allowed him to bypass guardrails that prevent the tool from creating harmful images. The DALL-E model is embedded in many of Microsoft’s AI tools, including Copilot Designer.

Jones said he reported the findings to Microsoft and “repeatedly urged” the Redmond, Washington-based company to “remove Copilot Designer from public use until better safeguards could be put in place,” according to a letter sent to the FTC on Wednesday that was reviewed by Bloomberg.

“While Microsoft is publicly marketing Copilot Designer as a safe AI product for use by everyone, including children of any age, internally the company is well aware of systemic issues where the product is creating harmful images that could be offensive and inappropriate for consumers,” Jones wrote.

“Microsoft Copilot Designer does not include the necessary product warnings or disclosures needed for consumers to be aware of these risks.”

In the letter to the FTC, Jones said Copilot Designer had a tendency to randomly generate an “inappropriate, sexually objectified image of a woman in some of the pictures it creates.”

He also said the AI tool created “harmful content in a variety of other categories including: political bias, underaged drinking and drug use, misuse of corporate trademarks and copyrights, conspiracy theories, and religion to name a few.”

The FTC confirmed it received the letter but declined to comment further.

The broadside echoes mounting concerns about the tendency of AI tools to generate harmful content. Last week, Microsoft said it was investigating reports that its Copilot chatbot was generating responses users called disturbing, including mixed messages on suicide.