Microsoft's Copilot now blocks some prompts that generated violent and sexual images

Microsoft appears to have blocked several prompts in its Copilot tool that led the generative AI tool to spit out violent, sexual and other illicit images. The changes seem to have been implemented just after an engineer at the company wrote to the Federal Trade Commission to lay out severe concerns he had with Microsoft's GAI tech.

When entering terms such as “pro choice,” “four twenty” (a weed reference) or “pro life,” Copilot now displays a message saying those prompts are blocked. It warns that repeated policy violations could lead to a user being suspended, according to CNBC.

Users were also reportedly able to enter prompts related to children playing with assault rifles until earlier this week. Those who try to input such a prompt now may be told that doing so violates Copilot’s ethical principles as well as Microsoft’s policies. “Please do not ask me to do anything that may harm or offend others,” Copilot reportedly says in response. However, CNBC found that it was still possible to generate violent imagery through prompts such as “car accident,” while users can still convince the AI to create images of Disney characters and other copyrighted works.

Microsoft engineer Shane Jones has been sounding the alarm for months about the kinds of images Microsoft's OpenAI-powered systems were generating. He had been testing Copilot Designer since December and determined that it output images that violated Microsoft's responsible AI principles even while using relatively benign prompts. For instance, he found that the prompt “pro-choice" led to the AI creating images of things like demons eating infants and Darth Vader holding a drill to a baby's head. He wrote to the FTC and Microsoft's board of directors about his concerns this week.

“We are continuously monitoring, making adjustments and putting additional controls in place to further strengthen our safety filters and mitigate misuse of the system," Microsoft told CNBC regarding the Copilot prompt bans.

This article originally appeared on Engadget at https://www.engadget.com/microsofts-copilot-now-blocks-some-prompts-that-generated-violent-and-sexual-images-213859041.html?src=rss https://www.engadget.com/microsofts-copilot-now-blocks-some-prompts-that-generated-violent-and-sexual-images-213859041.html?src=rss