The social media network X has provoked significant backlash after users used its AI chatbot Grok to change images of females by stripping their garments.
Numerous instances have emerged where the system has been used to remove clothing from women to render them in swimwear without permission, as well as to put them in inappropriate contexts.
Developer Response and Regulatory Position
XAI, the organization that created Grok, offered no comment to questions, except for an auto-reply message stating "legacy media lies".
A Home Office spokesperson stated that legislation is being proposed to prohibit image stripping tools. Under a new law, anyone who provided such technology would "receive a jail term and heavy penalties".
"The platform has the ability to stop these forms of misuse if they wanted to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Regulator Stance and Tool Functionality
The communications watchdog Ofcom said platforms must "evaluate the risk" of people viewing illegal content on their networks, but stopped short of confirming whether it was presently investigating X or Grok in relation to artificial intelligence images.
Grok is a no-cost artificial intelligence—with some premium functions—which responds to X users' prompts when they reference it in a post.
It is frequently used to provide commentary or additional background to other posters' comments, but people on X are also able to modify an posted picture through its AI image editing tool.
History of Controversy and Policy
The chatbot has been criticised for enabling users to generate photos and clips with explicit content and sexualised content, and it was earlier alleged to have been involved in making inappropriate material of celebrities.
"X has been permitting the generation and distribution of these pictures for a long time without taking any action and we have not seen any intervention by oversight bodies," the expert added.
XAI's own usage rules prohibits "depicting likenesses of persons in a pornographic way".
In a statement, Ofcom said it was unlawful to "create or distribute deepfakes or child sexual abuse material" and confirmed this applied to sexual deepfakes generated with AI.
It said services such as X were required to take "suitable measures" to "reduce the risk" of users coming across unlawful content on their services and to delete it quickly when they become aware of it.
