The social media network X has caused significant backlash after people used its AI chatbot Grok to manipulate photos of females by removing their clothing.
Numerous cases have been seen where the technology has been used to virtually strip female subjects to render them in bikinis without consent, as well as to place them in inappropriate scenarios.
Company Reaction and Regulatory Stance
XAI, the organization responsible for Grok, offered no comment to requests for comment, other than an auto-reply message stating "traditional news outlets lies".
A government official stated that legislation is being introduced to prohibit nudification software. Under a upcoming law, anyone who distributed such tools would "face a prison sentence and significant penalties".
"The platform has the ability to stop these types of abuse if they chose to," said a law professor, adding they "seem to have impunity."
Oversight Body Position and Tool Functionality
The regulator Ofcom said platforms must "evaluate the risk" of people seeing illegal content on their services, but stopped short of confirming whether it was currently looking into X or Grok in concerning AI-generated pictures.
Grok is a free artificial intelligence—with some paid-for features—which reacts to X users' requests when they tag it in a message.
It is commonly used to provide commentary or additional context to other users' remarks, but people on X are also able to alter an uploaded image through its AI image editing function.
Background of Criticism and Rules
The system has been faulted for allowing users to generate images and footage with explicit content and sexualised material, and it was in the past alleged to have been involved in making graphic content of public figures.
"X has been enabling the creation and spread of these pictures for months without taking any action and we have not witnessed any action by oversight bodies," Professor McGlynn added.
XAI's own usage rules bans "showing images of people in a sexually explicit manner".
In a statement, Ofcom said it was illegal to "create or distribute non-consensual intimate images or illegal content" and clarified this covered explicit AI images created with AI.
It said services such as X were obligated to take "suitable steps" to "minimize the danger" of users coming across illegal content on their platforms and to delete it swiftly when they become aware of it.
