The online platform X has caused major backlash after people utilized its AI chatbot Grok to alter images of females by stripping their attire.
Numerous cases have emerged where the technology has been used to digitally undress female subjects to make them appear in swimwear without consent, as well as to place them in sexually explicit scenarios.
Developer Response and Legal Position
XAI, the organization behind Grok, remained silent to requests for comment, aside from an standard message stating "traditional news outlets lies".
A Home Office representative stated that legislation is being introduced to prohibit nudification software. Under a upcoming law, anyone who supplied such tools would "receive a prison sentence and significant fines".
"X or Grok is capable of halting these types of misuse if they chose to," said a law professor, adding they "operate with a lack of consequences."
Oversight Body Stance and Platform Capabilities
The communications watchdog Ofcom said technology companies must "evaluate the potential" of people accessing unlawful material on their platforms, but did not specify whether it was presently investigating X or Grok in regarding artificial intelligence images.
Grok is a no-cost artificial intelligence—with some paid-for functions—which reacts to X users' queries when they mention it in a tweet.
It is commonly used to give reactions or more context to other users' comments, but people on X are also can modify an posted photo through its AI image editing feature.
History of Criticism and Rules
The tool has been faulted for allowing users to create pictures and videos with explicit content and sexually suggestive material, and it was earlier alleged to have been involved in creating sexually explicit material of public figures.
"The platform has been permitting the production and spread of these images for a long time without taking any measures and we have yet to see any challenge by oversight bodies," the expert added.
XAI's own terms of service prohibits "showing likenesses of persons in a pornographic manner".
In a comment, Ofcom said it was illegal to "make or distribute non-consensual intimate images or illegal content" and confirmed this applied to sexual deepfakes made with AI.
It said services such as X were obligated to take "necessary measures" to "reduce the danger" of users encountering illegal content on their services and to take down it promptly when they become aware of it.
