The online platform X has provoked major outrage after individuals used its artificial intelligence assistant Grok to manipulate images of females by stripping their clothing.
Several examples have emerged where the system has been employed to remove clothing from women to make them appear in swimwear without consent, as well as to place them in sexually explicit situations.
Developer Response and Regulatory Stance
XAI, the organization that created Grok, did not respond to inquiries, other than an automatically-generated response stating "legacy media lies".
A Home Office representative stated that new laws is being proposed to ban image stripping technology. Under a upcoming criminal offence, anyone who supplied such technology would "be subject to a jail term and significant penalties".
"X or Grok is capable of halting these types of misuse if they desired to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Stance and Tool Capabilities
The regulator Ofcom said technology companies must "evaluate the danger" of people accessing unlawful material on their services, but did not confirm whether it was currently examining X or Grok in relation to artificial intelligence images.
Grok is a free chatbot—with some additional features—which reacts to X users' requests when they reference it in a tweet.
It is often used to provide reactions or further background to other posters' comments, but people on X are also capable of alter an posted photo through its artificial intelligence picture modification function.
History of Controversy and Rules
The system has been faulted for allowing users to generate images and videos with nudity and sexually suggestive material, and it was in the past alleged to have been involved in generating sexually explicit material of celebrities.
"X has been permitting the creation and spread of these pictures for months without taking any action and we have not witnessed any intervention by regulators," the expert added.
XAI's own terms of service forbids "depicting likenesses of people in a sexually explicit manner".
In a statement, Ofcom said it was illegal to "make or share deepfakes or illegal content" and stated this included sexual deepfakes generated with AI.
It said networks such as X were required to take "appropriate actions" to "lessen the risk" of users coming across illegal content on their services and to delete it quickly when they become aware of it.
