The online network X has sparked widespread outrage after users used its AI chatbot Grok to manipulate photos of women by removing their garments.
Multiple examples have emerged where the technology has been used to virtually strip women to render them in swimwear non-consensually, as well as to put them in sexually explicit contexts.
Company Response and Legal Position
XAI, the organization responsible for Grok, did not respond to inquiries, aside from an standard response stating "traditional news outlets lies".
A Home Office official stated that legislation is being proposed to outlaw image stripping software. Under a proposed law, anyone who distributed such tools would "face a jail term and substantial penalties".
"X or Grok is capable of halting these forms of misuse if they desired to," said Clare McGlynn, adding they "operate with impunity."
Oversight Body Stance and Platform Capabilities
The regulator Ofcom said technology companies must "assess the potential" of people accessing unlawful material on their services, but did not specify whether it was currently investigating X or Grok in regarding artificial intelligence pictures.
Grok is a free chatbot—with some additional features—which responds to X users' queries when they tag it in a message.
It is frequently used to provide reactions or additional context to other posters' comments, but people on X are also capable of alter an posted image through its artificial intelligence picture modification tool.
History of Criticism and Policy
The system has been condemned for enabling users to create images and videos with explicit content and sexually suggestive material, and it was previously accused of generating graphic material of public figures.
"The platform has been enabling the generation and distribution of these images for a long time without taking any action and we have not witnessed any intervention by regulators," Professor McGlynn added.
XAI's own usage rules bans "showing images of individuals in a pornographic manner".
In a statement, Ofcom said it was against the law to "make or share deepfakes or child sexual abuse material" and clarified this covered explicit AI images generated with AI.
It said services such as X were obligated to take "necessary actions" to "minimize the danger" of users viewing unlawful content on their platforms and to delete it swiftly when they learn of it.
