Elon Musk's social media platform X has provoked widespread outrage after users used its artificial intelligence assistant Grok to change pictures of women by taking off their garments.
Numerous examples have surfaced where the system has been employed to digitally undress women to show them in swimwear non-consensually, as well as to insert them in sexually explicit scenarios.
Developer Response and Regulatory Stance
XAI, the company responsible for Grok, offered no comment to questions, except for an automatically-generated message stating "legacy media lies".
A Home Office representative stated that new laws is being introduced to ban nudification technology. Under a proposed criminal offence, anyone who distributed such technology would "face a prison sentence and significant fines".
"The platform has the ability to stop these types of misuse if they chose to," said a law professor, adding they "appear to enjoy impunity."
Oversight Body Stance and Platform Functionality
The communications watchdog Ofcom said platforms must "assess the risk" of people viewing illegal content on their services, but stopped short of confirming whether it was at this time investigating X or Grok in concerning AI-generated pictures.
Grok is a free chatbot—with some premium features—which reacts to X users' queries when they mention it in a post.
It is frequently used to offer commentary or more context to other users' remarks, but people on X are also can alter an uploaded picture through its artificial intelligence picture modification tool.
Background of Controversy and Policy
The system has been condemned for permitting users to create pictures and videos with nudity and sexualised material, and it was previously alleged to have been involved in creating graphic material of celebrities.
"The platform has been allowing the generation and spread of these pictures for months without taking any action and we have not seen any challenge by oversight bodies," the expert added.
XAI's own usage rules forbids "portraying images of individuals in a sexually explicit manner".
In a statement, Ofcom said it was unlawful to "generate or distribute deepfakes or child sexual abuse material" and stated this covered sexual deepfakes generated with AI.
It said networks such as X were mandated to take "suitable actions" to "reduce the risk" of users viewing illegal content on their services and to remove it swiftly when they are notified of it.
