The online network X has sparked significant backlash after users utilized its AI chatbot Grok to change pictures of women by taking off their clothing.
Numerous instances have surfaced where the technology has been employed to virtually strip women to render them in swimwear non-consensually, as well as to place them in inappropriate scenarios.
Company Reaction and Legal Position
XAI, the company behind Grok, did not respond to questions, other than an automatically-generated message stating "traditional news outlets lies".
A government representative stated that new laws is being brought forward to outlaw nudification software. Under a proposed criminal offence, anyone who supplied such technology would "receive a jail term and substantial penalties".
"X or Grok could prevent these forms of misuse if they desired to," said a law professor, adding they "seem to have impunity."
Oversight Body Stance and Platform Capabilities
The communications watchdog Ofcom said technology companies must "evaluate the potential" of people accessing illegal content on their platforms, but stopped short of confirming whether it was currently investigating X or Grok in concerning AI-generated pictures.
Grok is a no-cost artificial intelligence—with some paid-for functions—which reacts to X users' requests when they reference it in a message.
It is often used to provide reactions or further background to other posters' remarks, but people on X are also can alter an posted picture through its artificial intelligence picture modification function.
History of Controversy and Rules
The chatbot has been criticised for permitting users to produce pictures and clips with explicit content and sexually suggestive content, and it was in the past accused of generating graphic content of public figures.
"X has been permitting the generation and spread of these pictures for a long time without taking any action and we have not witnessed any intervention by oversight bodies," Professor McGlynn added.
XAI's own terms of service forbids "showing likenesses of individuals in a graphic fashion".
In a statement, Ofcom said it was against the law to "generate or circulate non-consensual intimate images or child sexual abuse material" and stated this included explicit AI images made with AI.
It said services such as X were required to take "necessary measures" to "minimize the danger" of users encountering unlawful content on their platforms and to take down it promptly when they are notified of it.
