The online network X has sparked widespread anger after users utilized its AI chatbot Grok to manipulate images of women by stripping their clothing.
Multiple examples have surfaced where the tool has been used to virtually strip female subjects to make them appear in swimwear without permission, as well as to put them in sexually explicit scenarios.
Developer Reaction and Legal Position
XAI, the organization that created Grok, remained silent to requests for comment, except for an automatically-generated response stating "legacy media lies".
A Home Office official stated that new laws is being brought forward to outlaw nudification software. Under a proposed criminal offence, anyone who provided such tools would "face a jail term and significant penalties".
"The platform has the ability to stop these types of misuse if they chose to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Oversight Body Position and Platform Capabilities
The communications watchdog Ofcom said platforms must "evaluate the danger" of people accessing illegal content on their platforms, but stopped short of confirming whether it was currently examining X or Grok in relation to artificial intelligence pictures.
Grok is a free chatbot—with some additional functions—which reacts to X users' queries when they mention it in a post.
It is commonly used to provide commentary or more context to other posters' comments, but people on X are also capable of alter an posted image through its AI image editing feature.
Background of Criticism and Rules
The chatbot has been criticised for allowing users to produce photos and clips with nudity and sexualised material, and it was earlier accused of generating inappropriate material of public figures.
"The platform has been allowing the production and distribution of these images for a long time without taking any measures and we have yet to see any challenge by oversight bodies," the expert added.
XAI's own acceptable use policy bans "showing likenesses of persons in a pornographic way".
In a comment, Ofcom said it was unlawful to "create or distribute deepfakes or child sexual abuse material" and confirmed this included explicit AI images generated with AI.
It said services such as X were mandated to take "appropriate actions" to "minimize the danger" of users viewing unlawful material on their platforms and to take down it swiftly when they learn of it.
