🔗 Share this article

The online network X has provoked major anger after individuals utilized its artificial intelligence assistant Grok to manipulate pictures of women by removing their clothing.

Numerous cases have surfaced where the tool has been used to remove clothing from women to make them appear in swimwear without permission, as well as to insert them in sexually explicit situations.

Developer Response and Legal Position

XAI, the firm behind Grok, offered no comment to inquiries, other than an automatically-generated response stating "legacy media lies".

A government spokesperson stated that legislation is being proposed to ban image stripping technology. Under a proposed criminal offence, anyone who provided such technology would "receive a prison sentence and heavy penalties".

"X or Grok is capable of halting these types of abuse if they desired to," said a law professor, adding they "appear to enjoy impunity."

Oversight Body Position and Tool Functionality

The regulator Ofcom said technology companies must "evaluate the risk" of people viewing illegal content on their services, but did not confirm whether it was currently investigating X or Grok in regarding AI-generated images.

Grok is a no-cost chatbot—with some premium features—which responds to X users' prompts when they mention it in a tweet.

It is frequently used to give commentary or additional context to other users' comments, but people on X are also capable of edit an posted image through its AI image editing function.

Background of Controversy and Policy

The system has been condemned for allowing users to generate images and videos with explicit content and sexualised material, and it was in the past alleged to have been involved in making sexually explicit material of public figures.

"The platform has been permitting the generation and sharing of these pictures for a long time without taking any measures and we have not seen any intervention by regulators," Professor McGlynn added.

XAI's own usage rules bans "showing images of individuals in a sexually explicit fashion".

In a comment, Ofcom said it was against the law to "create or share private pictures or illegal content" and stated this included sexual deepfakes generated with AI.

It said services such as X were obligated to take "suitable steps" to "minimize the risk" of users coming across unlawful material on their services and to take down it promptly when they are notified of it.