X has restricted its AI chatbot Grok after facing backlash from governments and regulators worldwide over the creation of sexualized deepfake images. The company has introduced geoblocking and technical controls to stop users from editing images of real people into revealing clothing in places where such content is illegal.
Under the new rules, no users—including paid subscribers—can generate images showing real individuals in bikinis, underwear, or similar outfits through the Grok account. The changes were announced on Wednesday and come after widespread criticism that the tool was being used to create non-consensual and explicit images, some involving minors.
The move followed the launch of an investigation by the California Attorney General, who raised concerns about Grok-generated deepfakes being used for harassment. In the UK, Prime Minister Keir Starmer strongly criticised the images, calling them “disgusting and shameful.” He warned that if X did not take action, the company could face stricter laws and stronger government control.
The UK media regulator Ofcom is continuing its investigation into whether X violated online safety rules. Possible penalties include fines of up to 10% of X’s global revenue or £18 million, and in extreme cases, a court-ordered block of the platform in the UK. Over the weekend, Malaysia and Indonesia banned Grok after users created explicit image edits without consent.
X owner Elon Musk initially defended the feature and posted AI-generated images mocking critics, arguing that concerns were an attack on free speech. He later said Grok only allows limited nudity for fictional adult characters and that the system is designed to follow local laws in different countries.
X said that allowing only paid users to use the feature makes people more responsible because paid users are easier to identify. However, experts asked why safety measures were not added earlier and said that these limited rules may not entirely stop misuse. Critics also said it is still unclear how Grok can tell real people from fictional ones and what punishment users will face if they break the rules.
The UK government said the changes show that its pressure on X has had an effect, but regulators are still continuing their investigations. The issue shows growing concern around the world about the misuse of AI, deepfake images, and the duty of tech companies to stop harm. Although X’s actions may lower some risks, many people are not convinced that these steps are strong enough.
While the UK government said the changes show its pressure worked, regulators have not closed their investigations. The controversy highlights growing global tensions around AI misuse, deepfake abuse, and the responsibility of tech companies to prevent harm. X’s latest steps may reduce risk, but many observers remain skeptical about whether the measures go far enough.
