End of the line for child abuse deepfakes? The Grok crisis and its impact

Florian Bodoky

14/1/2026

Translation: Natalie McKay

The UK is planning to pass new legislation in response to Grok creating explicit deepfake images upon request – some of them of children. Despite Musk’s accusations of censorship, politicians there are taking decisive action. What about Switzerland?

Update from 15 January 2026

Tech company X and affiliated AI company xAI have announced new technological restrictions for their AI chatbot Grok. These are intended to prevent the creation and editing of sexualised images of real people. This means that requests to generate digital images of people in bikinis, lingerie or other revealing outfits will be blocked, even for paying users. That’s not all – X has also modified Grok’s image and photo editing features so that only paying users can access them. According to the company, this is another step intended to improve content control. This is X’s response to criticism primarily aimed at the non-consensual creation of explicit deepfakes.

The new measures came in direct response to the criticism of xAI made by the Attorney General of the State of California, Rob Bonta. Governor Gavin Newsom called for a swift investigation, and warned that the company could face serious consequences. At federal level, the U.S. Senate has passed the Defiance Act, giving those affected by non-consensual AI deepfakes the right to take civil action against the creators of such content. The bill is now set to be discussed in the House of Representatives.

However, there are considerable doubts as to whether these measures will work. For example, The Verge has noticed that measures such as blocking certain prompts or limiting image editing to paying users can be easily circumvented. These images can still be created via the Grok website, the Grok app and the X app. It’s also easy to get around the age checks that occasionally pop up, as they don’t require any proof, and users can enter a false date of birth. The portal also criticises xAI owner Elon Musk for shifting all responsibility for the incidents onto users.

Original article

The UK is taking action in response to deepfake images, including some of minors, created by the Grok chatbot. In other words, the artificial intelligence assistant that’s part of Elon Musk’s company xAI. The British government stated in a press release that it considers the creation and sharing of such content a criminal offence, and affirms that the relevant authorities «will have the backing of this government to use the full powers which Parliament has given them».

Liz Kendall, Secretary of State for Science, Innovation and Technology, announced that the government was planning to make the creation of non-consensual deepfakes depicting people undressed or in minimal attire a priority offence. Media regulator Ofcom has also opened an investigation into whether X has complied with its duties under the Online Safety Act.

The investigation is likely to focus on the question of whether X and its operators are taking proactive and sufficient action against abuse. The fact that X restricts image generation to paying users isn’t enough for the British authorities, which also consider the measures the platform has taken in response to be inadequate. Prime Minister Keir Starmer has made his stance very clear, declaring: «If X cannot control Grok, we will». He’s asked Ofcom for «all options to be on the table», meaning that Grok could potentially be banned in the UK.

Grok, X and Elon: what actually happened?

In 2025, xAI gave the chatbot an image and video generation feature. Users can also edit images via prompts. Within just a few days, sexualised images of real people, mainly women, spread on the platform. Some of this content depicted minors. The usual protective mechanisms didn’t always work: requests that clearly violated ethical and legal boundaries were carried out anyway. Despite repeated complaints and reports, the feature remained active at times. Owner Elon Musk initially reacted to the criticism and plans of the British government in his usual manner, publicly criticising them and denouncing the interventions as «censorship».

X recently limited Grok’s image feature to paying subscribers to enable traceability. X emphasises that illegal content will be deleted and the accounts responsible for it blocked. Grok itself also generated an apology. However, critics point out that technical blocks can still be circumvented, for example via external Grok pages.

What’s happening in the EU and Switzerland?

In response to the Grok incidents, the European Union developed regulatory frameworks such as the Digital Services Act (DSA) and the AI Act as early as last year to curb the spread of harmful AI content. The authority has requested X (linked page in German) to retain all internal Grok-related data and documents, and is checking that the company complies with European requirements. The EU announced that it considers sexualised deepfake images, especially of minors, to be «illegal», «appalling» and «disgusting», and is actively attempting to get them removed and to prevent them being created and shared in the first place. Switzerland doesn’t have specific legislation dedicated to deepfake technologies. In May 2025, the National Council rejected a motion by the Green Party (linked page in German) calling for a specific regulation to address deepfakes by a clear majority. The Swiss Federal Council argued that the existing «technology-neutral» legislation was sufficient. The Swiss executive cited the provision against identity fraud Swiss Criminal Code Art. 179 ff..

In addition, following the signing of the European Council’s AI Convention, the Federal Department of Justice and Police (FDJP) has been tasked with preparing a consultation draft, set to be available by the end of 2026. The Federal Data Protection and Information Commissioner (FDPIC) also referred to the Federal Act on Data Protection Act (FADP), revised in 2023, which «is directly applicable to AI».

However, a recent case shows that Switzerland is also taking real action against AI-generated content: the Swiss Federal Supreme Court (linked document in German) recently ruled that pornographic content featuring digitally de-aged adults who look like minors is illegal. This is based on provisions to protect against «non-genuine» child pornography in the Criminal Code. It turns out Swiss jurisdiction isn’t so toothless after all.

Florian Bodoky

Editor

Florian.Bodoky@digitecgalaxus.ch

I've been tinkering with digital networks ever since I found out how to activate both telephone channels on the ISDN card for greater bandwidth. As for the analogue variety, I've been doing that since I learned to talk. Though Winterthur is my adoptive home city, my heart still bleeds red and blue. more

---

Background information

Interesting facts about products, behind-the-scenes looks at manufacturers and deep-dives on interesting people.

Show all

These articles might also interest you