The United Kingdom’s online safety regulator has opened a formal investigation into social media platform X following concerns over the circulation of sexualised artificial intelligence–generated imagery. The probe focuses on whether the platform has adequately enforced safeguards to prevent the creation and spread of explicit AI-generated content, particularly imagery that appears to depict real individuals without consent.

Focus on Compliance With Online Safety Rules

Regulators are examining whether X has breached obligations under the UK’s digital safety framework, which requires platforms to mitigate harms caused by illegal and abusive content. Authorities are assessing how X identifies, moderates, and removes AI-generated sexual material, and whether its existing systems are robust enough to protect users, especially minors.

Growing Concerns Over Non-Consensual AI Imagery

The rise of generative AI tools has intensified fears about deepfake pornography and manipulated sexual images being used for harassment, blackmail, and reputational harm. UK officials have highlighted that non-consensual sexualised AI imagery poses serious psychological and social risks, making platform accountability a key priority for regulators.

Scrutiny of Content Moderation Systems

As part of the probe, investigators are expected to review X’s content moderation policies, reporting mechanisms, and automated detection technologies. The regulator will also look at whether the platform responds quickly to user complaints and whether repeat offenders are effectively sanctioned.

Potential Consequences for the Platform

If violations are confirmed, X could face substantial penalties, including significant financial fines or legally binding directives to improve safety measures. The regulator has stressed that enforcement action is aimed at ensuring tech companies take proactive responsibility for managing emerging AI-related risks.

X Yet to Respond Publicly

At the time of reporting, X has not issued a detailed public response addressing the investigation. The company has previously stated that it supports free expression while working to limit harmful content, but regulators will determine whether those assurances translate into effective real-world protections.

A Test Case for AI Regulation

The outcome of this probe could set an important precedent for how AI-generated content is regulated in the UK and beyond. As governments worldwide grapple with the rapid evolution of generative technologies, the case underscores increasing pressure on digital platforms to balance innovation with user safety.