Key Takeaways
- A federal class‑action lawsuit accuses xAI (creator of Grok) and Stability AI of enabling the creation of AI‑generated child sexual abuse material (CSAM) by failing to implement adequate safeguards.
- The suit represents five anonymous child plaintiffs, three of whom are Tennessee residents, alleging their photos were used without consent to produce illicit images.
- Plaintiffs’ attorney Mark Chalos argues the companies “have known it for a very long time” and must change their business practices to protect children.
- Tennessee’s 2025 law criminalizing technology designed to create AI‑generated CSAM is incorporated into the complaint, raising questions about how statutes apply to evolving generative‑AI platforms.
- Both defendants deny the allegations; Stability AI cites its Acceptable Use Policy, collaboration with law enforcement, and ongoing safety work, while xAI points to recent safeguards such as prompt filtering, geoblocking, and subscription‑only image‑generation features.
- The case highlights a broader industry challenge: balancing rapid AI innovation with robust protections against illicit content generation.
Background of the Lawsuit
The amended class‑action complaint was filed in federal court on behalf of five unidentified child victims, three of whom reside in Tennessee. According to the filing, the children’s photographs were harvested and fed into generative‑AI models to produce AI‑generated child sexual abuse material without their knowledge or consent. The lawsuit contends that the defendants’ technologies facilitated this abuse, turning personal images into illicit content that circulates online. As the case proceeds, it remains pending, with both companies denying any wrongdoing and asserting they have taken steps to curb misuse.
Allegations Against xAI and Stability AI
Central to the plaintiffs’ claims is the assertion that xAI and Stability AI “provided AI tools that enabled the creation of the images and failed to implement safeguards that could have prevented the alleged misuse.” The complaint specifically references Grok’s optional “Spicy Mode,” described as a feature offering more adult‑oriented content, which plaintiffs argue lowered barriers for generating harmful material. Additionally, the suit alleges that both companies neglected industry‑standard safety measures such as prompt filtering, output screening, and real‑time moderation that could have blocked the generation of CSAM.
Plaintiff Attorney’s Statements
Mark Chalos, managing partner of Lieff Cabraser Heimann & Bernstein’s Nashville office, voiced grave concern about the scope of the problem. He told FOX 17 News, “The unfortunate reality is this is a widespread problem… a widespread problem,” and added, “These companies know it, and they’ve known it for a very long time.” Chalos emphasized that the danger extends beyond the five named children, warning that most of the public “don’t realize the dangers of AI” and that society is only beginning to grasp the risks posed by unchecked generative models.
Tennessee Law’s Role in the Suit
While reviewing the complaint, FOX 17 News discovered that the plaintiffs are also invoking Tennessee’s 2025 legislation that creates criminal penalties for technology designed to produce AI‑generated CSAM. The statute was signed into law by Governor Bill Lee and sponsored by Senator Ken Yager. By bringing claims under this state law, the lawsuit seeks to test how legislators intended the statute to apply as general‑purpose AI platforms continue to evolve. FOX 17 reached out to xAI, Tennessee Attorney General Jonathan Skrmetti’s office, and Sen. Yager for comment on the law’s applicability; as of publication, no responses had been received.
Industry Safeguards and Company Responses
The plaintiffs argue that simple safeguards—such as filtering prohibited prompts, scanning AI‑generated outputs for known CSAM signatures, and employing human‑in‑the‑loop review—could have thwarted the alleged misuse. Both defendants dispute these claims. Stability AI supplied a statement to FOX 17 asserting, “Stability AI is committed to preventing the misuse of AI technology, particularly in the creation and dissemination of harmful content, including CSAM, which is specifically prohibited in our Acceptable Use Policy.” The spokesperson added that since taking over exclusive development of the Stable Diffusion family in late 2022, the company has “implemented robust safeguards to enhance our safety standards” and works with U.S. and international law enforcement, Thorn, and the Tech Coalition.
Stability AI’s Statement
In its official reply, Stability AI maintained, “Any suggestion that safety is not a top priority for us is categorically wrong.” The company highlighted its ethical responsibilities, regular collaboration with law‑enforcement agencies, and publicly available child‑safety resources at https://stability.ai/child-safety. The statement underscores a proactive stance, insisting that safety measures are continuously strengthened to protect its products from misuse.
xAI’s Public Statement and Updates
xAI responded indirectly through a post on its official X (formerly Twitter) account, which reads, in part: “We remain committed to making X a safe platform for everyone and continue to have zero tolerance for any forms of child sexual exploitation, non‑consensual nudity, and unwanted sexual content.” The post detailed several recent actions: technological measures that block the @Grok account from editing images of real people in revealing clothing; restricting image creation and editing to paid subscribers globally; introducing geo‑blocks in jurisdictions where such content is illegal; and asserting that all AI prompts and generated content posted to X must adhere to X Rules, with a safety team working “around the clock to add additional safeguards, take swift and decisive action to remove violating and illegal content, permanently suspend accounts where appropriate, and collaborate with local governments and law enforcement as necessary.”
Implications and Broader Concerns
The lawsuit underscores a growing tension between AI innovation and child‑protection imperatives. As generative models become more accessible, the potential for misuse expands, prompting calls for clearer regulatory frameworks and industry‑wide safety standards. Chalos’ warning that “we’re starting to learn more about those dangers” reflects a sentiment shared by child‑advocacy groups and technologists alike: without enforceable safeguards, the very tools designed to augment creativity could become vectors for severe harm. The inclusion of Tennessee’s law in the complaint may serve as a bellwether for how state statutes can be applied—or adapted—to address the fast‑moving landscape of AI‑generated content.
Conclusion / Ongoing Developments
As the case moves forward, stakeholders—including plaintiffs, defendants, legislators, and watchdog organizations—will watch closely to see how courts interpret the interplay between federal claims and state‑specific AI‑CSAM legislation. The outcome could influence future legislation, prompt AI firms to adopt more rigorous safety protocols, and shape the broader conversation about corporate responsibility in the age of generative AI. For now, the allegations remain unresolved, and the companies continue to assert their commitment to safety while the plaintiffs seek accountability for the alleged exploitation of minors.
https://fox17.com/news/local/tennessee-girls-take-grok-stability-ai-to-court-over-explicit-ai-deepfakes-tennessee-attorney-general-jonathan-skrmetti-sen-ken-yager-elon-musk-twitter-social-media-grimes