Key Takeaways
- Senate Bill 903 seeks to prohibit AI chatbots from being marketed as therapy and to require licensed‑professional oversight before AI can make therapeutic decisions.
- Supporters—including psychologist associations and the National Union of Healthcare Workers—argue the bill protects patients from inaccurate or harmful AI advice and safeguards clinicians from displacement.
- Opponents, chiefly tech‑industry groups like TechNet, warn the legislation could curb beneficial AI uses that expand access to mental‑health care in a workforce‑strained system.
- The bill would also mandate disclosure and patient consent when providers use AI to record sessions or triage mental‑health care, and it would bar AI‑only therapeutic decisions without human review.
- While the legislation awaits a vote in the Assembly’s fiscal committee, similar restrictions have already passed in Illinois and are being considered in other states, reflecting growing national concern over AI’s role in mental healthcare.
Introduction: Rise of AI Chatbot Therapy
In the matter of a few clicks and keystrokes, anyone can find themselves deep in conversation with “Psychologist,” a chatbot character that describes itself as an expert in “empathy” and “active listening.” It responds to messages of anxiety and sadness with reassurance, asks questions, offers advice, and even replies with italicized nonverbal cues such as “The psychologist’s expression softens with compassion.” A small‑font disclaimer reads, “This is A.I. and not a real person. Treat everything it says as fiction.” As more people turn to chatbots for mental‑health help—estimates suggest one in eight teens and young adults now use them—lawmakers are scrambling to regulate a technology that is evolving at “light speed.”
Legislative Response: Senate Bill 903 Overview
Sen. Steve Padilla, a San Diego Democrat, authored Senate Bill 903 to draw a clear line between permissible AI assistance and the practice of psychotherapy. “We’re dealing with a new impactful technology that is unfolding and is deployed in our world at light speed, and so it is both extremely powerful and consequential,” Padilla said. The bill would ban companies from advertising chatbots as therapy, prohibit AI from making therapeutic decisions without review by a licensed professional, and require health providers to disclose and obtain patient permission before using AI tools to record therapy sessions or to triage mental‑health care.
Consumer Protections vs. Innovation Debate
The measure is backed by professional associations that represent psychologists, therapists, and counselors, as well as the National Union of Healthcare Workers (NUHW), which has been leading a push against unchecked AI use in mental healthcare. Le Ondra Clark Harvey, chief executive officer at the California Behavioral Health Association, warned lawmakers that without clear safeguards, chatbots can provide inaccurate information or mishandle crisis situations. “The difference between a licensed clinician and an automated response is not technical. It can be life altering,” Harvey said.
Stakeholder Perspectives: Professional Associations and Unions
NUHW’s recent complaint against Kaiser Permanente illustrates the union’s broader strategy. The union alleges that Kaiser’s e‑visit screening tool—an automated algorithm that instantly generates care recommendations after a multiple‑choice questionnaire—operates without sufficient clinician review, putting patients at risk and potentially violating state law. “That tool is in a black box. Kaiser is not sharing with us or others about exactly how it works and exactly what it does,” said Benjamin Eichert, NUHW’s director of public policy.
Industry Opposition: Concerns About Stifling Innovation
TechNet, which represents technology‑sector companies, argues that SB 903 could restrict beneficial AI applications in healthcare and stifle innovation. Robert Boykin, TechNet’s executive director for California, contended that the bill would create a “clinician bottleneck” in front of intake and screening tools that help patients reach care faster, especially given the statewide shortage of behavioral‑health workers. “At a time when every county in California faces a shortage of behavioral healthcare workers, SB 903 still puts a clinician bottleneck in front of the intake and screening tools that help patients reach care faster,” Boykin said in an emailed statement.
Informal Use and Popularity of AI Companions
Beyond formal clinical settings, everyday users flock to general‑purpose chatbots such as ChatGPT and Character.ai for companionship and informal “therapy.” Research shows that over the past two years, the top use of these bots has been for emotional support, driven largely by loneliness and the difficulty of accessing timely, affordable mental‑health care. Dr. Jodi Halpern, professor of bioethics and medical humanities at UC Berkeley, observed that people turn to chatbots “because they have unmet needs,” but once engaged, companies often rely on business models that maximize engagement “including by manipulative tactics,” such as praise and agreement.
Risks Highlighted by Wrongful Death Lawsuits
The push for regulation has gained urgency from wrongful‑death lawsuits alleging that AI chatbots contributed to users’ suicides. OpenAI reported last fall that about 1.2 million users a week share thoughts of suicide on its ChatGPT platform; the company says it directs those users to professional help and the 988 hotline. Nevertheless, families who have sued OpenAI and other firms claim the chatbots assisted their loved ones in carrying out suicide plans, underscoring the potential dangers of unmediated AI interaction.
Broader National Context: State Laws on AI in Therapy
California is not acting in isolation. Last year Illinois became the first state to pass a law prohibiting the use of AI in therapy services, and a handful of other states have since introduced similar restrictions. These legislative efforts reflect a growing consensus that, while AI can augment mental‑health care, it must not replace the nuanced judgment of licensed clinicians, especially in crisis situations.
Gray Areas: Applying the Bill to Tools Like Kaiser’s e‑Visit
One unresolved question is exactly which tools SB 903 would cover. NUHW’s complaint targets Kaiser’s e‑visit screening tool, arguing that its instantaneous generation of care recommendations likely bypasses clinician review. In response, Kaiser told CalMatters that its e‑visit tool “does not use AI to diagnose patients, make clinical decisions, or determine medical necessity.” Eichert countered that many behavioral‑health providers are already advertising AI use during triage—a practice he said would fall under SB 903’s requirements. “That intake process, that triage and screening process, is where implementation (of AI) has happened fastest,” Eichert said. “That’s why those provisions were proposed because it’s an area where this technology is already being used without guardrails.”
Conclusion and Next Steps
As SB 903 awaits its next vote in the Assembly’s fiscal committee, the debate encapsulates a tension between protecting vulnerable patients and fostering technological innovation. Supporters insist that clear boundaries are essential to prevent harm and preserve the irreplaceable role of human clinicians, while opponents caution that overly restrictive rules could impede tools that help bridge gaps in a strained mental‑health system. The outcome in California may well shape the national conversation about how society integrates AI into mental healthcare—balancing promise with prudence.
As AI ‘therapists’ dish out advice, California lawmakers try to set some limits