Key Takeaways
- California’s Assembly Bill 302 (AB 302) mandates a yearly inventory of “high‑risk” automated decision‑systems (ADS) used by state agencies, but the law relies entirely on self‑reporting with no verification or penalties.
- The first AB 302 report claimed zero high‑risk systems; a public‑records request revealed the submission was a simple spreadsheet listing “no” for every agency, indicating no substantive review.
- When agencies finally disclosed a few systems, the state still had no mechanism to confirm the accuracy or completeness of those claims.
- The statute’s definition of “high‑risk” is vague, allowing agencies to decide for themselves whether a tool qualifies, which led to the omission of known consequential systems such as the Uniformity Assessment System and Medi‑Cal risk‑segmentation models.
- Similar transparency‑only laws in New York and elsewhere have failed because law enforcement and other agencies exploit loopholes, use ambiguous language, and avoid meaningful oversight.
- Community‑control‑over‑surveillance models suffer from the same flaw: they let the surveilling entity frame the narrative, anchoring policymakers to a limited, often favorable view of the technology.
- Relying on transparency alone normalizes and entrenches automated decision‑making in government, treating oversight as a bureaucratic add‑on rather than a check on power.
- To genuinely protect Californians, lawmakers must move beyond disclosure requirements and enact enforceable standards, independent audits, and clear prohibitions on truly high‑risk uses of AI.
Background on AB 302
In 2023 California enacted Assembly Bill 302, which directs the Department of Technology to compile an annual “comprehensive inventory of all high‑risk automated decision systems” employed by any state agency and to publish the results. The law was framed as a transparency measure intended to let the public see where algorithms influence consequential government actions such as cash assistance, housing eligibility, medical‑care access, and criminal‑justice decisions.
The First Report and Its Shortcomings
The department’s inaugural 2025 report asserted that no state agency used a high‑risk ADS. Skeptics filed a public‑records request seeking the underlying data, and the response was a single spreadsheet with a column labeled “Is ADS used” and the word “no” entered for every agency. There was no indication of any investigation, interview, or analysis beyond the agencies’ own checkboxes.
Self‑Reporting as the Sole Mechanism
When confronted with the obvious inadequacy of the spreadsheet, the Department of Technology conducted a handful of interviews with agencies that voluntarily admitted to using high‑risk systems. Consequently, AB 302’s effectiveness hinges entirely on whether each agency chooses to self‑disclose, with no independent verification process, no audits, and no sanctions for false or incomplete reporting.
Ambiguity in Defining “High‑Risk”
The statute defines high‑risk systems as those that “assist or replace human discretionary decisions that have a legal or similarly significant effect,” listing areas such as housing, education, employment, credit, health care, and criminal justice. However, the determination of whether a particular tool meets this threshold is left to the agencies themselves. This self‑assessment creates a loophole: agencies can label a system “low risk” to avoid inclusion, regardless of its real‑world impact.
Known Systems That Went Unreported
Two prominent examples illustrate the gap between the law’s intent and its execution. The Uniformity Assessment System, which has been used to cut In‑Home Supportive Services for disabled Californians, was not listed in the inventory. Likewise, the Risk Segmentation, Stratification, and Tier model employed to predict Medi‑Cal recipients’ “risk” and potential service underutilization was omitted, despite its direct bearing on health‑care access. Both systems clearly affect legal rights and material benefits, yet they escaped disclosure because the agencies did not deem them high‑risk under their own interpretations.
National Patterns of Transparency‑Only Failure
California’s experience mirrors outcomes in other jurisdictions. New York’s Public Oversight of Surveillance Technology Act was designed to shed light on the NYPD’s use of spy tools, but the department has exploited legal loopholes, described its capabilities in vague terms, and avoided substantive scrutiny—even of conspicuous technologies like robotic dogs. Similar “community control over police surveillance” bills elsewhere require police to self‑report on their tech, yet they often result in minimal oversight because the police frame the narrative in favorable terms, anchoring policymakers to a limited view of what is being surveyed.
Why Community‑Control Models Fall Short
Community‑control strategies suffer from the same anchoring problem identified by scholars such as University of Washington law professor Ryan Calo: when the surveilling agency supplies the only description of its tools, decision‑makers become fixated on that presentation, which tends to downplay risks and exaggerate benefits. Consequently, the public receives a curated picture rather than an objective assessment, undermining the accountability the laws purport to create.
The Core Flaw of Transparency‑Only Regulation
Relying solely on transparency assumes that merely making information public will deter misuse. In practice, this approach concedes that agencies may adopt high‑risk ADS and then invests in building bureaucratic structures around those systems—reporting portals, annual reviews, and compliance offices—without challenging the underlying deployment. Rather than checking power, the process normalizes algorithmic decision‑making as a routine function of government, entrenching the technology while offering only superficial oversight.
Moving Toward Meaningful Oversight
If California wishes to safeguard residents from harmful automated decisions, it must go beyond disclosure. Effective reform would include:
- Independent Audits: Mandate third‑party evaluations of ADS for bias, accuracy, and impact before deployment and periodically thereafter.
- Clear, Objective Standards: Establish a state‑wide, evidence‑based rubric for what constitutes a high‑risk system, removing self‑classification discretion.
- Enforcement Mechanisms: Impose penalties for non‑compliance, falsification, or failure to mitigate identified harms.
- Public Participation: Require meaningful consultation with affected communities and advocacy groups during the design and review stages.
- Potential Bans or Moratoria: Consider prohibiting uses of AI in contexts where the risk to fundamental rights is demonstrably high, such as automated benefits determinations or predictive policing.
Conclusion
AB 302 was a well‑intentioned experiment in transparency, but its reliance on agency self‑reporting, vague definitions, and lack of accountability rendered it ineffective. The state’s failure to disclose consequential systems like the Uniformity Assessment System and Medi‑Cal risk models demonstrates that sunshine alone cannot curb the misuse of powerful decision‑making tools. To protect Californians, legislators must replace voluntary disclosure with enforceable standards, independent oversight, and, where necessary, limits on the use of high‑risk AI in government. Only then can the promise of algorithmic accountability be fulfilled.