Key Takeaways
- Weak or poorly targeted AI regulation can reduce overall safety more than having no regulation at all.
- Regulation that focuses mainly on downstream users encourages model developers to “free‑ride” on safety investments.
- Strong, well‑placed regulation that obliges both model developers and downstream deployers improves safety for everyone and can also raise economic returns.
- The AI supply chain behaves like a prisoner’s dilemma: without confidence that others will invest in safety, each actor cuts back, leading to collectively worse outcomes.
- Policymakers should consider the entire AI value chain—model providers, integrators, and end‑users—rather than regulating isolated applications or developers alone.
Introduction to the Study’s Premise
A new academic paper published in the Proceedings of the National Academy of Sciences argues that inadequate or misdirected AI regulation can be counterproductive, potentially producing worse safety outcomes than no regulation at all. Conducted by researchers from Cornell University and Carnegie Mellon University, the study uses theoretical economics and game theory to examine how regulatory requirements shape incentives for safety investments across the AI supply chain. Rather than treating AI as a monolithic technology, the authors emphasize the need to view it as a network of interconnected actors whose decisions influence one another.
Distinguishing Two Regulatory Leverage Points
The researchers identify two distinct points where governments can intervene: the creators of general‑purpose AI models (e.g., OpenAI, Google, Anthropic) and the downstream firms that adapt those models for specific uses such as medical diagnostics, e‑commerce recommendation engines, or customer‑service chatbots. While regulating downstream applications seems intuitive—because many risks surface when AI is deployed in concrete settings—the study shows that this focus can unintentionally relieve model developers of safety responsibilities.
How Downstream‑Focused Regulation Triggers Free‑Riding
When downstream companies are held accountable for meeting regulatory safety standards, general‑purpose model providers may respond by cutting back on their own safety measures, such as third‑party audits or robust testing protocols. Principal author Benjamin Laufer describes this as a “free‑riding behavior” whereby the regulation enables model developers to offload the safety burden onto downstream specialists. Consequently, the overall level of safety investment can drop, even though each party appears to be complying with the rules placed on them.
The Upstream‑Downstream Feedback Loop
The study’s game‑theoretic model reveals that safety investments are strategic complements: each actor’s willingness to spend on safety rises when they expect the other party to do the same. If downstream firms anticipate that model providers will skimp on safety, they have less incentive to invest heavily themselves, fearing their efforts will be undermined. Conversely, if model providers expect downstream users to shoulder the safety load, they reduce their own expenditures. This reciprocal expectation can drive the system toward a sub‑optimal equilibrium where total safety is lower than it could be under cooperative investment.
Parallels to the Prisoner’s Dilemma
The dynamic mirrors the classic prisoner’s dilemma: absent assurance that others will contribute adequately to safety, each company rationally chooses to protect its short‑term economic interests by minimizing safety spending and relying on the other party. The result is a collectively worse outcome—lower safety for all—even though mutual cooperation would yield higher safety and, as the study shows, greater economic utility. Utility is defined here as a firm’s share of revenue minus its investment costs, capturing both profit and the net benefit of safety spending.
Benefits of Strong, Well‑Placed Regulation
When regulation is sufficiently strong and correctly positioned—requiring meaningful safety investments from both model developers and downstream deployers—the uncertainty that fuels free‑riding diminishes. Clear obligations create a framework in which each party knows the other will also be investing, aligning incentives toward complementary safety efforts. Under these conditions, the model predicts mutually beneficial outcomes: end‑product safety rises, and both upstream and downstream firms enjoy higher utility because their safety expenditures are justified by reduced risk and enhanced market trust.
Policy Implications for the United States
The findings are especially relevant to the current U.S. regulatory landscape, where federal efforts have largely targeted frontier model developers (addressing model safety, testing, and national security), while state legislatures have concentrated on high‑impact downstream uses such as AI in hiring, healthcare, and insurance. The study cautions against treating these layers in isolation. Instead, policymakers should design regulations that consider how obligations at one level affect behavior at the other, ensuring that safety responsibilities are shared rather than shifted.
Beyond the Regulation Versus Deregulation Debate
The research challenges the prevailing framing of AI policy as a simple choice between regulation and deregulation. It argues that the design and placement of regulatory measures are at least as important as their stringency. Thoughtful regulation that maps onto the actual structure of the AI supply chain can mitigate the incentive problems identified, whereas blunt or narrowly focused rules may exacerbate them. As Laufer notes, “AI involves a very complicated set of stakeholders and actors that each have their own contributions to the technology,” underscoring the need for a holistic regulatory perspective.
Conclusion: Toward Coordinated Safety Investment
In sum, the study demonstrates that weak or misaligned AI regulation can backfire, leading to lower safety than a laissez‑faire approach. Effective governance must target both ends of the AI value chain, establishing clear, reciprocal safety obligations that discourage free‑riding and encourage cooperative investment. By aligning incentives across model providers and downstream applicators, regulators can achieve safer AI systems while also supporting economic returns—a win‑win scenario that moves the debate beyond simplistic dichotomies toward nuanced, supply‑chain‑aware policy.