Key Takeaways
- The Trump administration’s June 2 AI executive order opted for a voluntary pre‑release review rather than a mandatory licensing regime, reflecting a preference for market‑based solutions over heavy‑handed regulation.
- Comparing AI models to FDA‑approved drugs is flawed: AI systems are not fixed compounds, their risks evolve with use, and conditioning release on government approval raises prior‑restraint concerns under the First Amendment.
- The Federal Risk and Authorization Management Program (FedRAMP) offers a workable model: it conditions federal contracts on independent, continuous security assessments without blocking commercial sale or speech.
- Extending the FedRAMP approach to AI would require Congress to condition federal acquisition of “high‑risk” models on independent evidence of the model’s safety, leaving public release to existing consumer‑protection and tort law.
- Implementation would proceed through the Federal Acquisition Regulation (FAR) via notice‑and‑comment rulemaking, ensuring transparency, reviewability, and adherence to the unconstitutional‑conditions doctrine.
- To avoid entrenching only large vendors, the statute should mandate open, no‑cost evaluation options, publish the evaluation methodology, and bar exclusive review channels.
- Real‑world stakes are illustrated by the DoD’s designation of Anthropic as a supply‑chain risk and the limited voluntary evaluations already performed by the Center for AI Standards and Innovation.
The Administration’s Choice: Voluntary Review Over Licensing
President Donald J. Trump postponed a planned AI executive order on May 21 and signed a revised version on June 2. The final order “declined to create the licensing regime that OpenAI’s Sam Altman had urged the Senate in 2023 to create—a federal agency empowered to license powerful models before release—and that Gary Marcus, testifying beside him, wanted modeled on the Food and Drug Administration.” Instead, it chose a voluntary federal review of advanced AI models before public release. As National Economic Council director Kevin Hassett framed the administration’s thinking, the goal was to release models “in the wild after they’ve been proven safe, just like an FDA drug.”
Why the Drug Analogy Falls Short – Technical Limits
The pharmaceutical analogy fails for two reasons. First, technically, “a drug is a fixed compound tested within a defined population before approval. A ‘frontier model’—a cutting‑edge, general‑purpose AI model—however, is not fixed. Its capabilities shift with fine‑tuning and updates, its attack surface expands when it is connected to external tools and agents, and its hazards vary with the user and the deployment context.” A one‑time regulatory clearance would certify a system that looks altogether different by the time it is actually used.
Why the Drug Analogy Falls Short – Constitutional Limits
Second, constitutionally, conditioning the public release of a general‑purpose AI system that generates text and code on prior government clearance implicates the prior‑restraint doctrine, which calls for a heavy presumption against the validity of ex ante government restrictions on expression. “Drug licensing raises no comparable problem because a pill is not a form of expression. An approval regime for AI models would have to survive constitutional scrutiny that a licensing regime for drugs never faces.”
FedRAMP as a Working Alternative
An existing alternative avoids both problems: the Federal Risk and Authorization Management Program (FedRAMP), codified in 2022, conditions the awarding of federal agency contracts on an independent assessment of security controls against a defined baseline. “A vendor without an authorization is not barred from the commercial market, but it is excluded from the federal one until it passes that assessment.” The author, who leads FedRAMP compliance strategy for a managed‑services platform, notes that the model works because the government acts as a purchaser rather than a regulator of primary conduct. Vendors remain free to sell commercially without authorization and are ineligible only for federal contracts until their product is approved, so the condition does not restrict public release or commercial sale and avoids the constitutional problem that licensing creates.
Continuous Assessment Mirrors AI’s Evolving Nature
FedRAMP’s strength lies in its continuous nature: it requires monthly vulnerability scans and plan‑of‑action reporting, an annual independent assessment, and re‑authorization when a system makes a significant change. “The standard tracks a system as it evolves—appropriate for a frontier model’s needs, since its capabilities shift after release.” While FedRAMP tests whether a cloud system is secure, it does not evaluate whether the model inside that system is dangerous. A service hosting a frontier model may need FedRAMP authorization today, yet that authorization does not ask whether the model can materially assist a cyberattack, the synthesis of a pathogen, or another high‑consequence misuse. The proposal borrows FedRAMP’s mechanism and aims it at that gap: independent evidence about the safety of the model, not only the security of the system.
Translating FedRAMP to Frontier AI
The U.S. Congress should condition the federal acquisition of high‑risk AI models on independent evidence of the model’s safety. Public release would remain a company decision, governed by the consumer‑protection, privacy, security, and tort law that already applies. Federal adoption, however, would require more: “Federal agencies and defense contractors should not use a model that can materially assist autonomous cyber operations, biological or chemical weapons development, large‑scale fraud, or military targeting based on a company’s own representations, alone.” A procurement condition is an exercise of the spending power, and the relevant limit is the unconstitutional‑conditions doctrine. In Rust v. Sullivan, the U.S. Supreme Court distinguished procurement conditions that merely define a federal program from conditions that leverage federal funds to control conduct outside of that program. A condition requiring an independent safety assessment would stay on the program‑defining side if drafted with discipline: it must test whether a model that the government would adopt has been independently evaluated for the capabilities relevant to that use, and it must not reach the developer’s unrelated speech, corporate structure, or commercial conduct. Drafted this way, it would regulate the federal purchasing relationship, not the expressive act of releasing a model.
Rulemaking and Transparency Through the FAR
Procurement conditions are implemented through the Federal Acquisition Regulation, meaning the substantive rule would proceed through notice‑and‑comment rulemaking rather than bypassing it. “The contours of ‘covered models’—the high‑capability systems Congress would designate by capability threshold as eligible for government contracts—the evidentiary standard, and any waiver process by which an agency could procure a model that lacks full authorization, would be built on a public record and remain reviewable.” That is a feature of the procurement route, not an evasion of administrative law.
Congressional Specificity After Loper Bright
The statutes that empower federal agencies matter more after the Supreme Court’s decision in Loper Bright Enterprises v. Raimondo, under which courts will no longer defer to an agency’s reasonable reading of an ambiguous statute. Congress would therefore need to specify the elements of a federal procurement condition for covered AI systems: define what models the regulation covers based on capability thresholds, state what information an AI developer must address in the evidence provided for federal use, place the Center for AI Standards and Innovation—the NIST body that already runs voluntary frontier‑model evaluations—in statute as a national‑security test range directed to publish its evaluation methodology, test categories, and evidentiary criteria, so that the public record shows what the government tests for and how, and instruct the Federal Acquisition Regulation Council to implement the proposed procurement condition. Specificity would protect this proposed scheme from concerns that an agency is encroaching upon Congress’s legislative authority.
Guarding Against Vendor Entrenchment
A co‑founder of the defense software company Palantir, Joe Lonsdale, warned that an FDA‑style regime would entrench dominant companies able to absorb the costs of bespoke review for each new AI model. The procurement model resists that outcome only if the statute requires the Center to publish its methodology for review, mandates no‑cost evaluation options for small developers and open‑weight projects (whose trained model parameters are published for anyone to download and run), and bars the creation of review channels available only to the largest firms.
Real‑World Urgency: The DoD’s Anthropic Designation
The need for such a program is not hypothetical. The U.S. Department of Defense designated the AI developer Anthropic a supply‑chain risk this year, a label once reserved for foreign adversaries, after negotiations over its safety guardrails collapsed. The Center for AI Standards and Innovation has completed more than 40 frontier‑model evaluations but under voluntary agreements, and the new order sets the threshold for a covered model through a classified process the public cannot see. Federal trust in frontier AI developers is currently granted and withdrawn by discretion. A statutory release‑evidence condition would replace that discretion with a reviewable, evidence‑based standard. The signed order chose voluntary review over a license. That instinct was right, but the mechanism was wrong. “A framework that a developer can decline, with no criteria yet for who qualifies as a trusted partner, leaves federal trust in the gift of whoever holds office. The instrument that fixes that should be procurement, not an honor system.”
By translating the lessons of FedRAMP into a procurement‑focused safety requirement, Congress can harness the government’s purchasing power to incentivize rigorous, independent evaluation of frontier AI models while preserving First Amendment freedoms, encouraging broad participation, and maintaining accountability through transparent, reviewable rulemaking.