Key Takeaways
- Agentic AI can rapidly analyze massive data sets, offering powerful tools for both defending against and enabling cybercrime.
- Recent IBM data shows a 44 % year‑over‑year increase in AI‑utilized attacks on public‑facing software and systems.
- High‑profile incidents, such as the Anthropic breach, illustrate how attackers employ their own AI models to uncover weaknesses in source code.
- AI‑driven phishing has eroded traditional tell‑tale signs (misspellings, odd phrasing), making social engineering far more convincing.
- Panelists at the Berkman Klein Center urged proactive regulation, proposing a “safe harbor” framework that rewards baseline security practices while limiting liability for unavoidable harms.
- Experts warn against allowing private‑sector “hack‑back” or autonomous offensive AI, citing risks of escalation and chaotic vigilante‑style cyber conflicts.
- Reliable digital identity verification is seen as a long‑term defense against AI‑enhanced phishing, but practical hurdles (privacy, pseudonymity) must be resolved.
Introduction
The rise of agentic AI—systems that can autonomously perceive, plan, and act—has sparked both excitement and alarm within the cybersecurity community. While these models can sift through vast quantities of data at unprecedented speed, the same capabilities can be turned against defenders, enabling attackers to discover vulnerabilities, craft convincing social‑engineering lures, and automate offensive operations. A recent Berkman Klein Center discussion brought together leading experts who agreed that the time for decisive regulatory action is now, before the technology outpaces our ability to contain its misuse.
The Growing Scale of AI‑Powered Cybercrime
According to a 2026 IBM study, cyberattacks that leverage AI against public‑facing software and applications have surged by 44 % year over year. This sharp increase underscores how quickly malicious actors are integrating AI into their toolkits, turning what was once a niche advantage into a mainstream threat. The data signal that traditional defenses, which often rely on static signatures and human‑driven analysis, are struggling to keep pace with the velocity and adaptability of AI‑driven attacks.
Case Study: The Anthropic Breach
One illustrative example is the November data breach of Anthropic, the creator of the Claude Code assistant. Attackers deployed their own AI models to scan Anthropic’s source code for weaknesses, ultimately exposing internal workings that could be exploited further. The incident highlights a troubling symmetry: while organizations use AI to improve productivity and security, adversaries can employ the same technology to reverse‑engineer defenses and identify blind spots that might otherwise remain hidden.
AI‑Enhanced Phishing: The End of Obvious Clues
Robert Knake, former deputy national cyber director, noted that phishing emails a year ago often contained conspicuous misspellings or non‑colloquial phrasing, making them easy to spot for a vigilant user. Today, AI models fine‑tune language to mimic legitimate correspondence, stripping away those tell‑tale signals. Consequently, even trained employees can struggle to differentiate genuine messages from sophisticated forgeries, increasing the success rate of credential‑stealing and malware‑delivery campaigns.
Regulatory Perspectives: Need for a Safe‑Harbor Approach
Panelists consensus was that business and government leaders must move beyond voluntary best practices and establish clear regulatory frameworks. Knake advocated for a “safe harbor” model: companies that adopt baseline security measures—such as running the most current, known‑secure versions of open‑source packages—would be shielded from liability for unavoidable harms, whereas those neglecting these basics could be held responsible. This approach aims to incentivize prudent security without stifling innovation through excessive legal exposure.
Challenges in Defining Liability and Scope
James Mickens, a Harvard computer‑science professor, cautioned that crafting such a regime is easier said than done. Historically, firms like Microsoft and Amazon have incorporated internal stopgaps to mitigate traditional breaches without formal government mandates. The advent of agentic AI shifts the threat model: an external human can issue “evil commands” to AI‑driven systems inside data centers, manipulating them to act maliciously. Any regulation must therefore precisely delineate which liabilities apply, what hardware and software standards constitute compliance, and how to address the dynamic nature of AI‑based threats.
The Complexity of Proactive Vulnerability Management
Josephine Wolff, associate dean for research at Tufts’ Fletcher School, highlighted another regulatory hurdle: expecting the private sector to continuously inventory and document every piece of code running across expansive networks is both essential and extraordinarily difficult. Without accurate inventories, organizations cannot swiftly locate and patch vulnerabilities when they arise, undermining the effectiveness of any reactive or preventive measures. Wolff argued that while documentation is critical, achieving it at scale demands substantial investment in tooling, processes, and expertise.
Rejecting “Hack‑Back” and Autonomous Offensive AI
All panelists warned against permitting private entities to engage in retaliation or “hack‑back” operations. Wolff contended that proliferating actors seeking self‑defense would likely increase chaos rather than deter attackers, comparing the scenario to deputizing vigilantes in the physical world. Mickens added that envisioning corporations deploying autonomous agentic firewalls that trace attackers and launch offensive counter‑strikes risks creating a high‑frequency‑trading‑like cyber arms race, where algorithms react in real time without human oversight—a scenario fraught with unintended escalation and collateral damage.
Digital Identity Verification as a Long‑Term Defense
To counter AI‑fueled phishing, the group envisioned a future where online identities could be verified with high confidence. Knake stressed that knowing, with certainty, whether a correspondent is a genuine person is essential for restoring trust in digital interactions. Mickens acknowledged the promise of digital IDs but pointed out practical obstacles: many users need to reveal only a facet of their identity (e.g., a pseudonym) for safety reasons, such as victims of abuse or whistleblowers. Any identity system must therefore accommodate selective disclosure while preventing abuse, a balance that remains elusive today.
Conclusion: Balancing Innovation and Protection
The discussion underscored a paradox: agentic AI offers extraordinary defensive potential—continuous monitoring, anomaly detection, and rapid response—but also equips adversaries with equally potent offensive capabilities. As AI capabilities evolve, so too must the policies, technical safeguards, and organizational practices that govern their use. Establishing clear liability frameworks, encouraging baseline security through safe‑harbor incentives, resisting the temptation of offensive vigilante tactics, and investing in reliable identity verification are critical steps toward a resilient cyber ecosystem. Only through coordinated, forward‑looking action can society harness the benefits of agentic AI while curbing its potential to undermine personal privacy, economic stability, and national security.