Key Takeaways
- Daniel Stenberg, lead developer of cURL, tested Anthropic’s Mythos model via the Linux Foundation’s Project Glasswing program but never received direct access; an external party ran the scan and shared the results.
- Mythos initially flagged five items as “confirmed security vulnerabilities” in cURL, but after review only one was validated as a genuine issue; the rest were false positives or minor bugs.
- The single confirmed vulnerability is slated for a low‑severity CVE in the upcoming cURL 8.21.0 release and is described as insignificant.
- Stenberg concludes that the publicity surrounding Mythos is largely marketing hype, as its performance does not surpass that of existing AI‑powered code analysis tools.
- While AI tools excel at finding known classes of flaws, they have not yet discovered novel vulnerability types; effective use still requires human creativity in prompting and interpreting results.
- Stenberg remains open to experimenting with Mythos if direct access is granted, but he views current AI assistance as an augmentation of, not a replacement for, human security expertise.
Introduction and Context
Daniel Stenberg, the longtime maintainer of the widely used cURL library, recently participated in an evaluation of Anthropic’s Mythos model through the Linux Foundation’s Project Glasswing initiative. Glasswing aims to give high‑profile open‑source projects early access to cutting‑edge AI tools for security testing. Although Stenberg was promised access to Mythos, he never obtained a direct account; instead, a colleague with privileges executed the model against cURL’s codebase and forwarded the resulting report. Stenberg noted that, given his limited time for extensive prompt engineering, simply receiving a first‑pass scan and analysis would already be valuable. This setup frames his subsequent critique of Mythos’s capabilities and the surrounding hype.
Initial Scan Results
The Mythos scan, performed on a recent master‑branch commit of cURL’s git repository, returned a list of five items that the model labeled as “confirmed security vulnerabilities.” Stenberg admitted he had anticipated a more extensive catalogue of issues, given the model’s marketed prowess in uncovering security flaws. Consequently, the report felt underwhelming at first glance. He promptly shared the findings with his cURL security team for a thorough verification process, setting the stage for a more nuanced assessment of Mythos’s actual utility.
Team Evaluation and Filtering
After several hours of joint examination, Stenberg and his fellow security contributors reduced the initial five‑item list to a single confirmed vulnerability. Three of the original findings were identified as false positives; they highlighted shortcomings already documented in cURL’s API specifications rather than exploitable bugs. The fourth item was deemed a simple coding mistake that did not rise to the level of a security concern. This filtering exercise revealed that Mythos’s raw output required substantial human vetting to separate genuine threats from noise, a pattern familiar from earlier AI‑assisted analyses.
Nature of the Confirmed Issue
The lone validated vulnerability is scheduled for publication as a low‑severity CVE in tandem with cURL’s forthcoming 8.21.0 release, expected in late June. Stenberg characterized the flaw as trivial, noting that it would not cause anyone to “grasp for breath.” In addition to the security finding, Mythos surfaced several non‑security bugs that the cURL team is actively addressing; he praised the model’s descriptions and explanations for these issues as well‑crafted. Nonetheless, the overall security yield was modest, reinforcing his impression that Mythos did not deliver a breakthrough in vulnerability discovery.
Assessment of Hype vs. Reality
Stenberg’s blog post concludes that the considerable publicity surrounding Mythos is primarily a marketing effort rather than evidence of a transformative AI security advance. He sees no indication that Mythos uncovers issues at a higher or more sophisticated level than existing tools such as AISLE, Zeropath, or OpenAI Codex Security, which have already contributed hundreds of bug fixes to cURL over the past eight to ten months. While acknowledging that Mythos can perform competent analysis, he argues that its incremental improvement does not justify the exaggerated claims made by Anthropic.
Historical AI Interaction with cURL
The cURL project has a long history of employing static analyzers, fuzzers, and, more recently, AI‑powered code scanners. Stenberg remarked that these tools have collectively triggered between two and three hundred bugfixes merged into cURL in the recent 8‑10‑month window, with a dozen or more of those findings confirmed as CVEs. This extensive background positions cURL as an ideal testbed for evaluating whether a new AI model like Mythos offers any substantive advantage over its predecessors.
Limitations of Current AI Tools
Stenberg emphasizes a fundamental constraint: AI security tools are only as effective as the human knowledge encoded in their training data. Consequently, they excel at rediscovering established error patterns but have not yet demonstrated the ability to identify entirely novel vulnerability classes. He notes that, to date, no AI has reported a security flaw that represents a fundamentally new kind of weakness; instead, they surface fresh instances of known problems. This observation tempers expectations about AI’s autonomous discovery potential and underscores the continued necessity of human insight.
Role of Human Creativity and Future Outlook
Looking ahead, Stenberg envisions AI as a powerful adjunct to human researchers rather than a replacement. He believes that future security breakthroughs will arise when humans devise novel prompts, angles, or methodologies that steer AI models toward unexplored code pathways. In his words, “adding AIs to the mix gives the humans even more powerful tools to use, more ways to find problems.” Although he remains hopeful that he will eventually obtain direct access to Mythos for personal experimentation, he acknowledges that the promised access may be delayed or uncertain. Ultimately, Stenberg’s experience reinforces the view that AI‑enhanced code analysis is valuable, but its true impact hinges on skilled human guidance.