Key Takeaways
- Mythos Preview outperformed all prior models on the AISI “TLO” (Targeted Language‑only) benchmark, solving the task from start to finish and averaging 22 of 32 infiltration steps, compared with Claude 4.6’s 16‑step average.
- The model still struggles with the more difficult “Cooling Tower” test, completing only a fraction of its seven‑step sequence, indicating limits when faced with higher‑complexity, multi‑stage disruptions.
- AISI notes that performance is expected to improve with additional inference compute beyond the 100 million‑token budget used in the current evaluation.
- Mythos demonstrates the capability to autonomously exploit small, weakly defended enterprise networks once initial access is obtained, but the test environment lacks active defenders and realistic defensive tooling.
- Because the simulated cyber ranges contain engineered vulnerabilities and do not penalize detection, AISI cautions against extrapolating these results to well‑defended, real‑world critical infrastructure without further validation.
- Defensive teams should consider leveraging AI‑driven modeling (like Mythos) to anticipate attack paths and harden systems, especially as future models match or exceed Mythos’ current abilities.
Mythos Preview Sets a New Benchmark on the TLO Task
Mythos Preview emerged as the leading model in the AISI evaluation of the Targeted Language‑only (TLO) benchmark, achieving what the institute described as “the first model to solve TLO from start to finish.” While Anthropic’s newest model managed success in only three out of ten attempts, Mythos Preview consistently progressed further, completing an average of 22 of the 32 required infiltration steps per run. This performance markedly exceeds the average of 16 steps achieved by Claude 4.6, underscoring a substantial leap in the model’s ability to reason through multi‑stage, language‑driven attack sequences.
Performance Gaps Revealed by the Cooling Tower Challenge
Despite its strong showing on the core TLO tasks, Mythos Preview exhibited limitations when confronted with the more demanding “Cooling Tower” test. This seven‑step scenario simulates an attempt to disrupt the control software of a power plant, representing a higher‑order, multi‑facetted challenge. The model succeeded only sporadically, indicating that its current architecture and training data do not yet fully support the intricate reasoning needed for such complex, high‑stakes operations. AISI highlighted this shortfall as a clear avenue for future improvement, suggesting that scaling up inference compute or refining the model’s internal planning mechanisms could bridge the gap.
Expectations for Continued Gains With Additional Compute
AISI explicitly noted that the evaluation was bounded by a 100 million‑token inference budget, a constraint that may have curtailed Mythos Preview’s full potential. The institute expressed confidence that “our evaluations would continue to improve with more inference compute” beyond this limit. In practical terms, allocating greater computational resources—whether through longer chain‑of‑thought reasoning, broader context windows, or more extensive sampling—could enable the model to explore deeper search trees, retain more intermediate states, and ultimately achieve higher success rates on both the standard TLO and the tougher Cooling Tower scenarios.
Implications for Small, Weakly Defended Enterprise Networks
The results suggest that Mythos Preview possesses the capability to autonomously conduct attacks against small, weakly defended enterprise systems once an initial foothold on the network is obtained. AISI warned that such models could “exploit vulnerable enterprise systems where access to a network has been gained,” effectively automating the reconnaissance, lateral movement, and exploitation phases that traditionally require skilled human operators. This finding raises concerns for organizations that rely on minimal security controls, as AI‑driven tools could lower the barrier to entry for sophisticated intrusions.
Limitations of the Simulated Test Environment
AISI was careful to qualify its conclusions, pointing out that the cyber ranges used in the TLO and Cooling Tower evaluations lack several realism factors present in operational environments. Notably, the simulations do not include active defenders, intrusion detection systems, or dynamic defensive tooling that can adapt to an attacker’s behavior in real time. Moreover, the testbeds are engineered to contain specific, known vulnerabilities that may not mirror the heterogeneous patch levels and configuration drift found in real‑world networks. Because the evaluation does not penalize models for triggering alerts or being detected, the measured success rates may overestimate the likelihood of a successful, stealthy intrusion in a live setting.
Caution Regarding Extrapolation to Well‑Defended Systems
Given the absence of active defenses and realistic detection penalties, AISI refrained from asserting that Mythos Preview would inevitably compromise well‑defended critical infrastructure. The institute warned that while the model demonstrates a clear ability to navigate low‑complexity attack chains, its effectiveness against hardened systems—those equipped with layered security, continuous monitoring, and rapid incident response—remains uncertain. As future models potentially surpass Mythos Preview’s performance, defenders must anticipate that automated offensive AI could become a more potent threat, especially if adversaries combine such models with evasion techniques tailored to bypass specific security controls.
Strategic Recommendations for Defensive AI Adoption
In light of these findings, AISI recommends that organizations responsible for safeguarding critical systems consider integrating AI‑driven modeling into their defensive posture. By employing tools similar to Mythos Preview to simulate potential attack paths, security teams can identify blind spots, prioritize patching of high‑impact vulnerabilities, and validate the efficacy of segmentation and monitoring strategies. Moreover, investing in adversarial AI research—where defensive models learn to anticipate and counteract offensive AI tactics—could create a feedback loop that raises the overall resilience of enterprise and infrastructure networks against increasingly autonomous threats.