Key Takeaways
- The U.S. military’s push for an “AI‑first warfighting force” risks eroding human judgment through automation bias and skill atrophy.
- Even well‑trained leaders can defer to AI recommendations, potentially accepting flawed or overly aggressive suggestions.
- Rapid AI development outpaces current DOD guidance, which was last updated in 2023 before models like Claude reached multiple generations.
- Effective mitigation requires continual, field‑centric training, embedded AI‑savvy experts, and informal knowledge‑sharing networks moderated by the military.
- Maintaining appropriate human control means focusing not only on the technology itself but also on how soldiers and commanders think, learn, and decide when AI is present.
The Terminator Myth vs. Real‑World AI Risks
When people imagine AI and armed forces going awry, they often picture the Terminator franchise: a self‑aware Skynet launching nuclear war and deploying killer robots. The article notes, “There is no doubt that people need to maintain control over AI systems, and so it is essential for analysts and engineers to think about how to avert a Skynet‑style outcome.” Yet focusing solely on runaway models obscures a more insidious danger: AI’s impact on human cognition within the military.
Automation Bias and Skill Degradation
Decades of research show that reliance on automated advice can produce automation bias—the tendency to defer to machine judgments over one’s own. The piece cites classic examples: “people who blindly followed their navigation systems and drove their cars into lakes.” With AI, this bias is amplified; studies reveal that “relying on AI can impair skill development for new learners while degrading knowledge among experts.” When computer scientists and oncologists had AI assistance removed, both groups performed worse than before they used it, illustrating a loss of competence.
Implications for an “AI‑First” Force
Secretary of Defense Pete Hegseth’s vision of an “AI‑first warfighting force” could paradoxically weaken the very judgment it seeks to enhance. The article warns that commanders using AI for war‑game analysis or course‑of‑action development “risk diminishing the ability of commanders and soldiers to independently evaluate or develop creative military options.” Fatigued sailors reviewing threat images may accept a computer’s label of “enemy warship” even when their eyes suggest a harmless cargo tanker, while battlefield commanders might blindly adopt algorithm‑generated plans, losing their tactical edge.
Why Training Alone Isn’t Enough
The U.S. military invests heavily in selecting and disciplining senior leaders, yet the article argues that “AI is more capable than earlier technologies, and it is already overwhelming workers in other similarly disciplined fields.” In medicine, even seasoned doctors struggle to catch incorrect AI recommendations, and sycophantic chatbots have led educated users into delusions. With shrinking staff sizes driven by perceived AI efficiencies, these cognitive pitfalls could become more acute, especially as AI evolves faster than policy can keep pace.
Outdated Guidance and Training Gaps
The Department of Defense’s directive on autonomous weapons was last revised in 2023, “before Anthropic had even launched its first commercial model, Claude.” Since then, Claude has undergone more than twenty versions and gained “superhuman hacking capabilities.” The article points out that even if policies were updated as quickly as commercial models, the Pentagon would struggle to keep soldiers current: service members are juggling deployments in Iran, the Red Sea, Pacific patrols, maintenance, and readiness tests, often in remote locations with limited internet access. Flying personnel back for frequent retraining is technically possible but impractical for a technology that changes biweekly.
A Field‑Centric Learning Model
To preserve “appropriate levels of human judgment,” the article proposes overhauling military training. Junior officers need a broad, continuously updated education on AI’s capabilities and limits, while senior leaders must shift from occasional, schoolhouse‑style sessions to a “distributed, field‑centric approach.” This could involve YouTube‑style tutorials, GitHub‑like repositories for effective AI prompts, and developer communities akin to Discord, Reddit, or Slack where lessons are exchanged. The military can moderate such networks—requiring posters to identify themselves and employing expert moderators—to filter out harmful content while surfacing valuable insights.
Embedding Experts and Evaluating Impact
Beyond training, the Pentagon should embed knowledgeable engineers, data scientists, and legal advisers to help senior leaders navigate human‑AI interaction on the battlefield. These specialists could advise fire officers on how target displays affect perception, counsel generals on the merits and risks of AI‑enabled plans, and draft guidance to safeguard independent judgment. The article also stresses the need to evaluate how AI affects units over time: “getting a baseline understanding of judgment without the help of AI is essential to tracking AI’s positive and negative effects.” Such assessments would let commanders recognize when AI is being “overly solicitous,” much like they currently spot junior officers seeking favor, and ensure soldiers have sufficient time to decide.
Balancing Speed with Control
Ultimately, avoiding a Terminator‑like dystopia hinges on controlling not just the AI itself but also the human behavior surrounding it. The article concludes that the Pentagon must “rework how the military trains, deploys, and enables AI systems—and not just on engineering or buying them.” Doing so will take time and clash with Washington’s impulse to acquire and field AI rapidly, but it is necessary to preserve the judgment that underpins effective warfighting. By marrying rigorous, ongoing education with expert oversight and adaptive knowledge‑sharing, the armed forces can harness AI’s benefits while safeguarding the human mind that must remain in command.
https://www.foreignaffairs.com/united-states/how-ai-could-hollow-out-us-military