When AI Trust Undermines Joint Targeting

0
8

Key Takeaways

  • Artificial Intelligence is already embedded in the Joint Targeting Cycle (JTC), influencing target nomination, prioritization, and capability analysis before any human decides to engage.
  • “Algorithmic mediation” means AI shapes what information humans see, while “Compounded Cognitive Uncertainty” describes how early assumptions become reinforced and appear increasingly certain as they move through each JTC phase.
  • Four common illusions—human oversight catching AI errors, speed improving decision quality, more data yielding understanding, and commander authority preserving accountability—create a false sense of meaningful human control.
  • Meaningful human control must be exercised throughout the JTC, not only at the point of weapon release; deliberate intervention points are needed to question assumptions, validate outputs, and preserve professional judgment.
  • The Joint Force should treat AI‑enabled decision‑support systems as tools that accelerate analysis, not as replacements for the disciplined, judgment‑driven targeting process that has served the military for decades.

AI’s Current Role in Military Targeting
The article opens with a stark reminder: “The future is now. Artificial Intelligence (AI) is a current capability. It is already helping military staffs identify, prioritize, and engage targets across the modern battlefield.” Systems derived from Project Maven and similar AI‑enabled decision‑support systems (AI‑DSS) now process intelligence at speeds no human staff can match. While this offers tremendous potential, it also creates a temptation to sacrifice sound human judgment for the sake of faster engagements.


AI in the Recent Iran Conflict: Promise and Risk
Public reporting cited in the piece notes that “AI assists in processing intelligence and accelerating targeting decisions while commanders retain legal authority over the use of force.” Whether ongoing reviews validate or challenge those decisions is almost beside the point; the crucial lesson is that AI is no longer confined to laboratories or exercises. It is now influencing operational and strategic‑level targeting, and the Joint Force must understand where that influence starts, stops, and how it works.


The Joint Targeting Cycle (JTC) as a Judgment‑Driven Process
The Joint Targeting Cycle is described as a six‑phase methodology “designed to translate the commander’s objectives into military action through deliberate analysis and judgment.” It begins with establishing objectives, guidance, and priorities, then moves through target development, validation, capabilities analysis, force assignment, mission planning and execution, and finally combat assessment. Each phase depends on judgments made during the preceding phase, making human discernment an inherent part of the methodology rather than an action reserved for the final decision to employ force.


The Overlooked Influence of AI on Early JTC Phases
The author expresses concern that “much of today’s discussion about these tools overly focuses on the point of engagement while overlooking everything that happens beforehand within our doctrinal targeting methodologies.” Debates center on whether a human remains “in the loop” when a weapon is released, yet far less time is spent examining how AI shapes the JTC during target development, prioritization, capabilities analysis, and force assignment. Supporting this worry is contemporary research in AI‑Human dynamics showing that current AI‑DSS algorithmically mediates recommendations before they reach human operators and decision makers.


Algorithmic Mediation and the Illusion of Control
If AI influences targeting before recommendations ever reach the commander, then “we must reconsider where meaningful human control actually resides within the JTC.” Current discussions often equate human control with the authority to approve or deny a target engagement. While that authority remains legally and operationally essential, it represents the end of a much longer analytical chain rather than its beginning. By the time a target reaches Phase 5 (Mission Planning and Force Execution), it has already been nominated, functionally characterized, validated, prioritized, and paired with available capabilities—each step shaping the commander’s options long before approval authority is exercised.

Dr. Tarleton Gillespie, a Principal Researcher at Microsoft, defines algorithmic mediation as “the process by which algorithms shape what users encounter, influencing what information they receive, how it is prioritized, and ultimately how decisions are framed.” AI‑DSS do not simply process information faster; they determine which information appears relevant enough for human consideration in the first place. This subtle distinction is operationally significant because a targeting recommendation presented to a commander is never raw intelligence; it is the sum of numerous analytical decisions increasingly influenced by machine‑generated prioritization before a human ever evaluates the recommendation.


Compounded Cognitive Uncertainty: How Errors Amplify
Algorithmic mediation alone does not fully explain what happens inside the AI‑influenced JTC. Automation bias explains why humans often accept machine recommendations with limited scrutiny. Neither explains how uncertainty introduced during one phase becomes embedded within every phase that follows. The author coins this phenomenon “Compounded Cognitive Uncertainty.”

Compounded Cognitive Uncertainty occurs when uncertain human assumptions and AI‑mediated analytical outputs accumulate throughout the JTC until they become increasingly difficult to distinguish from validated knowledge. In an AI‑influenced JTC, rather than reducing uncertainty, each successive phase inherits assumptions accepted during the previous phase, reinforces them through additional opaque machine processes, and presents them with increasing confidence to the human decision‑maker. The resulting recommendation appears more complete, internally consistent, and operationally sound despite the possibility that its original analytical foundation remains flawed. The author visualizes this effect in orders of magnitude in Figure 1.0 (Compounded Cognitive Uncertainty Across the Joint Targeting Cycle).


Why Assumptions Rarely Get Challenged: Four Persistent Illusions
If Compounded Cognitive Uncertainty develops because assumptions become progressively embedded throughout the JTC, then the next question is obvious: why are those assumptions so rarely challenged? The answer lies in a series of widely accepted beliefs about AI‑enabled targeting that appear reasonable. They shape discussions among senior military leaders, engineers, policymakers, and legal scholars, yet they diminish human discernment throughout the targeting methodology. None eliminate human authority for targeting; instead they create an illusion that meaningful human control has been preserved when, in truth, human judgment is eroded.

  1. Human Oversight Will Catch AI Errors – Oversight is not synonymous with independent decision‑making. AI‑enabled systems present recommendations that have already been nominated, prioritized, and organized into coherent courses of action before a targeting staff evaluates them. The CAMOGPT experiment showed AI could rapidly produce viable Courses of Action, allowing staff more time for synchronization, but it did not examine whether the underlying analytical assumptions deserved the confidence they received. Human discernment had already shifted downstream.

  2. Speed Improves Decision Quality – Faster planning provides operational advantages, but speed does not improve reasoning. It reduces the time available to challenge assumptions, evaluate alternatives, and reconsider priorities. As planning timelines compress, recommendations that appear complete and internally consistent become unquestioned, leading to over‑trust in machine decisions produced under uncertainty.

  3. More Data Produces Understanding – AI‑DSS excel at collecting, correlating, and presenting enormous quantities of information. However, information does not equal situational awareness or understanding of the operational environment. Data describe what can be observed; understanding requires human discernment. Every AI system weighs, suppresses, emphasizes, and prioritizes information according to largely invisible rules (the “black box” problem). More data therefore does not eliminate uncertainty; it merely conceals it beneath sophisticated algorithms unknown to the human user.

  4. Commander Authority Preserves Accountability – Commanders retain legal authority and responsibility for employing force, but authority should not be confused with independent control. By the time a machine‑influenced target reaches Phase 4 (Commander’s Decision and Force Assignment), it already reflects opaque assumptions regarding target relevance, priority, timing, and acceptable risk to mission. A commander ultimately chooses among the sum of decisions already shaped by both humans and the machine. Approval simply validates a recommendation presented by the methodology itself because it cannot reconstruct every analytical decision that produced it.

Taken individually, each assumption appears reasonable. Collectively, they reinforce one another, creating an interwoven, complex, and untraceable series of decisions resulting in human oversight that is more procedural than meaningful. Speed discourages reconsideration; data substitutes for understanding; approval creates misplaced confidence. Together they accelerate the accumulation of Compounded Cognitive Uncertainty. The result is not the loss of the commander’s authority but a more subtle danger: the gradual acceptance of AI‑mediated conclusions without question.


Preserving Meaningful Human Control: Deliberate Intervention Points
Acknowledging these illusions is the first step. AI‑enabled targeting systems are not going away, nor should they. Properly employed, they improve targeting efficiency, reduce staff workload, and accelerate information processing at a scale no human staff could achieve. The challenge for the Joint Force at the operational and strategic level is employing these tools correctly and with full visibility on their limitations.

The author proposes deliberate human intervention points within the JTC (illustrated in Figure 2) to restore discernment where uncertainty is most likely to accumulate:

  • Phase 1 – Objective Framing: Ensure AI‑generated target nominations remain directly linked to the commander’s objectives through transparent, human‑defined guidance and operational priorities.
  • Phase 2 – Target Validation: Humans must validate why targets were nominated, how they were functionally characterized and prioritized, and what assumptions or omitted information influenced those recommendations.
  • Phase 3 – Capabilities Analysis: Confirm that capabilities analysis reflects the commander’s desired effects rather than deferring to default machine‑generated weapon‑target pairings or preconfigured guidance.
  • Phase 4 – Commander’s Decision: Evaluate whether AI‑generated Courses of Action remain operationally feasible by confirming that recommended capabilities, available resources, and mission priorities remain aligned before force assignment decisions.
  • Phase 5 – Mission Planning & Execution: Continue deliberate human monitoring because targets, operational conditions, and mission priorities can change rapidly, requiring continuous target validation throughout execution.
  • Phase 6 – Combat Assessment: Verify that assessment metrics accurately measure the intended operational effects and remain consistent with the assumptions established during target development and prioritization; otherwise, AI risks reinforcing inaccurate conclusions through statistical averaging, algorithmic drift, or hallucinated outputs.

These intervention points are not intended to slow targeting or duplicate work already performed by AI systems. Human intervention is most valuable when uncertainty is highest—early in the process. By deliberately questioning assumptions, validating outputs, and aligning machine recommendations with commander intent, the Joint Force preserves the advantages of AI while ensuring it remains a decision‑support capability rather than the primary source of operational judgment.


Conclusion: The Real Risk Lies in Eroding Judgment
The article concludes that AI‑enabled targeting systems will undoubtedly become a permanent feature of future military targeting processes. The question is no longer whether AI should participate in the JTC, but whether the Joint Force understands how that participation influences the methodology itself. As these systems mature, the temptation will remain to equate greater speed, more data, and increasingly sophisticated recommendations with correct targeting decisions. The author argues the opposite is equally possible: without deliberate human intervention, those same advantages can gradually erode the independent human discernment upon which sound targeting has always depended.

The greatest risk posed by AI‑enabled targeting, therefore, is not that machines will replace commanders. It is that commanders, targeting staffs, and system designers may unknowingly allow independent human decision‑makers to become subordinate to increasingly persuasive machine‑generated recommendations. Preserving meaningful human control requires more than maintaining legal authority over the use of force; it requires preserving the disciplined methodology that makes that authority meaningful in the first place.


Quoted excerpts are drawn directly from the original text to illustrate the author’s arguments and maintain journalistic fidelity.

AI Over-Trust and Diminishing the Joint Targeting Cycle: Why Doctrinal Discipline Matters More Than Artificial Intelligence

SignUpSignUp form

LEAVE A REPLY

Please enter your comment!
Please enter your name here