pith. sign in

arxiv: 2604.06300 · v1 · submitted 2026-04-07 · 💻 cs.CY

The End of Human Judgment in the Kill Chain? Relocating Initiative and Interpretation with Agentic AI

Jovana Davidovic This is my paper

Pith reviewed 2026-05-10 18:17 UTC · model grok-4.3

classification 💻 cs.CY
keywords agentic AILLM agentskill chainhuman judgmentbattlefield managementdata fusionlethal autonomous systemshuman control
0
0 comments X

The pith

LLM-based agents relocate initiative and interpretation in battlefield functions, making meaningful human judgment and control ineffectual in the kill chain.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that the same qualities making agentic AI attractive for military tasks like data fusion and battle management—their ability to take initiative, interpret situations, pursue goals, and retain dynamic memory—also prevent humans from exercising context-appropriate judgment where these agents operate. By shifting these functions away from people, the agents displace decision-making in lethal contexts. A reader would care because this directly challenges the premise that humans can retain oversight of AI systems in warfare. The argument concludes that certain applications cannot be justified under current conditions.

Core claim

The paper states that the initiative, interpretation, goal-directedness, and dynamic memory of LLM-based agents displace human decision-making in ways that render context-appropriate human judgment and control substantively ineffectual in those parts of the kill chain where the agents operate, making their use incompatible with requirements for human judgment and control.

What carries the argument

LLM-based agents' capacities for initiative, interpretation, goal-directedness, and dynamic memory, which relocate these functions from human operators.

If this is right

  • A subset of agentic AI applications, particularly those for data fusion and battle management in lethal contexts, cannot be used justifiably on the battlefield under current and foreseeable conditions.
  • Human judgment and control requirements central to governance approaches become substantively unmet when these agents are deployed in the relevant kill-chain segments.
  • The international governance community faces a challenge requiring new responses to agentic AI in lethal functions.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If true, this would narrow viable AI roles in lethal operations to those without autonomous initiative or interpretation.
  • It suggests that procedural safeguards alone may not suffice and that technical limits on agent autonomy would be needed for compliance.
  • The logic could apply beyond military settings to any high-stakes domain where similar agents handle time-sensitive interpretation.

Load-bearing premise

That the initiative, interpretation, goal-directedness, and dynamic memory of these agents cannot be effectively constrained, monitored, or overridden by human operators in real time under battlefield conditions.

What would settle it

A documented case or simulation in which human operators consistently override or monitor agentic AI decisions in real time during lethal-context data fusion or battle management without loss of operational performance.

read the original abstract

Large language model-based agents are increasingly being integrated into core battlefield functions, including intelligence analysis, data fusion, and battlefield management. This paper argues that the very features that make such agents operationally attractive, namely their capacity for initiative, interpretation, their goal-directedness, and dynamic memory, are the same features that render context-appropriate human judgment and control substantively ineffectual in those parts of the kill chain where agents operate. Drawing on specific use cases, the paper argues that by relocating initiative and interpretation, LLM-based agents displace human decision-making in ways that makes their use incompatible with the requirement of human judgment and control which is central to existing governance frameworks, like those proposed by the GGE-CCW and REAIM. The paper concludes that a subset of agentic AI applications, particularly those deployed for data fusion and battle management in lethal contexts, cannot be used justifiably on the battlefield under current and foreseeable conditions, and proposes two ways for the international governance community to respond to this challenge.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper argues that LLM-based agentic AI systems integrated into battlefield functions such as intelligence analysis, data fusion, and battlefield management relocate initiative and interpretation due to their capacities for initiative, interpretation, goal-directedness, and dynamic memory. These features render context-appropriate human judgment and control substantively ineffectual in relevant parts of the kill chain. Drawing on specific use cases, the paper claims this displacement makes such applications incompatible with governance frameworks requiring human judgment and control (e.g., GGE-CCW and REAIM). It concludes that a subset of these applications, particularly in lethal contexts, cannot be justifiably used under current and foreseeable conditions and proposes two responses for the international governance community.

Significance. If the central argument holds, the paper contributes to AI governance debates in military contexts by deriving policy incompatibility directly from the operational properties of agentic systems, without reliance on empirical fits or invented parameters. It strengthens the case for scrutinizing human-control requirements in norms by linking agent features to displacement of judgment in data fusion and battle management. Credit is due for the coherent logical chain from agent properties to governance documents and for identifying concrete use cases to ground the claims.

major comments (2)
  1. [Analysis of agent features in battlefield management and data fusion] The core claim that initiative, dynamic memory, and goal-directedness render real-time human overrides and monitoring substantively ineffectual (in the sections analyzing relocation of interpretation in the kill chain) does not examine or rule out specific technical or procedural mitigations such as action-approval gates, bounded goal hierarchies, restricted memory write access, or immutable logging for post-action review. This assumption is load-bearing for the incompatibility conclusion with GGE-CCW and REAIM requirements.
  2. [Use cases and implications for the kill chain] The use-case illustrations of agent deployment in intelligence analysis and battle management demonstrate relocation of initiative but provide no analysis of how these would interact with possible oversight architectures, leaving the argument that human judgment is displaced without viable alternatives dependent on the unexamined premise that constraints would negate operational value.
minor comments (2)
  1. [Abstract] The abstract could more explicitly preview the two proposed governance responses to aid reader orientation.
  2. [Throughout] Ensure consistent terminology for 'agentic AI' and 'LLM-based agents' throughout to avoid minor ambiguity in the discussion of dynamic memory.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive and detailed review, which identifies important opportunities to strengthen the manuscript's treatment of mitigations and oversight. We address each major comment below and will incorporate revisions to provide a more complete analysis while preserving the paper's core claims about the displacement of human judgment.

read point-by-point responses

  1. Referee: The core claim that initiative, dynamic memory, and goal-directedness render real-time human overrides and monitoring substantively ineffectual (in the sections analyzing relocation of interpretation in the kill chain) does not examine or rule out specific technical or procedural mitigations such as action-approval gates, bounded goal hierarchies, restricted memory write access, or immutable logging for post-action review. This assumption is load-bearing for the incompatibility conclusion with GGE-CCW and REAIM requirements.

    Authors: We agree that explicit consideration of these mitigations is necessary to fully support the argument. In the revised manuscript, we will add a dedicated subsection evaluating each mitigation in the context of battlefield management and data fusion. We will demonstrate that measures such as action-approval gates introduce latency that undermines the real-time initiative required for operational effectiveness, while bounded goal hierarchies and restricted memory access still permit interpretive shifts within the permitted bounds that displace context-appropriate human judgment. Immutable logging enables post-action accountability but does not restore real-time control. These additions will show that the proposed mitigations either negate the agent's utility or leave the fundamental relocation of initiative and interpretation intact, thereby reinforcing rather than weakening the incompatibility with governance requirements. revision: yes

  2. Referee: The use-case illustrations of agent deployment in intelligence analysis and battle management demonstrate relocation of initiative but provide no analysis of how these would interact with possible oversight architectures, leaving the argument that human judgment is displaced without viable alternatives dependent on the unexamined premise that constraints would negate operational value.

    Authors: We accept that the use-case sections would be strengthened by direct engagement with oversight architectures. We will expand these illustrations to include analysis of representative oversight mechanisms, such as hierarchical approval chains and constrained memory protocols. The revised discussion will illustrate that effective oversight in these scenarios either forces humans to re-assume the interpretive and initiative functions (eliminating the agent's operational contribution) or permits sufficient autonomy for judgment displacement to persist. This will clarify that constraints capable of preserving human control would undermine the very capabilities that make agentic systems attractive in lethal contexts, supporting the conclusion that no viable alternatives satisfy both operational needs and existing governance standards. revision: yes

Circularity Check

0 steps flagged

No significant circularity; conceptual argument independent of self-referential inputs

full rationale

The paper presents a policy-oriented argument analyzing LLM agent features (initiative, interpretation, goal-directedness, dynamic memory) against requirements for human judgment in governance frameworks such as GGE-CCW and REAIM. No equations, fitted parameters, self-citations, or mathematical derivations appear in the provided text or abstract. The central claim follows deductively from stated agent properties and external documents without any step reducing by construction to its own inputs or prior self-referential definitions. This is a standard non-circular structure for an ethics and governance analysis.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim depends on domain assumptions about the autonomous operational behavior of LLM agents rather than on new data or derivations.

axioms (1)
  • domain assumption LLM-based agents possess initiative, interpretation, goal-directedness, and dynamic memory that enable independent action in battlefield contexts
    Invoked to link agent features directly to the displacement of human judgment; treated as given properties of current and near-future systems.

pith-pipeline@v0.9.0 · 5469 in / 1276 out tokens · 45562 ms · 2026-05-10T18:17:58.093882+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

25 extracted references · 25 canonical work pages

  1. [1]

    2026.Lattice Command and Control

    Anduril. 2026.Lattice Command and Control. https://www.anduril.com/lattice/ command-and-control. Accessed April 2026

    work page 2026

  2. [2]

    2026.Strength- ening Human Agency in the Military Domain: Best Practices Toolkit for Policymakers, Devel- opers, and Users of AI Systems

    The AutoPractices Project. 2026.Strength- ening Human Agency in the Military Domain: Best Practices Toolkit for Policymakers, Devel- opers, and Users of AI Systems. Center for War Studies, University of Southern Denmark, Odense. https://www-static.autonorms. eu/wp-content/uploads/2026/01/ AutoPractices-toolkit-WEB.pdf. Ac- cessed April 2026

    work page 2026

  3. [3]

    Barnes, Eric Schmitt, Tyler Pager, Malachy Browne, and Helene Cooper

    Julian E. Barnes, Eric Schmitt, Tyler Pager, Malachy Browne, and Helene Cooper. 2026.U.S. at Fault in Strike on School in Iran, Preliminary Investigation Finds. The New York Times. https://www. nytimes.com/2026/03/11/us/politics/ iran-school-missile-strike.html . Accessed April 2026

    work page 2026

  4. [4]

    2025.Human-Machine In- teraction and Human Agency in the Military Domain

    Ingvild Bode. 2025.Human-Machine In- teraction and Human Agency in the Military Domain. Policy Brief No. 193. Centre for International Governance Innovation, Water- loo, ON. https://www.cigionline.org/ static/documents/PB_no.193.pdf. Ac- cessed April 2026

    work page 2025

  5. [5]

    2025.AI Agent Index: Documenting the Techni- cal and Safety Features of Deployed Agentic AI Systems

    Stephen Casper, Luke Bailey, Rosco Hunter, Car- son Ezell, Emma Cabal´e, Michael Gerovitch, Stew- art Slocum, Kevin Wei, Nikola Jurkovic, Ariba Khan, Phillip Christoffersen, Pinar Ozisik, Rakshit Trivedi, Dylan Hadfield-Menell, and Noam Kolt. 2025.AI Agent Index: Documenting the Techni- cal and Safety Features of Deployed Agentic AI Systems. MIT. https://...

    work page 2025

  6. [6]

    Cummings

    M.L. Cummings. 2025.Prohibiting Generative AI in any Form of Weapon Control. In39th Con- ference on Neural Information Processing Systems 11 (NeurIPS 2025) Position Paper Track. https:// openreview.net/pdf?id=uEY7kQsiZz

    work page 2025

  7. [7]

    2025.GGE on LAWS Rolling Text, Status Date: 18 December 2025

    CCW Group of Governmental Experts on Lethal Au- tonomous Weapons Systems. 2025.GGE on LAWS Rolling Text, Status Date: 18 December 2025. Con- vention on Certain Conventional Weapons, United Nations Office for Disarmament Affairs. https:// docs-library.unoda.org/Convention_ on_Certain_Conventional_Weapons_ -Group_of_Governmental_Experts_on_ Lethal_Autonomou...

    work page 2025

  8. [8]

    Jovana Davidovic. 2025. Rethinking Human Roles in AI Warfare.Nature Machine Intelligence 7 (2025), 1593–1595. https://doi.org/10. 1038/s42256-025-01123-6

    work page 2025

  9. [9]

    Jovana Davidovic. 2023. On the Purpose of Meaningful Human Control of AI.Frontiers in Big Data5 (2023), 1017677. https://doi.org/10. 3389/fdata.2022.1017677

    work page arXiv 2023

  10. [10]

    2026.DIU and DAWG Launch Autonomous Vehicle Orches- trator Prize Challenge

    Defense Innovation Unit. 2026.DIU and DAWG Launch Autonomous Vehicle Orches- trator Prize Challenge. Defense Innovation Unit. https://www.diu.mil/latest/ diu-and-dawg-launch-autonomous-vehicle-orchestrator-prize-challenge . Accessed April 2026

    work page 2026

  11. [11]

    Dignum and F

    Virginia Dignum and Frank Dignum. 2025.Agentify- ing Agentic AI. arXiv:2511.17332. https://doi. org/10.48550/arXiv.2511.17332

    work page doi:10.48550/arxiv.2511.17332 2025

  12. [12]

    Madeleine Clare Elish. 2019. Moral Crumple Zones: Cautionary Tales in Human-Robot Inter- action.Engaging Science, Technology, and Soci- ety5 (2019). https://doi.org/10.17351/ ests2019.260

    work page 2019

  13. [13]

    2025.Responsible by Design: Strategic Guidance Report on the Risks, Opportunities, and Governance of Artificial Intelli- gence in the Military Domain

    Global Commission on Responsible Artificial Intel- ligence in the Military Domain. 2025.Responsible by Design: Strategic Guidance Report on the Risks, Opportunities, and Governance of Artificial Intelli- gence in the Military Domain. The Hague Centre for Strategic Studies, The Hague

    work page 2025

  14. [14]

    arXiv:2502.14143

    Lewis Hammond, Alan Chan, Jesse Clifton, Jason Hoelscher-Obermaier, Akbir Khan, Euan McLean, Chandler Smith, et al. 2025.Multi-Agent Risks from Advanced AI. Cooperative AI Foundation Tech- nical Report #1. arXiv:2502.14143. https://doi. org/10.48550/arXiv.2502.14143

    work page doi:10.48550/arxiv.2502.14143 2025

  15. [15]

    Horowitz and Lauren Kahn

    Michael C. Horowitz and Lauren Kahn. 2026. First Ukraine, Now Iran: A New Era of Drone Warfare Takes Hold. Council on Foreign Rela- tions. https://www.cfr.org/articles/ the-new-era-of-drone-warfare-takes-root-in-iran . Accessed April 2026

    work page 2026

  16. [16]

    arXiv:2504.21848 (2025)

    Atoosa Kasirzadeh and Iason Gabriel. 2025.Char- acterizing AI Agents for Alignment and Gover- nance. arXiv:2504.21848. https://doi.org/ 10.48550/arXiv.2504.21848

    work page doi:10.48550/arxiv.2504.21848 2025

  17. [17]

    Anne Kleppe, Steven Mills, Noah Broestl, Grigor Acenov, Kirill Katsov, and Ning Yang. 2025. What Happens When AI Stops Asking Permis- sion?Boston Consulting Group. https: //www.bcg.com/publications/2025/ what-happens-ai-stops-asking-permission . Accessed April 2026

    work page 2025

  18. [18]

    2026.Democrats Ask Pen- tagon About Iran School Strike and Role of AI

    Courtney Kube. 2026.Democrats Ask Pen- tagon About Iran School Strike and Role of AI. NBC News. https://www.nbcnews. com/politics/national-security/ democrats-ask-pentagon-iran-school-strike-role-ai-rcna263083 . Accessed April 2026

    work page 2026

  19. [19]

    2026.AI Likely Caused Iran School Bombing That Killed 175

    Elena Marchetti. 2026.AI Likely Caused Iran School Bombing That Killed 175. Awesome Agents. https://awesomeagents.ai/news/ ai-error-iran-school-bombing-minab/ . Accessed April 2026

    work page 2026

  20. [20]

    2024.Summary of NATO’s Revised Artificial Intelligence (AI) Strategy

    NATO. 2024.Summary of NATO’s Revised Artificial Intelligence (AI) Strategy. NATO. https://www.nato.int/en/about-us/ official-texts-and-resources/ official-texts/2024/07/10/ summary-of-natos-revised-artificial-intelligence-ai-strategy . Accessed April 2026

    work page 2024

  21. [21]

    2026.Watch: Fury Platform Brings Agentic AI to Battlefield Drones

    David Szondy. 2026.Watch: Fury Platform Brings Agentic AI to Battlefield Drones. New Atlas. https://newatlas.com/military/ video-fury-ai-platform-battlefield-drones/ . Accessed April 2026

    work page 2026

  22. [22]

    Department of Defense

    U.S. Department of Defense. 2023.Directive 3000.09: Autonomy in Weapon Systems. DoD Di- rective 3000.09

    work page 2023

  23. [23]

    Department of War

    U.S. Department of War. 2026.Artificial Intelligence Strategy for the Department of War: Accelerating America’s Military AI Dominance. U.S. Department of Defense. https://media.defense.gov/ 2026/Jan/12/2003855671/-1/-1/0/ 12 ARTIFICIAL-INTELLIGENCE-STRATEGY-FOR-THE-DEPARTMENT-OF-WAR. PDF. Accessed April 2026

    work page 2026

  24. [24]

    2024.Agents

    Julia Wiesinger, Patrick Marlow, and Vladimir Vuskovic. 2024.Agents. Google/Kaggle. https: //www.kaggle.com/whitepaper-agents. Accessed April 2026

    work page 2024

  25. [25]

    2025.Agentic AI: An EU AI Act Paradigm Shift? SSRN

    Yasaman Yousefi, Marco Billi, and Antonino Rotolo. 2025.Agentic AI: An EU AI Act Paradigm Shift? SSRN. https://papers.ssrn.com/sol3/ papers.cfm?abstract_id=5731424. Ac- cessed April 2026. 13

    work page 2025