pith. sign in

arxiv: 2606.12442 · v1 · pith:XCAIRLV2new · submitted 2026-05-19 · 💻 cs.CY · cs.AI

Reframing AI Loss of Control: What It Is, How to Have It, How to Lose It

Ze Shen Chin , Maurice Chiodo , Dennis M\"uller , Coleman Snell This is my paper

Pith reviewed 2026-06-30 18:22 UTC · model grok-4.3

classification 💻 cs.CY cs.AI
keywords AI loss of controlgoal settingcyberneticscontrol theoryrequisite varietyAI safetygoal alignmentAI governance
0
0 comments X

The pith

Loss of control to AI is possible today with systems far below superintelligence when humans cannot set or achieve goals.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper defines control as the ability to set and get goals, then uses ideas from cybernetics and management to identify what is needed for control such as a working loop, enough variety to handle situations, and alignment on goals. It shows how AI can interfere with these elements to cause loss of control for people or groups. This reframing matters because it moves the discussion from distant superintelligent risks to scenarios that can already happen with current technology. The authors also outline steps to keep or regain control under this definition.

Core claim

By anchoring control to the setting and getting of goals and requiring a functional control loop, requisite variety, and goal alignment, the paper claims that AI behavior can produce loss of control in individuals and groups at levels well below superintelligence, and that such scenarios have already existed for a long time.

What carries the argument

Control defined as the setting and getting of goals, supported by a functional control loop, requisite variety, and goal alignment.

If this is right

  • Individuals and groups can already experience varying degrees of control loss due to AI that is not superintelligent.
  • Loss of control scenarios have existed for a long time under this definition.
  • Recommendations focused on preserving goal-setting ability, control loops, variety, and alignment can be applied immediately.
  • Control analysis applies to who or what sets goals and whether alignment holds.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The framework could be tested by measuring whether specific AI tools measurably reduce people's reported ability to set and reach personal or organizational goals.
  • Policy efforts might shift toward monitoring current AI deployments that affect goal alignment rather than waiting for advanced systems.
  • The same control elements could be examined in non-AI domains such as social media platforms or automated decision systems.

Load-bearing premise

Defining control strictly as the setting and getting of goals provides a sufficient and non-circular foundation that captures the essential features of control relevant to AI loss-of-control discussions.

What would settle it

A documented case in which widespread use of current AI systems prevents humans from setting or achieving goals yet produces no measurable reduction in control for the affected individuals or groups would falsify the central claim.

Figures

Figures reproduced from arXiv: 2606.12442 by Coleman Snell, Dennis M\"uller, Maurice Chiodo, Ze Shen Chin.

Figure 1
Figure 1. Figure 1: A hierarchy of goals, where lower-level (sub-)goals sit under higher-level goals. Not all goals have sub [PITH_FULL_IMAGE:figures/full_fig_p016_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The control loop. The control loop is also illustrated in [PITH_FULL_IMAGE:figures/full_fig_p017_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: A system is able to respond towards some, but not all, varieties of the environment and result in outcomes [PITH_FULL_IMAGE:figures/full_fig_p019_3.png] view at source ↗
read the original abstract

At present, loss of control risks have gained much prominence in public discussion, particularly in relation to AI, with extensive discourse present among academics, frontier labs, and even governments. However, in the existing literature, the concept seems to rest on surprisingly weak foundations, where even those that discuss loss of control extensively do not first establish what control is and what exactly is being lost. Our paper aims to address these gaps. We establish a working definition of control by anchoring it to the "setting and getting of goals". Then, we discuss various aspects of control, built on foundational concepts from related fields like cybernetics, management control, and control theory. This includes who (or what) can be in control, and the things they require to be in control, such as the ability to set goals, having a functional control loop, having requisite variety, and having sufficient goal alignment. Once a framework for control is established, we then discuss how control can be lost, how AIs can contribute to such loss of control, and offer relevant recommendations for how one can maintain control. One interesting consequence of our work is that humanity, as individuals and as groups, can lose varying degrees of control as a result of AI behaviour that is far below the level of superintelligence; the potential for loss of control scenarios (as we define them) already exist, and have existed for a long time.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 2 minor

Summary. The paper defines control as the setting and getting of goals, then builds a framework drawing from cybernetics, management control, and control theory that includes functional control loops, requisite variety, and goal alignment. It analyzes how AI systems can contribute to loss of control and offers recommendations for maintaining control. The central claim is that loss-of-control scenarios, as defined, can and do occur with sub-superintelligent AI and have existed for some time.

Significance. If the framework holds, it could expand AI governance discussions beyond superintelligence to include present-day algorithmic influences on human behavior and goal achievement. The attempt to integrate concepts from established fields is a positive step, but the absence of formal derivations, empirical validation, or falsifiable tests limits its ability to improve predictive accuracy over existing accounts.

major comments (3)
  1. [Definition section (early in manuscript)] The definition of control as 'setting and getting of goals' (introduced to support the subsequent framework) is not independently derived from control-theoretic or cybernetic literature; this makes the claim that loss-of-control scenarios already exist with current AI circular rather than demonstrated against external benchmarks.
  2. [Loss of control and AI contribution sections] The assertion that humanity can lose varying degrees of control due to sub-superintelligent AI behavior rests entirely on the chosen definition without concrete mappings to specific systems (e.g., recommender algorithms) or quantitative checks against measures like feedback stability or Ashby's law of requisite variety.
  3. [Recommendations section] Recommendations for maintaining control are presented qualitatively without linkage to falsifiable criteria or tests derived from the control-loop and requisite-variety elements of the framework.
minor comments (2)
  1. [Abstract] The abstract could more clearly distinguish the paper's novel contributions from prior work in cybernetics and AI alignment.
  2. [Framework sections] Notation for control concepts (e.g., control loops) remains informal; introducing simple equations or diagrams would improve precision.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for their constructive feedback. We respond point by point to the major comments below.

read point-by-point responses

  1. Referee: [Definition section (early in manuscript)] The definition of control as 'setting and getting of goals' (introduced to support the subsequent framework) is not independently derived from control-theoretic or cybernetic literature; this makes the claim that loss-of-control scenarios already exist with current AI circular rather than demonstrated against external benchmarks.

    Authors: We present the definition explicitly as a working definition chosen to anchor the framework for the purpose of analyzing AI systems, drawing inspiration from the cited literatures without claiming a formal derivation. This choice enables application to goal-directed AI behavior in a manner that highlights existing risks. We will revise the manuscript to state this more clearly, distinguish the definition from prior ones in control theory, and note that the framework's value lies in its subsequent application of established concepts such as control loops and requisite variety rather than in the definition alone. revision: yes

  2. Referee: [Loss of control and AI contribution sections] The assertion that humanity can lose varying degrees of control due to sub-superintelligent AI behavior rests entirely on the chosen definition without concrete mappings to specific systems (e.g., recommender algorithms) or quantitative checks against measures like feedback stability or Ashby's law of requisite variety.

    Authors: The manuscript is conceptual in nature and uses illustrative examples rather than quantitative validation. We will expand the loss-of-control and AI contribution sections with more detailed mappings to specific systems, including recommender algorithms, showing how they can disrupt human goal alignment and control loops. Quantitative checks against stability or requisite variety fall outside the paper's scope; we will add a limitations paragraph noting this and suggesting how the framework could support such analyses in future work. revision: partial

  3. Referee: [Recommendations section] Recommendations for maintaining control are presented qualitatively without linkage to falsifiable criteria or tests derived from the control-loop and requisite-variety elements of the framework.

    Authors: We will revise the recommendations section to explicitly tie each recommendation to the framework elements, for instance by indicating how control-loop functionality or requisite variety might be assessed through observable indicators. Full falsifiable tests require empirical studies beyond this conceptual paper; we will include suggestions for potential evaluation approaches derived from the framework. revision: yes

Circularity Check

0 steps flagged

No significant circularity; derivation self-contained via external fields

full rationale

The paper opens by establishing a working definition of control anchored to 'setting and getting of goals' and then explicitly builds subsequent elements (functional control loops, requisite variety, goal alignment) on concepts drawn from cybernetics, management control, and control theory. The central claim—that loss-of-control scenarios as defined already exist with sub-superintelligent AI—is presented as a consequence of applying this externally grounded framework rather than a quantity fitted to or defined in terms of the target result. No equations, fitted parameters, self-citation chains, or uniqueness theorems appear in the abstract or described structure that would reduce any prediction or conclusion to the inputs by construction. The derivation therefore remains independent of the chosen definition and does not meet the criteria for any enumerated circularity pattern.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The paper rests on a single ad-hoc definition of control chosen by the authors and standard background assumptions from cybernetics without independent empirical grounding or formal verification.

axioms (1)
  • ad hoc to paper Control is defined as the setting and getting of goals
    This working definition is introduced in the abstract as the foundation for all subsequent discussion of loss of control.

pith-pipeline@v0.9.1-grok · 5794 in / 1267 out tokens · 33786 ms · 2026-06-30T18:22:02.237334+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

13 extracted references · 11 canonical work pages · 2 internal anchors

  1. [1]

    NORMAL ACCIDENTS

    ISSN 2644-0865. doi: 10.1287/inte.2022.1143. URL https://pubsonline.informs.org/doi/10.12 87/inte.2022.1143. Silvia Amaro. Dutch government resigns after childcare benefits scandal. CNBC, January 2021. URL https://ww w.cnbc.com/2021/01/15/dutch-government-resigns-after-childcare-benefits-scandal-.html . Section: Europe News. Dario Amodei, Chris Olah, Jaco...

    work page doi:10.1287/inte.2022.1143 2022

  2. [2]

    ChatGPT Psychosis

    ISSN 1573-0964. doi: 10.1007/s11229-023-04367-0. URL https://link.springer.com/article/10 .1007/s11229-023-04367-0 . Maggie Harrison Dupr´e. People Are Being Involuntarily Committed, Jailed After Spiraling Into “ChatGPT Psychosis”. Futurism, June 2025. URL https://futurism.com/commitment-jail-chatgpt-psychosis . Emile Durkheim. The Division of Labor in So...

    work page doi:10.1007/s11229-023-04367-0 2025

  3. [3]

    URL https://www.cambridge.org/core/product/identifier/S303 3373325100410/type/journal_article

    doi: 10.1017/cfl.2025.10041. URL https://www.cambridge.org/core/product/identifier/S303 3373325100410/type/journal_article. Nicolas Falliere, Liam O Murchu, and Eric Chien. W32.Stuxnet Dossier. Symantec, 2011. URL https://nsarchiv e.gwu.edu/document/21440-document-44. 49 Molly Q Feldman and Carolyn Jane Anderson. Non-Expert Programmers in the Generative A...

    work page doi:10.1017/cfl.2025.10041 2025

  4. [4]

    Karen Hao

    URL https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4282704. Karen Hao. Empire of AI: Dreams and Nightmares in Sam Altman’s OpenAI. Penguin, May 2025. ISBN 978-0-593- 65751-5. John Hattie. Visible Learning for Teachers: Maximizing Impact on Learning. Routledge, London, March 2012. ISBN 978-0-203-18152-2. doi: 10.4324/9780203181522. 50 Connor Heaton, S...

    work page doi:10.4324/9780203181522 2025

  5. [5]

    Christopher Hilton

    URL https://www.nytimes.com/2025/06/13/technology/chatgpt-ai-chatbots-conspiracie s.html. Christopher Hilton. Ayrton Senna: The Whole Story. Haynes, 2004. ISBN 978-1-84425-096-7. Richard Hollingham. Apollo in 50 numbers: The workers. BBC Future, June 2019. URL https://www.bbc.com/ future/article/20190617-apollo-in-50-numbers-the-workers . Koen Holtman. Co...

    work page doi:10.1109/icsc65596.2025.11140447 2025

  6. [6]

    Bruno Latour

    URL https://proceedings.mlr.press/v162/langosco22a.html. Bruno Latour. Reassembling the Social: An Introduction to Actor-Network-Theory . Oxford University Press, July

  7. [7]

    and Zelnikov, Andrei , title =

    ISBN 978-0-19-925604-4. doi: 10.1093/oso/9780199256044.001.0001. URL https://doi.org/10.109 3/oso/9780199256044.001.0001. Michael Lawrence, Thomas Homer-Dixon, Scott Janzwood, Johan Rockstr¨om, Ortwin Renn, and Jonathan F. Donges. Global polycrisis: the causal mechanisms of crisis entanglement. Global Sustainability, 7:e6, 2024. ISSN 2059-

    work page doi:10.1093/oso/9780199256044.001.0001 2024

  8. [8]

    URL https://www.cambridge.org/core/product/identifier/S2059479 824000012/type/journal_article

    doi: 10.1017/sus.2024.1. URL https://www.cambridge.org/core/product/identifier/S2059479 824000012/type/journal_article. Edward A. Lee. Are We Losing Control? In Perspectives on Digital Humanism. Springer, November 2021. doi: 10.1 007/978-3-030-86144-5 1. URL https://link.springer.com/chapter/10.1007/978-3-030-86144-5_1 . Ren Bin Dixon Lee and Heather Fras...

    work page doi:10.1017/sus.2024.1 2024

  9. [9]

    The Mythos of Model Interpretability

    arXiv:1606.03490 [cs]. Shaoshan Liu, Anina Schwarzenbach, and Yiyu Shi. Human Resilience in the AI Era – What Machines Can’t Replace, October 2025. URL http://arxiv.org/abs/2510.25218. arXiv:2510.25218 [cs]. Walter Lord. A Night to Remember. Macmillan, December 2004. URL https://us.macmillan.com/books/978 0805077643/anighttoremember/. Igor Lutsenko. Princ...

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.15587/1729-4061.2016.79356 2025

  10. [10]

    Dennis M ¨uller

    URL https://www.reuters.com/business/swedens-klarna-shifts-ai-focus-cost-cuts-gro wth-2025-09-10/ . Dennis M ¨uller. The Mathematisation of the World: Uncovering the Socio-Economic Tensions for Ethics in Mathe- matics Education, 2025. URL https://arxiv.org/abs/2510.23349. Version Number: 1. Dennis M ¨uller, Maurice Chiodo, and James Franklin. A Hippocrati...

    work page doi:10.1007/s11948-022-00389-y 2025

  11. [11]

    Elika Somani, Anjay Friedman, Henry Wu, Marianne Lu, and Christopher Byrd

    doi: 10.1037/0022-3514.71.3.549. Elika Somani, Anjay Friedman, Henry Wu, Marianne Lu, and Christopher Byrd. Strengthening Emergency Pre- paredness and Response for AI Loss of Control Incidents. RAND Research Reports , 2025. URL https: //www.rand.org/pubs/research_reports/RRA3847-1.html. Lisa K. Son and Bennett L. Schwartz. Applied Metacognition. Cambridge...

    work page doi:10.1037/0022-3514.71.3.549 2025

  12. [12]

    Devisetty Sai Tharun, Panguluri Sai Srija, Peeta Vamsi Krishna, Shaiju Panchikkil, and V .M

    ISBN 978-0-679-64527-6. Devisetty Sai Tharun, Panguluri Sai Srija, Peeta Vamsi Krishna, Shaiju Panchikkil, and V .M. Manikandan. Defor- estation Detection from Remote Sensing Images using Machine Learning. In 2024 15th International Confer- ence on Computing Communication and Networking Technologies (ICCCNT) , pages 1–7, Kamand, India, June

    2024

  13. [13]

    Wireless Networks with Asynchronous Users

    IEEE. ISBN 979-8-3503-7024-9. doi: 10.1109/ICCCNT61001.2024.10724972. URL https: //ieeexplore.ieee.org/document/10724972/. Mariami Tkeshelashvili, Ritika Verma, and Steven M Kelly. AI Loss of Control Risk: Indications & Warning.AI Risk Reduction Initiative Report, February 2026. URL https://securityandtechnology.org/virtual-library /report/ai-loss-of-cont...

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.1109/icccnt61001.2024.10724972 2024