REVIEW 3 major objections 6 minor 36 references
Regulating autonomous and agentic AI
T0 review · 3 major / 6 minor · reviewed 2026-08-01 · deepseek-v4-flash
Pith's one-line read The paper argues that agentic AI breaks the knowledge-and-control link that regulation assumes, and that regulators must extend into the supply chain, set human-equivalent standards for AI decisions, and supervise in real time.
desk verdict A serious, well-integrated proposal for UK/EU AI governance; the main weakness is an asserted rather than measured premise about agentic AI's unpredictability, which should be flagged in review. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is the 'accountability gap' produced by agentic AI's solution autonomy, combined with the AI supply chain as the structure that distributes risk knowledge. Solution autonomy means the AI selects and adapts the means to a user-set goal, making its conduct unexpected and unpredictable; the supply chain of developer, deployer, user, and recipient means that the party regulation holds responsible lacks the knowledge that other actors hold, and no one holds both knowledge and control at the same time. These two ideas carry the argument: they explain why user-focused, retrospective regulation fails, why internal governance risks becoming performative, and why the paper's reme
What would settle it
A controlled study in a regulated sector, such as a bank running an agentic system, that showed its regulatory-relevant actions were predictable from its instructions and that its post-hoc reasoning explanations matched its actual decision process—demonstrating user-side foreseeability and explainability—would undercut the accountability gap on which the paper's whole reform programme rests.
Extended reading notes
Core claim
The paper's central claim is that autonomous and agentic AI do not require abandoning the foundations of regulation—outcomes, guidelines, governance—but they do invalidate the assumptions those foundations rest on: that regulated firms can foresee the risks of their activities, control the technology involved, and explain failures after the fact. Agentic AI is defined by 'solution autonomy': the user sets a goal, but the system chooses the means, adapts them, recruits sub-agents, and acts directly on outside systems, so its actions are unpredictable and its LLM-based reasoning cannot be reliably explained. Because the relevant knowledge is fragmented across developers, deployers, and users,
Load-bearing premise
The load-bearing premise is that agentic AI systems are, and will remain, sufficiently unpredictable and unexplainable that the humans who use them cannot foresee or control the risks they create.
Editorial extensions
If this is right
- Regulators should extend their reach to AI supply-chain actors whose choices create or amplify risk, while keeping the regulated user as the primary responsible party.
- Absent reliable metrics, AI decisions should be judged against the standard of a careful and competent human, which translates agentic AI responsibility into familiar legal terms.
- Retrospective oversight should give way to real-time system-level monitoring with escalation and suspension powers, in practice requiring automated, AI-based supervisory tools.
- Regulatees will need to share far more information across the supply chain, and regulators may need to let firms devise their own risk mitigations rather than prescribing uniform sector-wide checklists.
- Platform regimes that assume content originates from intentional user actions will misfire when agentic AI generates content autonomously.
Reading between the lines
- If supervisory AI becomes the mechanism of oversight, the same inexplicability problem may migrate upward: regulators will need explainability and audit trails for their own AI supervisors, a recursive problem the paper leaves implicit.
- The human-equivalence standard embeds a moving target: once AI demonstrably beats human performance on measured tasks, regulators could rationally raise the bar above human level, redistributing risk in ways the paper does not explore.
- The supply-chain extension, applied beyond finance to content and data protection, would push toward global coordination; without it, overlapping national regimes would impose contradictory duties on the same AI providers.
- A testable extension: in a regulatory sandbox, compare a cohort supervised retrospectively with one under real-time AI monitoring; the paper's account predicts earlier detection of systemic patterns in the real-time cohort.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper argues that agentic AI's 'solution autonomy' breaks the knowledge-and-control assumptions underlying current regulatory regimes, producing an accountability gap that cannot be closed by user-focused, retrospective, human-centred oversight. It examines four regimes—UK online content regulation (OSA), data protection (GDPR/UK GDPR), UK financial services (FCA, SM&CR, Mills Review, CTP regime), and the EU AI Act—and proposes that regulation must extend to the AI supply chain, establish decision-making standards for AI (possibly 'as good as a human'), and shift oversight to real-time, system-level, AI-driven supervision. The paper is explicitly forward-looking and is honest about some open questions, but its central conclusions rest on an empirical premise about agentic AI's unpredictability and inexplicability that is asserted rather than demonstrated.
Significance. If the analysis holds, the paper makes a significant contribution to the ongoing UK/EU regulatory debate: it synthesises four regulatory domains, draws on recent official reviews (Mills Review, CTP regime, Ofcom investigation), and identifies a concrete set of reform directions. Its strengths include the comparative structure, the careful parsing of the AI Act's value-chain roles, and the transparency about unresolved issues (§4.2 explicitly says the decision-standard question is 'posed rather than answered'; §4.3 concedes the regulator-expertise gap). The paper contains no fitted parameters and is not a derivation, so the usual circularity failure modes are absent. However, the significance is conditional: the main reform proposals are load-bearing on an empirical claim about agentic AI's unpredictability and non-explainability, for which the evidence is largely conceptual or anecdotal. The paper would be more persuasive if it either supplied systematic evidence or explicitly framed its conclusions as conditional on that premise.
major comments (3)
- [§1.1, fns 11, 14; §5] The central conclusions in §5—extension of regulation to the AI supply chain, establishment of a human-equivalence decision standard, and replacement of retrospective with real-time AI-driven oversight—rest on the assertion in §1.1 that agentic AI's solution autonomy makes its actions 'unexpected and unpredictable' and its reasoning 'not reliably explainable'. The support cited (fns 11, 12, 14) is a systems-theory preprint, a survey, and a single Times anecdote about a rogue OpenAI agent; no systematic measurement of current or near-future agentic deployments is provided. Without this premise, the accountability gap that motivates the entire regulatory overhaul shrinks substantially. Please either supply empirical evidence for the premise or explicitly frame the paper's recommendations as conditional on it.
- [§4.3] The recommendation that regulators adopt an AI-enabled agentic supervisory model, following the Mills Review (cited at n 114), is made without addressing the feasibility of that model in light of the paper's own arguments. If agentic AI systems are as unpredictable and non-explainable as §1.1 claims, then a supervisory AI would face the same opacity and drift problems that the paper says make human oversight ineffective. This is a potential circularity: using AI to supervise AI presupposes that the very problem identified earlier in the paper has been solved. The section should either provide evidence that supervisory AI can achieve the required visibility and explainability, or soften the claim to a research direction.
- [§4.2 vs §5] Section 4.2 states that the question of an AI decision-making standard is 'posed rather than answered', and it identifies serious difficulties in applying open-textured standards such as reasonable care to AI decisions. Yet §5 asserts that 'Decision-making standards for AI will need to be established' and suggests 'as good as a human' as a workable standard. The paper does not resolve the open-textured-standard problem it itself raises, so the conclusion goes beyond what the analysis supports. Please either downgrade the conclusion to a conditional proposal or develop a concrete mechanism for implementing the human-equivalence test in regulated practice.
minor comments (6)
- [Abstract] Typo: 'This paper investigate' should be 'This paper investigates'.
- [§3.2] Typo: 'and process and standards-based' should be 'process and standards-based'.
- [§2.5] Typo: 'already recognise' should be 'already recognises' or 'already recognized'.
- [n 23] Name typo: 'Motagnani' should be 'Montagnani'.
- [§4.1.2] The sentence 'The aim of the CTP regime is to prevent systemic disruption, and the small number of major AI foundation model suppliers suggests that all of them will become candidates for designation once the use of their AI models in the UK financial sector becomes entrenched' is a run-on; consider splitting for clarity.
- [§1.2] Minor wording: 'who is the person or organisation that experiences the AI's action' might be better as 'which is the person or organisation...' since it refers to 'the recipient'.
Circularity Check
No significant circularity: the analysis is an independent legal/policy argument; the first author's self-citations are minor and not load-bearing.
full rationale
The paper is a doctrinal and policy analysis rather than a formal derivation, so the main circularity failure modes—fitted parameters renamed as predictions, self-definitional equations, imported uniqueness theorems, or ansatz smuggled via citation—do not apply. There are no equations, no fitted values, and no statistical predictions. The central recommendations (extending regulation to the AI supply chain, adopting a human-equivalence decision standard, and moving to real-time AI-assisted oversight) are supported by an analytical argument from the structure of the AI supply chain and by external sources such as the FCA's Mills Review, Hacker et al., Cobbe et al., and the EU AI Act. The paper's reliance on the empirical premise that agentic AI is unpredictable and not reliably explainable is a correctness risk, not a circularity: the premise is asserted with conceptual support and a newspaper anecdote, but circularity would require the conclusion to be equivalent to the inputs by construction, which is not the case here. The self-citations to the first author's own work (n 25, n 27, n 75, n 108, n 116) support background propositions about liability, embedded business models, and fairness; they are ancillary rather than load-bearing, and the principal conclusions would stand without them. Accordingly, the paper is substantially self-contained against external benchmarks and receives a low circularity score.
Assumptions & free parameters
assumptions (5)
- domain assumption Regulatory responsibility is premised on regulatees having both knowledge of risks and control over the activity.
- domain assumption Agentic AI's 'solution autonomy' makes its actions unpredictable and its reasoning not reliably explainable.
- domain assumption The four regimes examined (DSA/OSA, GDPR, UK financial services, EU AI Act) sufficiently represent the range of regulatory models.
- domain assumption A standard of 'at least as well as a human' is socially acceptable and workable for AI decision-making.
- domain assumption Regulators can obtain or build the expertise and tools (including agentic supervisory AI) needed for real-time oversight.
invented entities (1)
-
AI-enabled agentic supervisory model
Cite this review
Pith. "Pith review of Regulating autonomous and agentic AI." pith.science (2026). https://pith.science/paper/H453M3ID
@misc{pith2026260721345,
author = {Pith},
title = {Pith review of: Regulating autonomous and agentic AI},
year = {2026},
howpublished = {\url{https://pith.science/paper/H453M3ID}},
note = {Machine review of arXiv:2607.21345}
}
read the original abstract
Regulating activities where regulatees use autonomous and agentic AI is challenging. Regulatory assumptions about regulatee knowledge and control no longer hold true; much of that lies elsewhere in the AI supply chain which thus needs to be brought within the scope of regulation. Governance systems for autonomous AI cannot replicate existing governance models, but need a fresh approach. Retrospective supervisory oversight becomes ineffective as a risk management tool, and AI autonomy generates new systemic risks which require new solutions. This paper investigate four regulatory systems: UK regulation of content platforms, data protection, UK financial services, and the EU AI Act\'92s cross-sectoral regime. It analyses the challenges posed by autonomous and agentic AI and proposes potential solutions which regulators might adopt. These will transform regulation from a reactive process to an active one, and assist it in adapting to the challenges of AI autonomy.
Reference graph
Works this paper leans on
-
[1]
puts. In practice there is a clear risk that those humans will at some point cease to exercise independent judgment and just rubber-stamp the AI’s suggested actions,2 but it remains the responsibility of regulatees to counter this danger via staff training and organisational processes. Full autonomy is where the AI can make decisions and also put them int...
2024
-
[2]
40 Regulation (EU) 2022/2554 of the European Parliament and of the Council on digital operational resilience for the financial sector
In particular, the problems of knowledge and control make it difficult to specify how regulatees should act when implementing and operating agentic AI systems, and they also make it harder to justify imposing responsibility and sanctions where the harm was caused by an AI’s autonomous acts. 40 Regulation (EU) 2022/2554 of the European Parliament and of th...
2022
-
[4]
8 Luca Nannini et al, ‘AI Agents Under EU Law: A Compliance Architecture for AI Providers’ (2026) arXiv preprint arXiv:2604.04604v1,
arXiv 2026
-
[5]
13 Hayley Clatterbuck, Clinton Castro and Arvo Muñoz Morán, ‘Risk alignment in agentic AI systems’ (2024) arXiv preprint arXiv:2410.01927, 6-21. 14 An LLM-based AI can be asked to explain its reasoning and will produce a plausible explanation, a ‘chain-of- thought’. However, that explanation has no necessary connection with the truth. Fazl Barez et al, ‘C...
arXiv 2024
-
[6]
10 Alan Chan, Rebecca Salganik, Alva Markelius Chris Pang, Nitarshan Rajkumar, Dmitrii Krasheninnikov, Lauro Langosco et al, ‘Harms from increasingly agentic algorithmic systems’ (2023) Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency 651,
2023
-
[8]
3 • The AI user sets its goal, but the AI itself determines the means to be used to achieve that goal rather than following a process set by its user.5 The AI also has power to adapt those means as the information it possesses is updated.6 • The AI can recruit other AIs as assistants and use information tools (tool-invocation capability), in effect assemb...
arXiv 2025
-
[11]
The aims of that governance are to identify risks and implement controls to prevent or minimise their occurrence
2.3 Risk management through governance As we will see in section 3.4, the primary compliance mechanism required by regulation is the implementation of appropriate governance by the regulated entity. The aims of that governance are to identify risks and implement controls to prevent or minimise their occurrence. Thus the issues of lack of foreseeability ab...
2025
-
[16]
37 Nannini et al, n 8, 13–14. 38 House of Commons Treasury Committee, Artificial Intelligence in Financial Services (Fifteenth Report of Session 2024–26, HC 684, 2026), https://committees.parliament.uk/publications/51128/documents/283671/default/,
2024
Show all 36 references
-
[17]
The financial system has devised macroprudential instruments to manage these risks, but they focus on human failure, not failures arising from emergent inter-agent interactions
This will bring designated providers partially within the UK’s financial sector regulation, primarily by imposing similar governance obligations to regulated firms.42 Similar issues arise for any data controller which uses third parties or agentic AI to process personal data, ...
2026
-
[20]
41 See section 4.1.2. 42 The regulatory obligations are set out in Bank of England Supervisory Statement SS6/24, Operational resilience: Critical third parties to the UK financial sector, https://www.bankofengland.co.uk/- /media/boe/files/prudential-regulation/supervisory-stat...
2024
-
[21]
45 See eg UK Financial Services and Markets Act 2000 s 19, which prohibits any person from carrying out the specified activities (s
6 (recommending AI-specific stress testing by the end of 2026). 45 See eg UK Financial Services and Markets Act 2000 s 19, which prohibits any person from carrying out the specified activities (s
2026
-
[22]
46 The AI Act does not state this explicitly, but its origins are in the EU’s product safety regime and its main focus is on risk prevention
unless they are authorised or exempt. 46 The AI Act does not state this explicitly, but its origins are in the EU’s product safety regime and its main focus is on risk prevention. 10 3.1 Outcome-based v care and skill obligations Responsibility, leading to liability, is greatl...
2023
-
[24]
60 UK ICO, Regulatory Action Policy, https://ico.org.uk/media2/about-the-ico/documents/2259467/regulatory- action-policy.pdf
59 UK FCA , ‘The Responsibilities of Providers and Distributors for the Fair Treatment of Customers (RPPD)’, para 1.3 https://api-handbook.fca.org.uk/files/document/rppd/RPPD_Full_20190329.pdf. 60 UK ICO, Regulatory Action Policy, https://ico.org.uk/media2/about-the-ico/docume...
-
[25]
90 DSA, arts 34–35; OSA, ss 9–12
89Ann Cavoukian, Privacy by Design: The 7 Foundational Principles (Information and Privacy Commissioner of Ontario 2009). 90 DSA, arts 34–35; OSA, ss 9–12. 91 FCA, Risk assessment processes and controls in firms: our findings (2025) https://www.fca.org.uk/publications/good-and...
2009
-
[27]
https://www.ofcom.org.uk/online-safety/illegal-and-harmful-content/investigation-into-x-and-scope-of-the- online-safety-act. 83 Uta Kohl, ‘Toxic Recommender Algorithms: Immunities, Liabilities and the Regulated Self-Regulation of the Digital Services Act and the Online Safety ...
2024
-
[28]
85 UK ICO, Guidance on AI and data protection (2023) https://ico.org.uk/for-organisations/uk-gdpr-guidance- and-resources/artificial-intelligence/guidance-on-ai-and-data-protection/; Explaining decisions made with AI, https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-r...
2023
-
[30]
https://ico.org.uk/about-the-ico/media- centre/news-and-blogs/2024/05/ico-warns-organisations-must-not-ignore-data-protection-risks-as-it- concludes-snap-my-ai-chatbot-investigation/. 93 See eg Manish Shukla, ‘Adaptive monitoring and real-world evaluation of agentic AI systems...
2024
-
[31]
97Hacker, Engel and Mauer, n 1, 1115–16
96 Directive 2014/65/EU, arts 24-25; Directive 2008/48/EC, art 5; Directive 2005/29/EC, arts 5-9; Council Directive 2000/78/EC, arts 2-4. 97Hacker, Engel and Mauer, n 1, 1115–16. 98Rodríguez de Las Heras Ballell, n 29, 7–8,
2014
-
[32]
18 user might still be able to foresee general categories of risk, but particular outputs and effects on third parties may be difficult to predict in advance or to guard against.99 This analysis suggests that regulation which focuses solely on users is unlikely to be effective...
2025 arXiv
-
[34]
outsourcing
100 This would not oblige suppliers to provide that information, but would make it harder for them to resist doing so in contractual negotiations. 19 4.1.1 Through contracts Where the purchaser of a service is exposed to possible regulatory liability in respect of the service,...
2025
-
[72]
72 AI Act, Recital 104 (human oversight must be genuine and proportionate to the risks of the specific high-risk AI system)
71 AI Act, art 26(1)-(6). 72 AI Act, Recital 104 (human oversight must be genuine and proportionate to the risks of the specific high-risk AI system). 13 regulator can verify whether testing occurred, whether documentation was completed, whether incidents were reported, withou...
2024
-
[106]
22 AD Selbst, ‘Negligence and AI’s Human Users’ (2020) 100(4) Boston University Law Review
2020
-
[158]
20 suppliers suggests that all of them will become candidates for designation once the use of their AI models in the UK financial sector becomes entrenched. Designation as a CTP would subject an AI service provider to obligations to ensure its operational risk management and r...
2024
-
[289]
48 DSA, arts 16–17, 27, 34–35; OSA, ss 9–12. 49 Garante per la protezione dei dati personali, Order of 30 March 2023, web doc no 9870847 https://www.garanteprivacy.it/home/docweb/-/docweb-display/docweb/9870847; EDPB, 'Report of the Work Undertaken by the ChatGPT Taskforce' (2...
2023
-
[640]
35European Parliament Research Service, Andrea Bertolini and others, 'Artificial Intelligence and Civil Liability: A European Perspective' (Study for JURI Committee, January
34Information Commissioner's Office, 'What is the impact of Article 22 of the UK GDPR on fairness?' (2025) https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/artificial-intelligence/guidance-on-ai- and-data-protection/how-do-we-ensure-fairness-in-ai/what-is-th...
2025
-
[1116]
meaningful human control
5 foresight and devise controls, so responsibility and accountability devolves to those humans. This is explicit in the UK’s financial regulation20 and implicit in the other regulatory systems examined. Regulation also assumes that those humans both know the risks created by t...
2025
-
[1194]
32Anirban Mukherjee and Hannah Hanwen Chang, 'Operational Agency: A Permeable Legal Fiction for Tracing Culpability in AI Systems' (forthcoming
31 Jennifer Cobbe and Jatinder Singh, 'Artificial Intelligence as a Service: Legal Responsibilities, Liabilities, and Policy Challenges' (2021) 42 Computer Law & Security Review 105573, 7; Dimitra Kamarinou, Christopher Millard and Jatinder Singh, ‘Machine learning with person...
2021
-
[1362]
23 Maria Lillà Montagnani, Marie-Claire Najjar and Antonio Davola, ‘The EU Regulatory Approach(es) to AI Liability, and its Application to the Financial Services Market’ (2024) Computer Law & Security Review 14–15. 24 UK FCA, Review into the long-term impact of AI on retail fi...
2024
-
[1837]
However, readers should note the possibility of socially acceptable cost-benefit trade-offs if the AI is delivering scarce resources
111 We think this true for the regulatory systems examined here. However, readers should note the possibility of socially acceptable cost-benefit trade-offs if the AI is delivering scarce resources. Thus society might accept a medical diagnosis AI which performs a little less ...
2024 arXiv
-
[1994]
128-36. 21 Where human and AI performance can be measured objectively, as for example in medical scan analysis,110 regulators could use those metrics to establish minimum performance standards. To achieve social acceptance of AI the minimum standard is likely to require the AI...
2017
-
[2016]
For further discussion, see Christopher Kuner and others, 'Expanding the Artificial Intelligence-Data Protection Debate' (2018) 8 International Data Privacy Law
OJ L119/1 (GDPR), art 5(1). For further discussion, see Christopher Kuner and others, 'Expanding the Artificial Intelligence-Data Protection Debate' (2018) 8 International Data Privacy Law
2018
-
[2022]
77 For a comparison of the two, see Benjamin Farrand, ‘How do we understand online harms? The impact of conceptual divides on regulatory divergence between the Online Safety Act and Digital Services Act’ (2024) 16 Journal of Media Law
2024
-
[2023]
1112, 1115-16. 2 Alisa Küper and Nicole Krämer, ‘Psychological traits and appropriate reliance: Factors shaping trust in AI’ (2025) 41 International Journal of Human–Computer Interaction 4115; Artur Klingbeil, Cassandra Grützner and Philipp Schreck, ‘Trust and reliance on AI—A...
2025
-
[2024]
Unhelpfully, the AI Act defines the developer of an AI systems as its ‘provider’ (art 3(3)) and its ‘deployer’ (art 3(4)) is what we term the ‘user’ in this paper
OJ L (the AI Act). Unhelpfully, the AI Act defines the developer of an AI systems as its ‘provider’ (art 3(3)) and its ‘deployer’ (art 3(4)) is what we term the ‘user’ in this paper. We think it important to identify the entities who are intermediate between developer and user...
2023
-
[2025]
8 which is within the range of responses of a human actor, but is not certain to do so,36 and its precise response is unpredictable and irreproducible
PE 776.426. 8 which is within the range of responses of a human actor, but is not certain to do so,36 and its precise response is unpredictable and irreproducible. If the AI has autonomous capabilities to implement its decisions, this means that its actions are hard to predict...
2024
-
[2026]
7 necessarily regulatory) responsibility across the AI supply chain (see sections 1.2 and 2.2) generates uncertainty about where governance should be undertaken, and by whom
SMU Science and Technology Law Review, 50 https://ssrn.com/abstract=5680063; Fraser and Suzor, n 21, 135–36. 7 necessarily regulatory) responsibility across the AI supply chain (see sections 1.2 and 2.2) generates uncertainty about where governance should be undertaken, and by...
2025
Reviewed August 1, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.