REVIEW 4 major objections 4 minor 39 references
Agentic Cloud Decoys: A Deception-Driven Framework for Autonomous Intrusion Investigation
T0 review · 4 major / 4 minor · reviewed 2026-07-31 · deepseek-v4-flash
Pith's one-line read Pairing a planted cloud decoy with an autonomous language-model agent can compress intrusion reconstruction from hours or days to minutes, while keeping every report claim traceable to observed evidence.
desk verdict Worth reading for the design and the injection analysis; the headline numbers are not yet established because the evaluation is self-scored and baseline-free. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the 'Cloud Decoy AI Agent' framework, whose load-bearing pieces are: a decoy S3 bucket that produces signal only when touched; a session-aggregation operator that groups CloudTrail events by the pivot tuple (source address, access key, principal, bucket)—all provider-derived, so the adversary cannot influence how their activity is partitioned; a small set of query tools that enrich the session with identity context, security context, and cloud-trail history; and two-stage dynamic prompt generation, in which a fixed system prompt sets the reasoning constraints and a per-session user prompt carries exactly the fields the agent observed, omitting missing fields rather than
What would settle it
Have an independent set of assessors apply the paper's rubric to the same ten scenarios, and also run the paper's own raw-log baseline condition—full CloudTrail records passed to the report model under a fixed template prompt, with no session aggregation, no enrichment, and no dynamic assembly—on the same scenarios. If the independent scoring does not reproduce nine complete reconstructions and zero unsupported assertions, or if the static-template baseline matches the reported accuracy, the central claim about the framework's contribution collapses.
Extended reading notes
Core claim
On its own terms, the paper establishes that a decoy-anchored agent can reconstruct a cloud intrusion session end-to-end, produce a report every factual assertion of which traces to a field in the evidence package, and do so within four to five minutes, by making the decoy the gate that bounds the agent's evidence horizon, grouping events by a four-element pivot tuple drawn only from provider-derived fields, and constructing the report prompt from only the evidence the agent fetched. It further claims that per-incident investigation cost is linear in the decoy interaction rate and independent of total account activity, which is the precise sense in which deception changes the economics of ag
Load-bearing premise
The load-bearing premise is that the evaluation is a valid test of the framework: ten scenarios were designed and scored by the authors themselves, with no baseline, no ablation, and no independent assessment, so 'complete reconstruction' and 'zero unsupported assertions' are defined by the authors' own rubric and could reflect the authors' expectations rather than intrinsic framework capability.
Editorial extensions
If this is right
- If the result holds, a cloud tenant can go from a decoy alert to an analyst-ready incident report in minutes rather than hours or days, because the decoy defines a small, high-confidence investigation universe.
- Investigation cost becomes decoupled from account size: per-incident cost scales with decoy-interaction volume, so an agentic responder is economically viable even in large environments.
- A prompt that carries only observed fields can eliminate the placeholder-triggered fabrication failure mode in LLM reporting, at least across the controlled scenarios tested.
- The delayed-access failure shows that sessionization by an idle threshold has an inherent trade-off against slow adversaries; the paper specifies a cross-session correlation algorithm as the required next step.
- The adversary-authored telemetry exposure implies that any production deployment of such a pipeline must treat object keys and user-agent strings as untrusted input, and the paper is explicit that its own prototype does not yet do so.
Reading between the lines
- If the zero-unsupported-assertion result is reproduced under independent scoring and against the paper's own static-template baseline, it would suggest the prompt-construction discipline transfers to other LLM-based forensic and log-analysis pipelines, not only decoy-anchored ones.
- The manuscript's configuration table still contains FIXME placeholders for the controller and report model identifiers, so the evaluation is not exactly reproducible from the paper alone; the authors claim to report model identifiers explicitly but the draft does not fully deliver that.
- The identified prompt-injection channel could be tested directly with a probe suite that places instruction text in object keys and user-agent strings, measuring whether the crafted text appears verbatim in the report (which is acceptable) versus whether the agent acts on it or issues unintended tool calls (which is the failure).
- The decoy-fingerprinting limitation suggests that real deployments will need multiple rotated decoy surfaces; an adversary who successfully identifies and avoids all of them defeats the entire pipeline, a risk the paper names but does not quantify.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents Cloud Decoy AI Agent, a framework pairing an S3 decoy with a language-model agent to autonomously investigate suspicious cloud activity. The authors formalize session reconstruction via a pivot tuple over provider-derived fields, define a grounding invariant for report generation, specify attack graph construction with a closed-world condition, and propose a two-stage dynamic prompt assembly. They identify indirect prompt injection as an unaddressed risk in this class of system and give an analytic cost model. Evaluation on ten self-constructed AWS S3 scenarios reports 9/10 complete reconstructions, zero unsupported assertions, and 4–5 minute latency, while explicitly acknowledging the absence of baselines, ablations, independent scoring, and multi-provider coverage.
Significance. If validated, the framework would be a useful contribution to cloud incident response: the formal definitions (pivot tuple, session, Invariant 1, attack graph closure) are coherent, the dynamic prompt discipline is a reasonable way to bound hallucination, and the cost model gives a clear argument for why decoy-gated agentic investigation can decouple cost from environment size. The paper is also unusually transparent about its limitations, naming the comparisons that would resolve them. However, the central empirical claims—complete reconstruction, zero unsupported assertions, and latency—are not yet established by the evaluation as reported. The assessment is based on author-designed scenarios, author-only scoring, no baseline or ablation, and missing model identifiers. The formal and architectural contributions are real, but the empirical evidence needs substantial strengthening.
major comments (4)
- [§VII-B/C, §VIII-D] The evaluation cannot support the framework-level empirical claims. The ten scenarios were constructed by the authors and scored by the authors, with no independent assessors or inter-rater reliability. The paper concedes this is a material weakness. The reported 9/10 complete reconstruction and 0 unsupported assertions may reflect the authors' implicit reconstruction model rather than an objective property of the system. A valid test requires blinded independent scoring with condition-identifying material removed, and ideally external scenario construction.
- [§XI-A] No baseline or ablation is included, so the results cannot isolate the contribution of the framework's components. In particular, the abstract's headline claim is compatible with a static-template prompt plus a compliant model; the paper names the required comparisons (raw log baseline, static template, non-deception trigger, deterministic renderer) but does not run them. At minimum, a static-template condition and a non-deception trigger condition are needed to show that dynamic prompt generation and the decoy gate are load-bearing. This is a central missing component for a paper whose main claim is empirical.
- [Table III] The model configuration table lists 'FIXME: controller model, vendor and version' and 'FIXME: report model, vendor and version.' The paper states that it reports model identifiers explicitly so the prompt discipline can be judged, but the actual identifiers are absent. This makes the evaluation irreproducible and prevents readers from assessing model dependence. The identifiers must be filled in before the empirical claims can be evaluated.
- [§IV, §VI-D, §VIII] The measurement of zero violations of Invariant 1 is partly circular. The invariant is enforced structurally by constructing the user prompt to contain only fields the agent observed, so a zero violation rate is in part a check that the construction was implemented correctly rather than an independent outcome. Moreover, the 'unsupported assertions' judgment was made by the authors, who also built the evidence package and the report prompts. To make the grounding result meaningful, the paper should implement per-claim citations (as noted in §XI-E) or have independent raters trace each claim to the evidence package.
minor comments (4)
- [References [8], [29], [30]] Several references contain '[VERIFY]' placeholders (page ranges, co-author names). These should be resolved before submission.
- [Abstract] Minor typographical issue: 'A WS S3' appears in the abstract; should be 'AWS S3'.
- [§IX] In the cost model, the notation 'c λd' is unclear; it should be defined or typeset more cleanly to distinguish the per-event cost from the Lambda duration cost.
- [§XI-B, Algorithm 1] In the scoring formula, 'e^{-g/H}' uses 'e' for the exponential; this is fine, but the weights w1...w4 are introduced without a normalization constraint. A brief note on how they are intended to be set would help.
Circularity Check
No load-bearing circularity: the main claims are architectural definitions or explicitly bounded empirical observations; self-citations are peripheral.
full rationale
The paper's derivation chain does not contain a fitted parameter renamed as a prediction or a result assumed by definition. Invariant 1 (Section IV) is stated as a requirement and then addressed by dynamic prompt construction; the zero-unsupported-assertions result (Section VIII) is an empirical check of LLM compliance, not a logical consequence, and the paper explicitly concedes it "does not establish that the prompt assembly discipline is what produced that result" (Section XI-A). Likewise, the 9.5/10 reconstruction score is scored against driver-script ground truth with a published rubric (Appendix A), and the paper openly flags author-only scoring as "a material weakness" (Section VII-C) and the absence of baseline/ablation (Section XI-A). Those are validity limits, not circular steps. The cost model (Section IX) is an analytic identity of the architecture's gating property; it is labeled illustrative rather than measured, and its conclusion (linear in decoy interactions, independent of total account volume) follows from the stated cost accounting, which is normal model-building rather than a circular prediction. Several references are by the co-authors (e.g., [3], [11], [12], [22], [37]), but none is used to justify the framework's load-bearing design claims; those rest on external citations ([7], [10], [17], [21]) and on the system's own definitions. The most serious manuscript-flagged weaknesses—no independent scoring, no baseline/ablation, FIXME model identifiers in Table III, no released artifact (Section XI-A), and the unmitigated prompt-injection surface (Sections III-A, XI-C)—are evidence-quality threats that should be weighed as correctness risk, not as circularity. Under the hard rule requiring a quotable reduction to the paper's own inputs, no such reduction is present.
Assumptions & free parameters
free parameters (1)
- idle threshold tau =
15 minutes
assumptions (5)
- domain assumption CloudTrail records the provider-derived fields (t, iota, kappa, rho, beta, a, epsilon) faithfully and they are not adversary-controllable.
- domain assumption No legitimate workflow has reason to touch the decoy, so every decoy interaction is adversarial by construction.
- domain assumption Grouping events by the pivot tuple with idle threshold tau yields sessions that match what a competent analyst would call one incident.
- domain assumption An LLM, given a prompt containing exactly the observed evidence package and no placeholders, will not fabricate facts outside that evidence.
- domain assumption Cloud providers record client-supplied fields such as object keys and user agents verbatim, making telemetry partially adversary-authored.
Cite this review
Pith. "Pith review of Agentic Cloud Decoys: A Deception-Driven Framework for Autonomous Intrusion Investigation." pith.science (2026). https://pith.science/paper/EBV6WTWX
@misc{pith2026260724006,
author = {Pith},
title = {Pith review of: Agentic Cloud Decoys: A Deception-Driven Framework for Autonomous Intrusion Investigation},
year = {2026},
howpublished = {\url{https://pith.science/paper/EBV6WTWX}},
note = {Machine review of arXiv:2607.24006}
}
read the original abstract
Cloud telemetry arrives at a scale that, paradoxically, makes intrusion understanding harder rather than easier. Attackers operate through legitimate identity, federated session tokens, and cloud native APIs indistinguishable from routine administration, and analysts spend an incident reconstructing context the logs already contain. We present Cloud Decoy AI Agent, a framework pairing a high fidelity cloud decoy with an autonomous language model agent that compresses the path from suspicious activity to an analyst ready report. Connecting a decoy to an agent is not a wiring exercise. The unit of investigation is the session rather than the event, and the session key is obscured by the identity layering federated credentials introduce. The agent's evidence horizon must be bounded, since an agent free to query full control plane history inherits the cost and false positive profile deception was meant to remove. And cloud telemetry is partly adversary authored, since object keys and user agent strings are attacker chosen values providers record verbatim, which makes any log to prompt path an indirect prompt injection channel that a decoy widens rather than narrows. We address the first two with a session aggregation operator over a pivot tuple drawn only from provider derived fields, and with dynamic prompt generation, a two stage prompt assembly enforcing a grounding invariant by carrying only fields the agent observed. We identify the third as an unaddressed exposure in this class of system, specify the mitigation it requires, and note our prototype does not implement it. Across ten controlled AWS S3 scenarios, nine were reconstructed completely, no report contained an assertion untraceable to an observed artifact, and latency was four to five minutes. We also state what this evaluation does not establish and name the comparisons that would settle it.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
Cloud forensics,
K. Ruan, J. Carthy, T. Kechadi, and M. Crosbie, “Cloud forensics,” Advances in Digital Forensics VII, vol. 361, pp. 35–46, 2011
2011
-
[2]
Cloud forensics: A meta-study of challenges, approaches, and open problems,
S. Zawoad and R. Hasan, “Cloud forensics: A meta-study of challenges, approaches, and open problems,”arXiv preprint arXiv:1302.6312, 2013
arXiv 2013
-
[3]
Cloud incident response framework and AI-based forensics using reinforcement learning and graph neural networks,
D. N. Alharthi, “Cloud incident response framework and AI-based forensics using reinforcement learning and graph neural networks,” in 2024 IEEE 15th Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON). IEEE, 2024, pp. 164–170
2024
-
[4]
Honeypots: Catching the insider threat,
L. Spitzner, “Honeypots: Catching the insider threat,” inProceedings of the 19th Annual Computer Security Applications Conference (ACSAC). IEEE, 2003, pp. 170–179
2003
-
[5]
Baiting inside attackers using decoy documents,
B. M. Bowen, S. Hershkop, A. D. Keromytis, and S. J. Stolfo, “Baiting inside attackers using decoy documents,” inProceedings of the 5th In- ternational ICST Conference on Security and Privacy in Communication Networks (SecureComm). Springer, 2009, pp. 51–70
2009
-
[6]
BotSwindler: Tamper resistant injection of believable decoys in VM-based hosts for crimeware detection,
B. M. Bowen, P. Prabhu, V . P. Kemerlis, S. Sidiroglou, A. D. Keromytis, and S. J. Stolfo, “BotSwindler: Tamper resistant injection of believable decoys in VM-based hosts for crimeware detection,” inRecent Advances in Intrusion Detection (RAID), ser. Lecture Notes in Computer Science, vol. 6307. Springer, 2010, pp. 118–137
2010
-
[7]
Using honey- buckets to characterize cloud storage scanning in the wild,
K. Izhikevich, G. V oelker, S. Savage, and L. Izhikevich, “Using honey- buckets to characterize cloud storage scanning in the wild,” 2024
2024
-
[8]
Not what you’ve signed up for: Compromising real-world LLM- integrated applications with indirect prompt injection,
K. Greshake, S. Abdelnabi, S. Mishra, C. Endres, T. Holz, and M. Fritz, “Not what you’ve signed up for: Compromising real-world LLM- integrated applications with indirect prompt injection,” inProceedings of the 16th ACM Workshop on Artificial Intelligence and Security (AISec). ACM, 2023, pp. 79–90, [VERIFY] page range
2023
Show all 39 references
-
[9]
OW ASP top 10 for large language model applications,
OW ASP Foundation, “OW ASP top 10 for large language model applications,” https://owasp.org/ www-project-top-10-for-large-language-model-applications/, 2025, accessed: 2026-07-26
2025
-
[10]
A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions,
L. Huang, W. Yu, W. Ma, W. Zhong, Z. Feng, H. Wang, Q. Chen, W. Peng, X. Feng, B. Qin, and T. Liu, “A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions,” arXiv preprint arXiv:2311.05232, 2023
2023 arXiv
-
[11]
LLM-powered automated cloud forensics: From log analysis to investigation,
D. N. Alharthi and R. Yasaei, “LLM-powered automated cloud forensics: From log analysis to investigation,” in2025 IEEE 18th International Conference on Cloud Computing (CLOUD). IEEE, 2025
2025
-
[12]
Cloud investigation automation framework (CIAF): An AI-driven approach to cloud forensics,
D. N. Alharthi and I. R. K. Garcia, “Cloud investigation automation framework (CIAF): An AI-driven approach to cloud forensics,” 2025
2025
-
[13]
Secure cloud migration strategy (SCMS): A safe jour- ney to the cloud,
D. N. Alharthi, “Secure cloud migration strategy (SCMS): A safe jour- ney to the cloud,” inProceedings of the 18th International Conference on Cyber Warfare and Security (ICCWS). Academic Conferences International Limited, 2023, pp. 1–6
2023
-
[14]
The NIST definition of cloud computing,
P. Mell and T. Grance, “The NIST definition of cloud computing,” National Institute of Standards and Technology, Tech. Rep. Special Publication 800-145, 2011
2011
-
[15]
Containerized cloud-based honeypot deception for tracking attackers,
S. Srinivasa, J. M. Pedersen, and E. Vasilomanolakis, “Containerized cloud-based honeypot deception for tracking attackers,”Scientific Re- ports, vol. 13, p. 1437, 2023
2023
-
[16]
A com- prehensive survey on cyber deception techniques to improve honeypot performance,
A. Javadpour, F. Ja’fari, T. Taleb, M. Shojafar, and C. Benza ¨ıd, “A com- prehensive survey on cyber deception techniques to improve honeypot performance,”Computers & Security, vol. 140, p. 103792, 2024
2024
-
[17]
ReAct: Synergizing reasoning and acting in language models,
S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y . Cao, “ReAct: Synergizing reasoning and acting in language models,” in International Conference on Learning Representations (ICLR), 2023
2023
-
[18]
Chain-of-thought prompting elicits reasoning in large language models,
J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. H. Chi, Q. V . Le, and D. Zhou, “Chain-of-thought prompting elicits reasoning in large language models,” inAdvances in Neural Information Processing Systems (NeurIPS), vol. 35, 2022, pp. 24 824–24 837
2022
-
[19]
LangGraph: Building stateful, multi-actor applications with LLMs,
LangChain, “LangGraph: Building stateful, multi-actor applications with LLMs,” https://langchain-ai.github.io/langgraph/, 2024, accessed: 2026- 07-26
2024
-
[20]
Introducing the Model Context Protocol,
Anthropic, “Introducing the Model Context Protocol,” https://www.anthropic.com/news/model-context-protocol, Nov. 2024, accessed: 2026-07-26
2024
-
[21]
Retrieval-augmented generation for knowledge-intensive NLP tasks,
P. Lewis, E. Perez, A. Piktus, F. Petroni, V . Karpukhin, N. Goyal, H. K ¨uttler, M. Lewis, W. tau Yih, T. Rockt ¨aschel, S. Riedel, and D. Kiela, “Retrieval-augmented generation for knowledge-intensive NLP tasks,” inAdvances in Neural Information Processing Systems (NeurIPS),...
2020
-
[22]
ProvenAI: Provenance-native traces of evidence in generated answers,
M. Faizan and D. N. Alharthi, “ProvenAI: Provenance-native traces of evidence in generated answers,” 2026
2026
-
[23]
A comprehensive overview of large lan- guage models (LLMs) for cyber defences: Opportunities and directions,
M. Hassanin and N. Moustafa, “A comprehensive overview of large lan- guage models (LLMs) for cyber defences: Opportunities and directions,” arXiv preprint arXiv:2405.14487, 2024
2024 arXiv
-
[24]
LogPr ´ecis: Unleashing language models for automated malicious log analysis,
M. Boffa, I. Drago, M. Mellia, L. Vassio, D. Giordano, R. V . Valen- tim, and Z. Ben-Houidi, “LogPr ´ecis: Unleashing language models for automated malicious log analysis,”Computers & Security, vol. 141, p. 103805, 2024
2024
-
[25]
LLMCloudHunter: Harnessing LLMs for automated extraction of de- tection rules from cloud-based CTI,
Y . Schwartz, L. Benshimol, D. Mimran, Y . Elovici, and A. Shabtai, “LLMCloudHunter: Harnessing LLMs for automated extraction of de- tection rules from cloud-based CTI,” 2024
2024
-
[26]
Employing LLM agents for tabletop incident response: A mock backdoors and breaches scenario,
S. Hays and J. White, “Employing LLM agents for tabletop incident response: A mock backdoors and breaches scenario,” 2024
2024
-
[27]
In-context autonomous network incident response: An end-to-end large language model agent approach,
T. Li, Z. Zhu, and Q. Zhu, “In-context autonomous network incident response: An end-to-end large language model agent approach,” 2025
2025
-
[28]
MCP safety audit: LLMs with the model context protocol allow major security exploits,
B. Radosevich and J. T. Halloran, “MCP safety audit: LLMs with the model context protocol allow major security exploits,” 2025
2025
-
[29]
A zero-trust reinforcement learning policy for mitigating cyberattacks on emergency vehicle preemption systems,
D. N. Alharthi and M. Abbas, “A zero-trust reinforcement learning policy for mitigating cyberattacks on emergency vehicle preemption systems,” in2024 IEEE 15th Annual Information Technology, Electron- ics and Mobile Communication Conference (IEMCON). IEEE, 2024, [VERIFY] co-au...
2024
-
[30]
Phishing detection based on webpage content: Static and dynamic analysis,
A. O’Mara, I. Alsmadi, A. Aleroud, and D. N. Alharthi, “Phishing detection based on webpage content: Static and dynamic analysis,” in 2023 3rd Intelligent Cybersecurity Conference (ICSC). IEEE, 2023, pp. 39–45, [VERIFY] first-author given name
2023
-
[31]
MITRE ATT&CK: Design and philosophy,
B. E. Strom, A. Applebaum, D. P. Miller, K. C. Nickels, A. G. Penning- ton, and C. B. Thomas, “MITRE ATT&CK: Design and philosophy,” The MITRE Corporation, Tech. Rep. MP180360R1, 2020
2020
-
[32]
MITRE ATT&CK for enterprise: Cloud ma- trix,
The MITRE Corporation, “MITRE ATT&CK for enterprise: Cloud ma- trix,” https://attack.mitre.org/matrices/enterprise/cloud/, 2024, accessed: 2026-07-26
2024
-
[33]
A literature survey and analysis on social engineering defense mechanisms and InfoSec policies,
D. N. Alharthi and A. C. Regan, “A literature survey and analysis on social engineering defense mechanisms and InfoSec policies,”Interna- tional Journal of Network Security & Its Applications, vol. 13, no. 2, pp. 41–61, 2021
2021
-
[34]
Social engineering defense mechanisms: A taxonomy and a survey of employees’ awareness level,
——, “Social engineering defense mechanisms: A taxonomy and a survey of employees’ awareness level,” inIntelligent Computing: Pro- ceedings of the 2020 Computing Conference. Springer, 2020
2020
-
[35]
A taxonomy of social engineering defense mechanisms,
D. N. Alharthi, M. Hammad, and A. C. Regan, “A taxonomy of social engineering defense mechanisms,” inFuture of Information and Communication Conference (FICC). Springer, 2020
2020
-
[36]
Intelligence-driven computer network defense informed by analysis of adversary campaigns and intrusion kill chains,
E. M. Hutchins, M. J. Cloppert, and R. M. Amin, “Intelligence-driven computer network defense informed by analysis of adversary campaigns and intrusion kill chains,”Leading Issues in Information Warfare and Security Research, vol. 1, no. 1, pp. 80–106, 2011
2011
-
[37]
A call to action for a secure-by- design generative AI paradigm,
D. N. Alharthi and I. R. K. Garcia, “A call to action for a secure-by- design generative AI paradigm,” 2025
2025
-
[38]
Canary files: Generating fake files to detect critical data loss from complex computer networks,
B. Whitham, “Canary files: Generating fake files to detect critical data loss from complex computer networks,” inProceedings of the 2nd International Conference on Cyber Security, Cyber Peacefare and Digital Forensic (CyberSec). The Society of Digital Information and Wireless ...
2013
-
[39]
Automated generation and analysis of attack graphs,
O. Sheyner, J. Haines, S. Jha, R. Lippmann, and J. M. Wing, “Automated generation and analysis of attack graphs,” inProceedings of the 2002 IEEE Symposium on Security and Privacy. IEEE, 2002, pp. 254–265. APPENDIXA SCORINGRUBRIC The criteria below were applied to each generate...
2002
Reviewed July 31, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.