REVIEW 4 major objections 4 minor 6 cited by
ProbGuard: Proactive Runtime Monitoring for LLM Agent Safety via Probabilistic Prediction
T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read ProbGuard claims a Markov chain learned from an agent's traces lets a runtime monitor estimate the probability of reaching an unsafe state, and that intervening at a risk threshold stops violations reactive rules catch too late.
desk verdict Legitimate new combination of DTMC learning and PCTL reachability for LLM-agent monitoring, but the empirical case is in-sample and the early-warning numbers should not be taken at face value. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is a learned Discrete-Time Markov Chain, a finite directed graph whose edges carry the probability of moving from one symbolic state to another, built from predicate-abstracted execution traces. Everything else is in service of making that object faithful and cheap to query. Validity-aware Laplace smoothing ensures no probability mass is assigned to semantically impossible transitions, which matters because sparse data would otherwise make rare unsafe rows look deterministic or unreachable. PCTL reachability in the form $P_{\le\theta}[\mathrm{F}\,\mathit{unsafe}]$ is the query that converts the model into a decision, and a PAC bound (Equation 2) converts collected trace counts into a formal statement about how close the learned reachability probabilities are to the ground-truth model. A cache over the finite abstract state space turns repeated model checking into a constant-time lookup, which is what keeps per-decision overhead at 5 to 28 ms.
What would settle it
Collect a fresh batch of traces from the same agent under the same task distribution, partition them by abstract state, and compare, for each state, the DTMC's predicted probability of eventually reaching the unsafe state against the empirical frequency with which runs from that state actually end in the unsafe state; systematic miscalibration, particularly predictions that worsen as the number of relevant prior steps grows or after a modest shift in the task or environment distribution, would refute the central claim. A cheaper version in the driving setup is to replay recorded scenarios in which the monitor said the risk was below threshold and count how many still produced violations, testing the claim that thresholds like $\theta=0.3$ yield 100% prediction.
Extended reading notes
Core claim
On its own terms, the paper's discovery is that a deliberately small probabilistic model, not the LLM itself, can serve as the safety oracle: agent executions are projected onto a state space of predicate valuations (is the fork in the microwave, is the microwave on, is vehicle speed below 0.5 km/h, is a priority NPC ahead), the resulting abstract traces are pooled into a DTMC with Laplace smoothing applied only to transitions that pass domain-specific validity checks, and the monitor answers PCTL queries of the form $P_{\le\theta}[\mathrm{F}\,\mathit{unsafe}]$ from the current abstract state. The framework appears as Pro2Guard in the paper's main text and as ProbGuard in the title and abstract. The empirical payload is that this learned reachability probability anticipates violations before they materialize: 100% prediction in all evaluated driving scenarios at $\theta=0.3$ with 0.77 to 38.66 seconds of lead time, though at the looser $\theta=0.7$ detection collapses to 0% for most law scenarios, and in embodied tasks the predict-then-enforce loop brings unsafe outcomes down from 40.63% to 2.60% in halt mode and to 14.07% with reflection while keeping 80.4% of the baseline completion rate. The paper's statistical claim is that the learned model is $(\varepsilon,\delta)$-PAC-correct for reachability properties, with a per-state sample-count bound (Equation 2) that also serves as a stopping rule for trace collection.
Load-bearing premise
The load-bearing premise is that what the agent does next depends only on its current abstracted situation, and that the situations seen in offline traces match what happens at runtime; the paper's Section 3.3 admits this is an idealization, so if an agent's next action is shaped by a long past that the abstraction discards, or by circumstances that shift after deployment, the predicted risk estimate is untrustworthy no matter how many traces were collected.
Editorial extensions
If this is right
- A reactive rule framework can be upgraded to proactive enforcement without discarding its symbolic predicates or its intervention actions: the rules' conditions become the DTMC's state space, and halting, user inspection, or LLM self-reflection fire on predicted risk rather than on imminent violation.
- Safety becomes a tunable dial rather than a fixed rule: at threshold 0.1 with halting, unsafe outcomes in embodied tasks drop from 40.63% to 2.60% at the cost of completing only 10.42% of tasks, while the reflect mode keeps 47.74% completion (80.4% of the baseline rate) and still cuts unsafe outcomes to 14.07%, a 65.37% relative reduction.
- The PAC bound doubles as a sampling budget: a developer can collect traces until every abstract state satisfies Equation 2 for the desired $(\varepsilon,\delta)$, and only then deploy the monitor with a stated guarantee on the learned reachability probability.
- Because the abstract state space is finite, the reachability probability for every state is precomputed and cached, making each monitoring decision a table lookup with 5 to 28 ms overhead, which the paper measures as acceptable for soft real-time planning loops.
- The model produces explainable warnings: in the driving experiments the monitor anticipated 100% of evaluated law violations and collisions at $\theta=0.3$ with 0.77 to 38.66 seconds of lead time, and it can report the estimated collision probability per state (e.g., 56.78% when a priority NPC is ahead at speed); the paper also reports an average 12.05% token saving over the reactive baseline from
Reading between the lines
- Editorial extension: the Markov assumption could be probed directly with an adversarial-history experiment, comparing predicted reachability probabilities for abstractly identical states that were entered through different multi-step prompt or memory histories and correlating any prediction error with history length; the paper does not run this test.
- Editorial extension: the reflect mode's success hints that feeding the numeric risk estimate back into the agent's context, rather than only a flag, could act as a behavioral shaping signal, and one could test whether repeated episodes with that feedback reduce the frequency of high-risk states on their own, independent of halting.
- Editorial extension: because the predicate vocabulary is drawn from known safety rules, harm expressible only through unmodeled predicates is invisible to the monitor by construction; an audit worth running is adversarial fuzzing that measures how many unsafe outcomes occur on abstract states the monitor rates as safe.
- Editorial extension: a hierarchical variant, one chain per task or environment glued by a task-choice action, or a single chain whose transition rows are conditioned on a task-identity feature, could transfer risk estimates across scenarios and shrink the per-task trace budget; the paper notes the MDP direction as future work but does not develop it.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes Pro2Guard (also titled 'ProbGuard' in the header/abstract), a runtime monitoring framework for LLM agents. It abstracts agent executions into Boolean predicate states, learns a Discrete-Time Markov Chain (DTMC) from execution traces with validity-aware Laplace smoothing, and at runtime computes the probability of eventually reaching a user-specified unsafe state via PCTL reachability. If the estimated probability exceeds a threshold, an intervention (stop, user inspection, or LLM self-reflection) is triggered. The authors claim that this predict-then-enforce paradigm outperforms reactive rule-based enforcement, reporting reduced unsafe outcomes in embodied household tasks and '100% prediction' of traffic violations and collisions in autonomous driving scenarios, with warnings up to 38.66 seconds ahead. A PAC-style guarantee is cited as justification for the learned model's reliability. The paper includes a discussion of domain abstraction, runtime overhead, and generalizability.
Significance. If the empirical claims were properly supported, this would be a useful contribution to LLM-agent runtime safety: it combines a standard formal-methods toolchain (DTMC learning + PCTL model checking) with a realistic deployment interface, and the open-source implementation and domain-agnostic predicate interface are concrete strengths. The conceptual pipeline is coherent and the use of a cited PAC bound is appropriate. However, the current evaluation does not separate training from test data, so the headline empirical results are not yet established. The idea itself is not highly novel, as DTMC-based probabilistic verification is well known, but the application to proactive LLM-agent enforcement and the explicit treatment of semantic validity constraints give it practical value. The paper's significance depends on a rigorous evaluation that distinguishes genuine forecasting from in-sample curve fitting, which the present manuscript does not provide.
major comments (4)
- [Section 4.1, Table 2] The AV experiments report '100% prediction' and time-ahead values of 0.77–38.66 seconds, but the evaluation never separates the traces used to learn the DTMC (Section 3.3) from those used to measure prediction. Section 4.1 states only that each configuration is replayed five times; it does not state whether the DTMC is trained on the same runs that are then replayed for evaluation. Without a held-out split or cross-validation, the numbers in Table 2 are compatible with an in-sample plug-in estimate that simply reflects the base rate of violations in the training data. The claim is therefore not presently distinguishable from in-sample curve fitting.
- [Section 3.4, Eq. (2)] The PAC bound is cited but never checked against the actual data. The paper does not report the per-state counts n_p, the ratios n_pq/n_p, the state space size m, or the chosen epsilon and delta, so the reader cannot verify whether Equation (2) holds for the traces used in Section 4. This matters because the bound is the main justification that finite-sample learning yields statistically reliable reachability estimates. Without reporting these quantities, the claim that Pro2Guard provides 'statistical reliability' remains unsupported.
- [Table 2 / Abstract] The abstract states that a threshold yields 'no false alarms', yet no false-alarm rate, specificity, or confusion-matrix statistic is reported anywhere in the AV evaluation. At low thresholds (e.g., theta = 0.3), a monitor that almost always raises a warning would achieve large time-ahead values and 100% recall on a small set of scenarios while having an arbitrarily poor false-alarm rate. The paper needs a proper detection-theoretic evaluation: true positives, false positives, and true negatives over a held-out set of safe and unsafe episodes, ideally with a comparison against a base-rate forecaster that always predicts eventual violation.
- [Section 3.3] The load-bearing modeling assumption—that abstract transitions are IID draws from a stationary distribution—is explicitly conceded as 'idealized' in Section 3.3. This assumption is not validated for the evaluated domains; the paper does not report tests of stationarity, episode-length distributions, or sensitivity to the 100-frame STL lookback window. Since the reachability probability is computed from the fitted transition matrix, any violation of the IID/stationarity assumption directly biases the risk estimates used for intervention. The paper should at least report a stability analysis (e.g., model trained on different trace subsets, varying smoothing alpha) to show the reported numbers are not artifacts of a particular trace collection.
minor comments (4)
- [Title/Abstract] The paper title in the header and abstract reads 'ProbGuard' while the body and running head use 'Pro2Guard'. This inconsistency must be resolved before resubmission.
- [Section 4.3] The text reports an average inference overhead of approximately 430 ms per decision cycle, then reports cached per-decision runtimes of 5–28 ms. It should be clarified whether the 430 ms figure is a cold-cache cost and whether the 5–28 ms figures include the abstraction and I/O overhead reported separately.
- [Table 2] The property names 'Law38_2', 'Law51_5', 'Law53', and 'No Collision' should be defined or referenced to the LawBreaker specification, and the 'sub-1'/'sub-5' notation in the text is not explained.
- [Section 4.1] The phrase 'Predication successful rates' should be 'Prediction success rates', and throughout the paper 'predication' is used in several places where 'prediction' is intended.
Circularity Check
No significant circularity: Pro2Guard's reachability predictions are model-based derivations from a learned DTMC and external PAC theorems; in-sample evaluation is a validity concern, not a circular reduction.
full rationale
The paper's derivation chain is: collect execution traces, abstract them into symbolic states via user/domain predicates, count valid transitions, apply Laplace smoothing to obtain a DTMC, then compute PCTL reachability probabilities P[F unsafe] at runtime. The predicted risk is a function of the fitted transition matrix, but this is ordinary model-based prediction rather than a circular reduction: the unsafe state and threshold are user inputs, and the reachability probability is not defined as the empirical violation frequency or as the fitted parameter itself. The PAC guarantee in Eq. (2) is cited to prior work [10, 43] with overlapping authorship, but it is a published mathematical theorem with stated assumptions that do not include the present empirical results; under the rubric, that counts as independent support rather than load-bearing self-citation. The evaluation in Section 4.1 does not report a held-out train/test split, false-alarm denominators, or verification of Eq. (2) sample sizes, and the limitations in Section 5.2 acknowledge the per-task DTMC decomposition and the lack of time-bounded reasoning. Those are methodological and correctness concerns, not circularity: the paper's central claim does not reduce to its inputs by construction. No self-definitional, fitted-input-called-prediction, renaming, or ansatz-smuggling pattern is exhibited with the required specific reduction.
Assumptions & free parameters
free parameters (4)
- Intervention threshold theta =
0.1, 0.3, 0.5, 0.7
- Laplace smoothing alpha =
not reported
- STL lookback window =
100 time frames
- Predicate set P =
domain-specific predicates from LawBreaker and SafeAgentBench
assumptions (6)
- domain assumption LLM agent transitions over the abstraction are IID and stationary
- domain assumption The Markov property holds for abstract states
- domain assumption The ground-truth system is a DTMC with known support
- domain assumption is_valid_state and is_valid_transition correctly encode semantic feasibility
- standard math The PAC bound from Sun et al. [43] applies to this learned DTMC setting
- domain assumption Runtime distribution matches offline sampling distribution
Cite this review
Pith. "Pith review of ProbGuard: Proactive Runtime Monitoring for LLM Agent Safety via Probabilistic Prediction." pith.science (2026). https://pith.science/paper/I7G5YX24
@misc{pith2026250800500,
author = {Pith},
title = {Pith review of: ProbGuard: Proactive Runtime Monitoring for LLM Agent Safety via Probabilistic Prediction},
year = {2026},
howpublished = {\url{https://pith.science/paper/I7G5YX24}},
note = {Machine review of arXiv:2508.00500}
}
read the original abstract
Large Language Model (LLM) agents increasingly operate across domains such as robotics, virtual assistants, and web automation. However, their stochastic decision-making introduces safety risks that are difficult to anticipate during execution. Existing runtime monitoring frameworks, such as AgentSpec, primarily rely on reactive safety rules that detect violations only when unsafe behavior is imminent or has already occurred, limiting their ability to handle long-horizon dependencies. We present ProbGuard, a proactive runtime monitoring framework for LLM agents that anticipates safety violations through probabilistic risk prediction. ProbGuard abstracts agent executions into symbolic states and learns a Discrete-Time Markov Chain (DTMC) from execution traces to model behavioral dynamics. At runtime, the monitor estimates the probability that execution will remain safe from the current state, and triggers an intervention when this probability falls below a user-defined threshold. To improve robustness, ProbGuard incorporates semantic validity constraints in the abstraction and admits a PAC-style analysis that characterizes the sample complexity required to certify the learned model under standard assumptions. We evaluate ProbGuard in two safety-critical domains: autonomous driving and embodied household agents. Across evaluated scenarios, ProbGuard consistently predicts traffic law violations and collisions in advance, with warnings up to 15.84 seconds at a threshold yielding no false alarms, and up to 38.66 seconds at stricter thresholds. In embodied agent tasks, ProbGuard's re-prompting intervention mode reduces unsafe behavior by 65.37% relative to the unmonitored baseline while retaining 80.4% of the baseline task completion; a stricter halting configuration reduces unsafe behavior by 93.60% at a larger cost in completion.
Figures
Figures from the paper (1 more)
Forward citations
Cited by 6 Pith papers
-
Resume Means Resume: A Machine-Checked Conformance Contract for Checkpoint, Interrupt, and Resume Semantics in Workflow Persistence Layers
The paper formalizes six resume-semantics properties, machine-checks a reference model, measures five agent workflow frameworks against them, and finds no two frameworks share a conformance profile.
-
DreamGuard: Efficient Runtime Guardrail for LLM Agents via Risk-Aware World Model
A lightweight learned model of an agent's trajectory can flag both immediate hazards and slowly accumulating risks before actions execute, improving safety while preserving task utility.
-
$S^3$: Improving Agent Safety through Multi-Stage Defense
S3 composes stage-specific safety skills through a guard agent, achieving near-zero attack success on six risk types in its own benchmark while preserving benign task completion.
-
Real-Time Detection and Repair of LLM Agent Failures
A telemetry-based watchdog with deterministic verification detects most injected failures and repairs 45% of real failures, raising task success from 52% to 73%.
-
Securing Agentic AI: From Per-Action Checks to Trajectory Assurance
A vision paper organizing agentic AI security into eleven research directions, with the thesis that safety requires verifiable trajectory-level behavioral guarantees, not per-action checks.
-
Why Formal Monitors Fail: Attack Distribution Entropy as a Coverage Bound for LTL-Based LLM Agent Safety
The central bound is a definitional restatement, the entropy-to-coverage corollary is false, and the empirical correlation is built on the monitor's own misses.
Reference graph
Works this paper leans on
-
[43]
Probabilistic verification of neural networks against group fairness
Bing Sun, Jun Sun, Ting Dai, and Lijun Zhang. Probabilistic verification of neural networks against group fairness. In Formal Methods–The Next 30 Years , pages 93–110. Springer, 2021
work page 2021
-
[1]
Safe reinforcement learning via shielding
Mohammad Alshiekh, Roderick Bloem, Ruediger Ehlers, Bettina Könighofer, Scott Niekum, and Ufuk Topcu. Safe reinforcement learning via shielding. In Proceedings of the AAAI Conference on Artificial Intelligence , pages 2669–2678, 2018
work page 2018
-
[2]
Concrete problems in ai safety
Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. Concrete problems in ai safety. arXiv preprint arXiv:1606.06565, 2016
arXiv 2016
-
[3]
Agentharm: A benchmark for measuring harmfulness of llm agents, 2024
Maksym Andriushchenko, Alexandra Souly, Mateusz Dziemian, Derek Duenas, Maxwell Lin, Justin Wang, Dan Hendrycks, Andy Zou, Zico Kolter, Matt Fredrik- son, Eric Winsor, Jerome Wynne, Yarin Gal, and Xander Davies. Agentharm: A benchmark for measuring harmfulness of llm agents, 2024
work page 2024
- [4]
-
[5]
Apollo open source platform 9.0, 2023
Apollo Auto. Apollo open source platform 9.0, 2023. Released December 18, 2023; accessed 2025-06-12
work page 2023
-
[6]
Christel Baier and Joost-Pieter Katoen. Principles of Model Checking . MIT press, 2008
work page 2008
-
[7]
Introduction to Runtime Verification, volume 10457 of Lecture Notes in Computer Science , pages 1–33
Ezio Bartocci, Yliès Falcone, Adrian Francalanza, and Giles Reger. Introduction to Runtime Verification, volume 10457 of Lecture Notes in Computer Science , pages 1–33. Springer Nature, 2018
work page 2018
Show all 65 references
-
[8]
Smolka, Scott D
Ezio Bartocci, Radu Grosu, Atul Karmarkar, Scott A. Smolka, Scott D. Stoller, Justin Seyster, and Erez Zadok. Adaptive runtime verification. In Proceedings of the 3rd International Conference on Runtime Verification (RV 2012) , volume 7687 of Lecture Notes in Computer Science ...
2012
-
[9]
Probabilistic verification of fairness properties via concentration
Osbert Bastani, Stratis Ioannidis, Manolis Lam, Shivaram Venkataraman, and Morteza Zadimoghaddam. Probabilistic verification of fairness properties via concentration. In Proceedings of the ACM on Programming Languages (POPL) , 2017
2017
-
[10]
Global pac bounds for learning discrete time markov chains
Hugo Bazille, Blaise Genest, Cyrille Jegourel, and Jun Sun. Global pac bounds for learning discrete time markov chains. In International Conference on Computer Aided Verification (CA V), volume 12225 of LNCS, pages 304–326. Springer, 2020
2020
-
[11]
Lmql: A query language for large language models
Leonhard Beurer-Kellner and Alexander Koller. Lmql: A query language for large language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL), 2023
2023
-
[12]
Bhattacharyya, Derek J
Raunak P. Bhattacharyya, Derek J. Phillips, Changliu Liu, Jayesh K. Gupta, Kather- ine Driggs-Campbell, and Mykel J. Kochenderfer. Simulating emergent properties of human driving behavior using multi-agent reward augmented imitation learn- ing. arXiv preprint arXiv:1903.05766, 2019
1903 arXiv
-
[13]
Boyajian, Jens Clausen, Lea M
Walter L. Boyajian, Jens Clausen, Lea M. Trenkwalder, Vedran Dunjko, and Hans J. Briegel. On the convergence of projective-simulation-based reinforcement learning in markov decision processes. Quantum Machine Intelligence, 2:13, 2020
2020
-
[14]
Pstmonitor: Monitor synthesis from probabilistic session types
Christian Bartolo Burlò, Adrian Francalanza, Alceste Scalas, Catia Trubiani, and Emilio Tuosto. Pstmonitor: Monitor synthesis from probabilistic session types. arXiv, 2022
2022
-
[15]
Agenteval: Evaluating llms as general-purpose agents
Andy Chen et al. Agenteval: Evaluating llms as general-purpose agents. arXiv preprint arXiv:2310.08560, 2023
2023 arXiv
-
[16]
Shieldagent: Shielding agents via verifiable safety policy reasoning, 2025
Zhaorun Chen, Mintong Kang, and Bo Li. Shieldagent: Shielding agents via verifiable safety policy reasoning, 2025
2025
-
[17]
Agentdojo: A dynamic environment to evaluate prompt injection attacks and defenses for llm agents, 2024
Edoardo Debenedetti, Jie Zhang, Mislav Balunović, Luca Beurer-Kellner, Marc Fischer, and Florian Tramèr. Agentdojo: A dynamic environment to evaluate prompt injection attacks and defenses for llm agents, 2024
2024
-
[18]
LLM-empowered embodied agent for memory-augmented task planning in household robotics
Marc Glocker et al. LLM-empowered embodied agent for memory-augmented task planning in household robotics. arXiv preprint arXiv:2504.21716, 2025. Conference’17, July 2017, Washington, DC, USA Haoyu Wang, Christopher M. Poskitt, Jun Sun, and Jiali Wei
2025 arXiv
-
[19]
Real estate listing gaffe exposes widespread use of ai in australian industry – and potential risks
The Guardian. Real estate listing gaffe exposes widespread use of ai in australian industry – and potential risks. 2024
2024
-
[20]
Redcode: Risky code execution and generation benchmark for code agents, 2024
Chengquan Guo, Xun Liu, Chulin Xie, Andy Zhou, Yi Zeng, Zinan Lin, Dawn Song, and Bo Li. Redcode: Risky code execution and generation benchmark for code agents, 2024
2024
-
[21]
In- tegrating expert knowledge into logical programs via llms
Franciszek Górski, Oskar Wysocki, Marco Valentino, and Andre Freitas. In- tegrating expert knowledge into logical programs via llms. arXiv preprint arXiv:2502.12275, February 2025
2025 arXiv
-
[22]
Understanding the planning of llm agents: A survey
Xu Huang, Weiwen Liu, Xiaolong Chen, Xingmei Wang, Hao Wang, Defu Lian, Yasheng Wang, Ruiming Tang, and Enhong Chen. Understanding the planning of llm agents: A survey. arXiv preprint arXiv:2402.02716, 2024
2024 arXiv
-
[23]
Language agents: A benchmark for llms as agents
Yujia Huang et al. Language agents: A benchmark for llms as agents. arXiv preprint arXiv:2308.00352, 2023
2023 arXiv
-
[24]
Sebastian Junges, Hazem Torfah, and Sanjit A. Seshia. Runtime monitors for markov decision processes. In CA V, 2021
2021
-
[25]
Prism 4.0: Verification of probabilistic real-time systems
Marta Kwiatkowska, Gethin Norman, and David Parker. Prism 4.0: Verification of probabilistic real-time systems. International conference on computer aided verification, pages 585–591, 2011
2011
-
[26]
Langchain, 2025
LangChain. Langchain, 2025. Accessed: 2025-01-14
2025
-
[27]
A brief account of runtime verification
Martin Leucker and Christian Schallhart. A brief account of runtime verification. Journal of Logic and Algebraic Programming , 78(5):293–303, 2009
2009
-
[28]
A review of prominent paradigms for LLM-based agents: Tool use (including rag), planning, and feedback learning
Xinzhe Li. A review of prominent paradigms for LLM-based agents: Tool use (including rag), planning, and feedback learning. arXiv preprint arXiv:2406.05804, 2024
2024 arXiv
-
[29]
Holistic evaluation of language models
Percy Liang et al. Holistic evaluation of language models. arXiv preprint arXiv:2211.09110, 2022
2022 arXiv
-
[30]
Truthfulqa: Measuring how models mimic human falsehoods
Stephanie Lin, Jacob Hilton, and Amanda Askell. Truthfulqa: Measuring how models mimic human falsehoods. arXiv preprint arXiv:2109.07958, 2023
2023 arXiv
-
[31]
Evaluating the harmfulness of llm agents via simulation
Jialu Liu et al. Evaluating the harmfulness of llm agents via simulation. arXiv preprint arXiv:2307.15852, 2023
2023 arXiv
-
[32]
Codeact: Tool-augmented code generation agents via action plans
Yujia Lu, Jialu Shen, Ziniu Dong, Xiang Ren, et al. Codeact: Tool-augmented code generation agents via action plans. arXiv preprint arXiv:2401.15772, 2024
2024 arXiv
-
[33]
Manning, Prabhakar Raghavan, and Hinrich Schütze.Introduction to Information Retrieval
Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schütze.Introduction to Information Retrieval. Cambridge University Press, 2008
2008
-
[34]
Training language models to follow instructions with human feedback
Long Ouyang et al. Training language models to follow instructions with human feedback. arXiv preprint arXiv:2203.02155, 2022
2022 arXiv
-
[35]
Unveiling ai agent vulnerabilities part v: Securing llm services
Sean Park. Unveiling ai agent vulnerabilities part v: Securing llm services. Trend Micro. Surveys vulnerabilities in code execution, data exfiltration, database access
-
[36]
When ai thinks it will lose, it sometimes cheats, study finds
Palisade Research. When ai thinks it will lose, it sometimes cheats, study finds. Time, 2025
2025
-
[37]
Ai agents: greater capabilities and enhanced risks
Reuters. Ai agents: greater capabilities and enhanced risks. Reuters. Risks include privacy violations, unintended modifications, and misaligned actions
-
[38]
Beyond accuracy: Behavioral testing of nlp models with checklist
Marco Tulio Ribeiro et al. Beyond accuracy: Behavioral testing of nlp models with checklist. In ACL, 2020
2020
-
[39]
Tool- former: Language models can teach themselves to use tools
Timo Schick, Arun Tejasvi Chaganty Dwivedi-Yu, Hinrich Schütze, et al. Tool- former: Language models can teach themselves to use tools. arXiv preprint arXiv:2302.04761, 2023
2023 arXiv
-
[40]
Reflexion: Language agents with verbal reinforcement learning
Noah Shinn, Erick Chien, and Pieter Abbeel. Reflexion: Language agents with verbal reinforcement learning. arXiv preprint arXiv:2303.11366, 2023
2023 arXiv
-
[41]
Imprompter
Clementine Star. This prompt can make an ai chatbot identify and extract personal details from your chats.Wired. Describes “Imprompter” prompt injection exfiltrating personal data 80% success
-
[42]
Stoller, Ezio Bartocci, Justin Seyster, Radu Grosu, Klaus Havelund, Scott A
Scott D. Stoller, Ezio Bartocci, Justin Seyster, Radu Grosu, Klaus Havelund, Scott A. Smolka, and Erez Zadok. Runtime verification with state estimation. In Runtime Verification (RV), 2011
2011
-
[44]
Poskitt, Jun Sun, Yuqi Chen, and Zijiang Yang
Yang Sun, Christopher M. Poskitt, Jun Sun, Yuqi Chen, and Zijiang Yang. Law- breaker: An approach for specifying traffic laws and fuzzing autonomous vehicles. In Proceedings of the 37th IEEE/ACM International Conference on Automated Soft- ware Engineering (ASE), pages 1–12. ACM, 2022
2022
-
[45]
Poskitt, Kun Wang, and Jun Sun
Yang Sun, Christopher M. Poskitt, Kun Wang, and Jun Sun. Lawbreaker: An approach for specifying traffic laws and fuzzing autonomous vehicles. In Pro- ceedings of the 47th IEEE/ACM International Conference on Software Engineering (ICSE 2025)
2025
-
[46]
Cyber crime is surging
Financial Times. Cyber crime is surging. will ai make it worse? Financial Times. AI-driven ransomware and phishing scaled by agentic systems
-
[47]
Poskitt, and Jun Sun
Haoyu Wang, Christopher M. Poskitt, and Jun Sun. Agentspec: Customizable runtime enforcement for safe and reliable llm agents, 2025. arXiv preprint
2025
-
[48]
Improving proba- bility estimation through active probabilistic model learning
Jingyi Wang, Xiaohong Chen, Jun Sun, and Shengchao Qin. Improving proba- bility estimation through active probabilistic model learning. In International Conference on Formal Engineering Methods (ICFEM) , volume 10551 of Lecture Notes in Computer Science , pages 56–72, Xi’an, C...
2017
-
[49]
Automatically ’verifying’ discrete-time complex systems through learning, abstraction and refinement
Jingyi Wang, Jun Sun, Shengchao Qin, and Cyrille Jegourel. Automatically ’verifying’ discrete-time complex systems through learning, abstraction and refinement. IEEE Transactions on Software Engineering , 47(1):189–203, 2021
2021
-
[50]
Should we learn probabilistic models for model checking? a new approach and an empirical study
Jingyi Wang, Jun Sun, Qixia Yuan, and Jun Pang. Should we learn probabilistic models for model checking? a new approach and an empirical study. In Inter- national Conference on Fundamental Approaches to Software Engineering (FASE) , volume 10202 of Lecture Notes in Computer Sc...
-
[51]
Learning probabilistic models for model checking: An evolutionary approach and an empirical study
Jingyi Wang, Jun Sun, Qixia Yuan, and Jun Pang. Learning probabilistic models for model checking: An evolutionary approach and an empirical study. International Journal on Software Tools for Technology Transfer , 20(4):367–384, 2018
2018
-
[52]
Poskitt, Yang Sun, Jun Sun, Jingyi Wang, Peng Cheng, and Jiming Chen
Kun Wang, Christopher M. Poskitt, Yang Sun, Jun Sun, Jingyi Wang, Peng Cheng, and Jiming Chen. 𝜇drive: User-controlled autonomous driving, 2024
2024
-
[53]
A survey on large language model based autonomous agents
Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhi-Yuan Chen, Jiakai Tang, Wayne Xin Zhao, Zhewei Wei, and Ji-Rong Wen. A survey on large language model based autonomous agents. Frontiers of Computer Science (arXiv:2308.11432), 2023
2023 arXiv
-
[54]
Llm-powered autonomous agents
Lilian Weng. Llm-powered autonomous agents. Lil’Log blog, 2023
2023
-
[55]
Guardagent: Safeguard llm agents by a guard agent via knowledge-enabled reasoning, 2025
Zhen Xiang, Linzhi Zheng, Yanjie Li, Junyuan Hong, Qinbin Li, Han Xie, Jiawei Zhang, Zidi Xiong, Chulin Xie, Carl Yang, Dawn Song, and Bo Li. Guardagent: Safeguard llm agents by a guard agent via knowledge-enabled reasoning, 2025
2025
-
[56]
Danning Xie, Yitong Li, Mijung Kim, Hung Viet Pham, Lin Tan, and Michael W. Godfrey. Docter: Documentation guided fuzzing for testing deep learning api functions. In Proceedings of the 29th ACM Joint European Software Engineering Conference and Symposium on the Foundations of ...
2021
-
[57]
Voyager: An open-ended embodied agent with large language models
Jerry Xu, Kahlil Zhang, Fei Xia, et al. Voyager: An open-ended embodied agent with large language models. In International Conference on Machine Learning (ICML), 2023
2023
-
[58]
Foundation agents as a general-purpose autonomy stack
Fan Yang et al. Foundation agents as a general-purpose autonomy stack. arXiv preprint arXiv:2310.02294, 2023
2023 arXiv
-
[59]
React: Synergizing reasoning and acting in language models, 2023
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models, 2023
2023
-
[60]
Survey on evaluation of llm-based agents
Asaf Yehudai, Lilach Eden, Alan Li, Guy Uziel, Yilun Zhao, Roy Bar-Haim, Arman Cohan, and Michal Shmueli-Scheuer. Survey on evaluation of llm-based agents. arXiv preprint arXiv:2503.16416, 2025
2025 arXiv
-
[61]
Safeagentbench: A benchmark for safe task planning of embodied llm agents, 2025
Sheng Yin, Xianghe Pang, Yuanzhuo Ding, Menglan Chen, Yutong Bi, Yichen Xiong, Wenhao Huang, Zhen Xiang, Jing Shao, and Siheng Chen. Safeagentbench: A benchmark for safe task planning of embodied llm agents, 2025
2025
-
[62]
Rvllm: Llm runtime verification with domain knowledge
Yedi Zhang, Sun Yi Emma, Annabelle Lee Jia En, and Jin Song Dong. Rvllm: Llm runtime verification with domain knowledge. arXiv preprint arXiv:2505.18585, May 2025
2025
-
[63]
Prompt automatic evaluation and jailbreak detection
Yifan Zhang et al. Prompt automatic evaluation and jailbreak detection. arXiv preprint arXiv:2401.12345, 2024
2024 arXiv
-
[64]
Agentdam: Privacy leakage evaluation for autonomous web agents, 2025
Arman Zharmagambetov, Chuan Guo, Ivan Evtimov, Maya Pavlova, Ruslan Salakhutdinov, and Kamalika Chaudhuri. Agentdam: Privacy leakage evaluation for autonomous web agents, 2025
2025
-
[65]
Jailbreak chat: A benchmark for jailbreak detection in language models
Zhen Zheng et al. Jailbreak chat: A benchmark for jailbreak detection in language models. arXiv preprint arXiv:2307.15043, 2023
2023 arXiv
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.