A survey of LLM agent self-security threats and mitigations alongside their applications in the cybersecurity lifecycle, introducing a synergy concept and empowerment framework.
hub
ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilities
11 Pith papers cite this work, alongside 6 external citations. Polarity classification is still indexing.
11
Pith papers citing it
6
external citations · OpenAlex
hub tools
years
2026 11representative citing papers
VAMPS benchmark shows multimodal LLMs solve graph-assisted math problems better by direct analysis than by tool-enabled visual solving.
EngiAI introduces a LangGraph-based multi-agent framework and a three-part benchmark suite for LLM-driven engineering design, reporting high task completion rates for proprietary models on Beams2D and Photonics2D problems.
Counterfactual no-search vs forced-search outcomes yield a model-specific oracle that trains search-routing policies, raising macro-F1 from ~0.71 to ~0.82–0.84 on oracle-eligible examples.
LedgerAgent is an inference-time method that uses a structured ledger to track task states and enforce domain policies in tool-calling agents, improving average pass^k over standard prompt-based approaches across four domains.
Evoflux applies evolutionary search at inference time to repair executable tool workflows for compact agents, outperforming SFT and SFT+DPO on held-out MCP-Bench tasks with live servers and 250 tools.
GAIS synthesizes diverse, high-fidelity agentic tasks from real-world MCP servers and adversarial planning, outperforming LLM-only baselines on BFCL, τ²-Bench, and ACEBench with greater data efficiency.
MirrorBench defines a reproducible benchmark combining lexical metrics (MATTR, Yule's K, HD-D) and LLM-judge metrics with calibration controls to measure human-likeness of user-proxy agents across four datasets.
The paper presents a threat model, taxonomy, and six-dimension measurement framework for AI sandboxes to clarify valid testing claims for safety, security, and regulatory assurance.
Q-Evolve unifies automatic process-reward labeling via advantage estimation and behavior-proximal policy optimization inside an in-distribution RL loop to enable self-evolving LLM agents on interactive tasks.
Across 15 LLMs, first-answer accuracy on arithmetic procedural execution falls from 63% at 5 steps to 20% at 95 steps, with under-execution increasing.
citing papers explorer
-
LLM agents security duality: a comprehensive survey of self-security and empowered cybersecurity
A survey of LLM agent self-security threats and mitigations alongside their applications in the cybersecurity lifecycle, introducing a synergy concept and empowerment framework.
-
VAMPS: Visual-Assisted Mathematical Problem Solving Benchmark
VAMPS benchmark shows multimodal LLMs solve graph-assisted math problems better by direct analysis than by tool-enabled visual solving.
-
EngiAI: A Multi-Agent Framework and Benchmark Suite for LLM-Driven Engineering Design
EngiAI introduces a LangGraph-based multi-agent framework and a three-part benchmark suite for LLM-driven engineering design, reporting high task completion rates for proprietary models on Beams2D and Photonics2D problems.
-
When Should LLMs Search? Counterfactual Supervision for Search Routing
Counterfactual no-search vs forced-search outcomes yield a model-specific oracle that trains search-routing policies, raising macro-F1 from ~0.71 to ~0.82–0.84 on oracle-eligible examples.
-
LedgerAgent: Structured State for Policy-Adherent Tool-Calling Agents
LedgerAgent is an inference-time method that uses a structured ledger to track task states and enforce domain policies in tool-calling agents, improving average pass^k over standard prompt-based approaches across four domains.
-
Evoflux: Inference-Time Evolution of Executable Tool Workflows for Compact Agents
Evoflux applies evolutionary search at inference time to repair executable tool workflows for compact agents, outperforming SFT and SFT+DPO on held-out MCP-Bench tasks with live servers and 250 tools.
-
Scaling Agentic Capabilities via Grounded Interaction Synthesis
GAIS synthesizes diverse, high-fidelity agentic tasks from real-world MCP servers and adversarial planning, outperforming LLM-only baselines on BFCL, τ²-Bench, and ACEBench with greater data efficiency.
-
MirrorBench: A Benchmark to Evaluate Conversational User-Proxy Agents for Human-Likeness
MirrorBench defines a reproducible benchmark combining lexical metrics (MATTR, Yule's K, HD-D) and LLM-judge metrics with calibration controls to measure human-likeness of user-proxy agents across four datasets.
-
AI Sandboxes: A Threat Model, Taxonomy, and Measurement Framework
The paper presents a threat model, taxonomy, and six-dimension measurement framework for AI sandboxes to clarify valid testing claims for safety, security, and regulatory assurance.
-
Self-evolving LLM agents with in-distribution Optimization
Q-Evolve unifies automatic process-reward labeling via advantage estimation and behavior-proximal policy optimization inside an in-distribution RL loop to enable self-evolving LLM agents on interactive tasks.
-
When LLMs Stop Following Steps: A Diagnostic Study of Procedural Execution in Language Models
Across 15 LLMs, first-answer accuracy on arithmetic procedural execution falls from 63% at 5 steps to 20% at 95 steps, with under-execution increasing.