REVIEW 10 cited by
AgentAlign: Navigating Safety Alignment in the Shift from Informative to Agentic Large Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
AgentAlign: Navigating Safety Alignment in the Shift from Informative to Agentic Large Language Models
read the original abstract
The acquisition of agentic capabilities has transformed LLMs from "knowledge providers" to "action executors", a trend that while expanding LLMs' capability boundaries, significantly increases their susceptibility to malicious use. Previous work has shown that current LLM-based agents execute numerous malicious tasks even without being attacked, indicating a deficiency in agentic use safety alignment during the post-training phase. To address this gap, we propose AgentAlign, a novel framework that leverages abstract behavior chains as a medium for safety alignment data synthesis. By instantiating these behavior chains in simulated environments with diverse tool instances, our framework enables the generation of highly authentic and executable instructions while capturing complex multi-step dynamics. The framework further ensures model utility by proportionally synthesizing benign instructions through non-malicious interpretations of behavior chains, precisely calibrating the boundary between helpfulness and harmlessness. Evaluation results on AgentHarm demonstrate that fine-tuning three families of open-source models using our method substantially improves their safety (35.8% to 79.5% improvement) while minimally impacting or even positively enhancing their helpfulness, outperforming various prompting methods. The dataset and code have both been open-sourced.
Forward citations
Cited by 10 Pith papers
-
Tool Specifications Matter: Uncovering and Mitigating Safety Risks in AI Agents
Schema-formatted tool specifications degrade refusal in LLM agents; using flattened textual specifications during safety judgment substantially restores refusal and blocks prompt injection.
-
Tool Use Enables Undetectable Steganography in Multi-Agent LLM Systems
Tool-using LLM agents can implement undetectable stegosystems, shifting the primary barrier to covert multi-agent collusion from technical feasibility to coordination without explicit agreement.
-
The Cold-Start Safety Gap in LLM Agents
The paper identifies a cold-start safety gap in LLM agents where safety improves 9-52% after 0-20 regular tasks and introduces the SODA benchmark to measure it.
-
Relevance as a Vulnerability: How Web Retrieval Degrades Safety Alignment in LLM Agents
Web retrieval degrades safety alignment in LLM agents, with relevance activating vulnerabilities including a Safe Source Paradox where oppositional content increases harmful compliance.
-
SafeHarbor: Defining Precise Decision Boundaries via Hierarchical Memory-Augmented Guardrail for LLM Agent Safety
SafeHarbor retrieves LLM-written prohibition/exemption rules from a hierarchical memory to reject harmful agent actions while reducing false refusals of benign queries.
-
Lingering Authority: Revocable Resource-and-Effect Capabilities for Coding Agents
PORTICO is a revocable capability reference monitor for coding agents that enforces task contracts via grant-invoke-closure lifecycles and rejects post-closure reuses while preserving task success.
-
From Risk Classification to Action Plan Remediation: A Guardrail Feedback Driven Framework for LLM Agents
TRIAD finetunes a guardrail model to output proceed/refuse/update decisions with natural-language feedback that is injected back into the agent, cutting average attack success rate to 10.42% on ASB and AgentHarm bench...
-
SafeHarbor: Defining Precise Decision Boundaries via Hierarchical Memory-Augmented Guardrail for LLM Agent Safety
SafeHarbor introduces a hierarchical memory-augmented guardrail with adversarial rule extraction and entropy-driven self-evolution to balance safety and utility in LLM agents.
-
SafeHarbor: Defining Precise Decision Boundaries via Hierarchical Memory-Augmented Guardrail for LLM Agent Safety
SafeHarbor uses hierarchical memory with adversarial rule extraction and entropy-driven self-evolution to achieve over 93% refusal on harmful requests while reaching 63.6% benign utility on GPT-4o.
-
S3LoRA: Safe Spectral Sharpness-Guided Pruning in Adaptation of Agent Planner
S3LoRA prunes LoRA layers with the sharpest spectral update concentration to improve safety in fine-tuned LLM agents without needing base models or extra data.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.