{"record_type":"pith_number_record","schema_url":"https://pith.science/schemas/pith-number/v1.json","pith_number":"pith:2025:OYHSK3MNS5ENCRPOOBFH6ZDCRR","short_pith_number":"pith:OYHSK3MN","schema_version":"1.0","canonical_sha256":"760f256d8d9748d145ee704a7f64628c5655c72569cb06d38ee983c35846fddb","source":{"kind":"arxiv","id":"2503.18666","version":3},"attestation_state":"computed","paper":{"title":"AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agents","license":"http://creativecommons.org/licenses/by/4.0/","headline":"AgentSpec lets users write runtime rules that stop LLM agents from unsafe actions in code, robots, and cars.","cross_cats":["cs.CL"],"primary_cat":"cs.AI","authors_text":"Christopher M. Poskitt, Haoyu Wang, Jun Sun","submitted_at":"2025-03-24T13:31:48Z","abstract_excerpt":"Agents built on LLMs are increasingly deployed across diverse domains, automating complex decision-making and task execution. However, their autonomy introduces safety risks, including security vulnerabilities, legal violations, and unintended harmful actions. Existing mitigation methods, such as model-based safeguards and early enforcement strategies, fall short in robustness, interpretability, and adaptability. To address these challenges, we propose AgentSpec, a lightweight domain-specific language for specifying and enforcing runtime constraints on LLM agents. With AgentSpec, users define "},"verification_status":{"content_addressed":true,"pith_receipt":true,"author_attested":false,"weak_author_claims":0,"strong_author_claims":0,"externally_anchored":false,"storage_verified":false,"citation_signatures":0,"replication_records":0,"graph_snapshot":true,"references_resolved":true,"formal_links_present":true},"canonical_record":{"source":{"id":"2503.18666","kind":"arxiv","version":3},"metadata":{"license":"http://creativecommons.org/licenses/by/4.0/","primary_cat":"cs.AI","submitted_at":"2025-03-24T13:31:48Z","cross_cats_sorted":["cs.CL"],"title_canon_sha256":"4b5b3b3fe1a5d5c061c5e8df5be7179fca37d0e7698584aea28f285ed734c5bf","abstract_canon_sha256":"a2e3c0a4df27cd97e92328bed6942fd95bd3568fca2d7c1d1377f4dc0e9f7dc2"},"schema_version":"1.0"},"receipt":{"kind":"pith_receipt","key_id":"pith-v1-2026-05","algorithm":"ed25519","signed_at":"2026-05-17T23:39:21.633516Z","signature_b64":"iujN/yCtZ3xNIPiosgmIBiAwtGTlh5cUTTE7+iEFnj0aV7jo56OsbZO10qvnKfi4mR+CPSL/2MgUUzX3+VAzBg==","signed_message":"canonical_sha256_bytes","builder_version":"pith-number-builder-2026-05-17-v1","receipt_version":"0.3","canonical_sha256":"760f256d8d9748d145ee704a7f64628c5655c72569cb06d38ee983c35846fddb","last_reissued_at":"2026-05-17T23:39:21.632828Z","signature_status":"signed_v1","first_computed_at":"2026-05-17T23:39:21.632828Z","public_key_fingerprint":"8d4b5ee74e4693bcd1df2446408b0d54"},"graph_snapshot":{"paper":{"title":"AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agents","license":"http://creativecommons.org/licenses/by/4.0/","headline":"AgentSpec lets users write runtime rules that stop LLM agents from unsafe actions in code, robots, and cars.","cross_cats":["cs.CL"],"primary_cat":"cs.AI","authors_text":"Christopher M. Poskitt, Haoyu Wang, Jun Sun","submitted_at":"2025-03-24T13:31:48Z","abstract_excerpt":"Agents built on LLMs are increasingly deployed across diverse domains, automating complex decision-making and task execution. However, their autonomy introduces safety risks, including security vulnerabilities, legal violations, and unintended harmful actions. Existing mitigation methods, such as model-based safeguards and early enforcement strategies, fall short in robustness, interpretability, and adaptability. To address these challenges, we propose AgentSpec, a lightweight domain-specific language for specifying and enforcing runtime constraints on LLM agents. With AgentSpec, users define "},"claims":{"count":4,"items":[{"kind":"strongest_claim","text":"Our evaluation shows that AgentSpec successfully prevents unsafe executions in over 90% of code agent cases, eliminates all hazardous actions in embodied agent tasks, and enforces 100% compliance by autonomous vehicles (AVs).","source":"verdict.strongest_claim","status":"machine_extracted","claim_id":"C1","attestation":"unclaimed"},{"kind":"weakest_assumption","text":"That comprehensive safety rules can be predefined to cover all relevant unsafe scenarios while remaining practical to write and that runtime interception of agent actions is feasible and accurate across domains without introducing unacceptable false positives.","source":"verdict.weakest_assumption","status":"machine_extracted","claim_id":"C2","attestation":"unclaimed"},{"kind":"one_line_summary","text":"AgentSpec introduces a customizable DSL for runtime enforcement of safety constraints on LLM agents, achieving over 90% prevention of unsafe code actions, zero hazardous embodied actions, and 100% AV compliance in evaluations.","source":"verdict.one_line_summary","status":"machine_extracted","claim_id":"C3","attestation":"unclaimed"},{"kind":"headline","text":"AgentSpec lets users write runtime rules that stop LLM agents from unsafe actions in code, robots, and cars.","source":"verdict.pith_extraction.headline","status":"machine_extracted","claim_id":"C4","attestation":"unclaimed"}],"snapshot_sha256":"564f5008fd3ee679e543bb896676fc0b0acb1594740adde8e6eb76042eaa719f"},"source":{"id":"2503.18666","kind":"arxiv","version":3},"verdict":{"id":"d7ee02ef-132c-40af-8e19-23b4b90a0753","model_set":{"reader":"grok-4.3"},"created_at":"2026-05-14T21:20:33.486922Z","strongest_claim":"Our evaluation shows that AgentSpec successfully prevents unsafe executions in over 90% of code agent cases, eliminates all hazardous actions in embodied agent tasks, and enforces 100% compliance by autonomous vehicles (AVs).","one_line_summary":"AgentSpec introduces a customizable DSL for runtime enforcement of safety constraints on LLM agents, achieving over 90% prevention of unsafe code actions, zero hazardous embodied actions, and 100% AV compliance in evaluations.","pipeline_version":"pith-pipeline@v0.9.0","weakest_assumption":"That comprehensive safety rules can be predefined to cover all relevant unsafe scenarios while remaining practical to write and that runtime interception of agent actions is feasible and accurate across domains without introducing unacceptable false positives.","pith_extraction_headline":"AgentSpec lets users write runtime rules that stop LLM agents from unsafe actions in code, robots, and cars."},"references":{"count":59,"sample":[{"doi":"","year":2025,"title":"AgentSpec. https://github.com/haoyuwang99/AgentSpec, 2025","work_id":"fa785603-cf6e-43f7-beb4-32c853d10e4c","ref_index":1,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2023,"title":"Runtime verification for trustworthy computing","work_id":"7dddb913-3bd7-4d9c-9ff7-1e6e4d038c0a","ref_index":2,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":null,"title":"Apollo Self-Driving","work_id":"909de431-0bfc-410b-99c4-7496d8d63eec","ref_index":3,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2025,"title":"Accessed: 2025-02-11","work_id":"f11b9b96-ef0b-487b-87b8-f12dbff93e81","ref_index":4,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2008,"title":"Principles of model checking","work_id":"f2d9a6fd-6cf6-4179-8614-511d89cb8ee9","ref_index":5,"cited_arxiv_id":"","is_internal_anchor":false}],"resolved_work":59,"snapshot_sha256":"b23902b547bd3abb1d736e1acd1600982b394d745678c2c73a35151fd545f40b","internal_anchors":2},"formal_canon":{"evidence_count":2,"snapshot_sha256":"d991a4f9e79cef635c52d38f7f260ca68ddb3570c50ad81e13281dc30287d677"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"},"aliases":[{"alias_kind":"arxiv","alias_value":"2503.18666","created_at":"2026-05-17T23:39:21.632946+00:00"},{"alias_kind":"arxiv_version","alias_value":"2503.18666v3","created_at":"2026-05-17T23:39:21.632946+00:00"},{"alias_kind":"doi","alias_value":"10.48550/arxiv.2503.18666","created_at":"2026-05-17T23:39:21.632946+00:00"},{"alias_kind":"pith_short_12","alias_value":"OYHSK3MNS5EN","created_at":"2026-05-18T12:33:37.589309+00:00"},{"alias_kind":"pith_short_16","alias_value":"OYHSK3MNS5ENCRPO","created_at":"2026-05-18T12:33:37.589309+00:00"},{"alias_kind":"pith_short_8","alias_value":"OYHSK3MN","created_at":"2026-05-18T12:33:37.589309+00:00"}],"events":[],"event_summary":{},"paper_claims":[],"inbound_citations":{"count":32,"internal_anchor_count":32,"sample":[{"citing_arxiv_id":"2605.20923","citing_title":"Causal Past Logic for Runtime Verification of Distributed LLM Agent Workflows","ref_index":47,"is_internal_anchor":true},{"citing_arxiv_id":"2605.16282","citing_title":"Taxonomy and Consistency Analysis of Safety Benchmarks for AI Agents","ref_index":51,"is_internal_anchor":true},{"citing_arxiv_id":"2605.16471","citing_title":"From AI-Generated Content to Agentic Action: Security and Safety Threats in Generative AI","ref_index":134,"is_internal_anchor":true},{"citing_arxiv_id":"2605.00424","citing_title":"Skills as Verifiable Artifacts: A Trust Schema and a Biconditional Correctness Criterion for Human-in-the-Loop Agent Runtimes","ref_index":27,"is_internal_anchor":true},{"citing_arxiv_id":"2605.14859","citing_title":"Do Coding Agents Understand Least-Privilege Authorization?","ref_index":39,"is_internal_anchor":true},{"citing_arxiv_id":"2510.23883","citing_title":"Agentic AI Security: Threats, Defenses, Evaluation, and Open Challenges","ref_index":206,"is_internal_anchor":true},{"citing_arxiv_id":"2602.09725","citing_title":"Efficient Remote KV Cache Reuse with GPU-native Video Codec","ref_index":63,"is_internal_anchor":true},{"citing_arxiv_id":"2602.16708","citing_title":"Formal Policy Enforcement for Real-World Agentic Systems","ref_index":64,"is_internal_anchor":true},{"citing_arxiv_id":"2603.17418","citing_title":"PowerDAG: Reliable Agentic AI System for Automating Distribution Grid Analysis","ref_index":42,"is_internal_anchor":true},{"citing_arxiv_id":"2603.29665","citing_title":"Near-Miss: Latent Policy Failure Detection in Agentic Workflows","ref_index":1,"is_internal_anchor":true},{"citing_arxiv_id":"2604.02022","citing_title":"ATBench: A Diverse and Realistic Agent Trajectory Benchmark for Safety Evaluation and Diagnosis","ref_index":31,"is_internal_anchor":true},{"citing_arxiv_id":"2605.13044","citing_title":"No Attack Required: Semantic Fuzzing for Specification Violations in Agent Skills","ref_index":28,"is_internal_anchor":true},{"citing_arxiv_id":"2604.02022","citing_title":"ATBench: A Diverse and Realistic Agent Trajectory Benchmark for Safety Evaluation and Diagnosis","ref_index":31,"is_internal_anchor":true},{"citing_arxiv_id":"2604.03070","citing_title":"Credential Leakage in LLM Agent Skills: A Large-Scale Empirical Study","ref_index":57,"is_internal_anchor":true},{"citing_arxiv_id":"2604.04978","citing_title":"Measuring the Permission Gate: A Stress-Test Evaluation of Claude Code's Auto Mode","ref_index":4,"is_internal_anchor":true},{"citing_arxiv_id":"2605.03378","citing_title":"ARGUS: Defending LLM Agents Against Context-Aware Prompt Injection","ref_index":150,"is_internal_anchor":true},{"citing_arxiv_id":"2605.06393","citing_title":"Constraining Host-Level Abuse in Self-Hosted Computer-Use Agents via TEE-Backed Isolation","ref_index":34,"is_internal_anchor":true},{"citing_arxiv_id":"2604.22136","citing_title":"Sovereign Agentic Loops: Decoupling AI Reasoning from Execution in Real-World Systems","ref_index":13,"is_internal_anchor":true},{"citing_arxiv_id":"2605.05501","citing_title":"SOCpilot: Verifying Policy Compliance for LLM-Assisted Incident Response","ref_index":10,"is_internal_anchor":true},{"citing_arxiv_id":"2605.03034","citing_title":"Stable Agentic Control: Tool-Mediated LLM Architecture for Autonomous Cyber Defense","ref_index":11,"is_internal_anchor":true},{"citing_arxiv_id":"2605.00314","citing_title":"Semia: Auditing Agent Skills via Constraint-Guided Representation Synthesis","ref_index":41,"is_internal_anchor":true},{"citing_arxiv_id":"2605.00081","citing_title":"Alignment Contracts for Agentic Security Systems","ref_index":43,"is_internal_anchor":true},{"citing_arxiv_id":"2604.19657","citing_title":"An AI Agent Execution Environment to Safeguard User Data","ref_index":72,"is_internal_anchor":true},{"citing_arxiv_id":"2604.10134","citing_title":"PlanGuard: Defending Agents against Indirect Prompt Injection via Planning-based Consistency Verification","ref_index":16,"is_internal_anchor":true},{"citing_arxiv_id":"2604.08059","citing_title":"Governed Capability Evolution: Lifecycle-Time Compatibility Checking and Rollback for AI-Component-Based Systems, with Embodied Agents as Case Study","ref_index":36,"is_internal_anchor":true}]},"formal_canon":{"evidence_count":2,"sample":[],"anchors":[]},"links":{"html":"https://pith.science/pith/OYHSK3MNS5ENCRPOOBFH6ZDCRR","json":"https://pith.science/pith/OYHSK3MNS5ENCRPOOBFH6ZDCRR.json","graph_json":"https://pith.science/api/pith-number/OYHSK3MNS5ENCRPOOBFH6ZDCRR/graph.json","events_json":"https://pith.science/api/pith-number/OYHSK3MNS5ENCRPOOBFH6ZDCRR/events.json","paper":"https://pith.science/paper/OYHSK3MN"},"agent_actions":{"view_html":"https://pith.science/pith/OYHSK3MNS5ENCRPOOBFH6ZDCRR","download_json":"https://pith.science/pith/OYHSK3MNS5ENCRPOOBFH6ZDCRR.json","view_paper":"https://pith.science/paper/OYHSK3MN","resolve_alias":"https://pith.science/api/pith-number/resolve?arxiv=2503.18666&json=true","fetch_graph":"https://pith.science/api/pith-number/OYHSK3MNS5ENCRPOOBFH6ZDCRR/graph.json","fetch_events":"https://pith.science/api/pith-number/OYHSK3MNS5ENCRPOOBFH6ZDCRR/events.json","actions":{"anchor_timestamp":"https://pith.science/pith/OYHSK3MNS5ENCRPOOBFH6ZDCRR/action/timestamp_anchor","attest_storage":"https://pith.science/pith/OYHSK3MNS5ENCRPOOBFH6ZDCRR/action/storage_attestation","attest_author":"https://pith.science/pith/OYHSK3MNS5ENCRPOOBFH6ZDCRR/action/author_attestation","sign_citation":"https://pith.science/pith/OYHSK3MNS5ENCRPOOBFH6ZDCRR/action/citation_signature","submit_replication":"https://pith.science/pith/OYHSK3MNS5ENCRPOOBFH6ZDCRR/action/replication_record"}},"created_at":"2026-05-17T23:39:21.632946+00:00","updated_at":"2026-05-17T23:39:21.632946+00:00"}