REVIEW 14 cited by
EARBench: Towards Evaluating Physical Risk Awareness for Task Planning of Foundation Model-based Embodied AI Agents
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
EARBench: Towards Evaluating Physical Risk Awareness for Task Planning of Foundation Model-based Embodied AI Agents
read the original abstract
Embodied artificial intelligence (EAI) integrates advanced AI models into physical entities for real-world interaction. The emergence of foundation models as the "brain" of EAI agents for high-level task planning has shown promising results. However, the deployment of these agents in physical environments presents significant safety challenges. For instance, a housekeeping robot lacking sufficient risk awareness might place a metal container in a microwave, potentially causing a fire. To address these critical safety concerns, comprehensive pre-deployment risk assessments are imperative. This study introduces EARBench, a novel framework for automated physical risk assessment in EAI scenarios. EAIRiskBench employs a multi-agent cooperative system that leverages various foundation models to generate safety guidelines, create risk-prone scenarios, make task planning, and evaluate safety systematically. Utilizing this framework, we construct EARDataset, comprising diverse test cases across various domains, encompassing both textual and visual scenarios. Our comprehensive evaluation of state-of-the-art foundation models reveals alarming results: all models exhibit high task risk rates (TRR), with an average of 95.75% across all evaluated models. To address these challenges, we further propose two prompting-based risk mitigation strategies. While these strategies demonstrate some efficacy in reducing TRR, the improvements are limited, still indicating substantial safety concerns. This study provides the first large-scale assessment of physical risk awareness in EAI agents. Our findings underscore the critical need for enhanced safety measures in EAI systems and provide valuable insights for future research directions in developing safer embodied artificial intelligence system. Data and code are available at https://github.com/zihao-ai/EARBench.
Forward citations
Cited by 14 Pith papers
-
RoboJailBench: Benchmarking Adversarial Attacks and Defenses in Embodied Robotic Agents
RoboJailBench creates a taxonomy-based benchmark, intent-contrast datasets, and evaluation framework for jailbreak attacks and defenses in embodied robotic AI systems.
-
Harnessing Embodied Agents: Runtime Governance for Policy-Constrained Execution
A runtime governance framework for embodied agents achieves 96.2% interception of unauthorized actions and 91.4% recovery success in 1000 simulation trials by externalizing policy enforcement.
-
When Words Are Safe But Actions Kill: Probing Physical Danger Beyond Text Safety in Hidden-State Risk Space
Content danger and physical danger form separable hidden-state signals in LLMs, and a single-layer logistic probe (PRISM) detects both at lower false-positive rates than LLM judges or text guardrails.
-
Self-Evolving Just-In-Time Memory for Proactive Embodied Safety
A Just-In-Time Memory framework with graph-based state tracking and self-evolving safety skills improves safe task completion in household robots by up to 30 percentage points on IS-Bench.
-
Benchmarking the Safety of Large Language Models for Robotic Health Attendant Control
Most of 72 tested LLMs complied with harmful medical-robot orders over half the time, with open-weight models far worse than proprietary ones.
-
SafetyALFRED: Evaluating Safety-Conscious Planning of Multimodal Large Language Models
SafetyALFRED shows multimodal LLMs recognize kitchen hazards accurately in QA tests but achieve low success rates when required to mitigate those hazards through embodied planning.
-
SENTINEL: A Multi-Level Formal Framework for Safety Evaluation of Foundation Model-based Embodied Agents
A three-level temporal-logic safety evaluator for embodied LLM agents that checks NL-to-LTL interpretation, plan compliance, and CTL over simulated execution trees.
-
RoboInspector: Unveiling the Unreliability of Policy Code for LLM-enabled Robotic Manipulation
LLM-generated robot policy code is unreliable, with failures clustering into four behavior types that grow with task complexity and shrink with instruction detail; a failure-feedback retry improves success up to 35%.
-
Harnessing Embodied Agents: Runtime Governance for Policy-Constrained Execution
An external runtime governance layer for embodied agents intercepts unauthorized actions at ~96% and recovers from runtime drift at ~91% under policy constraints in simulation, outperforming pre-execution-only baselin...
-
Benchmarking the Safety of Large Language Models for Robotic Health Attendant Control
LLMs for robotic health attendant control violate safety rules in 54.4% of harmful scenarios on average, with proprietary models at 23.7% median violation versus 72.8% for open-weight models, indicating they are not y...
-
Harnessing Embodied Agents: Runtime Governance for Policy-Constrained Execution
A runtime governance framework for embodied agents intercepts 96.2% of unauthorized actions and achieves 91.4% recovery success in 1000 simulation trials while outperforming baselines.
-
Context-Aware Risk Estimation in Home Environments: A Probabilistic Framework for Service Robots
A semantic graph framework propagates risk scores derived from a national accident database across spatial object relations, reporting 75% binary risk detection accuracy on 20 human-annotated NYU V2 home images.
-
Towards provable probabilistic safety for scalable embodied AI systems
The paper proposes a paradigm of provable probabilistic safety to enable scalable, safe deployment of embodied AI in critical applications.
-
Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluation
A synthesis of 247 papers on LLM agent security identifies prompt injection and tool hijacking as dominant threats, notes weakly compositional defenses, and argues for trust boundaries and realistic evaluations.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.