Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

A three-module reliability layer keeps LLM-driven robots on task when instructions are injected with malicious prompts.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A unified framework of secure prompting, state memory, and rule-based safety validation improves LLM-driven robot navigation under prompt injection attacks and obstacle-heavy environments, with modest real-robot verification.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection Worth refereeing, but the 325%/30.8% numbers don't survive contact with the paper's own formula: zero baselines make GI underdetermined. the 3 major comments →

arxiv 2509.02163 v1 pith:6HEXSP3D submitted 2025-09-02 cs.RO cs.AI

Enhancing Reliability in LLM-Integrated Robotic Systems: A Unified Approach to Security and Safety

Wenxiao Zhang , Xiangrui Kong , Conan Dewitt , Thomas Br\"aunl , Jin B. Hong This is my paper
classification cs.RO cs.AI
keywords LLM-driven mobile roboticsprompt injection defensegoal hijacking attacksafety validationstate managementmission-oriented exploration ratesim-to-real robot evaluation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that LLM-controlled mobile robots can be made more trustworthy by wrapping the model in a three-part reliability layer: structured prompt assembly with a security prefix, state management that feeds past commands and outcomes back into the model, and a LiDAR-based rule checker that validates move commands before execution. It defines two prompt-injection classes, Obvious Malicious Injection and Goal Hijacking Injection, and reports that its framework detects more of them and preserves mission performance that collapses in a baseline zero-shot LLM controller. In simulation, the combined improvement over baseline is 325% under adversarial conditions across obstacle environments and 30.8% in a deeper security-focused attack-rate study; a physical robot with camera and LiDAR replicates the trend with smaller but consistent gains. If the claim holds, a practical, model-agnostic reliability layer can bridge safety and security for embodied LLM agents without retraining the model.

Core claim

The paper's central claim is that a pipeline combining prompt assembling, state management, and safety validation makes a GPT-4o-driven mobile robot both safer and more attack-resistant. At each decision step, the LLM receives a system prompt containing a security directive warning that human instructions may come from attackers, plus multimodal inputs, and is required to emit structured reasoning including an is_attack flag before any control command. Before a Move command reaches the actuators, a rule checks that all LiDAR distances in the heading cone exceed the move distance plus a safety margin; invalid commands trigger up to three retries and, if still invalid, mission failure. In the

What carries the argument

A modular reliability layer with three interacting components. Prompt Assembling wraps each LLM request in a system prompt that includes a Security Prefix warning that human instructions may be adversarial, together with a user prompt combining camera image, LiDAR image, human instruction, and the previous step's reference state. State Management stores each command-execution pair and validation outcome in a lightweight database and feeds the last executed state back into the next prompt. Safety Validation evaluates Move commands with a rule: for all angles in a heading cone, the LiDAR distance minus the commanded distance must be at least a safety margin; violations trigger retries with fai

Load-bearing premise

The security metrics assume the attacked LLM can reliably recognize and report that its own input was manipulated, because detection is measured by asking the same model for an is_attack flag; a successful injection that changes behavior without being flagged, or over-flagging of benign phrasing, would weaken the central security claim.

What would settle it

Run goal-hijacking attacks that physically redirect the robot, record ground-truth attack labels for every prompt, and count how many behavior-changing attacks produce is_attack=false. If a substantial share of successful injections go unflagged, or if benign rephrasings are flagged as attacks, then the reported detection metrics overstate the defense; an independent human or external classifier could serve as the ground-truth labeler.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • In obstacle-filled simulated environments, the unprotected LLM controller fails every trial under obvious malicious injection; with the reliability layer it completes missions, so the defense is not only raising detection but also saving mission success.
  • Goal Hijacking Injection is the harder threat class: without defense, precision, recall, and F1 all drop to zero, and even with defense F1 recovers only to about 0.45 while mission performance stays below baseline, indicating future defenses must target subtle goal misalignment.
  • The defense adds real cost: token usage rises slightly and response time increases by roughly 18-28% in simulation, but the physical robot shows only about a 1% latency increase, suggesting the overhead is acceptable for real-time operation.
  • The framework requires no fine-tuning or architectural changes, and sim-to-real trials on a physical robot show the same ordering of defended versus undefended performance, supporting the transferability of this reliability pattern.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The security results rest on the attacked model's own is_attack flag, so detection is measured by asking the same LLM to report whether its own input was manipulated. The paper itself admits silent failures and overreactions in Section 8.4, meaning reported ADR, precision, recall, and F1 may overstate how many attacks are genuinely caught.
  • The safety validator only constrains Move commands; Turn and Stop are treated as inherently safe. An attacker who steers the robot through a sequence of turns, or who corrupts the LiDAR image directly, would not be blocked by this rule, so the claimed safety guarantee is narrower than general operational safety.
  • The 325% headline is an aggregate over metrics where the baseline collapses to zero in hard environments; real-world relative gains are much smaller (+40% for OMI, +28.6% for GHI). A conservative reading is that the framework converts catastrophic failures into degraded but functional performance rather than enabling near-perfect operation.
  • A testable extension would apply the same three-module pattern to other embodied agents, such as manipulators or quadrotors, replacing the LiDAR move-check with the corresponding safety predicate and comparing mission success under the same two attack classes.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a unified reliability layer for LLM-integrated mobile robots, combining structured prompt assembly (with a security prefix), state management, and rule-based safety validation. It introduces new metrics (MOER, TLR, ADR) and evaluates the framework against two prompt-injection classes (OMI and GHI) in simulation and on a physical Pioneer robot. The headline claims are a 325% improvement under adversarial conditions in Scenario 1 (GI = 3.25) and a 30.8% improvement in Scenario 2 (GI = 0.308). The framework is open-sourced and a real-robot deployment is reported.

Significance. If the quantitative claims are reproducible, the framework is a useful step toward practical reliability layers for LLM-driven robots: it couples prompt-level defence with deterministic safety validation, and the sim-to-real check is valuable. The paper is also unusually transparent about limitations, including tuning and edge cases. However, the headline improvement magnitudes rest on an aggregation formula that is undefined for zero baselines, and the security metrics rely on the attacked model's self-report. These issues must be resolved before the central numbers can be accepted.

major comments (3)
  1. [Section 5.4 with Sections 6.1.1 and 6.2.1] The headline GI values are not computable as defined. The relative difference ΔX(i) = (X_d,i − X_nd,i)/X_nd,i in Section 5.4 is undefined whenever the baseline X_nd,i is zero. The paper states that in Scenario 1 the SO, DO, and MO baselines 'register zero in key metrics' under OMI attacks (§6.1.1), and in Scenario 2 the no-defence precision and recall for GHI are 0.0 (§6.2.1). Nevertheless, §6.1.2 reports GI = 3.25 and §6.2.4 reports GI = 0.308 without stating how zero denominators were handled (dropped, clipped, or replaced). Dropping conditions changes the metric set; replacing with an arbitrary constant changes the result. These two central quantitative claims are therefore not reproducible from the information given. Please specify the exact handling rule and provide a robustness analysis using alternative aggregations (e.g., additive improvements or bounded imputation).
  2. [Section 4.3, Table 4, and Section 8.4] The security detection metrics (ADR, precision, recall, F1) are computed from the attacked GPT-4o's own 'is_attack' flag in its structured output, not from an independent ground-truth label or from observed behavioral consequences. The paper itself acknowledges in §8.4 that the system 'may fail silently—e.g., ignoring adversarial prompts without triggering defensive responses—or overreact to benign variations,' and that these cases were 'not included in the main figures.' This makes the detection metrics partly circular and potentially optimistic. Please validate the self-reported flags against human or rule-based labels, or clearly reframe the metrics as measuring the model's introspection accuracy and temper the corresponding security claims.
  3. [Section 5.3.1 and Section 6.3.4] The primary performance metric MOER is constructed with penalty weights α = 0.6 and β = 0.3 that were 'tuned to provide reasonable discrimination' on pilot runs. Because the GI values are averages of relative improvements in MOER (together with other metrics), the reported 325% and 30.8% magnitudes inherit this tuning. The sensitivity analysis in Table 5 shows that varying α and β by ±0.1 changes absolute MOER values considerably (e.g., OMI no-defence ranges from 0.16 to 0.29 across the table), even if the ordinal ranking of configurations is stable. The paper should report the GI values under the nine parameter combinations in Table 5, or otherwise quantify how much of the headline improvement is an artifact of the chosen penalty weights. This is needed before the abstract's percentage claims can be evaluated.
minor comments (5)
  1. [Section 5.4.1] The weighting term AR_i in the definition of W_X is described as 'attack rate or condition weight,' but in Scenario 1 only a single attack rate (0.5) is used and no explicit condition weights are given. Please state explicitly which weights were used for each condition and scenario.
  2. [Section 6.3.4 / Table 5] The sentence 'GHI consistently outperforms OMI' appears to be the reverse of what the data show: GHI yields lower MOER values and is described elsewhere as the more disruptive attack. Please correct the wording to 'GHI consistently underperforms OMI' or similar.
  3. [Section 5.3.1] In the MOER formula, for completed trials the factor t_j = |S_max|/s_j cancels the exploration ratio, so completed trials contribute exactly 1 regardless of path efficiency. The description of MOER as an 'exploration rate' is therefore misleading; it is closer to a weighted completion rate. Please clarify.
  4. [Figures 7 and 8; Table 6] No number of trials per condition or confidence intervals/error bars are reported. Given the stochastic nature of LLM outputs and sensor noise, the reported point estimates (e.g., Table 6) need at least the number of runs and ideally dispersion measures.
  5. [Section 7] The real-world validation uses a single static map and a small subset of Scenario 2. This is acknowledged in §8.4, but the abstract's 'sim-to-real consistency' phrasing should be softened to reflect the limited scope of the physical trials.

Circularity Check

0 steps flagged

No significant circularity: the framework's safety and performance claims are not derived from their own inputs; the main quantitative caveat is an undefined zero-baseline ratio in the GI computation, which is a reproducibility problem rather than circularity.

full rationale

The paper's derivation chain is not circular. The proposed reliability layer (prompt assembly, state management, safety validation) is implemented as an external pipeline: safety validation uses a geometric LiDAR rule (Eq. 8) independent of the LLM's own assertions, and the attack-detection metrics compare the LLM's `is_attack` field against experimentally known injected labels, so the ground truth is the attack set, not the metric. The security prefix is explicitly borrowed from prior external work [41], and the only self-citation [43] supplies a simulator/setting figure and is not load-bearing. The MOER penalty weights (alpha=0.6, beta=0.3) are empirically tuned, but the sensitivity analysis (Section 6.3.4) shows the ordinal conclusions do not depend on them, so this is not a fitted parameter being renamed as a prediction. The most serious concern is that the headline GI values are not reproducible: Section 5.4 defines Delta X(i) = (X_d - X_nd)/X_nd, but Sections 6.1.1 and 6.2.1 report baseline values of zero for key metrics (SO/DO/MO under OMI; GHI precision/recall/F1), making the ratio undefined; Section 6.1.2/6.2.4 report GI=3.25 and 0.308 without stating the imputation rule. This undermines the quantitative headline and should be corrected, but it is an identification/statistical-reporting flaw, not a claim whose output is equivalent to its input by construction.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

The framework depends on several empirically tuned constants (MOER penalties, retry threshold, timeout) and on the untested assumptions that the LLM's self-reported attack flag is reliable and that LiDAR truthfully represents the environment. These are reasonable engineering choices but they delimit the claim's strength.

free parameters (4)
  • MOER penalty weights alpha, beta = alpha=0.6, beta=0.3
    Selected 'based on empirical observations' (Section 5.3.1); sensitivity analysis shows ordering is stable but absolute values vary 0.09-0.53 (Table 5).
  • retry threshold N = 3
    Chosen to limit LLM retry loops; additional experiments with higher thresholds showed marginal gains but increased cost (Section 6.3.1).
  • timeout tau = 100 s
    Selected via sensitivity analysis across 60-150 s to balance completion and long-tail retries (Section 6.3.2).
  • safety validation range r and margin dist = not reported
    Used in Eq. 8; values are not disclosed in the paper, making the collision gate's strictness unreproducible.
axioms (5)
  • domain assumption The attacked LLM can reliably self-report whether its input is an attack when given a security prefix and structured output format.
    Underlies ADR, precision, recall and F1 computations (Section 4.3, Table 4).
  • domain assumption Only Move commands pose collision risk; Turn and Stop are inherently safe.
    Section 4.1.4 states safety validation focuses on Move because Turn/Stop are non-translational/passive.
  • domain assumption LiDAR distance measurements faithfully represent obstacles for the validation rule.
    Eq. 8 validates Moves against li(theta); a spoofed or noisy LiDAR feed bypasses the gate, which the threat model allows.
  • domain assumption A zero-shot structured-prompt LLM without the three defenses is a representative baseline.
    Section 5.1 defines the baseline to isolate effect of each component.
  • ad hoc to paper MOER with pilot-tuned penalties is a valid proxy for mission robustness.
    Metric is defined with fitted weights and used as primary performance measure (Section 5.3.1).

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Enhancing Reliability in LLM-Integrated Robotic Systems: A Unified Approach to Security and Safety." pith.science (2026). https://pith.science/paper/6HEXSP3D

@misc{pith2026250902163,
  author       = {Pith},
  title        = {Pith review of: Enhancing Reliability in LLM-Integrated Robotic Systems: A Unified Approach to Security and Safety},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6HEXSP3D}},
  note         = {Machine review of arXiv:2509.02163}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Integrating large language models (LLMs) into robotic systems has revolutionised embodied artificial intelligence, enabling advanced decision-making and adaptability. However, ensuring reliability, encompassing both security against adversarial attacks and safety in complex environments, remains a critical challenge. To address this, we propose a unified framework that mitigates prompt injection attacks while enforcing operational safety through robust validation mechanisms. Our approach combines prompt assembling, state management, and safety validation, evaluated using both performance and security metrics. Experiments show a 30.8% improvement under injection attacks and up to a 325% improvement in complex environment settings under adversarial conditions compared to baseline scenarios. This work bridges the gap between safety and security in LLM-based robotic systems, offering actionable insights for deploying reliable LLM-integrated mobile robots in real-world settings. The framework is open-sourced with simulation and physical deployment demos at https://llmeyesim.vercel.app/

Figures

Figures reproduced from arXiv: 2509.02163 by Conan Dewitt, Jin B. Hong, Thomas Br\"aunl, Wenxiao Zhang, Xiangrui Kong.

Figure 1
Figure 1. Figure 1: The Threat Model of the LLM-Integrated Mobile Robotic System [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 3
Figure 3. Figure 3: LiDAR Processing [43] 3.2. Adversarial Attack Strategies [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: The Workflow of the Proposed LLM-Integrated Mobile Robot System [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Experimental Settings of Safety Evaluation [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Experimental Settings of Security Evaluation [ [PITH_FULL_IMAGE:figures/full_fig_p011_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Scenario 1 Evaluation Results: Performance comparison between Baseline and our approach across multiple metrics in [PITH_FULL_IMAGE:figures/full_fig_p012_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Scenario 2 Evaluation Results: Performance comparison across di [PITH_FULL_IMAGE:figures/full_fig_p013_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Sim-to-real deployment setup: (a) shows the physical [PITH_FULL_IMAGE:figures/full_fig_p014_9.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Software Engineering for Self-Adaptive Robotics: A Research Agenda

    cs.SE 2025-05 unverdicted novelty 3.0

    This paper proposes a research agenda for software engineering of self-adaptive robotic systems along lifecycle stages and enabling technologies, identifying challenges and a roadmap to 2030.

Reference graph

Works this paper leans on

46 extracted references · 26 canonical work pages · cited by 1 Pith paper

  1. [1]

    Do As I Can, Not As I Say: Grounding Language in Robotic A ffordances

    Ahn, M., Brohan, A., Brown, N., Chebotar, Y ., Cortes, O., David, B., Finn, C., Fu, C., Gopalakrishnan, K., Hausman, K., et al., 2022. Do As I Can, Not As I Say: Grounding Language in Robotic A ffordances. arXiv preprint arXiv:2204.01691

    Pith/arXiv arXiv 2022

  2. [2]

    A Survey on Deep Learning Tools Dealing with Data Scarcity: Definitions, Challenges, Solutions, Tips, and Applications

    Alzubaidi, L., Bai, J., Al-Sabaawi, A., Santamaría, J., Albahri, A.S., Al- Dabbagh, B.S.N., Fadhel, M.A., Manoufali, M., Zhang, J., Al-Timemy, A.H., et al., 2023. A Survey on Deep Learning Tools Dealing with Data Scarcity: Definitions, Challenges, Solutions, Tips, and Applications. Journal of Big Data 10, 46

    work page 2023

  3. [3]

    Remembr: Building and reasoning over long-horizon spatio-temporal memory for robot navigation

    Anwar, A., Welsh, J., Biswas, J., Pouya, S., Chang, Y ., 2024. Remembr: Building and reasoning over long-horizon spatio-temporal memory for robot navigation. arXiv preprint arXiv:2409.13682

    Pith/arXiv arXiv 2024

  4. [4]

    LLM-Driven Robots Risk Enacting Discrimination, Violence, and Unlawful Actions

    Azeem, R., Hundt, A., Mansouri, M., Brandão, M., 2024. LLM-Driven Robots Risk Enacting Discrimination, Violence, and Unlawful Actions. arXiv preprint arXiv:2406.08824

    arXiv 2024

  5. [5]

    A Survey on Prompting Techniques in LLMs

    Bhandari, P., 2024. A Survey on Prompting Techniques in LLMs. arXiv preprint arXiv:2312.03740

    Pith/arXiv arXiv 2024

  6. [6]

    Cyber Security of Robots: a Comprehensive Survey

    Botta, A., Rotbei, S., Zinno, S., Ventre, G., 2023. Cyber Security of Robots: a Comprehensive Survey. Intelligent Systems with Applications , 200237

    work page 2023

  7. [7]

    Mobile Robot Programming: Adventures in Python and C

    Bräunl, T., 2023. Mobile Robot Programming: Adventures in Python and C. Springer International Publishing

    work page 2023

  8. [8]

    Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning

    Brunke, L., Gree ff, M., Hall, A.W., Yuan, Z., Zhou, S., Panerati, J., Schoellig, A.P., 2022. Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning. Annual Review of Control, Robotics, and Autonomous Systems 5, 411–444

    work page 2022

  9. [9]

    Ding, Y ., Fan, W., Ning, L., Wang, S., Li, H., Yin, D., Chua, T.S., Li, Q.,

    work page

  10. [10]

    PaLM-E: An Embodied Multimodal Language Model

    Driess, D., Xia, F., Sajjadi, M.S.M., Lynch, C., Chowdhery, A., Ichter, B., Wahid, A., Tompson, J., Vuong, Q., Yu, T., Huang, W., Chebotar, Y ., Sermanet, P., Duckworth, D., Levine, S., Vanhoucke, V ., Hausman, K., Toussaint, M., Greff, K., Zeng, A., Mordatch, I., Florence, P., 2023. PaLM-E: An Embodied Multimodal Language Model. arXiv preprint arXiv:2303.03378

    Pith/arXiv arXiv 2023

  11. [11]

    A Survey of Embodied AI: from Simulators to Research Tasks

    Duan, J., Yu, S., Tan, H.L., Zhu, H., Tan, C., 2022. A Survey of Embodied AI: from Simulators to Research Tasks. IEEE Transactions on Emerging Topics in Computational Intelligence 6, 230–244

    work page 2022

  12. [12]

    Foundation Models in Robotics: Applications, Challenges, and the Future

    Firoozi, R., Tucker, J., Tian, S., Majumdar, A., Sun, J., Liu, W., Zhu, Y ., Song, S., Kapoor, A., Hausman, K., et al., 2025. Foundation Models in Robotics: Applications, Challenges, and the Future. The International Journal of Robotics Research 44, 701–739

    work page 2025

  13. [13]

    Safe LLM-Controlled Robots with Formal Guarantees via Reachability Anal- ysis

    Hafez, A., Naderi Akhormeh, A., Hegazy, A., Alanwar, A., 2025. Safe LLM-Controlled Robots with Formal Guarantees via Reachability Anal- ysis. arXiv preprint arXiv:2503.03911

    Pith/arXiv arXiv 2025

  14. [14]

    Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysis

    Hu, Y ., Xie, Q., Jain, V ., Francis, J., Patrikar, J., Keetha, N., Kim, S., Xie, Y ., Zhang, T., Fang, H.S., et al., 2023. Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysis. arXiv preprint arXiv:2312.08782

    Pith/arXiv arXiv 2023

  15. [15]

    Language Mod- els as Zero-Shot Planners: Extracting Actionable Knowledge for Embod- ied Agents, in: International Conference on Machine Learning (ICML), PMLR

    Huang, W., Abbeel, P., Pathak, D., Mordatch, I., 2022. Language Mod- els as Zero-Shot Planners: Extracting Actionable Knowledge for Embod- ied Agents, in: International Conference on Machine Learning (ICML), PMLR. pp. 8948–8970. URL: https://proceedings.mlr.press/ v162/huang22a.html

    work page 2022

  16. [16]

    Inner Monologue: Embodied Reasoning through Planning with Language Models, in: Con- ference on Robot Learning, PMLR

    Huang, W., Xia, F., Xiao, T., Chan, H., Liang, J., Florence, P., Zeng, A., Tompson, J., Mordatch, I., Chebotar, Y ., et al., 2023. Inner Monologue: Embodied Reasoning through Planning with Language Models, in: Con- ference on Robot Learning, PMLR. pp. 1769–1782

    work page 2023

  17. [17]

    Robots Enact Malignant Stereotypes, in: ACM Conference on Fairness, Accountability, and Transparency (FAccT), pp

    Hundt, A., Agnew, W., Zeng, V ., Kacianka, S., Gombolay, M., 2022. Robots Enact Malignant Stereotypes, in: ACM Conference on Fairness, Accountability, and Transparency (FAccT), pp. 743–756

    work page 2022

  18. [18]

    MInference 1.0: Acceler- ating Pre-filling for Long-Context LLMs via Dynamic Sparse Attention

    Jiang, H., Li, Y ., Zhang, C., Wu, Q., Luo, X., Ahn, S., Han, Z., Abdi, A.H., Li, D., Lin, C.Y ., Yang, Y ., Qiu, L., 2024. MInference 1.0: Acceler- ating Pre-filling for Long-Context LLMs via Dynamic Sparse Attention. arXiv preprint arXiv:2407.02490

    Pith/arXiv arXiv 2024

  19. [19]

    Text-Based Prompt Injection Attack using Mathematical Functions in Modern Large Language Models

    Kwon, H., Pak, W., 2024. Text-Based Prompt Injection Attack using Mathematical Functions in Modern Large Language Models. Electronics 13, 5008

    work page 2024

  20. [20]

    Aligning Cyber Space with Physi- cal World: A Comprehensive Survey on Embodied AI

    Liu, Y ., Chen, W., Bai, Y ., Luo, J., Song, X., Jiang, K., Li, Z., Zhao, G., Lin, J., Li, G., et al., 2024a. Aligning Cyber Space with Physi- cal World: A Comprehensive Survey on Embodied AI. arXiv preprint arXiv:2407.06886

    Pith/arXiv arXiv

  21. [21]

    Formalizing and Benchmarking Prompt Injection Attacks and Defenses

    Liu, Y ., Jia, Y ., Geng, R., Jia, J., Gong, N.Z., 2024b. Formalizing and Benchmarking Prompt Injection Attacks and Defenses

    work page

  22. [22]

    OpenAI Vision Guide

    OpenAI, 2024. OpenAI Vision Guide. URL: https://platform. openai.com/docs/guides/vision. accessed: 2024-07-28

    work page 2024

  23. [23]

    Guardian: A Multi- Tiered Defence Architecture for Thwarting Prompt Injection Attacks on LLMs

    Rai, P., Sood, S., Madisetti, V .K., Bahga, A., 2024. Guardian: A Multi- Tiered Defence Architecture for Thwarting Prompt Injection Attacks on LLMs. Journal of Software Engineering and Applications 17, 43–68

    work page 2024

  24. [24]

    Safety Guardrails for LLM-Enabled Robots, in: RSS 2025 Workshop on Reliable Robotics: Safety and Security in the Face of Generative AI, pp

    Ravichandran, Z., Robey, A., Kumar, V ., Pappas, G.J., Hassani, H., 2025. Safety Guardrails for LLM-Enabled Robots, in: RSS 2025 Workshop on Reliable Robotics: Safety and Security in the Face of Generative AI, pp. 9493–9500

    work page 2025

  25. [25]

    Programming for reliability and safety in robotics: The role of domain-specific languages: Domain specific programming for safe and reliable robots

    Rizwan, M., 2024. Programming for reliability and safety in robotics: The role of domain-specific languages: Domain specific programming for safe and reliable robots. Licentiate thesis 2024

    work page 2024

  26. [26]

    Jailbreaking LLM-Controlled Robots

    Robey, A., Ravichandran, Z., Kumar, V ., Hassani, H., Pappas, G.J., 2024. Jailbreaking LLM-Controlled Robots. arXiv preprint arXiv:2410.13691

    Pith/arXiv arXiv 2024

  27. [27]

    LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action, in: Proceedings of the 6th Conference on Robot Learning (CoRL), pp

    Shah, D., Osi ´nski, B., Ichter, B., Levine, S., 2023. LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action, in: Proceedings of the 6th Conference on Robot Learning (CoRL), pp. 492–504. URL: https://proceedings.mlr.press/ v205/shah23b.html

    work page 2023

  28. [28]

    Putting GPT-4o to the Sword: A Comprehensive Evaluation of Language, Vision, Speech, and Multimodal Proficiency

    Shahriar, S., Lund, B.D., Mannuru, N.R., Arshad, M.A., Hayawi, K., Bevara, R.V .K., Mannuru, A., Batool, L., 2024. Putting GPT-4o to the Sword: A Comprehensive Evaluation of Language, Vision, Speech, and Multimodal Proficiency. Applied Sciences 14, 7782

    work page 2024

  29. [29]

    Defending Language Models Against Image-Based Prompt Attacks via User-Provided Spec- ifications, in: 2024 IEEE Security and Privacy Workshops (SPW), IEEE

    Sharma, R.K., Gupta, V ., Grossman, D., 2024. Defending Language Models Against Image-Based Prompt Attacks via User-Provided Spec- ifications, in: 2024 IEEE Security and Privacy Workshops (SPW), IEEE. pp. 112–131

    work page 2024

  30. [30]

    From LLMs to Actions: Latent Codes as Bridges in Hierarchical Robot Control, in: 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), IEEE

    Shentu, Y ., Wu, P., Rajeswaran, A., Abbeel, P., 2024. From LLMs to Actions: Latent Codes as Bridges in Hierarchical Robot Control, in: 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), IEEE. pp. 8539–8546

    work page 2024

  31. [31]

    Large Language Models Can Be Easily Distracted by Irrelevant Context, in: International Conference on Machine Learning, PMLR

    Shi, F., Chen, X., Misra, K., Scales, N., Dohan, D., Chi, E.H., Schärli, N., Zhou, D., 2023. Large Language Models Can Be Easily Distracted by Irrelevant Context, in: International Conference on Machine Learning, PMLR. pp. 31210–31227

    work page 2023

  32. [32]

    Exploring the Adversarial Vulnerabilities of Vision- Language-Action Models in Robotics

    Wang, T., Liu, D., Liang, J.C., Yang, W., Wang, Q., Han, C., Luo, J., Tang, R., 2024a. Exploring the Adversarial Vulnerabilities of Vision- Language-Action Models in Robotics. arXiv preprint arXiv:2411.13587

    Pith/arXiv arXiv

  33. [33]

    SRLM: Human-in-Loop Interactive Social Robot Navigation with Large Language Model and Deep Rein- forcement Learning

    Wang, W., Obi, I., Min, B.C., 2024b. SRLM: Human-in-Loop Interactive Social Robot Navigation with Large Language Model and Deep Rein- forcement Learning. arXiv preprint arXiv:2403.15648

    Pith/arXiv arXiv

  34. [34]

    Corelocker: Neuron-Level Usage Control, in: 2024 IEEE Symposium on Security and Privacy (SP), IEEE

    Wang, Z., Ma, Z., Feng, X., Sun, R., Wang, H., Xue, M., Bai, G., 2024c. Corelocker: Neuron-Level Usage Control, in: 2024 IEEE Symposium on Security and Privacy (SP), IEEE. pp. 2497–2514

    work page 2024

  35. [35]

    KARMA: Augmenting Embodied AI Agents with Long- and Short-Term Memory Systems

    Wang, Z., Yu, B., Zhao, J., Sun, W., Hou, S., Liang, S., Hu, X., Han, Y ., Gan, Y ., 2024d. KARMA: Augmenting Embodied AI Agents with Long- and Short-Term Memory Systems. arXiv preprint arXiv:2409.14908

    Pith/arXiv arXiv

  36. [36]

    Chain-of-Thought Prompting Elicits Reasoning in Large Language Models

    Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q.V ., Zhou, D., et al., 2022. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. Advances in neural information processing systems 35, 24824–24837

    work page 2022

  37. [37]

    How Secure Are Large Language Models (LLMs) for Navigation in Urban Environments? arXiv preprint arXiv:2402.09546

    Wen, C., Liang, J., Yuan, S., Huang, H., Fang, Y ., 2024. How Secure Are Large Language Models (LLMs) for Navigation in Urban Environments? arXiv preprint arXiv:2402.09546

    Pith/arXiv arXiv 2024

  38. [38]

    Autogen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation Framework

    Wu, Q., Bansal, G., Zhang, J., Wu, Y ., Zhang, S., Zhu, E., Li, B., Jiang, L., Zhang, X., Wang, C., 2023. Autogen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation Framework. arXiv preprint arXiv:2308.08155

    Pith/arXiv arXiv 2023

  39. [39]

    On the Safety Concerns of Deploy- ing LLMs/VLMs in Robotics: Highlighting the Risks and Vulnerabilities

    Wu, X., Xian, R., Guan, T., Liang, J., Chakraborty, S., Liu, F., Sadler, B., Manocha, D., Bedi, A.S., 2024. On the Safety Concerns of Deploy- ing LLMs/VLMs in Robotics: Highlighting the Risks and Vulnerabilities. arXiv preprint arXiv:2402.10340

    Pith/arXiv arXiv 2024

  40. [40]

    Leveraging Error- Assisted Fine-Tuning Large Language Models for Manufacturing Excel- lence

    Xia, L., Li, C., Zhang, C., Liu, S., Zheng, P., 2024. Leveraging Error- Assisted Fine-Tuning Large Language Models for Manufacturing Excel- lence. Robotics and Computer-Integrated Manufacturing 88, 102728

    work page 2024

  41. [41]

    Defensive Prompt Patch: 17 a Robust and Interpretable Defense of LLMs Against Jailbreak Attacks

    Xiong, C., Qi, X., Chen, P.Y ., Ho, T.Y ., 2024. Defensive Prompt Patch: 17 a Robust and Interpretable Defense of LLMs Against Jailbreak Attacks. arXiv preprint arXiv:2405.20099

    Pith/arXiv arXiv 2024

  42. [42]

    LiDAR-LLM: Exploring the Potential of Large Language Models for 3D LiDAR Understanding

    Yang, S., Liu, J., Zhang, R., Pan, M., Guo, Z., Li, X., Chen, Z., Gao, P., Guo, Y ., Zhang, S., 2023. LiDAR-LLM: Exploring the Potential of Large Language Models for 3D LiDAR Understanding. arXiv preprint arXiv:2312.14074

    Pith/arXiv arXiv 2023

  43. [43]

    Zhang, W., Kong, X., Dewitt, C., Braunl, T., Hong, J.B., 2024. A Study on Prompt Injection Attack Against LLM-Integrated Mobile Robotic Sys- tems, in: 2024 IEEE 35th International Symposium on Software Reliabil- ity Engineering Workshops (ISSREW), IEEE. pp. 361–368

    work page 2024

  44. [44]

    Zhen, H., Qiu, X., Chen, P., Yang, J., Yan, X., Du, Y ., Hong, Y ., Gan, C.,

    work page

  45. [46]

    arXiv preprint arXiv:2403.09631

    3D-VLA: A 3D Vision-Language-Action Generative World Model. arXiv preprint arXiv:2403.09631 . Wenxiao Zhang is a Ph.D. student at the University of Western Australia, researching the application and security of large lan- guage model-based agents in cyber-physical systems. He holds a Master’s degree in Software Engineering from the University of Western A...

    Pith/arXiv arXiv 2021

  46. [2024]

    arXiv preprint arXiv:2405.06211

    A Survey on RAG Meets LLMs: Towards Retrieval-Augmented Large Language Models. arXiv preprint arXiv:2405.06211

    Pith/arXiv arXiv

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.