Pith. sign in

REVIEW 3 major objections 3 minor 28 references

Not an A11y: How Android Accessibility Exposes Mobile AI Agents to Indirect Prompt Injection

T0 review · 3 major / 3 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper argues that mobile AI agents that read Android screens through accessibility-tree metadata can be hijacked by text embedded in the interface itself: notifications, overlays, web content, calendar entries, contact fields, hidden…

desk verdict A solid, systematically mapped empirical study of A11y-mediated indirect prompt injection in Android agents; the qualitative finding is credible, but the headline rates are descriptive, not causal, until a no-injection baseline and more trials are added. read the letter →

arxiv 2608.08939 v1 pith:2KKBH3MN submitted 2026-08-09 cs.AI

Rahul Deivasigamani , Sayeda Faatin Alvi , Derqui Andrea , Kaushal Punjabi , Stjepan Picek This is my paper · ORCID
classification cs.AI
keywords indirectpromptinjectionmobileAIagentsAndroidaccessibilitytreemetadataagentsecuritycontextdriftunauthorizedactions
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that mobile AI agents that read Android screens through accessibility-tree metadata are systemically vulnerable to indirect prompt injection: text embedded in notifications, overlays, web content, calendar entries, contact notes, hidden accessibility nodes, and planning-layer guides can be mistaken for trusted user instructions. This matters because these agents act on real devices, so injected text can make them abandon the original task, cross app boundaries, and perform actions the user never requested. The authors demonstrate the vulnerability with nine attack vectors across two agent frameworks, two language models, and multiple perception/execution configurations, reporting aggregate attack-success rates up to 0.822 and showing that even the most robust tested combination leaves context drift and unauthorized actions. The paper concludes that accessibility-derived content must be treated as untrusted data and that current frameworks lack the provenance tracking and data-instruction separation needed for safe deployment.

What carries the argument

The central object is the Android accessibility (A11y) tree: the structured dump of UI elements, text, content descriptions, bounds, and state that agents use instead of, or alongside, screenshots. The failure mechanism is data-instruction conflation: user goals and environmental text enter the same natural-language channel with no marker separating trusted intent from untrusted content, so the planner treats passive UI text as authoritative. The evaluation machinery is the A1-A9 attack taxonomy—runtime injections (notifications, same-app and cross-app overlays), application-content injections (hidden accessibility metadata, HTML, calendar, notes, contacts), and planning-layer injection (poisoned AppCards)—together with the metrics ASR, CDR, and UAR, which distinguish clean rejection, partial compromise, and completed attack. This design lets the paper attribute failures to the injection surface, framework architecture, model backend, and perception modality rather than to a single implementation bug.

What would settle it

Re-run the benchmark with full device-image restoration or randomized trial order, then compare first-trial attack success with aggregate attack success. If first-trial success rates are markedly lower than the reported 0.822 and 0.150 aggregates, carry-over device state is inflating the attack rates; if first-trial rates match the aggregates, the injected instructions are the cause.

Watch

Extended reading notes

Core claim

The central claim is that Android's accessibility tree is not a neutral observation channel but an attacker-influenced trust boundary. Because agent frameworks feed user goals and application-controlled UI text into the same reasoning stream without a provenance marker, an adversary who controls any piece of accessible content can inject instructions that redirect the agent. The paper supports this with a nine-vector taxonomy (A1-A9), a standardized adversarial objective (open Settings and retrieve the device Build Number), and three outcome metrics—Attack Success Rate, Context Drift Rate, and Unauthorized Action Rate—that separate completed hijack from reasoning compromise. Across 340 MobileRun trials and 80 Mobile-Use trials, MobileRun with Gemma4:31B reaches aggregate attack success of 0.822, while the most robust configuration, Mobile-Use with Qwen3.6:35B, lowers aggregate attack success to 0.150 but leaves a context-drift rate of 0.325 and an unauthorized-action rate of 0.275. The paper also reports that screenshot-based vision gives inconsistent protection and that AppCard poisoning, which bypasses the screen entirely, succeeds on every evaluated trial.

Load-bearing premise

The paper's headline numbers assume each trial starts from a restored baseline, and the authors note that residual state—recent searches, previously opened Settings pages, cached agent memory—may leak between trials; if that leakage is large, reported attack-success rates overstate the injected payloads' effect.

Editorial extensions

If this is right

  • Any accessibility-driven mobile agent that feeds unsanitized UI text into its planner inherits the vulnerability; the paper's results make it an architectural property rather than a quirk of one model or app.
  • Screenshot-based vision should not be treated as a security boundary: the paper finds it lowers attack success in some configurations, raises it in others, and leaves application-content and planning-layer attacks effective.
  • Stronger model robustness reduces but does not solve the problem: switching to Qwen3.6:35B in Mobile-Use cuts aggregate attack success from 0.750 to 0.150 while context drift and unauthorized actions persist.
  • Hidden-metadata and planning-layer surfaces are the highest-risk channels: A4 (hidden accessibility nodes) and A9 (poisoned AppCards) are among the strongest vectors, the latter achieving full success in every evaluated trial.
  • Deployment on real user accounts would turn the benchmark's harmless Build-Number objective into unauthorized cross-app navigation, information disclosure, or permission-sensitive actions, so the measured rates are a lower bound on impact.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not test desktop or iOS agents, but the hidden-metadata mechanism suggests any agent consuming accessibility trees, OCR, or screen-reader streams is exposed wherever machine-visible text diverges from human-visible text.
  • An untested extension of the defense discussion: attaching provenance (source app, human visibility, trust level) to every UI element before planning, and gating cross-app actions on explicit confirmation, should sharply reduce hidden-node and planning-layer attack success.
  • A falsifiable prediction of the paper's conflation account is that a simple system-prompt rule telling the model to ignore instructions found in content descriptions would lower hidden-metadata success rates but not to zero, because the root cause is planner-level trust rather than prompt wording.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. The paper investigates indirect prompt injection against mobile AI agents that rely on Android accessibility (A11y) trees as their primary observation channel. It introduces a nine-vector taxonomy (A1–A9) spanning notifications, runtime overlays, application content, hidden accessibility metadata, and planning-layer AppCard poisoning. The evaluation runs MobileRun and Mobile-Use with Gemma4:31B and Qwen3.6:35B over 340 and 80 trials, respectively, and scores each trial for Attack Success Rate (ASR), Context Drift Rate (CDR), and Unauthorized Action Rate (UAR). The paper reports aggregate ASR of 0.822 for MobileRun/Gemma4:31B, 0.544 for MobileRun/Qwen3.6:35B, 0.750 for Mobile-Use/Gemma4:31B, and 0.150 for Mobile-Use/Qwen3.6:35B, concluding that the A11y channel is a systemic trust boundary and that no evaluated configuration fully enforces data–instruction separation.

Significance. If the causal attribution to the injected payloads is sound, this is a valuable and timely measurement of an under-explored attack surface. The paper's separation of ASR, CDR, and UAR gives a useful compromise taxonomy, and the comparison across two frameworks, two models, and vision/no-vision configurations is a constructive step for the mobile-agent security community. The release of benchmark tasks, payloads, sanitized traces, and analysis scripts is a concrete reproducibility contribution. However, the headline success rates are not yet cleanly attributable to the injections because the metrics are computed only over attack trials and the paper itself acknowledges residual-state effects; the significance of the quantitative claims therefore depends on adding a no-injection control arm and reporting uncertainty.

major comments (3)
  1. [§5.1.1, Eqs. (1)–(3)] The metrics are defined only over attack trials, with no no-injection control condition. The paper's own text states that residual device or framework state, including recently opened Settings pages and cached agent memory, may influence subsequent trials. Since each trial is binary and only five trials run per configuration, one spurious Settings opening or Build Number disclosure shifts the per-cell rate by 0.2. The central claim that the observed out-of-scope actions are caused by the injected instructions therefore requires a control arm in which the same benign tasks are run with the payload absent or replaced by benign text. I ask the authors to add such controls, measure how often the agent spontaneously navigates to Settings and discloses the Build Number, and report the delta between attack and control conditions with confidence intervals.
  2. [§5.1.1, §7.2, Table 4] The small sample size (five trials per cell) makes the aggregate distinctions and the claimed persistence of context drift fragile. For example, Mobile-Use with Qwen3.6:35B has ASR=0.150, CDR=0.325, and UAR=0.275 over 40 trials; a few re-labeled trials or one or two residual-state artifacts would materially change these numbers. The paper acknowledges the descriptive nature in Section 7.2, but the abstract and conclusion present these magnitudes without uncertainty bounds. I request Wilson or bootstrap confidence intervals for the reported rates and a clear statement of which cross-model or cross-framework differences are distinguishable from noise.
  3. [§4.4, §7.2] Trial annotation is manual and performed by the authors, who also designed the attack prompts. The definitions of context drift and unauthorized action require judgment, and the paper notes that traces may be incomplete or may not faithfully represent the model's internal decision process. No inter-annotator agreement is reported, and the labels are not described as blinded. Given that the central measurements depend on these labels, I ask the authors to report inter-annotator agreement on a random sample, publish the full annotation guidelines, and describe the blinding protocol used when labeling trials.
minor comments (3)
  1. [§4.2] The statement 'across nine attack vectors, two frameworks' appears in the conclusion, but A9 is evaluated only on MobileRun; this asymmetry should be stated more prominently wherever framework-level aggregates are discussed.
  2. [Table 2] The merged cell layout in the MobileRun results table is difficult to read, especially for ASR2/CDR2/UAR2 columns; a long-format table with one row per vector-configuration-model or separate sub-tables per metric would improve readability.
  3. [Appendix C.1] The paper says the displayed payloads are 'representative' and that wording variations were introduced across configurations; this makes cross-configuration comparisons less controlled. I recommend reporting the exact payloads used for each configuration in the released artifact and quoting them in a supplementary table.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: this is an empirical measurement study whose reported rates are observed outcomes, not derivations from fitted inputs or self-referential definitions.

full rationale

The paper's central claim is that Android accessibility trees expose mobile agents to indirect prompt injection. This claim is supported by a benchmark in which attack success, context drift, and unauthorized action are manually labeled from execution traces and then aggregated by definitions in Section 4.3 and Equations (1)–(3). These definitions are operationalizations, not fitted parameters renamed as predictions: ASR, CDR, and UAR are computed from observed trial outcomes, and the attack-success label requires the concrete, independently checkable event of Build Number disclosure. The attack vectors place adversarial instructions in environmental content, but whether the agent follows them is an empirical question; the reported variation across models, frameworks, and configurations (e.g., MobileRun Gemma4:31B ASR3 = 0.822 versus Mobile-Use Qwen3.6:35B ASR3 = 0.150) shows the outcomes are not forced by construction. The paper's own residual-state caveat in Section 5.1.1 and Section 7.2, and the absence of a no-injection baseline, are threats to causal attribution and external validity, not evidence of circularity; they do not make the measured rates equivalent to the benchmark definitions. I found no load-bearing self-citation, no imported uniqueness theorem, and no renaming of a known result presented as a derivation. The benchmark is self-designed, which affects independence and generality, but that is a methodology concern rather than a circularity finding.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the assumption that the evaluated frameworks' A11y extraction and prompting match how production agents operate, and that the manual labeling captures the relevant behavior. No free parameters are fitted; the rates are direct measurements. No new entities are postulated.

assumptions (4)
  • domain assumption The Android accessibility tree content observed by the agent is faithfully passed to the LLM without sanitization or provenance labeling in the tested frameworks.
    The entire attack model depends on frameworks conveying A11y text into the model prompt; the paper states this as the architectural premise in Sections 1 and 2.1.
  • domain assumption The adversarial objective (Build Number disclosure) is a valid proxy for more impactful unauthorized actions such as data exfiltration or account changes.
    The paper uses this proxy to measure attack success across vectors and acknowledges in Section 7.2 that the benchmark does not cover account modification or financial transactions, so the proxy's representativeness is assumed.
  • domain assumption Manual annotation of traces reliably distinguishes context drift from mere quoting.
    Section 4.3 defines the criteria and Section 7.2 admits that context drift may be difficult to identify when traces are incomplete or do not faithfully represent the model's internal decision process.
  • domain assumption The tested frameworks (MobileRun v0.6.0 and Mobile-Use v3.3.0) are representative of accessibility-driven mobile agents.
    The systemic conclusion generalizes from two frameworks; Section 7.2 flags this as a limitation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Not an A11y: How Android Accessibility Exposes Mobile AI Agents to Indirect Prompt Injection." pith.science (2026). https://pith.science/paper/2KKBH3MN

@misc{pith2026260808939,
  author       = {Pith},
  title        = {Pith review of: Not an A11y: How Android Accessibility Exposes Mobile AI Agents to Indirect Prompt Injection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2KKBH3MN}},
  note         = {Machine review of arXiv:2608.08939}
}
read the original abstract

The rise of autonomous AI agents represents a major paradigm shift in how users interact with mobile devices. Frameworks such as MobileRun and Mobile-Use can autonomously navigate Android applications and execute complex multi-step tasks. To interpret user interfaces, these frameworks rely primarily on Android accessibility (A11y) trees and secondarily on visual screenshots. In this paper, we demonstrate that this architectural dependence on unsanitized accessibility metadata, together with visual input, introduces a systemic vulnerability to indirect prompt injection. We show that adversarial prompts can cause autonomous agents to abandon their original objectives, violate context boundaries, and perform unauthorized device actions. Our empirical evaluation demonstrates goal hijacking, context drift, and unauthorized actions across visually hidden and fully exposed attack scenarios. In aggregate, MobileRun reaches an attack success rate of 0.822 with Gemma4:31B, while Mobile-Use with Qwen3.6:35B reduces this to 0.150 but does not eliminate context drift or unauthorized actions. These findings reveal that current mobile agent frameworks fail to enforce semantic context boundaries, treating passive environmental text as trusted instructions. Finally, we present a taxonomy of these attacks and discuss the need for zero-trust input validation, dedicated security agents, and strict context isolation within mobile agent architectures.

Figures

Figures reproduced from arXiv: 2608.08939 by the authors.

Figure 1
Figure 1. MobileRun architecture and evaluated adversarial [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Mobile-Use architecture and the flow of task and interface context through its specialized agent components [ [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Representative workflow for A4, an application [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Aggregate ASR, CDR, and UAR by framework and [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

28 extracted references · 15 canonical work pages

  1. [1]

    d.].Create an Accessibility Service

    Android Developers [n. d.].Create an Accessibility Service. Android Developers. https://developer.android.com/guide/topics/ui/accessibility/service Not an A11y: How Android Accessibility Exposes Mobile AI Agents to Indirect Prompt Injection

    work page

  2. [2]

    Deepak Bhaskar Acharya, Karthigeyan Kuppan, and B. Divya. 2025. Agentic AI: Autonomous Intelligence for Complex Goals—A Comprehensive Survey.IEEE Access13 (2025), 18912–18936. doi:10.1109/ACCESS.2025.3532853

    arXiv 2025

  3. [3]

    Edoardo Debenedetti, Ilia Shumailov, Tianyu Fan, Jamie Hayes, Nicholas Carlini, Daniel Fabian, Christoph Kern, Cathy Shi, Andreas Terzis, and Florian Tramèr

    work page

  4. [4]

    Edoardo Debenedetti, Jie Zhang, Mislav Balunovic, Luca Beurer-Kellner, Marc Fischer, and Florian Tramèr. 2024. AgentDojo: A Dynamic Environment to Evaluate Attacks and Defenses for LLM Agents. InAdvances in Neural Information Processing Systems

    work page 2024

  5. [5]

    Chenghao Du, Quanfeng Huang, Tingxuan Tang, Zihao Wang, Adwait Nad- karni, and Yue Xiao. 2025. Measuring the Security of Mobile LLM Agents under Adversarial Prompts from Untrusted Third-Party Channels.arXiv preprint arXiv:2510.27140(2025). doi:10.48550/arXiv.2510.27140

    work page Pith review arXiv doi:10.48550/arxiv.2510.27140 2025

  6. [6]

    Roush, Judah Goldfeder, and Ravid Shwartz-Ziv

    Pierre-Louis Favreau, Jean-Pierre Lo, Clement Guiguet, Charles Simon- Meunier, Nicolas Dehandschoewercker, Allen G. Roush, Judah Goldfeder, and Ravid Shwartz-Ziv. 2026. Do Multi-Agents Dream of Electric Screens? Achieving Perfect Accuracy on AndroidWorld Through Task Decomposition. arXiv e-prints, Article arXiv:2602.07787 (Feb. 2026), arXiv:2602.07787 pag...

    doi:10.48550/arxiv.2602.07787 2026

  7. [7]

    Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz. 2023. Not What You’ve Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection. In Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security (Copenhagen, Denmark)(AISec ’23). Association for Computin...

    arXiv 2023

  8. [8]

    Jie Huang, Michael Backes, and Sven Bugiel. 2021. A11y and Privacy Don’t Have to Be Mutually Exclusive: Constraining Accessibility Service Misuse on Android. In30th USENIX Security Symposium (USENIX Security 21). USENIX Association, 2983–3000

    work page 2021

Show all 28 references
  1. [9]

    Quyu Kong, Xu Zhang, Zhenyu Yang, Nolan Gao, Chen Liu, Panrong Tong, Chenglin Cai, Hanzhang Zhou, Jianan Zhang, Liangyu Chen, Zhidan Liu, Steven Hoi, and Yue Wang. 2025. MobileWorld: Benchmarking Autonomous Mobile Agents in Agent-User Interactive, and MCP-Augmented Environment...

    2025 doi

  2. [10]

    Sijia Li, Xiaoyu Tan, Shahir Ali, Niels Schmidt, Gengchen Ma, and Xihe Qiu. 2026. Curiosity Driven Knowledge Retrieval for Mobile Agents. arXiv:2601.19306 [cs.AI] https://arxiv.org/abs/2601.19306

    2026

  3. [11]

    Yanda Li, Chi Zhang, Wanqi Yang, Bin Fu, Pei Cheng, Xin Chen, Ling Chen, and Yunchao Wei. 2024. AppAgent v2: Advanced Agent for Flexible Mobile Interactions.arXiv preprint arXiv:2408.11824(2024). doi:10.48550/arXiv.2408. 11824

    2024 doi

  4. [12]

    Yupei Liu et al. 2025. MELON: Indirect Prompt Injection Defense via Masked Re-Execution and Tool Comparison.arXiv preprint arXiv:2502.05174(2025). doi:10.48550/arXiv.2502.05174

    2025 doi arXiv

  5. [13]

    Forough Mehralian et al. 2022. Too Much Accessibility Is Harmful! Automated Detection and Analysis of Overly Accessible Elements in Android Apps. In Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering. Association for Computing Machinery...

    2022

  6. [14]

    2026.App Cards Feature Documentation

    MobileRun. 2026.App Cards Feature Documentation. MobileRun. https://docs. mobilerun.ai/framework/features/app-cards

    2026

  7. [15]

    MobileRun. 2026. MobileRun Framework Overview. https://docs.mobilerun.ai/ framework/overview. Accessed: 2026-05-25

    2026

  8. [16]

    Fábio Perez and Ian Ribeiro. 2022. Ignore previous prompt: Attack techniques for language models.arXiv preprint arXiv:2211.09527(2022)

    2022 arXiv

  9. [17]

    Christopher Rawles, Sarah Clinckemaillie, Yifan Chang, Jonathan Waltz, Gabrielle Lau, Marybeth Fair, Alice Li, William Bishop, Wei Li, Folawiyo Campbell-Ajala, Daniel Toyama, Robert Berry, Divya Tyamagundlu, Timothy Lillicrap, and Oriana Riva. 2024. AndroidWorld: A Dynamic Ben...

    2024 doi arXiv

  10. [18]

    2026.Gemma 4 Technical Report

    Gemma Team. 2026.Gemma 4 Technical Report. arXiv:2607.02770 [cs.CL] doi:10. 48550/arXiv.2607.02770

    2026 doi arXiv

  11. [19]

    2026.Qwen Studio

    Qwen Team. 2026.Qwen Studio. Qwen. https://qwenlm.github.io/blog/qwen3.6/

    2026

  12. [20]

    Junyang Wang, Haiyang Xu, Haitao Jia, Xi Zhang, Ming Yan, Weizhou Shen, Ji Zhang, Fei Huang, and Jitao Sang. 2024. Mobile-Agent-v2: Mobile Device Operation Assistant with Effective Navigation via Multi- Agent Collaboration. InAdvances in Neural Information Processing Sys- tems...

    2024

  13. [21]

    Junyang Wang, Haiyang Xu, Jiabo Ye, Ming Yan, Weizhou Shen, Ji Zhang, Fei Huang, and Jitao Sang. 2024. Mobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perception. arXiv:2401.16158 [cs.CL] https://arxiv. org/abs/2401.16158

    2024 arXiv

  14. [22]

    Haichuan Xu et al . 2024. DVa: Extracting Victims and Abuse Vectors from Android Accessibility Malware. In33rd USENIX Security Symposium (USENIX Security 24). USENIX Association

    2024

  15. [23]

    Qiusi Zhan, Zhixiang Liang, Zifan Ying, and Daniel Kang. 2024. InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents. InFindings of the Association for Computational Linguistics: ACL 2024

    2024

  16. [24]

    Chi Zhang, Zhao Yang, Jiaxuan Liu, Yucheng Han, Xin Chen, Zebiao Huang, Bin Fu, and Gang Yu. 2023. AppAgent: Multimodal Agents as Smartphone Users. arXiv:2312.13771 [cs.CV] https://arxiv.org/abs/2312.13771

    2023 arXiv

  17. [25]

    Yanzhe Zhang et al. 2024. Evaluating Safety of Autonomous Agents in Mobile Device Control.arXiv preprint arXiv:2410.17520(2024). doi:10.48550/arXiv.2410. 17520

    2024 doi

  18. [26]

    Zidong Zhang, Zhentao Xie, Wenrui Diao, and Jianliang Wu. 2026. (A)I Sees What You Don’t: Exploiting New Attack Surfaces in Third-Party Mobile Agents. arXiv preprint arXiv:2607.00333(2026). doi:10.48550/arXiv.2607.00333

    2026 doi arXiv

  19. [27]

    testtag

    Zilong Zou et al . 2026. Architecting a Secure, Intent-Centric Mobile Agent Operating System.arXiv preprint arXiv:2602.10915(2026). doi:10.48550/arXiv. 2602.10915 A Open Science We share all of our empirical demonstrations and attack prompts to foster future research and contr...

    2026 doi

  20. [2025]

    doi:10.48550/arXiv.2503.18813

    Defeating Prompt Injections by Design.arXiv preprint arXiv:2503.18813 (2025). doi:10.48550/arXiv.2503.18813

    2025 doi arXiv

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.