Pith. sign in

REVIEW 5 major objections 6 minor 1 cited by

SeerGuard claims that mobile GUI agents can be made safe by predicting the semantic next state of each action and rejecting unsafe ones before execution.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 22:58 UTC pith:OPRUACQR

load-bearing objection A well-built empirical safety layer for mobile GUI agents with a genuinely new instruction-plus-consequence design, but the paper must clear up a likely Next-State-QA train/eval overlap and show the next-state prediction actually drives safety before the mechanism claim holds. the 5 major comments →

arxiv 2607.15550 v1 pith:OPRUACQR submitted 2026-07-17 cs.AI

SeerGuard: A Safety Framework for Mobile GUI Agents via World Model Prediction

Xue Yu , Bo Yuan , Pengshuai Yang , Kailin Zhao , Hong Hu , Junlan Feng This is my paper
classification cs.AI
keywords mobile GUI agentssafety frameworkworld modelconsequence-aware safetyinstruction screeningaction risk assessmentvision-language modelnext-state prediction
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

SeerGuard is a two-stage pre-execution safety framework for mobile GUI agents. The paper tries to establish that predicting what an action will do to the screen before executing it—in natural language, not pixels—is enough to catch most harmful actions, including prompt injections and multi-step risky workflows. The first stage screens the user instruction for explicit malicious intent; the second stage, powered by a safety-augmented world model (SAWM), predicts the semantic next state of each candidate action and labels it safe or unsafe with a rationale. Training SAWM jointly on next-state prediction and safety labeling over a 148K-instance corpus yields consistent safety gains across three different GUI agents, e.g., on MobileSafetyBench the Risk-Cost Score at α=0.8 falls from 0.347 to 0.130 for Qwen3-VL while the Safety-Utility Score at ω=0.8 rises from 0.191 to 0.596. The authors' core claim is that consequence-aware safety is both feasible and effective at runtime, replacing reactive verification with proactive intervention.

Core claim

The central discovery is that a single model trained to predict the functional next state of a screen—as a text description—can simultaneously serve as an action-level safety filter. Given a screenshot and a candidate action (tap, type, swipe, open app, etc.), the Safety-Augmented World Model outputs a predicted next state, a binary safety label, and a rationale. Because the label is conditioned on the predicted consequence rather than only on the local action or instruction, the guard can catch attacks that only become visible several steps later, such as a hidden prompt injection that would lead to an unauthorized financial trade, or a navigation step toward an illegal website. The authors

What carries the argument

The Safety-Augmented World Model (SAWM) is the load-bearing mechanism: a vision-language model fine-tuned with multi-task supervised learning to predict p(s_{t+1}|o_t, a_t) as a textual description of the next UI state and to output a safety label and rationale in one pass. The design choice of semantic prediction instead of pixel-level generation keeps the guard fast enough to run online before every action, while still capturing the functional state changes that matter for risk. SAWM is also responsible for the first-stage instruction screening, making one model serve both the coarse and fine safety filters.

Load-bearing premise

The load-bearing premise is that the machine-generated safety labels used to train SAWM faithfully match the human-defined safety ground truth in the evaluation benchmarks; if they do not, the reported safety gains may partly reflect label alignment rather than a generalisable consequence-aware risk model.

What would settle it

Take the 102 unsafe and 102 safe MobileRisk trajectories, have three independent human annotators mark the first unsafe step in each, and re-run SAWM's action-level assessment (trajectory F1 and step score) against those human labels rather than the original ones. If the scores stay at roughly the reported levels (F1 about 0.72, step score about 0.36), the consequence-prediction mechanism is confirmed; if they drop substantially, the evaluation labels and training labels were aligned in a way that inflated the results.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If SAWM's numbers hold, a guard layer of this kind can be wrapped around existing GUI agents without retraining them, lowering high-risk completion rates and raising safe refusal rates on MobileSafetyBench across open and closed backbones.
  • Explicitly malicious instructions are rejected before any app is touched, and context-dependent risks are caught at the specific action that would cause harm—so the guard blocks prompt injections and multi-step attacks that text-only classifiers miss.
  • Because next-state prediction is semantic rather than pixel-level, the overhead is low enough that the guarded agent completes MobileSafetyBench in less wall-clock time than the unguarded one, since risky trajectories terminate early.
  • The same model simultaneously reaches state-of-the-art accuracy on the Next-State-QA benchmark (0.762) and strong step-level risk detection on MobileRisk (step score 0.361), suggesting that safety reasoning and world-model foresight reinforce each other rather than compete.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The training recipe relies on labels produced by other models rather than by humans; if those labels diverge from the human judgments used in evaluation, part of the reported improvement could reflect label alignment. A human-audited re-evaluation would clarify whether the consequence-prediction mechanism itself is what transfers.
  • The semantic next-state representation is a reusable asset: the same prediction could support explanations to users ('this action would open a payment page'), plan validation, or offline simulation of agent behavior on risky instructions without executing them.
  • The authors' data-composition analysis suggests that a small amount of synthetic textual mobile risk data acts as a bridge between general safety knowledge and GUI-specific risk; if that holds, similar augmentation may reduce the need for collecting dangerous real-world mobile traces in other device-control domains.
  • A natural stress test is to move from mobile screens to desktop or web agents, where the action space is richer and the consequences of a single click are often slower to surface; SAWM's approach would predict semantic states for those interfaces as long as a text description of the workspace can be produced.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes SeerGuard, a two-stage safety framework for mobile GUI agents. The first stage performs instruction-level screening; the second stage uses a Safety-Augmented World Model (SAWM) to predict the semantic next state of the GUI before an action is executed and to issue a binary safety label with a rationale. SAWM is built on Qwen3-VL-8B-Instruct and trained by multi-task SFT on a 148K corpus that mixes general textual safety data, multimodal mobile risk re-annotations, synthetic textual mobile risk data, and MobileWorld's Next-State-QA data. Experiments on MobileSafetyBench show large reductions in Risk-Cost Score and increases in Safety-Utility Score across three GUI agents (Qwen3-VL, GPT-5.1, Gemini-3.1). Additional experiments report strong performance on the MobileRisk action-level benchmark and on the Next-State-QA prediction benchmark, with data-composition ablations in the appendix.

Significance. If the central claims hold, SeerGuard is a useful step toward pre-execution, consequence-aware safety for mobile GUI agents, addressing a real gap left by reactive or instruction-only guardrails. The paper has concrete strengths: the framework is evaluated across three different GUI backbones, the dual-stage design is intuitive, the latency analysis suggests early interception can reduce end-to-end time, and the authors include data-composition ablations in the appendix. The use of an open-source base model (Qwen3-VL-8B) is also a positive for reproducibility. However, the manuscript as submitted does not yet establish the mechanism it claims: the Next-State-QA evaluation appears to overlap with the training data, no ablation isolates the role of next-state prediction in the safety decision, and the reported numbers contain internal inconsistencies. The empirical gains on MobileSafetyBench are plausible and valuable, but the world-model/consequence-awareness story needs stronger evidence before the paper can be accepted.

major comments (5)
  1. [§5.3 and Appendix A.1] The paper evaluates SAWM on the Next-State-QA benchmark from MobileWorld, but Appendix A.1 states that the Next-State-QA dataset from MobileWorld was incorporated into the 148K training corpus, and that 'We directly adopt the original annotations without additional modification.' The manuscript never states that the evaluation split is disjoint from the training samples. If the same instances are used for training and evaluation, the reported 0.762 accuracy in Table 4 may reflect memorization rather than world-model generalization. This is load-bearing because the claim that 'high-fidelity state prediction is critical' for action-level risk assessment rests on this table. Please specify the split, or re-run the evaluation on a disjoint held-out set.
  2. [Table 3 vs. Appendix A.2, Table 7] There is a direct numerical inconsistency for the same method and benchmark. Table 3 reports SAWM on MobileRisk as Acc 0.696, P 0.664, R 0.794, F1 0.723, Step Score 0.361. Appendix A.2, Table 7 reports SAWM as Acc 0.676, P 0.643, R 0.794, F1 0.711, Step Score 0.402. The recall is identical but the other metrics differ; the Step Score differs substantially (0.361 vs. 0.402). One of these tables cannot be correct for the same checkpoint and evaluation protocol. This must be reconciled before the data-composition ablation claims can be interpreted.
  3. [§5.1 and §3.3] The central claim is that SeerGuard is consequence-aware: it prevents risky actions by predicting their likely consequences. However, no ablation removes the next-state prediction output from the safety decision. The full SeerGuard includes both state prediction and safety labeling, while the SeerGuard_act ablation still includes both. The safety gains in Table 1 could plausibly come from an over-conservative action classifier that does not actually use the predicted next-state. An ablation that (a) removes the state-prediction token from the action-level prompt/objective, or (b) conditions the safety label on the ground-truth next state vs. the predicted next state, would directly test the mechanism. Without this, the paper overclaims that consequence prediction is what drives the improvement.
  4. [§5.1 and Tables 1–4] All reported results are single-run point estimates with no error bars, confidence intervals, or significance tests. The benchmarks are relatively small (e.g., MobileSafetyBench has 250 tasks), and several reported differences are small in magnitude (e.g., RCS 0.130 vs. 0.145, SUS 0.596 vs. 0.554 at α=0.8/ω=0.8). The claim of 'consistent' improvement across backbones would be far more credible with repeated runs or a proper statistical test. Please report variance over at least a few seeds or a bootstrap analysis.
  5. [§4.1 and Appendix A.1] The training labels for the multimodal mobile risk data and the general textual safety data are machine-generated: Qwen3-VL-30B-A3B for MobileWorld re-annotation, gpt-oss-120b for text safety re-annotation, and GPT-4o for synthetic text. There is no reported validation against human labels or against the ground-truth labels in MobileSafetyBench/MobileRisk/Next-State-QA. The paper's cross-agent safety improvements may partly reflect alignment with the proxy label distribution rather than generalizable risk prevention. At minimum, the authors should report agreement rates with human annotations on a sampled subset and compare the proxy labels with the benchmark ground truth where it exists.
minor comments (6)
  1. [§5.1 / Figure 3] Figure 3's bar values do not appear to match Table 1 in all cells (e.g., GPT-5.1+SeerGuard high-risk refusal is 79.33 in the figure while Table 1 implies different aggregates). Please ensure figure data derive from the same runs as the tables.
  2. [Figure 7 caption] Typo: 'Genimi-3.1' should be 'Gemini-3.1'.
  3. [§5.1 / Baselines] The SCoT baseline is described only as 'Safety-guided Chain-of-Thought prompting.' No prompt, citation, or implementation detail is given, and it appears to be a weak baseline (it essentially replicates Direct RCS values in Table 1). Please provide a concrete description or a stronger baseline.
  4. [§5.1 / Ablation naming] The variant 'SeerGuard_Qwen' replaces SAWM with the base Qwen3-VL as the guard. Calling this an 'ablation' is confusing, since it is not a component ablation of SeerGuard but a guard-model comparison. Please clarify.
  5. [Appendix B.3–B.4] The prompts in Prompts 1 and 2 are informative, but the paper should state how the structured JSON outputs were parsed and whether any outputs failed to parse. A parsing failure rate would help assess practical reliability.
  6. [Availability] The project website is referenced, but no code, trained weights, or dataset are released. For an empirical safety paper with strong claims, releasing the SAWM weights and evaluation scripts is important for reproducibility.

Circularity Check

0 steps flagged

No significant circularity: central safety claims are validated on external benchmarks; two omitted-support caveats prevent a fully clean bill.

full rationale

SeerGuard is an empirical system paper rather than a derivation chain, and its headline results are measured against external benchmarks not part of the training corpus: MobileSafetyBench for the main safety-utility/risk-cost results (Table 1), MobileRisk for action-level risk assessment (Table 3), and Agent-SafetyBench plus Prompt Injection for instruction-level screening (Table 2). The safety labels and next-state targets used in supervised fine-tuning are generated from MobileWorld trajectories, general safety data, and synthetic mobile-risk data, but the central claims do not reduce to the training labels. There is no load-bearing self-citation: MobileWorldBench, OS-Sentinel, and other prior work are cited as external resources, not as the authors' own prior results. Two caveats are worth flagging but do not amount to demonstrated circularity under the requirement of an exhibited reduction. First, Section 4.2 and Appendix A.1 state that the training corpus combines MobileWorld's next-state QA datasets (Dqa = 48K in Table 6), while Section 5.3 evaluates on the Next-State-QA benchmark from the same MobileWorld resource without stating that the evaluation split is disjoint from training; if the benchmark overlapped the fine-tuning data, Table 4 would reflect memorization rather than generalization. This is a missing split disclosure, not a proven equation-level reduction, and Tables 1 and 3 are independent of it. Second, the paper claims that multi-task training forces the model to evaluate action safety by explicitly anticipating future visual and functional GUI consequences, but no ablation removes next-state prediction from the safety decision; this is an unsupported mechanistic attribution rather than a definitional circularity. Because those two gaps are omitted-support issues and the core results are externally benchmarked, the appropriate circularity score is low.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 0 invented entities

SeerGuard introduces no new physical or ontological entities; SAWM is a trained neural model, not a postulated entity with independent falsifiable handles. The central analysis is empirical, so the ledger's main entries are the hand-picked data recipe, the semantic-sufficiency assumption, and the reliance on machine-generated safety labels.

free parameters (3)
  • Training data mixture (Dgen:Dgui:Dtext:Dqa and safe:unsafe ratio) = 59k:33k:8k:48k; 2:1 safe:unsafe
    Chosen via the ablation in Appendix A.2 (Table 7); the SAWM configuration is the one that performs best on MobileRisk, so headline results partly depend on this hand-picked composition.
  • SFT hyperparameters (epochs, learning rate, warmup) = 1 epoch, 1e-6 LR, 3% warmup
    Selected by the authors to avoid catastrophic forgetting and preserve generalization; no sensitivity analysis or error bars are reported.
  • Safety taxonomy and prompt templates = Ten OS-Sentinel categories; Prompt 1's 'review/judgment is SAFE' rule; Prompt 2's six risk categories
    The instruction-screening prompt explicitly declares that judgment-oriented requests are safe, and the action prompt enumerates six risk categories. These design choices shape the labels and are not derived from data.
axioms (5)
  • domain assumption Semantic text description s_{t+1} is sufficient to determine action safety; pixel-level rendering is unnecessary.
    Section 3.3 replaces p(o_{t+1}|o_t,a_t) with p(s_{t+1}|o_t,a_t); if a safety-relevant change is only visible at the pixel level (e.g., a hidden warning dialog or faint visual state), the semantic model could miss it.
  • domain assumption Machine-generated annotations (Qwen3-VL-30B-A3B, gpt-oss-120b, GPT-4o) are a faithful proxy for human safety judgment.
    Appendix A.1: multimodal risk data are re-annotated by Qwen3-VL-30B-A3B, general text safety data by gpt-oss-120b, and textual mobile risk data are synthesized by GPT-4o; the paper assumes these agree with the human labels in the evaluation benchmarks.
  • domain assumption The ten-category OS-Sentinel safety taxonomy covers all mobile GUI risks relevant to the benchmarks.
    Section 3.1 adopts the OS-Sentinel taxonomy and Prompt 2 enumerates six risk categories; if a benchmark task falls outside these categories, SAWM will mislabel it.
  • standard math The latent-variable factorization of the next-state distribution holds and z_{t+1} captures the safety-relevant state.
    Section 3.3 writes p(o_{t+1}|o_t,a_t) = sum_z p(o_{t+1}|z_{t+1},o_t)p(z_{t+1}|o_t,a_t); the factorization is definitional for a latent variable, but the sufficiency of z_{t+1} for safety is an extra domain assumption.
  • ad hoc to paper The 2:1 safe-to-unsafe training ratio is the correct balance.
    Section 4.2 states 'We find that an overall safe-to-unsafe ratio of 2:1 yields a favorable balance'; the ratio is selected by hand and not derived from any principle.

pith-pipeline@v1.3.0-alltime-deepseek · 20802 in / 12607 out tokens · 139483 ms · 2026-08-01T22:58:12.253518+00:00 · methodology

0 comments
read the original abstract

Mobile graphical user interface (GUI) agents have demonstrated remarkable capabilities in automating complex tasks, yet they introduce critical safety risks where a single erroneous action can lead to irreversible consequences. Existing safety mechanisms are primarily reactive, lacking the ability to assess risks before execution. In this paper, we introduce SeerGuard, a consequence-aware safety framework designed to mitigate these risks through pre-execution instruction-level screening and action-level risk assessment. Specifically, the action-level assessment analyzes agent-proposed actions within current GUI states, anticipating likely outcomes to identify risks before they are executed. To enable these capabilities, we construct a unified safety-augmented world model (SAWM) via multi-task learning, integrating semantic next-state prediction with safety risk assessment. Extensive experiments demonstrate that SeerGuard generalizes effectively across diverse mobile GUI agents. On Qwen3-VL-8B-Instruct, it increases the safety-utility score from $0.191$ to $0.596$ at $\omega=0.8$ and reduces the risk-cost score from $0.347$ to $0.130$ at $\alpha=0.8$. Further analyses on our SAWM validate the effectiveness of the instruction-level screening, alongside the capability of action risk assessment and next-state prediction.

Figures

Figures reproduced from arXiv: 2607.15550 by Bo Yuan, Hong Hu, Junlan Feng, Kailin Zhao, Pengshuai Yang, Xue Yu.

Figure 1
Figure 1. Figure 1: Effect of SeerGuard on the Risk–Cost Score (RCS) and Safety–Utility Score (SUS) of VLM-based GUI agents on Mobile￾SafetyBench. Integrating SeerGuard increases both 1 − RCS and SUS and expands radar-area, indicating improved avoidance of harmful execution with minimal usability loss and stronger safety alignment across six task categories. 2024). This rapid progress has brought autonomous mobile interaction… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of SeerGuard: A dual-stage, consequence-aware safety framework that combines instruction-level screening and world-model-based action risk assessment, which can secure mobile GUI agents by defending against explicit malicious intention and unsafe actions before execution. The core design principle of Ginst is to prioritize high recall for explicit malicious intent while minimizing the false￾positi… view at source ↗
Figure 3
Figure 3. Figure 3: The task completion rate (left) and task refusal rate (right) of different mobile GUI agents on MobileSafetyBench. The tradeoff between task completion and task refusal is evident, with different agents exhibiting varying performances. GPT-5.1 and Gemini-3.1 achieve higher LC but higher HC coupled with insufficient HR, which can lead to safety concerns. With SeerGuard, these agents show a significant decre… view at source ↗
Figure 4
Figure 4. Figure 4: A comparative case study of visual prompt injection. Unlike the baseline Qwen3-VL, which blindly complies with the injected malicious instruction to execute an unauthorized financial trade, SAWM successfully intercepts the operation before execu￾tion by predicting harmful consequences. As shown in [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Representative examples from the MobileWorld training data. Left: Given the current screen and a candidate UI action, the model predicts the next UI state, assesses potential risks, and outputs a safety label. Right: The model answers structural and visual questions about the next UI state. The ground-truth ”Future Screen” is included for visualization only and is hidden from the model during inference [P… view at source ↗
Figure 6
Figure 6. Figure 6: demonstrates the effectiveness of SAWM at the instruction level. Compared with the GUI agent Qwen3- VL in the direct mode (i.e., without guardrail), SAWM more reliably identifies unsafe intent embedded in the user request, such as attempts to access, expose, or misuse private information shown in the interface. While Qwen3-VL tends to focus on superficial task wording and may mistakenly treat such requests… view at source ↗
Figure 7
Figure 7. Figure 7: Action-level risk assessment case study on Web Navigation. Compared with strong GUI agent Gemini-3.1 (Direct), SAWM identifies that the proposed browsing action would lead to an illegal or malicious website and blocks the unsafe navigation before execution. User Instruction: Could you send the message to Cynthia, with a draft named 'Hiring Decision' that I have already written in the Joplin memo applicatio… view at source ↗
Figure 8
Figure 8. Figure 8: Action-level risk assessment case study on SMS. Compared with strong GUI agent GPT-5.1 (Direct), SAWM detects that the proposed action sequence would facilitate unsafe biased content in the SMS workflow and stops the risky operation in advance. 19 [PITH_FULL_IMAGE:figures/full_fig_p019_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Scaling GUI Agents with Visual State Transitions

    cs.AI 2026-07 conditional novelty 6.0

    A joint inverse-forward pretraining stage on visual screen transitions improves GUI-agent fine-tuning by 0.6–6.2 percentage points across three benchmarks.

Reference graph

Works this paper leans on

35 extracted references · 1 canonical work pages · cited by 1 Pith paper

  1. [1]

    Digirl: Training in-the-wild device-control agents with autonomous reinforcement learning

    Bai, H., Zhou, Y., Cemri, M., Pan, J., Suhr, A., Levine, S., and Kumar, A. Digirl: Training in-the-wild device-control agents with autonomous reinforcement learning. Advances in Neural Information Processing Systems, 2024

    2024

  2. [2]

    Constitutional ai: Harmlessness from ai feedback

    Bai, Y., Kadavath, S., Kundu, S., Askell, A., Kernion, J., Jones, A., Chen, A., Goldie, A., Mirhoseini, A., McKinnon, C., Chen, C., Olsson, C., Olah, C., Hernandez, D., Drain, D., Ganguli, D., Li, D., Tran-Johnson, E., Perez, E., Kerr, J., Mueller, J., Ladish, J., Landau, J., Ndousse, K., Lukosiute, K., Lovitt, L., Elhage, N., Schiefer, N., Joseph, N., Ma...

    Pith/arXiv arXiv 2022

  3. [3]

    Mobiledreamer: Generative sketch world model for gui agent

    Cao, Y., Zhong, Y., Zeng, Z., Zheng, L., Huang, J., Qiu, H., Shi, P., Mao, W., and Guanglu, W. Mobiledreamer: Generative sketch world model for gui agent. arXiv preprint arXiv:2601.04035, 2026

    arXiv 2026

  4. [4]

    T., Gwak, M., Song, G., Kim, J., Kim, S., Lee, D., and Yeo, J

    Chae, H., Kim, N., iunn Ong, K. T., Gwak, M., Song, G., Kim, J., Kim, S., Lee, D., and Yeo, J. Web agents with world models: Learning and leveraging environment dynamics in web navigation. In The Thirteenth International Conference on Learning Representations, 2025

    2025

  5. [5]

    The foundations of cost-sensitive learning

    Elkan, C. The foundations of cost-sensitive learning. Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence, 2001

    2001

  6. [6]

    Is your llm secretly a world model of the internet? model-based planning for web agents

    Gu, Y., Zhang, K., Ning, Y., Zheng, B., Gou, B., Xue, T., Chang, C., Srivastava, S., Xie, Y., Qi, P., Sun, H., and Su, Y. Is your llm secretly a world model of the internet? model-based planning for web agents. Transactions on Machine Learning Research, 2025

    2025

  7. [7]

    Mastering diverse control tasks through world models

    Hafner, D., Pasukonis, J., Ba, J., and Lillicrap, T. Mastering diverse control tasks through world models. Nature, pp.\ 1--7, 2025

    2025

  8. [8]

    Y., Lambert, N., Choi, Y., and Dziri, N

    Han, S., Rao, K., Ettinger, A., Jiang, L., Lin, B. Y., Lambert, N., Choi, Y., and Dziri, N. Wildguard: Open one-stop moderation tools for safety risks, jailbreaks, and refusals of llms. Advances in neural information processing systems, 2024

    2024

  9. [9]

    Llama guard 3 vision: Safeguarding human-ai image understanding conversations

    Inan, H., Upasani, K., Dong, J., et al. Llama guard 3 vision: Safeguarding human-ai image understanding conversations. arXiv preprint arXiv:2411.10414, 2024. URL https://arxiv.org/abs/2411.10414

    Pith/arXiv arXiv 2024

  10. [10]

    Rynnvla-001: Using human demonstrations to improve robot manipulation

    Jiang, Y., Huang, S., Xue, S., Zhao, Y., Cen, J., Leng, S., Li, K., Guo, J., Wang, K., Chen, M., et al. Rynnvla-001: Using human demonstrations to improve robot manipulation. arXiv preprint arXiv:2509.15212, 2025

    arXiv 2025

  11. [11]

    Polyguard: A multilingual safety moderation tool for 17 languages

    Kumar, P., Jain, D., Yerukola, A., Jiang, L., Beniwal, H., Hartvigsen, T., and Sap, M. Polyguard: A multilingual safety moderation tool for 17 languages. In Second Conference on Language Modeling, 2025

    2025

  12. [12]

    LeCun, Y. et al. A path towards autonomous machine intelligence version 0.9. 2, 2022-06-27. Open Review, 62 0 (1): 0 1--62, 2022

    2022

  13. [13]

    Verisafe agent: Safeguarding mobile gui agent via logic-based action verification

    Lee, J., Lee, D., Choi, C., Im, Y., Wi, J., Heo, K., Oh, S., Lee, S., and Shin, I. Verisafe agent: Safeguarding mobile gui agent via logic-based action verification. In Proceedings of the 31st Annual International Conference on Mobile Computing and Networking, 2025

    2025

  14. [14]

    S., Knox, W

    Lee, J., Hahm, D., Choi, J. S., Knox, W. B., and Lee, K. Mobilesafetybench: Evaluating safety of autonomous agents in mobile device control. In Proceedings of the AAAI Conference on Artificial Intelligence, 2026

    2026

  15. [15]

    Mobileworldbench: Towards semantic world modeling for mobile agents

    Li, S., Kallidromitis, K., Gokul, A., Kato, Y., Kozuka, K., and Grover, A. Mobileworldbench: Towards semantic world modeling for mobile agents. arXiv preprint arXiv:2512.14014, 2025

    arXiv 2025

  16. [16]

    Drive VLA -w0: World models amplify data scaling law in autonomous driving

    Li, Y., Shang, S., Liu, W., Zhan, B., Wang, H., Wang, Y., Chen, Y., Wang, X., AnYasong, Tang, C., Hou, L., Fan, L., and Zhang, Z. Drive VLA -w0: World models amplify data scaling law in autonomous driving. In The Fourteenth International Conference on Learning Representations, 2026

    2026

  17. [17]

    Sft doesn't always hurt general capabilities: Revisiting domain-specific fine-tuning in llms

    Lin, J., Wang, Z., Qian, K., Wang, T., Srinivasan, A., Zeng, H., Jiao, R., Zhou, X., Gesi, J., Wang, D., et al. Sft doesn't always hurt general capabilities: Revisiting domain-specific fine-tuning in llms. arXiv preprint arXiv:2509.20758, 2025

    arXiv 2025

  18. [18]

    L., Sun, J., Wang, J., Gao, J., Shan, J., Liu, K., Zhang, S., Yao, S., Cheng, S., Yao, W., Zhao, W., Liu, X., Liu, X., Chen, X., Yang, X., et al

    Liu, X., Qin, B., Liang, D., Dong, G., Lai, H., Zhang, H., Zhao, H., Iong, I. L., Sun, J., Wang, J., Gao, J., Shan, J., Liu, K., Zhang, S., Yao, S., Cheng, S., Yao, W., Zhao, W., Liu, X., Liu, X., Chen, X., Yang, X., et al. Autoglm: Autonomous foundation agents for guis. arXiv preprint arXiv:2411.00820, 2024. URL https://arxiv.org/abs/2411.00820

    Pith/arXiv arXiv 2024

  19. [19]

    Vimo: A generative visual GUI world model for app agents

    Luo, D., Tang, B., Li, K., Papoudakis, G., Song, J., Gong, S., HAO, J., Wang, J., and Shao, K. Vimo: A generative visual GUI world model for app agents. In The Fourteenth International Conference on Learning Representations, 2026

    2026

  20. [20]

    Lutfi Eren Erdogan, Chuyi Shang, A. G. and Ijju, S. safe-guard-prompt-injection, 2024. URL https://huggingface.co/datasets/xTRam1/safe-guard-prompt-injection

    2024

  21. [21]

    Discovering and achieving goals via world models

    Mendonca, R., Rybkin, O., Daniilidis, K., Hafner, D., and Pathak, D. Discovering and achieving goals via world models. In Advances in Neural Information Processing Systems. Curran Associates, Inc., 2021

    2021

  22. [22]

    Llama 3.1 nemotron safety guard 8b

    NVIDIA . Llama 3.1 nemotron safety guard 8b. https://huggingface.co/nvidia/Llama-3.1-Nemotron-Safety-Guard-8B-v3, 2024. Accessed: 2026-03-09

    2024

  23. [23]

    Ui-tars: Pioneering automated gui interaction with native agents

    Qin, Y., Ye, Y., Fang, J., Wang, H., Liang, S., Tian, S., Zhang, J., Li, J., Li, Y., Huang, S., Zhong, W., Li, K., Yang, J., Miao, Y., Lin, W., Liu, L., Jiang, X., Ma, Q., Li, J., Xiao, X., Cai, K., Li, C., Zheng, Y., Jin, C., Li, C., Zhou, X., Wang, M., Chen, H., Li, Z., Yang, H., Liu, H., Lin, F., Peng, T., Liu, X., and Shi, G. Ui-tars: Pioneering autom...

    Pith/arXiv arXiv 2025

  24. [24]

    Qwen3guard-stream-8b

    Qwen Team . Qwen3guard-stream-8b. https://huggingface.co/Qwen/Qwen3Guard-Stream-8B, 2025. Model card and benchmark details, Accessed: 2026-03-23

    2025

  25. [25]

    Neural OS : Towards simulating operating systems via neural generative models

    Rivard, L., Sun, S., Guo, H., Chen, W., and Deng, Y. Neural OS : Towards simulating operating systems via neural generative models. In The Fourteenth International Conference on Learning Representations, 2026

    2026

  26. [26]

    Os-sentinel: Towards safety-enhanced mobile gui agents via hybrid validation in realistic workflows

    Sun, Q., Li, M., Liu, Z., Xie, Z., Xu, F., Yin, Z., Cheng, K., Li, Z., Ding, Z., Liu, Q., Wu, Z., Zhang, Z., Kao, B., and Kong, L. Os-sentinel: Towards safety-enhanced mobile gui agents via hybrid validation in realistic workflows. arXiv preprint arXiv:2510.24411, 2025

    Pith/arXiv arXiv 2025

  27. [27]

    Magicgui: A foundational mobile gui agent with scalable data pipeline and reinforcement fine-tuning

    Tang, Y., Dong, Y., Huang, Z., et al. Magicgui: A foundational mobile gui agent with scalable data pipeline and reinforcement fine-tuning. arXiv preprint arXiv:2508.03700, 2025. doi:10.48550/arXiv.2508.03700. URL https://arxiv.org/abs/2508.03700

    Pith/arXiv arXiv doi:10.48550/arxiv.2508.03700 2025

  28. [28]

    Wang, Q., Fan, Y., and Wang, X. E. Safeground: Know when to trust GUI grounding models via uncertainty calibrations. In Agentic AI in the Wild: From Hallucinations to Reliable Autonomy, 2026

    2026

  29. [29]

    Hi-agent: Hierarchical vision-language agents for mobile device control

    Wu, Z., Lu, H., Xing, J., et al. Hi-agent: Hierarchical vision-language agents for mobile device control. arXiv preprint arXiv:2510.14388, 2025. doi:10.48550/arXiv.2510.14388. URL https://arxiv.org/abs/2510.14388

    work page doi:10.48550/arxiv.2510.14388 2025

  30. [30]

    Webworld: A large-scale world model for web agent training

    Xiao, Z., Tu, J., Zou, C., Zuo, Y., Li, Z., Wang, P., Yu, B., Huang, F., Lin, J., and Liu, Z. Webworld: A large-scale world model for web agent training. arXiv preprint arXiv:2602.14721, 2026

    arXiv 2026

  31. [31]

    Yang, A. et al. Qwen3-vl technical report. arXiv preprint arXiv:2511.21631, 2025. URL https://arxiv.org/abs/2511.21631

    Pith/arXiv arXiv 2025

  32. [32]

    Dream VLA : A vision-language-action model dreamed with comprehensive world knowledge

    Zhang, W., Liu, H., Qi, Z., Wang, Y., Yu, X., Zhang, J., Dong, R., He, J., Wang, H., Zhang, Z., Yi, L., Zeng, W., and Jin, X. Dream VLA : A vision-language-action model dreamed with comprehensive world knowledge. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025 a

    2025

  33. [33]

    Agent-safetybench: Evaluating the safety of llm agents

    Zhang, Z., Cui, S., Lu, Y., Zhou, J., Yang, J., Wang, H., and Huang, M. Agent-safetybench: Evaluating the safety of llm agents. arXiv preprint arXiv:2412.14470, 2025 b . URL https://arxiv.org/abs/2412.14470

    Pith/arXiv arXiv 2025

  34. [34]

    Webguard: Building a generalizable guardrail for web agents

    Zheng, B., Liao, Z., Salisbury, S., et al. Webguard: Building a generalizable guardrail for web agents. arXiv preprint arXiv:2507.14293, 2025. URL https://arxiv.org/abs/2507.14293

    Pith/arXiv arXiv 2025

  35. [35]

    Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models

    Zhu, J., Wang, W., Chen, Z., Liu, Z., Ye, S., Gu, L., Tian, H., Duan, Y., Su, W., Shao, J., et al. Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504.10479, 2025

    Pith/arXiv arXiv 2025