Pith. sign in

REVIEW 4 major objections 5 minor 22 references

DragonCrawl shows that mobile end-to-end testing can be reframed as goal-directed reasoning: an LLM agent picks the next action from the live UI, a vision model verifies the result by looking at a screenshot, and the pair serves as a per-co

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 00:28 UTC pith:TNT3H3DZ

load-bearing objection Credible, unusually candid industry report repositioning LLM-based E2E testing as a per-commit regression gate; architecture and ablation are worth reading, but headline pass rates rest on the system's own visual oracle, so treat the numbers as directional. the 4 major comments →

arxiv 2607.28750 v1 pith:TNT3H3DZ submitted 2026-07-30 cs.SE cs.AI

DragonCrawl: A Generative, Intent-Based Framework for Scalable Mobile End-to-End Testing

Sowjanya Puligadda , Mengdie Zhang , Ali Zamani , Dhruva Dixith Kurra , Eric Chen , Juan Marcano This is my paper
classification cs.SE cs.AI
keywords mobile testingend-to-end testingLLM-based testingregression testingmultimodal assertionstool callingCI/CD quality gategenerative testing
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that mobile end-to-end testing should answer a different question than exploratory testing asks: not "what can we discover?" but "does this code change break a critical flow?" DragonCrawl is a production system built around that question. It replaces hardcoded element locators and similarity-matched reference screens with a generative reasoning loop: given the flow's intent, the current screen, and the action history, an LLM picks the next action, and a multimodal vision model verifies the end state from a screenshot. The paper reports that this lifts pass rates from 80–82% with the embedding-based V1 to 91.6% on iOS and 92.2% on Android across 1,013 tests, cuts flow onboarding from 96–120 hours to under 4, and reduces test maintenance from an estimated 30–40% of engineering time to about 5%. If the results hold, LLM-based testing moves from a crash-finding novelty to a practical deployment gate that blocks bad commits.

Core claim

The central claim is that framing test execution as intent-based reasoning instead of nearest-neighbor screen matching yields a qualitative jump in reliability and coverage. V1 asked "which past screen is most similar?" and failed on complex flows; V2 asks "given my intent, the current UI snapshot, and what I have done so far, what is the best next action?" At each decision point the system sends a canonized view hierarchy, the flow's natural-language intent, the full action history, and the list of interactable elements to an LLM that returns the single next action in structured JSON. Completion is judged not by resource IDs but by sending a base64-encoded screenshot with a yes/no assertion

What carries the argument

The load-bearing mechanism is the intent-based reasoning loop: a context package containing the canonized screen state (a raw 144-node view hierarchy is compressed to 51 executable actions), the flow intent, action history, and available actions is sent to an LLM service that selects one action; a multimodal assertion endpoint evaluates screenshots against natural-language conditions; and a tool-calling layer performs backend state transitions. The component doing the heaviest work is the shared model used both for navigation and for verification — a design that gives the system its low maintenance burden but also concentrates authority in one oracle. Prompt constraints (valid JSON, action s

Load-bearing premise

The pass/fail verdicts are produced by the same model family that executes the tests, and the paper supports its claim of near-perfect visual-checking accuracy by citing its own prior work rather than an independent evaluation; if that oracle systematically approves screens a human QA would reject, the reported pass rates and the assertion that red builds reflect genuine regressions collapse.

What would settle it

Take a random sample of test-execution screenshots paired with their assertions, have several human testers independently label each as pass or fail, and measure agreement with the model's verdicts; materially lower than near-perfect agreement would undermine the regression gate's blocking decisions. A complementary check: introduce an externally verified UI regression into a staging build and see whether the gate blocks the commit.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Complex flows that were previously un-automatable become testable: flows with conditional branching and repeated actions went from 0% pass rate under V1 to 89% under V2, and the 61 core flows sustain near-99% pass rates.
  • Test authoring shifts from scripting to specification: a flow is defined as intent plus assertions plus optional backend tool templates, cutting onboarding from days to hours and letting coverage grow from 48 to 1,013 tests.
  • Maintenance effort drops to roughly 5% of engineering time, so red builds regain signal; the paper attributes faster bug resolution (median 30.4 hours vs. ~5 days) to automatic ticket filing with zero reassignments.
  • The system can act as a real deployment gate: low-priority assertion mismatches are tracked but non-blocking, meaning policy—not model error—decides what blocks a commit.
  • The approach is not self-contained: 234 of the 1,013 flows (23.1%) require at least one backend tool call, and the infrastructure to support that is a prerequisite for the reported coverage.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The most consequential untested risk is circularity: the same model family that navigates the app also certifies whether the end state is correct, so a shared visual blind spot would inflate both the action-prediction numbers and the pass rates; an independent audit against human-labeled screenshots would put the gate on firmer ground.
  • The architecture is a reusable template for any high-stakes UI domain where a view hierarchy or accessibility tree exists—web, desktop, embedded—but the true bottleneck is the assertion oracle, not the navigation.
  • The paper's own breakdown suggests the aggregate 91.6/92.2% understates the claim that matters: the 61 core flows run at near-99%, with the gap coming from lower-priority, less-hardened flows. If the oracle is sound, the headline number is the floor, not the ceiling.
  • A concrete extension the paper does not report: inject known UI regressions into a staging build and measure whether the gate blocks them, which would quantify detection sensitivity directly.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper reports on DragonCrawl, Uber's LLM-based mobile E2E regression testing framework, which evolved from embedding-based similarity matching (V1) to generative intent-based reasoning (V2). The central claims are that V2 achieves 91.6% (iOS) and 92.2% (Android) pass rates across 1,013 tests running in CI/CD, reduces test onboarding from 96–120 hours to under 4 hours, and saves an estimated 27 developer-years. The architecture uses GPT-4o for next-action selection, multimodal visual assertions for end-state and mid-state verification, and tool calling for backend state transitions. The evaluation covers action prediction precision, pass-rate analysis, an ablation study, latency, cost optimization, and operational experience.

Significance. If the empirical claims are trustworthy, this is a meaningful industrial contribution: it demonstrates that an LLM-driven agent can act as a per-commit regression gate at production scale, and it describes architectural and operational lessons that are valuable to the software testing community. The paper is unusually candid about limitations (external model dependency, cost, nondeterminism, single-company scope), and the ablation study with repeated runs is a useful internal check. However, the central measurement chain is self-referential: the system's own visual assertions define pass/fail, and no independent oracle or human-audited sample is provided. The headline pass rates and the classification of non-passing runs as 'genuine regressions' therefore remain unvalidated as measurements of product quality.

major comments (4)
  1. [§5.2, with §3.2] The pass/fail verdict is system-determined: a test passes when GPT-4o's /assert_flow_end and /verify_mid_state endpoints answer 'yes' to natural-language assertions. The paper justifies the reliability of this oracle by citing prior work [11] for 'near-perfect accuracy,' but no evidence from that work is presented here, and no independent human-labeled audit is reported. Because the headline pass rates (91.6%/92.2%) and the claim that non-passing runs correspond to real regressions depend entirely on this oracle, the central empirical claim is not yet established. I recommend adding a random sample of test verdicts reviewed by human QA, reporting inter-rater agreement, or validating assertions against backend state / resource identifiers.
  2. [§5.2, §6.5, §6.7] The categorization of non-passing runs into infrastructure failures (~1%), agent errors (~0.5%), and 'genuine regressions (the remainder)' is not independently confirmed. The 'DragonCrawl Signal' claim that red builds almost always indicate legitimate regressions is supported only by the same system's self-assessed verdicts, not by external confirmation such as bug reports, developer acknowledgment, or rollback data. Please provide the number of runs classified as regressions and how many were subsequently confirmed by humans.
  3. [Table 2 and §5.2] The V2 pass rates are point estimates with no denominator, no time window, no variance, and no confidence intervals. Given the acknowledged nondeterminism of LLM agents, this is insufficient for a reliability claim. Please report the number of executions behind each rate (e.g., unique tests vs. per-commit runs), the observation period, and error bars or raw counts. Also clarify whether the V1 vs. V2 comparison is on the same test set or different flows; the current text suggests different sets, which would weaken the claimed improvement.
  4. [§6.4 and abstract] The estimated '27 developer-years saved' is built on assumed baselines of 16 developer-hours per test authoring and 20 hours per test maintenance, with H_year = 1,380 productive hours. These baselines are not justified by data or cited sources. Since this number appears in the abstract and is used as an ROI argument, please provide evidence for the baselines, a sensitivity analysis, or qualify the estimate more explicitly as an order-of-magnitude calculation.
minor comments (5)
  1. [Abstract vs. §6.1] The abstract states onboarding goes from 96–120 hours to 'under 4 hours,' but §6.1 reports a median of 2.5 hours, p75 = 3 hours, and p90 = 4 hours. The abstract should say 'median under 3 hours' or report the distribution more accurately.
  2. [§5.4 and Table 4] Table 4 lists V1 complex-flow latency as 'N/A,' but the text says 'V1 had a 0% pass rate on complex flows even after 3 retries.' Clarify whether complex flows were attempted in V1 and, if so, what latency was observed.
  3. [§5.2] The breakdown of non-passing runs (infrastructure ~1%, agent ~0.5%, genuine regressions 'the remainder') is hard to reconcile with the 91.6%/92.2% pass rates unless the remainder is about 6–8% of all runs. Please make the relationship explicit and provide the actual counts.
  4. [Reference [11]] The claim of 'near-perfect' visual QA accuracy rests entirely on reference [11]. Since this is the linchpin of the oracle reliability argument, give a brief description of its methodology or findings in this paper.
  5. [§6.5] The 'feature velocity increased by approximately 15%' claim is not accompanied by any measurement method or data. Either provide supporting evidence or describe it as a qualitative observation.

Circularity Check

2 steps flagged

Pass/fail verdicts are produced by the same GPT-4o oracle used to compute the headline pass rates; the only cited support for oracle accuracy is the authors' own prior work [11].

specific steps
  1. self definitional [Section 5.2 (Pass Rate Analysis), first paragraph; see also Section 3.2 Multimodal End-State Detection]
    "A test passes when DragonCrawl reaches the flow's defined end state (e.g. the trip rating screen) and every visual assertion specified by the test owner is satisfied (e.g., "does the screen show the driver's license plate?"). The verdict is system-determined, with the end state and each true/false assertion evaluated from the captured screens."

    The reported pass rates are computed from verdicts issued by the same GPT-4o endpoints (/assert_flow_end, /verify_mid_state) that the agent uses to determine success. The paper provides no independent human-labeled or externally validated ground-truth set for these visual verdicts. Therefore the 91.6% (iOS) and 92.2% (Android) figures measure the system agreeing with its own judgments rather than objectively verified flow completion; pass/fail is defined as the LLM's yes/no answer, so the evaluation is circular with respect to the oracle.

  2. self citation load bearing [Section 5.2, sentence citing [11]; reference [11]]
    "The underlying visual question answering is highly reliable, with near-perfect accuracy reported in our prior work [11], so blocking is governed by policy rather than model error: a commit is blocked when the end state is not reached or a high-priority assertion fails..."

    The only support offered for the oracle's near-perfect reliability is the authors' own prior paper [11], whose author list overlaps with this paper (Marcano, Zhang, Zamani, Puligadda). No accuracy figure, dataset, or independent reproducibility check appears in this paper. This citation is load-bearing: it is what lets the paper attribute residual non-passing runs to 'genuine regressions' and claim red builds are real signals. Without independent support for [11]'s near-perfect accuracy, the headline pass-rate claim and the regression-gate interpretation collapse.

full rationale

The paper is not mathematically circular: the V1-to-V2 pass-rate comparison, latency measurements, ablation results, and operator-reported savings are empirical measurements. However, the central evaluation chain is circular in the measurement-oracle sense. A test 'passes' when GPT-4o's /assert_flow_end and /verify_mid_state say so; the same model's self-verdicts are then used to compute the 91.6%/92.2% pass rates, and the oracle's reliability is justified by citing the authors' own prior work [11]. The Limitations section concedes that visual assertions 'can produce false verdicts,' but gives no rate. Thus the headline regression-gate claim rests on a self-referential loop plus a self-citation, not on an independently audited ground truth. Score 6: partial circularity, because the operational/bug-detection evidence and external comparison provide independent content, but the headline pass-rate metric and the 'genuine regressions' classification reduce to the system's own judgment.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 0 invented entities

No novel physical entities are introduced; the software components (DragonCrawl V2, Golden Dataset, tool templates) are engineering constructs with internal evidence, not independent falsifiable entities. The central claim rests on the free parameters and domain assumptions listed above, especially the self-referential visual QA oracle and the assumed effort baselines.

free parameters (6)
  • authoring baseline 16 developer-hours per test = 16 h/test
    Assumed in §6.4 to estimate 12 developer-years of authoring savings; no measurement from a traditional scripting team is provided.
  • maintenance baseline 20 hours per test = 20 h/test
    Assumed in §6.4 to estimate 15 developer-years of maintenance savings; no empirical baseline is shown.
  • productive developer-hours per year = 1,380 h/yr
    Standard labor assumption used to convert hours to developer-years in §6.4.
  • Golden Dataset accuracy alert threshold = 95%
    Chosen alert threshold in §4.2; no justification for why 95% is the correct cutoff.
  • token cost decline rate = 10x every 6-12 months
    Industry trend assumption in §2.2 used to justify the business case for switching to vendor LLMs.
  • near-perfect visual QA accuracy = not quantified
    Invoked in §5.2 via self-cited [11]; treated as high but no error rates or validation data are given.
axioms (5)
  • domain assumption GPT-4o multimodal visual assertions provide an accurate oracle for end-state and mid-state success.
    Section 5.2: passes are defined by LLM answers to visual questions; accuracy is asserted via self-cited prior work [11], with no external benchmark.
  • domain assumption The Golden Dataset of ~10,000 validated interaction sequences is correct ground truth for action correctness.
    Section 4.2 uses it to compute precision@k and to flag divergences; its own validity is not independently established.
  • domain assumption CI environment, device farms, and test accounts are stable enough that pass-rate differences reflect agent quality rather than infrastructure.
    Section 5.2 attributes ~1% to infrastructure and ~0.5% to agent errors but does not report how these are measured or how retries affect the pass-rate denominator.
  • domain assumption External task-completion rates from Table 3 are meaningful reference points despite different settings.
    Section 5.3 itself says they are 'context, not a like-for-like result'; the comparison therefore cannot be used to validate DragonCrawl's absolute numbers.
  • standard math Standard arithmetic for combinatorial test-count estimates (61 flows x 150 cities x 60 languages = 549,000).
    Section 2.1 simple multiplication; no substantive assumption.

pith-pipeline@v1.3.0-alltime-deepseek · 15263 in / 12780 out tokens · 127337 ms · 2026-08-03T00:28:47.964269+00:00 · methodology

0 comments
read the original abstract

As mobile applications grow in complexity, traditional End-to-End (E2E) testing frameworks struggle with UI volatility, maintenance overhead, and cross-platform scalability. This paper presents DragonCrawl, an AI-driven mobile testing system for continuous regression testing that has evolved from embedding-based similarity matching to generative intent-based reasoning using large language models. Unlike prior LLM-based testing research focused on exploratory testing and crash detection, DragonCrawl validates specific user flows on every code change, blocking commits that break critical functionality. By leveraging GPT-4o's multimodal capabilities, DragonCrawl achieves 91.6% pass rate on iOS and 92.2% on Android across 1,013 automated tests running continuously in CI/CD pipelines. The system reduces test onboarding time from 96-120 hours to under 4 hours and has saved an estimated 27 developer years in test maintenance effort. We present the architectural evolution from V1 (semantic embedding matching) to V2 (generative intent-based reasoning), discuss implementation challenges including token explosion and memory constraints, and report operational experience from production deployment. The integration of multimodal vision for end-state detection and tool calling for backend state transitions enables comprehensive regression testing that bridges UI interactions with system state. Our results demonstrate that AI-driven testing can maintain stability while eliminating the brittleness of traditional automated tests, enabling continuous quality assurance at scale.

Figures

Figures reproduced from arXiv: 2607.28750 by Ali Zamani, Dhruva Dixith Kurra, Eric Chen, Juan Marcano, Mengdie Zhang, Sowjanya Puligadda.

Figure 1
Figure 1. Figure 1: Architecture of DragonCrawl. The core of DragonCrawl V2 is an intent-based reasoning engine that generates next actions based on rich context. For each decision point during test execution, the system constructs a context package containing: • Current UI Snapshot: A textual representation of the XML view hierarchy of the screen, including all interactive ele￾ments, their properties, and their relationships… view at source ↗
Figure 2
Figure 2. Figure 2: Dataflow of Visual Assertions. when layouts change. DragonCrawl V2 uses multimodal vision to detect test completion. At each step, the system captures a screenshot and encodes it as base64. When the model believes the test goal may be achieved, it triggers an end-state detection query. The system sends both the screenshot and a natural language question to GPT-4o: "Based on this image, has the user success… view at source ↗
Figure 3
Figure 3. Figure 3: Context enhancement from view hierarchy. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Context-Conditioned Backend Orchestration in [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Coverage growth over time [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Distribution of high priority bug resolutions. [PITH_FULL_IMAGE:figures/full_fig_p010_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

22 extracted references · 7 canonical work pages · 4 internal anchors

  1. [1]

    Nadia Alshahwan, Xinbo Gao, Mark Harman, Yue Jia, Ke Mao, Alexander Mols, Taijin Tei, and Ilya Zorin. 2018. Deploying Search Based Software Engineering with Sapienz at Facebook. InProceedings of the 10th International Symposium on Search Based Software Engineering (SSBSE). Springer, Cham, Switzerland, 3–45. doi:10.1007/978-3-319-99241-9_1

    work page doi:10.1007/978-3-319-99241-9_1 2018

  2. [2]

    Riccardo Coppola, Luca Ardito, Maurizio Morisio, and Marco Torchiano. 2020. Mobile Testing: New Challenges and Perceived Difficulties from Developers of the Italian Industry.IT Professional22, 5 (2020), 32–39. doi:10.1109/MITP.2019. 2942810

    work page doi:10.1109/mitp.2019 2020

  3. [3]

    Google. 2012. UI/Application Exerciser Monkey. Android Developers. https: //developer.android.com/studio/test/monkey

    2012

  4. [4]

    Towards Test Generation from Task Description for Mobile Testing with Multi-modal Reasoning

    Hieu Huynh, Hai Phung, Hung Pham, Tien N. Nguyen, and Vu Nguyen. 2025. Towards Test Generation from Task Description for Mobile Testing with Multi- Modal Reasoning. arXiv:2504.15917. doi:10.48550/arXiv.2504.15917

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2504.15917 2025

  5. [5]

    Yuanchun Li, Ziyue Yang, Yao Guo, and Xiangqun Chen. 2017. DroidBot: A Lightweight UI-Guided Test Input Generator for Android. InProceedings of the 39th International Conference on Software Engineering Companion (ICSE-C). IEEE, Buenos Aires, Argentina, 23–26. doi:10.1109/ICSE-C.2017.8

    doi:10.1109/icse-c.2017.8 2017

  6. [6]

    Yuanchun Li, Ziyue Yang, Yao Guo, and Xiangqun Chen. 2019. Humanoid: A Deep Learning-Based Approach to Automated Black-box Android App Testing. In Proceedings of the 34th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, San Diego, CA, USA, 1070–1073. doi:10.1109/ASE.2019. 00104

    work page doi:10.1109/ase.2019 2019

  7. [7]

    Zhe Liu, Chunyang Chen, Junjie Wang, Xing Che, Yuekai Huang, Jun Hu, and Qing Wang. 2022. Fill in the Blank: Context-Aware Automated Text Input Gener- ation for Mobile GUI Testing. arXiv:2212.04732. doi:10.48550/arXiv.2212.04732

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2212.04732 2022

  8. [8]

    Zhe Liu, Chunyang Chen, Junjie Wang, Mengzhuo Chen, Boyu Wu, Xing Che, Dandan Wang, and Qing Wang. 2023. Chatting with GPT-3 for Zero-Shot Human- Like Mobile Automated GUI Testing. arXiv:2305.09434. doi:10.48550/arXiv.2305. 09434

    Pith/arXiv arXiv doi:10.48550/arxiv.2305.09434 2023

  9. [9]

    Zhe Liu, Chunyang Chen, Junjie Wang, Yuekai Huang, Jun Hu, and Qing Wang

  10. [10]

    Ke Mao, Mark Harman, and Yue Jia. 2016. Sapienz: Multi-objective Automated Testing for Android Applications. InProceedings of the 25th International Sympo- sium on Software Testing and Analysis (ISSTA). ACM, New York, NY, USA, 94–105. doi:10.1145/2931037.2931054

    arXiv 2016

  11. [11]

    Juan Marcano, Ashish Samant, Kai Song, Lingchao Chen, Kaelan Mikowicz, Tim Smyth, Mengdie Zhang, Ali Zamani, Arturo Bravo Rovirosa, Sowjanya Puligadda, Srikanth Prodduturi, and Mayank Bansal. 2026. Scaling Mobile Chaos Testing with AI-Driven Test Execution. InProceedings of the 48th International Conference on Software Engineering: Software Engineering in...

    arXiv 2026

  12. [12]

    Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al

    OpenAI, Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. 2024. GPT-4o System Card. arXiv:2410.21276. doi:10.48550/arXiv.2410.21276

    Pith/arXiv arXiv doi:10.48550/arxiv.2410.21276 2024

  13. [13]

    Minxue Pan, An Huang, Guoxin Wang, Tian Zhang, and Xuandong Li. 2020. Reinforcement Learning Based Curiosity-Driven Testing of Android Applications. InProceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA). ACM, New York, NY, USA, 153–164. doi:10.1145/ 3395363.3397354

    arXiv 2020

  14. [14]

    Chao Peng, Zhengwei Lv, Jiarong Fu, Jiayuan Liang, Zhao Zhang, Ajitha Rajan, and Ping Yang. 2023. Hawkeye: Change-Targeted Testing for Android Apps Based on Deep Reinforcement Learning. arXiv:2309.01519. doi:10.48550/arXiv. 2309.01519

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2309.01519 2023

  15. [15]

    Dezhi Ran, Zongyang Li, Chenxu Liu, Wenyu Wang, Weizhi Meng, Xionglin Wu, Hui Jin, Jing Cui, Xing Tang, and Tao Xie. 2022. Automated Visual Testing for Mobile Apps in an Industrial Setting. InProceedings of the 44th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP). ACM, New York, NY, USA, 55–64. doi:10.1145/35...

    arXiv 2022

  16. [16]

    Dezhi Ran, Hao Wang, Zihe Song, Mengzhou Wu, Yuan Cao, Ying Zhang, Wei Yang, and Tao Xie. 2024. Guardian: A Runtime Framework for LLM-Based UI Exploration. InProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA). ACM, New York, NY, USA, 958–970. doi:10.1145/3650212.3680334

    arXiv 2024

  17. [17]

    Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tie-Yan Liu. 2020. MPNet: Masked and Permuted Pre-Training for Language Understanding. arXiv:2004.09297. doi:10.48550/arXiv.2004.09297

    Pith/arXiv arXiv doi:10.48550/arxiv.2004.09297 2020

  18. [18]

    Ting Su, Guozhu Meng, Yuting Chen, Ke Wu, Weiming Yang, Yao Yao, Geguang Pu, Yang Liu, and Zhendong Su. 2017. Guided, Stochastic Model-Based GUI Testing of Android Apps. InProceedings of the 11th Joint Meeting on Foundations of Software Engineering (ESEC/FSE). ACM, New York, NY, USA, 245–256. doi:10. 1145/3106237.3106298

    arXiv 2017

  19. [19]

    Hao Wen, Yuanchun Li, Guohong Liu, Shanhui Zhao, Tao Yu, Toby Jia-Jun Li, Shiqi Jiang, Yunhao Liu, Yaqin Zhang, and Yunxin Liu. 2024. AutoDroid: LLM-Powered Task Automation in Android. InProceedings of the 30th Annual International Conference on Mobile Computing and Networking (MobiCom). ACM, New York, NY, USA, 543–557. doi:10.1145/3636534.3649379

    arXiv 2024

  20. [20]

    Hao Wen, Hongming Wang, Jiaxuan Liu, and Yuanchun Li. 2023. DroidBot-GPT: GPT-Powered UI Automation for Android. arXiv:2304.07061. doi:10.48550/arXiv. 2304.07061

    Pith/arXiv arXiv doi:10.48550/arxiv.2304.07061 2023

  21. [21]

    Juyeon Yoon, Robert Feldt, and Shin Yoo. 2023. Autonomous Large Language Model Agents Enabling Intent-Driven Mobile GUI Testing. arXiv:2311.08649. doi:10.48550/arXiv.2311.08649 Received 2026-04-21; accepted 2026-07-01

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2311.08649 2023

  22. [2020]

    InProceed- ings of the 35th IEEE/ACM International Conference on Automated Software Engi- neering (ASE)

    Owl Eyes: Spotting UI Display Issues via Visual Understanding. InProceed- ings of the 35th IEEE/ACM International Conference on Automated Software Engi- neering (ASE). ACM, New York, NY, USA, 398–409. doi:10.1145/3324884.3416547

    arXiv