Pith. sign in

REVIEW 3 major objections 4 minor 69 references

LogiDroid shows automated functional testing of mobile apps can be driven by retrieving and fusing business logic from similar apps, then adapting it to the live GUI; on two standard benchmarks it reaches 40–65% success.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 20:01 UTC pith:PCZ234S5

load-bearing objection A solid retrieval-augmented LLM agent system for Android functional testing, with a real contribution in generating assertions, but the evaluation needs a cleanup before its headline numbers can be trusted. the 3 major comments →

arxiv 2602.24108 v2 pith:PCZ234S5 submitted 2026-02-27 cs.SE

LogiDroid: Individual Functional Test Generation via Business Logic Extraction and Adaptation

Yakun Zhang , Zihan Wang , Xinzhi Peng , Zihao Xie , Xiaodong Wang , Xutao Li , Dan Hao , Lu Zhang
show 1 more author
Yunming Ye
This is my paper
classification cs.SE
keywords functional testingGUI test generationlarge language modelsbusiness logic extractionknowledge retrievaltest case generationAndroid appsassertions
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that functional test generation for mobile apps — long stuck on crash hunting and event sequences — can be turned into a knowledge-reuse problem. LogiDroid retrieves similar test cases from a curated repository, fuses them into an app-agnostic 'core business logic', and then adapts that logic to the live GUI of the target app, producing test cases that carry verification assertions. If the reported results hold, the approach would let industrial teams generate individual functional tests at scale, something existing LLM-based testers, which emit events but no assertions, cannot do. The central claim is a 40% success rate on one benchmark and 65% on another, with the paper reporting large relative gains over the state-of-the-art baselines.

Core claim

LogiDroid is a two-stage pipeline. Stage 1 (Knowledge Retrieval and Fusion) builds a 294-case repository from 71 apps, retrieves the Top_sim=3 cases semantically closest to a new requirement, and fuses them into a short, implementation-agnostic 'business logic' — a list of event and assertion steps. Stage 2 (Context-Aware Test Generation) lets a Perception-Interaction Agent capture GUI state (screenshot plus widget text/resource-id/content-desc) and a Decision-Generation Agent select which logic step fits the current state within a sliding window of Step_num=2, generate concrete instructions, judge completion, and finally synthesize a test case including assertions. The paper reports that th

What carries the argument

The central mechanism is the 'core business logic': an implementation-agnostic list of event and assertion steps distilled by the Knowledge-Fusion Agent from retrieved similar test cases. The Decision-Generation Agent adapts it to the target app via a sliding-window step selection (window size 2), a differentiated instruction generator for events versus assertions, and a completion-judgment loop with a retry threshold; assertion generation uses a state-backtracking mechanism when a widget must be verified as absent. This division of labor — retrieve, fuse, select, adapt — is what converts vague requirements into executable, verifiable test cases.

Load-bearing premise

The reported success rates rest on the two volunteer engineers who also wrote the requirement descriptions judging that a generated test case 'successfully tests' the target functionality, with no independent oracle.

What would settle it

Run LogiDroid against a mutated version of an app and check whether the generated assertions catch the mutation. If tests the volunteers labeled 'successful' fail to detect a fault the ground-truth test detects, the success-rate is inflated; if they catch all such faults, the label is credible.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • LogiDroid is the only approach among the three compared that generates complete test cases with assertions, not just event sequences; if the success rates are real, automated functional verification becomes feasible.
  • Cross-application knowledge reuse is a viable path: 294 test cases across 71 apps suffice to reach 40–65% success on 28 target apps and 190 requirements.
  • The architecture is model-agnostic: switching the underlying LLM keeps success at 60–75% on a subset, so the benefit comes from the pipeline, not the specific model.
  • The cost is modest — about 6 minutes and 32k tokens per test case — placing LogiDroid between the two baselines, so the accuracy gain does not imply prohibitive resource use.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper's own numbers are inconsistent: the abstract says a 25% improvement on the first benchmark while the evaluation section says 48%; unless reconciled, the exact margin over the baselines should be read as approximate.
  • The hyperparameters (Step_num=2, Top_sim=3) were tuned on 20% of the same test cases used for evaluation, so the reported rates may be optimistic; a strict held-out tuning protocol is an extension worth testing.
  • Perfect-rate is low (20–41%), but success-rate counts divergent valid sequences; many of the 'unsuccessful' tests may be acceptable with minor edits, suggesting the practical value is higher than the perfect-rate implies.
  • A testable extension: measure how success-rate degrades when Top_sim is reduced or when no same-category test exists in the repository — this would show whether retrieval, rather than generation, is the real bottleneck.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes LogiDroid, a two-stage LLM-agent framework for generating individual functional test cases for Android apps. Stage 1 (Knowledge Retrieval and Fusion) uses a Semantic-Retrieval Agent and a Knowledge-Fusion Agent to retrieve similar historical test cases from a constructed dataset and distill them into structured business logic. Stage 2 (Context-Aware Test Generation) uses a Perception-Interaction Agent and a Decision-Generation Agent to incrementally produce executable event/assertion sequences adapted to the live GUI state. The evaluation reports results on the FrUITeR and Lin datasets (28 apps, 190 requirement descriptions), comparing against AutoDroid and AppAgent. The headline claims are a 40% success-rate on FrUITeR (claimed as 48% improvement over baselines in the introduction, but 25% in the abstract) and a 65% success-rate on the Lin dataset (55% improvement), with LogiDroid being the only approach that generates verification assertions.

Significance. If the reported numbers are trustworthy, this is a meaningful advance: LogiDroid is the only method in the comparison that produces complete test cases with assertions, and its success-rate advantage over AutoDroid and AppAgent is substantial. The paper also has concrete strengths: it enforces a data-isolation protocol when retrieving from the constructed dataset (Section 4.2), it includes ablation and robustness experiments across two foundation models, and it promises to release the implementation and data. The central weakness is not the architecture but the evaluation protocol: hyperparameters are tuned on 20% of the evaluation test cases without evidence of holdout, the success labels are assigned by the same volunteers who wrote the requirement descriptions, and all performance numbers are single-run point estimates from a stochastic pipeline. These issues jointly make the current headline rates an optimistic upper bound rather than a clean estimate. The result remains plausible and worth publishing, but the evaluation must be made airtight before the central claim can be accepted.

major comments (3)
  1. [Section 4.1 (Implementation details) and Table 5] The hyperparameters Step_num and Top_sim are optimized 'based on 20% of the test cases' and then adopted for the final experiments, but the paper never states that this 20% was excluded from the final evaluation. If Table 5 includes the tuning subset, the reported success/perfect rates are selection-biased. On FrUITeR the headline improvement is 40% vs 27% (AutoDroid), i.e., a difference of about 8 successful cases out of 64; even a small validation/test gap could change the conclusion. Please clarify whether the tuning subset was held out, and if not, re-run the final evaluation on a disjoint test fold or report both validation and test performance.
  2. [Section 4.1 (Evaluation metrics, Success-rate)] The success-rate is decided by manual inspection performed by 'the two volunteers previously responsible for requirement drafting.' A high Fleiss kappa (0.93) only demonstrates that the two raters agree with each other; it does not demonstrate that their shared judgment is unbiased with respect to the system whose output they are rating. Since the success-rate is the metric that most distinguishes LogiDroid from the baselines, this is load-bearing. The authors should either use independent evaluators who did not author the requirement descriptions, adopt a pre-specified rubric for 'successfully test the target functionality,' or provide a blinded comparison where evaluators do not know whether a test case came from LogiDroid or a baseline.
  3. [Section 4.2 and Table 5 (RQ1)] All reported percentages are single-run point estimates from an LLM-based pipeline with stochastic sampling. No confidence intervals, multiple runs, seeds, or statistical significance tests are reported. For example, on FrUITeR, 40% vs 32% success-rate corresponds to about 5 successful cases out of 64; the binomial 95% intervals for these two proportions overlap considerably. The same issue affects the ablation (Table 6) and robustness (Table 7) numbers. The authors should run the pipeline multiple times (or at least report a significance test / effect size) to demonstrate that the differences are not noise.
minor comments (4)
  1. [Abstract vs. Section 1 / Section 4.2] The FrUITeR improvement is stated as 'over 25%' in the abstract and 'over 48%' in the introduction and RQ1 analysis. The two numbers correspond to different baselines (AppAgent vs AutoDroid), but this inconsistency will confuse readers. Please state the comparison baseline explicitly in each location.
  2. [Section 4.1, Eq. (1)] The notation 'Perfect-rate = Test_gt / Test_t' is ambiguous: the denominator should be the number of generated test cases, and the numerator should be the subset that matches ground truth. Please define the sets and counts explicitly.
  3. [Algorithm 2] The text refers to 'Line 6,' 'Line 17,' etc., but the rendered algorithm listing does not show line numbers. Please add visible line numbers or revise the references.
  4. [Section 5 (Threat to validity)] The internal-validity discussion does not mention the hyperparameter-tuning issue or the self-evaluation concern raised above. Please address these directly in the threats section; acknowledging them and explaining the mitigation would strengthen the paper.

Circularity Check

1 steps flagged

Reported success-rates are partly fitted: hyperparameters were tuned on 20% of the same test cases used to report the headline results.

specific steps
  1. fitted input called prediction [Section 4.1, Implementation details (hyperparameter tuning); Table 5]
    "Regarding parameter configuration, we optimize the two hyperparameters of LogiDroid through systematic preliminary experiments. For the sliding window size Step_num and the number of retrieved similar test cases Top_sim, we conduct validation experiments on a candidate set {1, 2, 3} based on 20% of the test cases. The results indicated that LogiDroid achieves optimal performance when Step_num = 2 and Top_sim = 3. Therefore, we adopt this configuration for the final experiments."

    The reported success-rates in Table 5 (40% on FrUITeR, 65% on Lin) are the headline evidence of effectiveness. Those same benchmark test cases were used, at least in part, to select Step_num and Top_sim: the configuration was chosen because it performed well on 20% of the test cases. The paper never states that this 20% was removed from the final evaluation, so the final success-rate is not an independent estimate; it is partly the same criterion used to pick the configuration. The claimed improvements over baselines are therefore partially an artifact of tuning rather than an out-of-sample prediction.

full rationale

LogiDroid is an empirical systems paper with no formal derivation chain, so the circularity question centers on whether the reported effectiveness numbers are independent of the choices that produced them. The main problem is the hyperparameter tuning: Section 4.1 says Step_num and Top_sim were optimized on 20% of the test cases, and then the paper says 'we adopt this configuration for the final experiments' without stating that those 20% were held out. If they were included, the headline success-rates are partly fitted values, and the comparison against AutoDroid/AppAgent is biased in LogiDroid's favor. The paper's data-isolation statement in Section 4.2 removes target-application cases from the retrieval dataset, which addresses direct retrieval shortcuts, but it does not address this tuning leakage. A second concern is that success-rate is judged by the same two volunteers who wrote the requirement descriptions; Fleiss' kappa 0.93 shows agreement but not independence from the test inputs. There are no load-bearing self-citations or uniqueness theorems, and the proposed pipeline itself is not definitionally circular. If the 20% tuning subset had been explicitly excluded from the final results, the evaluation would be much stronger and the score would drop to about 2-3. As written, the central success-rate claim is partially forced by the tuning procedure, giving score 6.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

The system assumes reusable business logic across similar apps, LLM summaries are accurate enough for retrieval, and GUI hierarchy/screenshot capture sufficient context. These are domain assumptions, not mathematical axioms. No new physical or formal entities are introduced.

free parameters (3)
  • Step_num (sliding window size) = 2
    Selected from candidate set {1,2,3} on 20% of the test cases; affects how many business-logic steps the Decision-Generation Agent considers at once.
  • Top_sim (number of retrieved test cases) = 3
    Selected from candidate set {1,2,3} on 20% of the test cases; controls how many similar cases are fused into business logic.
  • Maximum attempt threshold = not reported
    Algorithm 2 uses an attempt threshold to skip steps and prevent infinite loops; its value is not specified, so replication requires guessing.
axioms (4)
  • domain assumption Functionally similar mobile apps share reusable business logic and testing patterns.
    Section 1 states this observation as the motivation for knowledge reuse; if false, retrieval-based fusion would not help.
  • domain assumption LLM-generated functional summaries accurately represent test-case semantics for embedding-based retrieval.
    Section 3.1.1 uses LLM summaries as retrieval keys with no reported validation of summary correctness.
  • domain assumption GUI hierarchy plus screenshots provide sufficient information to map abstract business logic to concrete widgets.
    Section 3.2.1 assumes the three attributes text, content-desc, resource-id are enough for widget grounding.
  • domain assumption Evaluation data from FrUITeR/Lin is representative of industrial functional testing scenarios.
    Section 4.1 uses these datasets as ground truth; external validity depends on their representativeness.

pith-pipeline@v1.3.0-alltime-deepseek · 19979 in / 13049 out tokens · 114562 ms · 2026-08-02T20:01:57.924428+00:00 · methodology

0 comments
read the original abstract

Functional testing is essential for verifying that the business logic of mobile applications aligns with user requirements. Despite its importance, functional testing remains heavily dependent on manual effort due to two core challenges. First, acquiring and reusing business logic from unstructured requirements remains difficult, which hinders the understanding of specific functionalities. Second, a significant semantic gap exists when adapting business logic to the diverse GUI environments, which hinders the generation of test cases for specific mobile applications. To address the preceding challenges, we propose LogiDroid, a two-stage approach that generates individual functional test cases by extracting business logic and adapting it to target applications. First, in the Knowledge Retrieval and Fusion stage, two LLM-based agents are employed to construct a functional test dataset, retrieve relevant test cases, and extract structured business logic for the target functionality. Second, in the Context-Aware Test Generation stage, two other LLM-based agents jointly analyze the extracted business logic and the real time GUI environment to incrementally generate context adaptive functional test cases. This design allows LogiDroid to accurately understand application semantics and use domain expertise to generate complete test cases with verification assertions. We assess the effectiveness of LogiDroid using two widely-used datasets that cover 28 real-world applications and 190 functional requirements. Experimental results show that LogiDroid successfully tested 40% of functional requirements on the FrUITeR dataset (an improvement of over 25% compared to the state-of-the-art approaches) and 65% on the Lin dataset (an improvement of over 55% compared to the state-of-the-art approaches). These results demonstrate the significant effectiveness of LogiDroid in functional test generation.

Figures

Figures reproduced from arXiv: 2602.24108 by Dan Hao, Lu Zhang, Xiaodong Wang, Xinzhi Peng, Xutao Li, Yakun Zhang, Yunming Ye, Zihan Wang, Zihao Xie.

Figure 1
Figure 1. Figure 1: The test process of “Add and remove an item” [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of LogiDroid application in real-time to capture multimodal information, including GUI layouts and visual screenshots. By integrating the business logic acquired from the first stage with this multimodal context, the system employs a progressive decision￾making mechanism to decompose functional requirements into executable test sequences. Note that, unlike existing approaches [38, 62, 66] that onl… view at source ↗
Figure 3
Figure 3. Figure 3: Example of Decision-Generation Agent. of each step relative to the current GUI state by analyzing a limited set of possibilities. Second, the model performs reasoning according to the order of steps within the window to select the logic step that best matches the current context of the mobile application. Third, if no applicable step exists within the window, the model returns a specific identifier “(-1)",… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

69 extracted references · 14 linked inside Pith

  1. [1]

    https://abcnews.go.com/

    2025.ABC News - Breaking News, Latest News and Videos. https://abcnews.go.com/

    2025

  2. [2]

    https://huggingface.co/BAAI/bge-base-en-v1.5

    2025.Bge-base-en-v1.5. https://huggingface.co/BAAI/bge-base-en-v1.5

    2025

  3. [3]

    https://chat.openai.com/

    2025.ChatGPT. https://chat.openai.com/

    2025

  4. [4]

    https://f-droid.org/

    2025.F-Droid: free and open source Android app repository. https://f-droid.org/

    2025

  5. [5]

    https://play.google.com/store/apps/details?id=org.mozilla.firefox

    2025.Firefox browser: fast, private and secure Web browser. https://play.google.com/store/apps/details?id=org.mozilla.firefox

    2025

  6. [6]

    2025.FrUITeR dataset.https://felicitia.github.io/FrUITeR

    2025

  7. [7]

    https://play.google.com/store/games

    2025.Google Play store. https://play.google.com/store/games

    2025

  8. [8]

    https://openai.com/gpt-5/

    2025.GPT-5 model. https://openai.com/gpt-5/

    2025

  9. [9]

    https://42matters.com/google-play-statistics-and-trends?

    2025.How many apps are currently avilable. https://42matters.com/google-play-statistics-and-trends?

    2025

  10. [10]

    2025.Lin dataset.https://github.com/seal-hub/CraftDroid

    2025

  11. [11]

    https://www.tencent.com/

    2025.Tencent Inc. https://www.tencent.com/

    2025

  12. [12]

    https://developer.Android.com/studio/test/monkey

    2025.UI/application exerciser Monkey. https://developer.Android.com/studio/test/monkey

    2025

  13. [13]

    Saswat Anand, Mayur Naik, Mary Jean Harrold, and Hongseok Yang. 2012. Automated concolic testing of smartphone apps. InFSE. 1–11

    2012

  14. [14]

    Young-Min Baek and Doo-Hwan Bae. 2016. Automated model-based Android GUI testing using multi-level GUI comparison criteria. InASE. 238–249

    2016

  15. [15]

    Carlos Bernal-Cárdenas, Nathan Cooper, Kevin Moran, Oscar Chaparro, Andrian Marcus, and Denys Poshyvanyk. 2020. Translating video recordings of mobile app usages into replayable scenarios. InProceedings of the ACM/IEEE 42nd international conference on software engineering. 309–321

    2020

  16. [16]

    Benyamin Beyzaei, Saghar Talebipour, Ghazal Rafiei, Nenad Medvidović, and Sam Malek. 2025. Automated Test Transfer across Android Apps using Large Language Models.Proceedings of the ACM on Software Engineering2, ISSTA (2025), 2227–2250

    2025

  17. [17]

    Nataniel P Borges Jr, Maria Gómez, and Andreas Zeller. 2018. Guiding app testing with mined interaction models. InMOBILESoft. 133–143

    2018

  18. [18]

    Shaoheng Cao, Minxue Pan, Yuanhong Lan, and Xuandong Li. 2025. Intention-Based GUI Test Migration for Mobile Apps using Large Language Models.Proceedings of the ACM on Software Engineering2, ISSTA (2025), 2296–2318

    2025

  19. [19]

    Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.06261(2025)

    Pith/arXiv arXiv 2025

  20. [20]

    Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu, and Zhifang Sui. 2022. A survey on in-context learning.arXiv preprint arXiv:2301.00234(2022)

    Pith/arXiv arXiv 2022

  21. [21]

    Sidong Feng and Chunyang Chen. 2024. Prompting Is All You Need: Automated Android Bug Replay with Large Language Models. InProceedings of the 46th IEEE/ACM International Conference on Software Engineering. 1–13

    2024

  22. [22]

    Joseph L Fleiss. 1971. Measuring nominal scale agreement among many raters.Psychological bulletin76, 5 (1971), 378

    1971

  23. [23]

    Daniel Fried, Armen Aghajanyan, Jessy Lin, Sida Wang, Eric Wallace, Freda Shi, Ruiqi Zhong, Wen-tau Yih, Luke Zettlemoyer, and Mike Lewis. 2022. Incoder: A generative model for code infilling and synthesis.arXiv preprint arXiv:2204.05999(2022)

    Pith/arXiv arXiv 2022

  24. [24]

    Difei Gao, Lei Ji, Zechen Bai, Mingyu Ouyang, Peiran Li, Dongxing Mao, Qinchen Wu, Weichen Zhang, Peiyi Wang, Xiangwu Guo, et al. 2023. ASSISTGUI: Task-Oriented Desktop Graphical User Interface Automation.arXiv preprint arXiv:2312.13108(2023)

    Pith/arXiv arXiv 2023

  25. [25]

    Xiang Gao, Shin Hwei Tan, Zhen Dong, and Abhik Roychoudhury. 2018. Android testing via synthetic symbolic execution. InASE. 419–429

    2018

  26. [26]

    Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yixin Dai, Jiawei Sun, Haofen Wang, and Haofen Wang. 2023. Retrieval- augmented generation for large language models: A survey.arXiv preprint arXiv:2312.109972, 1 (2023). Manuscript submitted to ACM Context-Aware Functional Test Generation via Business Logic Extraction and Adaptation 23

    Pith/arXiv arXiv 2023

  27. [27]

    Tianxiao Gu, Chun Cao, Tianchi Liu, Chengnian Sun, Jing Deng, Xiaoxing Ma, and Jian Lü. 2017. AimDroid: activity-insulated multi-level automated testing for Android applications. InICSME. 103–114

    2017

  28. [28]

    Gang Hu, Linjie Zhu, and Junfeng Yang. 2018. AppFlow: using machine learning to synthesize robust, reusable UI tests. InESEC/FSE. 269–282

    2018

  29. [29]

    Kai Huang, Xiangxin Meng, Jian Zhang, Yang Liu, Wenjie Wang, Shuhao Li, and Yuqing Zhang. 2023. An empirical study on fine-tuning large language models of code for automated program repair. In2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 1162–1174

    2023

  30. [30]

    Ziwei Ji, YU Tiezheng, Yan Xu, Nayeon Lee, Etsuko Ishii, and Pascale Fung. 2023. Towards mitigating LLM hallucination via self reflection. InThe 2023 Conference on Empirical Methods in Natural Language Processing

    2023

  31. [31]

    Yue Jiang, Eldon Schoop, Amanda Swearngin, and Jeffrey Nichols. 2023. ILuvUI: Instruction-tuned LangUage-Vision modeling of UIs from Machine Conversations.arXiv preprint arXiv:2310.04869(2023)

    Pith/arXiv arXiv 2023

  32. [32]

    Yavuz Koroglu, Alper Sen, Ozlem Muslu, Yunus Mete, Ceyda Ulker, Tolga Tanriverdi, and Yunus Donmez. 2018. QBE: QLearning-based exploration of Android applications. InICST. 105–115

    2018

  33. [33]

    Duling Lai and Julia Rubin. 2019. Goal-driven exploration for Android applications. InASE. 115–127

    2019

  34. [34]

    Xiaolei Li, Jialun Cao, Yepang Liu, Shing-Chi Cheung, and Hailong Wang. 2025. Reusedroid: A vlm-empowered android ui test migrator boosted by active feedback.arXiv preprint arXiv:2504.02357(2025)

    Pith/arXiv arXiv 2025

  35. [35]

    Xiaoxi Li, Jiajie Jin, Yujia Zhou, Yuyao Zhang, Peitian Zhang, Yutao Zhu, and Zhicheng Dou. 2025. From matching to generation: A survey on generative information retrieval.ACM Transactions on Information Systems43, 3 (2025), 1–62

    2025

  36. [36]

    Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, et al. 2022. Competition-level code generation with alphacode.Science378, 6624 (2022), 1092–1097

    2022

  37. [37]

    Yang Li, Jiacong He, Xin Zhou, Yuan Zhang, and Jason Baldridge. 2020. Mapping natural language instructions to mobile UI action sequences.arXiv preprint arXiv:2005.03776(2020)

    Pith/arXiv arXiv 2020

  38. [38]

    Yuanchun Li, Ziyue Yang, Yao Guo, and Xiangqun Chen. 2017. Droidbot: a lightweight UI-guided test input generator for Android. InICSE-C. 23–26

    2017

  39. [39]

    Yuanchun Li, Ziyue Yang, Yao Guo, and Xiangqun Chen. 2019. Humanoid: a deep learning-based approach to automated black-box Android app testing. InASE. 1070–1073

    2019

  40. [40]

    Jun-Wei Lin, Reyhaneh Jabbarvand, and Sam Malek. 2019. Test transfer across mobile apps through semantic mapping. InASE. 42–53

    2019

  41. [41]

    Jun-Wei Lin and Sam Malek. 2022. Gui test transfer from web to android. InProceedings of the 15th IEEE International Conference on Software Testing, Verification and Validation (ICST 2022)

    2022

  42. [42]

    Zhe Liu, Chunyang Chen, Junjie Wang, Xing Che, Yuekai Huang, Jun Hu, and Qing Wang. 2023. Fill in the blank: Context-aware automated text input generation for mobile gui testing. In2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE). IEEE, 1355–1367

    2023

  43. [43]

    Zhe Liu, Chunyang Chen, Junjie Wang, Mengzhuo Chen, Boyu Wu, Xing Che, Dandan Wang, and Qing Wang. 2023. Chatting with gpt-3 for zero-shot human-like mobile automated gui testing.arXiv preprint arXiv:2305.09434(2023)

    Pith/arXiv arXiv 2023

  44. [44]

    Zhe Liu, Chunyang Chen, Junjie Wang, Yuekai Huang, Jun Hu, and Qing Wang. 2022. Guided bug crush: Assist manual gui testing of android apps via hint moves. InProceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1–14

    2022

  45. [45]

    Zhe Liu, Chunyang Chen, Junjie Wang, Yuekai Huang, Jun Hu, and Qing Wang. 2022. Nighthawk: Fully automated localizing ui display issues via visual understanding.IEEE Transactions on Software Engineering49, 1 (2022), 403–418

    2022

  46. [46]

    Aravind Machiry, Rohan Tahiliani, and Mayur Naik. 2013. Dynodroid: an input generation system for Android apps. InESEC/FSE. 224–234

    2013

  47. [47]

    Ke Mao, Mark Harman, and Yue Jia. 2016. Sapienz: multi-objective automated testing for Android applications. InISSTA. 94–105

    2016

  48. [48]

    Leonardo Mariani, Mauro Pezzè, Valerio Terragni, and Daniele Zuddas. 2023. An evolutionary approach to adapt tests across mobile apps. InAST. 70–79

    2023

  49. [49]

    Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Huan Wang, Yingbo Zhou, Silvio Savarese, and Caiming Xiong. 2022. Codegen: An open large language model for code with multi-turn program synthesis.arXiv preprint arXiv:2203.13474(2022)

    Pith/arXiv arXiv 2022

  50. [50]

    Ju Qian, Zhengyu Shang, Shuoyan Yan, Yan Wang, and Lin Chen. 2020. Roscript: a visual script driven truly non-intrusive robotic testing system for touch screen applications. InProceedings of the ACM/IEEE 42nd International Conference on Software Engineering. 297–308

    2020

  51. [51]

    Dezhi Ran, Hao Wang, Zihe Song, Mengzhou Wu, Yuan Cao, Ying Zhang, Wei Yang, and Tao Xie. 2024. Guardian: A runtime framework for LLM-based UI exploration. InProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis. 958–970

    2024

  52. [52]

    Freda Shi, Daniel Fried, Marjan Ghazvininejad, Luke Zettlemoyer, and Sida I Wang. 2022. Natural language to code translation with execution. arXiv preprint arXiv:2204.11454(2022)

    Pith/arXiv arXiv 2022

  53. [53]

    Helge Spieker, Arnaud Gotlieb, Dusica Marijan, and Morten Mossige. 2017. Reinforcement learning for automatic test case prioritization and selection in continuous integration. InISSTA. 12–22

    2017

  54. [54]

    Ting Su, Guozhu Meng, Yuting Chen, Ke Wu, Weiming Yang, Yao Yao, Geguang Pu, Yang Liu, and Zhendong Su. 2017. Guided, stochastic model-based GUI testing of Android apps. InESEC/FSE. 245–256

    2017

  55. [55]

    Ting Su, Yichen Yan, Jue Wang, Jingling Sun, Yiheng Xiong, Geguang Pu, Ke Wang, and Zhendong Su. 2021. Fully automated functional fuzzing of Android apps for detecting non-crashing logic bugs.Proceedings of the ACM on Programming Languages5, OOPSLA (2021), 1–31

    2021

  56. [56]

    Jingling Sun, Ting Su, Jiayi Jiang, Jue Wang, Geguang Pu, and Zhendong Su. 2023. Property-Based Fuzzing for Finding Data Manipulation Errors in Android Apps. InProceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 1088–1100. Manuscript submitted to ACM 24 Zhang et al

    2023

  57. [57]

    Jingling Sun, Ting Su, Kai Liu, Chao Peng, Zhao Zhang, Geguang Pu, Tao Xie, and Zhendong Su. 2023. Characterizing and finding system setting-related defects in android apps.IEEE Transactions on Software Engineering49, 4 (2023), 2941–2963

    2023

  58. [58]

    Jingling Sun, Ting Su, Jun Sun, Jianwen Li, Mengfei Wang, and Geguang Pu. 2024. Property-Based Testing for Validating User Privacy-Related Functionalities in Social Media Apps. InCompanion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering. 440–451

    2024

  59. [59]

    Maryam Taeb, Amanda Swearngin, Eldon School, Ruijia Cheng, Yue Jiang, and Jeffrey Nichols. 2023. Axnav: Replaying accessibility tests from natural language.arXiv preprint arXiv:2310.02424(2023)

    Pith/arXiv arXiv 2023

  60. [60]

    Chenxu Wang, Tianming Liu, Yanjie Zhao, Minghui Yang, and Haoyu Wang. 2025. LLMDroid: Enhancing Automated Mobile App GUI Testing Coverage with Large Language Model Guidance.Proceedings of the ACM on Software Engineering2, FSE (2025), 1001–1022

    2025

  61. [61]

    Jue Wang, Yanyan Jiang, Ting Su, Shaohua Li, Chang Xu, Jian Lu, and Zhendong Su. 2022. Detecting non-crashing functional bugs in Android apps via deep-state differential analysis. InProceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 434–446

    2022

  62. [62]

    Hao Wen, Yuanchun Li, Guohong Liu, Shanhui Zhao, Tao Yu, Toby Jia-Jun Li, Shiqi Jiang, Yunhao Liu, Yaqin Zhang, and Yunxin Liu. 2024. AutoDroid: LLM-powered Task Automation in Android(MobiCom ’24). Association for Computing Machinery, Washington D.C., DC, USA. https://doi.org/10.1145/3636534.3649379

    arXiv 2024

  63. [63]

    Hao Wen, Hongming Wang, Jiaxuan Liu, and Yuanchun Li. 2023. DroidBot-GPT: GPT-powered UI Automation for Android.arXiv preprint arXiv:2304.07061(2023)

    Pith/arXiv arXiv 2023

  64. [64]

    Shengcheng Yu, Chunrong Fang, Mingzhe Du, Zimin Ding, Zhenyu Chen, and Zhendong Su. 2024. Practical, Automated Scenario-based Mobile App Testing.IEEE Transactions on Software Engineering(2024)

    2024

  65. [65]

    Shengcheng Yu, Chunrong Fang, Mingzhe Du, Yuchen Ling, Zhenyu Chen, and Zhendong Su. 2024. Practical Non-Intrusive GUI Exploration Testing with Visual-based Robotic Arms. InProceedings of the IEEE/ACM 46th International Conference on Software Engineering. 1–13

    2024

  66. [66]

    Chi Zhang, Zhao Yang, Jiaxuan Liu, Yanda Li, Yucheng Han, Xin Chen, Zebiao Huang, Bin Fu, and Gang Yu. 2025. Appagent: Multimodal agents as smartphone users. InProceedings of the 2025 CHI Conference on Human Factors in Computing Systems. 1–20

    2025

  67. [67]

    Shun Zhang, Zhenfang Chen, Yikang Shen, Mingyu Ding, Joshua B Tenenbaum, and Chuang Gan. 2023. Planning with large language models for code generation.arXiv preprint arXiv:2303.05510(2023)

    Pith/arXiv arXiv 2023

  68. [68]

    Yakun Zhang, Chen Liu, Xiaofei Xie, Yun Lin, Jin Song Dong, Dan Hao, and Lu Zhang. 2025. GUI Test Migration via Abstraction and Concretization. ACM Transactions on Software Engineering and Methodology(2025)

    2025

  69. [69]

    Yixue Zhao, Justin Chen, Adriana Sejfia, Marcelo Schmitt Laser, Jie Zhang, Federica Sarro, Mark Harman, and Nenad Medvidovic. 2020. FrUITeR: a framework for evaluating UI test reuse. InESEC/FSE. 1190–1201. Manuscript submitted to ACM

    2020