Pith. sign in

REVIEW 3 major objections 5 minor 104 references

Automated Test Transfer Across Android Apps Using Large Language Models

T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A method called LLMigrate transfers UI tests across Android apps at a 97.5% success rate on the CraftDroid benchmark.

desk verdict A genuinely new LLM-based test-transfer pipeline with public artifacts and a thoughtful unseen-app study, but the benchmark-tuned parameters make the headline 97.5% in-sample; a referee should require a validation split or sensitivity analysis. read the letter →

arxiv 2411.17933 v2 pith:K6Q4AQA4 submitted 2024-11-26 cs.SE

Benyamin Beyzaei , Saghar Talebipour , Ghazal Rafiei , Nenad Medvidovic , Sam Malek This is my paper · ORCID
classification cs.SE
keywords mobileUItestingtesttransfermigrationlargelanguagemodelsmultimodalLLMAndroidappsGUIautomationoraclegeneration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a multimodal large language model can transfer an existing end-to-end UI test from one Android app to another app with the same functionality, without access to the source code. The method first rewrites the source test as a short natural-language description, then lets an LLM drive exploration of the target app by choosing the next event from a screenshot and a simplified layout hierarchy. On 120 transfers drawn from the CraftDroid benchmark, the paper reports a 97.5% success rate and a 91.1% reduction in manual test-writing effort, improving on the best prior technique by 9.1 and 38.2 percentage points. If this holds beyond the benchmark, testers could stop writing routine UI tests from scratch for every app in a domain.

What carries the argument

The load-bearing object is the two-phase LLMigrate pipeline: Source Abstraction produces the Abstract Source Test, a one-paragraph natural-language description of what the source test does and which step completes it, and Test Migration runs a loop in which a Screen Analyzer sub-agent interprets a screenshot plus cleaned XML layout into a widget report and an event-generation sub-agent picks the best next event from that report, the abstract test, and the already-performed events. Reliability mechanisms are temperature-zero sampling with majority voting (three queries, key threshold two), a repair prompt that feeds the last exception back to the model with chain-of-thought instructions, and an Explorer that records dead-end events, backtracks by restarting the app, and falls back when repeated guesses fail. The loop ends when all source oracles have been transferred or after three times the source event count.

What would settle it

Run LLMigrate on a held-out set of same-domain app pairs released after GPT-4o's knowledge cutoff, with ground-truth tests authored independently, and count successful transfers per flow; if the overall success rate falls clearly below the claimed range on a statistically meaningful sample, or if any reproducible flow loops forever without transferring all oracles, the central claim fails. A sharper check is to choose flows in which the target app inserts an extra confirmation step, such as a password-confirmation dialog or a terms checkbox, that the source flow does not contain, and measure how often the loop still reaches the final oracle.

Watch

Extended reading notes

Core claim

On its own terms, LLMigrate's discovery is that test transfer can be reframed as goal-directed exploration instead of widget-to-widget matching. The source test is executed once and summarized by an LLM into a natural-language Abstract Source Test; the migration phase then repeatedly captures the target app's current screen, translates the screenshot plus cleaned XML hierarchy into a widget report, selects the next GUI, system, or oracle event, and repairs invalid events with exception-feedback prompts and backtracking. The paper reports total average precision of 98.39% for GUI events and 94.71% for oracle events, total average recall of 99.53% and 94.71%, a 97.5% successful transfer rate, and a 91.1% reduction in manual effort across 120 transfers, with a 93.3% success rate on newly released apps outside GPT-4o's training data.

Load-bearing premise

The load-bearing premise is that a screenshot plus a simplified listing of the on-screen elements gives the language model enough information to choose the correct next action on every screen of an arbitrary target app, even when the model has never seen that app; if there is a class of apps or flows where that inference fails, the reported success rate will not generalize.

Editorial extensions

If this is right

  • A developer with one tested app in a domain can obtain usage-based tests for other Android apps in that domain without seeing their source code.
  • The transfer is fast and cheap enough for routine use: an average of 247 seconds and roughly USD 0.70 per transferred test in GPT-4o token costs.
  • Complex flows that defeated similarity-based transfer show the largest gains: in the shopping category, GUI-event precision rises from about 44–45% for prior techniques to 100% for LLMigrate.
  • Newer apps not present in the LLM's training data still transfer at a 93.3% average success rate, indicating the method is not merely replaying memorized apps.
  • The method handles input-field differences by generating plausible values for fields absent from the source test and skipping fields not used by the target app.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the natural-language Abstract Source Test could be treated as a reusable asset: once a functionality is described once, that description might serve any number of target apps, turning one-to-one test transfer into specification-based test generation.
  • Because the loop is guided by a goal rather than by source-app widget mappings, the same machinery could transfer shared sub-flows such as login, search, or form-fill across apps in different categories.
  • A direct user study measuring developer time to accept or edit an LLMigrate-produced test would test whether the 91.1% Levenshtein-based reduction metric translates into actual time savings; the paper does not present that study.
  • The shopping-category oracle gap suggests that pairing the multimodal screen analysis with stronger visual verification, such as icon classification or optical character recognition, could close the remaining transfer failures.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper presents LLMigrate, a technique that uses a multimodal LLM (GPT-4o) to transfer usage-based UI tests between Android apps in the same domain without requiring source code. LLMigrate first executes the source test on the source app, augments each event with widget attributes, and asks the LLM to produce a natural-language 'Abstract Source Test.' It then dynamically explores the target app, using screenshots and cleaned XML hierarchies as the current state, and repeatedly asks the LLM to select the next event, with majority voting, dead-end tracking, backtracking, and repair prompts. The evaluation on 120 transfers from 19 functional CraftDroid apps reports a 97.5% successful transfer rate, a 91.1% manual-effort reduction, and improvements over the best prior technique of 9.1% in success rate and 38.2% in effort reduction. Additional studies target modern popular apps (RQ4) and apps released after GPT-4o's knowledge cutoff (RQ5).

Significance. If the reported results hold, LLMigrate would be a substantial advance over similarity-based test transfer: it handles differing input fields, transfers oracle events, requires no source code, and achieves high success and effort reduction in complex domains where prior work shows negative reduction. The paper's public artifacts, its manually evaluated 120-transfer study on the standard CraftDroid benchmark, and its explicit RQ5 attempt to evaluate on apps released after the model's knowledge cutoff are genuine strengths. However, the headline numbers are produced by a configuration tuned on the same benchmark used for evaluation, and several supporting numbers in RQ4 and RQ5 are internally inconsistent or not traceable to individual transfers. The central idea is promising and within scope for the venue, but the evidence as reported is not yet as strong as the abstract claims.

major comments (3)
  1. [Section 4.1] The three main parameters (maximum wrong tries at the same step, majority-voting run count n, and majority-voting inclusion threshold m) are tuned directly on the same 120-transfer evaluation that produces the headline 97.5% success rate, 91.1% reduction, and the 9.1%/38.2% improvements over TREADROID. Section 4.1 states that the authors 'empirically observed the best-performing values' and set them to 3, 3, and 2, but no validation split or sensitivity analysis is reported. Since prior techniques are evaluated with fixed configurations, the comparison is effectively between a benchmark-tuned configuration and fixed baselines. Please add a hold-out validation split, a sensitivity analysis over the parameter grid, or the metrics of a reasonable untuned configuration; without this, the central numbers may be optimistic.
  2. [Section 4.5, Table 4] The text states that LLMigrate achieved 'an average success rate of 98%' and 'an average reduction score of 97%' on the new popular apps. Averaging the ten rows in Table 4 gives 95% success (nine 100% entries and one 50% entry) and approximately 83% reduction, not 98% and 97%. Please clarify the weighting or averaging procedure, or correct the stated values and qualify the 'strong performance' characterization accordingly.
  3. [Section 4.6, Tables 5 and 6] The RQ5 setup says the study involves 30 total test transfers, but Table 6 reports one row per category/test with a single value per metric. With five new categories and three apps each, it is not clear whether the rows aggregate 15 transfers, 30 transfers, or something else; as written, the 93.3% success rate is not traceable to individual transfers. Please report per-app or per-transfer results and reconcile the '30 total test transfers' statement with the displayed aggregation.
minor comments (5)
  1. [Abstract and Section 8] The abstract reports 97.5% success and 91.1% reduction, while the conclusion says '97% of tests' and 'more than 90%' of manual work; please use consistent figures.
  2. [Section 4.3 and Figure 7] Phrases such as 'outperform ... by almost 10%' and '44% lower' should be expressed in percentage points (9.1 points and 44.5 points, respectively) to avoid ambiguity.
  3. [Table 3] The parenthetical differences in Table 3 are not explained and do not always correspond to the difference from the same baseline; for example, the total oracle recall shows 5.7%, while LLMigrate (94.71) minus TREADROID (87.44) is 7.27. Define the comparison basis in the caption or remove the parentheticals.
  4. [Section 4.2 and Table 3] TEMdroid's published averages are from a non-identical dataset and the artifacts are unavailable; the paper acknowledges this, but presenting those averages in the same comparison table risks over-reading. Consider using a clearly separated row or footnote that states the 19/23 overlap and the lack of artifact-based re-evaluation.
  5. [Section 4.6] The claim that apps are 'unseen by LLMs' is operationalized only by release date after the knowledge cutoff; it would be useful to state explicitly that this is a proxy and that the model may still have seen similar app types or patterns.

Circularity Check

1 steps flagged · score 4.0 of 10

Headline metrics are reported after tuning three parameters on the same evaluation set, but RQ5 gives an out-of-sample check that keeps the central claim from being fully self-referential.

  1. fitted input called prediction [Section 4.1 Experimental Setup, parameter settings]
    "As discussed in Section 3, our approach has three adjustable parameters: 1) maximum wrong tries at the Same Step, 2) total number of runs for majority voting (n), and 3) threshold for majority voting to include a field (m). We empirically observed the best-performing values for all these parameters and set them to 3, 3, and 2, respectively, in our evaluation."

    The same 120 transfers that produce the headline 97.5% success rate, 91.1% reduction, and the 9.1% and 38.2% improvements over prior techniques are used to select the three parameters. Because the paper states the values were chosen as 'best-performing' on this evaluation set, the reported metrics are in-sample optima rather than the outcome of a fixed, pre-specified method. These parameters directly control the migration loop's backtracking threshold, majority-vote query count, and field-inclusion threshold, so they can materially change the transfer outcome. The headline claim is therefore partially fitted to the data used to measure it. The RQ5 unseen-apps study (93% success rate) is an independent, out-of-sample measurement, which prevents the circularity from being total.

full rationale

The only concrete circular element is the parameter-selection procedure in Section 4.1: three adjustable parameters are tuned on the same 120-transfer benchmark that yields the headline figures, so the top-line numbers include an in-sample fitting component. The paper does not use the ground-truth target tests during transfer, the comparison baselines are external (CraftDroid, TREADROID, TEMdroid), and RQ5 evaluates apps released after GPT-4o's knowledge cutoff with manually written tests, giving an independent out-of-sample check. No load-bearing self-citation chain or definitional equivalence is present; CraftDroid, though co-authored by one of the present authors, is a public benchmark also used by external works. The fitted-parameter issue is real but partial, so the overall circularity score is moderate.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The system introduces no new entities; it is a tool, and the abstract source test is an internal representation, not a new physical or conceptual object. The central results rest on five domain assumptions about app similarity, LLM competence, prompt generality, oracle-completing test flows, and ground-truth correctness. Five tunable numeric parameters were chosen by hand or tuned on the benchmark, which affects how the 97.5% figure should be read. RQ5 provides independent grounding by testing on unseen apps.

free parameters (5)
  • max_wrong_tries_at_same_step = 3
    Section 4.1: empirically observed best-performing value, used as threshold for wrong event attempts before backtracking.
  • majority_voting_runs_n = 3
    Number of repeated LLM queries per event selection; tuned on the benchmark.
  • majority_voting_threshold_m = 2
    Minimum frequency for a JSON key to be included in the final event; tuned on the benchmark.
  • max_target_events_multiplier = 3 (times number of source test events)
    Configurable end condition threshold for the migration loop, set to three times the source event count.
  • max_unsuccessful_attempts = 3
    Threshold for consecutive invalid events at one step before triggering a backtrack.
assumptions (5)
  • domain assumption Apps within the same domain share analogous functionalities despite different UIs.
    Stated in Section 1 as the foundational premise of test transfer; the whole approach exploits this overlap.
  • domain assumption GPT-4o can map between XML widget descriptions, screenshots, and functional intent.
    Invoked in Section 3.2 for screen analysis and event selection; all transferred tests depend on this.
  • domain assumption The manually authored prompt templates and hints are general and contain no app-specific details.
    Claimed in Section 3.2; if false, results would not generalize to new apps, as tested in RQ5.
  • domain assumption Tests typically conclude with an oracle event, so transferring all source oracles is a valid termination condition.
    Used as the migration loop end condition in Section 3.2.
  • domain assumption The manually written ground truth tests are correct and represent the expected behavior for each functionality.
    Used as the oracle for precision, recall, and reduction in Section 4.2; manual evaluation is subjective.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Automated Test Transfer Across Android Apps Using Large Language Models." pith.science (2026). https://pith.science/paper/K6Q4AQA4

@misc{pith2026241117933,
  author       = {Pith},
  title        = {Pith review of: Automated Test Transfer Across Android Apps Using Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/K6Q4AQA4}},
  note         = {Machine review of arXiv:2411.17933}
}
read the original abstract

The pervasiveness of mobile apps in everyday life necessitates robust testing strategies to ensure quality and efficiency, especially through end-to-end usage-based tests for mobile apps' user interfaces (UIs). However, manually creating and maintaining such tests can be costly for developers. Since many apps share similar functionalities beneath diverse UIs, previous works have shown the possibility of transferring UI tests across different apps within the same domain, thereby eliminating the need for writing the tests manually. However, these methods have struggled to accommodate real-world variations, often facing limitations in scenarios where source and target apps are not very similar or fail to accurately transfer test oracles. This paper introduces an innovative technique, LLMigrate, which leverages Large Language Models (LLMs) to efficiently transfer usage-based UI tests across mobile apps. Our experimental evaluation shows LLMigrate can achieve a 97.5% success rate in automated test transfer, reducing the manual effort required to write tests from scratch by 91.1%. This represents an improvement of 9.1% in success rate and 38.2% in effort reduction compared to the best-performing prior technique, setting a new benchmark for automated test transfer.

Figures

Figures reproduced from arXiv: 2411.17933 by the authors.

Figure 1
Figure 1. Registration test in DODuae [1] [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Registration test in Zalando [5]. transfer outcomes are evaluated based on the precision, recall, reduction, and success rate metrics, and the results were compared to those obtained from the existing techniques evaluated on the same benchmark [59, 66, 98]. LLMigrate achieved a 97.5% success rate, reducing the manual effort required to write tests from scratch by 91.1%. This represents an improvement of 9.1% in succ… view at source ↗
Figure 3
Figure 3. LLMigrate’s approach overview. after the transfer, which is the final outcome of the transfer technique, is termed the target test (e.g., the registration test on Zalando). To evaluate the effectiveness of our approach, we compare the target test against a manually created test for the target app, referred to as the ground truth test. This ground truth test mirrors the same functionality as the source test and serve… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Examples of GUI and oracle events after the test augmentation step. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: An example of the screen analysis task performed by the screen analyzer sub-agent. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: The reduction achieved by LLMigrate, CraftDroid, and TREADROID across various app categories. met their objectives, including executing the required functionality and asserting the appropriate conditions using the transferred oracles. This shows that LLMigrate was able…
Figure 7
Figure 7. Figure 7: The successful transfer rate achieved by [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

104 extracted references · 51 canonical work pages

  1. [1]

    DODuae - Women’s Online Store

    2024. DODuae - Women’s Online Store. https://tinyurl.com/mu5zkenz

    2024

  2. [2]

    Email - Fast & Secure Mail

    2024. Email - Fast & Secure Mail. https://tinyurl.com/53ensprk

    2024

  3. [3]

    Firefox Fast & Private Browser

    2024. Firefox Fast & Private Browser. https://tinyurl.com/yc5t5tkh

    2024

  4. [4]

    To Do List

    2024. To Do List. https://tinyurl.com/c8chz4fb

    2024

  5. [5]

    Zalando – Online Fashion Store

    2024. Zalando – Online Fashion Store. https://tinyurl.com/mpee366u

    2024

  6. [6]

    200TV - Live TV Movies App

    2025. 200TV - Live TV Movies App. https://tinyurl.com/3xf6wb8v

    2025

  7. [7]

    Ava Assistant - Movies & Shows

    2025. Ava Assistant - Movies & Shows. https://tinyurl.com/4e2pzxry

    2025

  8. [8]

    Cash Loan EMI Calcualtor

    2025. Cash Loan EMI Calcualtor. https://tinyurl.com/2tb59nr7

    2025

Show all 104 references
  1. [9]

    Chatbot - AI Smart Assistant

    2025. Chatbot - AI Smart Assistant. https://tinyurl.com/vk4t739r

    2025

  2. [10]

    ClipFix: Movie Shazam

    2025. ClipFix: Movie Shazam. https://tinyurl.com/avjp9bzv

    2025

  3. [11]

    Color SMS: Message & Messenger

    2025. Color SMS: Message & Messenger. https://tinyurl.com/428xy74b

    2025

  4. [12]

    Daily Notes - Easy Notebook

    2025. Daily Notes - Easy Notebook. https://tinyurl.com/rrr9j5ee

    2025

  5. [13]

    Deep Search - AI Chatbot

    2025. Deep Search - AI Chatbot. https://tinyurl.com/5c68xzdt

    2025

  6. [14]

    EMI Calculator & Financial

    2025. EMI Calculator & Financial. https://tinyurl.com/2kbtpd25

    2025

  7. [15]

    EMI Calculator : Loan Planner

    2025. EMI Calculator : Loan Planner. https://tinyurl.com/2vsb3h3y

    2025

  8. [16]

    Google Gemini

    2025. Google Gemini. https://tinyurl.com/nhe8hpty

    2025

  9. [17]

    LLMigrate open-source repository

    2025. LLMigrate open-source repository. https://github.com/seal-hub/llmigrate

    2025

  10. [18]

    Messages for SMS - DUAL SIM

    2025. Messages for SMS - DUAL SIM. https://tinyurl.com/mrd2rdus

    2025

  11. [19]

    Messages: Text SMS

    2025. Messages: Text SMS. https://tinyurl.com/bdfhk5xk

    2025

  12. [20]

    Notes - QuickNotes

    2025. Notes - QuickNotes. https://tinyurl.com/mue2jyau

    2025

  13. [21]

    Personal notes and tasks

    2025. Personal notes and tasks. https://tinyurl.com/393fn3ww

    2025

  14. [22]

    Toufique Ahmed and Premkumar Devanbu. 2022. Few-shot training LLMs for project-specific code-summarization. In Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering . 1–5

    2022

  15. [23]

    Domenico Amalfitano, Anna Rita Fasolino, Porfirio Tramontana, Salvatore De Carmine, and Atif M Memon. 2012. Using GUI ripping for automated testing of Android applications. In2012 Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering . IEE...

    2012

  16. [24]

    Amalfitano, A

    D. Amalfitano, A. R. Fasolino, P. Tramontana, B. D. Ta, and A. M. Memon. 2015. MobiGUITAR: Automated Model-Based Testing of Mobile Apps. IEEE Software 32, 5 (Sept 2015), 53–59. https://doi.org/10.1109/MS.2014.55

    2015 doi

  17. [25]

    Saswat Anand, Mayur Naik, Mary Jean Harrold, and Hongseok Yang. 2012. Automated Concolic Testing of Smartphone Apps. In Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering (Cary, North Carolina) (FSE ’12). ACM, New York, NY, U...

    2012 doi

  18. [26]

    Farnaz Behrang and Alessandro Orso. 2018. Test migration for efficient large-scale assessment of mobile app coding assignments. In Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis

    2018

  19. [27]

    Farnaz Behrang and Alessandro Orso. 2019. Test Migration Between Mobile Apps with Similar Functionality. In 34th International Conference on Automated Software Engineering (ASE 2019)

    2019

  20. [28]

    Farnaz Behrang and Alessandro Orso. 2019. To appear.. Test Migration Between Mobile Apps with Similar Functionality. In Proceedings of the The 34th IEEE/ACM International Conference on Automated Software Engineering (San Diego, USA) (ASE ’19)

    2019

  21. [29]

    Islem Bouzenia, Premkumar Devanbu, and Michael Pradel. 2024. Repairagent: An autonomous, llm-based agent for program repair. arXiv preprint arXiv:2403.17134 (2024)

    2024 arXiv

  22. [30]

    Jieshan Chen, Chunyang Chen, Zhenchang Xing, Xiwei Xu, Liming Zhu, Guoqiang Li, and Jinshui Wang. 2020. Unblind your apps: Predicting natural-language labels for mobile gui components by deep learning. In Proceedings of the ACM/IEEE 42nd international conference on software en...

    2020

  23. [31]

    Wontae Choi, George Necula, and Koushik Sen. 2013. Guided GUI Testing of Android Apps with Minimal Restart and Approximate Learning. In Proceedings of the 2013 ACM SIGPLAN International Conference on Object Oriented Programming Systems Languages & Applications (Indianapolis, I...

    2013

  24. [32]

    World Wide Web Consortium. 2025. https://www.w3.org/TR/webdriver/

    2025

  25. [33]

    Appium Contributors. [n. d.]. Appium. https://github.com/appium/appium

  26. [34]

    Xiang Deng et al. 2024. Mind2web: Towards a generalist agent for the web. Advances in Neural Information Processing Systems 36 (2024)

    2024

  27. [35]

    Zhen Dong, Marcel Böhme, Lucia Cojocaru, and Abhik Roychoudhury. 2020. Time-travel testing of Android apps. In 2020 IEEE/ACM 42nd International Conference on Software Engineering (ICSE) . IEEE, 481–492

    2020

  28. [37]

    Sidong Feng and Chunyang Chen. 2024. Prompting is all you need: Automated android bug replay with large language models. In Proceedings of the 46th IEEE/ACM International Conference on Software Engineering . 1–13

    2024

  29. [38]

    Yi Gao, Xing Hu, Tongtong Xu, Xin Xia, David Lo, and Xiaohu Yang. 2024. MUT: Human-in-the-Loop Unit Test Migration. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering . 1–12

    2024

  30. [39]

    Tianxiao Gu et al. 2019. Practical GUI testing of Android applications via model abstraction and refinement. In 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE) . IEEE, 269–280

    2019

  31. [40]

    Izzeddin Gur et al. 2022. Understanding html with large language models. arXiv preprint arXiv:2210.03945 (2022)

    2022 arXiv

  32. [41]

    Izzeddin Gur et al. 2023. A real-world webagent with planning, long context understanding, and program synthesis. arXiv preprint arXiv:2307.12856 (2023)

    2023 arXiv

  33. [42]

    Roman Haas et al. 2021. How can manual testing processes be optimized? developer survey, optimization guidelines, and case studies. In Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering...

    2021

  34. [43]

    Halfond, and Ramesh Govindan

    Shuai Hao, Bin Liu, Suman Nath, William G.J. Halfond, and Ramesh Govindan. 2014. PUMA: Programmable UI- automation for Large-scale Dynamic Analysis of Mobile Apps. In Proceedings of the 12th Annual International Conference on Mobile Systems, Applications, and Services (Bretton...

    2014

  35. [44]

    Gang Hu, Linjie Zhu, and Junfeng Yang. 2018. AppFlow: using machine learning to synthesize robust, reusable UI tests. In Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering . ACM, 269–282

    2018

  36. [45]

    Yuchao Huang et al. 2024. Crashtranslator: Automatically reproducing mobile application crashes directly from stack trace. In Proceedings of the 46th IEEE/ACM International Conference on Software Engineering . 1–13

    2024

  37. [46]

    Jensen, Mukul R

    Casper S. Jensen, Mukul R. Prasad, and Anders Møller. 2013. Automated Testing with Targeted Event Sequence Generation. In Proceedings of the 2013 International Symposium on Software Testing and Analysis (Lugano, Switzerland) (ISSTA 2013). ACM, New York, NY, USA, 67–77. https:/...

    2013

  38. [47]

    Ruihua Ji et al. 2023. Vision-Based Widget Mapping for Test Migration Across Mobile Platforms: Are We There Yet?. In 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE) . IEEE, 1416–1428

    2023

  39. [48]

    Bangyan Ju et al. 2024. A Study of Using Multimodal LLMs for Non-Crash Functional Bug Detection in Android Apps. arXiv preprint arXiv:2407.19053 (2024)

    2024 arXiv

  40. [49]

    Jouko Kaasila, Denzil Ferreira, Vassilis Kostakos, and Timo Ojala. 2012. Testdroid: automated remote UI testing on Android. In Proceedings of the 11th International Conference on Mobile and Ubiquitous Multimedia . 1–4

    2012

  41. [50]

    Sungmin Kang, Juyeon Yoon, Nargiz Askarbekkyzy, and Shin Yoo. 2023. Evaluating Diverse Large Language Models for Automatic and General Bug Reproduction. arXiv preprint arXiv:2311.04532 (2023)

    2023 arXiv

  42. [51]

    Sungmin Kang, Juyeon Yoon, and Shin Yoo. 2023. Large language models are few-shot testers: Exploring llm-based general bug reproduction. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) . IEEE, 2312–2323

    2023

  43. [52]

    Farideh Khalili, Leonardo Mariani, Ali Mohebbi, Mauro Pezzè, and Valerio Terragni. 2024. Semantic matching in GUI test reuse. Empirical Software Engineering 29, 3 (2024), 1–58

    2024

  44. [53]

    Pavneet Singh Kochhar et al. 2015. Understanding the Test Automation Culture of App Developers. In 2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST) . 1–10. https://doi.org/10.1109/ICST. 2015.7102609

    2015

  45. [54]

    Yavuz Koroglu et al. 2018. QBE: QLearning-based exploration of android applications. In Software Testing, Verification and Validation (ICST), 2018 IEEE 11th International Conference on . IEEE, 105–115

    2018

  46. [55]

    Firebase Test Lab. 2024. Robo test (Android). https://firebase.google.com/docs/test-lab/android/robo-ux-test

    2024

  47. [56]

    Vladimir I Levenshtein. 1966. Binary codes capable of correcting deletions, insertions, and reversals. In Soviet physics doklady, Vol. 10. 707–710

    1966

  48. [57]

    Kanglin Li and Mengqi Wu. 2006. Effective GUI testing automation: Developing an automated GUI testing tool . John Wiley & Sons

    2006

  49. [58]

    Jun-Wei Lin, Reyhaneh Jabbarvand, and Sam Malek. 2019. Test Transfer Across Mobile Apps Through Semantic Mapping. In 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE) . 42–53. https: //doi.org/10.1109/ASE.2019.00015

    2019

  50. [59]

    Jun-Wei Lin, Reyhaneh Jabbarvand, and Sam Malek. 2019. Test Transfer Across Mobile Apps Through Semantic Mapping. In 34th International Conference on Automated Software Engineering (ASE 2019)

    2019

  51. [60]

    Jun-Wei Lin and Sam Malek. 2022. Gui test transfer from web to android. In 2022 IEEE Conference on Software Testing, Verification and Validation (ICST). IEEE, 1–11

    2022

  52. [61]

    Mario Linares-Vásquez, Carlos Bernal-Cárdenas, Kevin Moran, and Denys Poshyvanyk. 2017. How do developers test android applications?. In 2017 IEEE International Conference on Software Maintenance and Evolution (ICSME) . Proc. ACM Softw. Eng., Vol. 2, No. ISSTA, Article ISSTA09...

    2017

  53. [62]

    Mario Linares-Vásquez, Martin White, Carlos Bernal-Cárdenas, Kevin Moran, and Denys Poshyvanyk. 2015. Mining Android App Usages for Generating Actionable GUI-based Execution Scenarios. In Proceedings of the 12th Working Conference on Mining Software Repositories (Florence, Ita...

    2015

  54. [63]

    Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. 2024. Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation. Advances in Neural Information Processing Systems 36 (2024)

    2024

  55. [64]

    Peng Liu et al. 2017. Automatic Text Input Generation for Mobile Testing. In Proceedings of the 39th International Conference on Software Engineering (Buenos Aires, Argentina) (ICSE ’17). IEEE Press, Piscataway, NJ, USA, 643–653. https://doi.org/10.1109/ICSE.2017.65

    2017 doi

  56. [65]

    Shuqi Liu et al. 2022. Test reuse based on adaptive semantic matching across android mobile applications. In 2022 IEEE 22nd International Conference on Software Quality, Reliability and Security (QRS) . IEEE, 703–709

    2022

  57. [66]

    Shuqi Liu et al. 2024. Enhancing test reuse with GUI events deduplication and adaptive semantic matching. Science of Computer Programming 232 (2024), 103052

    2024

  58. [67]

    Zhe Liu et al. 2023. Fill in the blank: Context-aware automated text input generation for mobile gui testing. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE). IEEE, 1355ś1367 (2023)

    2023

  59. [68]

    Zhe Liu et al. 2024. Vision-driven Automated Mobile GUI Testing via Multimodal Large Language Model. arXiv preprint arXiv:2407.03037 (2024)

    2024 arXiv

  60. [69]

    Zhe Liu, Chunyang Chen, Junjie Wang, Mengzhuo Chen, Boyu Wu, Xing Che, Dandan Wang, and Qing Wang. 2024. Make llm a testing expert: Bringing human-like interaction to mobile gui testing via functionality-aware decisions. In Proceedings of the IEEE/ACM 46th International Confer...

    2024

  61. [70]

    Aravind Machiry, Rohan Tahiliani, and Mayur Naik. 2013. Dynodroid: An Input Generation System for Android Apps. In Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering (Saint Petersburg, Russia) (ESEC/FSE 2013). ACM, New York, NY, USA, 224–234. http...

    2013

  62. [71]

    Riyadh Mahmood, Nariman Mirzaei, and Sam Malek. 2014. EvoDroid: Segmented Evolutionary Testing of Android Apps. In Proceedings of the 22Nd ACM SIGSOFT International Symposium on Foundations of Software Engineering (Hong Kong, China) (FSE 2014). ACM, New York, NY, USA, 599–609....

    2014

  63. [72]

    Ke Mao, Mark Harman, and Yue Jia. 2016. Sapienz: Multi-objective automated testing for android applications. In Proceedings of the 25th International Symposium on Software Testing and Analysis . 94–105

    2016

  64. [73]

    Ke Mao, Mark Harman, and Yue Jia. 2016. Sapienz: Multi-objective Automated Testing for Android Applications. In Proceedings of the 25th International Symposium on Software Testing and Analysis (Saarbrücken, Germany) (ISSTA 2016). ACM, New York, NY, USA, 94–105. https://do...

    2016

  65. [74]

    Leonardo Mariani, Mauro Pezzè, Valerio Terragni, and Daniele Zuddas. 2021. An Evolutionary Approach to Adapt Tests Across Mobile Apps. In The 2nd ACM/IEEE International Conference on Automation of Software Test (AST 2021)

    2021

  66. [75]

    Leonardo Mariani, Mauro Pezzè, and Daniele Zuddas. 2018. Augusto: Exploiting popular functionalities for the generation of semantic gui tests with oracles. InProceedings of the 40th International Conference on Software Engineering. 280–290

    2018

  67. [76]

    Mirzaei, H

    N. Mirzaei, H. Bagheri, R. Mahmood, and S. Malek. 2015. SIG-Droid: Automated system input generation for Android applications. In 2015 IEEE 26th International Symposium on Software Reliability Engineering (ISSRE) . 461–471. https://doi.org/10.1109/ISSRE.2015.7381839

    2015

  68. [77]

    Yash Mishra et al. 2023. Image Understanding of GUI Widgets for Test Reuse. In 2023 3rd International Conference on Pervasive Computing and Social Networking (ICPCSN) . IEEE, 572–579

    2023

  69. [78]

    Kevin Moran et al. 2016. Automatically Discovering, Reporting and Reproducing Android Application Crashes. In 2016 IEEE International Conference on Software Testing, Verification and Validation (ICST) . 33–44. https://doi.org/10. 1109/ICST.2016.34

    2016

  70. [79]

    OpenAI. [n. d.]. GPT-4o. https://platform.openai.com/docs/models/gpt-4o/

  71. [80]

    Xue Qin, Hao Zhong, and Xiaoyin Wang. 2019. TestMig: Migrating GUI Test Cases from iOS to Android. InProceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis (Beijing, China) (ISSTA 2019). ACM, New York, NY, USA, 284–295. https://doi.org/10....

    2019

  72. [81]

    Andreas Rau, Jenny Hotzkow, and Andreas Zeller. 2018. Transferring tests across web applications. In International Conference on Web Engineering. Springer, 50–64

    2018

  73. [82]

    Ting Su et al. 2017. Guided, Stochastic Model-based GUI Testing of Android Apps. In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering (Paderborn, Germany) (ESEC/FSE 2017). ACM, New York, NY, USA, 245–256. https://doi.org/10.1145/3106237.3106298

    2017

  74. [83]

    Ting Su, Guozhu Meng, Yuting Chen, Ke Wu, Weiming Yang, Yao Yao, Geguang Pu, Yang Liu, and Zhendong Su

  75. [84]

    Haotian Sun et al. 2024. Adaplanner: Adaptive planning from feedback with language models. Advances in Neural Information Processing Systems 36 (2024)

    2024

  76. [85]

    Saghar Talebipour, Yixue Zhao, Luka Dojcilović, Chenggang Li, and Nenad Medvidović. 2022. UI test migration across mobile platforms. In Proceedings of the 36th IEEE/ACM International Conference on Automated Software Engineering (Melbourne, Australia) (ASE ’21). IEEE Press, 756...

    2022

  77. [86]

    Appium Team. 2024. Appium UiAutomator2 Driver. https://github.com/appium/appium-uiautomator2-driver

    2024

  78. [87]

    Android Studio Team. 2023. UI/Application Exerciser Monkey. https://developer.android.com/studio/test/monkey

    2023

  79. [88]

    Dingbang Wang, Yu Zhao, Sidong Feng, Zhaoxu Zhang, William GJ Halfond, Chunyang Chen, Xiaoxia Sun, Jiangfan Shi, and Tingting Yu. 2024. Feedback-driven automated whole bug report reproduction for android apps. InProceedings of the 33rd ACM SIGSOFT International Symposium on So...

    2024

  80. [89]

    Jue Wang et al. 2020. ComboDroid: generating high-quality test inputs for Android apps via use case combinations. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering . 469–480

    2020

  81. [90]

    Junjie Wang et al. 2024. Software testing with large language models: Survey, landscape, and vision.IEEE Transactions on Software Engineering (2024)

    2024

  82. [91]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems 35 (2022), 24824–24837

    2022

  83. [92]

    Hao Wen, Hongming Wang, Jiaxuan Liu, and Yuanchun Li. 2023. Droidbot-gpt: Gpt-powered ui automation for android. arXiv preprint arXiv:2304.07061 (2023)

    2023 arXiv

  84. [93]

    Prasad, and Tao Xie

    Wei Yang, Mukul R. Prasad, and Tao Xie. 2013. A Grey-Box Approach for Automated GUI-Model Generation of Mobile Applications. In Fundamental Approaches to Software Engineering , Vittorio Cortellessa and Dániel Varró (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 250–265

    2013

  85. [94]

    Juyeon Yoon, Robert Feldt, and Shin Yoo. 2023. Autonomous Large Language Model Agents Enabling Intent-Driven Mobile GUI Testing. arXiv preprint arXiv:2311.08649 (2023)

    2023 arXiv

  86. [95]

    Shengcheng Yu et al. 2023. Llm for test script generation and migration: Challenges, capabilities, and opportunities. In 2023 IEEE 23rd International Conference on Software Quality, Reliability, and Security (QRS) . IEEE, 206–217

    2023

  87. [96]

    Prasad, and Sarfraz Khurshid

    Razieh Nokhbeh Zaeem, Mukul R. Prasad, and Sarfraz Khurshid. 2014. Automated Generation of Oracles for Testing User-Interaction Features of Mobile Apps. In Proceedings of the 2014 IEEE International Conference on Software Testing, Verification, and Validation (ICST ’14) . IEEE...

    2014 doi

  88. [97]

    Hailong Zhang and Atanas Rountev. 2017. Analysis and Testing of Notifications in Android Wear Applications. In Proceedings of the 39th International Conference on Software Engineering (Buenos Aires, Argentina) (ICSE ’17). IEEE Press, Piscataway, NJ, USA, 347–357. https://doi.o...

    2017 doi

  89. [98]

    Yakun Zhang et al. 2024. Learning-based Widget Matching for Migrating GUI Test Cases. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering (Lisbon, Portugal) (ICSE ’24). Association for Com- puting Machinery, New York, NY, USA, Article 69, 13 pa...

    2024

  90. [99]

    Yakun Zhang et al. 2024. LLM-based Abstraction and Concretization for GUI Test Migration. arXiv:2409.05028 [cs.SE] https://arxiv.org/abs/2409.05028

    2024 arXiv

  91. [100]

    Yakun Zhang et al. 2024. Synthesis-Based Enhancement for GUI Test Case Migration. In Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis . 869–881

    2024

  92. [101]

    Yu Zhao et al. 2019. Recdroid: automatically reproducing android application crashes from bug reports. In 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE) . IEEE, 128–139

    2019

  93. [102]

    Yixue Zhao, Justin Chen, Adriana Sejfia, Marcelo Schmitt Laser, Jie Zhang, Federica Sarro, Mark Harman, and Nenad Medvidovic. 2020. FrUITeR: a framework for evaluating UI test reuse. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and S...

    2020

  94. [103]

    Yu Zhao, Kye Miller, Tingting Yu, Wei Zheng, and Minchao Pu. 2019. Automatically extracting bug reproducing steps from android bug reports. In Reuse in the Big Data Era: 18th International Conference on Software and Systems Reuse, ICSR 2019, Cincinnati, OH, USA, June 26–28, 20...

    2019

  95. [104]

    Boyuan Zheng et al. 2024. Gpt-4v (ision) is a generalist web agent, if grounded. arXiv preprint arXiv:2401.01614 (2024). Received 2024-10-31; accepted 2025-03-31 Proc. ACM Softw. Eng., Vol. 2, No. ISSTA, Article ISSTA098. Publication date: July 2025

    2024 arXiv

  96. [2017]

    In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering

    Guided, stochastic model-based GUI testing of Android apps. In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering . 245–256. Proc. ACM Softw. Eng., Vol. 2, No. ISSTA, Article ISSTA098. Publication date: July 2025. ISSTA098:24 Benyamin Beyzaei, Sa...

    2017

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.