Pith. sign in

REVIEW 4 major objections 5 minor 59 references

Agent for User: Testing Multi-User Interactive Features in TikTok

T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A team of LLM agents, each playing a user on its own virtual phone, can automate multi-user interactive feature tests in TikTok from a plain-language task description.

desk verdict A useful industrial multi-agent LLM system for multi-user app testing, but its success rate depends on an input-format assumption that the paper has not yet validated. read the letter →

arxiv 2504.15474 v1 pith:I64C4PLI submitted 2025-04-21 cs.SE

Sidong Feng , Changhao Du , Huaxiao Liu , Qingnan Wang , Zhengwei Lv , Gang Huo , Xu Yang , Chunyang Chen This is my paper · ORCID
classification cs.SE
keywords multi-agentLLMmulti-userinteractivefeaturesAndroidapptestingvirtualdevicefarmGUItaskautomationLLM-drivenagentsTikTok
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes that the hard part of testing multi-user interactive features—like live streams and voice calls—can be handed to a team of LLM-driven agents, one per virtual device, each impersonating a user. Given a high-level task description, the system allocates devices, splits the task per user, and lets the agents click through the GUI in a coordinated order. On 24 TikTok tasks the approach completes 75% of them and matches 85.9% of the reference action sequence, beating three baselines and saving developers 87% of the time they would spend writing scripts. Integrated into TikTok's production testing flow for one month, it processed 3,318 task descriptions and helped surface 26 real multi-user interaction bugs.

What carries the argument

The load-bearing mechanism is the inter-agent handoff loop. Each LLM agent perceives its device screen as a simplified XML view hierarchy—keeping only resource id, class, clickable, text, and content-desc—and chooses from a five-action vocabulary: tap, input, back, switch user, and end task. When an agent's assigned user step is complete, it emits a [switch][user] signal to hand control to the next device's agent; the end-task signal stops the run. The virtual device farm supplies the required number of emulators, determined by a regex that counts distinct 'UserN' labels in the task description.

What would settle it

Take the same 24 tasks, strip the explicit 'UserN' labels, and re-run device allocation and task assignment on the paraphrased free-form text; if success rate falls well below 75%, the rule-based labeling assumption is confirmed as the load-bearing dependency.

Watch

Extended reading notes

Core claim

The central claim is that a natural-language multi-user task description can be turned into an executable, synchronized multi-device test without any hand-written script. Each user in the description is mapped to an LLM agent on its own emulated Android device; agents step through the GUI one action at a time, and when one finishes its sub-task it sends a switch-user signal so the next agent acts. The only ordering rule is that User 1 always moves first—the rest of the interleaving emerges from the agents' mutual interaction. The authors report that this design, rather than generic random exploration or single-agent LLM bug replay, is what lets the system reach 75% task success and 85.9% action similarity.

Load-bearing premise

The whole pipeline assumes every task description explicitly labels the users as 'User 1, User 2, ...' and that the first action always belongs to User 1, yet the paper gives no evidence that real-world descriptions, which average about 20 words, reliably follow that format.

Editorial extensions

If this is right

  • Writing a multi-device automation script drops from about 20 minutes per task to 2.46 minutes with the automated pipeline, an 87% time saving for developers.
  • The single rule that User 1 acts first, with later order emerging from agent handoffs, removes the need to pre-plan the full interaction sequence in the script.
  • Because the agents read the same XML accessibility data any Android app exposes, the approach can in principle be pointed at other social apps without changing the agent design.
  • Running the agents on emulated devices in a virtual farm, rather than physical phones, lowers the cost of multi-device testing and makes it practical to run in CI/CD loops.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the user-label regex is the only device-counting signal, then the approach's reach depends on task descriptions being written with explicit 'UserN' markers; inferring roles from free-form text is an obvious next step the paper does not address.
  • The reported failure causes—sub-second timing constraints, cross-app jumps, and ambiguous GUI widgets—suggest that pairing the LLM with a low-latency local model or adding a tiny rule-based fast path for time-critical taps could lift the 75% ceiling.
  • The [switch] handoff is a generic coordination primitive; the same design could test multiplayer games, collaborative editing, or account-sync scenarios that also require ordered actions across devices.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a multi-agent LLM framework for automating tests of multi-user interactive features in the TikTok app. The approach comprises two main phases: a virtual device farm that allocates devices based on the number of users extracted from a task description, and a multi-agent automation phase where each device is controlled by an LLM-based agent that navigates the GUI and uses explicit actions such as tap, input, back, switch user, and end task. The authors evaluate the approach on 24 curated multi-user interactive tasks, reporting a 75% success rate, 85.9% action similarity, and 87% time savings compared with professional developers. They also report integrating the system into TikTok's Fastbot-based testing platform, where it processed 3,318 task descriptions and helped identify 26 bugs.

Significance. If the reported results withstand scrutiny, the paper addresses a genuine and under-served problem: multi-user interactive features constitute a substantial portion of real-world app testing tasks, and existing automated app testing tools are largely single-user oriented. The industrial deployment and the 26 reported bugs are a meaningful practical signal, and the failure analysis in Section IV-A is honest and identifies concrete causes (timing, cross-app interaction, ambiguous GUI). The paper does not ship a replication package or detailed per-task data, so the evidence is currently preliminary. The central contribution is an empirical system, not a formal derivation, so the main risks are external validity and evaluation rigor rather than internal circularity.

major comments (4)
  1. [Section III-A2 and III-B1, Table I] The first two stages of the pipeline are rule-based and depend on the task description containing explicit 'UserN' labels. Device allocation uses a regex pattern 'User[1-9]', task assignment uses pattern recognition around 'UserX', and Section III-B1 states that the initial action would invariably originate from User1. Every task in Table I is pre-formatted as an explicit 'User1: ...; User2: ...' sequence, so the rules are satisfied by construction. The paper does not report how many of the 7,870 real-world task descriptions discussed in Section II-B actually contain explicit user labels, whether they start with User1, or whether they use single-digit user numbers. If descriptions are free-form, omit labels, or begin with a different user, the allocator and segmenter fail before any LLM inference occurs. The RQ1 and RQ3 claims are therefore conditional on an input-formatting assumption that is asserted but not demonstrated. Please report the prevalence of the required format in the real corpus and evaluate on non-preformatted descriptions.
  2. [Section IV-A, Table II] The statement that the approach 'significantly outperforms' the baselines is not supported by any statistical analysis. Table II reports mean success rates and mean action similarities over three runs, with no per-task outcome matrix, no standard deviations or ranges, and no significance test. Because the success metric is binary over 24 tasks, three runs produce very limited evidence for the 75% and 85.9% figures. In addition, Monkey and Humanoid are random exploration tools not designed to execute a given task, and the paper does not compute action similarity for them, so the 30.3% improvement comparison applies only to AdbGPT. Please provide run-level data, confidence intervals or exact per-task results, and appropriate statistical tests, and clearly separate task-driven baselines from random-exploration baselines when interpreting the comparison.
  3. [Section IV-B, RQ2] The 87% time-saving claim rests on an underspecified comparison. The paper reports an average of 2.46 minutes for the automated approach and 20.02 minutes for two developers who wrote scripts for the 24 tasks, but it does not report the distribution of per-task times, the number of paired observations used in the Mann-Whitney U test, or whether the automated time includes environment setup, LLM inference retries, and manual verification of the generated actions. Because the developers' manual scripting and the automated approach consume different inputs and workflows, the comparison is meaningful, but the missing variance and setup-cost details make the magnitude of the claimed savings difficult to assess.
  4. [Section IV-C, RQ3] The practicality claim of detecting 26 bugs is not quantified rigorously. The paper states that 3,318 task descriptions were run, that the completion rate was 70.09%, and that failing instances were manually inspected to identify 26 bugs, but it gives no baseline, no inter-rater agreement for bug identification, no reproduction or severity information, and no false-positive count. It is also unclear whether these bugs would have been discovered by the existing Fastbot-based process without the multi-agent automation layer. Please provide a breakdown of the 26 bugs by type, the number of failure cases examined, how bugs were confirmed, and a comparison with the prior testing process.
minor comments (5)
  1. [Abstract and Conclusion] The abstract says 'aiding in the detection of 26 multi-user interactive bugs,' while the conclusion says 'improving 26 bug detection'; the latter appears to be a wording error and should be made consistent.
  2. [Section II-B] The sentence describing the curated 24 tasks states that each task is structured as '<User1: ...; User2: ...>', but the preceding prevalence analysis of 7,870 descriptions does not clarify whether those original descriptions already followed this format; this is directly related to Major Comment 1 and should be clarified in the text.
  3. [Section III-C] The phrase 'we assign separate LLM instances of each agent' should read 'to each agent'; this is a grammatical typo.
  4. [Section IV-A] The paper does not release the 24 task descriptions, ground-truth traces, or the exact prompts used; making these artifacts available would substantially improve the reproducibility of the evaluation.
  5. [Section IV-A] The claim that the tasks were 'freshly identified and labeled by human annotators specifically for this study, mitigating the potential bias for data leakage' is only partially convincing, since a model like GPT-4 may still have knowledge of common TikTok GUI patterns; a more direct check would be to test on task descriptions that are semantically similar but use different wording.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reported success rate is an empirical outcome of a deployed LLM pipeline, not a quantity derived from fitted inputs or load-bearing self-citations.

full rationale

This paper is an empirical systems paper rather than a derivation, and I find no circular step that reduces a claimed prediction to its own inputs. The central quantitative claims—75% task success, 85.9% action similarity, 87% time savings, and 26 industrial bugs—are measured results of running a GPT-4-based multi-agent pipeline on 24 TikTok tasks and on 3,318 real-world task descriptions, with ground-truth traces collected by two authors; none of these values is obtained by fitting a parameter to the target quantity or by renaming an input. The regex-based device allocation and rule-based task assignment assume that task descriptions carry explicit 'UserN' labels and begin with User1; Table I tasks satisfy that format, and Section III-B1 states the User1-first rule. This is an input-format scope condition, not a construction that makes success tautological, because the LLM agents must still discover the correct GUI actions. The evaluation uses the authors' own AdbGPT as one baseline, but it also uses external Monkey and Humanoid baselines, and no load-bearing claim is justified solely by a self-citation. The industrial deployment result is an external outcome reported from TikTok's testing platform, not a derivation from the paper's own assumptions. Hence no specific circular reduction can be quoted, and the appropriate finding is no circularity.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the reliability of GPT-4 for GUI action inference, the structured 'UserX' task format, the faithfulness of the virtual device farm, and the correctness of manually produced ground-truth traces. None of these are proven outside the paper's own evaluation. No numerical parameters are fitted to data, and no new physical entities are introduced.

assumptions (4)
  • domain assumption GPT-4 can infer the correct next GUI action from a simplified XML hierarchy and a short task prompt.
    The entire Multi-agent Task Automation phase (Section III-B2 and Figure 4) depends on this capability; the only support is the 24-task evaluation, and the paper itself reports failures due to ambiguous GUIs.
  • domain assumption Task descriptions always follow the 'UserX' pattern and the initiating action always comes from User1.
    Device allocation (Section III-A2) and task assignment (Section III-B1) use regex and rule-based segmentation that assume this format, based on a pilot study; no evidence is given for general free-form descriptions.
  • domain assumption The virtual device farm faithfully reproduces physical device behavior for the tested features.
    Section III-A1 replaces a physical farm with ARM emulators; fidelity is asserted from prior load testing [15] rather than from app-behavior validation.
  • domain assumption The manually constructed ground-truth execution traces are correct and complete.
    Action similarity in RQ1 is measured against traces created by two authors (Section IV-A), and the RQ3 bug count depends on manual monitoring (Section IV-C).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Agent for User: Testing Multi-User Interactive Features in TikTok." pith.science (2026). https://pith.science/paper/I64C4PLI

@misc{pith2026250415474,
  author       = {Pith},
  title        = {Pith review of: Agent for User: Testing Multi-User Interactive Features in TikTok},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/I64C4PLI}},
  note         = {Machine review of arXiv:2504.15474}
}
read the original abstract

TikTok, a widely-used social media app boasting over a billion monthly active users, requires effective app quality assurance for its intricate features. Feature testing is crucial in achieving this goal. However, the multi-user interactive features within the app, such as live streaming, voice calls, etc., pose significant challenges for developers, who must handle simultaneous device management and user interaction coordination. To address this, we introduce a novel multi-agent approach, powered by the Large Language Models (LLMs), to automate the testing of multi-user interactive app features. In detail, we build a virtual device farm that allocates the necessary number of devices for a given multi-user interactive task. For each device, we deploy an LLM-based agent that simulates a user, thereby mimicking user interactions to collaboratively automate the testing process. The evaluations on 24 multi-user interactive tasks within the TikTok app, showcase its capability to cover 75% of tasks with 85.9% action similarity and offer 87% time savings for developers. Additionally, we have also integrated our approach into the real-world TikTok testing platform, aiding in the detection of 26 multi-user interactive bugs.

Figures

Figures reproduced from arXiv: 2504.15474 by the authors.

Figure 1
Figure 1. Illustration of LIVE together. testing tasks within TikTok, as discussed in Section II, reveals that 40.31% of these tasks pertain to the interactions between multiple users. For instance, as depicted in [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Example of the script for multi-user feature testing. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. The overview of our approach [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: The example of prompting agent. the task to identify discrete components that can be indepen￾dently executed by different devices. We employ rule-based pattern recognition to identify tasks associated with specific “UserX”, where “X” represents the user index. For inst…
Figure 5
Figure 5. Figure 5: Failure examples of our approach. Baselines. We set up three state-of-the-art methods as baselines for comparison with our approach. These methods include one task-driven (AdbGPT) and two random-based (Monkey, Humanoid), all commonly utilized in automated app testing. …
Figure 6
Figure 6. Figure 6: Comparison of time performance mutual discussion. To ensure an equitable comparison with our automated approach, we record the time they spent from the moment they began writing a task automation script to the point where the script was fully automated on devices. Resu…
Figure 7
Figure 7. Figure 7: Illustration of interactive bug. the internal developer-friendly system to identify functional bugs. We set up the number of bugs as the evaluation metric to assess the practicality of our approach. Results. We run the experiment in their testing system, with over 3,31…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

59 extracted references · 48 canonical work pages

  1. [1]

    An empirical study on leveraging images in automated bug report reproduction,

    D. Wang, Z. Zhang, S. Feng, W. G. Halfond, and T. Yu, “An empirical study on leveraging images in automated bug report reproduction,” arXiv preprint arXiv:2502.15099, 2025

    arXiv 2025

  2. [2]

    Gifdroid: Automated replay of visual bug reports for android apps,

    S. Feng and C. Chen, “Gifdroid: Automated replay of visual bug reports for android apps,” in Proceedings of the 44th International Conference on Software Engineering , 2022, pp. 1045–1057

    work page 2022

  3. [3]

    Auto-icon: An automated code generation tool for icon designs assisting in ui development,

    S. Feng, S. Ma, J. Yu, C. Chen, T. Zhou, and Y . Zhen, “Auto-icon: An automated code generation tool for icon designs assisting in ui development,” in Proceedings of the 26th International Conference on Intelligent User Interfaces , 2021, pp. 59–69

    work page 2021

  4. [4]

    Read it, don’t watch it: Captioning bug recordings automatically,

    S. Feng, M. Xie, Y . Xue, and C. Chen, “Read it, don’t watch it: Captioning bug recordings automatically,” in 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE). IEEE, 2023, pp. 2349–2361

    work page 2023

  5. [5]

    Gifdroid: an automated light-weight tool for replaying visual bug reports,

    S. Feng and C. Chen, “Gifdroid: an automated light-weight tool for replaying visual bug reports,” in Proceedings of the ACM/IEEE 44th International Conference on Software Engineering: Companion Pro- ceedings, 2022, pp. 95–99

    work page 2022

  6. [6]

    Agent-based software testing: A definition and systematic mapping study,

    P. P. Kumaresen, M. Frasheri, and E. P. Enoiu, “Agent-based software testing: A definition and systematic mapping study,” in 2020 IEEE 20th International Conference on Software Quality, Reliability and Security Companion (QRS-C). IEEE, 2020, pp. 24–31

    work page 2020

  7. [7]

    Agent-oriented approaches for model-based software testing: A mapping study,

    J. Ram ´ırez-M´endez, C. Quesada-L ´opez, A. Mart ´ınez, and M. Jenk- ins, “Agent-oriented approaches for model-based software testing: A mapping study,” in Information Technology and Systems: ICITS 2021, Volume 1. Springer, 2021, pp. 340–349

    work page 2021

  8. [8]

    Meet claude,

    “Meet claude,” https://www.anthropic.com/claude, 2024

    work page 2024

Show all 59 references
  1. [9]

    Droidbot: a lightweight ui-guided test input generator for android,

    Y . Li, Z. Yang, Y . Guo, and X. Chen, “Droidbot: a lightweight ui-guided test input generator for android,” in 2017 IEEE/ACM 39th International Conference on Software Engineering Companion (ICSE-C) . IEEE, 2017, pp. 23–26

    2017

  2. [10]

    Ui/application exerciser monkey,

    “Ui/application exerciser monkey,” https://developer.android.com/studio/ test/other-testing-tools/monkey, 2024

    2024

  3. [11]

    Make llm a testing expert: Bringing human-like interaction to mobile gui testing via functionality-aware decisions,

    Z. Liu, C. Chen, J. Wang, M. Chen, B. Wu, X. Che, D. Wang, and Q. Wang, “Make llm a testing expert: Bringing human-like interaction to mobile gui testing via functionality-aware decisions,” arXiv preprint arXiv:2310.15780, 2023

    arXiv

  4. [12]

    Agent-based software engineering,

    M. Wooldridge, “Agent-based software engineering,” IEE Proceedings- software, vol. 144, no. 1, pp. 26–37, 1997

    1997

  5. [13]

    Amazon.com: Aws device farm,

    “Amazon.com: Aws device farm,” https://aws.amazon.com/device-farm/, 2024

    2024

  6. [14]

    Google.com: Google firebase test lab,

    “Google.com: Google firebase test lab,” https://firebase.google.com/ products/test-lab, 2024

    2024

  7. [15]

    Virtual device farms for mobile app testing at scale: A pursuit for fidelity, efficiency, and accessibility,

    H. Lin, J. Qiu, H. Wang, Z. Li, L. Gong, D. Gao, Y . Liu, F. Qian, Z. Zhang, P. Yang et al., “Virtual device farms for mobile app testing at scale: A pursuit for fidelity, efficiency, and accessibility,” in Proceedings of the 29th Annual International Conference on Mobile Comp...

    2023

  8. [16]

    Top 10 cloud service providers globally in 2022,

    “Top 10 cloud service providers globally in 2022,” https://dgtlinfra.com/ top-10-cloud-service-providers-2022, 2024

    2022

  9. [17]

    Dynamic binary translation,

    M. Probst, “Dynamic binary translation,” in UKUUG Linux Developer’s Conference, vol. 2002, 2002

    2002

  10. [18]

    Sleight of arm: Demystifying intel houdini,

    “Sleight of arm: Demystifying intel houdini,” https://talks.toorcon.net/ toorcon-2021/talk/WRQEVD/, 2024

    2021

  11. [19]

    Android: Wifimanager,

    “Android: Wifimanager,” https://cs.android.com/android/platform/ superproject/+/master:packages/modules/Wifi/framework/java/android/ net/wifi/WifiManager.java, 2024

    2024

  12. [20]

    Android: Dctracker,

    “Android: Dctracker,” https://cs.android.com/android/platform/ superproject/+/master:frameworks/opt/telephony/src/java/com/android/ internal/telephony/dataconnection/DcTracker.java, 2024

    2024

  13. [21]

    The dawn of lmms: Preliminary explorations with gpt-4v (ision),

    Z. Yang, L. Li, K. Lin, J. Wang, C.-C. Lin, Z. Liu, and L. Wang, “The dawn of lmms: Preliminary explorations with gpt-4v (ision),” arXiv preprint arXiv:2309.17421, vol. 9, no. 1, p. 1, 2023

    2023 arXiv

  14. [22]

    Google accessibilitynodeinfo,

    “Google accessibilitynodeinfo,” https://developer.android.com/reference/ android/view/accessibility/AccessibilityNodeInfo, 2024

    2024

  15. [23]

    Prompting is all your need: Automated android bug replay with large language models,

    S. Feng and C. Chen, “Prompting is all your need: Automated android bug replay with large language models,” arXiv preprint arXiv:2306.01987, 2023

    2023 arXiv

  16. [24]

    Mud: Towards a large-scale and noise-filtered ui dataset for modern style ui modeling,

    S. Feng, S. Ma, H. Wang, D. Kong, and C. Chen, “Mud: Towards a large-scale and noise-filtered ui dataset for modern style ui modeling,” in Proceedings of the CHI Conference on Human Factors in Computing Systems, 2024, pp. 1–14

    2024

  17. [25]

    Language mod- els are few-shot learners,

    T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al., “Language mod- els are few-shot learners,” Advances in neural information processing systems, vol. 33, pp. 1877–1901, 2020

    1901

  18. [26]

    Android debug bridge (adb) - android developers,

    “Android debug bridge (adb) - android developers,” https://developer. android.com/studio/command-line/adb, 2024

    2024

  19. [27]

    Introducing chatgpt,

    “Introducing chatgpt,” https://chat.openai.com/, 2024

    2024

  20. [28]

    Android uiautomator2 python wrapper,

    “Android uiautomator2 python wrapper,” https://github.com/openatx/ uiautomator2, 2024

    2024

  21. [29]

    Video2action: Reducing human interactions in action annotation of app tutorial videos,

    S. Feng, C. Chen, and Z. Xing, “Video2action: Reducing human interactions in action annotation of app tutorial videos,” in Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, 2023, pp. 1–15

    2023

  22. [30]

    Prompting is all you need: Automated android bug replay with large language models,

    S. Feng and C. Chen, “Prompting is all you need: Automated android bug replay with large language models,” in Proceedings of the 46th IEEE/ACM International Conference on Software Engineering , 2024, pp. 1–13

    2024

  23. [31]

    Humanoid: A deep learning- based approach to automated black-box android app testing,

    Y . Li, Z. Yang, Y . Guo, and X. Chen, “Humanoid: A deep learning- based approach to automated black-box android app testing,” in 2019 34th IEEE/ACM International Conference on Automated Software En- gineering (ASE). IEEE, 2019, pp. 1070–1073

    2019

  24. [32]

    Wilcoxon-mann-whitney or t-test? on assumptions for hypothesis tests and multiple interpretations of decision rules,

    M. P. Fay and M. A. Proschan, “Wilcoxon-mann-whitney or t-test? on assumptions for hypothesis tests and multiple interpretations of decision rules,” Statistics surveys, vol. 4, p. 1, 2010

    2010

  25. [33]

    Fastbot2: Reusable automated model-based gui testing for android enhanced by reinforcement learning,

    Z. Lv, C. Peng, Z. Zhang, T. Su, K. Liu, and P. Yang, “Fastbot2: Reusable automated model-based gui testing for android enhanced by reinforcement learning,” in Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering, 2022, pp. 1–5

    2022

  26. [34]

    Continuous deployment at facebook and oanda,

    T. Savor, M. Douglas, M. Gentili, L. Williams, K. Beck, and M. Stumm, “Continuous deployment at facebook and oanda,” in Proceedings of the 38th International Conference on software engineering companion , 2016, pp. 21–30

    2016

  27. [35]

    Test selection for unified regression testing,

    S. Wang, X. Lian, D. Marinov, and T. Xu, “Test selection for unified regression testing,” in 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) . IEEE, 2023, pp. 1687–1699

    2023

  28. [36]

    Unveiling the tricks: Automated detection of dark patterns in mobile applications,

    J. Chen, J. Sun, S. Feng, Z. Xing, Q. Lu, X. Xu, and C. Chen, “Unveiling the tricks: Automated detection of dark patterns in mobile applications,” in Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, 2023, pp. 1–20

    2023

  29. [37]

    Sapienz: Multi-objective automated testing for android applications,

    K. Mao, M. Harman, and Y . Jia, “Sapienz: Multi-objective automated testing for android applications,” inProceedings of the 25th International Symposium on Software Testing and Analysis , 2016, pp. 94–105

    2016

  30. [38]

    Droidfuzzer: Fuzzing the android apps with intent-filter tag,

    H. Ye, S. Cheng, L. Zhang, and F. Jiang, “Droidfuzzer: Fuzzing the android apps with intent-filter tag,” in Proceedings of International Conference on Advances in Mobile Computing & Multimedia , 2013, pp. 68–74

    2013

  31. [39]

    Guided gui testing of android apps with minimal restart and approximate learning,

    W. Choi, G. Necula, and K. Sen, “Guided gui testing of android apps with minimal restart and approximate learning,” Acm Sigplan Notices , vol. 48, no. 10, pp. 623–640, 2013

    2013

  32. [40]

    Practical gui testing of android applications via model abstraction and refinement,

    T. Gu, C. Sun, X. Ma, C. Cao, C. Xu, Y . Yao, Q. Zhang, J. Lu, and Z. Su, “Practical gui testing of android applications via model abstraction and refinement,” in 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE) . IEEE, 2019, pp. 269–280

    2019

  33. [41]

    Psychologically- inspired, unsupervised inference of perceptual groups of gui widgets from gui images,

    M. Xie, Z. Xing, S. Feng, X. Xu, L. Zhu, and C. Chen, “Psychologically- inspired, unsupervised inference of perceptual groups of gui widgets from gui images,” in Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Soft...

    2022

  34. [42]

    Uied: a hybrid tool for gui element detection,

    M. Xie, S. Feng, Z. Xing, J. Chen, and C. Chen, “Uied: a hybrid tool for gui element detection,” in Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering , 2020, pp. 1655–1659

    2020

  35. [43]

    Automated concolic testing of smartphone apps,

    S. Anand, M. Naik, M. J. Harrold, and H. Yang, “Automated concolic testing of smartphone apps,” in Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering , 2012, pp. 1–11

    2012

  36. [44]

    Gallery dc: Auto-created gui component gallery for design search and knowledge discovery,

    S. Feng, C. Chen, and Z. Xing, “Gallery dc: Auto-created gui component gallery for design search and knowledge discovery,” in Proceedings of the ACM/IEEE 44th International Conference on Software Engineering: Companion Proceedings, 2022, pp. 80–84

    2022

  37. [45]

    Gallery dc: Design search and knowledge discovery through auto-created gui component gallery,

    C. Chen, S. Feng, Z. Xing, L. Liu, S. Zhao, and J. Wang, “Gallery dc: Design search and knowledge discovery through auto-created gui component gallery,” Proceedings of the ACM on Human-Computer Interaction, vol. 3, no. CSCW, pp. 1–22, 2019

    2019

  38. [46]

    Feedback-driven automated whole bug report reproduction for android apps,

    D. Wang, Y . Zhao, S. Feng, Z. Zhang, W. G. Halfond, C. Chen, X. Sun, J. Shi, and T. Yu, “Feedback-driven automated whole bug report reproduction for android apps,” in Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis , 2024, pp. 1048–1060

    2024

  39. [47]

    Reran: Timing-and touch-sensitive record and replay for android,

    L. Gomez, I. Neamtiu, T. Azim, and T. Millstein, “Reran: Timing-and touch-sensitive record and replay for android,” in 2013 35th Interna- tional Conference on Software Engineering (ICSE) . IEEE, 2013, pp. 72–81

    2013

  40. [48]

    Towards efficient record and replay: A case study in wechat,

    S. Feng, H. Lu, T. Xiong, Y . Deng, and C. Chen, “Towards efficient record and replay: A case study in wechat,” in Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering , 2023, pp. 1681–1692

    2023

  41. [49]

    Efficiency matters: Speeding up automated testing with gui rendering inference,

    S. Feng, M. Xie, and C. Chen, “Efficiency matters: Speeding up automated testing with gui rendering inference,” in 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) . IEEE, 2023, pp. 906–918

    2023

  42. [50]

    Sikuli: using gui screenshots for search and automation,

    T. Yeh, T.-H. Chang, and R. C. Miller, “Sikuli: using gui screenshots for search and automation,” in Proceedings of the 22nd annual ACM symposium on User interface software and technology , 2009, pp. 183– 192

    2009

  43. [51]

    How do developers test android applications?

    M. Linares-V ´asquez, C. Bernal-C ´ardenas, K. Moran, and D. Poshy- vanyk, “How do developers test android applications?” in 2017 IEEE International Conference on Software Maintenance and Evolution (IC- SME). IEEE, 2017, pp. 613–622

    2017

  44. [52]

    Appflow: using machine learning to synthesize robust, reusable ui tests,

    G. Hu, L. Zhu, and J. Yang, “Appflow: using machine learning to synthesize robust, reusable ui tests,” in Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, 2018, pp. 269– 282

    2018

  45. [53]

    Enabling cost-effective ui automation testing with retrieval-based llms: A case study in wechat,

    S. Feng, H. Lu, J. Jiang, T. Xiong, L. Huang, Y . Liang, X. Li, Y . Deng, and A. Aleti, “Enabling cost-effective ui automation testing with retrieval-based llms: A case study in wechat,” in Proceedings of the 39th IEEE/ACM International Conference on Automated Software Enginee...

    2024

  46. [54]

    Agent-based test management for software system test,

    C. Malz and N. Jazdi, “Agent-based test management for software system test,” in 2010 IEEE International Conference on Automation, Quality and Testing, Robotics (AQTR) , vol. 2. IEEE, 2010, pp. 1–6

    2010

  47. [55]

    A systematic review of agent-based test case generation for regression testing,

    P. K. Arora and R. Bhatia, “A systematic review of agent-based test case generation for regression testing,” Arabian Journal for Science and Engineering, vol. 43, no. 2, pp. 447–470, 2018

    2018

  48. [56]

    Test agents: The next generation of test cases,

    E. Enoiu and M. Frasheri, “Test agents: The next generation of test cases,” in 2019 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW). IEEE, 2019, pp. 305– 308

    2019

  49. [57]

    Towards automation in software test life cycle based on multi- agent,

    J. Tang, “Towards automation in software test life cycle based on multi- agent,” in 2010 International Conference on Computational Intelligence and Software Engineering . IEEE, 2010, pp. 1–4

    2010

  50. [58]

    A new approach in development of distributed framework for automated software testing using agents,

    P. Dhavachelvan, G. Uma, and V . Venkatachalapathy, “A new approach in development of distributed framework for automated software testing using agents,” Knowledge-Based Systems, vol. 19, no. 4, pp. 235–247, 2006

    2006

  51. [59]

    Multi-agent-based integrated framework for intra-class testing of object-oriented software,

    P. Dhavachelvan and G. Uma, “Multi-agent-based integrated framework for intra-class testing of object-oriented software,” Applied Soft Com- puting, vol. 5, no. 2, pp. 205–222, 2005

    2005

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.