Pith. sign in

REVIEW 2 major objections 1 minor 54 references

Reviewed by Pith at T0; open to challenge.

T0 means a machine referee read the full paper against a public rubric. The mark states how deep the mechanical check went, never who wrote it. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

WebCQ uses deep multi-agent RL with feature-based action vectors to explore more states and actions in large web GUIs than tabular methods.

2026-06-26 09:52 UTC pith:54AGMKBR

load-bearing objection WebCQ adds QTRAN coordination and feature-concatenated DQN to MARL web testing and reports clear gains over MARG on real sites, but the dynamic action space mechanism is not shown in enough detail to explain the results. the 2 major comments →

arxiv 2606.22502 v1 pith:54AGMKBR submitted 2026-06-21 cs.SE

WebCQ: Cooperative Multi-Agent Deep Reinforcement Learning for Scalable Web GUI Testing

Yujia Fan , Sinan Wang , Zebang Fei , Yao Qin , Huaxuan Li , Yepang Liu This is my paper
classification cs.SE
keywords web GUI testingmulti-agent reinforcement learningdeep Q-networkGUI explorationscalable testingaction vectorfailure triggeringasynchronous coordination
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces WebCQ to overcome scaling limits in web GUI testing where prior multi-agent reinforcement learning approaches rely on tabular algorithms that cannot handle large state spaces. It combines QTRAN coordination with a DQN policy that receives action vectors built from semantic and exploration features of UI events. Under fixed time and agent budgets this produces higher state coverage and more unique actions across commercial sites. The approach also includes a synchronization mechanism suited to asynchronous web testing. If the method works as described, testing tools could reach more interactive elements and surface more failures without requiring proportionally more resources.

Core claim

WebCQ incorporates QTRAN for multi-agent coordination and a lightweight synchronization mechanism allowing it to work under asynchronous web testing scenarios. It extracts semantic and exploration features for each UI event to form an action vector. This vector is concatenated with the current state vector and fed into the policy network, enabling DQN-based decision making within a dynamic action space. On eight large-scale commercial websites, under the same time budget and agent count, WebCQ explored 33.3 percent more states and executed 42.2 percent more unique actions than MARG while triggering more failures on six of the eight sites. It maintained higher action throughput during twenty-

What carries the argument

The action vector formed by concatenating semantic and exploration features of each UI event, concatenated with the state vector and passed to a DQN policy network under QTRAN coordination.

Load-bearing premise

The assumption that concatenating semantic and exploration features into an action vector and feeding it to a DQN policy network will produce effective decisions in the dynamic action spaces of real web applications.

What would settle it

A controlled comparison on one of the eight websites or a similar large site in which WebCQ under the same time budget and agent count explores fewer states or executes fewer unique actions than MARG.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 1 minor

Summary. The paper proposes WebCQ, a MARL approach for web GUI testing that combines QTRAN for multi-agent coordination with a DQN policy network. Semantic and exploration features are extracted per UI event to form action vectors, which are concatenated with the state vector and passed to the DQN for decisions in dynamic action spaces. On eight commercial websites, under fixed time budgets and agent counts, it reports exploring 33.3% more states and executing 42.2% more unique actions than MARG, triggering more failures on six sites, and showing improved throughput and scaling with more agents in 20-hour runs.

Significance. If the empirical gains and architectural choices hold under rigorous controls, WebCQ would address a practical scalability gap in MARL-based GUI testing by moving beyond tabular methods, offering a concrete path to higher coverage on large web applications.

major comments (2)
  1. [Abstract / policy network description] The description of the policy network (abstract and method) states that per-event feature vectors are concatenated with the state and fed to DQN for dynamic action spaces, yet provides no equation, pseudocode, or ablation addressing how variable-length action sets are managed (e.g., action masking, per-action heads, or invalid-action filtering). Standard DQN assumes a fixed discrete output dimension; without this mechanism the reported 33.3% state and 42.2% action gains cannot be attributed to the coordination or feature design rather than baseline or environment artifacts.
  2. [Evaluation] The evaluation section reports quantitative improvements but supplies no information on experimental controls, statistical significance tests, how failures were defined and counted, or whether the eight websites and 20-hour scalability runs used identical configurations across methods. These omissions make the central performance claims unverifiable from the given text.
minor comments (1)
  1. [Abstract] The abstract contains a typographical error: 'WebCQovercomes' should be 'WebCQ overcomes'.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments, which highlight important areas for improving the clarity and rigor of our manuscript. We address each major comment below and will revise the paper accordingly.

read point-by-point responses

  1. Referee: [Abstract / policy network description] The description of the policy network (abstract and method) states that per-event feature vectors are concatenated with the state and fed to DQN for dynamic action spaces, yet provides no equation, pseudocode, or ablation addressing how variable-length action sets are managed (e.g., action masking, per-action heads, or invalid-action filtering). Standard DQN assumes a fixed discrete output dimension; without this mechanism the reported 33.3% state and 42.2% action gains cannot be attributed to the coordination or feature design rather than baseline or environment artifacts.

    Authors: We agree that the manuscript's description of the policy network is insufficiently detailed regarding the handling of dynamic action spaces. The text states that action vectors are concatenated with the state vector and fed to DQN, but provides no equation, pseudocode, or explicit mechanism (such as masking) for variable-length sets. This omission prevents clear attribution of the reported gains. We will revise the method section to include a formal description of the input construction, the action selection process, and how invalid actions are filtered, along with any relevant implementation details from our experiments. revision: yes

  2. Referee: [Evaluation] The evaluation section reports quantitative improvements but supplies no information on experimental controls, statistical significance tests, how failures were defined and counted, or whether the eight websites and 20-hour scalability runs used identical configurations across methods. These omissions make the central performance claims unverifiable from the given text.

    Authors: We acknowledge that the evaluation section lacks necessary details on experimental controls, statistical significance, failure definitions, and confirmation of identical configurations across methods and runs. While the manuscript indicates that comparisons used the same time budget and agent count on eight commercial sites with 20-hour scalability experiments, these specifics are not elaborated. We will expand the evaluation section to document the experimental setup, failure criteria (e.g., how crashes or errors were identified), any statistical tests applied, and explicit statements confirming consistent configurations, thereby making the performance claims verifiable. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical comparisons rest on external benchmarks

full rationale

The paper's derivation chain consists of a proposed MARL architecture (QTRAN coordination plus DQN on concatenated semantic/exploration features) followed by direct experimental evaluation on eight commercial websites. Reported gains (33.3% more states, 42.2% more actions than MARG) are measured outcomes under fixed time/agent budgets, not predictions or quantities derived from fitted parameters. No equations, uniqueness theorems, or self-citations are invoked to force the central results; the evaluation is externally falsifiable against the websites and baseline. This satisfies the self-contained criterion with no load-bearing reductions to inputs.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract provides no explicit free parameters, axioms, or invented entities; all technical details are omitted.

pith-pipeline@v0.9.1-grok · 5806 in / 1095 out tokens · 18618 ms · 2026-06-26T09:52:38.331141+00:00 · methodology

0 comments
read the original abstract

Multi-agent reinforcement learning (MARL)-based techniques have shown promise for GUI testing. However, as the complexity of modern GUI software increases, existing MARL-based approaches (e.g., MARG and Fastbot) struggle to scale due to the inherent limitations of their underlying tabular reinforcement learning algorithms. This limits their applicability to large-scale commercial GUI software, especially web applications with vast state spaces and many interactive elements. To fill this gap, we propose WebCQ, a novel MARL-based approach for scalable web GUI testing. WebCQ incorporates QTRAN for multi-agent coordination and a lightweight synchronization mechanism, allowing it to work under asynchronous web testing scenarios. It extracts semantic and exploration features for each UI event to form an action vector. This vector is concatenated with the current state vector and fed into the policy network, enabling DQN-based decision making within a dynamic action space. We evaluated WebCQ on eight large-scale commercial websites. Under the same time budget and agent count, WebCQ explored 33.3% more states and executed 42.2% more unique actions than MARG, while triggering more failures on six of the eight websites under test. It also demonstrated strong scalability, maintaining higher action throughput during 20-hour experiments, and achieving greater performance improvements as the number of agents increased. These results show that WebCQovercomes key limitations of existing MARL-based approaches, providing a scalable and effective solution for enhancing modern web GUI testing.

Figures

Figures reproduced from arXiv: 2606.22502 by Huaxuan Li, Sinan Wang, Yao Qin, Yepang Liu, Yujia Fan, Zebang Fei.

Figure 1
Figure 1. Figure 1: QTRAN’s architecture [40] state-of-the-art performance in terms of exploration efficiency, failure-triggering effectiveness, and scalability. • To facilitate future research and industry practice, we have pub￾lished the source code of WebCQ and all experimental data in: https://doi.org/10.5281/zenodo.19222038. 2 Preliminaries 2.1 Reinforcement Learning Reinforcement learning (RL) studies how an autonomous … view at source ↗
Figure 2
Figure 2. Figure 2: An overview of WebCQ This allows the joint model to capture complex interactions among agents without being restricted by additivity or monotonicity as￾sumptions. Then, QTRAN’s loss function is defined as the weighted sum of three components: Ltd = E h (𝑟 +𝛾 max 𝒂 ′ 𝑄jt(𝒔 ′ , 𝒂 ′ ; 𝜽 − ) − 𝑄jt(𝒔, 𝒂))2 i Lopt = E h (max 𝒂 ′ 𝑄 ′ jt(𝒔, 𝒂 ′ ) − max 𝒂 ′ 𝑄ˆ jt(𝒔, 𝒂 ′ ) +𝑉jt(𝒔))2 i Lnopt = E h (min[𝑄 ′ jt(𝒔, 𝒂) −… view at source ↗
Figure 3
Figure 3. Figure 3: An example of tag-depth-based state vector [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 5
Figure 5. Figure 5: Hourly executed actions (Y-axis) of WebCQ and MARG on Gap over 20 hours, where solid lines represent the mean values of six independent runs, and shaded areas represent their standard deviations. MARG on the Gap website over 20 hours. At the beginning, both approaches exhibit a similar number of executed actions. This is because, in the initial phase, few states have been explored and the Q-table in MARG h… view at source ↗
Figure 6
Figure 6. Figure 6: Comparisons of WebCQ and MARG on Gap with increasing number of agents. actions. Although WebCQ exhibits wider variability across runs, its median values are generally higher than those of MARG, re￾flecting better overall effectiveness. This indicates that WebCQ can effectively leverage additional agents to enhance decision-making, mitigating the impact of communication overhead. An Exception oc￾curs at the… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

54 extracted references · 5 canonical work pages · 3 internal anchors

  1. [1]

    Release glove-wiki-gigaword-300·piskvorky/gensim-data

    2017. Release glove-wiki-gigaword-300·piskvorky/gensim-data. https://github. com/piskvorky/gensim-data/releases/tag/glove-wiki-gigaword-300. [Accessed 11-04-2025]

    2017

  2. [2]

    Top Websites in the World - Most Visited & Popular Rankings - Semrush — semrush.com

    2025. Top Websites in the World - Most Visited & Popular Rankings - Semrush — semrush.com. https://semrush.com/website/top/. [Accessed 01-04-2025]

    2025

  3. [3]

    W3Schools.com — w3schools.com

    2025. W3Schools.com — w3schools.com. https://www.w3schools.com/js/js_ htmldom.asp. [Accessed 19-05-2025]

    2025

  4. [4]

    Andrea Arcuri and Lionel Briand. 2014. A hitchhiker’s guide to statistical tests for assessing randomized algorithms in software engineering.Software Testing, Verification and Reliability24, 3 (2014), 219–250

    2014

  5. [5]

    Ishan Banerjee, Bao Nguyen, Vahid Garousi, and Atif Memon. 2013. Graphical user interface (GUI) testing: Systematic mapping and repository.Information and Software Technology55, 10 (2013), 1679–1694

    2013

  6. [6]

    Sebastian Bauersfeld and Tanja Vos. 2012. A reinforcement learning approach to automated gui robustness testing. InFast abstracts of the 4th symposium on search-based software engineering (SSBSE 2012). 7–12

    2012

  7. [7]

    Sebastian Bauersfeld and Tanja EJ Vos. 2014. User interface level testing with TESTAR; what about more sophisticated action specification and selection?. In SATToSE. 60–78

    2014

  8. [8]

    Matteo Biagiola, Filippo Ricca, and Paolo Tonella. 2017. Search based path and input data generation for web application testing. InSearch Based Software Engi- neering: 9th International Symposium, SSBSE 2017, Paderborn, Germany, September 9-11, 2017, Proceedings 9. Springer, 18–32

    2017

  9. [9]

    Matteo Biagiola, Andrea Stocco, Filippo Ricca, and Paolo Tonella. 2019. Diversity- based web test generation. InProceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 142–153

    2019

  10. [10]

    Tianqin Cai, Zhao Zhang, and Ping Yang. 2020. Fastbot: A Multi-Agent Model- Based Test Generation System Beijing Bytedance Network Technology Co., Ltd.. InProceedings of the IEEE/ACM 1st International Conference on Automation of Software Test. 93–96

    2020

  11. [11]

    Yuanchao Chen, Yuliang Lu, Zulie Pan, Juxing Chen, Fan Shi, Yang Li, and Yonghui Jiang. 2024. APIMiner: Identifying Web Application APIs Based on Web Page States Similarity Analysis.Electronics13, 6 (2024), 1112

    2024

  12. [12]

    Eliane Collins, Arilo Neto, Auri Vincenzi, and José Maldonado. 2021. Deep reinforcement learning based android application gui testing. InProceedings of the XXXV Brazilian Symposium on Software Engineering. 186–194

    2021

  13. [13]

    Jingjing Cui, Yuanwei Liu, and Arumugam Nallanathan. 2019. The Application of Multi-Agent Reinforcement Learning in UAV Networks. In2019 IEEE International Conference on Communications Workshops (ICC Workshops). 1–6. doi:10.1109/ ICCW.2019.8756984

    work page arXiv 2019

  14. [14]

    Chris Dann, Yishay Mansour, Mehryar Mohri, Ayush Sekhari, and Karthik Srid- haran. 2022. Guarantees for epsilon-greedy reinforcement learning with function approximation. InInternational conference on machine learning. PMLR, 4666– 4689

    2022

  15. [15]

    Yujia Fan, Sinan Wang, Zebang Fei, Yao Qin, Huaxuan Li, and Yepang Liu. 2024. Can Cooperative Multi-Agent Reinforcement Learning Boost Automatic Web Testing? An Exploratory Study. InProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering. 14–26

    2024

  16. [16]

    Yujia Fan, Siyi Wang, Sinan Wang, Yepang Liu, Guoyao Wen, and Qi Rong. 2023. A Comprehensive Evaluation of Q-Learning Based Automatic Web GUI Testing. In 2023 10th International Conference on Dependable Systems and Their Applications (DSA). IEEE, 12–23

    2023

  17. [17]

    Jakob Foerster, Nantas Nardelli, Gregory Farquhar, Triantafyllos Afouras, Philip HS Torr, Pushmeet Kohli, and Shimon Whiteson. 2017. Stabilising ex- perience replay for deep multi-agent reinforcement learning. InInternational conference on machine learning. PMLR, 1146–1155

    2017

  18. [18]

    Hui Guo, Ting Su, Xiaoqiang Liu, Siyi Gu, and Jingling Sun. 2023. Effectively Finding ICC-related Bugs in Android Apps via Reinforcement Learning. In2023 IEEE 34th International Symposium on Software Reliability Engineering (ISSRE). IEEE, 403–414

    2023

  19. [19]

    Jayesh K Gupta, Maxim Egorov, and Mykel Kochenderfer. 2017. Cooperative multi-agent control using deep reinforcement learning. InAutonomous Agents and Multiagent Systems: AAMAS 2017 Workshops, Best Papers, São Paulo, Brazil, May 8-12, 2017, Revised Selected Papers 16. Springer, 66–83

    2017

  20. [20]

    Kenish Rajesh Halani, Rahul Saxena, et al . 2021. Critical analysis of manual versus automation testing. In2021 International Conference on Computational Performance Evaluation (ComPE). IEEE, 132–135

    2021

  21. [21]

    Eric A Hansen, Daniel S Bernstein, and Shlomo Zilberstein. 2004. Dynamic programming for partially observable stochastic games. InAAAI, Vol. 4. 709– 715

    2004

  22. [22]

    Weinberger

    Gao Huang, Zhuang Liu, Laurens van der Maaten, and Kilian Q. Weinberger

  23. [23]

    Densely Connected Convolutional Networks

    Densely Connected Convolutional Networks. arXiv:1608.06993 [cs.CV] https://arxiv.org/abs/1608.06993

    work page internal anchor Pith review Pith/arXiv arXiv

  24. [24]

    Yavuz Koroglu, Alper Sen, Ozlem Muslu, Yunus Mete, Ceyda Ulker, Tolga Tan- riverdi, and Yunus Donmez. 2018. QBE: QLearning-based exploration of android applications. In2018 IEEE 11th International Conference on Software Testing, Veri- fication and Validation (ICST). IEEE, 105–115

    2018

  25. [25]

    Yuanhong Lan, Yifei Lu, Zhong Li, Minxue Pan, Wenhua Yang, Tian Zhang, and Xuandong Li. 2024. Deeply Reinforcing Android GUI Testing with Deep Rein- forcement Learning. InProceedings of the 46th IEEE/ACM International Conference on Software Engineering. 1–13

    2024

  26. [26]

    Tao Li, Chenhui Cui, Rubing Huang, Dave Towey, and Lei Ma. [n. d.]. Large Language Models for Automated Web-Form-Test Generation: An Empirical Study. ACM Transactions on Software Engineering and Methodology([n. d.])

  27. [27]

    Chenxu Liu, Junheng Wang, Wei Yang, Ying Zhang, and Tao Xie. [n. d.]. Judge: Effective State Abstraction for Guiding Automated Web GUI Testing.ACM Transactions on Software Engineering and Methodology([n. d.])

  28. [28]

    Leonardo Mariani, Mauro Pezzè, Oliviero Riganelli, and Mauro Santoro. 2011. AutoBlackTest: a tool for automatic black-box testing. InProceedings of the 33rd international conference on software engineering. 1013–1015

    2011

  29. [29]

    Leonardo Mariani, Mauro Pezzè, Oliviero Riganelli, and Mauro Santoro. 2014. Automatic testing of GUI-based applications.Software Testing, Verification and Reliability24, 5 (2014), 341–366

    2014

  30. [30]

    Ali Mesbah, Engin Bozdag, and Arie Van Deursen. 2008. Crawling Ajax by inferring user interface state changes. In2008 eighth international conference on web engineering. IEEE, 122–134

    2008

  31. [31]

    Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al. 2015. Human-level control through deep reinforcement learning. nature518, 7540 (2015), 529–533

    2015

  32. [32]

    Marco Mobilio, Diego Clerissi, Giovanni Denaro, and Leonardo Mariani. 2023. GUI Testing to the Power of Parallel Q-Learning. In2023 IEEE/ACM 45th International Conference on Software Engineering: Companion Proceedings (ICSE-Companion). IEEE, 55–59

    2023

  33. [33]

    2020.Grokking deep reinforcement learning

    Miguel Morales. 2020.Grokking deep reinforcement learning. Manning Publica- tions

    2020

  34. [34]

    2016.A concise introduction to decentralized POMDPs

    Frans A Oliehoek, Christopher Amato, et al . 2016.A concise introduction to decentralized POMDPs. Vol. 1. Springer

    2016

  35. [35]

    Minxue Pan, An Huang, Guoxin Wang, Tian Zhang, and Xuandong Li. 2020. Reinforcement learning based curiosity-driven testing of Android applications. InProceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis. 153–164

    2020

  36. [36]

    Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. InProceedings of the 2014 conference on empirical methods in natural language processing (EMNLP). 1532–1543. Conference’17, July 2017, Washington, DC, USA Yujia Fan, Sinan Wang, Zebang Fei, Yao Qin, Huaxuan Li, Yepang Liu

    2014

  37. [37]

    Simon Poulding and John A Clark. 2010. Efficient software verification: Statistical testing using automated search.IEEE Transactions on Software Engineering36, 6 (2010), 763–777

    2010

  38. [38]

    Andrea Romdhana, Alessio Merlo, Mariano Ceccato, and Paolo Tonella. 2022. Deep reinforcement learning for black-box testing of android apps.ACM Trans- actions on Software Engineering and Methodology (TOSEM)31, 4 (2022), 1–29

    2022

  39. [39]

    Tom Schaul, John Quan, Ioannis Antonoglou, and David Silver. 2015. Prioritized experience replay.arXiv preprint arXiv:1511.05952(2015)

    work page internal anchor Pith review Pith/arXiv arXiv 2015

  40. [40]

    Salman Sherin, Asmar Muqeet, Muhammad Uzair Khan, and Muhammad Zohaib Iqbal. 2023. QExplore: An exploration strategy for dynamic web applications using guided search.Journal of Systems and Software195 (2023), 111512

    2023

  41. [41]

    Kyunghwan Son, Daewoo Kim, Wan Ju Kang, David Earl Hostallero, and Yung Yi. 2019. Qtran: Learning to factorize with transformation for cooperative multi- agent reinforcement learning. InInternational conference on machine learning. PMLR, 5887–5896

    2019

  42. [42]

    András Vargha and Harold D Delaney. 2000. A critique and improvement of the CL common language effect size statistics of McGraw and Wong.Journal of Educational and Behavioral Statistics25, 2 (2000), 101–132

    2000

  43. [43]

    Oriol Vinyals, Timo Ewalds, Sergey Bartunov, Petko Georgiev, Alexander Sasha Vezhnevets, Michelle Yeo, Alireza Makhzani, Heinrich Küttler, John Agapiou, Julian Schrittwieser, John Quan, Stephen Gaffney, Stig Petersen, Karen Simonyan, Tom Schaul, Hado van Hasselt, David Silver, Timothy Lillicrap, Kevin Calderone, Paul Keet, Anthony Brunasso, David Lawrence...

    work page internal anchor Pith review Pith/arXiv arXiv 2017

  44. [44]

    Siyi Wang, Sinan Wang, Yujia Fan, Xiaolei Li, and Yepang Liu. 2024. Leveraging Large Vision-Language Model for Better Automatic Web GUI Testing. In2024 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 125–137

    2024

  45. [45]

    Christopher JCH Watkins and Peter Dayan. 1992. Q-learning.Machine learning 8 (1992), 279–292

    1992

  46. [46]

    Yaodong Yang and Jun Wang. 2020. An overview of multi-agent reinforcement learning from game theoretical perspective.arXiv preprint arXiv:2011.00583 (2020)

    work page arXiv 2020

  47. [47]

    Shengcheng Yu, Chunrong Fang, Xin Li, Yuchen Ling, Zhenyu Chen, and Zhen- dong Su. 2024. Effective, Platform-Independent GUI Testing via Image Embedding and Reinforcement Learning.ACM Transactions on Software Engineering and Methodology33, 7 (2024), 1–27

    2024

  48. [48]

    Kaiqing Zhang, Zhuoran Yang, and Tamer Başar. 2021. Multi-agent reinforce- ment learning: A selective overview of theories and algorithms.Handbook of reinforcement learning and control(2021), 321–384

    2021

  49. [49]

    Shuai Zhang, Hongkang Li, Meng Wang, Miao Liu, Pin-Yu Chen, Songtao Lu, Sijia Liu, Keerthiram Murugesan, and Subhajit Chaudhury. 2023. On the convergence and sample complexity analysis of deep q-networks with 𝜀-greedy exploration. Advances in Neural Information Processing Systems36 (2023), 13064–13102

    2023

  50. [50]

    Zhaoxu Zhang, Robert Winn, Yu Zhao, Tingting Yu, and William GJ Halfond

  51. [51]

    InProceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis

    Automatically reproducing android bug reports using natural language processing and reinforcement learning. InProceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis. 411–422

  52. [52]

    Yu Zhao, Brent Harrison, and Tingting Yu. 2024. Dinodroid: Testing android apps using deep q-networks.ACM Transactions on Software Engineering and Methodology33, 5 (2024), 1–24

    2024

  53. [53]

    Yan Zheng, Yi Liu, Xiaofei Xie, Yepang Liu, Lei Ma, Jianye Hao, and Yang Liu

  54. [54]

    In 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE)

    Automatic web testing using curiosity-driven reinforcement learning. In 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE). IEEE, 423–435

    2021