Pith. sign in

REVIEW 68 references

Under unified budgets, fast throughput finds the most RL crashes, while exploration-focused fuzzers find more varied ones—and both kinds of crashes can repair agents and train safety monitors.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Under unified budgets, MDPFuzz leads crash count and speed; SeqDivFuzz leads diversity; fuzz crashes improve robustness and train cross-fuzzer safety monitors.

T0 review reviewed 2026-07-31 challenge →

load-bearing objection Solid first unified bake-off of RL fuzzers with real downstream numbers; CARLA validity filter is the main soft spot but does not sink the core claims.

arxiv 2607.24577 v1 pith:IBMTD3HF submitted 2026-07-27 cs.LG cs.SE

Evaluating Fuzz Testing for Reinforcement Learning Agents

Zhibin Kang , Hanmo You , Dong Wang , Haiming Zheng , Junjie Chen This is my paper
classification cs.LG cs.SE
keywords reinforcement learningfuzz testingcrash diversityagent robustnesssafety monitoringempirical studyMDPFuzzCARLA
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that existing RL fuzzing papers cannot be compared fairly because they use different setups, baselines, and metrics. It runs the first head-to-head study of five published fuzzers plus plain random testing on three environments of rising difficulty—MountainCar, BipedalWalker, and the CARLA driving simulator—under the same time budget and validity checks. It measures not only how many unique crashes each method finds, but how diverse those crashes are, how quickly they appear, and whether the crashes help two practical tasks: repairing the agent and training a monitor that predicts crashes. The central result is a clear split: lightweight, high-throughput methods (especially MDPFuzz) win on crash count and speed; methods built for exploration (especially SeqDivFuzz) win on diversity; and the crashes themselves are useful downstream, cutting later crash counts and supporting monitors that often exceed 95% accuracy even across methods.

Core claim

When RL fuzzers are compared under identical configurations, throughput-oriented methods such as MDPFuzz discover the most unique crashes and do so most efficiently, while exploration-oriented methods such as SeqDivFuzz uncover more diverse crash behaviors; the resulting crashes can improve agent robustness (with the largest reported self-repair gain about 41.5%) and train safety monitors that generalize strongly across fuzzers.

What carries the argument

A four-axis unified benchmark—effectiveness (# unique crashes), diversity (input and output clusters), efficiency (discovery over time and generations per crash), and practical utility (repair and safety monitoring)—run for fixed 12-hour budgets with differential validity checking against a strong independent policy.

Load-bearing premise

A test input counts as a real agent crash only if a separate validation policy succeeds on it while the agent under test fails; if that validator is incomplete, some impossible or merely hard environments get counted as agent faults.

What would settle it

Re-run the same 12-hour campaigns with a strictly stronger or human-verified validity oracle (especially on CARLA) and check whether MDPFuzz’s lead on unique crashes, the diversity rankings, and the reported robustness and monitoring gains still hold on only the remaining validated crashes.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Practitioners should pick MDPFuzz-style high-throughput fuzzing when the goal is maximum crash count under a fixed time budget, and diversity-oriented methods when the goal is varied repair data.
  • No single fuzzer covers the others’ crash seeds, so combining complementary strategies is the practical way to reduce blind spots.
  • Crash sets from fuzzing are not only diagnostic: they can fine-tune agents and train monitors that transfer across fuzzers.
  • Future papers should keep random testing as a baseline and report multi-level diversity, not crash count alone.
  • Evaluation should include realistic high-complexity tasks; rankings shift as environments grow harder.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If seed overlap stays near zero across more agents and domains, automated fuzzer-portfolio or seed-to-fuzzer routing becomes a natural next system, not just a manual recipe.
  • The repair results hint that diversity of failure modes may matter more than raw crash volume for transferable robustness—worth testing with controlled diversity budgets.
  • Safety monitors trained on shared crash signatures could become a cheap runtime layer even when the training fuzzer differs from the deployment threat model.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Circularity Check

0 steps flagged

Empirical bake-off with no derivation chain that reduces claims to fitted inputs or self-justifying definitions.

full rationale

This paper is a controlled comparative evaluation of existing RL fuzzers (plus random testing) under fixed wall-clock budgets, shared seeds, and external environments/agents. Rankings of #UC, ID/OD, UD-AUC, AG/UC, robustness deltas, and monitor accuracy are measured outcomes of held-out runs, not quantities derived from parameters fitted to the same targets. Validity filtering (differential testing against independent policies) and clustering-based diversity are operational measurement choices; they do not make the reported method orderings true by construction. Self-citations appear as related work or as the baselines under test, which is appropriate and not load-bearing uniqueness import. No self-definitional loop, fitted-input-as-prediction, or renamed known law is present. Circularity score is therefore 0.

Axiom & Free-Parameter Ledger

6 free parameters · 7 axioms · 0 invented entities

The central comparative claims rest on operational definitions of crash, validity, uniqueness, and diversity, plus fixed experimental knobs (12 h budget, author hyperparameters, clustering rule, repair recipe). No new physical entities are postulated; the load-bearing content is measurement protocol and domain assumptions about simulators and validators.

free parameters (6)
  • wall_clock_budget = 12 hours
    All effectiveness/efficiency rankings are under a fixed 12-hour budget; relative order can change under different budgets or iteration caps.
  • silhouette_improvement_threshold = 20%
    ID/OD cluster counts follow Bartlett et al. with a 20% silhouette improvement rule; this threshold directly sets reported diversity.
  • crash_transition_sampling_weight = 10 vs 1; 10k crash / 90k normal
    Robustness-repair results depend on weighting crash transitions 10× versus normal transitions in a 10k/90k replay mix during fine-tuning.
  • fine_tune_learning_rate_and_updates = 1e-7, 50k updates
    Repair outcomes use lr=1e-7 and 50k offline gradient updates taken from prior recipes; different repair hyperparameters could change ⇑Rob and regression rates.
  • safety_monitor_sequence_length_and_class_balance = 25 steps; 3000 samples; 30% crash
    Monitor accuracy uses last-25-step crash windows, matched-length success windows, and a 3000-sample set with 30% crashes.
  • fuzzer_author_hyperparameters = per-paper defaults
    Each baseline is run with ‘optimal hyperparameters recommended by respective authors,’ which are free choices inherited from prior papers and can favor methods tuned to similar tasks.
axioms (7)
  • domain assumption Differential validity: an input is a valid agent fault iff a strong validation policy succeeds and the target fails.
    Section III-B; without this, crash counts could include infeasible configurations. Validator failure rates (0%, 0%, 2.5%) are assumed low enough not to dominate rankings.
  • ad hoc to paper Unique crash ≡ distinct (initial seed, crash trajectory) pair.
    Section III-D Effectiveness; this identity defines #UC and therefore the primary effectiveness ranking.
  • domain assumption Input/output diversity equals number of PCA+clustering groups of crash inputs/trajectories under the silhouette rule.
    Section III-D Diversity, following Bartlett et al.; diversity findings inherit this operationalization.
  • domain assumption Selected agents with low random-testing failure rates are appropriate subjects so intrinsic brittleness does not swamp fuzzer differences.
    Section III-A; agents chosen after 1000 random configs with reported failure rates 1.1%, 1.6%, 7.2%.
  • domain assumption Author-recommended hyperparameters and official/replicated implementations fairly represent each method’s capability.
    Sections III-C and III-E; MDPFuzz uses the updated simplified variant from the replication study rather than the original coverage mechanism.
  • domain assumption Prioritized replay fine-tuning on crash transitions is a valid proxy for ‘practical utility’ of crashes for robustness improvement.
    Section IV-D; alternative repair pipelines could reorder method usefulness.
  • standard math Wilcoxon signed-rank with Benjamini–Hochberg correction and rank-biserial effect sizes appropriately group methods.
    Section IV-A statistical protocol used to assign letter groups in Table I.

reviewed 2026-07-31 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Evaluating Fuzz Testing for Reinforcement Learning Agents." pith.science (2026). https://pith.science/paper/IBMTD3HF

@misc{pith2026260724577,
  author       = {Pith},
  title        = {Pith review of: Evaluating Fuzz Testing for Reinforcement Learning Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IBMTD3HF}},
  note         = {Machine review of arXiv:2607.24577}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Reinforcement Learning (RL) agents are increasingly deployed in safety-critical domains such as robotics, autonomous driving, and drone control, where unexpected behaviors may lead to severe real-world consequences. Fuzz testing has recently emerged as a promising method for exploring the vast state spaces of RL agents and exposing crashes. Although numerous RL fuzzing methods have been proposed, existing studies often differ in evaluation settings, baselines, and metrics, making it difficult to draw reliable conclusions about their relative effectiveness and practical usefulness. To address this gap, we present the first comprehensive empirical study that systematically evaluates RL fuzzing methods from four complementary perspectives: effectiveness, diversity, efficiency, and practical utility. We benchmark five state-of-the-art methods alongside random testing under unified configurations across three environments of increasing complexity (MountainCar, BipedalWalker, and CARLA), and further assess the downstream usefulness of detected crashes for agent robustness improvement and safety monitoring. Our results reveal several key insights. For instance,throughput-oriented methods like MDPFuzz demonstrate superior effectiveness and efficiency in crash discovery, while methods explicitly designed to encourage exploration like SeqDivFuzz excel at uncovering diverse crash behaviors. We also show that fuzzing-generated crashes can meaningfully improve agent robustness and enable accurate safety monitoring with strong cross-method generalization. Beyond these empirical findings, we distill actionable guidance for both researchers and practitioners, highlighting the benefits of combining complementary fuzzing strategies and adopting multi-level diversity analysis to achieve more comprehensive and practical RL testing.

Figures

Figures reproduced from arXiv: 2607.24577 by Dong Wang, Haiming Zheng, Hanmo You, Junjie Chen, Zhibin Kang.

Figure 1
Figure 1. Figure 1: The General Framework of RL Formally, RL problems are formulated based on the Markov Decision Process (MDP) [40]. MDP is typically represented as a tuple (S, A,P, R, γ). In this tuple, S denotes the state space. It corresponds to all possible observations that the agent can perceive at each step of execution. A represents the set of possible actions. These actions can be either continuous or discrete, and … view at source ↗
Figure 2
Figure 2. Figure 2: Examples from BipedalWalker for Qualitative Analysis [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Overlap of Initial Seeds for UC Detected [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Temporal Trends of UC Numbers Detected by Different [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Box-plot Analysis of AG/UC for Different Fuzzers [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

68 extracted references · 4 linked inside Pith

  1. [1]

    A hitchhiker’s guide to statistical tests for assessing randomized algorithms in software engineering,

    A. Arcuri and L. C. Briand, “A hitchhiker’s guide to statistical tests for assessing randomized algorithms in software engineering,”Softw. Test. Verification Reliab., vol. 24, no. 3, pp. 219–250, 2014

    2014

  2. [2]

    The pursuit of diversity: Multi-objective testing of deep rein- forcement learning agents,

    A. Bartlett, C. C. S. Liem, and A. Panichella, “The pursuit of diversity: Multi-objective testing of deep rein- forcement learning agents,” inSSBSE, ser. Lecture Notes in Computer Science, vol. 16228. Springer, 2025, pp. 97–112

    2025

  3. [3]

    Reinforcement learning: An introduction. by richard’s sutton,

    A. G. Barto, “Reinforcement learning: An introduction. by richard’s sutton,”SIAM Rev, vol. 6, no. 2, p. 423, 2021

    2021

  4. [4]

    Controlling the false discovery rate: a practical and powerful approach to multiple testing,

    Y . Benjamini and Y . Hochberg, “Controlling the false discovery rate: a practical and powerful approach to multiple testing,”Journal of the Royal statistical society: series B (Methodological), vol. 57, no. 1, pp. 289–300, 1995

    1995

  5. [5]

    Testing the plasticity of re- inforcement learning-based systems,

    M. Biagiola and P. Tonella, “Testing the plasticity of re- inforcement learning-based systems,”ACM Trans. Softw. Eng. Methodol., vol. 31, no. 4, pp. 80:1–80:46, 2022

    2022

  6. [6]

    Testing of deep reinforcement learning agents with surrogate models,

    ——, “Testing of deep reinforcement learning agents with surrogate models,”ACM Trans. Softw. Eng. Methodol., vol. 33, no. 3, pp. 73:1–73:33, 2024

    2024

  7. [7]

    Coverage- based greybox fuzzing as markov chain,

    M. Böhme, V . Pham, and A. Roychoudhury, “Coverage- based greybox fuzzing as markov chain,”IEEE Trans. Software Eng., vol. 45, no. 5, pp. 489–506, 2019

    2019

  8. [8]

    Openai gym,

    G. Brockman, V . Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba, “Openai gym,” CoRR, vol. abs/1606.01540, 2016

    Pith/arXiv arXiv 2016

  9. [9]

    Exploration by random network distillation,

    Y . Burda, H. Edwards, A. J. Storkey, and O. Klimov, “Exploration by random network distillation,” inICLR (Poster). OpenReview.net, 2019

    2019

  10. [10]

    Drlfailuremon- itor: A dynamic failure monitoring approach for deep reinforcement learning system,

    Y . Cai, X. Wan, Z. Liu, and Z. Zheng, “Drlfailuremon- itor: A dynamic failure monitoring approach for deep reinforcement learning system,” inISSRE. IEEE, 2024, pp. 487–498

    2024

  11. [11]

    Cohen,Statistical Power Analysis for the Behavioral Sciences, 2nd ed

    J. Cohen,Statistical Power Analysis for the Behavioral Sciences, 2nd ed. Routledge, 1988. [Online]. Available: https://doi.org/10.4324/9780203771587

    doi:10.4324/9780203771587 1988

  12. [12]

    Rank-biserial correlation,

    E. E. Cureton, “Rank-biserial correlation,”Psychome- trika, vol. 21, pp. 287–290, 1956

    1956

  13. [13]

    CARLA: an open urban driving simulator,

    A. Dosovitskiy, G. Ros, F. Codevilla, A. M. López, and V . Koltun, “CARLA: an open urban driving simulator,” in CoRL, ser. Proceedings of Machine Learning Research, vol. 78. PMLR, 2017, pp. 1–16

    2017

  14. [14]

    Prioritized replay for RL post-training,

    M. Fatemi, “Prioritized replay for RL post-training,” CoRR, vol. abs/2601.02648, 2026

    arXiv 2026

  15. [15]

    Reinforcement learning for online testing of autonomous driving systems: a replication and extension study,

    L. Giamattei, M. Biagiola, R. Pietrantuono, S. Russo, and P. Tonella, “Reinforcement learning for online testing of autonomous driving systems: a replication and extension study,”Empir. Softw. Eng., vol. 30, no. 1, p. 19, 2025

    2025

  16. [16]

    Towards comprehensive testing on the robustness of co- operative multi-agent reinforcement learning,

    J. Guo, Y . Chen, Y . Hao, Z. Yin, Y . Yu, and S. Li, “Towards comprehensive testing on the robustness of co- operative multi-agent reinforcement learning,” inCVPR Workshops. IEEE, 2022, pp. 114–121

    2022

  17. [17]

    Many-objective reinforcement learning for online testing of dnn-enabled systems,

    F. U. Haq, D. Shin, and L. C. Briand, “Many-objective reinforcement learning for online testing of dnn-enabled systems,” inICSE. IEEE, 2023, pp. 1814–1826

    2023

  18. [18]

    Curiosity-driven testing for sequen- JOURNAL OF LATEX CLASS FILES, VOL. 18, NO. 9, SEPTEMBER 2020 13 tial decision-making process,

    J. He, Z. Yang, J. Shi, C. Yang, K. Kim, B. Xu, X. Zhou, and D. Lo, “Curiosity-driven testing for sequen- JOURNAL OF LATEX CLASS FILES, VOL. 18, NO. 9, SEPTEMBER 2020 13 tial decision-making process,” inICSE. ACM, 2024, pp. 165:1–165:14

    2020

  19. [19]

    Deep reinforcement learning for drone navigation using sensor data,

    V . J. Hodge, R. Hawkins, and R. Alexander, “Deep reinforcement learning for drone navigation using sensor data,”Neural Comput. Appl., vol. 33, no. 6, pp. 2015– 2033, 2021

    2015

  20. [20]

    Homepage, https://github.com/RWO-zb/fuzzers, 2026

    2026

  21. [21]

    A novel DDPG method with prioritized experience replay,

    Y . Hou, L. Liu, Q. Wei, X. Xu, and C. Chen, “A novel DDPG method with prioritized experience replay,” in SMC. IEEE, 2017, pp. 316–321

    2017

  22. [22]

    Carl: Learning scalable plan- ning policies with simple rewards,

    B. Jaeger, D. Dauner, J. Beißwenger, S. Gerstenecker, K. Chitta, and A. Geiger, “Carl: Learning scalable plan- ning policies with simple rewards,” inConference on Robot Learning. PMLR, 2025, pp. 5301–5338

    2025

  23. [23]

    Residual reinforcement learning for robot control,

    T. Johannink, S. Bahl, A. Nair, J. Luo, A. Kumar, M. Loskyll, J. A. Ojea, E. Solowjow, and S. Levine, “Residual reinforcement learning for robot control,” in ICRA. IEEE, 2019, pp. 6023–6029

    2019

  24. [24]

    Concept bottleneck models,

    P. W. Koh, T. Nguyen, Y . S. Tang, S. Mussmann, E. Pierson, B. Kim, and P. Liang, “Concept bottleneck models,” inICML, ser. Proceedings of Machine Learning Research, vol. 119. PMLR, 2020, pp. 5338–5348

    2020

  25. [25]

    Anatomy of a robotaxi crash: Lessons from the cruise pedestrian dragging mishap,

    P. Koopman, “Anatomy of a robotaxi crash: Lessons from the cruise pedestrian dragging mishap,” inSAFECOMP, ser. Lecture Notes in Computer Science, vol. 14988. Springer, 2024, pp. 119–133

    2024

  26. [26]

    Offline reinforcement learning: Tutorial, review, and perspectives on open problems,

    S. Levine, A. Kumar, G. Tucker, and J. Fu, “Offline reinforcement learning: Tutorial, review, and perspectives on open problems,”CoRR, vol. abs/2005.01643, 2020

    Pith/arXiv arXiv 2005

  27. [27]

    Faster diffusion: Rethinking the role of the encoder for diffusion model inference,

    S. Li, T. Hu, J. van de Weijer, F. S. Khan, T. Liu, L. Li, S. Yang, Y . Wang, M. Cheng, and J. Yang, “Faster diffusion: Rethinking the role of the encoder for diffusion model inference,” inNeurIPS, 2024

    2024

  28. [28]

    Agentfuzz: Fuzzing for deep reinforcement learning systems,

    T. Li, X. Wan, and M. M. Özbek, “Agentfuzz: Fuzzing for deep reinforcement learning systems,” inISSRE Work- shops. IEEE, 2022, pp. 110–113

    2022

  29. [29]

    Generative model-based testing on decision-making policies,

    Z. Li, X. Wu, D. Zhu, M. Cheng, S. Chen, F. Zhang, X. Xie, L. Ma, and J. Zhao, “Generative model-based testing on decision-making policies,” inASE. IEEE, 2023, pp. 243–254

    2023

  30. [30]

    Todynet: temporal dynamic graph neural network for multivariate time series classification,

    H. Liu, D. Yang, X. Liu, X. Chen, Z. Liang, H. Wang, Y . Cui, and J. Gu, “Todynet: temporal dynamic graph neural network for multivariate time series classification,” Information Sciences, vol. 677, p. 120914, 2024

    2024

  31. [31]

    Finetuning from offline reinforcement learning: Chal- lenges, trade-offs and practical solutions,

    Y . Luo, J. Kay, E. Grefenstette, and M. P. Deisenroth, “Finetuning from offline reinforcement learning: Chal- lenges, trade-offs and practical solutions,”CoRR, vol. abs/2303.17396, 2023

    Pith/arXiv arXiv 2023

  32. [32]

    Enhancing multi-agent system testing with diversity-guided exploration and adaptive critical state exploitation,

    X. Ma, Y . Wang, J. Wang, X. Xie, B. Wu, S. Li, F. Xu, and Q. Wang, “Enhancing multi-agent system testing with diversity-guided exploration and adaptive critical state exploitation,” inISSTA. ACM, 2024, pp. 1491– 1503

    2024

  33. [33]

    Fault diversity in reinforcement learning policy testing,

    Q. Mazouni, A. Gotlieb, M. Acher, and H. Spieker, “Fault diversity in reinforcement learning policy testing,”ACM Transactions on Software Engineering and Methodology

  34. [34]

    Policy testing with mdpfuzz (replicability study),

    Q. Mazouni, H. Spieker, A. Gotlieb, and M. Acher, “Policy testing with mdpfuzz (replicability study),” in ISSTA. ACM, 2024, pp. 1567–1578

    2024

  35. [35]

    Learning, reward, and decision making,

    J. P. O’Doherty, J. Cockburn, and W. M. Pauli, “Learning, reward, and decision making,”Annual review of psychol- ogy, vol. 68, no. 1, pp. 73–100, 2017

    2017

  36. [36]

    Mdpfuzz: testing models solving markov decision processes,

    Q. Pang, Y . Yuan, and S. Wang, “Mdpfuzz: testing models solving markov decision processes,” inISSTA. ACM, 2022, pp. 378–390

    2022

  37. [37]

    Pytorch: An imperative style, high-performance deep learning library,

    A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Köpf, E. Z. Yang, Z. DeVito, M. Rai- son, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala, “Pytorch: An imperative style, high-performance deep learning library,” inNeurIPS, 2019, pp. 8024–8035

    2019

  38. [38]

    Deepxplore: automated whitebox testing of deep learning systems,

    K. Pei, Y . Cao, J. Yang, and S. Jana, “Deepxplore: automated whitebox testing of deep learning systems,” Commun. ACM, vol. 62, no. 11, pp. 137–145, 2019

    2019

  39. [39]

    Learning and testing resilience in cooperative multi-agent systems,

    T. Phan, T. Gabor, A. Sedlmeier, F. Ritz, B. Kempter, C. Klein, H. Sauer, R. N. Schmid, J. Wieghardt, M. Zeller, and C. Linnhoff-Popien, “Learning and testing resilience in cooperative multi-agent systems,” inAA- MAS. IFAAMAS, 2020, pp. 1055–1063

    2020

  40. [40]

    M. L. Puterman,Markov decision processes: discrete stochastic dynamic programming. John Wiley & Sons, 2014

    2014

  41. [41]

    Rl baselines3 zoo,

    A. Raffin, “Rl baselines3 zoo,” https://github.com/ DLR-RM/rl-baselines3-zoo, 2020

    2020

  42. [42]

    Stable-baselines3: Reliable reinforce- ment learning implementations,

    A. Raffin, A. Hill, A. Gleave, A. Kanervisto, M. Ernestus, and N. Dormann, “Stable-baselines3: Reliable reinforce- ment learning implementations,”J. Mach. Learn. Res., vol. 22, pp. 268:1–268:8, 2021

    2021

  43. [43]

    Vuzzer: Application-aware evolutionary fuzzing,

    S. Rawat, V . Jain, A. Kumar, L. Cojocar, C. Giuffrida, and H. Bos, “Vuzzer: Application-aware evolutionary fuzzing,” inNDSS. The Internet Society, 2017

    2017

  44. [44]

    Prior- itized experience replay,

    T. Schaul, J. Quan, I. Antonoglou, and D. Silver, “Prior- itized experience replay,” inICLR (Poster), 2016

    2016

  45. [45]

    Testing rein- forcement learning systems: A comprehensive review,

    A. Sunba, J. Hassine, and M. A. Ahmed, “Testing rein- forcement learning systems: A comprehensive review,”J. Syst. Softw., vol. 231, p. 112563, 2026

    2026

  46. [46]

    Search-based testing of reinforcement learning,

    M. Tappler, F. C. Córdoba, B. K. Aichernig, and B. Könighofer, “Search-based testing of reinforcement learning,” inIJCAI. ijcai.org, 2022, pp. 503–510

    2022

  47. [47]

    Learning and repair of deep reinforce- ment learning policies from fuzz-testing data,

    M. Tappler, A. Pferscher, B. K. Aichernig, and B. Könighofer, “Learning and repair of deep reinforce- ment learning policies from fuzz-testing data,” inICSE. ACM, 2024, pp. 6:1–6:13

    2024

  48. [48]

    PCLA: A framework for testing autonomous agents in the CARLA simulator,

    M. J. Tehrani, J. Kim, and P. Tonella, “PCLA: A framework for testing autonomous agents in the CARLA simulator,” inSIGSOFT FSE Companion. ACM, 2025, pp. 1040–1044

    2025

  49. [49]

    $\mu \text{PRL}$: A mutation testing pipeline for deep rein- forcement learning based on real faults,

    D. Thomas, M. Biagiola, N. Humbatova, M. War- dat, G. Jahangirova, H. Rajan, and P. Tonella, “$\mu \text{PRL}$: A mutation testing pipeline for deep rein- forcement learning based on real faults,” inICSE. IEEE, 2025, pp. 2238–2250

    2025

  50. [50]

    Does neuron coverage matter for deep reinforcement learning?: A preliminary JOURNAL OF LATEX CLASS FILES, VOL. 18, NO. 9, SEPTEMBER 2020 14 study,

    M. Trujillo, M. Linares-Vásquez, C. Escobar-Velásquez, I. Dusparic, and N. Cardozo, “Does neuron coverage matter for deep reinforcement learning?: A preliminary JOURNAL OF LATEX CLASS FILES, VOL. 18, NO. 9, SEPTEMBER 2020 14 study,” inICSE (Workshops). ACM, 2020, pp. 215–220

    2020

  51. [51]

    Rigorous agent evaluation: An adversarial approach to uncover catastrophic failures,

    J. Uesato, A. Kumar, C. Szepesvári, T. Erez, A. Rud- erman, K. Anderson, K. D. Dvijotham, N. Heess, and P. Kohli, “Rigorous agent evaluation: An adversarial approach to uncover catastrophic failures,” inICLR (Poster). OpenReview.net, 2019

    2019

  52. [52]

    A survey of reinforce- ment learning for software engineering,

    D. Wang, H. You, L. Zhu, K. Lin, Z. Chen, C. Yang, J. Yu, Z. Wang, and J. Chen, “A survey of reinforce- ment learning for software engineering,”arXiv preprint arXiv:2507.12483, 2025

    Pith/arXiv arXiv 2025

  53. [53]

    Fuzzing with sequence diversity inference for sequential decision- making model testing,

    K. Wang, Y . Wang, J. Wang, and Q. Wang, “Fuzzing with sequence diversity inference for sequential decision- making model testing,” inISSRE. IEEE, 2023, pp. 706– 717

    2023

  54. [54]

    Wilcoxon signed-rank test,

    R. F. Woolson, “Wilcoxon signed-rank test,”Wiley ency- clopedia of clinical trials, pp. 1–3, 2007

    2007

  55. [55]

    Regression fault detection and mitigation in the evolution of deep learning systems,

    H. You, “Regression fault detection and mitigation in the evolution of deep learning systems,” inSIGSOFT FSE Companion. ACM, 2025, pp. 1293–1294

    2025

  56. [56]

    Regression fuzzing for deep learning systems,

    H. You, Z. Wang, J. Chen, S. Liu, and S. Li, “Regression fuzzing for deep learning systems,” inICSE. IEEE, 2023, pp. 82–94

    2023

  57. [57]

    Mitigating regression faults induced by feature evolution in deep learning systems,

    H. You, Z. Wang, X. Chen, J. Chen, J. Sun, S. Liu, and Z. Dong, “Mitigating regression faults induced by feature evolution in deep learning systems,”ACM Trans. Softw. Eng. Methodol., vol. 34, no. 6, pp. 171:1–171:33, 2025

    2025

  58. [58]

    A comprehensive study of deep learning model fixing approaches,

    H. You, Z. Wang, Z. Dong, L. Mo, J. Zhao, and J. Chen, “A comprehensive study of deep learning model fixing approaches,”CoRR, vol. abs/2512.23745, 2025

    arXiv 2025

  59. [59]

    Navigating the testing of evolving deep learning systems: An exploratory interview study,

    H. You, Z. Wang, B. Lin, and J. Chen, “Navigating the testing of evolving deep learning systems: An exploratory interview study,” inICSE. IEEE, 2025, pp. 2726–2738

    2025

  60. [60]

    A white-box testing for deep neural networks based on neuron coverage,

    J. Yu, S. Duan, and X. Ye, “A white-box testing for deep neural networks based on neuron coverage,”IEEE Trans. Neural Networks Learn. Syst., vol. 34, no. 11, pp. 9185– 9197, 2023

    2023

  61. [61]

    End-to-end urban driving by imitating a reinforcement learning coach,

    Z. Zhang, A. Liniger, D. Dai, F. Yu, and L. V . Gool, “End-to-end urban driving by imitating a reinforcement learning coach,” inICCV. IEEE, 2021, pp. 15 202– 15 212

    2021

  62. [62]

    Iden- tifying the failure-revealing test cases in metamorphic testing: A statistical approach,

    Z. Zheng, D. Ren, H. Liu, T. Y . Chen, and T. Li, “Iden- tifying the failure-revealing test cases in metamorphic testing: A statistical approach,”ACM Transactions on Software Engineering and Methodology, vol. 34, no. 2, pp. 1–26, 2025

    2025

  63. [63]

    Parallel test prioritiza- tion,

    J. Zhou, J. Chen, and D. Hao, “Parallel test prioritiza- tion,”ACM Trans. Softw. Eng. Methodol., vol. 31, no. 1, pp. 8:1–8:50, 2022

    2022

  64. [64]

    Knowledge transfer from simple to complex: A safe and efficient reinforcement learning framework for au- tonomous driving decision-making,

    R. Zhou, J. Huang, M. Li, H. Li, H. Cao, and X. Song, “Knowledge transfer from simple to complex: A safe and efficient reinforcement learning framework for au- tonomous driving decision-making,”Adv. Eng. Informat- ics, vol. 65, p. 103188, 2025

    2025

  65. [65]

    Robustness testing for multi-agent reinforcement learning: State perturbations on critical agents,

    Z. Zhou and G. Liu, “Robustness testing for multi-agent reinforcement learning: State perturbations on critical agents,” inECAI, ser. Frontiers in Artificial Intelligence and Applications, vol. 372. IOS Press, 2023, pp. 3131– 3139

    2023

  66. [66]

    Fuzzing: A survey for roadmap,

    X. Zhu, S. Wen, S. Camtepe, and Y . Xiang, “Fuzzing: A survey for roadmap,”ACM Comput. Surv., vol. 54, no. 11s, pp. 230:1–230:36, 2022

    2022

  67. [67]

    A search-based testing approach for deep reinforcement learning agents,

    A. Zolfagharian, M. Abdellatif, L. C. Briand, M. Bagherzadeh, and R. S., “A search-based testing approach for deep reinforcement learning agents,”IEEE Trans. Software Eng., vol. 49, no. 7, pp. 3715–3735, 2023

    2023

  68. [68]

    SMARLA: A safety monitoring approach for deep reinforcement learning agents,

    A. Zolfagharian, M. Abdellatif, L. C. Briand, and S. Ramesh, “SMARLA: A safety monitoring approach for deep reinforcement learning agents,”IEEE Trans. Software Eng., vol. 51, no. 1, pp. 82–105, 2025

    2025

This paper was first reviewed by grok-4.5 on July 31, 2026.