Pith. sign in

REVIEW 3 major objections 8 minor 32 references

Two API hooks, all RL testing methods

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · glm-5.2

2026-07-09 21:19 UTC pith:3HABKAF4

load-bearing objection Open-source RL testing tool with a clean decorator architecture, but the evaluation is purely demonstrative and one core claim (white-box AT) has an undescribed mechanism. the 3 major comments →

arxiv 2607.07029 v1 pith:3HABKAF4 submitted 2026-07-08 cs.LG cs.AIcs.SE

Gimitest: A Comprehensive Tool for Testing Reinforcement Learning Policies

Dennis Gross , Quentin Mazouni , Helge Spieker , Arnaud Gotlieb This is my paper
classification cs.LG cs.AIcs.SE
keywords reinforcement learningpolicy testingsoftware testingsearch-based testingmetamorphic testingadversarial testingmulti-agent reinforcement learningdecorator pattern
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Reinforcement learning policies can fail in dangerous ways—crashing spacecraft, misjudging distances, or behaving erratically when conditions shift slightly from training. Testing these policies for such failures is fragmented: each testing technique (search-based testing, metamorphic testing, adversarial testing) typically requires its own custom implementation, and the diversity of RL environment APIs makes it hard to write tests that port across frameworks. This paper introduces a tool called Gimitest that aims to unify these testing approaches under a single architecture. The central mechanism is a decorator pattern: the tool wraps an RL environment's two core methods—reset (which initializes a scenario) and step (which advances the simulation by one tick)—and injects user-defined test logic at five hook points around each call (before and after configuration, before and after testing, and the original method itself). By subclassing a base test class and overriding methods at these hooks, a tester can modify the initial state for search-based testing, perturb observations for adversarial testing, alter environmental parameters for metamorphic testing, log execution data, or interface with external systems for automated test generation. The same wrapper works for both single-agent and multi-agent (parallel and turn-based) environments, because it targets the common API shared across major RL frameworks. The paper demonstrates the tool across six environments and four testing categories, showing that the hook-based interception is expressive enough to implement each method without modifying the environment or policy code directly.

Core claim

The paper claims that intercepting just two environment methods—step() and reset()—with pre- and post-execution hooks is sufficient to implement the major families of RL policy testing (search-based, metamorphic, adversarial) plus logging and automated code generation, across both single-agent and multi-agent settings. The GTest base class and its decorator wrapping provide a unified interface: subclasses override hook methods to inject testing logic at specific time steps, and the environment executes the original method plus the injected hooks in sequence. This means a tester writes testing logic once and applies it across any compatible RL framework without altering the environment's or政策

What carries the argument

The decorator design pattern applied to the environment's reset() and step() methods. A GTest base class defines hook methods (pre/post-configuration and pre/post-test) that are called in sequence around each reset or step invocation. Subclasses override these hooks to implement specific testing logic. A GLogger class can further decorate GTest instances to capture execution data. The same pattern extends to multi-agent environments because they share the same fundamental reset/step API.

Load-bearing premise

The paper's generality rests on the premise that all relevant testing logic can be injected through pre- and post-execution hooks around the environment's reset() and step() methods. Environments that rely on third-party simulators with internal state not exposed through these two methods, or that require parallel execution across multiple environment instances, fall outside what the tool can handle.

What would settle it

Find a commonly used RL testing method whose logic cannot be expressed through pre/post hooks on reset() and step() alone—for example, a method that requires intercepting the policy's internal forward pass at a layer not exposed through the observation/action interface, or one that needs to modify the environment's transition dynamics mid-step in a way that the step() wrapper cannot reach.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the two-hook architecture is as general as claimed, future RL testing methods could be implemented as GTest subclasses rather than as standalone tools, reducing duplication and enabling direct comparison of methods on identical environments.
  • The GPT-4 integration for automated test code generation suggests a pipeline where an LLM reads environment source code, extracts configurable parameters, and produces executable test suites—potentially scaling policy testing to environments that lack human-written test harnesses.
  • The multi-agent support (both parallel and turn-based) means the same testing infrastructure could evaluate coordination failures in multi-agent systems, not just individual policy weaknesses.
  • The logging and failure-visualization pipeline (recording episode frames for GPT-4 vision analysis) points toward automated failure categorization at scale, which could help triage the large volume of failures that testing methods produce.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 8 minor

Summary. The paper introduces Gimitest, an open-source Python tool for testing single- and multi-agent reinforcement learning (RL) policies. The tool uses the decorator design pattern to intercept the step() and reset() methods of environment simulators (Farama Gymnasium, PettingZoo, OpenAI Gym), injecting user-defined test logic via a GTest base class with pre/post-configuration and pre/post-test hooks. The paper claims support for search-based software testing (SBST), metamorphic testing (MT), adversarial testing (AT), execution logging, and automated test code generation via GPT-4. Six use cases demonstrate the tool on Lunar Lander (SBST), Mountain Car (AT), Cart Pole (MT), Lunar Lander (logging), Connect Four (turn-based MARL), Waterworld (parallel MARL), and Cart Pole (automated generation). The tool is released as open source.

Significance. The paper addresses a genuine gap: despite a growing body of RL policy testing research (SBST, MT, AT, fuzzing), there is no widely adopted unified tool that bridges these methods across single- and multi-agent settings and multiple gym APIs. The decorator-based architecture is a clean, lightweight design that requires only step()/reset() access, lowering the barrier to adoption. The open-source release (https://github.com/DennisGross/Gimitest) is a concrete strength, as is the demonstrated applicability to both turn-based and parallel MARL, which is often overlooked. The GPT-4 integration for automated test code generation, while preliminary, is an interesting forward-looking feature. However, the significance is tempered by the primarily demonstrative evaluation—no baselines, no quantitative comparison with existing tools, and no scalability measurements—which limits the ability to assess the tool's practical value relative to alternatives.

major comments (3)
  1. Section 4.3 claims that 'Gimitest lets the user access the internals of the policy for white-box attacks,' and Section 6.2 demonstrates an FGSM attack on Mountain Car, which requires computing the gradient of the policy's loss with respect to observations. However, the architecture in Section 5 and Algorithm 1 only describe decorating the environment's step() and reset() methods. The paper does not describe how GTest obtains a reference to the policy object, how it accesses the policy's computational graph for gradient computation, or how this works across different RL backends (PyTorch, TensorFlow, JAX). This is a gap between a claimed core functionality (white-box AT) and the described mechanism. The authors should either (a) describe the policy-access mechanism in the architecture section, or (b) clarify that white-box AT requires the user to manually provide policy access and is not,
  2. The evaluation in Section 6 is entirely demonstrative. There are no baselines: the SBST results in Section 6.1 (Figure 8) compare three search methods implemented within Gimitest but do not compare against any external testing tool or framework. The paper does not report execution times, scalability limits, or the effort required to implement each test method. Without at least a rough quantitative comparison or a usability study (e.g., lines of code to implement a test method with vs. without Gimitest), the claim of 'comprehensive' and 'effective' is not substantiated beyond proof-of-concept. The authors should add at least one comparative baseline or a quantitative usability metric.
  3. The claim of 'comprehensive' testing support is qualified in Section 5 by noting that environments relying on third-party simulators (e.g., CARLA) and parallel test execution are not yet supported. These are significant boundaries. The paper should either temper the 'comprehensive' claim in the abstract and title, or provide a clearer scoping statement early in the paper (Section 1.3 or Section 4) that delineates what classes of environments and testing scenarios are in-scope vs. out-of-scope.
minor comments (8)
  1. Section 1, Example 3: the claim that 'random noise is a special type of adversarial attack' is cited to Ohashi et al. (2021), but this is a non-standard framing. The paper should either justify this assumption more carefully or soften the language, as it conflates two distinct concepts in the adversarial ML literature.
  2. Figure 5 is referenced in Section 5 but the labels 'Pre' and 'Post' are not clearly defined in the surrounding text until later. Consider adding a brief inline definition when the figure is first introduced.
  3. Section 6.1: the input range [−1000, 1000]^2 for Lunar Lander forces is stated without justification. Why this range? Does it correspond to physically meaningful perturbations?
  4. Section 6.3: the paper states 'over 100 such bugs' were uncovered in Cart Pole but does not report the total number of test cases run, making it hard to assess the bug rate.
  5. The paper uses 'GTest' and 'GLogger' as class names but does not consistently capitalize or format them (e.g., 'GTest class' vs. 'GTest-class' in Example 13). Standardize.
  6. Section 7 mentions experiments were conducted on the ex3 cluster but provides no hardware specifications, runtime, or resource usage details.
  7. The automated testing pipeline (Figure 14) is described in Section 6.6 but the figure is placed at the end of the paper (after the conclusion). Consider moving it closer to the relevant discussion.
  8. Several references are incomplete or inconsistently formatted (e.g., 'CoRR, abs/...' entries lack access dates; some conference proceedings lack page numbers).

Circularity Check

0 steps flagged

No circularity: tool paper with self-contained demonstrations and no derivation chain to reduce.

full rationale

This is a software tool description paper, not a derivation or proof paper. The central claim—that Gimitest supports SBST, MT, AT, logging, and automated test generation by decorating step()/reset() methods—is substantiated through architectural description (Section 5) and empirical use cases (Section 6), not through a chain of mathematical reasoning that could be circular. The self-citations present (Gross et al. 2022 on COOL-MC, Mazouni et al. 2024a on MDPFuzz replicability) are used as related work or prior results, not as load-bearing premises that define the paper's outputs in terms of its inputs. The use cases in Section 6 (Lunar Lander SBST, Mountain Car AT, Cart Pole MT, Connect Four and Waterworld MARL) demonstrate the tool on external benchmarks with externally trained policies, so the results are not forced by construction. The skeptic's concern about the undescribed mechanism for white-box AT gradient access is a correctness/completeness gap, not a circularity issue—the claim is not self-definitional, it is simply under-specified in the architectural description. No step in the paper reduces to its own inputs by definition or by self-citation chain.

Axiom & Free-Parameter Ledger

0 free parameters · 3 axioms · 2 invented entities

The paper introduces no free parameters or fitted constants. The axioms are domain assumptions about RL environment APIs and the sufficiency of the decorator pattern. The invented entities (GTest, GLogger) are software classes in the open-source tool, not theoretical constructs, and their functionality is demonstrated empirically.

axioms (3)
  • domain assumption RL environment simulators expose action processing (step method) and environment reset (reset method) capabilities following the Gymnasium/PettingZoo API convention.
    Stated in Section 1.3 and Section 4. The entire architecture depends on this API contract. Environments that do not follow this convention (e.g., some third-party simulators like CARLA) are acknowledged as unsupported.
  • domain assumption The decorator design pattern can sufficiently intercept and modify all environment interactions needed for SBST, MT, and AT without requiring changes to the environment or policy internals (except for white-box AT).
    Section 5 describes the decorator pattern applied to step and reset. The assumption is that pre/post hooks around these two methods are sufficient for all supported testing methods.
  • ad hoc to paper Random noise is a special type of adversarial attack.
    Stated in Example 3 (Section 1): 'we assume that random noise is a special type of adversarial attack.' This assumption is used to justify treating noise injection under the AT category.
invented entities (2)
  • GTest class independent evidence
    purpose: Base class whose methods (pre/post-step-configuration, pre/post-step-test, pre/post-reset-configuration, pre/post-reset-test) are overridden by users to implement testing logic.
    The class is part of the shipped open-source tool. Its functionality is demonstrated in multiple use cases (Sections 6.1-6.5).
  • GLogger class independent evidence
    purpose: Decorator for GTest instances that logs the testing process at every point in time.
    Part of the shipped tool. Demonstrated in Section 6.4 (Policy Analysis) with action frequency logging.

pith-pipeline@v1.1.0-glm · 20307 in / 2376 out tokens · 239233 ms · 2026-07-09T21:19:38.063931+00:00 · methodology

0 comments
read the original abstract

Reinforcement learning (RL) policies can be unsafe and vulnerable to attacks. Ensuring their reliability is often a pain point as existing automated testing methods target only selected environments, testing scenarios, and RL algorithms. To address this, we propose a comprehensive framework for testing single- and multi-agent RL policies under varying conditions. Our implementation of this framework, Gimitest, is an open-source tool that supports various gym frameworks and allows for modifications of their integrated components. This article describes the framework and details Gimitest's functionality and architecture. It showcases its effectiveness in testing multiple RL policies in environments such as the official Farama Gymnasium and PettingZoo.

Figures

Figures reproduced from arXiv: 2607.07029 by Arnaud Gotlieb, Dennis Gross, Helge Spieker, Quentin Mazouni.

Figure 1
Figure 1. Figure 1: Single-agent RL system, where an agent receives an observation [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Parallel MARL setup with two agents performing a joint action [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Turn-based MARL system with two agents interacting sequentially with a shared environment. [PITH_FULL_IMAGE:figures/full_fig_p002_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Gimitest’s role within the RL workflow, which involves training, testing, and deploying policies. [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Interception of Gimitest in the RL policy-environment interaction. Pre refers to functionality added before the environment is called; post refers to functionality added afterward [PITH_FULL_IMAGE:figures/full_fig_p010_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: GTest base class. 10 [PITH_FULL_IMAGE:figures/full_fig_p010_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: GLogger base class. We can create custom GTest subclasses and override specific methods to suit our testing needs. Overwriting the GTest configuration methods allows us to modify the environment at specific time steps, and overwriting the GTest testing methods allows us to test the policy at specific time steps. By applying GTest decoration to the environment step method, the environment sequentially perfo… view at source ↗
Figure 8
Figure 8. Figure 8: SBST: Distribution of the input tests for the Lunar Lander use case. The inputs describe the initial [PITH_FULL_IMAGE:figures/full_fig_p012_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Results of the Adversarial Testing experiment on the Mountain Car policy. Adversarial states are [PITH_FULL_IMAGE:figures/full_fig_p013_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Action frequencies of the Lunar Lander policy, when tested with random, evolutionary, and fuzz SBST methods. “NOP” refers to the do-nothing action, while “LEFT”, “MAIN” and “RIGHT” refer to firing the three engines of the spacecraft. player 2 always wins when the coin is set into columns 0, 3, 4, and 6, and in the other cases, player 2 loses. In Waterworld, we tested how well the trained MARL policies per… view at source ↗
Figure 11
Figure 11. Figure 11: Connect Four results. The win rate for player 2 is on the y-axis, and the x-axis describes the [PITH_FULL_IMAGE:figures/full_fig_p014_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Waterworld results. The x-axis represents the different numbers of evaders, and the y-axis [PITH_FULL_IMAGE:figures/full_fig_p015_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: This recorded frame shows a failure. Analyzing this frame with GPT-4 helps categorize the failure, [PITH_FULL_IMAGE:figures/full_fig_p015_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: General automated RL policy SBST framework. First, GPT-4 extracts the environment source [PITH_FULL_IMAGE:figures/full_fig_p016_14.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

32 extracted references · 32 canonical work pages · 15 internal anchors

  1. [1]

    Code Generation Tools (Almost) for Free? A Study of Few-Shot, Pre-Trained Language Models on Code

    Patrick Bareiß, Beatriz Souza, Marcelo d’Amorim, and Michael Pradel. Code generation tools (almost) for free? A study of few-shot, pre-trained language models on code.CoRR, abs/2206.01335,

    work page internal anchor Pith review Pith/arXiv arXiv

  2. [2]

    Varun Bhatt, Bryon Tjanaka, Matthew C

    doi: 10.1109/TSE.2014.2372785. Varun Bhatt, Bryon Tjanaka, Matthew C. Fontaine, and Stefanos Nikolaidis. Deep surrogate assisted generation of environments.arXiv:2206.04199,

    work page doi:10.1109/tse.2014.2372785 2014

  3. [3]

    Deep Surrogate Assisted Generation of Environments

    URLhttps://arxiv.org/abs/2206.04199. Matteo Biagiola and Paolo Tonella. Testing of deep reinforcement learning agents with surrogate models. ACM Trans. Softw. Eng. Methodol., (73):1–33,

    work page internal anchor Pith review Pith/arXiv arXiv

  4. [4]

    OpenAI Gym

    16 Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. Openai gym.CoRR, abs/1606.01540,

    work page internal anchor Pith review Pith/arXiv arXiv

  5. [5]

    Patrick P. K. Chan, Yaxuan Wang, and Daniel S. Yeung. Adversarial attack against deep reinforcement learning with static reward impact map. InProceedings of the 15th ACM Asia Conference on Computer and Communications Security, ASIA CCS ’20, pp. 334–343, New York, NY, USA, 2020a. Association for Computing Machinery. ISBN 9781450367509. doi: 10.1145/3320269...

    work page doi:10.1145/3320269.3384715

  6. [6]

    doi: 10.1145/3143561

    ISSN 0360-0300. doi: 10.1145/3143561. URLhttps://doi.org/10.1145/3143561. T.Y. Chen, S.C. Cheung, and S.M. Yiu. Metamorphic Testing: A New Approach for Generating Next Test Cases. Technical Report HKUST-CS98-01, Department of Computer Science, Hong Kong University of Science and Technology,

    work page doi:10.1145/3143561

  7. [7]

    Effective Test Generation Using Pre-trained Large Language Models and Mutation Testing

    Arghavan Moradi Dakhel, Amin Nikanjam, Vahid Majdinasab, Foutse Khomh, and Michel C. Desmarais. Ef- fective test generation using pre-trained large language models and mutation testing.CoRR, abs/2308.16557,

    work page internal anchor Pith review Pith/arXiv arXiv

  8. [8]

    Validating a Deep Learning Framework by Metamorphic Testing

    Junhua Ding, Xiaojun Kang, and Xin Hua Hu. Validating a Deep Learning Framework by Metamorphic Testing. InProceedings - 2017 IEEE/ACM 2nd International Workshop on Metamorphic Testing, MET 2017, pp. 28–34,

    work page 2017

  9. [9]

    Explaining and Harnessing Adversarial Examples

    Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572,

    work page internal anchor Pith review Pith/arXiv arXiv

  10. [10]

    Mungojerrie: Reinforcement Learning of Linear-Time Objectives

    Ernst Moritz Hahn, Mateo Perez, Sven Schewe, Fabio Somenzi, Ashutosh Trivedi, and Dominik Wojtczak. Mungojerrie: Reinforcement learning of linear-time objectives.CoRR, abs/2106.09161,

    work page internal anchor Pith review Pith/arXiv arXiv

  11. [11]

    Schorlemmer, Rohan Sethi, Yung-Hsiang Lu, George K

    ISBN 9781665457019. doi: 10.1109/ICSE48619.2023.00155. URLhttps://doi.org/10.1109/ICSE48619.2023.00155. Sandy H. Huang, Nicolas Papernot, Ian J. Goodfellow, Yan Duan, and Pieter Abbeel. Adversarial attacks on neural network policies.CoRR, abs/1702.02284,

    work page doi:10.1109/icse48619.2023.00155 2023

  12. [12]

    Adversarial Attacks on Neural Network Policies

    URLhttp://arxiv.org/abs/1702.02284. Zubair Khaliq, Sheikh Umar Farooq, and Dawood Ashraf Khan. Artificial intelligence in software testing : Impact, problems, challenges and prospect.CoRR, abs/2201.05371,

    work page internal anchor Pith review Pith/arXiv arXiv

  13. [14]

    URL https://arxiv.org/abs/1705.06452. Joel Z. Leibo, Edgar A. Duéñez-Guzmán, Alexander Vezhnevets, John P. Agapiou, Peter Sunehag, Raphael Koster, Jayd Matyas, Charlie Beattie, Igor Mordatch, and Thore Graepel. Scalable evaluation of multi-agent reinforcement learning with melting pot. InICML 2021,

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  14. [16]

    Tactics of Adversarial Attack on Deep Reinforcement Learning Agents

    URL https: //arxiv.org/abs/1703.06748. Chengjie Lu, Yize Shi, Huihui Zhang, Man Zhang, Tiexin Wang, Tao Yue, and Shaukat Ali. Learning configurations of operating environment of autonomous vehicles to maximize their collisions.IEEE Transactions on Software Engineering, 49(1):384–402,

    work page internal anchor Pith review Pith/arXiv arXiv

  15. [17]

    Quentin Mazouni, Helge Spieker, Arnaud Gotlieb, and Mathieu Acher

    doi: 10.1109/TSE.2022.3150788. Quentin Mazouni, Helge Spieker, Arnaud Gotlieb, and Mathieu Acher. A review of validation and verification of neural network-based policies for sequential decision making. InRencontres des Jeunes Chercheurs en Intelligence Artificielle (RJCIA),

    work page doi:10.1109/tse.2022.3150788 2022

  16. [18]

    A Review of Validation and Verification of Neural Network-based Policies for Sequential Decision Making

    URLhttps://arxiv.org/abs/2312.09680. Quentin Mazouni, Helge Spieker, Arnaud Gotlieb, and Mathieu Acher. Policy testing with mdpfuzz (replica- bility study). InProceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA 2024, pp. 1567–1578, New York, NY, USA, 2024a. Association for Computing Machinery. ISBN 979840070...

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.1145/3650212.3680382 2024

  17. [19]

    Playing Atari with Deep Reinforcement Learning

    18 Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. Playing atari with deep reinforcement learning.arXiv:1312.5602,

    work page internal anchor Pith review Pith/arXiv arXiv

  18. [20]

    Kaiser, Lifeng Hu, and Leon Wu

    Christian Murphy, Gail E. Kaiser, Lifeng Hu, and Leon Wu. Properties of machine learning applications for use in metamorphic testing. InProceedings of the Twentieth International Conference on Software Engineering & Knowledge Engineering (SEKE’2008), San Francisco, CA, USA, July 1-3, 2008, pp. 867–872. Knowledge Systems Institute Graduate School,

    work page 2008

  19. [21]

    Kohei Ohashi, Kosuke Nakanishi, Wataru Sasaki, Yuji Yasui, and Shin Ishii

    doi: 10.7916/D8XK8PFD. Kohei Ohashi, Kosuke Nakanishi, Wataru Sasaki, Yuji Yasui, and Shin Ishii. Deep adversarial reinforcement learning with noise compensation by autoencoder.IEEE Access, 9:143901–143912,

    work page doi:10.7916/d8xk8pfd

  20. [22]

    GPT-4 Technical Report

    OpenAI. GPT-4 technical report.CoRR, abs/2303.08774,

    work page internal anchor Pith review Pith/arXiv arXiv

  21. [23]

    Justin K

    doi: 10.1145/3361566. Justin K. Pugh, Lisa B. Soros, and Kenneth O. Stanley. Quality diversity: A new frontier for evolutionary computation.Frontiers in Robotics and AI, 3,

    work page doi:10.1145/3361566

  22. [24]

    doi: 10.3389/frobt.2016.00040

    ISSN 2296-9144. doi: 10.3389/frobt.2016.00040. URL https://www.frontiersin.org/journals/robotics-and-ai/articles/10.3389/frobt.2016.00040. Antonin Raffin. Rl baselines3 zoo.https://github.com/DLR-RM/rl-baselines3-zoo,

    work page doi:10.3389/frobt.2016.00040 2016

  23. [25]

    doi: 10.1109/TSE

    ISSN 1939-3520. doi: 10.1109/TSE. 2016.2532875. URLhttps://ieeexplore.ieee.org/abstract/document/7422146. Helge Spieker and Arnaud Gotlieb. Adaptive metamorphic testing with contextual bandits.Journal of Systems and Software, 165:110574,

    work page doi:10.1109/tse 1939

  24. [26]

    doi: 10.1016/j.jss.2020.110574

    ISSN 0164-1212. doi: 10.1016/j.jss.2020.110574. Helge Spieker, Nassim Belmecheri, Arnaud Gotlieb, and Nadjib Lazaar. Evaluating Human Trajectory Predic- tion with Metamorphic Testing. InProceedings of the 9th ACM International Workshop on Metamorphic Testing, MET 2024, pp. 34–40. Association for Computing Machinery,

    work page doi:10.1016/j.jss.2020.110574 2020

  25. [27]

    doi: 10.1145/3679006.3685071

    ISBN 9798400711176. doi: 10.1145/3679006.3685071. URLhttps://dl.acm.org/doi/10.1145/3679006.3685071. Marcel Steinmetz, Daniel Fiser, Hasan Ferit Eniser, Patrick Ferber, Timo P. Gros, Philippe Heim, Daniel Höller, Xandra Schuler, Valentin Wüstholz, Maria Christakis, and Jörg Hoffmann. Debugging a policy: Automatic action-policy testing in AI planning. InIC...

    work page doi:10.1145/3679006.3685071

  26. [28]

    URLhttps://arxiv.org/abs/ 2005.07099. L. Sun and Z. Q. Zhou. Metamorphic Testing for Machine Translations: MT4MT. In2018 25th Australasian Software Engineering Conference (ASWEC), pp. 96–100, November

    work page internal anchor Pith review Pith/arXiv arXiv 2005

  27. [29]

    19 Richard S Sutton and Andrew G Barto.Reinforcement learning: An introduction

    doi: 10.1109/ASWEC.2018.00021. 19 Richard S Sutton and Andrew G Barto.Reinforcement learning: An introduction. MIT press,

    work page doi:10.1109/aswec.2018.00021 2018

  28. [31]

    Intriguing properties of neural networks

    URLhttps://arxiv.org/abs/ 1312.6199. Martin Tappler, Filip Cano Cordoba, Bernhard Aichernig, and Bettina Könighofer. Search-based testing of reinforcement learning. In31st International Joint Conference on Artificial Intelligence and the 25th European Conference on Artificial Intelligence: IJCAI-ECAI 2022, pp. 503–510. ijcai. org,

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  29. [32]

    Junjie Wang, Yuchao Huang, Chunyang Chen, Zhe Liu, Song Wang, and Qing Wang

    URL https://zenodo.org/record/8127025. Junjie Wang, Yuchao Huang, Chunyang Chen, Zhe Liu, Song Wang, and Qing Wang. Software testing with large language models: Survey, landscape, and vision.IEEE Trans. Software Eng., 50(4):911–936,

    work page arXiv

  30. [33]

    URLhttps://doi.org/10.1093/comjnl/25.4.465

    doi: 10.1093/ COMJNL/25.4.465. URLhttps://doi.org/10.1093/comjnl/25.4.465. Xiaoyuan Xie, Joshua W.K. Ho, Christian Murphy, Gail Kaiser, Baowen Xu, and Tsong Yueh Chen. Testing and validating machine learning classifiers by metamorphic testing.Journal of Systems and Software, 84 (4):544–558,

    work page doi:10.1093/comjnl/25.4.465

  31. [34]

    MinAtar: An Atari-Inspired Testbed for Thorough and Reproducible Reinforcement Learning Experiments

    ISSN 2158987413. doi: 10.1016/j.jss.2010.11.920. URLhttp://dx.doi.org/10.1016/ j.jss.2010.11.920. Kenny Young and Tian Tian. Minatar: An atari-inspired testbed for more efficient reinforcement learning experiments.CoRR, abs/1903.03176,

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.1016/j.jss.2010.11.920 2010

  32. [35]

    doi: 10.1109/TSE.2023.3269804

    ISSN 1939-3520. doi: 10.1109/TSE.2023.3269804. 20

    work page doi:10.1109/tse.2023.3269804 1939