Pith. sign in

REVIEW 4 major objections 6 minor 42 references

Search-based Generation of Waypoints for Triggering Self-Adaptations in Maritime Autonomous Vessels

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read WPgen generates minor waypoint modifications that steer maritime autonomous vessels into unstable navigation, offering test scenarios for triggering and validating self-adaptation systems.

desk verdict A genuinely new waypoint-perturbation test generator, evaluated honestly but with an unvalidated fitness proxy and no direct trigger demonstration—worth a serious referee, conditional on closing that gap. read the letter →

arxiv 2507.16327 v1 pith:RG27N2YT submitted 2025-07-22 cs.SE

Karoline Nyl{ae}nder , Aitor Arrieta , Shaukat Ali , Paolo Arcaini This is my paper · ORCID
classification cs.SE
keywords autonomousvesselsself-adaptationwaypointgenerationsearch-basedsoftwaretestingNSGA-IImulti-objectiveoptimizationsimulation-basedmaritime
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that minor modifications to the waypoints guiding a maritime autonomous vessel can be generated automatically, and that those small edits are enough to provoke the kind of unstable navigation that should trigger the vessel's self-adaptation logic. The proposed approach, WPgen, casts waypoint generation as a two-objective search problem: keep the new waypoints as close as possible to the original route while maximizing a measure of path instability. Experiments on three simulated vessels show the approach outperforms random search with large effect size on the overwater tanker, while on the two underwater vessels the outcome depends on how the initial population is seeded. The practical aim is to give developers test scenarios for validating that their self-adaptation implementations trigger correctly before deployment.

What carries the argument

The load-bearing mechanism is NSGA-II, a multi-objective evolutionary algorithm, configured to optimize two fitness functions: $\text{fit}_{\text{distWPs}}(\text{WP})$ (Eq. 2), the Euclidean distance between original and modified waypoints, and $\text{fit}_{\text{unstable}}(\text{WP})$ (Eq. 3), the sum over consecutive waypoint pairs of the ratio between the simulated path length and the original straight-line distance. The simulator supplies the vessel models and the simulated paths, a death-penalty filter enforces the minimum-distance constraint between consecutive waypoints, and three seeding strategies—fully seeded, combined, and fully random—define the three WPgen variants. The instability objective in Eq. (3) is the proxy that is supposed to align with the self-adaptation triggering condition, and its correctness is what the paper's utility ultimately rests on.

What would settle it

Run WPgen on any of the three vessels to produce a Pareto front of modified waypoints, simulate each solution, and compute the fraction of sub-paths that the autocorrelation-based instability classifier marks as unstable. If that fraction is no higher for WPgen solutions than for randomly generated waypoints of the same distance from the original route, then the Eq. (3) objective is not actually steering the search toward unstable navigation and the central claim fails.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central claim is that WPgen can produce waypoints that are minor variations of a vessel's predefined route and that nevertheless drive the navigation software into unstable behavior, giving testers a way to exercise self-adaptation mechanisms. The search is driven by two competing objectives: minimizing the Euclidean distance between the generated and original waypoints (Eq. 2) and maximizing the summed ratio of each simulated sub-path length to the straight-line distance between the corresponding original waypoints (Eq. 3). When evaluated on the Mariner overwater tanker, all three seeding variants of WPgen beat random search with large effect size; on the Remus100 underwater vessel only the combined and fully random seeding variants beat random search, while the fully seeded variant was significantly worse; and on the Nspauv underwater vessel no approach differed significantly from random search. The paper treats this vessel-dependence as the main empirical message: the choice of search strategy and seeding must be tuned to the vessel under test.

Load-bearing premise

The assumption that the path-length ratio in Eq. (3) is a faithful proxy for the unstable navigation that should trigger a self-adaptation is the load-bearing premise, because the paper never shows that optimizing this ratio produces waypoints that the autocorrelation-based classifier would label unstable.

Editorial extensions

If this is right

  • For the overwater tanker Mariner, using any of the three WPgen seeding variants instead of random search is the recommended practice, since all three yield significantly better hypervolume with large effect size.
  • For underwater vessels, seeding the initial population with randomness (combined or fully random) is preferable to fully seeding it, because the fully seeded variant can be significantly worse than random search, as happened on Remus100.
  • The generated waypoints provide ready-made test scenarios for validating the Plan and Legitimate components of a MAPLE-K self-adaptation loop.
  • The sub-path classification (stable, unstable, missing) can be reused as a more direct fitness function for future search-based generation.
  • WPgen's effectiveness is vessel-dependent, so approach selection should be informed by the vessel model being tested.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the search objective (Eq. 3) and the evaluation classifier (autocorrelation of vessel angles) are different measures, a natural next step is to validate their agreement; if they disagree, the generated tests may still fail to trigger adaptations even when the fitness function improves.
  • The same encoding and search operators could be applied to other waypoint-following robots, such as unmanned aerial vehicles, since the approach does not rely on maritime-specific dynamics beyond what the simulator provides.
  • The vessel-dependence of results suggests that a cheap pre-screening phase could estimate whether a given vessel's navigation software is search-testable, avoiding wasted computational effort on vessels like Nspauv where random search is sufficient.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper presents WPgen, a multi-objective search-based approach for generating minor modifications to predefined waypoints of maritime autonomous vessels, with the aim of producing unstable navigation behavior that should trigger self-adaptation. WPgen uses NSGA-II with three seeding strategies and two objectives: minimizing the deviation from the original waypoints (Eq. (2)) and maximizing a path-length-based instability proxy (Eq. (3)). The approach is evaluated on three vessel models in the Marine Systems Simulator against random search, using hypervolume as the quality indicator and an autocorrelation-based classifier for path stability (RQ3). The results show that WPgen variants significantly outperform random search on the Mariner tanker, are mixed on the underwater vessels (with random search beating the fully seeded variant on Remus100), and are not significantly different on Nspauv.

Significance. If the connection between the Eq. (3) proxy and actual vessel instability were established, WPgen would be a useful contribution to scenario-based testing of self-adaptive maritime navigation software, addressing a real need within the RoboSAPIENS project. The paper's strengths include the use of a recognized maritime simulator (MSS), three realistic vessel models, standard statistical methodology (Mann-Whitney U, Vargha-Delaney A12, 30 runs), a replication package, and clear reporting of results. However, the central claim that WPgen generates waypoints that trigger self-adaptations is not yet supported, because the search objective is an unvalidated surrogate for the instability criterion used in evaluation.

major comments (4)
  1. [Section 3.4, Eq. (3); Section 4.3.3] Section 3.4, Eq. (3) defines fit_unstable as the sum of ratios of simulated sub-path lengths to straight-line distances between original waypoints, but Section 4.3.3 defines instability operationally via autocorrelation of vessel angles with a threshold of 0.1. The manuscript never demonstrates that maximizing Eq. (3) is correlated with this classifier; a long smooth deviation and a short oscillatory path can score oppositely on the two measures. Since RQ1 and RQ2 and the HV comparisons are computed in the Eq. (2)-Eq. (3) objective space, significant hypervolume improvements do not entail that WPgen produces more waypoints that would be classified as unstable. The authors should validate the proxy, for example by reporting the correlation or classification agreement between Eq. (3) and the autocorrelation labels, or by using the classifier as the fitness function.
  2. [Section 4.3.3] Section 4.3.3 describes the stability classifier only in general terms: it is not stated which axes are measured, at what sampling rate autocorrelation is computed, how the peak threshold of 0.1 was determined, or how the 'more than one threshold peak' rule was validated. Reference [26] is a general discussion of correlation coefficients and does not justify this specific threshold for vessel-angle autocorrelation. Without these details, the RQ3 classification results cannot be independently reproduced or assessed, which is important because that classifier is the only operational definition of 'unstable' in the paper.
  3. [Section 5.3, Table 2] Section 5.3, Table 2 reports percentages of stable, unstable, and missing sub-paths across all Pareto-front solutions (e.g., 18%-25% unstable for the underwater vessels and around 82%-83% for Mariner). These are descriptive post-hoc rates; they do not validate that high values of Eq. (3) predict the unstable class. A proper validation would compare the distribution of Eq. (3) values for classifier-labeled stable versus unstable sub-paths, or report precision and recall of the proxy at relevant thresholds.
  4. [Section 5.1, Table 1 and Figure 3] Section 5.1, Table 1 and Figure 3: the RQ1 conclusion that the problem is complex enough to require a guided algorithm is based on HV computed in the surrogate objective space. On Nspauv there are no significant differences, and on Remus100 random search significantly outperforms WPgenseed in HV; moreover, because the surrogate is unvalidated, even the Mariner advantage does not directly support the claim that WPgen triggers more self-adaptations.
minor comments (6)
  1. [Section 3.4, Eq. (3)] Equation (3) sums over N-1 sub-paths, but the text notes that the vessel may fail to reach all waypoints, in which case there are fewer than N-1 sub-paths; the summation bound and the treatment of missing sub-paths in the fitness function should be clarified.
  2. [Section 4.2.3] The hyperparameter Delta is set to 400 for Mariner and 150 for the underwater vessels based on vessel characteristics, but the sensitivity of the results to this choice is not studied; a brief sensitivity discussion would strengthen the parameter-setting rationale.
  3. [Section 4.3.3] The statement that a full path can have 2^{N-1} + 2(N-2) different categorizations is unexplained and appears inconsistent with the three sub-path categories; the reasoning behind this count should be made explicit.
  4. [Algorithm 1] Line 8 of Algorithm 1 is formatted ambiguously ('while p < 0.5 numOfMutParams'); it should be written as a clear condition, likely p < 0.5^{numOfMutParams} or p < 0.5 * numOfMutParams.
  5. [Author information] The fourth author's email address is identical to the second author's (aarrieta@mondragon.edu); this appears to be a typographical error.
  6. [Section 6.1] The sentence 'For Remus100, WPgen proved to be the best' is ambiguous because Table 1 shows that the choice of variant matters (WPgenseed loses to RS while WPgencomb and WPgenrnd beat RS); the text should identify which variant is meant.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: WPgen's fitness functions are defined from scratch and the reported comparisons are against the same objective space; the main limitation is an unvalidated fitness-vs-classifier proxy, which is a construct validity issue, not a circular derivation.

full rationale

The paper's derivation chain is self-contained. Eq. (2) and Eq. (3) define the two objectives (closeness and instability-as-path-length-ratio) directly from the simulation output and the original waypoints; neither is defined in terms of the result it is used to support, and no parameter is fitted to the evaluation data. RQ1/RQ2 compare HV values computed in the same two-objective space, which is a standard algorithm-effectiveness comparison rather than a prediction from fitted inputs. RQ3 uses an independently specified autocorrelation-based classifier (Sect. 4.3.3) to categorize the generated sub-paths; the percentages in Table 2 are post-hoc measurements, not quantities forced by the fitness function. The self-citations ([1], [13], [31], [38]) provide background, project context, or related work and are not load-bearing for the technical claim. The genuine weakness is that Eq. (3) is never shown to correlate with the autocorrelation classifier that defines 'unstable' in the evaluation; this is a construct-validity threat, not an equation-level or self-citation circularity. Honest non-finding is therefore appropriate.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The paper's claims rest on the simulator's fidelity, on the fitness proxy in Eq. (3), and on the autocorrelation classifier in Section 4.3.3. None of these are independently validated; the classifier threshold was chosen during an exploratory study. There are no new physical entities or forces, and the only fitted constants are the per-vessel Δ and the instability threshold.

free parameters (2)
  • Delta (search-space bound, Δ) = 400 (Mariner), 150 (Remus100, Nspauv)
    Bounds each coordinate perturbation around the original waypoint. Set per AV based on vessel characteristics in Section 4.2.3; no formal tuning procedure is described.
  • Autocorrelation instability threshold = 0.1 (and 'more than one peak')
    In Section 4.3.3, an angle is counted unstable if it has more than one autocorrelation peak above 0.1. The value is asserted with a reference that does not support it, and no sensitivity analysis is provided.
assumptions (3)
  • domain assumption The MSS simulator faithfully reproduces the navigation behavior of Mariner, Remus100, and Nspauv such that unstable simulated paths imply unsafe real-world behavior.
    Used throughout Section 4.2.1 and Section 5. The paper cites MSS as a widely used simulator but does not validate the three vessel models against real-world data.
  • domain assumption Eq. (3), the ratio of actual path length to straight-line distance between consecutive waypoints, is a valid proxy for unstable navigation.
    This proxy guides the entire search in Section 3.4. The paper provides no evidence that high normalized path length correlates with oscillatory vessel motion or with situations that would trigger a MAPE-K adaptation.
  • domain assumption The autocorrelation rule with threshold 0.1 correctly classifies sub-paths into stable, unstable, and missing.
    Assumed in Section 4.3.3 and used for the RQ3 results in Section 5.3. The paper notes that other metrics were tried and rejected during an exploratory study, but the accepted classifier is not validated against ground truth.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Search-based Generation of Waypoints for Triggering Self-Adaptations in Maritime Autonomous Vessels." pith.science (2026). https://pith.science/paper/RG27N2YT

@misc{pith2026250716327,
  author       = {Pith},
  title        = {Pith review of: Search-based Generation of Waypoints for Triggering Self-Adaptations in Maritime Autonomous Vessels},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RG27N2YT}},
  note         = {Machine review of arXiv:2507.16327}
}
read the original abstract

Self-adaptation in maritime autonomous vessels (AVs) enables them to adapt their behaviors to address unexpected situations while maintaining dependability requirements. During the design of such AVs, it is crucial to understand and identify the settings that should trigger adaptations, enabling validation of their implementation. To this end, we focus on the navigation software of AVs, which must adapt their behavior during operation through adaptations. AVs often rely on predefined waypoints to guide them along designated routes, ensuring safe navigation. We propose a multiobjective search-based approach, called WPgen, to generate minor modifications to the predefined set of waypoints, keeping them as close as possible to the original waypoints, while causing the AV to navigate inappropriately when navigating with the generated waypoints. WPgen uses NSGA-II as the multi-objective search algorithm with three seeding strategies for its initial population, resulting in three variations of WPgen. We evaluated these variations on three AVs (one overwater tanker and two underwater). We compared the three variations of WPgen with Random Search as the baseline and with each other. Experimental results showed that the effectiveness of these variations varied depending on the AV. Based on the results, we present the research and practical implications of WPgen.

Figures

Figures reproduced from arXiv: 2507.16327 by the authors.

Figure 1
Figure 1. WPgen in the Context of the MAPLE-K loop focuses on identifying waypoints that should trigger adaptations to prevent navigation software from malfunctioning. In particular, we propose a multi-objective search approach, called WPgen, to generate a set of waypoints by introducing minor modifications to an existing set of waypoints that should trigger adaptations. To guide the search, we defined two objectives, i.e., m… view at source ↗
Figure 2
Figure 2. Example of paths of Remus100 (North-East view) with different sub-path types: stable, unstable, and missing. A waypoint is shown as a red circle and labeled as wpi for ith waypoint. A path consists of waypoints from wp1 to wp7 . Fig. 2b shows the AV’s path is unstable between some waypoints. Fig. 2c shows the AV missing several waypoints. 4.3.3 Classification of sub-paths and paths For RQ3, we are interested in stud… view at source ↗
Figure 3
Figure 3. RQ1 and RQ2 – Distribution of the calculated HV metric for each autonomous vessel and approach [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 26 canonical work pages

  1. [26]

    Correlation coefficients: appropriate use and interpretation

    Patrick Schober, Christa Boer, and Lothar A Schwarte. Correlation coefficients: appropriate use and interpretation. Anesthesia & analgesia, 126(5):1763–1768, 2018

    work page 2018

  2. [1]

    Modeling and analyzing MAPE-K feedback loops for self-adaptation

    Paolo Arcaini, Elvinia Riccobene, and Patrizia Scandurra. Modeling and analyzing MAPE-K feedback loops for self-adaptation. InProceedings of the 10th International Symposium on Software Engineering for Adaptive and Self-Managing Systems, SEAMS ’15, pages 13–23. IEEE Press, 2015. doi: 10.1109/SEAMS.2015.10

    doi:10.1109/seams.2015.10 2015

  3. [2]

    Thor Fossen.Handbook of Marine Craft Hydrodynamics and Motion Control. 04 2021. ISBN 978-1119575047. doi: 10.1002/9781119994138

    doi:10.1002/9781119994138 2021

  4. [3]

    On the verification and validation of AI navigation algorithms

    Ivan Porres, Sepinoud Azimi, Sébastien Lafond, Johan Lilius, Johanna Salokannel, and Mirva Salokorpi. On the verification and validation of AI navigation algorithms. InGlobal Oceans 2020: Singapore – U.S. Gulf Coast, pages 1–8, 2020. doi: 10.1109/IEEECONF38699.2020.9389133. 12 Running Title for Header

    arXiv 2020

  5. [4]

    Scenario-based testing of a ship collision avoidance system

    Ivan Porres, Sepinoud Azimi, and Johan Lilius. Scenario-based testing of a ship collision avoidance system. In2020 46th Euromicro Conference on Software Engineering and Advanced Applications (SEAA), pages 545–552. IEEE,

    work page

  6. [5]

    Lokukaluge P. Perera. Deep learning toward autonomous ship navigation and possible COLREGs failures. Journal of Offshore Mechanics and Arctic Engineering, 142(3):031102, 12 2019. ISSN 0892-7219. doi: 10.1115/1.4045372. URLhttps://doi.org/10.1115/1.4045372

    work page doi:10.1115/1.4045372 2019

  7. [6]

    Tobias Rye Torben, Jon Arne Glomsrud, Tom Arne Pedersen, Ingrid B Utne, and Asgeir J Sørensen. Automatic simulation-based testing of autonomous ships using Gaussian processes and temporal logic.Proceedings of the Institution of Mechanical Engineers, Part O: journal of Risk and Reliability, 237(2):293–313, 2023. doi: 10.1177/1748006X211069277. URLhttps://d...

    work page doi:10.1177/1748006x211069277 2023

  8. [7]

    Randomly testing an autonomous collision avoidance system with real-world ship encounter scenario from AIS data.Journal of Marine Science and Engineering, 10(11), 2022

    Feixiang Zhu, Zhengyu Zhou, and Hongrui Lu. Randomly testing an autonomous collision avoidance system with real-world ship encounter scenario from AIS data.Journal of Marine Science and Engineering, 10(11), 2022. ISSN 2077-1312. doi: 10.3390/jmse10111588. URLhttps://www.mdpi.com/2077-1312/10/11/1588

    work page doi:10.3390/jmse10111588 2022

Show all 42 references
  1. [8]

    A novel deterministic search-based algorithm for multi-ship collaborative collision avoidance decision-making.Ocean Engineering, 292:116524, 2024

    Rong Zhen, Ziqiang Shi, Qiyong Gu, and Shenhua Yang. A novel deterministic search-based algorithm for multi-ship collaborative collision avoidance decision-making.Ocean Engineering, 292:116524, 2024. ISSN 0029-8018. doi: https://doi.org/10.1016/j.oceaneng.2023.116524. URL http...

    2024

  2. [9]

    Search-based software engineering.Information and Software Technology, 43(14):833–839, 2001

    Mark Harman and Bryan F Jones. Search-based software engineering.Information and Software Technology, 43(14):833–839, 2001. ISSN 0950-5849. doi: https://doi.org/10.1016/S0950-5849(01)00189-6. URL https: //www.sciencedirect.com/science/article/pii/S0950584901001896

    2001 doi

  3. [10]

    [Online; accessed 25-March-2025]

    Marine Systems Simulator.https://www.fossen.biz/MSS/. [Online; accessed 25-March-2025]

    2025

  4. [11]

    A vessel’s mathematical model and its real counterpart: A comparative methodology based on a real-world study.The journal of Navigation, 69(6):1379–1392, 2016

    Krzysztof Czaplewski and Piotr Zwolan. A vessel’s mathematical model and its real counterpart: A comparative methodology based on a real-world study.The journal of Navigation, 69(6):1379–1392, 2016

    2016

  5. [12]

    [Online; accessed 25-March-2025]

    RoboSAPIENS.https://robosapiens-eu.tech/. [Online; accessed 25-March-2025]

    2025

  6. [13]

    Robotic safe adaptation in unprecedented situations: the robosapiens project.Research Directions: Cyber-Physical Systems, 2:e4, 2024

    Peter G Larsen, Shaukat Ali, Roland Behrens, Ana Cavalcanti, Claudio Gomes, Guoyuan Li, Paul De Meulenaere, Mikkel L Olsen, Nikolaos Passalis, Thomas Peyrucain, et al. Robotic safe adaptation in unprecedented situations: the robosapiens project.Research Directions: Cyber-Physi...

    2024

  7. [14]

    A review of constraint-handling techniques for evolution strategies.Appl

    Oliver Kramer. A review of constraint-handling techniques for evolution strategies.Appl. Comp. Intell. Soft Comput., 2010, January 2010. ISSN 1687-9724. doi: 10.1155/2010/185063. URL https://doi.org/10.1155/ 2010/185063

    2010 doi

  8. [15]

    The seed is strong: Seeding strategies in search-based software testing

    Gordon Fraser and Andrea Arcuri. The seed is strong: Seeding strategies in search-based software testing. In Proceedings of the 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation, ICST ’12, pages 121–130, USA, 2012. IEEE Computer Society....

    2012 doi

  9. [16]

    Agirre, and Goiuria Sagardui

    Aitor Arrieta, Pablo Valle, Joseba A. Agirre, and Goiuria Sagardui. Some seeds are strong: Seeding strategies for search-based test case selection.ACM Trans. Softw. Eng. Methodol., 32(1), February 2023. ISSN 1049-331X. doi: 10.1145/3532182. URLhttps://doi.org/10.1145/3532182

    2023 doi

  10. [17]

    Seeding strategies in search-based unit test generation

    José Miguel Rojas, Gordon Fraser, and Andrea Arcuri. Seeding strategies in search-based unit test generation. Softw. Test. Verif. Reliab., 26(5):366–401, August 2016. ISSN 0960-0833. doi: 10.1002/stvr.1601. URL https://doi.org/10.1002/stvr.1601

    2016 doi

  11. [18]

    PlatEMO: A MATLAB platform for evolutionary multi- objective optimization.IEEE Computational Intelligence Magazine, 12(4):73–87, 2017

    Ye Tian, Ran Cheng, Xingyi Zhang, and Yaochu Jin. PlatEMO: A MATLAB platform for evolutionary multi- objective optimization.IEEE Computational Intelligence Magazine, 12(4):73–87, 2017

    2017

  12. [19]

    An overview of the marine systems simulator (MSS): A Simulink toolbox for marine control systems.Modeling, identification and Control, 27(4):259–275, 2006

    Tristan Perez, Oyvind Smogeli, Thor Fossen, and Asgeir J Sorensen. An overview of the marine systems simulator (MSS): A Simulink toolbox for marine control systems.Modeling, identification and Control, 27(4):259–275, 2006

    2006

  13. [20]

    A Simulated 4-DOF Ship Motion Dataset for System Identification under Environmental Disturbances, 2022

    Alexandra Baier and Steffen Staab. A Simulated 4-DOF Ship Motion Dataset for System Identification under Environmental Disturbances, 2022. URLhttps://doi.org/10.18419/DARUS-2905

    2022 doi

  14. [21]

    Parameter identification of ship manoeuvring model under disturbance using support vector machine method.Ships and Offshore Structures, 16 (sup1):13–21, 2021

    Tongtong Wang, Guoyuan Li, Baiheng Wu, Vilmar Æsøy, and Houxiang Zhang. Parameter identification of ship manoeuvring model under disturbance using support vector machine method.Ships and Offshore Structures, 16 (sup1):13–21, 2021

    2021

  15. [22]

    K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan. A fast and elitist multiobjective genetic algorithm: NSGA- II.Trans. Evol. Comp, 6(2):182–197, April 2002. ISSN 1089-778X. doi: 10.1109/4235.996017. URL https://doi.org/10.1109/4235.996017. 13 Running Title for Header

    2002

  16. [23]

    A practical guide for using statistical tests to assess randomized algorithms in software engineering

    Andrea Arcuri and Lionel Briand. A practical guide for using statistical tests to assess randomized algorithms in software engineering. InProceedings of the 33rd International Conference on Software Engineering, ICSE ’11, pages 1–10, New York, NY , USA, 2011. Association for C...

    2011

  17. [24]

    A survey on the hypervolume indicator in evolutionary multiobjective optimization.IEEE Transactions on Evolutionary Computation, 25(1):1–20, 2021

    Ke Shang, Hisao Ishibuchi, Linjun He, and Lie Meng Pang. A survey on the hypervolume indicator in evolutionary multiobjective optimization.IEEE Transactions on Evolutionary Computation, 25(1):1–20, 2021. doi: 10.1109/ TEVC.2020.3013290

    2021

  18. [25]

    Summary and analysis of extension program evaluation in R, 2016, revised

    Salvatore S Mangiafico. Summary and analysis of extension program evaluation in R, 2016, revised

    2016

  19. [27]

    Betty H. Cheng, Rogério Lemos, Holger Giese, Paola Inverardi, Jeff Magee, Jesper Andersson, Basil Becker, Nelly Bencomo, Yuriy Brun, Bojan Cukic, Giovanna Marzo Serugendo, Schahram Dustdar, Anthony Finkelstein, Cristina Gacek, Kurt Geihs, Vincenzo Grassi, Gabor Karsai, Holger ...

    2009

  20. [28]

    Müller, Mary Shaw, Jesper Andersson, Marin Litoiu, Bradley Schmerl, Gabriel Tamura, Norha M

    Rogério de Lemos, Holger Giese, Hausi A. Müller, Mary Shaw, Jesper Andersson, Marin Litoiu, Bradley Schmerl, Gabriel Tamura, Norha M. Villegas, Thomas V ogel, Danny Weyns, Luciano Baresi, Basil Becker, Nelly Bencomo, Yuriy Brun, Bojan Cukic, Ron Desmarais, Schahram Dustdar, Gr...

    2013

  21. [29]

    A testing scheme for self-adaptive software systems with architectural runtime models

    Joachim Hänsel, Thomas V ogel, and Holger Giese. A testing scheme for self-adaptive software systems with architectural runtime models. In2015 IEEE International Conference on Self-Adaptive and Self-Organizing Systems Workshops, pages 134–139, Sep. 2015. doi: 10.1109/SASOW.2015.27

    2015 doi

  22. [30]

    Towards testing self-organizing, adaptive systems

    Benedikt Eberhardinger, Hella Seebach, Alexander Knapp, and Wolfgang Reif. Towards testing self-organizing, adaptive systems. InProceedings of the 26th IFIP WG 6.1 International Conference on Testing Software and Systems - Volume 8763, ICTSS 2014, pages 180–185, New York, NY ,...

    2014 doi

  23. [31]

    Model-based testing for MAPE-K adaptation control loops

    Paolo Arcaini, Raffaela Mirandola, Elvinia Riccobene, and Patrizia Scandurra. Model-based testing for MAPE-K adaptation control loops. In2020 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pages 43–51, 2020. doi: 10.1109/ICSTW...

    2020

  24. [32]

    Fredericks, Andres J

    Erik M. Fredericks, Andres J. Ramirez, and Betty H. C. Cheng. Towards run-time testing of dynamic adaptive systems. InProceedings of the 8th International Symposium on Software Engineering for Adaptive and Self- Managing Systems, SEAMS ’13, pages 169–174. IEEE Press, 2013. ISB...

    2013

  25. [33]

    Fredericks, Byron DeVries, and Betty H

    Erik M. Fredericks, Byron DeVries, and Betty H. C. Cheng. Towards run-time adaptation of test cases for self-adaptive systems in the face of uncertainty. InProceedings of the 9th International Symposium on Software Engineering for Adaptive and Self-Managing Systems, SEAMS 2014...

    2014

  26. [34]

    Afshin Mansouri, and Yuanyuan Zhang

    Mark Harman, S. Afshin Mansouri, and Yuanyuan Zhang. Search-based software engineering: Trends, techniques and applications.ACM Comput. Surv., 45(1), December 2012. ISSN 0360-0300. doi: 10.1145/2379776.2379787. URLhttps://doi.org/10.1145/2379776.2379787

    2012

  27. [35]

    A survey on automated driving system testing: Landscapes and trends.ACM Trans

    Shuncheng Tang, Zhenya Zhang, Yi Zhang, Jixiang Zhou, Yan Guo, Shuang Liu, Shengjian Guo, Yan-Fu Li, Lei Ma, Yinxing Xue, and Yang Liu. A survey on automated driving system testing: Landscapes and trends.ACM Trans. Softw. Eng. Methodol., 32(5), July 2023. ISSN 1049-331X. doi: ...

    2023 doi

  28. [36]

    AmbieGen: A search-based framework for au- tonomous systems testing.Science of Computer Programming, 230:102990, 2023

    Dmytro Humeniuk, Foutse Khomh, and Giuliano Antoniol. AmbieGen: A search-based framework for au- tonomous systems testing.Science of Computer Programming, 230:102990, 2023. ISSN 0167-6423. doi: https://doi.org/10.1016/j.scico.2023.102990. URL https://www.sciencedirect.com/scie...

    2023

  29. [37]

    Simulation-based test case generation for unmanned aerial vehicles in the neighborhood of real flights

    Sajad Khatiri, Sebastiano Panichella, and Paolo Tonella. Simulation-based test case generation for unmanned aerial vehicles in the neighborhood of real flights. In2023 IEEE Conference on Software Testing, Verification and Validation (ICST), pages 281–292, 2023. doi: 10.1109/IC...

    2023

  30. [38]

    Uncertainty-wise test case generation and minimization for cyber-physical systems.journal of Systems and Software, 153:1–21, 2019

    Man Zhang, Shaukat Ali, and Tao Yue. Uncertainty-wise test case generation and minimization for cyber-physical systems.journal of Systems and Software, 153:1–21, 2019. ISSN 0164-1212. doi: https://doi.org/10.1016/j.jss. 2019.03.011. URLhttps://www.sciencedirect.com/science/art...

    2019 doi

  31. [92]

    URLhttps://doi.org/10.1109/ICST.2012.92

    2012 doi

  32. [2014]

    ISBN 9781450328647

    Association for Computing Machinery. ISBN 9781450328647. doi: 10.1145/2593929.2593937. URL https://doi.org/10.1145/2593929.2593937

  33. [2020]

    URLhttps://doi.org/10.1109/SEAA51224.2020.00090

    doi: 10.1109/SEAA51224.2020.00090. URLhttps://doi.org/10.1109/SEAA51224.2020.00090

    2020

  34. [2024]

    (Pdf version: http://rcompanion.org/documents/ RHandbookProgramEvaluation.pdf)

    URL http://rcompanion.org/handbook/. (Pdf version: http://rcompanion.org/documents/ RHandbookProgramEvaluation.pdf). Version 1.20.07

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.