Pith. sign in

REVIEW 2 major objections 5 minor 1 cited by

Search-Based Fuzzing For RESTful APIs That Use MongoDB

T0 review · 2 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read MongoDB-aware fuzzing significantly improves white-box REST API testing, with max effect size on four of six APIs.

desk verdict Solid empirical extension of EvoMaster to MongoDB; just don't take the RQ1 gains as evidence for the distance heuristic until it's isolated from direct insertion. read the letter →

arxiv 2507.20848 v1 pith:E5TFFR43 submitted 2025-07-28 cs.SE cs.DB

Hernan Ghianni , Man Zhang , Juan P. Galeotti , Andrea Arcuri This is my paper · ORCID
classification cs.SEcs.DB
keywords RESTAPIfuzzingsearch-basedsoftwaretestingMongoDBNoSQLdatabasetestcasegenerationcodecoveragewhite-boxheuristics
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper's thesis is that a white-box fuzzer for REST APIs can be substantially more effective for MongoDB-backed services if it stops treating the database as a black box. It proposes two concrete mechanisms: a NoSQL distance that scores how close each empty query result is to returning documents, and a way for generated test cases to insert documents straight into MongoDB with field types inferred from the API's own result classes. The authors argue this matters because the database boundary blocks ordinary branch-distance guidance, and because many microservices cannot create via HTTP the data they serve, making direct insertion the only practical route. On six real-world APIs, the extended fuzzer improves average line coverage from 40.0% to 47.3%, with the maximum effect size (every Mongo run better than every baseline run) on four of the six and a peak gain of 18%.

What carries the argument

The load-bearing device is a NoSQL distance $H_c(F)=\min_{d\in c}H_d(F)$: for each filter $F$ observed on a find() call that returned no documents, the instrumented tool re-executes the find and scores every stored document by recursively translating the filter's operators into branch-distance-style quantities (using the standard branch distance $\rho$ and normalizing conjunctions with $\nu(x)=x/(x+1)$), so that a score of zero means the document satisfies the filter. This distance is added as a secondary objective in the search, breaking ties among test cases with equal primary coverage. A second device is direct document insertion: the fuzzer emits a MongoDB insertion call whose JSON payload is generated from field types inferred reflectively from the repository's result class, so the inserted document matches the format the API expects to read.

What would settle it

Run a controlled ablation on the same six APIs under the paper's one-hour, 30-repetition protocol, comparing Base, Base plus the NoSQL distance only, Base plus direct document insertion only, and the full combination. If the distance-only variant shows no coverage gain over Base while the full variant keeps $\hat{A}_{12}=1.00$ on the same four APIs, the heuristic component is not the load-bearing mechanism and the paper's explanation of the gain would need revision.

Watch

Extended reading notes

Core claim

The discovery is that MongoDB interactions can be turned from an opaque sink for search gradient into a measurable secondary objective. The tool instruments every call to the MongoDB driver's find() method, records filters that return no documents, and after each test execution re-runs those filters to compute a distance $H_c(F)=\min_{d\in c}H_d(F)$ over the documents currently in the collection; this distance is added to the fuzzer's fitness function as a tie-breaker. Alongside this, a new initialization call lets a generated test insert a JSON document into a specified collection, with the document's fields sampled to match the types inferred by reflection from the repository's result class, so the inserted data satisfies the implicit schema. The empirical claim is that, compared with the default white-box tool, this combination raises line coverage on five of the six studied APIs (with average effect size $\hat{A}_{12}=0.91$), finds more faults on three, and does so while generating tests that issue fewer HTTP calls on average.

Load-bearing premise

The load-bearing premise is that rewarding test cases whose empty MongoDB queries come closer to returning data will, often enough, translate into higher code coverage and more fault detection; the paper itself flags this as an uncertain secondary objective.

Editorial extensions

If this is right

  • On five of the six studied APIs, the MongoDB-aware variant beats the default white-box fuzzer in line coverage, with average effect size $\hat{A}_{12}=0.91$ and the maximum $\hat{A}_{12}=1.00$ on four APIs, meaning every one of the 30 runs was better than every baseline run there.
  • Average line coverage rises from 40.0% to 47.3% across the six APIs, with the largest gain from 57.3% to 75.7% on one API.
  • The technique detects more faults on three of the six APIs, with an average effect size of $\hat{A}_{12}=0.70$ for fault count.
  • Because monitoring happens at the MongoDB Java driver level, the approach works regardless of the object-document mapper the API uses internally.
  • Generated tests need fewer HTTP calls on average to reach higher coverage, since data can be inserted directly instead of being built through lengthy endpoint sequences.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: an ablation separating the two mechanisms would tell practitioners whether direct insertion alone, without the distance heuristic, already explains the $\hat{A}_{12}=1.00$ results; if so, that would be the cheaper configuration to adopt.
  • Editorial inference: because the distance is computed after the test run on a database state that may have changed, a real-time or incremental version of the heuristic could provide a more stable gradient for write-heavy APIs.
  • Editorial inference: the same filter-distance idea could be turned into a test oracle, searching for documents that satisfy a query's negation and thereby trigger error-handling paths, rather than only using the distance as a coverage guide.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper presents an extension of EvoMaster for white-box fuzzing of REST APIs that interact with MongoDB. Two mechanisms are introduced: (i) runtime instrumentation that monitors MongoCollection.find() calls and computes a "NoSQL distance" heuristic approximating how close a collection's existing documents are to satisfying a filter that returned no results, used as a secondary objective (§5.1–§5.3); and (ii) direct generation of MongoDB insertion calls from test cases, with a schema inferred reflectively from the expected result classes observed at find() sites (§6). The approach is evaluated on six JVM-based REST APIs from the EMB corpus that use MongoDB. RQ1 compares the extended tool (Mongo) against default EvoMaster (Base) over 30 one-hour runs per API, reporting statistically significant coverage gains on five of six APIs with average A12=0.91 and up to 18% average improvement, plus increased fault detection on three APIs. RQ2 compares against four black-box fuzzers (ARAT-RL, EvoMaster BB, Restler, Schemathesis), where Mongo obtains the best average rank (1.5) and best average JaCoCo line coverage (52.7%). The paper includes a replication package.

Significance. The work addresses a real and increasingly relevant problem: REST APIs whose control flow depends on data stored in schema-less NoSQL databases, where standard branch-distance heuristics give no gradient. The proposed instrumentation at the MongoDB driver level is a reasonable, ODM-independent design. The empirical study is solid in several respects: 30 repetitions, one-hour sessions, standard statistical tests, and honest reporting of cases where the method did not win (e.g., ocvn-rest, and session-service vs ARAT-RL). The authors also disclose the secondary nature of the NoSQL distance objective and the fact that a higher cost per test evaluation leads to fewer HTTP calls. The open-source implementation and replication package are valuable strengths. The main weakness is that the experimental design conflates the two proposed mechanisms, so the evidence for the NoSQL distance heuristic—explicitly listed as a first novelty in the introduction—is missing. This is fixable with an ablation, but until that is done the paper's strongest claims should be read as being about the combined technique.

major comments (2)
  1. [§7.3, Table 5, RQ1] The Base-vs-Mongo comparison does not isolate the NoSQL distance heuristic (§5) from direct MongoDB insertion (§6). In the Mongo configuration both mechanisms are always enabled, and direct insertion alone can trivially make a previously empty find() return data by writing documents before the SUT is called; this mechanism is an adaptation of the SQL technique in [22], which the paper itself cites as effective. Because the paper's first stated contribution is the NoSQL heuristic, and because §5.3 concedes that "we cannot be sure that having data returned by the database will necessarily have a beneficial impact on achieving coverage of g," the observed A12=1.00 on four of six APIs cannot be attributed to that heuristic. An ablation (e.g., a configuration with the heuristic but without direct insertion, and/or direct insertion without the heuristic) is needed to support the contribution as stated; if the authors intend to claim only the combined technique, the claims and title should be adjusted accordingly.
  2. [§7.3, Table 5, RQ1] The paper reports the average number of detected faults for Base and Mongo but never describes what constitutes a detected fault or which oracles were used. EvoMaster has several built-in fault oracles, and the choice of oracle materially affects the counts. Without this information the RQ1 statement "more faults are found" cannot be interpreted or reproduced. Please specify the oracle configuration, the counting procedure, and whether faults were counted per unique failing HTTP call or per root cause.
minor comments (5)
  1. [§7.2–§7.3] The coverage metric differs between the RQ1 and RQ2 experiments (EvoMaster's own reporting tooling for RQ1 vs JaCoCo on generated test suites for RQ2). Please state explicitly which metric underlies the percentages in the abstract and in each table, and discuss the implications for comparing effect sizes across the two research questions.
  2. [§5.2, Table 3] The definition of Hd('f', C) relies on branch distance rho for string equality, but Table 1 only defines rho for numeric operands; a brief sentence describing how Levenshtein distance is combined with the numeric branch distance would remove ambiguity.
  3. [§5.2 and §6] The heuristic is only computed when the target collection is non-empty; when the collection is empty, no gradient is available. This design choice, combined with the probabilistic insertion described in §6, further suggests that the direct-insertion mechanism may be the dominant driver of the reported results and should be examined in the ablation.
  4. [§7.3, Table 6] The Friedman test is reported globally, but no post-hoc pairwise comparisons are given; the claim that Mongo is the best tool rests on average ranks and coverage averages, which would benefit from pairwise effect sizes.
  5. [Throughout] There are several typos and nonstandard spellings: "dependant" should be "dependent," "Surpringly" should be "Surprisingly," and "standarized" should be "standardized."

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the MongoDB heuristics and direct-insertion techniques are assessed against independently measured coverage, and no claimed result reduces by construction to its inputs.

full rationale

The paper's central claim is empirical: the Mongo configuration of EvoMaster improves coverage and fault detection relative to Base on six APIs. Coverage is measured independently of the search heuristics (line/branch coverage reported by EvoMaster's tooling for RQ1 and JaCoCo for RQ2), so the heuristic Hc(F) = min_d Hd(F) is not used as the evaluation metric. The NoSQL distance is explicitly a secondary objective: 'this is just a secondary objective, as we cannot be sure that having data returned by the database will necessarily have a beneficial impact on achieving coverage of g' (Section 5.3). This disclaims any definitional link between the heuristic and the measured outcome. The direct MongoDB insertion technique infers a document format from the expected result class and mutates insertions, but its effect on coverage is again measured externally; no parameter is fitted to the test APIs and then relabeled as a prediction. The paper builds on prior work by the same authors ([22], [25]), and EvoMaster is a self-cited tool, but these citations provide the base technology rather than a forced conclusion: the contribution is an extension evaluated against Base, four black-box fuzzers, and the EMB corpus. The absence of an ablation separating the NoSQL distance heuristic from direct insertion is a real threat to attributing the RQ1 gains to the heuristic specifically, but that is an experimental confound, not circularity; there is no equation or definition in the paper that makes the measured improvement equal to the heuristic by construction.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new theoretical entities, forces, or dimensions. Its contributions are engineering heuristics and a direct data insertion mechanism. The main assumptions are about the validity of the heuristics in guiding the search and the completeness of the driver-level monitoring, both of which are reasonable but not proven.

free parameters (2)
  • Normalization function ν(x) = x/(x+1)
    Chosen to map scores to [0,1] and prevent one score from dominating, following prior work [15]. This is a design choice that affects the heuristic but is not fitted to the target results.
  • K constant in branch distance
    Used in the branch distance definition to ensure a positive value when a condition is false. Standard from the search-based testing literature.
assumptions (3)
  • domain assumption Monitoring calls to MongoCollection.find() at the driver level captures all relevant database interactions, regardless of the ODM used.
    Section 5.1 states that all ODMs ultimately invoke the Java driver's find() methods, but this is not formally proven and may not hold for every MongoDB driver version.
  • domain assumption The heuristic Hc(F) = min over documents of Hd(F) provides useful gradient for the search.
    Applied in Section 5.2; the paper acknowledges in Section 5.3 that returning data may not help coverage, making this a questionable but honest assumption.
  • domain assumption Inserting documents that match the inferred data format will not cause exceptions in the SUT's parsing code.
    Section 6 relies on reflection to infer the expected document format, but this inference can be incomplete or wrong for complex schemas.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Search-Based Fuzzing For RESTful APIs That Use MongoDB." pith.science (2026). https://pith.science/paper/E5TFFR43

@misc{pith2026250720848,
  author       = {Pith},
  title        = {Pith review of: Search-Based Fuzzing For RESTful APIs That Use MongoDB},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/E5TFFR43}},
  note         = {Machine review of arXiv:2507.20848}
}
read the original abstract

In RESTful APIs, interactions with a database are a common and crucial aspect. When generating whitebox tests, it is essential to consider the database's state (i.e., the data contained in the database) to achieve higher code coverage and uncover more hidden faults. This article presents novel techniques to enhance search-based software test generation for RESTful APIs interacting with NoSQL databases. Specifically, we target the popular MongoDB database, by dynamically analyzing (via automated code instrumentation) the state of the database during the test generation process. Additionally, to achieve better results, our novel approach allows inserting NoSQL data directly from test cases. This is particularly beneficial when generating the correct sequence of events to set the NoSQL database in an appropriate state is challenging or time-consuming. This method is also advantageous for testing read-only microservices. Our novel techniques are implemented as an extension of EvoMaster, the only open-source tool for white-box fuzzing RESTful APIs. Experiments conducted on six RESTful APIs demonstrated significant improvements in code coverage, with increases of up to 18% compared to existing white-box approaches. To better highlight the improvements of our novel techniques, comparisons are also carried out with four state-of-the-art black-box fuzzers.

Figures

Figures reproduced from arXiv: 2507.20848 by the authors.

Figure 1
Figure 1. Example of RESTful API in Java SpringBoot, with two endpoints (a [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. A JSON representation of the object that is stored in the database. The [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. A test case generated by EvoMaster that satisfies the true branch in Figure 1. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: A test case generated by EvoMaster that satisfies the true branch in [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Snippet of the RESTful API using the Object-Document Mapper Morphia instead of Spring Data [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 2
Figure 2. Figure 2: d = {‘‘ id” : ‘‘<auto-generated-Id>”, ‘‘x” : 42, ‘‘y” : ‘‘b”, ‘‘z” : 1.0} Based on the definition of the heuristic score Hd(F) for document d and a filter F, we need to compute the score Hd(‘‘x”, {‘‘$eq” : 17}) [PITH_FULL_IMAGE:figures/full_fig_p007_2.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. WFC/WFD: Web Fuzzing Commons, Dataset and Guidelines to Support Experimentation in REST API Fuzzing

    cs.SE 2025-09 conditional novelty 6.0 of 10

    The paper introduces standardized schemas for REST API authentication and fault reporting, a 36-API Docker-backed dataset, and a six-fuzzer comparison with guidelines for fair experimentation.

Reference graph

Works this paper leans on

58 extracted references · 41 canonical work pages · cited by 1 Pith paper

  1. [22]

    Andrea Arcuri and Juan P Galeotti. 2020. Handling SQL databases in automated system test generation. ACM Transactions on Software Engineering and Methodology (TOSEM)29, 4 (2020), 1--31

    work page 2020

  2. [1]

    EvoMaster Benchmark (EMB)

    [n.d.]. EvoMaster Benchmark (EMB). https://github.com/WebFuzzing/EMB. Online, Accessed July 28, 2025

    work page 2025

  3. [2]

    [n.d.]. JaCoCo. https://www.jacoco.org/. Online, Accessed July 28, 2025

    work page 2025

  4. [3]

    [n.d.]. KMongo. https://litote.org/kmongo/. Online, Accessed July 28, 2025

    work page 2025

  5. [4]

    Micronaut Data MongoDB

    [n.d.]. Micronaut Data MongoDB. https://micronaut-projects.github.io/micronaut- data/latest/guide/#mongo. Online, Accessed July 28, 2025

    work page 2025

  6. [5]

    [n.d.]. MongoDB. https://www.mongodb.com/. Online, Accessed July 28, 2025. 2https://github.com/WebFuzzing/EvoMaster 14

    work page 2025

  7. [6]

    MongoDB Java Drivers

    [n.d.]. MongoDB Java Drivers. https://www.mongodb.com/docs/drivers/java-drivers/. Online, Accessed July 28, 2025

    work page 2025

  8. [7]

    [n.d.]. Morphia. https://morphia.dev/. Online, Accessed July 28, 2025

    work page 2025

Show all 58 references
  1. [8]

    Query Operators for MongoDB

    [n.d.]. Query Operators for MongoDB. https://www.mongodb.com/docs/manual/reference/operator/query/. Online, Accessed July 28, 2025

    2025

  2. [9]

    RestAssured

    [n.d.]. RestAssured. https://github.com/rest-assured/rest-assured. Online, Accessed July 28, 2025

    2025

  3. [10]

    Spring Boot

    [n.d.]. Spring Boot. https://spring.io/projects/spring-boot. Online, Accessed July 28, 2025

    2025

  4. [11]

    Spring Data MongoDB

    [n.d.]. Spring Data MongoDB. https://spring.io/projects/spring-data-mongodb. Online, Accessed July 28, 2025

    2025

  5. [12]

    [n.d.]. SQL. https://www.iso.org/standard/63555.html. Online, Accessed July 28, 2025

    2025

  6. [13]

    Ali, L.C

    S. Ali, L.C. Briand, H. Hemmati, and R.K. Panesar-Walawege. 2010. A systematic review of the application and empirical investigation of search-based test-case generation. IEEE Transactions on Software Engineering (TSE) 36, 6 (2010), 742--762

    2010

  7. [14]

    Mohammad Alshraideh and Leonardo Bottaci. 2006. Search-based software test data generation for string data using program-specific search operators. Software Testing, Verification and Reliability (STVR)16, 3 (2006), 175--203

    2006

  8. [15]

    A. Arcuri. 2013. It really does matter how you normalize the branch distance in search-based software testing. Softw. Test., Verif. Reliab.23, 2 (2013), 119--147. https://doi.org/10.1002/stvr.457

    2013 doi

  9. [16]

    Andrea Arcuri. 2017. RESTful API Automated Test Case Generation. In IEEE International Conference on Software Quality, Reliability and Security (QRS). IEEE, 9--20

    2017

  10. [17]

    Andrea Arcuri. 2018. EvoMaster: Evolutionary Multi-context Automated System Test Generation. In IEEE International Conference on Software Testing, Verification and Validation (ICST). IEEE

    2018

  11. [18]

    Andrea Arcuri. 2018. Test suite generation with the Many Independent Objective (MIO) algorithm. Information and Software Technology104 (2018), 195--206

    2018

  12. [19]

    Andrea Arcuri. 2019. RESTful API Automated Test Case Generation with EvoMaster. ACM Transactions on Software Engineering and Methodology (TOSEM)28, 1 (2019), 3

    2019

  13. [20]

    Andrea Arcuri. 2020. Automated Black-and White-Box Testing of RESTful APIs With EvoMaster. IEEE Software 38, 3 (2020), 72--78

    2020

  14. [21]

    Arcuri and L

    A. Arcuri and L. Briand. 2014. A Hitchhiker’s Guide to Statistical Tests for Assessing Randomized Algorithms in Software Engineering. Software Testing, Verification and Reliability (STVR)24, 3 (2014), 219--250

    2014

  15. [23]

    Andrea Arcuri and Juan P Galeotti. 2021. Enhancing Search-based Testing with Testability Transformations for Existing APIs. ACM Transactions on Software Engineering and Methodology (TOSEM)31, 1 (2021), 1--34

    2021

  16. [24]

    Andrea Arcuri, Juan Pablo Galeotti, Bogdan Marculescu, and Man Zhang. 2021. EvoMaster: A Search-Based System Test Generation Tool. Journal of Open Source Software6, 57 (2021), 2153

    2021

  17. [25]

    Andrea Arcuri, Man Zhang, and Juan Pablo Galeotti. 2024. Advanced White-Box Heuristics for Search- Based Fuzzing of REST APIs. ACM Transactions on Software Engineering and Methodology (TOSEM) (2024). https://doi.org/10.1145/3652157

    2024 doi

  18. [26]

    Andrea Arcuri, Man Zhang, Amid Golmohammadi, Asma Belhadi, Juan P Galeotti, Bogdan Marculescu, and Susruthan Seran. 2023. EMB: A curated corpus of web/enterprise applications and library support for software testing research. In 2023 IEEE Conference on Software Testing, Verifi...

    2023

  19. [27]

    Andrea Arcuri, Man Zhang, Susruthan Seran, Asma Belhadi, Juan Pablo Galeotti, Bogdan, Amid Gol- mohammadi, Onur Duman, Agustina Aldasoro, Philip, Alberto Mart ´ ın L´ opez, Hernan Ghianni,¨Om¨ ur S ¸ahin, Annibale Panichella, Kyle Niemeyer, and Marcello Maugeri. 2025. WebFuzzi...

    2025 doi

  20. [28]

    Vaggelis Atlidakis, Patrice Godefroid, and Marina Polishchuk. 2019. RESTler: Stateful REST API Fuzzing. In ACM/IEEE International Conference on Software Engineering (ICSE). 748–758

    2019

  21. [29]

    Baresel and H

    A. Baresel and H. Sthamer. 2003. Evolutionary testing of flag conditions. In Genetic and Evolutionary Computation Conference (GECCO). 2442--2454

    2003

  22. [30]

    George Candea and Patrice Godefroid. 2019. Automated Software Test Generation: Some Challenges, Solutions, and Recent Advances. Springer International Publishing, Cham, 505--531. https://doi.org/ 10.1007/978-3-319-91908-9_24

    2019 doi

  23. [31]

    Davide Corradini, Amedeo Zampieri, Michele Pasqua, and Mariano Ceccato. 2021. Restats: A test coverage tool for RESTful APIs. In 2021 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 594--598

    2021

  24. [32]

    Roy Thomas Fielding. 2000. Architectural styles and the design of network-based software architectures. Ph.D. Dissertation. University of California, Irvine

    2000

  25. [33]

    Gordon Fraser and Andrea Arcuri. 2011. EvoSuite: Automatic Test Suite Generation for Object-Oriented Software. In Proceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on Foundations of software engineering. 416--419

    2011

  26. [34]

    Gordon Fraser and Andrea Arcuri. 2013. Whole Test Suite Generation. IEEE Transactions on Software Engineering 39, 2 (2013), 276--291

    2013

  27. [35]

    Felix Gessert, Wolfram Wingerath, Steffen Friedrich, and Norbert Ritter. 2017. NoSQL database systems: a survey and decision guidance. Comput. Sci. Res. Dev.32, 3-4 (2017), 353--365. https://doi.org/10. 1007/S00450-016-0334-3

    2017

  28. [36]

    Amid Golmohammadi, Man Zhang, and Andrea Arcuri. 2022. Testing RESTful APIs: A Survey. ACM Transactions on Software Engineering and Methodology(2022)

    2022

  29. [37]

    Amid Golmohammadi, Man Zhang, and Andrea Arcuri. 2023. Testing RESTful APIs: A Survey. ACM Transactions on Software Engineering and Methodology(aug 2023). https://doi.org/10.1145/3617175

    2023 doi

  30. [38]

    Harman, L

    M. Harman, L. Hu, R. Hierons, A. Baresel, and H. Sthamer. 2002. Improving evolutionary testing by flag removal. In Genetic and Evolutionary Computation Conference (GECCO). 1351--1358

    2002

  31. [39]

    Mark Harman, S Afshin Mansouri, and Yuanyuan Zhang. 2012. Search-based software engineering: Trends, techniques and applications. ACM Computing Surveys (CSUR)45, 1 (2012), 11

    2012

  32. [40]

    Zac Hatfield-Dodds and Dmitry Dygalo. 2022. Deriving Semantics-Aware Fuzzers from Web API Schemas. In 2022 IEEE/ACM 44th International Conference on Software Engineering: Companion Proceedings (ICSE-Companion). IEEE, 345--346

    2022

  33. [41]

    Pooyan Jamshidi, Claus Pahl, Nabor C Mendon¸ ca, James Lewis, and Stefan Tilkov. 2018. Microservices: The journey so far and challenges ahead. IEEE Software 35, 3 (2018), 24--35

    2018

  34. [42]

    Myeongsoo Kim, Saurabh Sinha, and Alessandro Orso. 2023. Adaptive rest api testing with reinforcement learning. In 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 446--458

    2023

  35. [43]

    Myeongsoo Kim, Qi Xin, Saurabh Sinha, and Alessandro Orso. 2022. Automated Test Generation for REST APIs: No Time to Rest Yet. In Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2022). Association for Computing Machinery, New...

    2022

  36. [44]

    Bogdan Korel. 1990. Automated software test data generation. IEEE Transactions on software engineering 16, 8 (1990), 870--879

    1990

  37. [45]

    Nuno Laranjeiro, Jo˜ ao Agnelo, and Jorge Bernardino. 2021. A black box tool for robustness testing of REST services. IEEE Access 9 (2021), 24738--24754. 16

    2021

  38. [46]

    Yi Liu, Yuekang Li, Gelei Deng, Yang Liu, Ruiyuan Wan, Runchao Wu, Dandan Ji, Shiheng Xu, and Minli Bao. 2022. Morest: Model-based RESTful API Testing with Execution Feedback. In ACM/IEEE International Conference on Software Engineering (ICSE)

    2022

  39. [47]

    Ke Mao, Mark Harman, and Yue Jia. 2016. Sapienz: Multi-objective automated testing for android applications. In ACM Int. Symposium on Software Testing and Analysis (ISSTA). ACM, 94--105

    2016

  40. [48]

    Bogdan Marculescu, Man Zhang, and Andrea Arcuri. 2022. On the Faults Found in REST APIs by Automated Test Generation. ACM Transactions on Software Engineering and Methodology (TOSEM)31, 3 (2022), 1--43

    2022

  41. [49]

    Alberto Martin-Lopez, Sergio Segura, and Antonio Ruiz-Cort´ es. 2019. Test coverage criteria for RESTful web APIs. In Proceedings of the 10th ACM SIGSOFT International Workshop on Automating TEST Case Design, Selection, and Evaluation. 15--21

    2019

  42. [50]

    Alberto Martin-Lopez, Sergio Segura, and Antonio Ruiz-Cort´ es. 2021. RESTest: Automated Black-Box Testing of RESTful Web APIs. In ACM Int. Symposium on Software Testing and Analysis (ISSTA). ACM, 682--685

    2021

  43. [51]

    Sam Newman. 2021. Building microservices. ” O’Reilly Media, Inc.”

    2021

  44. [52]

    Alessandro Orso and Gregg Rothermel. 2014. Software testing: a research travelogue (2000-2014). In Proceedings of the on Future of Software Engineering, FOSE 2014, Hyderabad, India, May 31 - June 7, 2014, James D. Herbsleb and Matthew B. Dwyer (Eds.). ACM, 117--132. https://do...

    2014

  45. [53]

    Annibale Panichella, Fitsum Kifetew, and Paolo Tonella. 2018. Automated Test Case Generation as a Many-Objective Optimisation Problem with Dynamic Selection of the Targets. IEEE Transactions on Software Engineering (TSE)44, 2 (2018), 122--158

    2018

  46. [54]

    Jos´ e Miguel Rojas, Jos´ e Campos, Mattia Vivanti, Gordon Fraser, and Andrea Arcuri. 2015. Combining multiple coverage criteria in search-based unit test generation. In International Symposium on Search Based Software Engineering. Springer, 93--108

    2015

  47. [55]

    Jos´ e Miguel Rojas, Gordon Fraser, and Andrea Arcuri. 2016. Seeding strategies in search-based unit test generation. Software Testing, Verification and Reliability26, 5 (2016), 366--401

    2016

  48. [56]

    Emanuele Viglianisi, Michael Dallago, and Mariano Ceccato. 2020. RESTTESTGEN: Automated Black- Box Testing of RESTful APIs. In IEEE International Conference on Software Testing, Verification and Validation (ICST). IEEE

    2020

  49. [57]

    Huayao Wu, Lixin Xu, Xintao Niu, and Changhai Nie. 2022. Combinatorial Testing of RESTful APIs. In ACM/IEEE International Conference on Software Engineering (ICSE)

    2022

  50. [58]

    Man Zhang and Andrea Arcuri. 2023. Open Problems in Fuzzing RESTful APIs: A Comparison of Tools. ACM Transactions on Software Engineering and Methodology (TOSEM)(may 2023). https: //doi.org/10.1145/3597205 17

    2023 doi

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.