Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

RT4CHART verifies RAG answers claim by claim, against local then full context, and outperforms existing detectors on strict faithfulness benchmarks.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 17:09 UTC pith:ADKSW3PG

load-bearing objection A solid, practical RAG hallucination detector with a genuinely useful hierarchical verification design, held back by a self-constructed benchmark whose key subset is not independently validated. the 4 major comments →

arxiv 2603.27752 v2 pith:ADKSW3PG submitted 2026-03-29 cs.CL cs.SE

Retromorphic Testing with Hierarchical Verification for Hallucination Detection in RAG

Boxi Yu , Yuzhong Zhang , Liting Lin , Lionel Briand , Emir Mu\~noz This is my paper
classification cs.CL cs.SE
keywords hallucination detectionretrieval-augmented generationcontext faithfulnessclaim decompositionhierarchical verificationretromorphic testingbenchmark re-annotationspan localization
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that detecting hallucinations in retrieval-augmented generation should be treated as a traceability problem: every claim in an answer must map back to evidence in the retrieved context. It proposes RT4CHART, which decomposes answers into claims, checks each claim against overlapping context chunks, then re-checks against the full context, labeling claims entailed, contradicted, or baseless and returning the exact context sentences used. On two improved benchmarks, this hierarchical verification achieves the best answer-level F1 among compared detectors, with most gains coming from the local claim-level step and global verification adding selective benefit. The paper also re-annotates a widely used benchmark and reports 1.68x more hallucination cases than the original labels, arguing that current benchmarks understate hallucination prevalence.

Core claim

RT4CHART's central claim is that a strict context-faithfulness verdict for a RAG answer is best produced by a backward verification program that decomposes the answer into atomic claims and verifies them hierarchically. Local verification scans each claim against overlapping context windows and merges chunk judgments with an OR-join; global verification then re-reads the full context, using the local label only as a hint, and assigns a final label. The answer is flagged hallucinated if any final claim is contradicted or baseless. On the 408-sample re-annotated subset the method reaches 0.776 answer-level F1, an 83% relative improvement over the strongest baseline, and on the broader 2,675-sa

What carries the argument

The load-bearing object is the verification trace T = {(Clm_i, y*_i, E*_i)} produced by the backward program: each answer claim gets a final label (entailed, contradicted, baseless) and a set of context-side evidence spans anchored to source sentences. The hierarchy couples a local verifier over overlapping chunks with an OR-join, and a global verifier over the full context with adaptive prompts conditioned on the local label; an AND-join then flags the whole answer on one unfaithful claim. This lets the system catch both chunk-boundary distributed evidence and locally plausible but globally contradicted claims, while returning auditable evidence rather than a scalar.

Load-bearing premise

The central fragile premise is that the re-annotated ground truth is accurate: the authors left 1,129 matched samples unchanged after a 70-sample sanity check and used an LLM auditor to select the 1,546 cases for two-author review; if that selection or the revised labels systematically over-mark plausible but unsupported statements, the F1 numbers, the prevalence claim, and the span-level scores all shift.

What would settle it

Take a random subset of the original benchmark, have three independent expert annotators label hallucination spans with no LLM prefilter, and compare (a) whether the detected hallucination prevalence stays near 1.68x the original labels and (b) whether RT4CHART's answer-level F1 and span F1 hold against this independent gold standard; a large drop would indicate the reported gains depend on the re-annotation protocol.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Answer-level holistic scorers miss partially hallucinated answers; claim-level hierarchical verification catches them, as shown by recall gains over the strongest baseline.
  • Auditors get not just a verdict but the exact answer spans and supporting or contradicting context sentences, making RAG outputs auditable end-to-end.
  • The 1.68x increase in hallucination cases found during re-annotation implies published benchmark numbers may understate real hallucination rates.
  • Local verification is the primary driver of performance; global verification adds value mainly when evidence is spread across multiple context chunks.
  • At roughly a cent per sample with a low-cost judge model, routine post-update faithfulness audits become practical.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the hierarchical verification result generalizes, any generation task with a fixed authoritative input—summarization, structured data-to-text, policy-grounded assistants—could inherit the same local-then-global evidence audit.
  • The re-annotation protocol implies that older detector comparisons on the original benchmark may need recalibration, since false-negative labels suppress measured recall for all detectors.
  • A testable extension is to apply RT4CHART to non-English or specialized-domain contexts; the paper's evaluation is limited to English QA, summarization, and data-to-text, so transfer remains open.
  • The coarse sentence-level span localization suggests a next step of finer-grained evidence alignment to raise span precision without losing recall.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper presents RT4CHART, a context-faithfulness hallucination detector for RAG. The method decomposes a generated answer into atomic claims, verifies each claim over overlapping context chunks (local verification) and then against the full retrieved context (global verification), assigns each claim one of Entailed/Contradicted/Baseless labels, maps claim labels back to answer spans, and returns context-side evidence. An AND-join produces an answer-level hallucination verdict. The authors evaluate on the external RAGTruth++ benchmark (408 samples) and on a newly constructed RAGTruth-Enhance benchmark (2,675 samples) created by re-annotating the RAGTruth evaluation split. They report answer-level F1 of 0.776 on RAGTruth++ (83% relative improvement over the strongest baseline) and 0.845 on RAGTruth-Enhance, span-level F1 of 47.5% on RAGTruth-Enhance, and a 1.68x increase in detected hallucination cases relative to the original RAGTruth labels. Ablations attribute most of the gain to local claim-level verification, with global verification selectively helpful.

Significance. If the empirical results are reliable, the paper makes a useful contribution: it moves beyond answer-level scoring toward evidence-grounded, span-level hallucination diagnosis, and the hierarchical local-to-global design is a plausible mechanism for handling long retrieved contexts. Strengths include the evaluation on the externally constructed RAGTruth++ benchmark, the consistent answer-level ordering across two datasets, the ablation evidence for local verification, and the practical cost/reproducibility analysis with released code and prompts. The central weakness is that the larger benchmark, RAGTruth-Enhance, is self-constructed with a protocol that has not been validated against an independent human gold standard; the answer-level, span-level, evidence-grounding, and prevalence claims all depend on that benchmark. RAGTruth++ provides only partial mitigation because it is much smaller and does not cover the span/evidence diagnostics or the prevalence claim.

major comments (4)
  1. [§4.2 (RAGTruth-Enhance construction)] The reliability of RAGTruth-Enhance is the load-bearing assumption for Table 3 (F1=0.845), Table 4 (span F1=47.5%), and the 1.68x prevalence claim. The protocol uses an LLM auditor to select 1,546 flagged cases for two-author review and leaves 1,129 matched cases unchanged after a 70-sample sanity check. That sanity check measures agreement with the original labels, which the paper itself shows are substantially under-annotated (86→865 spans on the RAGTruth++ subset). It therefore cannot establish the correctness of the matched subset. In addition, the flagged subset is selected by an LLM judge; if that judge shares the judgment tendencies of RT4CHART's LLM verifier, the benchmark is enriched for exactly the failure modes the method detects. The paper needs an independent human-validated sample of both matched and flagged cases, with inter-annotator agreement reported, before these numbe
  2. [§4.2 and Table 5] RAGTruth-Enhance is said to include 'refuting context evidence for contradiction labels,' but the construction of this evidence is not described and no validation is provided. The evidence-grounding diagnostic in Table 5 (F1=54.8%) is used to support the contribution of context-side evidence, yet if the gold refuting spans were produced by the same two-author review without an explicit annotation protocol, independence and reliability are unknown. At minimum, the paper should report the annotation instructions, the number of examples with mappable gold evidence, and agreement statistics for the evidence annotations.
  3. [§4.4 / Table 6 and abstract] The abstract states that 'the hierarchical verification design is the main driver of performance,' but Table 6 shows that on RAGTruth-Enhance global verification does not change answer-level F1 at all (0.845 with vs. without global verification). The ablation supports the more precise claim that local claim-level verification drives the gains, while global verification is complementary but selectively beneficial. The contribution framing should be adjusted to match the evidence; otherwise the central design claim is overstated.
  4. [§5.2 (Threats to validity)] The Threats to Validity section lists model dependence, implementation dependence, and bounded evaluation setting, but it does not list the most immediate threat: the reliability of the self-constructed RAGTruth-Enhance labels. Given that the paper's headline prevalence and span-level claims rest on this benchmark, the omission is significant. The paper should either add this threat explicitly or provide the missing independent validation.
minor comments (6)
  1. [§4.2] The LLM auditor that flags cases is not specified (model, prompt, threshold). Since the auditor's output drives the entire re-annotation protocol, these details are needed for reproducibility and for assessing selection bias.
  2. [§4.2] The 70-sample sanity check is very small. With 70 observations, an observed accuracy of 98% has a wide confidence interval; the paper should report the exact number of disagreements and, preferably, a confidence interval.
  3. [§4.2] Inter-annotator agreement for the two-author review is not reported. 'Residual disputes are rare (<0.5%)' after discussion is not a substitute for a kappa or agreement statistic computed before adjudication.
  4. [§4.3 and Table 4] The span-level comparison with Lettuce is informative, but the paper should clarify whether Lettuce was trained on the original RAGTruth labels; if so, a re-annotated gold standard can disadvantage it in ways unrelated to detection ability.
  5. [§4.6] The cost estimate of $0.0104 per sample assumes a particular GPT-4o mini pricing and does not report the number of LLM calls per sample (decomposition, each local chunk, global verification). Reporting call counts would make the cost analysis reproducible.
  6. [§6] MiniCheck and AlignScore are cited in related work but not included in the experimental comparison. Adding at least one modern NLI-based detector would strengthen the baseline set if feasible.

Circularity Check

0 steps flagged

No significant circularity: the detection pipeline is not fitted to its evaluation labels and the headline result is anchored by an external benchmark; the main caveat is benchmark-validity, not circular derivation.

full rationale

The paper's derivation chain is self-contained. RT4CHART is an LLM-based pipeline (claim decomposition, chunked local verification, global verification, AND-join aggregation), and its outputs are compared against ground-truth labels that are either external (RAGTruth++) or human-reviewed (RAGTruth-Enhance). No parameter of the detector is fitted to the evaluation labels and then reported as a prediction; the answer-level and span-level metrics are not forced by the method's definitions. The 'retromorphic testing' framing comes from a prior paper by the first author (ref. [24]), but it is used as a conceptual wrapper, not as a theorem from which the empirical results are derived; no load-bearing argument reduces to that citation. RAGTruth-Enhance is author-constructed, and its validation protocol—especially the LLM-auditor selection and the 70-case sanity check for matched samples—raises legitimate data-reliability concerns, but this is a benchmark-validity threat rather than a circular reduction of the method's predictions to its inputs. The paper also reports external RAGTruth++ results and acknowledges residual performance gaps and limitations, further indicating that no result is obtained by construction. Therefore no circular step is established.

Axiom & Free-Parameter Ledger

2 free parameters · 6 axioms · 0 invented entities

The ledger shows no fitted numeric model weights, but several design choices and data-generation assumptions carry the result: chunking hyperparameters are selected on the eval set, and the core evidence on the larger benchmark derives from the authors' own re-annotation. These are assumptions, not independent external facts.

free parameters (2)
  • Context chunk window W and overlap O = W=25, O=10
    Default hyperparameters; Table 9 shows (25,10) yields best answer-level F1 on RAGTruth-Enhance, indicating selection against the evaluation set. Sensitivity is low across nearby settings.
  • SelfCheckGPT decision threshold = 0.5
    Chosen by threshold sweep on validation runs (Section 4.2 footnote), affecting the baseline comparison. This is a baseline parameter, not RT4CHART's, but it shapes the reported margins.
axioms (6)
  • domain assumption Retrieved context C is the sole authoritative evidence source for faithfulness
    Adopted in Section 2; if C is incomplete or wrong, entailed/contradicted/baseless labels can be misleading for real-world truth.
  • domain assumption The LLM judge's per-chunk and full-context labels are reliable proxies for entailment/contradiction/support
    The entire pipeline is LLM-judged with temperature 0; no calibration or human audit of judge outputs is reported beyond aggregated metrics.
  • domain assumption RAGTruth-Enhance re-annotation labels are correct
    Section 4.2: two-author review of 1,546 flagged samples and 70-sample sanity check for matched cases; no inter-annotator agreement reported for the main set.
  • domain assumption Sliding-window chunking with W and O preserves all locally relevant evidence
    Section 2/3.4; if a claim's evidence is fragmented across chunks or boundary overlap is insufficient, local labels mislead.
  • domain assumption LLM claim decomposition preserves the answer's propositional content and qualifiers
    Section 3.3; dropped negation/quantifiers/modality would change claim semantics and downstream labels.
  • domain assumption Retromorphic testing provides a valid oracle framing for context faithfulness
    Section 3.1; the relation RR is defined to hold by construction, but its correctness as an oracle is assumed, not proven.

pith-pipeline@v1.3.0-alltime-deepseek · 15813 in / 14440 out tokens · 123978 ms · 2026-08-02T17:09:04.400807+00:00 · methodology

0 comments
read the original abstract

Large language models can still hallucinate in retrieval-augmented generation (RAG), producing claims that are unsupported by or conflict with the retrieved context. Detecting such errors remains challenging when faithfulness is judged solely against the retrieved context: many existing detectors return holistic answer-level scores, while others target open-domain factuality or fail to provide evidence-grounded diagnostics. We present RT4CHART, a retromorphic testing framework for context-faithfulness assessment. RT4CHART decomposes an answer into independently verifiable claims, performs hierarchical local-to-global verification against the retrieved context, and assigns each claim one of three labels: entailed, contradicted, or baseless. It further maps these claim-level decisions back to specific answer spans and returns explicit context-side evidence, enabling fine-grained auditing rather than opaque scoring. We evaluate RT4CHART on RAGTruth++ (408 samples) and our re-annotated RAGTruth-Enhance (2,675 samples). RT4CHART achieves the best answer-level hallucination-detection F1 score among the evaluated baselines. On RAGTruth++, it attains a precision of 0.845, a recall of 0.718, and an F1 score of 0.776, representing an 83% relative improvement over the strongest baseline. It also achieves a span-level F1 score of 47.5% on RAGTruth-Enhance. Ablation studies show that claim-based local processing drives most of the observed improvement, while global verification provides selective benefits across datasets. Finally, our re-annotation identifies 1.68X more hallucination cases than the original labels, suggesting that commonly used benchmarks substantially underestimate the prevalence of hallucination.

Figures

Figures reproduced from arXiv: 2603.27752 by Boxi Yu, Emir Mu\~noz, Lionel Briand, Liting Lin, Yuzhong Zhang.

Figure 1
Figure 1. Figure 1: Detection granularity on a motivating example. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of RT4CHART. The system decomposes [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. RAG-TESTER: Automated End-to-End Testing of Retrieval-Augmented Large Language Models

    cs.AI 2026-07 conditional novelty 4.0

    A coverage-oriented test generator that adds negative-rejection and complex-passage queries to RAG evaluation, reported to detect 6.6% more failures than a strategy-free baseline in 24 model configurations.

Reference graph

Works this paper leans on

29 extracted references · 4 linked inside Pith · cited by 1 Pith paper

  1. [1]

    2026.A toolkit for RT4CHART

    Anonymous. 2026.A toolkit for RT4CHART. https://anonymous.4open.science/ r/rt4hallucination-374D

    2026

  2. [2]

    Barr, Mark Harman, Phil McMinn, Muzammil Shahbaz, and Shin Yoo

    Earl T. Barr, Mark Harman, Phil McMinn, Muzammil Shahbaz, and Shin Yoo

  3. [3]

    Blue Guardrails. 2025. ragtruth-plus-plus. https://huggingface.co/datasets/blue- guardrails/ragtruth-plus-plus Dataset on the Hugging Face Hub

    2025

  4. [4]

    Bowman, Gabor Angeli, Christopher Potts, and Christopher D

    Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Man- ning. 2015. A large annotated corpus for learning natural language inference. InProceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP). 632–642. doi:10.18653/v1/D15-1075

    doi:10.18653/v1/d15-1075 2015

  5. [5]

    T. Y. Chen, F.-C. Kuo, H. Liu, P.-L. Poon, D. Towey, T. H. Tse, and Z. Q. Zhou

  6. [6]

    Tianyu Gao, Howard Yen, Jiatong Yu, and Danqi Chen. 2023. Enabling Large Language Models to Generate Text with Citations. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP). 6465–

    2023

  7. [7]

    Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of Hallucination in Natural Language Generation.Comput. Surveys55, 12 (2023), 248:1–248:38. doi:10.1145/3571730

    doi:10.1145/3571730 2023

  8. [8]

    Miriam Kümmel. 2025. RAGTruth++: Enhanced Hallucination Detection Bench- mark. Blue Guardrails blog. https://www.blueguardrails.com/en/blog/ragtruth- plus-plus-enhanced-hallucination-detection-benchmark Accessed: 2026-02-13

    2025

  9. [9]

    Bennett, and Marti A

    Philippe Laban, Tobias Schnabel, Paul N. Bennett, and Marti A. Hearst. 2022. SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summa- rization.Transactions of the Association for Computational Linguistics10 (2022), 163–177. doi:10.1162/tacl_a_00453

    doi:10.1162/tacl_a_00453 2022

  10. [10]

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. arXiv:2005.11401 [cs.CL] https://arxiv.org/abs/2005.11401

    Pith/arXiv arXiv 2020

  11. [11]

    Miaoran Li, Rogger Luo, and Ofer Mendelevitch. 2024. HHEM-2.1-Open. doi:10. 57967/hf/3240

    2024

  12. [12]

    Potsawee Manakul, Adian Liusie, and Mark JF Gales. 2023. SelfCheckGPT: Zero- Resource Black-Box Hallucination Detection for Generative Large Language Models. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP)

    2023

  13. [13]

    McKeeman

    William M. McKeeman. 1998. Differential Testing for Software.Digital Technical Journal10, 1 (1998), 100–107

    1998

  14. [14]

    Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Wei Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2023. FActScore: Fine- grained Atomic Evaluation of Factual Precision in Long Form Text Generation. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP). 12076–12100

    2023

  15. [15]

    Cheng Niu, Yuanhao Wu, Juno Zhu, Siliang Xu, KaShun Shum, Randy Zhong, Juntong Song, and Tong Zhang. 2024. RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 10862–10878. doi:10.18653/v1/2024.acl-long.585

    doi:10.18653/v1/2024.acl-long.585 2024

  16. [16]

    Selvan Sunitha Ravi, Bartosz Mielczarek, Anand Kannappan, Douwe Kiela, and Rebecca Qian. 2024. Lynx: An Open Source Hallucination Evaluation Model. arXiv:2407.08488 [cs.CL]

    Pith/arXiv arXiv 2024

  17. [17]

    Manuel Rigger and Zhendong Su. 2022. Intramorphic Testing: A New Approach to the Test Oracle Problem. InProceedings of the 2022 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software (Onward!). doi:10.1145/3563835.3567662

    arXiv 2022

  18. [18]

    Sánchez, and Antonio Ruiz-Cortés

    Sergio Segura, Gordon Fraser, Ana B. Sánchez, and Antonio Ruiz-Cortés. 2016. A Survey on Metamorphic Testing.IEEE Transactions on Software Engineering 42, 9 (2016), 805–824. doi:10.1109/TSE.2016.2532875

    arXiv 2016

  19. [19]

    Yixiao Song, Yekyung Kim, and Mohit Iyyer. 2024. VeriScore: Evaluating the Factuality of Verifiable Claims in Long-Form Text Generation. InFindings of the Association for Computational Linguistics: EMNLP 2024. 9447–9474. doi:10.18653/ v1/2024.findings-emnlp.552

    2024

  20. [20]

    Liyan Tang, Philippe Laban, and Greg Durrett. 2024. MiniCheck: Efficient Fact- Checking of LLMs on Grounding Documents. InProceedings of the 2024 Confer- ence on Empirical Methods in Natural Language Processing (EMNLP). 8818–8847. doi:10.18653/v1/2024.emnlp-main.499

    doi:10.18653/v1/2024.emnlp-main.499 2024

  21. [21]

    Vectara. 2026. Hallucination evaluation. Vectara Docs. https://docs.vectara.com/ docs/hallucination-and-evaluation/hallucination-evaluation Accessed: 2026-02- 13

    2026

  22. [22]

    Jerry Wei, Chengrun Yang, Xinying Song, Yifeng Lu, Nathan Hu, Dustin Tran, Daiyi Peng, Ruibo Liu, Da Huang, Cosmo Du, and Quoc V. Le. 2024. Long- form Factuality in Large Language Models. InAdvances in Neural Information Processing Systems (NeurIPS)

    2024

  23. [23]

    Zhang, and Gias Uddin

    Borui Yang, Md Afif Al Mamun, Jie M. Zhang, and Gias Uddin. 2025. Hallucination Detection in Large Language Models with Metamorphic Relations.Proceedings of the ACM on Software Engineering2, FSE (July 2025). doi:10.1145/3715735

    doi:10.1145/3715735 2025

  24. [24]

    Boxi Yu, Qiuyang Mang, Qingshuo Guo, and Pinjia He. 2023. Retromorphic Testing: A New Approach to the Test Oracle Problem. arXiv:2310.06433 [cs.SE]

    Pith/arXiv arXiv 2023

  25. [25]

    Yuheng Zha, Yichi Yang, Ruichen Li, and Zhiting Hu. 2023. AlignScore: Evaluating Factual Consistency with A Unified Alignment Function. InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 11328–11348. doi:10.18653/v1/2023.acl-long.634

    doi:10.18653/v1/2023.acl-long.634 2023

  26. [26]

    Ádám Kovács and Gábor Recski. 2025. LettuceDetect: A Hallucination Detection Framework for RAG Applications. arXiv:2502.17125 [cs.CL] https://arxiv.org/ abs/2502.17125

    Pith/arXiv arXiv 2025

  27. [2015]

    doi:10.1109/TSE.2014.2372785

    The Oracle Problem in Software Testing: A Survey.IEEE Transactions on Software Engineering41, 5 (2015), 507–525. doi:10.1109/TSE.2014.2372785

    arXiv 2015

  28. [2018]

    Surveys51, 1 (2018), 4:1–4:27

    Metamorphic Testing: A Review of Challenges and Opportunities.Comput. Surveys51, 1 (2018), 4:1–4:27. doi:10.1145/3143561

    doi:10.1145/3143561 2018

  29. [6488]

    doi:10.18653/v1/2023.emnlp-main.398

    doi:10.18653/v1/2023.emnlp-main.398 2023