Test It Before You Trust It: Applying Software Testing for Trustworthy In-context Learning

Chaiyong Ragkhitwetsagul; Chommakorn Sontesadisai; Teeradaj Racharak; Thanwadee Sunetnanta

REVIEW 4 major objections 6 minor 17 references

Test It Before You Trust It: Applying Software Testing for Trustworthy In-context Learning

T0 review · 4 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Software-testing-style metamorphic tests expose linguistic bugs in large language models that accuracy scores miss.

desk verdict A credible metamorphic-testing pass-rate study for LLMs in ICL, but several metamorphic relations are over-strict and the 'bug' claims run ahead of the data. read the letter →

arxiv 2504.18827 v3 pith:MQPDIBZ2 submitted 2025-04-26 cs.SE cs.AI

Teeradaj Racharak , Chaiyong Ragkhitwetsagul , Chommakorn Sontesadisai , Thanwadee Sunetnanta This is my paper · ORCID

classification cs.SEcs.AI

keywords in-contextlearningmetamorphictestingLLMevaluationadversarialperturbationstrustworthyAIsoftwarerobustnesspromptengineering

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that the trustworthiness of in-context learning in large language models should be tested the way software is tested: by checking whether model outputs remain consistent under controlled input perturbations, not merely whether they match gold labels. It introduces MMT4NL, a metamorphic-testing framework that builds nine perturbation operators—synonym substitution, named-entity replacement, negation, vocabulary, fairness, robustness, temporal, semantic-role, and coreference—and measures a pass rate, the fraction of perturbed inputs for which the expected output relation holds. On 50 sentiment-analysis and 50 question-answering items, GPT-4o and Gemini-2.0-Flash achieve high accuracy on original inputs yet show substantially lower pass rates under many perturbations, such as Gemini-2.0-Flash's low temporal pass rate and negation pass rates as low as 60% in QA. The paper interprets these gaps as linguistic bugs in prompt interpretation that standard accuracy evaluation misses.

What carries the argument

The central mechanism is the metamorphic relation, formally written as $MR(x, f(x), P(x), P(f(x))) \Rightarrow g(P(x)) = P(f(x))$, where $P$ is the LLM under test, $f$ is an input transformation, and $g$ is the expected output transformation, usually the identity. It converts the test-oracle problem—no known correct output for a novel input—into a label-free consistency check between original and perturbed outputs. MMT4NL pairs this relation with nine concrete perturbation operators and a pass-rate metric $N_{\text{pass}} / N_{\text{test}}$, which lets any LLM API be tested without internal access or human annotation for each perturbed example.

What would settle it

Human annotators rate whether each perturbed item truly preserves the original meaning and sentiment on the paper's 50 sentiment items; if a large fraction is judged not meaning-preserving, or if a model can achieve near-100% pass rate on all nine relations while producing clearly wrong answers on the original inputs, then the pass-rate metric is measuring the relations themselves rather than trustworthiness.

Watch

Extended reading notes

Core claim

On the paper's own terms, the core discovery is that treating an LLM as a black-box program and applying metamorphic testing reveals systematic interpretation flaws that accuracy on original inputs conceals. The framework defines metamorphic relations as implications $P(x) = P(f(x))$ (or an analogous output relation) for meaning-preserving transformations $f$; when the equality fails, the model has failed a trustworthiness check. Across the two models and two tasks, the authors observe concrete failures: gender-modified sentences shift sentiment, synonym substitutions like 'complicated' to 'complex' flip negative sentiment to Neutral, a single transposed character in a medical term changes a QA answer, and coreference restructuring flips answers about census questions. Because accuracy remains high on the same items, the paper concludes that pass rate under metamorphic relations is a complementary, achievable trustworthiness metric for in-context learning.

Load-bearing premise

The load-bearing premise is that a trustworthy LLM must give invariant or predictably related outputs under meaning-preserving perturbations; if synonym and pronoun substitutions are not a valid standard of trustworthiness, then low pass rates do not prove the model is buggy.

Editorial extensions

If this is right

Accuracy on original inputs should no longer be taken as sufficient evidence of ICL trustworthiness; pass rate under metamorphic relations should be reported alongside accuracy.
The framework supplies a reusable test template for any prompt construction: run the nine perturbation types, compute pass rates, and identify the specific linguistic capability that fails.
Few-shot prompting does not uniformly repair robustness: some categories, such as temporal consistency for Gemini-2.0-Flash and negation handling in QA for both models, remain weak or worsen.
Supplying context in question answering substantially raises both accuracy and pass rate, indicating that grounding can mitigate but not eliminate perturbation sensitivity.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The invariance standard is an editorial choice of the paper, not an external truth; a natural extension would be human validation of 'meaning preserved' for each perturbation to separate genuine semantic flips from acceptable re-readings.
The same metamorphic template could extend beyond classification to generation tasks by replacing equality with entailment or semantic-similarity output relations.
Pass-rate patterns across categories could be used as a diagnostic fingerprint—e.g., consistent Taxonomy failures with Fairness passes might point to embedding-space synonym drift rather than demographic bias.
Automating perturbation generation with controlled edit distance would scale the framework to large benchmarks and test whether pass rates degrade smoothly with perturbation severity.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Christian, quick take on arXiv:2504.18827. This is a methodology paper that recombines CheckList's metamorphic-testing vocabulary with PromptBench's adversarial perturbation ideas and applies the result to in-context learning. The empirical core is credible: GPT-4o and Gemini-2.0-Flash do flip answers on small perturbations, and the paper's tables give concrete pass rates across sentiment, QA with/without context, and zero/one/few-shot. That makes it a useful data point for anyone building LLM evaluations.

What's genuinely new: a named framework (MMT4NL) with formal metamorphic relations, a pass-rate metric for ICL, and a systematic sweep across prompt regimes. The observation that few-shot prompting does not reliably improve robustness is worth recording. The writing is clear, and the artifact link is a plus. Also, the self-citations are to the authors' own prior NLP work; nothing circular there.

Where it gets soft: several metamorphic relations are not actually meaning-preserving, so the pass rates overstate the model's instability. Taxonomy requires P(x)=P(f(x)) for synonyms, but 'hungry' vs. 'starving' and 'complicated' vs. 'complex' are not interchangeable in sentiment. Robustness uses 'tired' → 'tried' as if it were a typo, but that is a different word with a different meaning. Negation's formal MR ¬P(x)≈P(f(x)) is mis-specified for three-class sentiment, and the QA example 'not uncommon' does not truly invert the question. Fairness's 'female Chief Justice' changes the referent, so a changed answer may be correct. These are not minor nits; they undermine the quantitative claim that MMT4NL 'effectively revealed' bugs. The paper itself even shows failure examples that are reasonable model behavior, which complicates the interpretation.

Other concerns: 50 records per task is small, there are no error bars or repeated runs, and the perturbations were manually constructed except for typos. The selective display of 'surprising' examples is fine for illustration but not for evidence. If the authors fix the MR definitions, add uncertainty bounds, and soften the conclusions, the framework could be useful.

This paper deserves a serious referee: the question of whether metamorphic testing is a valid oracle for LLM behavior is worth airing. I would not desk-reject it, but I would send it for review expecting major revision.

Referee Report

4 major / 6 minor

Summary. The paper proposes MMT4NL, a metamorphic-testing framework for evaluating in-context learning in LLMs. It defines nine perturbation categories (Taxonomy, NER, Negation, Vocab, Fairness, Robustness, Temporal, SRL, Coreference) as metamorphic relations and introduces pass rate as a summary metric. The authors apply the framework to GPT-4o and Gemini-2.0-Flash on a 50-item sentiment dataset and a 50-item StrategyQA subset under zero-, one-, and few-shot prompting, with and without context for QA. They report pass rates in Tables 3–5 and manually inspect selected failures to argue that MMT4NL reveals interpretation flaws and biases in state-of-the-art LLMs.

Significance. The central observation that LLM outputs are unstable under small linguistic perturbations is empirically demonstrated and useful for the SE-for-AI community. The paper's strengths are its explicit use of metamorphic relations as external oracles, the absence of fitted parameters or result-dependent calibration, and the reproducible experimental structure (scripts, prompts, and data via the public repository). If the metamorphic relations were validated as meaning-preserving, the framework would provide a practical complement to accuracy-based evaluation. However, the current evidence is weakened by the oracle-validity and analysis issues discussed below, so the significance is conditional on a substantial revision.

major comments (4)

[Section 4, Table 2, Section 7.3] Several metamorphic relations assert output invariance under perturbations that the paper's own examples show are not meaning-preserving. The Taxonomy examples in Section 7.3, "I'm really hungry" → "I'm really starving" and "very complicated" → "very complex", change intensity and connotation, so a sentiment change can be a legitimate semantic response rather than a bug. The Robustness example in Table 2, "tired" → "tried", is not a minor typo but a different word, and the Fairness example in Section 7.3, "Chief Justice" → "female Chief Justice", changes the referent, so a changed answer can be factually correct. Because these metamorphic relations are asserted rather than validated (for example, with human annotation), the low pass rates in Tables 3–5 may substantially overstate the number of genuine LLM flaws. The authors should either restrict MRs to demonstrably meaning-preserving transformations, validate the transformations empirically, or reframe the reported rates as sensitivity rather than bugs.
[Section 4, Negation Handling] The Negation Handling MR is stated as ¬P(x) ≈ P(f(x)) for adding negation cues, but the examples do not implement a well-defined negation operation. For QA, "would it be not uncommon to find a penguin in Miami?" is not the negation of "would it be common to find a penguin in Miami?" — it introduces a double negative whose yes/no answer is not simply inverted. For sentiment, "I'm so not energetic" is not a negation of "I'm so tired." Consequently, the Negation pass rates in Tables 3–5 do not measure what the section claims, and the low rates cannot be attributed to a specific negation-handling failure.
[Section 5, Section 6.2, Tables 3–5] The experimental basis is too thin for the quantitative claims. Each task uses only 50 records, perturbations are mostly manually constructed, and results are reported as point-estimate pass rates without confidence intervals or significance tests. For instance, a difference between 84% and 88% in Table 3 corresponds to two examples out of 50, and the claim in Section 7.1 that MMT4NL "effectively revealed LLM interpretation flaws" rests on such small counts. The paper should provide uncertainty quantification, statistical comparisons across conditions, and a clear statement of how perturbations were generated and checked.
[Section 7.3] The manual analysis is presented as evidence of specific bug categories, but the authors state they "only showed surprising and severe scenarios" and report subset-relative rates such as 2/7, 3/8, and 5/7 without defining the denominator. This selective reporting cannot support general claims such as "Action Verb Replacement caused the highest failure rates" or "models anchor sentiment inconsistently." The authors should report the full failure inventory, define the subsets, and provide a systematic error analysis rather than selected examples.

minor comments (6)

[Section 8] The conclusion twice uses "NNT4ML" instead of "MMT4NL" ("We presented NNT4ML" and "NNT4ML can be applied").
[Table 2] The table entry "SLR QA" should be "SRL QA" to match the terminology used in Sections 3 and 4.
[Definition 2] The formula for pass rate has inconsistent spacing and subscripts ("Ntest,N pass") and should be typeset properly as N_pass / N_test.
[Section 6.2] Only the one-shot prompt templates are shown in the paper; the zero-shot and few-shot templates are not specified, so the experimental design is not fully reproducible from the text alone. The repository is referenced, but the paper should describe all templates or include them in an appendix.
[Section 5] The dataset provenance is thin: the Chat Sentiment dataset is cited only as a Kaggle dataset without version or license information, and StrategyQA is referenced via a Hugging Face user page rather than the original dataset publication.
[Section 7.3] The failure rates reported in the bullet points (e.g., 2/7 and 5/7) are not tied to the 50-item test sets described in Section 5; the paper should define the subsets from which these denominators come.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: pass rates are direct measurements against externally defined metamorphic relations; no fitted input is renamed as a prediction.

full rationale

The paper introduces MMT4NL as a metamorphic-testing framework for LLMs. Its central quantities are pass rates computed from predefined metamorphic relations (Section 4) and observed LLM outputs (Tables 3-5). There is no fitted parameter that is later renamed as a prediction: the metamorphic relations, such as P(x)=P(f(x)) for Taxonomy and Robustness, are chosen test oracles, and the reported pass rates are direct measurements of whether the model satisfies those relations on the selected 50-record datasets. The claim that MMT4NL 'revealed various linguistic bugs' is explicitly grounded in these pre-specified invariance relations, and the paper does not attempt to derive the MRs from the observed failures. The self-citations (refs 6, 15, 17) concern prior works by one author on negation QA and summarization, but they are used only as background references for LLM applications and are not load-bearing for the framework's validity or for the experimental conclusions. The only substantive weakness, that some perturbations may not be meaning-preserving (e.g., 'tired' to 'tried', 'hungry' to 'starving'), is a correctness/validity concern about the chosen test oracle, not a circularity: the paper does not assume the conclusion that LLMs are buggy; it tests an externally stated invariance condition. Therefore no step in the derivation chain reduces to its own inputs by construction, and no self-citation chain forces the results.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The central claim relies on these three assumptions, none of which are externally benchmarked. The validity of the metamorphic relations is the most consequential because low pass rates are interpreted as 'bugs' without independent justification.

assumptions (3)

domain assumption LLM outputs are deterministic or that a single response is representative.
The paper compares original and perturbed responses as if a single run is sufficient; temperature or other stochastic settings are not reported (Section 6.2).
ad hoc to paper Metamorphic relations, such as invariance under synonym substitution, are valid test oracles for trustworthiness.
The paper defines MRs such as P(x)=P(f(x)) for Taxonomy and interprets violations as bugs, without external justification that these invariances are always desirable (Section 4).
domain assumption The 50-record samples are representative of the broader tasks.
Datasets are subsampled with vague 'balanced' criteria; no detailed sampling or validation is provided (Section 5).

how reviews work

0 comments

Cite this review

Pith. "Pith review of Test It Before You Trust It: Applying Software Testing for Trustworthy In-context Learning." pith.science (2026). https://pith.science/paper/MQPDIBZ2

@misc{pith2026250418827,
  author       = {Pith},
  title        = {Pith review of: Test It Before You Trust It: Applying Software Testing for Trustworthy In-context Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MQPDIBZ2}},
  note         = {Machine review of arXiv:2504.18827}
}

read the original abstract

In-context learning (ICL) has emerged as a powerful capability of large language models (LLMs), enabling them to perform new tasks based on a few provided examples without explicit fine-tuning. Despite their impressive adaptability, these models remain vulnerable to subtle adversarial perturbations and exhibit unpredictable behavior when faced with linguistic variations. Inspired by software testing principles, we introduce a software testing-inspired framework, called MMT4NL, for evaluating the trustworthiness of in-context learning by utilizing adversarial perturbations and software testing techniques. It includes diverse evaluation aspects of linguistic capabilities for testing the ICL capabilities of LLMs. MMT4NL is built around the idea of crafting metamorphic adversarial examples from a test set in order to quantify and pinpoint bugs in the designed prompts of ICL. Our philosophy is to treat any LLM as software and validate its functionalities just like testing the software. Finally, we demonstrate applications of MMT4NL on the sentiment analysis and question-answering tasks. Our experiments could reveal various linguistic bugs in state-of-the-art LLMs.

Figures

Figures reproduced from arXiv: 2504.18827 by the authors.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

17 extracted references · 11 canonical work pages

[1]

Cambridge University Press, 2 edn

Ammann, P., Offutt, J.: Introduction to Software Testing. Cambridge University Press, 2 edn. (2016)

work page 2016
[2]

IEEE Transactions on Software Engineering41(5), 507–525 (2015).https://doi.org/10.1109/TSE.2014.2372785

Barr, E.T., Harman, M., McMinn, P., Shahbaz, M., Yoo, S.: The oracle problem in software testing: A survey. IEEE Transactions on Software Engineering41(5), 507–525 (2015).https://doi.org/10.1109/TSE.2014.2372785

arXiv 2015
[3]

Chen, T.Y., Cheung, S.C., Yiu, S.M.: Metamorphic testing: A new approach for generating next test cases. Tech. rep., The Hong Kong University of Science and Technology (1998)

work page 1998
[4]

Es, S., James, J., Espinosa-Anke, L., Schockaert, S.: Ragas: Automated evaluation of retrieval augmented generation (2023),https://arxiv.org/abs/2309.15217

arXiv 2023
[5]

confident-ai.com/(September 2024)

Inc., C.A.: Deepeval: The open-source llm evaluation framework.https://docs. confident-ai.com/(September 2024)

work page 2024
[6]

In: Nguyen, N.T., Boonsang, S., Fujita, H., Hnatkowska, B., Hong, T., Pasupa, K., Selamat, A

Jin, T., Racharak, T., Nguyen, L.M.: Negt5: A cross-task text-to-text framework for negation in question answering. In: Nguyen, N.T., Boonsang, S., Fujita, H., Hnatkowska, B., Hong, T., Pasupa, K., Selamat, A. (eds.) Intelligent Information and Database Systems - 15th Asian Conference, ACIIDS 2023, Proceedings, Part II. Lecture Notes in Computer Science, ...

work page 2023
[7]

Advances in neural information processing systems35, 22199–22213 (2022)

Kojima, T., Gu, S.S., Reid, M., Matsuo, Y., Iwasawa, Y.: Large language models are zero-shot reasoners. Advances in neural information processing systems35, 22199–22213 (2022)

2022
[8]

IEEE Transactions on Neural Networks and Learning Systems (2023)

Lu, Q., Sun, X., Long, Y., Gao, Z., Feng, J., Sun, T.: Sentiment analysis: Com- prehensive reviews, recent advances, and open challenges. IEEE Transactions on Neural Networks and Learning Systems (2023)

work page 2023

Show all 17 references

[9]

arXiv preprint arXiv:2108.12237 (2021)

Moradi, M., Samwald, M.: Evaluating the robustness of neural language models to input perturbations. arXiv preprint arXiv:2108.12237 (2021)

2021 arXiv
[10]

Kaggle (2023), available athttps://www

Nursyahrina: Chat sentiment dataset. Kaggle (2023), available athttps://www. kaggle.com/datasets/nursyahrina/chat-sentiment-dataset

2023
[11]

McGraw-Hill, Inc., USA, 7 edn

Pressman, R.: Software Engineering: A Practitioner’s Approach. McGraw-Hill, Inc., USA, 7 edn. (2009)

2009
[12]

In: Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence

Ribeiro,M.T.,Wu,T.,Guestrin,C.,Singh,S.:Beyondaccuracy:Behavioraltesting of nlp models with checklist (extended abstract). In: Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence. pp. 4824–4828. Interna- tional Joint Conferences on Artificial...

2021
[13]

IEEE Transactions on Software Engineering42, 805–824 (9 2016).https: //doi.org/10.1109/TSE.2016.2532875

Segura, S., Fraser, G., Sanchez, A.B., Ruiz-Cortes, A.: A survey on metamorphic testing. IEEE Transactions on Software Engineering42, 805–824 (9 2016).https: //doi.org/10.1109/TSE.2016.2532875

2016
[14]

Hugging Face Datasets (2023), available athttps:// huggingface.co/datasets/voidful/StrategyQA

Voidful: Strategyqa. Hugging Face Datasets (2023), available athttps:// huggingface.co/datasets/voidful/StrategyQA

2023
[15]

In: Amigó, E., Castells, P., Gonzalo, J., Carterette, B., Culpepper, J.S., Kazai, G

Xiong, Y., Racharak, T., Nguyen, M.L.: Extractive elementary discourse units for improving abstractive summarization. In: Amigó, E., Castells, P., Gonzalo, J., Carterette, B., Culpepper, J.S., Kazai, G. (eds.) SIGIR ’22: The 45th International ACM SIGIR Conference on Research ...

2022
[16]

Journal of Machine Learning Research 25(254), 1–22 (2024)

Zhu, K., Zhao, Q., Chen, H., Wang, J., Xie, X.: Promptbench: A unified library for evaluation of large language models. Journal of Machine Learning Research 25(254), 1–22 (2024)

2024
[17]

IEEE Access10, 67047–67057 (2022)

Zin, M.M., Racharak, T., Nguyen, M.L.: Dbape: Denoising-based APE system for improving english-myanmar NMT. IEEE Access10, 67047–67057 (2022)

2022

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.

[1] [1]

Cambridge University Press, 2 edn

Ammann, P., Offutt, J.: Introduction to Software Testing. Cambridge University Press, 2 edn. (2016)

work page 2016

[2] [2]

IEEE Transactions on Software Engineering41(5), 507–525 (2015).https://doi.org/10.1109/TSE.2014.2372785

Barr, E.T., Harman, M., McMinn, P., Shahbaz, M., Yoo, S.: The oracle problem in software testing: A survey. IEEE Transactions on Software Engineering41(5), 507–525 (2015).https://doi.org/10.1109/TSE.2014.2372785

arXiv 2015

[3] [3]

Chen, T.Y., Cheung, S.C., Yiu, S.M.: Metamorphic testing: A new approach for generating next test cases. Tech. rep., The Hong Kong University of Science and Technology (1998)

work page 1998

[4] [4]

Es, S., James, J., Espinosa-Anke, L., Schockaert, S.: Ragas: Automated evaluation of retrieval augmented generation (2023),https://arxiv.org/abs/2309.15217

arXiv 2023

[5] [5]

confident-ai.com/(September 2024)

Inc., C.A.: Deepeval: The open-source llm evaluation framework.https://docs. confident-ai.com/(September 2024)

work page 2024

[6] [6]

In: Nguyen, N.T., Boonsang, S., Fujita, H., Hnatkowska, B., Hong, T., Pasupa, K., Selamat, A

Jin, T., Racharak, T., Nguyen, L.M.: Negt5: A cross-task text-to-text framework for negation in question answering. In: Nguyen, N.T., Boonsang, S., Fujita, H., Hnatkowska, B., Hong, T., Pasupa, K., Selamat, A. (eds.) Intelligent Information and Database Systems - 15th Asian Conference, ACIIDS 2023, Proceedings, Part II. Lecture Notes in Computer Science, ...

work page 2023

[7] [7]

Advances in neural information processing systems35, 22199–22213 (2022)

Kojima, T., Gu, S.S., Reid, M., Matsuo, Y., Iwasawa, Y.: Large language models are zero-shot reasoners. Advances in neural information processing systems35, 22199–22213 (2022)

2022

[8] [8]

IEEE Transactions on Neural Networks and Learning Systems (2023)

Lu, Q., Sun, X., Long, Y., Gao, Z., Feng, J., Sun, T.: Sentiment analysis: Com- prehensive reviews, recent advances, and open challenges. IEEE Transactions on Neural Networks and Learning Systems (2023)

work page 2023