REVIEW 3 major objections 6 minor 1 cited by
Can LLMs Detect Intrinsic Hallucinations in Paraphrasing and Machine Translation?
T0 review · 3 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash
Pith's one-line read Open-access LLMs can detect intrinsic hallucinations in paraphrases and translations, and much smaller NLI models match or beat them.
desk verdict A solid, honest comparison of LLM and NLI hallucination detectors on HalluciGen, but the prompt-consistency claim is overstated and the manually constructed test set limits external validity. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the HalluciGen contrastive detection setup: each example supplies a source sentence and two candidate hypotheses—one faithful and one containing an intrinsic hallucination—and the detector must choose which hypothesis the source does not logically support. A hallucination is operationalized as the absence of entailment rather than the presence of an explicit contradiction, which relaxes the earlier intrinsic-hallucination definition of Ji et al. (2023) to include additions, substitutions, negation, tense changes, and other category-specific errors. The LLMs are prompted zero-shot with six prompt formulations, while the NLI baselines score entailment between the source and each hypothesis and pick the less supported one, so the whole comparison reduces hallucination detection to a textual-entailment judgement.
What would settle it
Collect a corpus of naturally occurring hallucinations—outputs produced by real paraphrase and translation systems and judged by human annotators as unsupported by their source—and run the same detectors on it; if F1 drops toward chance on that data, the conclusion that open-access LLMs are appropriate detectors would not extend beyond the manually constructed test sets.
Extended reading notes
Core claim
The paper's central claim is that open-access LLMs can serve as zero-shot detectors of intrinsic hallucinations in two conditional generation tasks, if a hallucination is defined as a hypothesis that is not entailed by its source. Over the HalluciGen test sets, the strongest models—Mixtral-8x7B-Instruct and Meta-Llama-3-70B-Instruct—perform consistently well across all five language directions, while the considerably smaller NLI baselines (BGE-M3-ZeroShot-v2.0 and, for Swedish, ScandiNLI-Large) are hard to beat, especially in the paraphrase scenario and for French-to-English translation. The paper treats this as evidence that LLMs are appropriate for the detection task but are not the only viable option, and it reports that prompt formulation has only a small effect on performance, with English prompts generally best and explicit mention of the word "hallucination" sometimes hurting certain models.
Load-bearing premise
The central premise is that the HalluciGen test sets, whose hallucinated hypotheses were largely constructed manually from a fixed set of error categories, represent how intrinsic hallucinations actually appear in real paraphrases and translations; if that premise fails, the reported F1 scores do not generalize to naturally occurring hallucinations.
Editorial extensions
If this is right
- Open-access LLMs such as Mixtral-8x7B-Instruct and Meta-Llama-3-70B-Instruct can be deployed directly as zero-shot hallucination detectors for paraphrases and translations across at least five language directions.
- Detection accuracy is largely stable across the six prompt formulations and prompt languages, so practical deployment does not require careful prompt engineering for each task.
- Small NLI models, which are far cheaper to run and cannot produce invalid outputs, match or exceed the best LLMs in paraphrase hallucination detection and in some translation directions, making them a cost-effective alternative.
- Model size and instruction tuning are not reliable predictors of detection skill: the larger Mixtral-8x22B-Instruct underperforms the smaller Mixtral-8x7B-Instruct, and instruction-tuned GPT-SW3 variants underperform their base versions on paraphrase scenarios.
- English prompts generally outperform prompts written in the target language, giving a concrete guideline for multilingual use of these detectors.
Reading between the lines
- Because the hallucinated hypotheses in the test sets were largely constructed manually from a fixed error taxonomy, the headline scores likely give an optimistic view of performance on naturally occurring hallucinations; a natural-corpus replication is the direct test of robustness.
- The near-parity of NLI models suggests that intrinsic hallucination detection in these tasks is essentially an entailment judgement, so advances in multilingual NLI could be the cheapest route to better detectors, rather than scaling LLMs.
- The finding that some models perform worse when the prompt explicitly says "hallucination" hints at unstable associations with the term from instruction tuning; a direct extension is to compare definition-only prompts across more languages and model families.
- Because cross-lingual detection may implicitly require the detector to understand both languages, an explicit translation or back-translation step before the entailment check is a testable upgrade to the LLM prompting setup.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper investigates whether open-access large language models (LLMs) can detect intrinsic hallucinations in paraphrasing and machine translation, building on the HalluciGen shared task. The authors evaluate a suite of LLMs (Llama-3, Mixtral, EuroLLM, GPT-SW3) across two paraphrase languages (English, Swedish) and four translation directions (en⇔fr, en⇔de), using six prompt formulations in two prompt languages. They compare against multilingual zero-shot NLI models. The main findings are that Mixtral-8x7B-Instruct and Meta-Llama-3-70B-Instruct perform consistently well, that performance is stable across prompts, and that NLI models—especially in the paraphrase scenario—are competitive with or better than the LLMs. The paper also analyzes effects of model size, instruction tuning, and prompt language, and conducts an error analysis by hallucination category.
Significance. If the results are taken at face value, this is a useful empirical contribution: it systematically compares a range of open-access LLMs and lightweight NLI baselines on a defined hallucination detection task, with careful control of prompt variations and documentation of inference cost. The finding that small NLI models rival much larger LLMs is practically important and well demonstrated on the HalluciGen test sets. However, the significance is limited by two factors: the test sets are constructed by manually injecting a fixed set of error types into otherwise faithful hypotheses, which the authors themselves acknowledge may not reflect real-world intrinsic hallucinations; and the test sets are small (100–119 examples per condition), with no confidence intervals or significance tests, so the ranking between models is not shown to be statistically robust. The paper's central comparative claims therefore hold only within the narrow HalluciGen setting unless additional validation is provided.
major comments (3)
- The evaluation sets are built by manually constructing hallucinated hypotheses from a fixed list of eleven error categories, and for translation most examples are sampled from the ACES challenge set. The paper's own Limitations section states that 'a number of hallucination examples were constructed manually and may not accurately reflect real-world intrinsic hallucinations.' This is a load-bearing threat to external validity: the headline F1 scores and the LLM/NLI comparison are all measured on this artificial distribution, where each item is a forced contrast between a faithful hypothesis and a localized corruption. The conclusion that 'LLMs are appropriate for this task' should be explicitly scoped to the HalluciGen task definition, not to intrinsic hallucination detection in naturally occurring paraphrases and translations. A concrete mitigation would be to add a small evaluation set of naturally occurring model hallucinations (e.g., from SHROOM or sampled from model generations) to check whether the relative ranking of LLMs and NLI baselines persists.
- The test sets contain only 100–119 examples per language condition. Many of the reported differences in F1 between the best LLMs and the NLI baselines, or between strong LLMs, are on the order of 0.03–0.10 (e.g., English paraphrase: NLI 0.90, Meta-Llama-3-70B-Instruct P5 0.94, Mixtral-8x7B-Instruct P6 0.86). With 119 examples, such differences are likely within sampling noise, and the paper provides no confidence intervals, bootstrap estimates, or significance tests (e.g., McNemar's test on paired predictions). Consequently, the claims that Mixtral and Llama-3-70B-Instruct 'perform consistently well across all languages and scenarios' and that the NLI baseline is 'hard to beat' are not statistically supported. Reporting bootstrap confidence intervals for the main comparisons would be a necessary addition.
- The paper defines a hallucination as a hypothesis that is not entailed by the source (Section 1), and the NLI baselines are explicitly trained to predict entailment (Section 4.4). The strong performance of the NLI models is therefore largely by construction, and the framing of their competitiveness as a notable finding ('suggesting that LLM-based detectors are not the only viable option') understates this circularity. This does not invalidate the results, but the comparison would be more informative if the authors either explicitly contextualized NLI performance as expected given the task definition, or added a non-entailment-based baseline (e.g., a lexical overlap heuristic or an MT metric) to calibrate task difficulty. Both options would strengthen the paper's analysis of what the task actually measures.
minor comments (6)
- The text states that GPT-SW3-20B-Instruct reaches an F1 of 0.07 for Prompt 2 on Swedish paraphrase, but Table 2 shows 0.04 for that cell; please correct the discrepancy.
- The sentence 'One exception is Swedish paraphrase, where GPT-SW3-20B-INSTRUCT performs best with Swedish prompts' is contradicted by Table 2: the instruct variant has a higher average with English prompts (0.35) than with Swedish prompts (0.13), whereas the base GPT-SW3-20B model shows the claimed pattern (0.31 vs 0.35). Please correct the model identifier.
- In the list of post-processing label variations, 'hypothesis12' is presumably a typo for 'hypothesis2'; please fix.
- The paper reports F1 but does not explicitly define the positive class; please state that F1 is computed for the hallucination (hyp−) label, and clarify whether micro- or macro-averaging is used.
- The hallucination categories are presented in a different order in Table 5 than in the text (e.g., 'Conversion' appears before 'Date' in the text but after 'Tense' in the table); please align the ordering for readability.
- The example output for Prompt 5 ('"2" ⇒ hyp2 ⇒ hyp1*') requires the footnote to be interpreted; consider adding a brief in-table explanation of the flip, or noting it explicitly in the caption.
Circularity Check
No significant circularity: the paper reports empirical benchmark results with human-annotated gold labels; the NLI/entailment alignment is an acknowledged design feature, not a fitted prediction.
full rationale
The paper is an evaluation study: it measures F1 scores of open-access LLMs and NLI baselines on the HalluciGen detection task. No parameter is fitted to the test set, and no 'prediction' is derived from the task definition. The strongest potential concern is that hallucination is defined as 'an output, or hypothesis, that is not entailed by the input, or source' (Section 1), while the NLI baselines are 'trained specifically for predicting textual entailment' (Section 4.4). This alignment is real, but it does not make the evaluation circular: gold labels come from human annotation with high inter-annotator agreement (0.90 English, 0.88 Swedish), and the NLI models must still empirically succeed on the specific constructed items. The authors explicitly acknowledge that NLI competence is 'arguably unsurprising' given this definition, which further shows they are not presenting a derivation. The use of the HalluciGen dataset from the same research group is a self-citation, but the dataset is public, built in part on external resources (SHROOM, ACES, PAWS-X), and the paper's Limitations candidly states that manually constructed hallucination examples 'may not accurately reflect real-world intrinsic hallucinations.' That is an external-validity caveat, not circular reasoning. Since the central claims are empirical comparisons on a fixed, human-labeled benchmark, there is no load-bearing reduction of the results to the inputs by construction.
Assumptions & free parameters
free parameters (2)
- Decoding hyperparameters (temperature, top-k, max_new_tokens) =
0.1, 20, 5
- NLI label decision rule =
E/NE comparison over the two hypotheses
assumptions (3)
- domain assumption Hallucination equals non-entailment of the hypothesis by the source.
- domain assumption The gold labels in HalluciGen are correct and unbiased.
- domain assumption Rule-based post-processing correctly maps model outputs to hyp1 or hyp2.
Cite this review
Pith. "Pith review of Can LLMs Detect Intrinsic Hallucinations in Paraphrasing and Machine Translation?." pith.science (2026). https://pith.science/paper/RC7CUKP4
@misc{pith2026250420699,
author = {Pith},
title = {Pith review of: Can LLMs Detect Intrinsic Hallucinations in Paraphrasing and Machine Translation?},
year = {2026},
howpublished = {\url{https://pith.science/paper/RC7CUKP4}},
note = {Machine review of arXiv:2504.20699}
}
read the original abstract
A frequently observed problem with LLMs is their tendency to generate output that is nonsensical, illogical, or factually incorrect, often referred to broadly as hallucination. Building on the recently proposed HalluciGen task for hallucination detection and generation, we evaluate a suite of open-access LLMs on their ability to detect intrinsic hallucinations in two conditional generation tasks: translation and paraphrasing. We study how model performance varies across tasks and language and we investigate the impact of model size, instruction tuning, and prompt choice. We find that performance varies across models but is consistent across prompts. Finally, we find that NLI models perform comparably well, suggesting that LLM-based detectors are not the only viable option for this specific task.
Figures
Forward citations
Cited by 1 Pith paper
-
$M^2PO$: Multi-Perspective Multi-Pair Preference Optimization for Machine Translation
M2PO combines a QE-plus-alignment reward with a dynamic curriculum and multi-pair DPO loss, and reports WMT21-22 gains for a 7B translation model, but the abstract's WMT23/24 9B parity claims are unsupported.
Reference graph
Works this paper leans on
-
[1]
online" 'onlinestring :=
ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...
-
[2]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
-
[3]
Chantal Amrhein, Nikita Moghe, and Liane Guillou. 2022. https://aclanthology.org/2022.wmt-1.44 ACES : Translation accuracy challenge sets for evaluating machine translation metrics . In Proceedings of the Seventh Conference on Machine Translation ( WMT ) , pages 479--513. Association for Computational Linguistics
work page 2022
-
[4]
Aleksandrs Berdicevskis, Gerlof Bouma, Robin Kurtz, Felix Morger, Joey \"O hman, Yvonne Adesam, Lars Borin, Dana Dann \'e lls, Markus Forsberg, Tim Isbister, Anna Lindahl, Martin Malmsten, Faton Rekathati, Magnus Sahlgren, Elena Volodina, Love B \"o rjeson, Simon Hengchen, and Nina Tahmasebi. 2023. Superlim: A S wedish language understanding evaluation be...
work page 2023
-
[5]
Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. https://arxiv.org/abs/2402.03216 Bge m3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation . Preprint, arXiv:2402.03216
arXiv 2024
-
[6]
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783
arXiv 2024
-
[7]
Luise D \"u rlich, Evangelia Gogoulou, Liane Guillou, Joakim Nivre, and Shorouq Zahra. 2024. Overview of the clef-2024 eloquent lab: Task 2 on hallucigen. In 25th Working Notes of the Conference and Labs of the Evaluation Forum, CLEF 2024. Grenoble. 9 September 2024 through 12 September 2024, volume 3740, pages 691--702. CEUR-WS
work page 2024
-
[8]
Ariel Ekgren, Amaru Cuba Gyllensten, Felix Stollenwerk, Joey \"O hman, Tim Isbister, Evangelia Gogoulou, Fredrik Carlsson, Judit Casademont, and Magnus Sahlgren. 2024. GPT - SW 3: An autoregressive language model for the S candinavian languages. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and ...
work page 2024
Show all 24 references
-
[9]
Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of hallucination in natural language generation. ACM Comput. Surv., 55(12)
2023
-
[10]
Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. 2024. Mixtral of experts. arXiv preprint arXiv:2401.04088
2024 arXiv
-
[11]
Ehsan Kamalloo, Nouha Dziri, Charles Clarke, and Davood Rafiei. 2023. https://doi.org/10.18653/v1/2023.acl-long.307 Evaluating open-domain question answering in the era of large language models . In Proceedings of the 61st Annual Meeting of the Association for Computational Li...
2023 doi
-
[12]
Jenna Kanerva, Filip Ginter, Li-Hsin Chang, Iiro Rastas, Valtteri Skantsi, Jemina Kilpel \"a inen, Hanna-Mari Kupari, Jenna Saarni, Maija Sev \'o n, and Otto Tarkka. 2021. F innish paraphrase corpus. In Proceedings of the 23rd Nordic Conference on Computational Linguistics (No...
2021
-
[13]
Zettlemoyer
Haoqiang Kang, Terra Blevins, and Luke S. Zettlemoyer. 2024. https://api.semanticscholar.org/CorpusID:267740651 Comparing hallucination detection metrics for multilingual generation . ArXiv, abs/2402.10496
2024 arXiv
-
[14]
Jussi Karlgren, Luise D \"u rlich, Evangelia Gogoulou, Liane Guillou, Joakim Nivre, Magnus Sahlgren, Aarne Talman, and Shorouq Zahra. 2024. Overview of eloquent 2024---shared tasks for evaluating generative language model quality. In Experimental IR Meets Multilinguality, Mult...
2024
-
[15]
Tom Kocmi, Eleftherios Avramidis, Rachel Bawden, Ond r ej Bojar, Anton Dvorkovich, Christian Federmann, Mark Fishel, Markus Freitag, Thamme Gowda, Roman Grundkiewicz, Barry Haddow, Philipp Koehn, Benjamin Marie, Christof Monz, Makoto Morishita, Kenton Murray, Makoto Nagata, To...
2023 doi
-
[16]
Junyi Li, Xiaoxue Cheng, Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.397 H alu E val: A large-scale hallucination evaluation benchmark for large language models . In Proceedings of the 2023 Conference on Empirical Methods in Natur...
2023 doi
-
[17]
Potsawee Manakul, Adian Liusie, and Mark Gales. 2023. S elf C heck GPT : Zero-resource black-box hallucination detection for generative large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 9004--9017, Singapore...
2023
-
[18]
Timothee Mickus, Elaine Zosa, Raul Vazquez, Teemu Vahtola, J \"o rg Tiedemann, Vincent Segonne, Alessandro Raganato, and Marianna Apidianaki. 2024. https://aclanthology.org/2024.semeval-1.273 S em E val-2024 task 6: SHROOM , a shared-task on hallucinations and related observab...
2024
-
[19]
Dan Saattrup Nielsen. 2022. https://aclanthology.org/D19-1382 Scandinli: Natural language inference for the scandinavian languages . https://github.com/alexandrainst/ScandiNLI
2022
-
[20]
Xiao Pu, Mingqi Gao, and Xiaojun Wan. 2023. https://arxiv.org/abs/2309.09558 Summarization is (almost) dead . Preprint, arXiv:2309.09558
2023 arXiv
-
[21]
Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. https://doi.org/10.18653/v1/N18-1101 A broad-coverage challenge corpus for sentence understanding through inference . In Proceedings of the 2018 Conference of the North A merican Chapter of the Association for Computation...
2018 doi
-
[22]
Yinfei Yang, Yuan Zhang, Chris Tar, and Jason Baldridge. 2019. https://doi.org/10.18653/v1/D19-1382 PAWS - X : A cross-lingual adversarial dataset for paraphrase identification . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the ...
2019 doi
-
[23]
Wenxuan Zhang, Yue Deng, Bing Liu, Sinno Pan, and Lidong Bing. 2024. https://doi.org/10.18653/v1/2024.findings-naacl.246 Sentiment analysis in the era of large language models: A reality check . In Findings of the Association for Computational Linguistics: NAACL 2024, pages 38...
2024 doi
-
[24]
Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian-Yun Nie, and Ji-Rong W...
2024 arXiv
Reviewed August 16, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.