Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

Can LLMs Detect Intrinsic Hallucinations in Paraphrasing and Machine Translation?

T0 review · 3 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Pith's one-line read Open-access LLMs can detect intrinsic hallucinations in paraphrases and translations, and much smaller NLI models match or beat them.

desk verdict A solid, honest comparison of LLM and NLI hallucination detectors on HalluciGen, but the prompt-consistency claim is overstated and the manually constructed test set limits external validity. read the letter →

arxiv 2504.20699 v1 pith:RC7CUKP4 submitted 2025-04-29 cs.CL cs.AI

classification cs.CLcs.AI
keywords hallucinationdetectionintrinsicparaphrasegenerationmachinetranslationlargelanguagemodelsnaturalinferencepromptrobustnessHalluciGen
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Hallucinations in machine translation and paraphrasing are outputs that are not logically supported by their source. This paper asks whether open-access large language models can be pointed at a source sentence, two candidate outputs, and asked which one is hallucinated, and whether the answer holds across languages and prompts. Using the HalluciGen contrastive test sets, it finds that the two strongest models, Mixtral-8x7B-Instruct and Meta-Llama-3-70B-Instruct, detect these hallucinations consistently across English and Swedish paraphrases and English-French/English-German translation in both directions. The notable result is that much smaller natural-language-inference (NLI) models, which only judge textual entailment, perform comparably well and sometimes beat the LLMs. The paper therefore concludes that LLMs are suitable detectors but not the only viable option, and that none of model size, instruction tuning, or prompt choice is a reliable predictor of performance.

What carries the argument

The central object is the HalluciGen contrastive detection setup: each example supplies a source sentence and two candidate hypotheses—one faithful and one containing an intrinsic hallucination—and the detector must choose which hypothesis the source does not logically support. A hallucination is operationalized as the absence of entailment rather than the presence of an explicit contradiction, which relaxes the earlier intrinsic-hallucination definition of Ji et al. (2023) to include additions, substitutions, negation, tense changes, and other category-specific errors. The LLMs are prompted zero-shot with six prompt formulations, while the NLI baselines score entailment between the source and each hypothesis and pick the less supported one, so the whole comparison reduces hallucination detection to a textual-entailment judgement.

What would settle it

Collect a corpus of naturally occurring hallucinations—outputs produced by real paraphrase and translation systems and judged by human annotators as unsupported by their source—and run the same detectors on it; if F1 drops toward chance on that data, the conclusion that open-access LLMs are appropriate detectors would not extend beyond the manually constructed test sets.

Watch

Extended reading notes

Core claim

The paper's central claim is that open-access LLMs can serve as zero-shot detectors of intrinsic hallucinations in two conditional generation tasks, if a hallucination is defined as a hypothesis that is not entailed by its source. Over the HalluciGen test sets, the strongest models—Mixtral-8x7B-Instruct and Meta-Llama-3-70B-Instruct—perform consistently well across all five language directions, while the considerably smaller NLI baselines (BGE-M3-ZeroShot-v2.0 and, for Swedish, ScandiNLI-Large) are hard to beat, especially in the paraphrase scenario and for French-to-English translation. The paper treats this as evidence that LLMs are appropriate for the detection task but are not the only viable option, and it reports that prompt formulation has only a small effect on performance, with English prompts generally best and explicit mention of the word "hallucination" sometimes hurting certain models.

Load-bearing premise

The central premise is that the HalluciGen test sets, whose hallucinated hypotheses were largely constructed manually from a fixed set of error categories, represent how intrinsic hallucinations actually appear in real paraphrases and translations; if that premise fails, the reported F1 scores do not generalize to naturally occurring hallucinations.

Editorial extensions

If this is right

  • Open-access LLMs such as Mixtral-8x7B-Instruct and Meta-Llama-3-70B-Instruct can be deployed directly as zero-shot hallucination detectors for paraphrases and translations across at least five language directions.
  • Detection accuracy is largely stable across the six prompt formulations and prompt languages, so practical deployment does not require careful prompt engineering for each task.
  • Small NLI models, which are far cheaper to run and cannot produce invalid outputs, match or exceed the best LLMs in paraphrase hallucination detection and in some translation directions, making them a cost-effective alternative.
  • Model size and instruction tuning are not reliable predictors of detection skill: the larger Mixtral-8x22B-Instruct underperforms the smaller Mixtral-8x7B-Instruct, and instruction-tuned GPT-SW3 variants underperform their base versions on paraphrase scenarios.
  • English prompts generally outperform prompts written in the target language, giving a concrete guideline for multilingual use of these detectors.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the hallucinated hypotheses in the test sets were largely constructed manually from a fixed error taxonomy, the headline scores likely give an optimistic view of performance on naturally occurring hallucinations; a natural-corpus replication is the direct test of robustness.
  • The near-parity of NLI models suggests that intrinsic hallucination detection in these tasks is essentially an entailment judgement, so advances in multilingual NLI could be the cheapest route to better detectors, rather than scaling LLMs.
  • The finding that some models perform worse when the prompt explicitly says "hallucination" hints at unstable associations with the term from instruction tuning; a direct extension is to compare definition-only prompts across more languages and model families.
  • Because cross-lingual detection may implicitly require the detector to understand both languages, an explicit translation or back-translation step before the entailment check is a testable upgrade to the LLM prompting setup.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper investigates whether open-access large language models (LLMs) can detect intrinsic hallucinations in paraphrasing and machine translation, building on the HalluciGen shared task. The authors evaluate a suite of LLMs (Llama-3, Mixtral, EuroLLM, GPT-SW3) across two paraphrase languages (English, Swedish) and four translation directions (en⇔fr, en⇔de), using six prompt formulations in two prompt languages. They compare against multilingual zero-shot NLI models. The main findings are that Mixtral-8x7B-Instruct and Meta-Llama-3-70B-Instruct perform consistently well, that performance is stable across prompts, and that NLI models—especially in the paraphrase scenario—are competitive with or better than the LLMs. The paper also analyzes effects of model size, instruction tuning, and prompt language, and conducts an error analysis by hallucination category.

Significance. If the results are taken at face value, this is a useful empirical contribution: it systematically compares a range of open-access LLMs and lightweight NLI baselines on a defined hallucination detection task, with careful control of prompt variations and documentation of inference cost. The finding that small NLI models rival much larger LLMs is practically important and well demonstrated on the HalluciGen test sets. However, the significance is limited by two factors: the test sets are constructed by manually injecting a fixed set of error types into otherwise faithful hypotheses, which the authors themselves acknowledge may not reflect real-world intrinsic hallucinations; and the test sets are small (100–119 examples per condition), with no confidence intervals or significance tests, so the ranking between models is not shown to be statistically robust. The paper's central comparative claims therefore hold only within the narrow HalluciGen setting unless additional validation is provided.

major comments (3)
  1. The evaluation sets are built by manually constructing hallucinated hypotheses from a fixed list of eleven error categories, and for translation most examples are sampled from the ACES challenge set. The paper's own Limitations section states that 'a number of hallucination examples were constructed manually and may not accurately reflect real-world intrinsic hallucinations.' This is a load-bearing threat to external validity: the headline F1 scores and the LLM/NLI comparison are all measured on this artificial distribution, where each item is a forced contrast between a faithful hypothesis and a localized corruption. The conclusion that 'LLMs are appropriate for this task' should be explicitly scoped to the HalluciGen task definition, not to intrinsic hallucination detection in naturally occurring paraphrases and translations. A concrete mitigation would be to add a small evaluation set of naturally occurring model hallucinations (e.g., from SHROOM or sampled from model generations) to check whether the relative ranking of LLMs and NLI baselines persists.
  2. The test sets contain only 100–119 examples per language condition. Many of the reported differences in F1 between the best LLMs and the NLI baselines, or between strong LLMs, are on the order of 0.03–0.10 (e.g., English paraphrase: NLI 0.90, Meta-Llama-3-70B-Instruct P5 0.94, Mixtral-8x7B-Instruct P6 0.86). With 119 examples, such differences are likely within sampling noise, and the paper provides no confidence intervals, bootstrap estimates, or significance tests (e.g., McNemar's test on paired predictions). Consequently, the claims that Mixtral and Llama-3-70B-Instruct 'perform consistently well across all languages and scenarios' and that the NLI baseline is 'hard to beat' are not statistically supported. Reporting bootstrap confidence intervals for the main comparisons would be a necessary addition.
  3. The paper defines a hallucination as a hypothesis that is not entailed by the source (Section 1), and the NLI baselines are explicitly trained to predict entailment (Section 4.4). The strong performance of the NLI models is therefore largely by construction, and the framing of their competitiveness as a notable finding ('suggesting that LLM-based detectors are not the only viable option') understates this circularity. This does not invalidate the results, but the comparison would be more informative if the authors either explicitly contextualized NLI performance as expected given the task definition, or added a non-entailment-based baseline (e.g., a lexical overlap heuristic or an MT metric) to calibrate task difficulty. Both options would strengthen the paper's analysis of what the task actually measures.
minor comments (6)
  1. The text states that GPT-SW3-20B-Instruct reaches an F1 of 0.07 for Prompt 2 on Swedish paraphrase, but Table 2 shows 0.04 for that cell; please correct the discrepancy.
  2. The sentence 'One exception is Swedish paraphrase, where GPT-SW3-20B-INSTRUCT performs best with Swedish prompts' is contradicted by Table 2: the instruct variant has a higher average with English prompts (0.35) than with Swedish prompts (0.13), whereas the base GPT-SW3-20B model shows the claimed pattern (0.31 vs 0.35). Please correct the model identifier.
  3. In the list of post-processing label variations, 'hypothesis12' is presumably a typo for 'hypothesis2'; please fix.
  4. The paper reports F1 but does not explicitly define the positive class; please state that F1 is computed for the hallucination (hyp−) label, and clarify whether micro- or macro-averaging is used.
  5. The hallucination categories are presented in a different order in Table 5 than in the text (e.g., 'Conversion' appears before 'Date' in the text but after 'Tense' in the table); please align the ordering for readability.
  6. The example output for Prompt 5 ('"2" ⇒ hyp2 ⇒ hyp1*') requires the footnote to be interpreted; consider adding a brief in-table explanation of the flip, or noting it explicitly in the caption.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper reports empirical benchmark results with human-annotated gold labels; the NLI/entailment alignment is an acknowledged design feature, not a fitted prediction.

full rationale

The paper is an evaluation study: it measures F1 scores of open-access LLMs and NLI baselines on the HalluciGen detection task. No parameter is fitted to the test set, and no 'prediction' is derived from the task definition. The strongest potential concern is that hallucination is defined as 'an output, or hypothesis, that is not entailed by the input, or source' (Section 1), while the NLI baselines are 'trained specifically for predicting textual entailment' (Section 4.4). This alignment is real, but it does not make the evaluation circular: gold labels come from human annotation with high inter-annotator agreement (0.90 English, 0.88 Swedish), and the NLI models must still empirically succeed on the specific constructed items. The authors explicitly acknowledge that NLI competence is 'arguably unsurprising' given this definition, which further shows they are not presenting a derivation. The use of the HalluciGen dataset from the same research group is a self-citation, but the dataset is public, built in part on external resources (SHROOM, ACES, PAWS-X), and the paper's Limitations candidly states that manually constructed hallucination examples 'may not accurately reflect real-world intrinsic hallucinations.' That is an external-validity caveat, not circular reasoning. Since the central claims are empirical comparisons on a fixed, human-labeled benchmark, there is no load-bearing reduction of the results to the inputs by construction.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new entities or fitted scientific constants. Its central claims rest on the task definition equating hallucination with non-entailment, on the quality of the manually constructed gold labels, and on the correctness of the rule-based label extraction. The decoding parameters and NLI decision rule are hand-chosen modeling choices, disclosed but not systematically varied.

free parameters (2)
  • Decoding hyperparameters (temperature, top-k, max_new_tokens) = 0.1, 20, 5
    Chosen after experimentation on the trial sets (Section 4.1), not optimized per model. The paper notes they were not optimized for individual models (Limitations).
  • NLI label decision rule = E/NE comparison over the two hypotheses
    Author-defined rule in Appendix F to map entailment scores to a single hyp label; no thresholds or alternative rules are compared.
assumptions (3)
  • domain assumption Hallucination equals non-entailment of the hypothesis by the source.
    Section 1 defines intrinsic hallucination this way, relaxing Ji et al. (2023). The entire dataset construction and evaluation depend on this equivalence.
  • domain assumption The gold labels in HalluciGen are correct and unbiased.
    Section 3.1: annotation by the authors with high inter-annotator agreement (0.90 en, 0.88 sv) and sampled from ACES and PAWS-X; however, examples were manually constructed and may not reflect real hallucinations (Limitations).
  • domain assumption Rule-based post-processing correctly maps model outputs to hyp1 or hyp2.
    Appendix D: outputs containing both labels or no label are invalid; mapping errors could affect F1, though the rules are described in detail.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Can LLMs Detect Intrinsic Hallucinations in Paraphrasing and Machine Translation?." pith.science (2026). https://pith.science/paper/RC7CUKP4

@misc{pith2026250420699,
  author       = {Pith},
  title        = {Pith review of: Can LLMs Detect Intrinsic Hallucinations in Paraphrasing and Machine Translation?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RC7CUKP4}},
  note         = {Machine review of arXiv:2504.20699}
}
read the original abstract

A frequently observed problem with LLMs is their tendency to generate output that is nonsensical, illogical, or factually incorrect, often referred to broadly as hallucination. Building on the recently proposed HalluciGen task for hallucination detection and generation, we evaluate a suite of open-access LLMs on their ability to detect intrinsic hallucinations in two conditional generation tasks: translation and paraphrasing. We study how model performance varies across tasks and language and we investigate the impact of model size, instruction tuning, and prompt choice. We find that performance varies across models but is consistent across prompts. Finally, we find that NLI models perform comparably well, suggesting that LLM-based detectors are not the only viable option for this specific task.

Figures

Figures reproduced from arXiv: 2504.20699 by the authors.

Figure 1
Figure 1. The average proportion of incorrectly labeled source-hyp [PITH_FULL_IMAGE:figures/full_fig_p016_1.png] view at source ↗
Figure 2
Figure 2. The average proportion of incorrectly labeled source-hyp [PITH_FULL_IMAGE:figures/full_fig_p016_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. $M^2PO$: Multi-Perspective Multi-Pair Preference Optimization for Machine Translation

    cs.CL 2025-10 reject novelty 5.0 of 10

    M2PO combines a QE-plus-alignment reward with a dynamic curriculum and multi-pair DPO loss, and reports WMT21-22 gains for a 7B translation model, but the abstract's WMT23/24 9B parity claims are unsupported.

Reference graph

Works this paper leans on

24 extracted references · 10 canonical work pages · cited by 1 Pith paper

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Chantal Amrhein, Nikita Moghe, and Liane Guillou. 2022. https://aclanthology.org/2022.wmt-1.44 ACES : Translation accuracy challenge sets for evaluating machine translation metrics . In Proceedings of the Seventh Conference on Machine Translation ( WMT ) , pages 479--513. Association for Computational Linguistics

  4. [4]

    O hman, Yvonne Adesam, Lars Borin, Dana Dann \'e lls, Markus Forsberg, Tim Isbister, Anna Lindahl, Martin Malmsten, Faton Rekathati, Magnus Sahlgren, Elena Volodina, Love B \

    Aleksandrs Berdicevskis, Gerlof Bouma, Robin Kurtz, Felix Morger, Joey \"O hman, Yvonne Adesam, Lars Borin, Dana Dann \'e lls, Markus Forsberg, Tim Isbister, Anna Lindahl, Martin Malmsten, Faton Rekathati, Magnus Sahlgren, Elena Volodina, Love B \"o rjeson, Simon Hengchen, and Nina Tahmasebi. 2023. Superlim: A S wedish language understanding evaluation be...

  5. [5]

    Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. https://arxiv.org/abs/2402.03216 Bge m3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation . Preprint, arXiv:2402.03216

  6. [6]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

  7. [7]

    Luise D \"u rlich, Evangelia Gogoulou, Liane Guillou, Joakim Nivre, and Shorouq Zahra. 2024. Overview of the clef-2024 eloquent lab: Task 2 on hallucigen. In 25th Working Notes of the Conference and Labs of the Evaluation Forum, CLEF 2024. Grenoble. 9 September 2024 through 12 September 2024, volume 3740, pages 691--702. CEUR-WS

  8. [8]

    Ariel Ekgren, Amaru Cuba Gyllensten, Felix Stollenwerk, Joey \"O hman, Tim Isbister, Evangelia Gogoulou, Fredrik Carlsson, Judit Casademont, and Magnus Sahlgren. 2024. GPT - SW 3: An autoregressive language model for the S candinavian languages. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and ...

Show all 24 references
  1. [9]

    Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of hallucination in natural language generation. ACM Comput. Surv., 55(12)

  2. [10]

    Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. 2024. Mixtral of experts. arXiv preprint arXiv:2401.04088

  3. [11]

    Ehsan Kamalloo, Nouha Dziri, Charles Clarke, and Davood Rafiei. 2023. https://doi.org/10.18653/v1/2023.acl-long.307 Evaluating open-domain question answering in the era of large language models . In Proceedings of the 61st Annual Meeting of the Association for Computational Li...

  4. [12]

    Jenna Kanerva, Filip Ginter, Li-Hsin Chang, Iiro Rastas, Valtteri Skantsi, Jemina Kilpel \"a inen, Hanna-Mari Kupari, Jenna Saarni, Maija Sev \'o n, and Otto Tarkka. 2021. F innish paraphrase corpus. In Proceedings of the 23rd Nordic Conference on Computational Linguistics (No...

  5. [13]

    Zettlemoyer

    Haoqiang Kang, Terra Blevins, and Luke S. Zettlemoyer. 2024. https://api.semanticscholar.org/CorpusID:267740651 Comparing hallucination detection metrics for multilingual generation . ArXiv, abs/2402.10496

  6. [14]

    Jussi Karlgren, Luise D \"u rlich, Evangelia Gogoulou, Liane Guillou, Joakim Nivre, Magnus Sahlgren, Aarne Talman, and Shorouq Zahra. 2024. Overview of eloquent 2024---shared tasks for evaluating generative language model quality. In Experimental IR Meets Multilinguality, Mult...

  7. [15]

    Tom Kocmi, Eleftherios Avramidis, Rachel Bawden, Ond r ej Bojar, Anton Dvorkovich, Christian Federmann, Mark Fishel, Markus Freitag, Thamme Gowda, Roman Grundkiewicz, Barry Haddow, Philipp Koehn, Benjamin Marie, Christof Monz, Makoto Morishita, Kenton Murray, Makoto Nagata, To...

  8. [16]

    Junyi Li, Xiaoxue Cheng, Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.397 H alu E val: A large-scale hallucination evaluation benchmark for large language models . In Proceedings of the 2023 Conference on Empirical Methods in Natur...

  9. [17]

    Potsawee Manakul, Adian Liusie, and Mark Gales. 2023. S elf C heck GPT : Zero-resource black-box hallucination detection for generative large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 9004--9017, Singapore...

  10. [18]

    Timothee Mickus, Elaine Zosa, Raul Vazquez, Teemu Vahtola, J \"o rg Tiedemann, Vincent Segonne, Alessandro Raganato, and Marianna Apidianaki. 2024. https://aclanthology.org/2024.semeval-1.273 S em E val-2024 task 6: SHROOM , a shared-task on hallucinations and related observab...

  11. [19]

    Dan Saattrup Nielsen. 2022. https://aclanthology.org/D19-1382 Scandinli: Natural language inference for the scandinavian languages . https://github.com/alexandrainst/ScandiNLI

  12. [20]

    Xiao Pu, Mingqi Gao, and Xiaojun Wan. 2023. https://arxiv.org/abs/2309.09558 Summarization is (almost) dead . Preprint, arXiv:2309.09558

  13. [21]

    Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. https://doi.org/10.18653/v1/N18-1101 A broad-coverage challenge corpus for sentence understanding through inference . In Proceedings of the 2018 Conference of the North A merican Chapter of the Association for Computation...

  14. [22]

    Yinfei Yang, Yuan Zhang, Chris Tar, and Jason Baldridge. 2019. https://doi.org/10.18653/v1/D19-1382 PAWS - X : A cross-lingual adversarial dataset for paraphrase identification . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the ...

  15. [23]

    Wenxuan Zhang, Yue Deng, Bing Liu, Sinno Pan, and Lidong Bing. 2024. https://doi.org/10.18653/v1/2024.findings-naacl.246 Sentiment analysis in the era of large language models: A reality check . In Findings of the Association for Computational Linguistics: NAACL 2024, pages 38...

  16. [24]

    Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian-Yun Nie, and Ji-Rong W...

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.