Pith. sign in

REVIEW 3 major objections 7 minor 23 references

UCSC at SemEval-2025 Task 3: Context, Models and Prompt Optimization for Automated Hallucination Detection in LLM Output

T0 review · 3 major / 7 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A retrieval, prompting, and span-mapping pipeline achieves the best average hallucination-detection rank across all fourteen Mu-SHROOM languages.

desk verdict Solid shared-task system report; the ranking claim holds up, but the 'better-than-human' claim is not supported by the reported human comparison. read the letter →

arxiv 2505.03030 v1 pith:4654MCGY submitted 2025-05-05 cs.CL

Sicong Huang , Jincheng He , Shiyuan Huang , Karthik Raja Anandan , Arkajyoti Chakraborty , Ian Lane This is my paper · ORCID
classification cs.CL
keywords hallucinationdetectionspanlabelinglargelanguagemodelsretrieval-augmentedfact-checkingpromptoptimizationmultilingualNLPMu-SHROOMSemEval2025Task3
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that span-level hallucination detection in LLM question answering does not require specialized training. A pipeline that retrieves context from a search API, prompts an off-the-shelf LLM to flag unverifiable text, and maps the flagged text back to character spans achieved the best average rank across all fourteen languages of the Mu-SHROOM shared task. The authors further report that adding retrieved context raised English IoU by 27% over a no-context baseline, that simple text extraction beat a knowledge-graph-based verifier, and that automatic prompt optimization gave additional gains. Their central practical claim is that with good context, a simple prompting approach can detect hallucinations with better-than-human accuracy.

What carries the argument

The load-bearing machinery is the three-stage pipeline. Stage one retrieves context by querying Perplexity Sonar Pro with either the question itself or claims extracted from the answer, assuming the returned content is sufficient to verify the answer. Stage two identifies unverifiable content in three ways: direct text extraction, in which the LLM finds spans unsupported by the context; knowledge-graph verification, in which the context is parsed into a graph and each answer fact is checked against it; and minimal-cost revision, in which a reasoning model rewrites the answer with the fewest changes and the edits are treated as hallucinations. Stage three maps those errors to character spans using substring matching, fact-to-span prompting, or edit distance. On top of this, prompt optimization with MiPROv2—a Bayesian search over instructions and few-shot demonstrations—tunes the detection prompt; the paper's key empirical finding is that the simplest path, direct text extraction with substring matching plus retrieved context, is the most reliable.

What would settle it

Re-run the manual annotation of the Mu-SHROOM English validation set using the exact official guidelines and the same Perplexity Sonar Pro context the system uses; if the best human annotator's IoU against the official hard labels reaches or exceeds the system's 0.57, the claim that a simple prompting approach beats human annotators would be refuted.

Watch

Extended reading notes

Core claim

The central discovery, in the paper's own terms, is that a three-stage pipeline—context retrieval via Perplexity Sonar Pro, hallucinated-content detection by prompting GPT-4o, GPT-4o-mini, o1, or DeepSeek-R1, and span mapping via substring match, fact-to-span mapping, or edit distance—ranks in the top two on IoU in 11 of 14 languages and on correlation in 10 of 14, and holds the best average position overall. Section 2 states that 'when paired with good context, a simple prompting-based approach can reliably detect hallucinations with better-than-human accuracy.' The evidence for that claim comes from internal annotation: the best individual human annotator reached IoU 0.48 and correlation 0.48 on the English validation set, while the best system reached 0.57 and 0.55. Prompt optimization with MiPROv2 improved the detector further, and combining five systems as pseudo-annotators raised soft-label correlation by about 5% on average at a typical 5% IoU cost.

Load-bearing premise

The system assumes the context returned by Perplexity Sonar Pro is complete and factually accurate enough to verify every claim in the answer, so any missing or erroneous search result becomes a missed or falsely flagged hallucination; the better-than-human comparison further assumes the authors' manual annotations follow the same labeling guidelines as the official Mu-SHROOM annotations.

Editorial extensions

If this is right

  • Off-the-shelf LLMs plus search-engine context are sufficient for competitive span-level hallucination detection; no fine-tuning or task-specific training is needed.
  • Retrieving context from the question is the single largest performance lever, lifting English IoU from 0.44 to 0.56 compared with no context.
  • Translating non-English questions to English before retrieval slightly hurts performance, so target-language retrieval is preferable for multilingual detection.
  • Combining diverse systems as pseudo-annotators raises soft-label Spearman correlation by up to 12% in some languages, at a typical 5% IoU cost.
  • Bayesian prompt optimization improves detection, though no single optimization target (IoU, Corr, or MaxIoU) dominates both validation and test sets.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension would be corrupting or truncating the retrieved context and measuring the IoU drop; this would isolate how much of the pipeline's edge comes from retrieval quality versus prompt strength.
  • The better-than-human comparison implicitly depends on the exact context and labeling guidelines given to human annotators; under a different annotation protocol the human-system gap could shrink or reverse.
  • Minimal-cost revision produces a corrected answer as a byproduct, so the detector could double as a lightweight text-editing tool for LLM outputs—an application the paper does not itself develop.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper describes the UCSC submission to SemEval 2025 Task 3 (Mu-SHROOM), a multilingual span-level hallucination detection benchmark. The proposed system is a three-stage pipeline: (i) context retrieval from Perplexity Sonar Pro, (ii) hallucinated content detection via direct text extraction, knowledge-graph verification, or minimal-cost revision, and (iii) mapping the detected errors back to character-level spans. The pipeline is enhanced by MiPROv2 prompt optimization and by combining multiple systems as pseudo-annotators to generate soft labels. The authors report top-two rankings in 11 of 14 languages on IoU and 10 of 14 on Corr, and claim the highest average position across all languages. They also claim that, when paired with good context, a simple prompting-based approach can detect hallucinations with better-than-human accuracy, supported by a comparison of their internal annotations against official gold labels.

Significance. If the central ranking claim holds, the paper demonstrates that strong multilingual span-level hallucination detection can be achieved with off-the-shelf LLMs, retrieved context, and optimized prompts, without task-specific fine-tuning. The paper's strengths include honest reporting of a failed knowledge-graph variant, a transparent description of the prompt-optimization protocol, and public release of code and results. The 'better-than-human' claim, however, is not established by the presented evidence, and the primary ranking claim is not fully verifiable from the manuscript as written.

major comments (3)
  1. [Section 2 and Section 4.2] The claim that the prompting approach achieves 'better-than-human accuracy' is not supported by the reported comparison. The human baseline is computed from the authors' own annotations scored against the official aggregated Mu-SHROOM labels, while the system is also scored against those same labels. The authors themselves state that 'potential differences in labeling guidelines' may explain the low human overlap. If the internal annotations follow a different protocol, the human IoU/Corr values are not a valid measure of human performance on the official task, and the comparison is apples-to-oranges. The appropriate baseline would be the agreement of the official Mu-SHROOM annotators against their aggregated gold labels, which is not reported. Without that, the 'better-than-human' conclusion does not follow; please either provide the official per-annotator agreement or remove/soften the claim.
  2. [Abstract and Table 2] The headline claim of 'ranking #1 in average position across all languages' is not verifiable from the manuscript. Table 2 reports only the authors' own per-language ranks and their averages (2.6 IoU, 2.4 Corr), but it does not include the full leaderboard or a citation to the official ranking. Since this is the central primary claim, please include the official overall ranking table (or a precise statement of the official metric and the relevant official source) so that the claim can be independently checked.
  3. [Tables 1, 3, and 4] The key comparisons in the paper are presented without any variance estimates or significance tests. For example, in Table 4 the validation IoU improves from 0.44 to 0.57 under prompt optimization, but the test IoU improves only from 0.55 to 0.60, and several differences in Table 1 are on the order of 0.01–0.03. Given that the paper draws conclusions such as 'prompt optimization improves system performance' and 'system combination improves Corr,' the absence of uncertainty quantification makes it difficult to assess which differences are reliable. Please add a limitation statement or, where feasible, confidence intervals or significance tests.
minor comments (7)
  1. [Section 5.1] The statement 'As our system development was focused only on English' is contradicted by the per-language prompt optimization and context-retrieval experiments described in Sections 3.5 and 4.1 and reported in Tables 1 and 8; please clarify that the pipeline design is language-agnostic and that language-specific prompt optimization was performed automatically on validation data.
  2. [Table 2] Please specify which submission the per-language ranks correspond to (for example, the best-performing single system or the system combination), since Table 1 reports multiple systems per language.
  3. [Section 3.1] The assumption that the retrieved context contains all information required to verify the answer is load-bearing for the pipeline but is only stated as an assumption; please add it to the limitations discussion in Section 5.3.
  4. [Appendix C, Table 6] The table would be more informative if it included a no-context baseline (for example, text extraction without any retrieved context) to directly show the size of the context-retrieval benefit.
  5. [Section 3.5] There is a typo: 'use used exactly the same methods' should read 'we used exactly the same methods.'
  6. [Appendix D.1] In the JSON example, 'co nfidence_ score' contains an erroneous space; please fix the formatting.
  7. [Section 4.2] There is a typo: 'Corr or 0.34' should read 'Corr of 0.34.'

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity; the system's ranking is validated against the external Mu-SHROOM test labels, and prompt optimization is confined to validation folds.

full rationale

The paper's central claim—ranking #1 in average position across 14 languages—is evaluated against official Mu-SHROOM test labels, which are external to the paper and not used for fitting. Prompt optimization with MiPROv2 is performed on validation data using 2-fold cross-validation (Section 4.1), and the reported test scores are held out from that process, so no fitted input is being relabeled as a prediction. The three-stage pipeline (context retrieval, detection, span mapping) is a system engineering contribution, not a derivation that reduces to its inputs. The only questionable passage is the 'better-than-human accuracy' claim in Sections 2 and 4.2, where the human baseline is the authors' own manual annotations scored against the official aggregated labels, and the authors themselves note 'potential differences in labeling guidelines.' This is a methodological weakness and an overstatement, but it is not circular: both the system and the human annotations are scored against the same official gold labels, and the system was not fitted to the authors' annotations. There is also no load-bearing self-citation: the cited tools (DSPy, MiPROv2, Perplexity, GPT-4o, DeepSeek-R1) are external, and no uniqueness theorem or prior same-author result is invoked to force the design. The derivation chain is self-contained against the external benchmark, so the circularity score is 0.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The system's performance rests on the quality of external context, on LLM outputs that are not formally verified, and on the benchmark labels. None of these are free parameters in a mathematical derivation sense, but the design choices listed below were selected using validation data.

free parameters (4)
  • Context source choice = Perplexity Sonar Pro
    Chosen after validation comparison in Table 6; all final results use this context source.
  • Optimized prompt text = not specified in paper
    MiPROv2 (Section 3.4) searches prompt instructions and few-shot demonstrations on validation subsets; exact optimized prompts are only promised in the code repository.
  • Per-language optimization target = IoU except English uses Corr
    Table 4 shows no single target dominates validation and test; the per-language choice introduces selection on validation data.
  • System combination membership = five systems listed in Appendix B
    Ensemble members were selected based on validation performance and diversity rather than a fixed rule.
assumptions (4)
  • domain assumption External search context is accurate and sufficient for verification.
    Section 3.1 assumes returned content contains all information required to answer the question; false or incomplete context breaks detection.
  • domain assumption LLM-provided span probabilities align with human annotation proportions.
    Soft-label Corr evaluation in Section 3.6 treats model confidence and agreement as a proxy for annotator agreement.
  • domain assumption Mu-SHROOM human labels are a valid external ground truth.
    All IoU and Corr scores are computed against these labels; the paper's own analysis notes annotator disagreement.
  • ad hoc to paper Author-generated annotations are comparable to official labels for human-vs-system comparison.
    Section 4.2 compares system scores to the authors' own labels and admits possible differences in labeling guidelines, so the 'better-than-human' claim is not grounded in the official gold standard.

how reviews work

0 comments
Cite this review

Pith. "Pith review of UCSC at SemEval-2025 Task 3: Context, Models and Prompt Optimization for Automated Hallucination Detection in LLM Output." pith.science (2026). https://pith.science/paper/4654MCGY

@misc{pith2026250503030,
  author       = {Pith},
  title        = {Pith review of: UCSC at SemEval-2025 Task 3: Context, Models and Prompt Optimization for Automated Hallucination Detection in LLM Output},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4654MCGY}},
  note         = {Machine review of arXiv:2505.03030}
}
read the original abstract

Hallucinations pose a significant challenge for large language models when answering knowledge-intensive queries. As LLMs become more widely adopted, it is crucial not only to detect if hallucinations occur but also to pinpoint exactly where in the LLM output they occur. SemEval 2025 Task 3, Mu-SHROOM: Multilingual Shared-task on Hallucinations and Related Observable Overgeneration Mistakes, is a recent effort in this direction. This paper describes the UCSC system submission to the shared Mu-SHROOM task. We introduce a framework that first retrieves relevant context, next identifies false content from the answer, and finally maps them back to spans in the LLM output. The process is further enhanced by automatically optimizing prompts. Our system achieves the highest overall performance, ranking #1 in average position across all languages. We release our code and experiment results.

Figures

Figures reproduced from arXiv: 2505.03030 by the authors.

Figure 1
Figure 1. The UCSC hallucination detection framework. We retrieve context from external sources, identify false [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

23 extracted references · 5 canonical work pages

  1. [1]

    Forrest Sheng Bao, Miaoran Li, Renyi Qu, Ge Luo, Erana Wan, Yujia Tang, Weisi Fan, Manveer Singh Tamber, Suleman Kazi, Vivek Sourabh, Mike Qi, Ruixuan Tu, Chenyu Xu, Matthew Gonzales, Ofer Mendelevitch, and Amin Ahmad. 2024. https://arxiv.org/abs/2410.13210 Faithbench: A diverse hallucination benchmark for summarization by modern llms . Preprint, arXiv:2410.13210

    arXiv 2024

  2. [2]

    Robert M Cronin, Daniel Fabbri, Joshua C Denny, S Trent Rosenbloom, and Gretchen Purcell Jackson. 2017. A comparison of rule-based and machine learning approaches for classifying patient portal messages. International journal of medical informatics, 105:110--120

    work page 2017

  3. [3]

    Peiyi Wang

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, and et al. Peiyi Wang. 2025. https://arxiv.org/abs/2501.12948 Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning . Preprint, arXiv:2501.12948

    arXiv 2025

  4. [4]

    Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Meng Wang, and Haofen Wang. 2024. https://arxiv.org/abs/2312.10997 Retrieval-augmented generation for large language models: A survey . Preprint, arXiv:2312.10997

    arXiv 2024

  5. [5]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, and Alex Vaughan et al. 2024. https://arxiv.org/abs/2407.21783 The llama 3 herd of models . Preprint, arXiv:2407.21783

    arXiv 2024

  6. [6]

    Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. 2025. https://doi.org/10.1145/3703155 A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions . ACM Transactions on Information Systems, 43(2):1–55

    doi:10.1145/3703155 2025

  7. [7]

    Zhengbao Jiang, Frank Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi-Yu, Yiming Yang, Jamie Callan, and Graham Neubig. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.495 Active retrieval augmented generation . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7969--7992, Singapore. Association for Compu...

    doi:10.18653/v1/2023.emnlp-main.495 2023

  8. [8]

    Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, and Christopher Potts

    Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Vardhamanan A, Saiful Haq, Ashutosh Sharma, Thomas T. Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, and Christopher Potts. 2024. https://openreview.net/forum?id=sY5N0zY5Od DSP y: Compiling declarative language model calls into state-of-the-art pipelines . In The Tw...

    2024

Show all 23 references
  1. [9]

    u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt\

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K\" u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt\" a schel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. In Proc...

    2020

  2. [10]

    Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. https://doi.org/10.18653/v1/2022.acl-long.229 T ruthful QA : Measuring how models mimic human falsehoods . In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pa...

    2022 doi

  3. [11]

    Andreas Marfurt and James Henderson. 2022. https://doi.org/10.18653/v1/2022.gem-1.21 Unsupervised token-level hallucination detection from summary generation by-products . In Proceedings of the 2nd Workshop on Natural Language Generation, Evaluation, and Metrics (GEM), pages 2...

    2022 doi

  4. [12]

    Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.741 FA ct S core: Fine-grained atomic evaluation of factual precision in long form text generatio...

    2023 doi

  5. [13]

    Abhika Mishra, Akari Asai, Vidhisha Balachandran, Yizhong Wang, Graham Neubig, Yulia Tsvetkov, and Hannaneh Hajishirzi. 2024. https://openreview.net/forum?id=dJMTn3QOWO Fine-grained hallucination detection and editing for language models . In First Conference on Language Modeling

    2024

  6. [14]

    Cheng Niu, Yuanhao Wu, Juno Zhu, Siliang Xu, KaShun Shum, Randy Zhong, Juntong Song, and Tong Zhang. 2024. https://doi.org/10.18653/v1/2024.acl-long.585 RAGT ruth: A hallucination corpus for developing trustworthy retrieval-augmented language models . In Proceedings of the 62n...

    2024 doi

  7. [15]

    OpenAI . 2024. https://cdn.openai.com/o1-system-card-20241205.pdf Openai o1 system card

    2024

  8. [16]

    Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, and et al

    OpenAI, Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, and et al. Alan Hayes. 2024. https://arxiv.org/abs/2410.21276 Gpt-4o system card . Preprint, arXiv:2410.21276

    2024 arXiv

  9. [17]

    Krista Opsahl-Ong, Michael J Ryan, Josh Purtell, David Broman, Christopher Potts, Matei Zaharia, and Omar Khattab. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.525 Optimizing instructions and demonstrations for multi-stage language model programs . In Proceedings of the 2...

    2024 doi

  10. [18]

    Rachneet Sachdeva, Yixiao Song, Mohit Iyyer, and Iryna Gurevych. 2024. https://arxiv.org/abs/2407.11930 Localizing and mitigating errors in long-form question answering . Preprint, arXiv:2407.11930

    2024 arXiv

  11. [19]

    Pranab Sahoo, Prabhash Meharia, Akash Ghosh, Sriparna Saha, Vinija Jain, and Aman Chadha. 2024. https://doi.org/10.18653/v1/2024.findings-emnlp.685 A comprehensive survey of hallucination in large language, image, video and audio foundation models . In Findings of the Associat...

    2024 doi

  12. [20]

    Ra\'ul V\'azquez, Timothee Mickus, Elaine Zosa, Teemu Vahtola, J\"org Tiedemann, Aman Sinha, Vincent Segonne, Fernando S\'anchez-Vega, Alessandro Raganato, Jindřich Libovický, Jussi Karlgren, Shaoxiong Ji, Jindřich Helcl, Liane Guillou, Ona de Gibert, Jaione Bengoetxea, Joseph...

    2025

  13. [21]

    Chunting Zhou, Graham Neubig, Jiatao Gu, Mona Diab, Francisco Guzm \'a n, Luke Zettlemoyer, and Marjan Ghazvininejad. 2021. https://doi.org/10.18653/v1/2021.findings-acl.120 Detecting hallucinated content in conditional neural sequence generation . In Findings of the Associati...

    2021 doi

  14. [22]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  15. [23]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.