Few-Shot Optimized Framework for Hallucination Detection in Resource-Limited NLP Systems

Ahmed Nasreldin; Ali Hamdi; Ammar Mohammed; Baraa Hikal

REVIEW 4 major objections 5 minor 1 cited by

Few-Shot Optimized Framework for Hallucination Detection in Resource-Limited NLP Systems

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read An ensemble of seven LoRA-tuned Mistral checkpoints trained on DeepSeek-v3 weak labels claims 85.5 percent accuracy and the top rank on SHROOM's model-agnostic hallucination detection track.

desk verdict A competent shared-task system paper whose headline ensemble result is probably right but under-reported; the missing checkpoints v4/v7 need an explanation before I'd trust the 0.855. read the letter →

arxiv 2501.16616 v1 pith:J4F7VA3Z submitted 2025-01-28 cs.CL

Baraa Hikal , Ahmed Nasreldin , Ali Hamdi , Ammar Mohammed This is my paper · ORCID

classification cs.CL

keywords few-shotlearninghallucinationdetectionweaksupervisionpromptengineeringensembleLoRAfine-tuningMistral-7BSHROOMsharedtask

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is trying to establish that a hallucination detector can be built to a competitive level even when labeled data is scarce. It uses DeepSeek-v3 with iteratively refined system instructions and an eight-shot prompt to generate weak labels for 30,000 unlabeled training samples, then restructures those labels into a conversational format. A 7-billion-parameter instruct model, Mistral-7B-Instruct-v0.3, is fine-tuned on this data with LoRA, and seven fine-tuned checkpoints are combined by majority voting. The authors report 85.5 percent accuracy on the SHROOM shared task's model-agnostic test set, which they state is the top rank in that track. If correct, this would show a practical route to reliable hallucination detection on modest hardware without a large annotation budget.

What carries the argument

The mechanism has four linked components. First, DeepSeek-v3 is used as a weak labeler, with its prompts improved in three stages: generic system instructions, task-specific system instructions, and an eight-shot prompt that includes labeled examples; validation accuracy climbs from 73.6 to 77.1 to 82.4 percent across these stages. Second, the raw classification-style dataset is restructured into conversational turns (system, user, assistant), so the task becomes a generation task rather than a classification task. Third, Mistral-7B-Instruct-v0.3 is fine-tuned on the resulting 30,000-sample dataset using LoRA with rank 64. Fourth, seven independently fine-tuned checkpoints are aggregated by majority voting, and this ensemble achieves the reported 85.5 percent test accuracy.

What would settle it

Run the pipeline again with checkpoint selection fixed before touching the test set: choose the seven variants using only the validation set, then evaluate the ensemble once on the SHROOM test set. If the resulting accuracy falls to 0.847 or below, the claimed 0.855 benchmark is an artifact of test-set-based selection rather than a property of the framework.

Watch

Extended reading notes

Core claim

The paper's central claim is that high-quality weak supervision, produced by prompt-optimized few-shot labeling, can replace large human-annotated datasets for hallucination detection. The authors show that DeepSeek-v3's weak-label accuracy rises from 73.6 percent with default settings to 82.4 percent when task-specific system instructions and an eight-shot prompt are added. After reformatting the weakly labeled data into system/user/assistant turns, fine-tuning Mistral-7B-Instruct-v0.3 with LoRA, and ensembling seven checkpoints by majority vote, the pipeline reaches 85.5 percent accuracy on the SHROOM model-agnostic test set. The authors present this as a new benchmark for the SHROOM task and as evidence that data restructuring, few-shot optimization, and ensembling can overcome data scarcity in resource-limited NLP systems.

Load-bearing premise

The 85.5 percent result relies on the assumption that the seven checkpoints used in the ensemble were chosen without using the test set; if some checkpoints were dropped because they scored lower on that same test set, the reported accuracy is optimistically biased.

Editorial extensions

If this is right

The same recipe of few-shot weak labeling, instruction-format data restructuring, LoRA fine-tuning, and majority-vote ensembling can be ported to other low-resource classification tasks.
A 7-billion-parameter model fine-tuned on weak labels can be competitive with larger or more specialized hallucination detectors, reducing both hardware and annotation costs.
The steady accuracy gains from prompt refinement (73.6 to 77.1 to 82.4 percent) suggest that iterative prompt engineering is a cheap way to raise weak-label quality before spending compute on fine-tuning.
Because the method is model-agnostic and relies only on generated text, it can be applied to closed large language models whose internal weights and logits are unavailable.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper does not compare DeepSeek-v3's weak labels against human labels on a sample; a small human audit of the 30,000 labels would show whether the framework's ceiling is set by label quality rather than by fine-tuning or ensembling.
The checkpoint-selection step is not fully specified in the paper; a fair re-analysis that fixes the ensemble before seeing the test set would establish whether the 0.855 score is robust, since the reported margin over the runner-up is only 0.008.
The data-restructuring trick likely transfers to other instruction-tuned classifiers, since converting labels into natural-language assistant answers is a cheap way to adapt classification tasks to generative models, but the paper only demonstrates it on hallucination detection.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things to know. First, this is a system description, not a methodological breakthrough: it combines prompt refinement, weak supervision, LoRA fine-tuning, and majority voting, all known pieces. Second, the headline 85.5% test accuracy on SHROOM's model-agnostic track is plausible but under-verified. The biggest gap is that Table 4 lists seven checkpoints out of what must have been at least nine (v4 and v7 are missing) with no explanation. The ensemble is exactly the kind of thing where dropping weak members can add points, so the omission is load-bearing.

What the paper does well: the validation progression (73.6 to 77.1 to 82.4) shows the prompt engineering actually works. The data restructuring into a generative format is clearly described and is a sensible practical move. The comparison table with other SHROOM systems gives useful context. The writing is plain, and the method is described well enough that another group could reimplement it without much pain.

Soft spots, in proportion. The v4/v7 omission is the real one. If those checkpoints were dropped for poor test-set performance, the 0.855 is partly a test-set artifact, and the "new benchmark" phrase oversells an 0.008 gain over Halu-NLP. If they were dropped for training instability or another a-priori reason, one sentence would clear it up. As written, it is a reporting hole. The math in Section 3.1 is decorative; it defines a probability with unknown weight matrices and then never uses it. Cut it. There are no confidence intervals or significance tests, and for a difference of 0.008 that matters. I would not call the framework "resource-limited" without qualification: LoRA on 30k samples with a 7B base is moderate, not tiny.

The citation pattern is fine. They cite the task paper and related shared-task systems. Nothing inflated or invented. The central result is likely real, but the paper needs verification steps before I would rely on the exact number.

Bottom line: this is a competent system description that would benefit from a cleanup pass. A serious referee should ask for an explanation of the checkpoint selection, removal of the decorative equations, and ideally release of the generated weak labels or code. I would engage with it, but I would insist on the fixes before treating the result as established.

Referee Report

4 major / 5 minor

Summary. The paper describes a hallucination-detection system for the SemEval-2024 SHROOM shared task. The authors use DeepSeek-v3 with 8-shot prompting and task-specific system instructions to generate weak labels for 30,000 unlabeled training examples, restructure the data into an instruction-following format, fine-tune Mistral-7B-Instruct-v0.3 with LoRA, and combine seven checkpoints via majority voting. The headline claim is 85.5% test accuracy on the model-agnostic track, which the paper says is a top ranking and a new SHROOM benchmark. Prompt-engineering choices are tuned on a validation set, and evaluation is on the human-annotated SHROOM test set.

Significance. If the 85.5% result is reproducible, the paper demonstrates a practical recipe for resource-limited hallucination detection: weak supervision from a strong LLM, data restructuring for generative models, and ensemble voting over LoRA checkpoints. The pipeline has a clean evaluation design in one respect: weak labels come from DeepSeek, fine-tuning is on those labels, and the final evaluation uses human-annotated SHROOM labels, so the main result is not circular. The prompt configurations are compared on a validation set, not the test set, which is appropriate. However, the paper's contribution is primarily empirical, and the evidence does not yet support the strength of the claims: the checkpoint-selection gap in Table 4 is a genuine load-bearing concern, and no code, checkpoints, confidence intervals, or official leaderboard citation are provided. The formal probability equations in Section 3.1 are not used in the experiments and function as decoration rather than a derivation.

major comments (4)

[§4.2, Table 4] Table 4 reports ensemble accuracy of 0.855 from seven checkpoints (v0, v1, v2, v3, v5, v6, v8) but omits v4 and v7 without any explanation. Because the naming pattern implies that at least nine checkpoints were trained, the exclusion criterion is load-bearing: if v4 and v7 were removed because of low test-set accuracy, the ensemble result is partly selected on the test set, and the phrase 'setting a new benchmark' in the abstract is inflated. Please report the total number of trained checkpoints, the selection rule, the accuracy of every checkpoint, and the ensemble accuracy both with and without the omitted checkpoints.
[§4.2, Table 4] The reported gain of the ensemble over the best single checkpoint is 0.855 versus 0.845, a difference of one percentage point, but no confidence intervals, bootstrap estimates, or significance tests are given, and the test-set size is not stated. Without this information, the ensemble's superiority over the single best checkpoint is not established. Please add uncertainty quantification and, if possible, McNemar's test on the paired predictions.
[§4.3, Table 5] The 'top ranking on the model-agnostic track' claim is supported only by a local table of ten systems with no citation to the official SHROOM results, no mention of the total number of participating systems, and no specification of how the ranking was queried. Please cite the official SemEval-2024 overview paper or leaderboard and report the exact metric and rank; otherwise the top-ranking claim is under-supported.
[§3.3, §4] No code, fine-tuned checkpoints, or final 8-shot prompt are released, and the paper does not specify the exact system instruction that achieved 82.4% weak-label accuracy. Since the central claim is empirical and the reported result depends on a particular set of LoRA checkpoints, the method is not reproducible as written. Please provide the final prompts, the checkpoint selection protocol, and at least the code or a detailed configuration file (LoRA alpha/dropout, sequence length, data split, number of epochs) in an appendix or repository.

minor comments (5)

[§3.1, Eqs. (1)-(3)] The probability equations introduce quantities f, W_initial, W_refined, and W_fewshot that are never defined in terms of DeepSeek-v3, and they are not used in Section 4 to derive any of the reported accuracies. Please either connect these equations to the experiments or remove them, because as written they give a formal appearance to what is actually prompt engineering.
[Table 3] The validation-set accuracies in Table 3 (73.6, 77.1, 82.4) are reported without the validation-set size, the number of runs, or any variance measure; please add these so readers can judge whether the improvements are stable.
[§3.2, Table 1] The transformation from the original SHROOM record to the reconstructed user prompt is not fully explained: the original 'hyp' and 'tgt' fields are clear, but the role of the 'ref' field and the choice of 'Context' text should be stated. A precise template would also help reproducibility.
[§3.2/§3.3] The paper says the weakly labeled dataset has 30,000 samples but does not report the class balance, the number of unique source examples, or how the data were split for fine-tuning and validation. Please add these dataset statistics.
[References] Several references are incomplete or inconsistently formatted (e.g., [7] lacks venue/publisher, [13] is missing a DOI/venue, and [17] has an invalid page range '717-724' with no DOI); please normalize the bibliography.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the empirical pipeline uses independent weak labels and human-annotated test labels; the checkpoint-selection gap is a transparency concern, not a demonstrated circular step.

full rationale

Walking the derivation chain: weak labels are produced by DeepSeek-v3 (Section 3.1, Table 3), the Mistral-7B-Instruct-v0.3 model is fine-tuned on those labels (Section 3.3), and evaluation is performed against the official SHROOM human-annotated test set (Section 4.3). The test labels are external to the weak-label generation and to the prompt-engineering loop, which is reported on the validation set (Table 3). The probability equations in Section 3.1 are definitions and notation rather than a derivation: expressions such as P(y|x) = softmax(W f(x,p,s)+b) and s* = argmax_s Eval(P(y|x,pfewshot,s), ytrue) do not constrain or entail the reported accuracies, so they cannot create circularity. The paper contains no self-citations that are load-bearing; the cited prior work consists of external systems and shared-task descriptions. The only substantive concern is the unexplained omission of checkpoints v4 and v7 from Table 4: if those checkpoints were dropped because of poor test-set performance, then the ensemble accuracy of 0.855 would be partly selected on the test set. However, the paper does not state any such selection criterion, and no quoted text or equation exhibits a reduction of the ensemble result to a fitted or test-set-derived quantity. Under the hard rule that circularity must be demonstrated by quotation and exhibited reduction, this remains a transparency and reproducibility risk, not a demonstrated circular step. The empirical claim is therefore self-contained with respect to its stated inputs, and the appropriate circularity score is 0.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

The framework's contribution is empirical, and it rests on unstated assumptions about weak label quality, data format transfer, and checkpoint selection. The free parameters are standard hyperparameters, but the checkpoint exclusion is an unexplained choice that directly affects the headline number.

free parameters (7)

Number of few-shot examples = 8
Chosen by the authors; no ablation across different shot counts is reported.
Task-specific system instruction text = Not fully disclosed
The instruction is iteratively refined, but the final text is not shown; selection is based on validation accuracy.
LoRA rank = 64
Fixed hyperparameter; no sensitivity analysis.
Training steps = 500
Fixed hyperparameter; no sensitivity analysis.
Batch size = 8
Fixed hyperparameter; no sensitivity analysis.
Learning rate = 2e-5
Fixed hyperparameter; no sensitivity analysis.
Ensemble checkpoint selection = 7 of 9 checkpoints
Only seven of nine fine-tuned checkpoints are included; v4 and v7 are omitted without explanation.

assumptions (5)

standard math Softmax with learned weights is a valid model for label probability
Used in Section 3.1 equations, but never fitted or used.
domain assumption DeepSeek-v3 weak labels are sufficiently accurate to train a downstream detector
Core to the weak supervision pipeline; only validated indirectly by final test accuracy.
domain assumption Reformulating classification data into a generative instruction format improves fine-tuning
Stated in Section 3.2; no controlled comparison with classification format.
domain assumption Majority voting over fine-tuned checkpoints improves accuracy
Ensemble result is slightly better than individual checkpoints, but the gain is within selection uncertainty.
domain assumption SHROOM validation and test sets are representative and correctly labeled
Relies on the shared task data; no independent verification.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Few-Shot Optimized Framework for Hallucination Detection in Resource-Limited NLP Systems." pith.science (2026). https://pith.science/paper/J4F7VA3Z

@misc{pith2026250116616,
  author       = {Pith},
  title        = {Pith review of: Few-Shot Optimized Framework for Hallucination Detection in Resource-Limited NLP Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/J4F7VA3Z}},
  note         = {Machine review of arXiv:2501.16616}
}

read the original abstract

Hallucination detection in text generation remains an ongoing struggle for natural language processing (NLP) systems, frequently resulting in unreliable outputs in applications such as machine translation and definition modeling. Existing methods struggle with data scarcity and the limitations of unlabeled datasets, as highlighted by the SHROOM shared task at SemEval-2024. In this work, we propose a novel framework to address these challenges, introducing DeepSeek Few-shot optimization to enhance weak label generation through iterative prompt engineering. We achieved high-quality annotations that considerably enhanced the performance of downstream models by restructuring data to align with instruct generative models. We further fine-tuned the Mistral-7B-Instruct-v0.3 model on these optimized annotations, enabling it to accurately detect hallucinations in resource-limited settings. Combining this fine-tuned model with ensemble learning strategies, our approach achieved 85.5% accuracy on the test set, setting a new benchmark for the SHROOM task. This study demonstrates the effectiveness of data restructuring, few-shot optimization, and fine-tuning in building scalable and robust hallucination detection frameworks for resource-constrained NLP systems.

Figures

Figures reproduced from arXiv: 2501.16616 by the authors.

**Figure 2.** shows the original dataset, and [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

MSA at SemEval-2025 Task 3: High Quality Weak Labeling and LLM Ensemble Verification for Multilingual Hallucination Detection
cs.CL 2025-05 reject novelty 4.0 of 10

An LLM ensemble that extracts hallucinated spans and votes on them, followed by fuzzy matching, achieved top ranks in Arabic and Basque at the Mu-SHROOM shared task.

Reference graph

Works this paper leans on

20 extracted references · 17 canonical work pages · cited by 1 Pith paper

[1]

arXiv preprint arXiv:2412.19437 (2024), https://arxiv.org/abs/2412.19437

AI@DeepSeek: Deepseek-v3: A versatile and high-perform ance language model. arXiv preprint arXiv:2412.19437 (2024), https://arxiv.org/abs/2412.19437

arXiv 2024
[2]

In: Proceedings of the 18th International Workshop on Semantic Evaluation ( SemEval-2024)

Allen, B.P., Polat, F., Groth, P.: SHROOM-INDElab at SemE val-2024 task 6: Zero- and few-shot LLM-based classiﬁcation for hallucination de tection. In: Proceedings of the 18th International Workshop on Semantic Evaluation ( SemEval-2024). As- sociation for Computational Linguistics (2024)

work page 2024
[3]

In: Proceedings of the 18th International Worksho p on Semantic Evalu- ation (SemEval-2024)

Arzt, V., Azarbeik, M.M., Lasy, I., Kerl, T., Recski, G.: T U wien at SemEval-2024 task 6: Unifying model-agnostic and model-aware technique s for hallucination de- tection. In: Proceedings of the 18th International Worksho p on Semantic Evalu- ation (SemEval-2024). pp. 1172–1186. Association for Comp utational Linguistics (2024)

work page 2024
[4]

In: Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024)

Borra, F., Savelli, C., Rosso, G., Koudounas, A., Gioberg ia, F.: MALTO at SemEval-2024 task 6: Leveraging synthetic data for LLM hall ucination detec- tion. In: Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024). pp. 1793–1808. Association for Computati onal Linguistics (2024)

work page 2024
[5]

In: Proceedings o f the 18th Interna- tional Workshop on Semantic Evaluation (SemEval-2024)

Brodoceanu, O.: Octavianb at semeval-2024 task 6: An expl oration of human- like qualities in hallucinated llm texts. In: Proceedings o f the 18th Interna- tional Workshop on Semantic Evaluation (SemEval-2024). pp . 1162–1165 (2024), https://aclanthology.org/2024.semeval-1.169/

work page 2024
[6]

In: Proceedings of t he 61st Annual Meet- ing of the Association for Computational Linguistics

Dale, D., Voita, E., Barrault, L., Costa-jussà, M.R.: Det ecting and mitigat- ing hallucinations in machine translation: Model internal workings alone do well, sentence similarity even better. In: Proceedings of t he 61st Annual Meet- ing of the Association for Computational Linguistics. vol. 1, pp. 36–50. As- sociation for Computational Linguistics, Toro...

work page 2023
[7]

In : Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2 024)

Fallah, P., Gooran, S., Jafarinasab, M., Sadeghi, P., Far nia, R., Tarabkhah, A., Taghavi, Z.S., Sameti, H.: SLPL SHROOM at SemEval2024 task 0 6: A compre- hensive study on models ability to detect hallucination. In : Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2 024). pp. 1137–1143. Association for Computational Linguis...

work page 2024
[8]

In: Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024)

Griogoriadou, N., Lymperaiou, M., Filandrianos, G., Sta mou, G.: Ails-ntua at semeval-2024 task 6: Eﬃcient model tuning for hallucinatio n detection and anal- ysis. In: Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024). pp. 1551–1557 (2024) 10 B. Hikal

work page 2024

Show all 20 references

[9]

In: Proceedings of the 17th Conference of the European Chapter of the Associati on for Computational Linguistics

Guerreiro, N.M., Voita, E., Martins, A.: Looking for a nee dle in a haystack: A com- prehensive study of hallucinations in neural machine trans lation. In: Proceedings of the 17th Conference of the European Chapter of the Associati on for Computational Linguistics. pp. 1059–10...

2023
[10]

In: Proceedings of the 2023 Conference on Empirica l Methods in Natural Language Processing

Lei, D., Li, Y., Hu, M.M., Wang, M., Yun, V., Ching, E., Kam al, E.: Chain of natural language inference for reducing large language mod el ungrounded halluci- nations. In: Proceedings of the 2023 Conference on Empirica l Methods in Natural Language Processing. pp. 3445–3454 (2023)

2023
[11]

arXiv preprint arXiv:2311.05556 (2023)

Liu, F., Lin, K., Li, L., Wang, J., Yacoob, Y., Wang, L.: MI TIGATING HALLUCI- NATION IN LARGE MULTIMODAL MODELS VIA ROBUST INSTRUCTION TUNING. arXiv preprint arXiv:2311.05556 (2023)

2023 arXiv
[12]

In: Proceed- ings of the 18th International Workshop on Semantic Evaluat ion (SemEval-2024)

Liu, W., Shi, W., Zhang, Z., Huang, H.: HIT-MI&T lab at Sem Eval-2024 task 6: DeBERTa-based entailment model is a reliable hallucinatio n detector. In: Proceed- ings of the 18th International Workshop on Semantic Evaluat ion (SemEval-2024). Association for Computational Lingui...

2024
[13]

arXiv preprint arXiv:2303.08896 (2023)

Manakul, P., Goyal, M., Chowdhury, A., Raghunathan, A., Bhardwaj, A.: Self- checkgpt: Zero-resource detection of machine-generated t ext. arXiv preprint arXiv:2303.08896 (2023)

2023 arXiv
[14]

In: Proceedings of the 18th Inte rnational Workshop on Semantic Evaluation (SemEval-2024)

Mehta, R., Hoblitzell, A., O’Keefe, J., Jang, H., Varma, V.: Halu-NLP at SemEval- 2024 task 6: MetaCheckGPT - a multi-task hallucination dete ction using LLM un- certainty and meta-models. In: Proceedings of the 18th Inte rnational Workshop on Semantic Evaluation (SemEval-2024...

2024
[15]

In: Proceedings of the 18th International Workshop on Semantic Evaluation ( SemEval-2024)

Mickus, T., Zosa, E., Vázquez, R., Vahtola, T., Tiedeman n, J., Segonne, V., Ra- ganato, A., Apidianaki, M.: SemEval-2024 shared task 6: SHR OOM, a shared-task on hallucinations and related observable overgeneration m istakes. In: Proceedings of the 18th International Workshop...

2024
[16]

In: Proceedings of the 61st Annual Meeting of the Assoc iation for Computa- tional Linguistics (Volume 1: Long Papers)

Müller, M., Rios, A., Sennrich, R.: Domain robustness in neural machine transla- tion. In: Proceedings of the 61st Annual Meeting of the Assoc iation for Computa- tional Linguistics (Volume 1: Long Papers). pp. 47–65. Asso ciation for Computa- tional Linguistics, Toronto, Cana...

2023 arXiv
[17]

In: Proceedi ngs of the 18th Interna- tional Workshop on Semantic Evaluation (SemEval-2024)

Pan, R., García-Díaz, J.A., Bernal-Beltrán, T., Valenc ia-García, R.: Umuteam at semeval-2024 task 6: Leveraging zero-shot learning for det ecting hallucinations and related observable overgeneration mistakes. In: Proceedi ngs of the 18th Interna- tional Workshop on Semantic E...

2024
[18]

In: Proceedings of the 18th International Works hop on Semantic Eval- uation (SemEval-2024)

Rykov, E.S., Shishkina, Y., Petrushina, K., Titova, K., Petrakov, S., Panchenko, A.: Smurfcat at semeval-2024 task 6: Leveraging synthetic d ata for hallucination detection. In: Proceedings of the 18th International Works hop on Semantic Eval- uation (SemEval-2024). pp. 869–88...

2024
[19]

In: Proceedings of the 18th International Workshop on Semantic Evaluation ( SemEval-2024)

Sanayei, R., Singh, A., Rezaei, M., Bethard, S.: Maria at semeval 2024 task-6: Hallucination detection through llms, mnli, and cosine sim ilarity. In: Proceedings of the 18th International Workshop on Semantic Evaluation ( SemEval-2024). pp. 1583–1588 (2024), https://aclanthol...

2024
[20]

In: Pro- ceedings of the 18th International Workshop on Semantic Eva luation (SemEval- 2024)

Wei, C., Chen, Z., Fang, S., He, J., Gao, M.: Opdai at semev al-2024 task 6: Small llms can accelerate hallucination detection with weakly su pervised data. In: Pro- ceedings of the 18th International Workshop on Semantic Eva luation (SemEval- 2024). pp. 720–729 (2024), https:...

2024

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

arXiv preprint arXiv:2412.19437 (2024), https://arxiv.org/abs/2412.19437

AI@DeepSeek: Deepseek-v3: A versatile and high-perform ance language model. arXiv preprint arXiv:2412.19437 (2024), https://arxiv.org/abs/2412.19437

arXiv 2024

[2] [2]

In: Proceedings of the 18th International Workshop on Semantic Evaluation ( SemEval-2024)

Allen, B.P., Polat, F., Groth, P.: SHROOM-INDElab at SemE val-2024 task 6: Zero- and few-shot LLM-based classiﬁcation for hallucination de tection. In: Proceedings of the 18th International Workshop on Semantic Evaluation ( SemEval-2024). As- sociation for Computational Linguistics (2024)

work page 2024

[3] [3]

In: Proceedings of the 18th International Worksho p on Semantic Evalu- ation (SemEval-2024)

Arzt, V., Azarbeik, M.M., Lasy, I., Kerl, T., Recski, G.: T U wien at SemEval-2024 task 6: Unifying model-agnostic and model-aware technique s for hallucination de- tection. In: Proceedings of the 18th International Worksho p on Semantic Evalu- ation (SemEval-2024). pp. 1172–1186. Association for Comp utational Linguistics (2024)

work page 2024

[4] [4]

In: Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024)

Borra, F., Savelli, C., Rosso, G., Koudounas, A., Gioberg ia, F.: MALTO at SemEval-2024 task 6: Leveraging synthetic data for LLM hall ucination detec- tion. In: Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024). pp. 1793–1808. Association for Computati onal Linguistics (2024)

work page 2024

[5] [5]

In: Proceedings o f the 18th Interna- tional Workshop on Semantic Evaluation (SemEval-2024)

Brodoceanu, O.: Octavianb at semeval-2024 task 6: An expl oration of human- like qualities in hallucinated llm texts. In: Proceedings o f the 18th Interna- tional Workshop on Semantic Evaluation (SemEval-2024). pp . 1162–1165 (2024), https://aclanthology.org/2024.semeval-1.169/

work page 2024

[6] [6]

In: Proceedings of t he 61st Annual Meet- ing of the Association for Computational Linguistics

Dale, D., Voita, E., Barrault, L., Costa-jussà, M.R.: Det ecting and mitigat- ing hallucinations in machine translation: Model internal workings alone do well, sentence similarity even better. In: Proceedings of t he 61st Annual Meet- ing of the Association for Computational Linguistics. vol. 1, pp. 36–50. As- sociation for Computational Linguistics, Toro...

work page 2023

[7] [7]

In : Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2 024)

Fallah, P., Gooran, S., Jafarinasab, M., Sadeghi, P., Far nia, R., Tarabkhah, A., Taghavi, Z.S., Sameti, H.: SLPL SHROOM at SemEval2024 task 0 6: A compre- hensive study on models ability to detect hallucination. In : Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2 024). pp. 1137–1143. Association for Computational Linguis...

work page 2024

[8] [8]

In: Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024)

Griogoriadou, N., Lymperaiou, M., Filandrianos, G., Sta mou, G.: Ails-ntua at semeval-2024 task 6: Eﬃcient model tuning for hallucinatio n detection and anal- ysis. In: Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024). pp. 1551–1557 (2024) 10 B. Hikal

work page 2024