Pith. sign in

REVIEW 5 major objections 5 minor 35 references

Data-efficient Meta-models for Evaluation of Context-based Questions and Answers in LLMs

T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that a two-stage pipeline of internal-state features plus a small tabular classifier can match strong proprietary LLM judges on hallucination detection for RAG question answering using as few as 250 labeled examples.

desk verdict A useful low-data hallucination-detection paper whose abstract overstates 250-sample parity with RAGAS; the reported tables never isolate that operating point. read the letter →

arxiv 2505.23299 v1 pith:7NQ3DBOS submitted 2025-05-29 cs.CL

Julia Belikova , Konstantin Polev , Rauf Parchiev , Dmitry Simakov This is my paper
classification cs.CL
keywords hallucinationdetectionretrieval-augmentedgenerationquestionansweringdataefficiencymodelprobingLookbackLensTabPFNmeta-classification
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish that hallucination detection in retrieval-augmented question answering can be done accurately with very little labeled data. It shows a two-stage pipeline: extract internal activations from a small open LLM, compress them, and feed them to a lightweight tabular classifier. With 250 training samples, the best configuration reaches ROC-AUC scores comparable to a GPT-4o-based RAGAS judge and beats zero-shot GPT-4o. The point is that annotation-constrained industrial deployments could build local, private detectors without costly proprietary APIs. The paper also positions TabPFNv2, a tabular foundation model for small data, as the strongest classifier in this setting.

What carries the argument

Two complementary feature streams are extracted from a frozen extractor LLM. The first pools per-token hidden states using mean, max, and last-token aggregation from middle layers and compresses them with PCA or UMAP to 30 components. The second adapts Lookback Lens by computing, for each layer and attention head, the mean ratio of attention weight given to context tokens versus newly generated tokens, again compressed to 30 components or restricted to selected middle layers. These features are concatenated and fed to TabPFNv2, a tabular foundation model that performs in-context learning on small datasets, with logistic regression and CatBoost as alternatives. PCA on lookback features and TabPFNv2's handling of few examples are the two components that carry most of the reported gains.

What would settle it

Run the proposed TabPFN pipeline with the original span-based Lookback Lens features instead of mean lookback ratios on EManual, ExpertQA, and RAGTruth at 250 training samples. If span-based features produce substantially higher ROC-AUC, the mean-ratio adaptation erases signal and the data-efficiency claim is specific to this simplification rather than to the original method.

Watch

Extended reading notes

Core claim

The central claim is that data-efficient meta-classifiers over internal state features can close most of the gap to proprietary LLM judges. Combining lookback-ratio features from attention with pooled hidden states, reduced to 30 PCA components, and classifying with TabPFNv2 gives average ROC-AUC around 0.78 across the three QA benchmarks, with the best configuration reaching 0.82 on ExpertQA. This matches or approaches RAGAS GPT-4o, which scores 0.81 to 0.84, while using only 250 training samples and open-source 7 to 9 billion parameter extractor LLMs. The authors frame this as evidence that the bottleneck to practical hallucination detection is not model capacity but the interface between internal signals and small-data classifiers.

Load-bearing premise

The load-bearing premise is that reducing attention lookback dynamics to a single mean ratio per layer and head preserves enough hallucination signal for the classifier, since the paper does not compare against the original span-level Lookback Lens.

Editorial extensions

If this is right

  • Teams with only a few hundred annotated QA pairs can train a local hallucination detector instead of calling a proprietary judge for every generation.
  • Small open-weight extractor models in the 7 to 9 billion parameter range are sufficient, removing the need to access the generator's hidden states or send sensitive data to external APIs.
  • PCA-compressed lookback features are more useful than raw or UMAP-compressed features in most tested configurations.
  • TabPFNv2 consistently outperforms logistic regression, CatBoost, and attention-pooling probes under low-data conditions.
  • Performance improves sharply from 50 to 250 training samples and then plateaus, making 250 samples a practical operating point for deployment.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: a direct ablation that concatenates hidden-state and lookback features into one classifier input may yield further gains, since the paper evaluates the two feature streams in separate configurations rather than jointly.
  • Beyond the paper: because the mean lookback ratio discards per-span information, restoring span-level lookback features inside the tabular pipeline could push data efficiency higher if the hallucination signal is concentrated in specific token spans.
  • Beyond the paper: the 250-sample result likely transfers best to tasks similar to the three benchmarks tested; transfer to multi-hop or highly specialized domains remains untested.
  • Beyond the paper: the reported MRR results suggest these detectors could also rank candidate answers, not just flag individual generations, which would be a cheap reranking extension for RAG pipelines.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes a two-stage pipeline for contextual hallucination detection in RAG-based question answering: extract internal-state features from a smaller open LLM (pooled hidden states and a mean lookback ratio derived from attention maps), reduce their dimensionality with PCA or UMAP, and classify with a lightweight meta-classifier (logistic regression, CatBoost, or TabPFNv2). The method is evaluated on EManual, ExpertQA, and RAGTruth under training-set sizes from 50 to 1000 samples, with GPT-4o zero-shot judging and the RAGAS GPT-4o pipeline as strong proprietary baselines. The central claim is that this approach achieves performance comparable to those proprietary baselines with only 250 labeled training samples.

Significance. If the central claim were fully supported, this would be a useful result for annotation-constrained industrial RAG deployments: a local, lightweight detector that matches proprietary LLM judges at 250 samples would reduce both annotation cost and privacy exposure. The paper has real strengths: it uses public benchmarks, three open extractor models, and a genuine train/test split with reported ROC-AUC and MRR numbers, and the out-of-sample evaluation design is sound. The exploration of TabPFNv2 for this task is also a reasonable novelty. However, the headline parity claim is not supported by the tables as printed, and several methodological details needed for reproducibility and for interpreting the data-efficiency results are missing.

major comments (5)
  1. [Abstract, §4.2 (Tables 5 and 6)] The claim that the approach 'achieves performance comparable to strong proprietary LLM-based baselines with only 250 training samples' is not supported by the reported point estimates. Table 5, which does not state the training-set size, shows that across the lookback+TabPFN variants the best EManual score is 0.6972 versus RAGAS GPT-4o's 0.8208, and the best RAGTruth score is 0.8037 versus 0.8386; only ExpertQA reaches parity (0.8165 versus 0.8160). Table 6's 250-sample row, which also does not identify the feature/classifier/extractor configuration, reports 0.5903, 0.7648, and 0.7368 for EManual, ExpertQA, and RAGTruth, respectively, all below the RAGAS baselines. As printed, the data-efficiency paragraph and the abstract overstate the result; the paper should present the specific configuration at the 250-sample operating point that attains parity and state the gap threshold used for 'comparable.'
  2. [§3 Feature extraction (2)] The adaptation of Lookback Lens to a single mean lookback ratio per layer and head, averaged over the whole sequence, is not validated against the original span-based method. If the hallucination signal in the original method resides in per-token-span attention dynamics, the mean collapse could erase that signal, and the reported data efficiency would not transfer to the original method. Please include an ablation comparing the mean-collapsed features with span-based lookback features on at least one dataset/extractor pair, or provide evidence that the mean preserves the relevant signal.
  3. [§4.1 Main methods] The layer-range selection for raw lookback features is underspecified: the sentence 'specific layer ranges (Qwen: 5–21, Llama: 8–22, Gemma: 5–35) are selected to approximate this 500-feature ceiling' does not explain how these ranges were chosen or whether the choice involved the test labels. If the ranges were selected based on test-set performance, the reported ROC-AUC values are optimistic and the method is not fully specified. Please state the selection criterion and report sensitivity to range boundaries.
  4. [§4.1 Training and Evaluation] Only three random seeds are used, and Figure 1's shaded intervals are labeled as ±95% confidence intervals over those three seeds. With n=3 these intervals are very wide, and the point estimates at the 250-sample operating point cannot reliably support a parity claim against fixed LLM-judge baselines. Please report per-seed values or increase the number of seeds, and state whether the same default test split is used in every run.
  5. [§3 Feature extraction (1), §4.1] The manuscript does not state whether the PCA/UMAP dimensionality-reduction transformations are fitted on the training subset only or on the full dataset before splitting. If they are fitted on the full dataset, the unsupervised transforms can leak test-set distributional information and inflate the data-efficiency gains. Please specify the fitting protocol and, if necessary, re-run the experiments with the reductions fitted only on the training folds.
minor comments (5)
  1. [§4.1, Figure 1] The figure caption refers to 'three response generators' but the body text never names the generator models; clarify whether they are the same as the three extractor models and how the 'independent extractor' setting is realized.
  2. [§4.1 Main methods] Specify the exact transformer layer index used as the 'middle layer' for each extractor model, since the current text only cites empirical findings from [5].
  3. [§4.2, Table 3] The MRR numbers in Table 3 are not accompanied by a definition of how mean reciprocal rank is computed for binary hallucination labels; add the ranking procedure used.
  4. [§3] The term 'meta-classification' is used without definition; clarify that the classifiers operate on extracted features rather than on raw text.
  5. [Table 5] Table 5 contains inconsistent spacing in several decimal numbers (e.g., '0 .8064', '0 .8122'); fix the typography.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the pipeline is a standard supervised evaluation on held-out splits, and no fitted quantity or self-citation defines the target labels.

full rationale

The paper's contribution is empirical: it extracts internal-state features (pooled hidden states and mean lookback ratios), trains lightweight classifiers on labeled subsets, and reports ROC-AUC and MRR on default test splits. There is no step in which a fitted parameter is renamed as a prediction or in which the target is defined in terms of the features. The middle-layer choice is inherited from the external Lookback Lens work ('based on empirical findings from [5]'), not from the authors' own prior claims, so no self-citation chain is load-bearing. The mean-lookback adaptation is an unvalidated design choice, not a circular one: the hallucination labels come from human annotation (RAGTruth) or GPT-4 annotation (RAGBench), independent of the attention-ratio features. Classifiers (TabPFNv2, logistic regression, CatBoost) are trained on 50-1000 sample subsets with a 20% validation holdout and tested on the default splits, so the reported numbers are genuine out-of-sample estimates. The abstract's 'comparable ... with only 250 training samples' claim is not well supported by Tables 5 and 6 (e.g., EManual at 250 samples is 0.5903 vs RAGAS GPT-4o's 0.8208), but this is an overstatement relative to the reported point estimates, not a circularity. No circularity score above 0 is warranted.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new entities. Its central claim rests on three hand-chosen hyperparameters (dimensionality target, layer ranges, middle layer) and on four domain assumptions inherited from prior work or introduced without direct validation. The most fragile is the mean-lookback-ratio adaptation, which is a novel modification of Lookback Lens that is not benchmarked against the original.

free parameters (3)
  • Dimensionality reduction target size = 30 components per feature type
    Chosen to keep total features under TabPFNv2's 500-feature limit; no sensitivity analysis reported (Section 4.1).
  • Extractor layer ranges for lookback features = Qwen 5-21, Llama 8-22, Gemma 5-35
    Selected to approximate the 500-feature ceiling; the selection procedure is not reported, so test-set tuning cannot be ruled out (Section 4.1, Main methods).
  • Hidden-state middle layer = unspecified
    The paper uses middle layer hidden states citing prior work but does not state which layer; this choice is load-bearing for the hidden-state features (Section 3, feature extraction (1)).
assumptions (4)
  • domain assumption Extractor LLM internal states (hidden states and attention) contain information about contextual hallucination
    The whole feature extraction rests on this, inherited from Lookback Lens [5] and probing literature [28].
  • ad hoc to paper A single mean lookback ratio per layer and head, averaged over the whole sequence, preserves the hallucination signal of the original token-span Lookback Lens
    Introduced in Section 3 feature extraction (2) with no validation against the original span-based computation.
  • domain assumption PCA and UMAP compression to 30 components retains hallucination-relevant variance
    Standard assumption for dimensionality reduction, but no reconstruction error or sensitivity analysis is provided.
  • domain assumption RAGBench's GPT-4-generated labels and RAGTruth's human labels are valid ground truth
    The evaluation treats these benchmark labels as correct without analysis of label noise or annotator agreement.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Data-efficient Meta-models for Evaluation of Context-based Questions and Answers in LLMs." pith.science (2026). https://pith.science/paper/7NQ3DBOS

@misc{pith2026250523299,
  author       = {Pith},
  title        = {Pith review of: Data-efficient Meta-models for Evaluation of Context-based Questions and Answers in LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7NQ3DBOS}},
  note         = {Machine review of arXiv:2505.23299}
}
read the original abstract

Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) systems are increasingly deployed in industry applications, yet their reliability remains hampered by challenges in detecting hallucinations. While supervised state-of-the-art (SOTA) methods that leverage LLM hidden states -- such as activation tracing and representation analysis -- show promise, their dependence on extensively annotated datasets limits scalability in real-world applications. This paper addresses the critical bottleneck of data annotation by investigating the feasibility of reducing training data requirements for two SOTA hallucination detection frameworks: Lookback Lens, which analyzes attention head dynamics, and probing-based approaches, which decode internal model representations. We propose a methodology combining efficient classification algorithms with dimensionality reduction techniques to minimize sample size demands while maintaining competitive performance. Evaluations on standardized question-answering RAG benchmarks show that our approach achieves performance comparable to strong proprietary LLM-based baselines with only 250 training samples. These results highlight the potential of lightweight, data-efficient paradigms for industrial deployment, particularly in annotation-constrained scenarios.

Figures

Figures reproduced from arXiv: 2505.23299 by the authors.

Figure 1
Figure 1. Test ROC-AUC versus training-set size for the pro￾posed evaluators (solid lines) across the three benchmarks (rows) and three response generators (columns). Horizontal dashed lines correspond to the zero-shot GPT-4o judge (yel￾low) and the RAGAS GPT-4o pipeline (cyan). Shaded areas indicate ±95% confidence intervals over three random seeds. Method EManual ExpertQA RAGTruth PCA + lookback + tabpfn 0.6972 0.8165 0.767… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

35 extracted references · 15 canonical work pages

  1. [1]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Floren- cia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. GPT-4 technical report. arXiv preprint arXiv:2303.08774 (2023)

    arXiv 2023

  2. [2]

    Ingeol Baek, Hwan Chang, ByeongJeong Kim, Jimin Lee, and Hwanhee Lee. 2025. Probing-RAG: Self-Probing to Guide Language Models in Selective Document Retrieval. InFindings of the Association for Computational Linguistics: NAACL 2025, Luis Chiruzzo, Alan Ritter, and Lu Wang (Eds.). Association for Computational Linguistics, Albuquerque, New Mexico, 3287–330...

    work page 2025

  3. [3]

    Masha Belyi, Robert Friel, Shuai Shao, and Atindriyo Sanyal. 2025. Luna: A Lightweight Evaluation Model to Catch Language Model Hallucinations with High Accuracy and Low Cost. In Proceedings of the 31st International Conference on Computational Linguistics: Industry Track . 398–409

    work page 2025

  4. [4]

    Yuemin Chen, Feifan Wu, Jingwei Wang, Hao Qian, Ziqi Liu, Zhiqiang Zhang, Jun Zhou, and Meng Wang. 2024. Knowledge-augmented Financial Market Analysis and Report Generation. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track . 1207–1217

    2024

  5. [5]

    Yung-Sung Chuang, Linlu Qiu, Cheng-Yu Hsieh, Ranjay Krishna, Yoon Kim, and James Glass. 2024. Lookback Lens: Detecting and Mitigating Contextual Halluci- nations in Large Language Models Using Only Attention Maps. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing . 1419–1436

    work page 2024

  6. [6]

    Jinhao Duan, Hao Cheng, Shiqi Wang, Alex Zavalny, Chenan Wang, Renjing Xu, Bhavya Kailkhura, and Kaidi Xu. 2024. Shifting Attention to Relevance: Towards the Predictive Uncertainty Quantification of Free-Form Large Language Models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , Lun-Wei ...

    doi:10.18653/v1/2024.acl 2024

  7. [7]

    Shahul Es, Jithin James, Luis Espinosa Anke, and Steven Schockaert. 2024. Ragas: Automated evaluation of retrieval augmented generation. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations. 150–158

    2024

  8. [8]

    Marina Fomicheva, Shuo Sun, Lisa Yankovskaya, Frédéric Blain, Francisco Guzmán, Mark Fishel, Nikolaos Aletras, Vishrav Chaudhary, and Lucia Spe- cia. 2020. Unsupervised Quality Estimation for Neural Machine Translation. Transactions of the Association for Computational Linguistics 8 (2020), 539–555. doi:10.1162/tacl_a_00330

    doi:10.1162/tacl_a_00330 2020

Show all 35 references
  1. [9]

    Robert Friel, Masha Belyi, and Atindriyo Sanyal. 2024. Ragbench: Explain- able benchmark for retrieval-augmented generation systems. arXiv preprint arXiv:2407.11005 (2024)

    2024 arXiv

  2. [10]

    Giacomo Frisoni, Miki Mizutani, Gianluca Moro, and Lorenzo Valgimigli. 2022. BioReader: a Retrieval-Enhanced Text-to-Text Transformer for Biomedical Lit- erature. In Proceedings of the 2022 conference on empirical methods in natural language processing. 5770–5793

    2022

  3. [11]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783 (2024)

    2024 arXiv

  4. [12]

    Noah Hollmann, Samuel Müller, Katharina Eggensperger, and Frank Hutter. 2022. Tabpfn: A transformer that solves small tabular classification problems in a second. arXiv preprint arXiv:2207.01848 (2022)

    2022 arXiv

  5. [13]

    Noah Hollmann, Samuel Müller, Lennart Purucker, Arjun Krishnakumar, Max Körfer, Shi Bin Hoo, Robin Tibor Schirrmeister, and Frank Hutter. 2025. Accurate predictions on small data with a tabular foundation model. Nature 637, 8045 (2025), 319–326

    2025

  6. [14]

    Yuheng Huang, Jiayang Song, Zhijie Wang, Shengming Zhao, Huaming Chen, Felix Juefei-Xu, and Lei Ma. 2025. Look Before You Leap: An Exploratory Study of Uncertainty Analysis for Large Language Models. IEEE Transactions on Software Engineering 51, 2 (2025), 413–429. doi:10.1109/...

    2025

  7. [15]

    Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of hallucination in natural language generation. Comput. Surveys 55, 12 (2023), 1–38

    2023

  8. [16]

    Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. 2023. Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Genera- tion. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023 . Open...

    2023

  9. [17]

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Cheng- gang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. 2024. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437 (2024)

    2024 arXiv

  10. [18]

    Chaitanya Malaviya, Subin Lee, Sihao Chen, Elizabeth Sieber, Mark Yatskar, and Dan Roth. 2023. Expertqa: Expert-curated questions and attributed answers. arXiv preprint arXiv:2309.07852 (2023)

    2023 arXiv

  11. [19]

    Potsawee Manakul, Adian Liusie, and Mark J. F. Gales. 2023. SelfCheckGPT: Zero- Resource Black-Box Hallucination Detection for Generative Large Language Models. arXiv:2303.08896 [cs.CL] https://arxiv.org/abs/2303.08896

    2023 arXiv

  12. [20]

    Aakanksha Naik, Sravanthi Parasa, Sergey Feldman, Lucy Lu Wang, and Tom Hope. 2022. Literature-Augmented Clinical Outcome Prediction. In Findings of the Association for Computational Linguistics: NAACL 2022 . 438–453

    2022

  13. [21]

    Abhilash Nandy, Soumya Sharma, Shubham Maddhashiya, Kapil Sachdeva, Pawan Goyal, and Niloy Ganguly. 2021. Question answering over electronic de- vices: A new benchmark dataset and a multi-task learning based QA framework. arXiv preprint arXiv:2109.05897 (2021)

    2021 arXiv

  14. [22]

    Cheng Niu, Yuanhao Wu, Juno Zhu, Siliang Xu, Kashun Shum, Randy Zhong, Juntong Song, and Tong Zhang. 2023. Ragtruth: A hallucination corpus for developing trustworthy retrieval-augmented language models. arXiv preprint arXiv:2401.00396 (2023)

    2023 arXiv

  15. [23]

    Hadas Orgad, Michael Toker, Zorik Gekhman, Roi Reichart, Idan Szpektor, Hadas Kotek, and Yonatan Belinkov. 2024. LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinations. arXiv:2410.02707 [cs.CL] https://arxiv.org/abs/2410.02707

    2024 arXiv

  16. [24]

    Liudmila Prokhorenkova, Gleb Gusev, Aleksandr Vorobev, Anna Veronika Doro- gush, and Andrey Gulin. 2018. CatBoost: unbiased boosting with categorical features. Advances in neural information processing systems 31 (2018)

    2018

  17. [25]

    Jon Saad-Falcon, Omar Khattab, Christopher Potts, and Matei Zaharia. 2024. ARES: An Automated Evaluation Framework for Retrieval-Augmented Genera- tion Systems. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics...

    2024

  18. [26]

    Pranab Sahoo, Prabhash Meharia, Akash Ghosh, Sriparna Saha, Vinija Jain, and Aman Chadha. 2024. A Comprehensive Survey of Hallucination in Large Lan- guage, Image, Video and Audio Foundation Models. In Findings of the Association for Computational Linguistics: EMNLP 2024 . 11709–11724

    2024

  19. [27]

    Bhaskarjit Sarmah, Dhagash Mehta, Benika Hall, Rohan Rao, Sunil Patel, and Stefano Pasquali. 2024. Hybridrag: Integrating knowledge graphs and vector retrieval augmented generation for efficient information extraction. InProceedings of the 5th ACM International Conference on A...

    2024

  20. [28]

    CH-Wang Sky, Benjamin Van Durme, Jason Eisner, and Chris Kedzie. 2024. Do Androids Know They’re Only Dreaming of Electric Sheep?. In Findings of the Association for Computational Linguistics ACL 2024 . 4401–4420

    2024

  21. [29]

    Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, et al. 2024. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118 (2024)

    2024 arXiv

  22. [30]

    Gabriel Tjio, Ping Liu, Joey Tianyi Zhou, and Rick Siow Mong Goh. 2022. Adver- sarial semantic hallucination for domain generalized semantic segmentation. In Proceedings of the IEEE/CVF winter conference on applications of computer vision . 318–327

    2022

  23. [31]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017)

    2017

  24. [32]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Me...

    2024 arXiv

  25. [33]

    Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, et al . 2023. Siren’s song in the AI ocean: a survey on hallucination in large language models. arXiv preprint arXiv:2309.01219 (2023)

    2023 arXiv

  26. [34]

    Yiyun Zhao, Prateek Singh, Hanoz Bhathena, Bernardo Ramos, Aviral Joshi, Swaroop Gadiyaram, and Saket Sharma. 2024. Optimizing LLM based retrieval augmented generation pipelines in the financial domain. InProceedings of the 2024 Conference of the North American Chapter of the ...

    2024

  27. [35]

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems 36 (2023), 46595–46623

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.