Pith. sign in

REVIEW 3 major objections 5 minor 49 references

IRIS detects LLM hallucinations without labeled data by training a lightweight probe on the model's own verification reasoning, using its verbalized confidence as a soft truth label.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 18:19 UTC pith:NNWVVLLO

load-bearing objection IRIS is a genuinely new recipe for unsupervised hallucination detection—verification-response embeddings with verbalized-confidence pseudolabels—and it mostly works, but the MIND comparison and the degenerate-pseudolabel robustness result need strengthening before the claims fully land. the 3 major comments →

arxiv 2509.10004 v1 pith:NNWVVLLO submitted 2025-09-12 cs.CL cs.AI

Unsupervised Hallucination Detection by Inspecting Reasoning Processes

Ponhvoan Srey , Xiaobao Wu , Anh Tuan Luu This is my paper
classification cs.CL cs.AI
keywords hallucination detectionunsupervised learninginternal representationsverbalized confidencesoft pseudolabelchain-of-thought reasoningprobe trainingfactuality
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

IRIS is an attempt to show that an LLM's internal states, recorded while it reasons about whether a claim is true, carry enough truth-related signal to power a hallucination detector without any human labels. The method asks the model to verify a statement step by step, takes the last hidden state of that reasoning, and uses the model's verbalized confidence as a soft label for correctness. A compact probe is trained on these embeddings and outperforms earlier unsupervised methods on three benchmarks, with gains of 3.2%, 7.0%, and 10.2% over the best baselines. It needs only a single query per statement and works with as few as 32 training examples, making real-time, open-source detection plausible.

Core claim

The paper's central claim is that scrutiny of the model's own verification process—not the statement alone—yields internal representations that track factual correctness. IRIS prompts an LLM to evaluate a statement step by step, records the contextualized embeddings of its response, and treats the model's verbalized confidence as a soft pseudolabel for truth. A small MLP trained on these features with bootstrapped, noise-resistant targets generalizes across topics and datasets, beating direct prompting, multi-sample uncertainty estimates, and earlier internal-state probes in the unsupervised setting.

What carries the argument

The load-bearing component is the verification-response embedding: the last-token, last-layer hidden state of the LLM's chain-of-thought evaluation of the statement, paired with a soft pseudolabel equal to the model's verbalized confidence (normalized so 1 means true). The probe is a three-layer MLP (256,128,64) with ReLU and sigmoid, trained with soft bootstrapping—mixing the pseudolabel with its own current prediction—and a symmetric cross-entropy loss that resists fitting to noisy targets.

Load-bearing premise

The load-bearing premise is that an LLM's stated confidence in its own verification is a trustworthy proxy for factual correctness, so that these soft pseudolabels genuinely point the probe toward truth-related features.

What would settle it

Take a set of factual statements about a topic the LLM has never seen (for example, invented entities with plausible-sounding attributes), where the truth labels are known. If IRIS still reports above-chance accuracy, it would be exploiting a superficial pattern rather than genuine knowledge; if accuracy is at chance, the method's dependence on the model's internal knowledge is confirmed.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • IRIS requires only one LLM call per statement, enabling real-time hallucination detection with low computational overhead.
  • Detection accuracy degrades gracefully out of distribution (about 3.1% average drop), so a probe trained in one domain remains useful in another.
  • With as few as 32 training statements—even ones the model generates itself—IRIS maintains competitive accuracy, removing the need for large curated datasets.
  • Features from verification reasoning outperform features from the statement itself, confirming that reasoning traces are richer sources of truth signal.
  • Using SAR as an alternative pseudolabel shows the framework can absorb stronger uncertainty estimates, suggesting further gains as uncertainty metrics improve.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The robustness to adversarial prompts reported in the appendix hints that the hidden states, not the verbalized confidence, do most of the work; a decisive test would be training with deliberately reversed pseudolabels and checking whether accuracy collapses.
  • Because the pseudolabel slot accepts any uncertainty metric, IRIS can be seen as a general bridge between uncertainty estimation and internal-state probing, so replacing verbalized confidence with a better-calibrated score could extend the method to models that do not produce reliable self-reports.
  • The method opens a practical path to per-user or per-domain hallucination detectors: a few unlabeled statements, even self-generated by the target model, suffice to train a personalized probe without human annotation.
  • Passage-level hallucination detection, which the paper leaves for future work, could be built by decomposing a long answer into atomic claims and running IRIS on each.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes IRIS, an unsupervised hallucination detection method. For each statement, IRIS prompts an LLM to verify correctness with chain-of-thought reasoning, extracts the last-token contextualized embedding of the verification response, and uses the model's verbalized confidence as a soft pseudolabel. A small MLP probe is trained on these embeddings with soft bootstrapping and a symmetric cross-entropy loss. The authors report that IRIS outperforms existing unsupervised methods on True-False, HaluEval2, and HELM datasets, including gains of 3.2%, 7.0%, and 10.2% over the best baselines, and that it remains effective with as few as 32 training statements.

Significance. If the central claims hold, IRIS would be a practically appealing unsupervised hallucination detector: it requires only one LLM query per statement, uses lightweight probe training, and appears to transfer across topics and models. The paper includes useful analyses of out-of-distribution transfer, model size, layer depth, and training-data size, and provides a public code repository. However, the headline comparison with the internal-activation baseline MIND is confounded by a modified access assumption, and Appendix B exposes an unresolved tension in how IRIS obtains supervision when verbalized confidence collapses. These issues bear directly on the paper's main claim of consistent superiority over prior unsupervised internal-state methods.

major comments (3)
  1. [§3.2, Table 1] The comparison with MIND is not apples-to-apples. MIND is originally designed to use the generating model's contextualized embeddings; here it is evaluated with a proxy model (Llama-3.1-8B-Instruct) and a probe trained on a different automatically labeled dataset. For HELM, the original generating models are openly available, so the original access pattern is feasible. The large gaps (90.38 vs. 52.36 on True-False; 68.43 vs. 54.10 on HELM) therefore confound method quality with training-domain mismatch and unavailable internal states. The claim that IRIS beats the closest internal-activation baseline is not yet supported; please either rerun MIND with its intended access or explicitly label this as an adapted baseline and soften the corresponding conclusion.
  2. [§2.2 and Appendix B] There is an unresolved contradiction about the pseudolabel mechanism. Appendix B states that under adversarial prompts the verbalized confidence scores are 'all pushed to zero, rendering the pseudolabels ineffective for training the classifier probe,' yet IRIS still reports high accuracy. Since IRIS trains on these verbalized-confidence pseudolabels, all-zero labels would leave no supervisory signal except the classifier's own bootstrapped predictions. The paper does not explain how training proceeds in this regime, what the pseudolabel distribution actually is, or why the probe still separates true from false statements. This is load-bearing because the method's stated source of supervision collapses exactly in the setting used to demonstrate robustness. Please provide the pseudolabel distributions, the effective targets after bootstrapping, and an explanation of the mechanism, or remov
  3. [§2.2, Figure 3] The central assumption that verbalized confidence is a valid proxy for statement truthfulness is asserted but not directly validated against ground truth. Figure 3 compares verbalized confidence with token-probability entropy on an unlabeled dataset, but no calibration metrics (e.g., reliability diagrams, ECE, correlation with correctness labels) are reported. Since all pseudolabels are derived from this confidence, evidence of calibration is needed. Please add calibration analyses across datasets and model sizes, and ideally ablate the noise level in pseudolabels to show the training objective is robust to realistic miscalibration.
minor comments (5)
  1. [Eq. (1)] The cross-entropy term is written as H(ŷ_i, t_i) = t_i log ŷ_i + (1 - t_i) log(1 - ŷ_i) without a minus sign. In standard notation, minimizing this expression would maximize log-likelihood; please clarify whether the loss is negated or whether the equation denotes negative log-likelihood.
  2. [Appendix A] The prompt templates are referenced as Figure 6, but the figure content is not included in the text provided. Please include the actual prompts so the method is fully reproducible.
  3. [Section 3.1 / Table 1] The HELM model names 'LLB-7B', 'LLC-7B', and 'LLC-13B' are not defined. Please expand or explain these abbreviations.
  4. [Section 2.2] The normalization of the soft pseudolabel is described only verbally. Specify the exact mapping from verbalized confidence or entropy to ỹ_i ∈ [0,1], including how '0 indicates hallucination' is operationalized.
  5. [Table 1] No variance or significance measures are reported. Given the small training sizes (e.g., 32 statements) and stochastic MLP training, reporting means and standard deviations over multiple seeds would strengthen the empirical claims.

Circularity Check

0 steps flagged

No significant circularity: IRIS's self-referential pseudolabel loop is grounded by external ground-truth evaluation, and no load-bearing self-citation or definitional reduction is present.

full rationale

The derivation chain is: (1) prompt an LLM to verify a statement; (2) take the last-token embedding of the verification response as a feature; (3) use the LLM's verbalized confidence as a soft pseudolabel; (4) train a small MLP to predict the pseudolabel from the embedding; (5) evaluate the probe on held-out ground-truth labels. The only self-referential element is that features and pseudolabels come from the same LLM, so the probe is effectively learning to reproduce the model's confidence from its hidden states. This is not circular by construction: the pseudolabel is not defined in terms of the probe, and the probe's success is measured against external ground-truth labels on held-out datasets (True-False, HaluEval2, HELM). The central claim—that this self-supervised probe outperforms existing unsupervised detectors—is an empirical claim, not a tautology. Using the same model for both features and labels does bound the method by that model's calibration, but that is a stated assumption (Section 2.2), not a reduction of the output to the input. Baseline fairness concerns (e.g., MIND adapted with a proxy model and out-of-domain training data) are experimental-design issues, not circularity. Self-citations in the related work are not load-bearing. No equation or definition in the paper makes the predicted label equivalent to the input by construction.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 0 invented entities

The method introduces no new physical or model-level entities. It relies on two domain assumptions about LLM internals and self-confidence, plus standard mathematical tools for noisy-label learning. The free parameters are limited to loss weights and standard hyperparameters.

free parameters (3)
  • beta (soft bootstrapping weight) = 0.8 (used in loss-function ablation; final value not explicitly stated in main text)
    Controls the weight of the pseudolabel versus the classifier prediction in the regression target t_i = beta * y~ + (1-beta) * yhat. Appendix C reports that careful tuning is required.
  • phi (reverse cross-entropy weight) = not stated
    Balances the symmetric cross-entropy terms in Eq. (1). Appendix C shows phi > 1 causes a considerable accuracy drop, so the value must be tuned.
  • MLP architecture and optimizer settings = 256-128-64 with ReLU; lr=1e-2; wd=1e-5; 10 epochs; patience 5
    Chosen by hand in Section 2.3. These affect the probe but are standard and not fitted to the scientific claim itself.
axioms (5)
  • domain assumption LLM internal states contain knowledge about the truthfulness of statements, accessible via embeddings
    Invoked in Section 2.1 and based on prior work (Azaria and Mitchell 2023, Ji et al. 2024). This motivates using verification-response hidden states as features.
  • domain assumption The model's verbalized confidence or token entropy when appraising statement correctness reflects the likelihood the statement is true
    Core pseudolabel assumption in Section 2.2. If calibration fails, the soft targets are wrong and the probe learns a corrupted signal.
  • domain assumption Verification-response embeddings are more informative about factuality than statement embeddings
    Stated as a belief in Section 2.1 and empirically supported by the supervised Ceiling versus SAPLMA comparison in Table 1, but it is a design premise for the pipeline.
  • domain assumption Llama-3.1-8B-Instruct serves as an adequate proxy model for statements generated by other LLMs across all benchmarks
    Section 3.2 uses this one model for all datasets. The HELM dataset contains completions from six different LLMs; the proxy mismatch could affect results.
  • standard math Soft bootstrapping and symmetric cross-entropy are effective for learning with noisy labels
    Uses established techniques from Reed et al. (2014) and Wang et al. (2019), which are accepted in the literature.

pith-pipeline@v1.3.0-alltime-deepseek · 14300 in / 20999 out tokens · 205485 ms · 2026-08-04T18:19:10.015919+00:00 · methodology

0 comments
read the original abstract

Unsupervised hallucination detection aims to identify hallucinated content generated by large language models (LLMs) without relying on labeled data. While unsupervised methods have gained popularity by eliminating labor-intensive human annotations, they frequently rely on proxy signals unrelated to factual correctness. This misalignment biases detection probes toward superficial or non-truth-related aspects, limiting generalizability across datasets and scenarios. To overcome these limitations, we propose IRIS, an unsupervised hallucination detection framework, leveraging internal representations intrinsic to factual correctness. IRIS prompts the LLM to carefully verify the truthfulness of a given statement, and obtain its contextualized embedding as informative features for training. Meanwhile, the uncertainty of each response is considered a soft pseudolabel for truthfulness. Experimental results demonstrate that IRIS consistently outperforms existing unsupervised methods. Our approach is fully unsupervised, computationally low cost, and works well even with few training data, making it suitable for real-time detection.

Figures

Figures reproduced from arXiv: 2509.10004 by Anh Tuan Luu, Ponhvoan Srey, Xiaobao Wu.

Figure 1
Figure 1. Figure 1: Comparison between MIND (Su et al., 2024) and our method. MIND incorrectly identifies a state￾ment as hallucination. Our method extracts model inter￾nal knowledge by asking it to think carefully whether the statement is correct. Its confidence is obtained as a soft pseudolabel. information (Zhang et al., 2023b). As LLMs have been observed to confidently generate false infor￾mation, it is challenging for us… view at source ↗
Figure 2
Figure 2. Figure 2: Illustrations of eliciting uncertainty. In uncertainty-based methods, given a statement, a question to which [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Distribution of confidence in determining the [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Average accuracy (%) on True-False dataset [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Accuracy (%) using embeddings at different [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 7
Figure 7. Figure 7: HELM average accuracy (%) with Llama-3.1- [PITH_FULL_IMAGE:figures/full_fig_p012_7.png] view at source ↗
Figure 6
Figure 6. Figure 6: Prompts to verify statement correctness and [PITH_FULL_IMAGE:figures/full_fig_p012_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

49 extracted references · 1 canonical work pages

  1. [1]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. https://arxiv.org/abs/2303.08774 Gpt-4 technical report . arXiv preprint arXiv:2303.08774

    Pith/arXiv arXiv 2023

  2. [2]

    Amos Azaria and Tom Mitchell. 2023. https://arxiv.org/abs/2304.13734 The internal state of an llm knows when it's lying . arXiv preprint arXiv:2304.13734

    Pith/arXiv arXiv 2023

  3. [3]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. https://papers.nips.cc/paper/2020/hash/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html Language models are few-shot learners . Advances in neural information processing systems, 33:1877--1901

    2020

  4. [4]

    Collin Burns, Haotian Ye, Dan Klein, and Jacob Steinhardt. 2022. https://arxiv.org/abs/2212.03827 Discovering latent knowledge in language models without supervision . arXiv preprint arXiv:2212.03827

    Pith/arXiv arXiv 2022

  5. [5]

    Chao Chen, Kai Liu, Ze Chen, Yi Gu, Yue Wu, Mingyuan Tao, Zhihang Fu, and Jieping Ye. 2024. https://arxiv.org/abs/2402.03744 Inside: Llms' internal states retain the power of hallucination detection . arXiv preprint arXiv:2402.03744

    Pith/arXiv arXiv 2024

  6. [6]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. 2021. https://arxiv.org/abs/2110.14168 Training verifiers to solve math word problems . arXiv preprint arXiv:2110.14168

    Pith/arXiv arXiv 2021

  7. [7]

    Shehzaad Dhuliawala, Mojtaba Komeili, Jing Xu, Roberta Raileanu, Xian Li, Asli Celikyilmaz, and Jason Weston. 2024. https://doi.org/10.18653/v1/2024.findings-acl.212 Chain-of-verification reduces hallucination in large language models . In Findings of the Association for Computational Linguistics: ACL 2024, pages 3563--3578, Bangkok, Thailand. Association...

    doi:10.18653/v1/2024.findings-acl.212 2024

  8. [8]

    Danny Driess, Fei Xia, Mehdi SM Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, et al. 2023. https://arxiv.org/abs/2303.03378 Palm-e: An embodied multimodal language model . arXiv preprint arXiv:2303.03378

    Pith/arXiv arXiv 2023

  9. [9]

    Jinhao Duan, Hao Cheng, Shiqi Wang, Alex Zavalny, Chenan Wang, Renjing Xu, Bhavya Kailkhura, and Kaidi Xu. 2024. https://aclanthology.org/2024.acl-long.276 Shifting attention to relevance: Towards the predictive uncertainty quantification of free-form large language models . In Proceedings of the 62nd Annual Meeting of the Association for Computational Li...

    2024

  10. [10]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. https://arxiv.org/abs/2407.21783 The llama 3 herd of models . arXiv preprint arXiv:2407.21783

    Pith/arXiv arXiv 2024

  11. [11]

    Ekaterina Fadeeva, Aleksandr Rubashevskii, Artem Shelmanov, Sergey Petrakov, Haonan Li, Hamdy Mubarak, Evgenii Tsymbalov, Gleb Kuzmin, Alexander Panchenko, Timothy Baldwin, Preslav Nakov, and Maxim Panov. 2024. https://aclanthology.org/2024.findings-acl.558 Fact-checking the output of large language models via token-level uncertainty quantification . In F...

    2024

  12. [12]

    Ekaterina Fadeeva, Roman Vashurin, Akim Tsvigun, Artem Vazhentsev, Sergey Petrakov, Kirill Fedyanin, Daniil Vasilev, Elizaveta Goncharova, Alexander Panchenko, Maxim Panov, et al. 2023. https://arxiv.org/abs/2311.07383 Lm-polygraph: Uncertainty estimation for language models . arXiv preprint arXiv:2311.07383

    Pith/arXiv arXiv 2023

  13. [13]

    Jinlan Fu, See-Kiong Ng, Zhengbao Jiang, and Pengfei Liu. 2023. Gptscore: Evaluate as you desire. arXiv preprint arXiv:2302.04166

    Pith/arXiv arXiv 2023

  14. [14]

    Ziwei Ji, Delong Chen, Etsuko Ishii, Samuel Cahyawijaya, Yejin Bang, Bryan Wilie, and Pascale Fung. 2024. https://doi.org/10.18653/v1/2024.blackboxnlp-1.6 LLM internal states reveal hallucination risk faced with a query . In Proceedings of the 7th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP, pages 88--104, Miami, Florida, US. ...

    doi:10.18653/v1/2024.blackboxnlp-1.6 2024

  15. [15]

    Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. https://proceedings.neurips.cc/paper_files/paper/2022/hash/8bb0d291acd4acf06ef112099c16f326-Abstract-Conference.html Large language models are zero-shot reasoners . Advances in neural information processing systems, 35:22199--22213

    2022

  16. [16]

    Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. 2023. https://arxiv.org/abs/2302.09664 Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation . arXiv preprint arXiv:2302.09664

    Pith/arXiv arXiv 2023

  17. [17]

    Junyi Li, Jie Chen, Ruiyang Ren, Xiaoxue Cheng, Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. 2024. https://aclanthology.org/2024.acl-long.586 The dawn after the dark: An empirical study on factuality hallucination in large language models . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), page...

    2024

  18. [18]

    Junyi Li, Xiaoxue Cheng, Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. 2023 a . https://doi.org/10.18653/v1/2023.emnlp-main.397 H alu E val: A large-scale hallucination evaluation benchmark for large language models . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6449--6464, Singapore. Association for Computat...

    doi:10.18653/v1/2023.emnlp-main.397 2023

  19. [19]

    Kenneth Li, Oam Patel, Fernanda Vi \'e gas, Hanspeter Pfister, and Martin Wattenberg. 2023 b . https://proceedings.neurips.cc/paper_files/paper/2023/hash/81b8390039b7302c909cb769f8b6cd93-Abstract-Conference.html Inference-time intervention: Eliciting truthful answers from a language model . Advances in Neural Information Processing Systems, 36

    2023

  20. [20]

    Zhen Lin, Shubhendu Trivedi, and Jimeng Sun. 2024. https://arxiv.org/abs/2305.19187 Generating with confidence: Uncertainty quantification for black-box large language models . Preprint, arXiv:2305.19187

    Pith/arXiv arXiv 2024

  21. [21]

    Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.153 G -eval: NLG evaluation using gpt-4 with better human alignment . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 2511--2522, Singapore. Association for Computational Linguistics

    doi:10.18653/v1/2023.emnlp-main.153 2023

  22. [22]

    Potsawee Manakul, Adian Liusie, and Mark JF Gales. 2023. https://arxiv.org/abs/2303.08896 Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models . arXiv preprint arXiv:2303.08896

    Pith/arXiv arXiv 2023

  23. [23]

    Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Wei Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2023. https://arxiv.org/abs/2305.14251 Factscore: Fine-grained atomic evaluation of factual precision in long form text generation . arXiv preprint arXiv:2305.14251

    Pith/arXiv arXiv 2023

  24. [24]

    MistralAI. 2024. https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3 Mistral-7b-instruct-v0.3

    2024

  25. [25]

    Niels M \"u ndler, Jingxuan He, Slobodan Jenko, and Martin Vechev. 2023. https://arxiv.org/abs/2305.15852 Self-contradictory hallucinations of large language models: Evaluation, detection and mitigation . arXiv preprint arXiv:2305.15852

    Pith/arXiv arXiv 2023

  26. [26]

    Fengjun Pan, Xiaobao Wu, Zongrui Li, and Anh Tuan Luu. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.794 Are LLM s good zero-shot fallacy classifiers? In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 14338--14364, Miami, Florida, USA. Association for Computational Linguistics

    doi:10.18653/v1/2024.emnlp-main.794 2024

  27. [27]

    Scott Reed, Honglak Lee, Dragomir Anguelov, Christian Szegedy, Dumitru Erhan, and Andrew Rabinovich. 2014. https://arxiv.org/abs/1412.6596 Training deep neural networks on noisy labels with bootstrapping . arXiv preprint arXiv:1412.6596

    Pith/arXiv arXiv 2014

  28. [28]

    Pranab Sahoo, Ayush Kumar Singh, Sriparna Saha, Vinija Jain, Samrat Mondal, and Aman Chadha. 2024. https://arxiv.org/abs/2402.07927 A systematic survey of prompt engineering in large language models: Techniques and applications . arXiv preprint arXiv:2402.07927

    Pith/arXiv arXiv 2024

  29. [29]

    William Saunders, Catherine Yeh, Jeff Wu, Steven Bills, Long Ouyang, Jonathan Ward, and Jan Leike. 2022. https://arxiv.org/abs/2206.05802 Self-critiquing models for assisting human evaluators . arXiv preprint arXiv:2206.05802

    Pith/arXiv arXiv 2022

  30. [30]

    Weihang Su, Changyue Wang, Qingyao Ai, Yiran Hu, Zhijing Wu, Yujia Zhou, and Yiqun Liu. 2024. https://arxiv.org/abs/2403.06448 Unsupervised real-time hallucination detection based on the internal states of large language models . arXiv preprint arXiv:2403.06448

    Pith/arXiv arXiv 2024

  31. [31]

    Katherine Tian, Eric Mitchell, Allan Zhou, Archit Sharma, Rafael Rafailov, Huaxiu Yao, Chelsea Finn, and Christopher D Manning. 2023. https://arxiv.org/abs/2305.14975 Just ask for calibration: Strategies for eliciting calibrated confidence scores from language models fine-tuned with human feedback . arXiv preprint arXiv:2305.14975

    Pith/arXiv arXiv 2023

  32. [32]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. https://arxiv.org/abs/2307.09288 Llama 2: Open foundation and fine-tuned chat models . arXiv preprint arXiv:2307.09288

    Pith/arXiv arXiv 2023

  33. [33]

    Miles Turpin, Julian Michael, Ethan Perez, and Samuel Bowman. 2023. Language models don't always say what they think: Unfaithful explanations in chain-of-thought prompting. Advances in Neural Information Processing Systems, 36:74952--74965

    2023

  34. [34]

    Jiaan Wang, Yunlong Liang, Fandong Meng, Zengkui Sun, Haoxiang Shi, Zhixu Li, Jinan Xu, Jianfeng Qu, and Jie Zhou. 2023. https://doi.org/10.18653/v1/2023.newsum-1.1 Is C hat GPT a good NLG evaluator? a preliminary study . In Proceedings of the 4th New Frontiers in Summarization Workshop, pages 1--11, Singapore. Association for Computational Linguistics

    doi:10.18653/v1/2023.newsum-1.1 2023

  35. [35]

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2022. https://arxiv.org/abs/2203.11171 Self-consistency improves chain of thought reasoning in language models . arXiv preprint arXiv:2203.11171

    Pith/arXiv arXiv 2022

  36. [36]

    Yisen Wang, Xingjun Ma, Zaiyi Chen, Yuan Luo, Jinfeng Yi, and James Bailey. 2019. https://openaccess.thecvf.com/content_ICCV_2019/papers/Wang_Symmetric_Cross_Entropy_for_Robust_Learning_With_Noisy_Labels_ICCV_2019_paper.pdf Symmetric cross entropy for robust learning with noisy labels . In Proceedings of the IEEE/CVF international conference on computer v...

    2019

  37. [37]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. https://proceedings.neurips.cc/paper_files/paper/2022/hash/9d5609613524ecf4f15af0f7b31abca4-Abstract-Conference.html Chain-of-thought prompting elicits reasoning in large language models . Advances in neural information processing systems, 35:24824--24837

    2022

  38. [38]

    Xiaobao Wu. 2025. https://arxiv.org/pdf/2505.02686 Sailing ai by the stars: A survey of learning from rewards in post-training and test-time scaling of large language models . arXiv preprint arXiv:2505.02686

    Pith/arXiv arXiv 2025

  39. [39]

    Xiaobao Wu, Xinshuai Dong, Thong Nguyen, and Anh Tuan Luu. 2023. https://arxiv.org/pdf/2306.04217 Effective neural topic modeling with embedding clustering regularization . In Proceedings of the 40th International Conference on Machine Learning, ICML'23. JMLR.org

    Pith/arXiv arXiv 2023

  40. [40]

    Xiaobao Wu, Thong Nguyen, and Anh Tuan Luu. 2024 a . https://doi.org/10.1007/s10462-023-10661-7 A survey on neural topic models: Methods, applications, and challenges . Artificial Intelligence Review

    work page doi:10.1007/s10462-023-10661-7 2024

  41. [41]

    Xiaobao Wu, Thong Thanh Nguyen, Delvin Ce Zhang, William Yang Wang, and Anh Tuan Luu. 2024 b . https://openreview.net/forum?id=7t6aq0Fa9D FAST opic: Pretrained transformer is a fast, adaptive, stable, and transferable topic model . In The Thirty-eighth Annual Conference on Neural Information Processing Systems

    2024

  42. [42]

    Xiaobao Wu, Liangming Pan, William Yang Wang, and Anh Tuan Luu. 2024 c . https://doi.org/10.18653/v1/2024.emnlp-main.843 AKEW : Assessing knowledge editing in the wild . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 15118--15133, Miami, Florida, USA. Association for Computational Linguistics

    doi:10.18653/v1/2024.emnlp-main.843 2024

  43. [43]

    Xiaobao Wu, Liangming Pan, Yuxi Xie, Ruiwen Zhou, Shuai Zhao, Yubo Ma, Mingzhe Du, Rui Mao, Anh Tuan Luu, and William Yang Wang. 2024 d . https://arxiv.org/abs/2412.13670 Antileak-bench: Preventing data contamination by automatically constructing benchmarks with updated real-world knowledge . arXiv preprint arXiv:2412.13670

    Pith/arXiv arXiv 2024

  44. [44]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. 2024. https://arxiv.org/abs/2412.15115 Qwen2. 5 technical report . arXiv preprint arXiv:2412.15115

    Pith/arXiv arXiv 2024

  45. [45]

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. https://arxiv.org/abs/2305.10601 Tree of thoughts: Deliberate problem solving with large language models, 2023 . arXiv preprint arXiv:2305.10601

    Pith/arXiv arXiv 2023

  46. [46]

    Tianhang Zhang, Lin Qiu, Qipeng Guo, Cheng Deng, Yue Zhang, Zheng Zhang, Chenghu Zhou, Xinbing Wang, and Luoyi Fu. 2023 a . https://arxiv.org/abs/2311.13230 Enhancing uncertainty-based hallucination detection with stronger focus . arXiv preprint arXiv:2311.13230

    Pith/arXiv arXiv 2023

  47. [47]

    Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, et al. 2023 b . https://arxiv.org/abs/2309.01219 Siren's song in the ai ocean: a survey on hallucination in large language models . arXiv preprint arXiv:2309.01219

    Pith/arXiv arXiv 2023

  48. [48]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  49. [49]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...