Pith. sign in

REVIEW 3 major objections 5 minor 53 references

The Curious Case of Factuality Finetuning: Models' Internal Beliefs Can Improve Factuality

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Training on a model's own internally filtered beliefs yields the most factual long-form generations, beating gold-document finetuning by 12–30 percentage points.

desk verdict Careful long-form extension of the familiarity story, but the central result may be a verifier artifact from vagueness; deserves revision, not desk reject. read the letter →

arxiv 2507.08371 v1 pith:SCYREAMJ submitted 2025-07-11 cs.CL

Benjamin Newman , Abhilasha Ravichander , Jaehun Jung , Rui Xin , Hamish Ivison , Yegor Kuznetsov , Pang Wei Koh , Yejin Choi This is my paper · ORCID
classification cs.CL
keywords hallucinationlong-formgenerationfinetuningdataselectioninternalknowledgeprobesself-generatedtrainingepistemicfactualityevaluationfiltering
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

What data should you finetune on to reduce hallucination? This paper answers: not necessarily the most factually accurate data you can find. Across three long-form generation domains and three instruction-tuned language models, training on model-generated claims that a hidden-state probe labels as supported—what the model itself believes—consistently improves factuality more than training on gold documents or on generated claims filtered by external entailment. The improvements hold even though the internally filtered training data is less factual by the external standard, and they transfer across domains. The central practical claim is that a model's own internal knowledge signal, extracted cheaply with linear probes, can be a powerful guide for post-training data selection.

What carries the argument

The central mechanism is EPISTEMIC TRAINING: generate candidate text from the target model, atomize it into atomic claims, and filter those claims with a binary logistic-regression probe applied to the last-token hidden state at a middle layer, trained on a separate set of entities with silver labels from an external entailment model. The probe acts as a read-out of what the model's representations encode as true. A second ingredient is length control: for every training sample the pipeline keeps the same number of atoms across all compared conditions, ranking atoms by probe probability or entailment score, so factuality differences cannot be explained simply by shorter generations. The argument also depends on an automatic entailment model as both the source of silver labels for the probe and the evaluator of final generation factuality.

What would settle it

Run the same four-way comparison with human-annotated atomic claims as the evaluation oracle. If EPISTEMIC TRAINING no longer beats gold-only or externally filtered training when factuality is judged by people instead of the entailment model, the central claim would be refuted. A simpler check: ask human raters to score the informativeness and factual support of generations from each condition, and compare whether the internal-filter advantage persists.

Watch

Extended reading notes

Core claim

The paper's central claim is that finetuning data selected by the model's own internal beliefs produces more factual long-form generations than data selected by external human-authored gold documents. The authors call this procedure EPISTEMIC TRAINING: generate multiple documents about an entity from the model, split them into atomic claims, and keep only claims a linear probe trained on hidden states predicts are true. In controlled experiments that equalize the number of claims in each training sample, this combination outperforms training on gold data alone, on gold data filtered by the same probes, and on generated data filtered by entailment to gold documents, with gains of 12 to 30 factuality percentage points depending on the model and domain. The paper further claims that the gains generalize across domains, so the internal-belief signal transfers beyond the entities used for probe training.

Load-bearing premise

The load-bearing premise is that a linear probe trained on external-verifier labels faithfully captures what the model genuinely believes, and that the same entailment model is a fair judge for both producing those labels and scoring final factuality; if the probe merely reflects the verifier, or the verifier rewards vagueness, the central comparison could be a measurement artifact rather than evidence about internal knowledge.

Editorial extensions

If this is right

  • If correct, practitioners can reduce hallucination using minimal gold data: only a small held-out entity set is needed to train the probe, and the gains transfer when the model is finetuned on one domain and tested on another.
  • Because the internally filtered data contains claims that are vague and sometimes unsupported by gold documents, the method implies a factuality–informativeness trade-off, with vaguer generations as a price for fewer errors.
  • The finding that generated data beats gold data, and that internal filtering beats external entailment, suggests that familiarity with a claim matters more than its objective verifiability for post-training hallucination control.
  • Improvements concentrate on low-frequency entities, consistent with the idea that training on unfamiliar gold facts encourages hallucination, and that internal filters preferentially remove those risky claims.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • We infer, beyond the paper, that because the probe is trained on silver labels from the same entailment model used to evaluate final outputs, part of the reported 12–30 point advantage could reflect the filter and evaluator sharing a bias rather than an independent internal-knowledge signal.
  • We infer, beyond the paper, that the internal probe's preference for shorter, vaguer claims could be abusing the entailment evaluator's tendency to mark under-specific claims as supported; a human evaluation would be needed to confirm the gains are real.
  • We infer, beyond the paper, that hidden-state probe scores could serve as a continuous familiarity measure for any entity, potentially replacing corpus-frequency proxies for hallucination risk.
  • We infer, beyond the paper, that a direct practical test would be to apply EPISTEMIC TRAINING on a domain the model was not pretrained on and measure whether the model abstains rather than hallucinates, because the current method may only preserve existing knowledge.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper asks what data practitioners should finetune on to reduce hallucination in long-form generation, and proposes EPISTEMIC TRAINING: finetuning on claims generated by the model itself and filtered by logistic-regression probes applied to the model's hidden states. The probe is trained on silver labels produced by an external entailment verifier (MiniCheck) on held-out entities, and the same verifier is used to compute the final factuality metric. Across three models (Mistral-7B-Instruct, Llama-2-7B-Chat, Gemma-7B-it) and three domains (biographies, plots, medical terms), the authors report that EPISTEMIC TRAINING gives the highest factuality, improving over gold data alone by 12–30 percentage points, that internal filtering outperforms external entailment filtering for generated data by up to 8 points, and that the gains transfer across domains. The experiments include a length-control budget of atoms, a random-filter baseline, standard deviations over three seeds, and a cross-domain transfer table.

Significance. If the mechanism claimed in the title is correct, this is a practically valuable result: practitioners could improve long-form factuality with very little gold data by finetuning on self-generated, self-filtered claims, and the cross-domain transfer would further reduce annotation cost. The experimental scaffolding is strong in several respects: length is controlled through an atom budget, a random filter baseline is included, three open models and three domains are used, and results are averaged over seeds. The paper also releases a repository for reproducibility. The weakness is that the central interpretation—that the probe measures internal beliefs—is not separated from a plausible metric-bias explanation in which the same verifier is used for training labels, filtering, and evaluation; this is the main reason the reported gains cannot yet be taken at face value as evidence about internal knowledge.

major comments (3)
  1. [§2.2, §3, Appendix F.3, Appendix H.3] The central claim that EPISTEMIC TRAINING exploits the model's internal beliefs rather than verifier bias is not yet supported, because fint is trained on MiniCheck silver labels (§2.2) and the final evaluation uses the same MiniCheck function (§3), while fint-filtered claims are significantly shorter and vaguer than fext-filtered claims (Appendix F.3, Table 9; Table 8). MiniCheck is an entailment model with no specificity penalty, so short generic claims such as "Von Raumer died" are easy to mark as entailed; the paper's own human evaluation in Appendix H.3 finds that EPISTEMIC TRAINING generations receive a mean detail score of 0.95 on a 0–2 scale, with score 1 explicitly defined as "vague or disfluent." The length control and random-filter baseline control the number of atoms but not per-claim specificity, so the 12–30 percentage point improvement and the fint-over-fext gains could reflect verifier bias rather than internal knowledge. Please report results with a verifier that penalizes vagueness (or with human factuality judgments), or otherwise show that the effect survives when claims are matched on specificity.
  2. [Appendix E] The probe layer index is selected by plotting probe F1 on claims from the evaluation set: the caption of Figure 2 states that the plot is "on claims from the evaluation set of model generations," and layer 15 is then used in all experiments. This is a selection on the evaluation entities, which can optimistically bias fint's measured quality and the downstream factuality advantage in Table 2. The layer should be chosen on the probe-training entities or on a separate validation split, with the sensitivity of Table 2 to that choice reported.
  3. [§2.2] Because fint's labels are themselves generated by fext, the paper needs a direct test that the probe captures internal belief rather than a learned approximation of the external verifier. The held-out entity split reduces but does not eliminate the concern: fint could be a noisy fext, and the claim-length divergence in Appendix F.3 would then simply be verifier bias inherited from fext. A concrete test is to train the probe on human-labeled claims (or on labels from a second, independent verifier) and check whether both probe accuracy and the fint advantage in Table 2 persist; in addition, human-verifying a sample of fint-only claims would establish that they are true, not merely entailed by a no-specificity verifier.
minor comments (5)
  1. [Appendix C.1] There is a typo "Mitral" for "Mistral" in the description of the atomization validation ("the maximum of the RougeL and exact match scores between each Mitral atom and each InstructGPT atom").
  2. [Appendix B] The Plots dataset description reads "This dataset provides contains~3,000 media titles"; the doubled verb should be corrected.
  3. [Appendix I] The sample generation for Nicolas Sarkozy contains visible HTML-entity artifacts (e.g., "s¡C3¿¡A9¿gol¡C3¿¡A8¿ne"); if these appear in the actual model output rather than in the PDF rendering, they should be cleaned or explicitly noted.
  4. [Appendix H.2] The caption of Table 11 copies the sentence "Dashes are where the Bios internal knowledge filter for Llama-2-7B-Chat predicts all claims are unsupported," but Table 11 contains no dashes; the sentence should be removed or adapted.
  5. [Table 3] The header row of Table 3 is difficult to parse because the column labels (Gold/Generated, model names) are compressed into a single line; a two-level header would make the quartile comparison readable.

Circularity Check

0 steps flagged · score 2.0 of 10

No central circularity: the fint filter is a learned probe, not fext itself, and the fint-vs-fext comparison is evaluated with the same fext metric, so the result is not forced by construction; the shared oracle weakens the 'internal beliefs' interpretation but does not reduce the derivation to its inputs.

full rationale

The paper's central comparison — generated data filtered by fint versus generated data filtered by fext and versus gold data — is not circular in the sense of a prediction being equivalent to its input by construction. fint is a linear probe trained on silver labels produced by fext using a separate held-out set of entities (Section 2.2: 'silver labels generated by fext as training labels'), and final factuality is measured with the same fext MiniCheck verifier (Section 3: 'we use the same approach as in the external knowledge filtering setting—fext'). This shared oracle is a real construct-validity limitation: the 'internal knowledge' signal is operationalized as a fext surrogate, and Appendix F.3 (Table 9) plus Appendix H.3 show that fint preferentially selects shorter claims and that EPISTEMIC TRAINING generations are rated vague on average (mean detail 0.95 on a 0–2 scale). Those observations support a metric-bias critique of the interpretation, but they do not make the empirical result circular: fint demonstrably diverges from fext (fint-supported training data is only 63.5–85.6% fext-supported, Table 7), all conditions are scored by the same fext metric, and the fint advantage over fext is an empirical outcome rather than a definitional identity. The probe-layer selection in Appendix E uses claims from the evaluation set, which adds optimistic bias but is a leakage concern rather than a reduction of the central claim. No load-bearing self-citation or imported uniqueness theorem is present; the probe-motivation citations (Azaria & Mitchell 2023, Joshi et al. 2024) are external and used only to motivate plausibility. Accordingly, no circular step meets the quote-and-reduction standard; the score reflects a partially self-referential evaluation oracle, not a circular derivation.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on several domain assumptions that are acknowledged or implicit: gold documents define ground truth, MiniCheck is a fair judge, and a probe trained on external labels can stand for internal belief. The probe layer and length-control budget are data-dependent choices that affect results but are not derived from theory. No new physical or ontological entities are introduced.

free parameters (3)
  • Probe layer index = 15
    Selected because probe F1 peaks at early-middle layers (Figure 2, Appendix E); the plot is computed on claims from the evaluation set, so this choice may leak evaluation information.
  • Length-control atom budget p = per-sample minimum across the four conditions
    Training samples are truncated to the minimum number of supported atoms across conditions to control for length; this value is data-dependent and shapes all training sets (Section 3, Length Control).
  • LoRA and optimization hyperparameters = rank 8, alpha 16, lr 3e-4, 500 steps, batch 16
    Chosen via hyperparameter search on held-out biography data (Appendix G); standard but not derived from first principles.
assumptions (4)
  • domain assumption Gold documents contain all information needed to judge a claim's factuality.
    Claims that are correct but not in the Wikipedia gold document are counted as unsupported; authors acknowledge this in Limitations A.
  • domain assumption MiniCheck entailment is an accurate factuality oracle.
    Used both to produce fext training labels and to evaluate final generations (Sections 2.2 and 3); validated against FActScore labels within 1 point of GPT-4o, but it is still an approximate model.
  • ad hoc to paper A linear probe trained on fext silver labels reflects the model's internal beliefs.
    This interpretive assumption is central to the internal beliefs claim; probes are trained on external labels for held-out entities, not on the model's own certainty judgments (Section 2.2).
  • domain assumption Atomization, decontextualization, and merging are faithful enough not to change conclusions.
    Pipeline errors could propagate; authors validate components in Appendix C but note compounding errors in Limitations A.

how reviews work

0 comments
Cite this review

Pith. "Pith review of The Curious Case of Factuality Finetuning: Models' Internal Beliefs Can Improve Factuality." pith.science (2026). https://pith.science/paper/SCYREAMJ

@misc{pith2026250708371,
  author       = {Pith},
  title        = {Pith review of: The Curious Case of Factuality Finetuning: Models' Internal Beliefs Can Improve Factuality},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SCYREAMJ}},
  note         = {Machine review of arXiv:2507.08371}
}
read the original abstract

Language models are prone to hallucination - generating text that is factually incorrect. Finetuning models on high-quality factual information can potentially reduce hallucination, but concerns remain; obtaining factual gold data can be expensive and training on correct but unfamiliar data may potentially lead to even more downstream hallucination. What data should practitioners finetune on to mitigate hallucinations in language models? In this work, we study the relationship between the factuality of finetuning data and the prevalence of hallucinations in long-form generation tasks. Counterintuitively, we find that finetuning on factual gold data is not as helpful as finetuning on model-generated data that models believe to be factual. Next, we evaluate filtering strategies applied on both factual gold data and model-generated data, and find that finetuning on model-generated data that is filtered by models' own internal judgments often leads to better overall factuality compared to other configurations: training on gold data filtered by models' judgments, training on gold data alone, or training on model-generated data that is supported by gold data. These factuality improvements transfer across three domains we study, suggesting that a models' own beliefs can provide a powerful signal for factuality.

Figures

Figures reproduced from arXiv: 2507.08371 by the authors.

Figure 1
Figure 1. We manipulate two aspects of finetuning data: the [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Plot of probe F1 versus layer of the model on claims from the evaluation set of [PITH_FULL_IMAGE:figures/full_fig_p019_2.png] view at source ↗
Figure 3
Figure 3. Histograms showing the proportion of claims predicted to be supported for each [PITH_FULL_IMAGE:figures/full_fig_p020_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Histogram of the log-frequencies of the terms in our evaluation datasets. [PITH_FULL_IMAGE:figures/full_fig_p023_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

53 extracted references · 19 canonical work pages

  1. [1]

    Factcheckmate: Preemptively detecting and mitigating hallucinations in lms

    Deema Alnuhait, Neeraja Kirtane, Muhammad Khalifa, and Hao Peng. Factcheckmate: Preemptively detecting and mitigating hallucinations in lms. ArXiv, abs/2410.02899, 2024. URL https://api.semanticscholar.org/CorpusID:273163183

    arXiv 2024

  2. [2]

    The internal state of an llm knows when it’s lying

    Amos Azaria and Tom Mitchell. The internal state of an llm knows when it’s lying. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the Association for Computational Linguistics: EMNLP 2023, pp.\ 967–976, Singapore, December 2023. Association for Computational Linguistics. doi:10.18653/v1/2023.findings-emnlp.68. URL https://aclanthology.org...

    doi:10.18653/v1/2023.findings-emnlp.68 2023

  3. [3]

    Quantifying memorization across neural language models

    Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang. Quantifying memorization across neural language models. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=TatRHT_1cK

    work page 2023

  4. [4]

    Sky CH-Wang, Benjamin Van Durme, Jason Eisner, and Chris Kedzie. Do androids know they ' re only dreaming of electric sheep? In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Findings of the Association for Computational Linguistics: ACL 2024, pp.\ 4401--4420, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi:10.18653/v...

    doi:10.18653/v1/2024.findings-acl.260 2024

  5. [5]

    Decontextualization: Making sentences stand-alone

    Eunsol Choi, Jennimaria Palomaki, Matthew Lamm, Tom Kwiatkowski, Dipanjan Das, and Michael Collins. Decontextualization: Making sentences stand-alone. Transactions of the Association for Computational Linguistics, 9: 0 447--461, 2021. doi:10.1162/tacl_a_00377. URL https://aclanthology.org/2021.tacl-1.27

    doi:10.1162/tacl_a_00377 2021

  6. [6]

    The llama 3 herd of models

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. ArXiv, abs/2407.21783, 2024. URL https://arxiv.org/abs/2407.21783

    arXiv 2024

  7. [7]

    Wiki medical terms, 2023

    Gamino. Wiki medical terms, 2023. URL https://huggingface.co/datasets/gamino/wiki_medical_terms

    work page 2023

  8. [8]

    Zorik Gekhman, Gal Yona, Roee Aharoni, Matan Eyal, Amir Feder, Roi Reichart, and Jonathan Herzig. Does fine-tuning LLM s on new knowledge encourage hallucinations? In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp.\ 7765--7784, Miami, Florida, USA, Novemb...

    doi:10.18653/v1/2024.emnlp-main.444 2024

Show all 53 references
  1. [9]

    Understanding finetuning for factual knowledge extraction

    Gaurav Ghosal, Tatsunori Hashimoto, and Aditi Raghunathan. Understanding finetuning for factual knowledge extraction. In Proceedings of the 41st International Conference on Machine Learning, ICML'24. JMLR.org, 2024. URL https://openreview.net/pdf?id=cPsn9AcOYh

    2024

  2. [10]

    Language models hallucinate, but may excel at fact verification

    Jian Guan, Jesse Dodge, David Wadden, Minlie Huang, and Hao Peng. Language models hallucinate, but may excel at fact verification. In Kevin Duh, Helena Gomez, and Steven Bethard (eds.), Proceedings of the 2024 Conference of the North American Chapter of the Association for Com...

    2024 doi

  3. [11]

    Molecular facts: Desiderata for decontextualization in LLM fact verification

    Anisha Gunjal and Greg Durrett. Molecular facts: Desiderata for decontextualization in LLM fact verification. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Findings of the Association for Computational Linguistics: EMNLP 2024, pp.\ 3751--3768, Miami, Florida, US...

    2024 doi

  4. [12]

    Training language models on the knowledge graph: Insights on hallucinations and their detectability

    Jiri Hron, Laura A Culp, Gamaleldin Fathy Elsayed, Rosanne Liu, Jasper Snoek, Simon Kornblith, Alex Rizkowsky, Isabelle Simpson, Jascha Sohl-Dickstein, Noah Fiedel, Aaron T Parisi, Alexander A Alemi, Azade Nova, Ben Adlam, Bernd Bohnet, Gaurav Mishra, Hanie Sedghi, Izzeddin Gu...

    2024

  5. [13]

    Lo RA : Low-rank adaptation of large language models

    Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lo RA : Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=nZeVKeeFYf9

    2022

  6. [14]

    Training language models to generate text with citations via fine-grained rewards

    Chengyu Huang, Zeqiu Wu, Yushi Hu, and Wenya Wang. Training language models to generate text with citations via fine-grained rewards. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistic...

    2024 doi

  7. [15]

    Calibrating long-form generations from large language models

    Yukun Huang, Yixin Liu, Raghuveer Thirukovalluru, Arman Cohan, and Bhuwan Dhingra. Calibrating long-form generations from large language models. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Findings of the Association for Computational Linguistics: EMNLP 2024, ...

    2024 doi

  8. [16]

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas ...

    2023 arXiv

  9. [17]

    Teaching language models to hallucinate less with synthetic tasks

    Erik Jones, Hamid Palangi, Clarisse Sim \ o es Ribeiro, Varun Chandrasekaran, Subhabrata Mukherjee, Arindam Mitra, Ahmed Hassan Awadallah, and Ece Kamar. Teaching language models to hallucinate less with synthetic tasks. In The Twelfth International Conference on Learning Repr...

    2024

  10. [18]

    T rivia QA : A large scale distantly supervised challenge dataset for reading comprehension

    Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. T rivia QA : A large scale distantly supervised challenge dataset for reading comprehension. In Regina Barzilay and Min-Yen Kan (eds.), Proceedings of the 55th Annual Meeting of the Association for Computational Lin...

    2017 doi

  11. [19]

    Personas as a way to model truthfulness in language models

    Nitish Joshi, Javier Rando, Abulhair Saparov, Najoung Kim, and He He. Personas as a way to model truthfulness in language models. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing...

    2024 doi

  12. [20]

    Language models (mostly) know what they know

    Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova DasSarma, Eli Tran-Johnson, Scott Johnston, Sheer El-Showk, Andy Jones, Nelson Elhage, Tristan Hume, Anna Chen, Yuntao Bai, Sam Bowman, Stanislav For...

    2022 arXiv

  13. [21]

    Unfamiliar finetuning examples control how language models hallucinate

    Katie Kang, Eric Wallace, Claire Tomlin, Aviral Kumar, and Sergey Levine. Unfamiliar finetuning examples control how language models hallucinate. In Luis Chiruzzo, Alan Ritter, and Lu Wang (eds.), Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the...

    2025

  14. [22]

    Gonzalez, Haotong Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Haotong Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. Proceedings of the 29th Symposium on Operating Systems Principle...

    2023

  15. [23]

    Factuality enhanced language models for open-ended text generation

    Nayeon Lee, Wei Ping, Peng Xu, Mostofa Patwary, Pascale Fung, Mohammad Shoeybi, and Bryan Catanzaro. Factuality enhanced language models for open-ended text generation. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS '22, Red ...

    2022

  16. [24]

    Inference-time intervention: Eliciting truthful answers from a language model

    Kenneth Li, Oam Patel, Fernanda Vi \'e gas, Hanspeter Pfister, and Martin Wattenberg. Inference-time intervention: Eliciting truthful answers from a language model. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https://openreview.net/forum?id...

    2023

  17. [25]

    Flame : Factuality-aware alignment for large language models

    Sheng-Chieh Lin, Luyu Gao, Barlas Oguz, Wenhan Xiong, Jimmy Lin, Wen-tau Yih, and Xilun Chen. Flame : Factuality-aware alignment for large language models. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (eds.), Advances in Neural Informati...

    2024

  18. [26]

    Infini-gram: Scaling unbounded n-gram language models to a trillion tokens

    Jiacheng Liu, Sewon Min, Luke Zettlemoyer, Yejin Choi, and Hannaneh Hajishirzi. Infini-gram: Scaling unbounded n-gram language models to a trillion tokens. In First Conference on Language Modeling, 2024. URL https://openreview.net/forum?id=u2vAyMeLMm

    2024

  19. [27]

    Kevin Liu, Stephen Casper, Dylan Hadfield-Menell, and Jacob Andreas. Cognitive dissonance: Why do language model outputs disagree with internal representations of truthfulness? In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical...

    2023 doi

  20. [28]

    When not to trust language models: Investigating effectiveness of parametric and non-parametric memories

    Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. When not to trust language models: Investigating effectiveness of parametric and non-parametric memories. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Proceedings o...

    2023 doi

  21. [29]

    Locally typical sampling

    Clara Meister, Tiago Pimentel, Gian Wiher, and Ryan Cotterell. Locally typical sampling. Transactions of the Association for Computational Linguistics, 11: 0 102--121, 2023. doi:10.1162/tacl_a_00536. URL https://aclanthology.org/2023.tacl-1.7

    2023 doi

  22. [30]

    Gemma Team Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, L. Sifre, Morgane Rivi \`e re, Mihir Kale, J Christopher Love, Pouya Dehghani Tafti, L'eonard Hussenot, Aakanksha Chowdhery, Adam Roberts, Aditya Barua, Alex Botev, Alex Castro-Ros, Am...

    2024 arXiv

  23. [31]

    FA ct S core: Fine-grained atomic evaluation of factual precision in long form text generation

    Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. FA ct S core: Fine-grained atomic evaluation of factual precision in long form text generation. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.)...

    2023 doi

  24. [32]

    Fine-grained hallucination detection and editing for language models

    Abhika Mishra, Akari Asai, Vidhisha Balachandran, Yizhong Wang, Graham Neubig, Yulia Tsvetkov, and Hannaneh Hajishirzi. Fine-grained hallucination detection and editing for language models. In First Conference on Language Modeling, 2024. URL https://openreview.net/forum?id=dJMTn3QOWO

    2024

  25. [33]

    LLM s know more than they show: On the intrinsic representation of LLM hallucinations

    Hadas Orgad, Michael Toker, Zorik Gekhman, Roi Reichart, Idan Szpektor, Hadas Kotek, and Yonatan Belinkov. LLM s know more than they show: On the intrinsic representation of LLM hallucinations. In The Thirteenth International Conference on Learning Representations, 2025. URL h...

    2025

  26. [34]

    Wikiplots, 2017

    Mark Riedl and John Bell. Wikiplots, 2017. URL https://github.com/markriedl/WikiPlots

    2017

  27. [35]

    Diversity

    Chantal Shaib. Diversity. https://github.com/cshaib/diversity, 2024

    2024

  28. [36]

    Trusting your evidence: Hallucinate less with context-aware decoding

    Weijia Shi, Xiaochuang Han, Mike Lewis, Yulia Tsvetkov, Luke Zettlemoyer, and Wen-tau Yih. Trusting your evidence: Hallucinate less with context-aware decoding. In Kevin Duh, Helena Gomez, and Steven Bethard (eds.), Proceedings of the 2024 Conference of the North American Chap...

    2024 doi

  29. [37]

    The curious case of hallucinatory (un)answerability: Finding truths in the hidden states of over-confident large language models

    Aviv Slobodkin, Omer Goldman, Avi Caciularu, Ido Dagan, and Shauli Ravfogel. The curious case of hallucinatory (un)answerability: Finding truths in the hidden states of over-confident large language models. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of th...

    2023 doi

  30. [38]

    V eri S core: Evaluating the factuality of verifiable claims in long-form text generation

    Yixiao Song, Yekyung Kim, and Mohit Iyyer. V eri S core: Evaluating the factuality of verifiable claims in long-form text generation. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Findings of the Association for Computational Linguistics: EMNLP 2024, pp.\ 9447--...

    2024 doi

  31. [39]

    M ini C heck: Efficient fact-checking of LLM s on grounding documents

    Liyan Tang, Philippe Laban, and Greg Durrett. M ini C heck: Efficient fact-checking of LLM s on grounding documents. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp.\ 8818--...

    2024 doi

  32. [40]

    Fine-tuning language models for factuality

    Katherine Tian, Eric Mitchell, Huaxiu Yao, Christopher Manning, and Chelsea Finn. Fine-tuning language models for factuality. In NeurIPS 2023 Workshop on Instruction Tuning and Instruction Following, 2023 a . URL https://openreview.net/forum?id=kEK08VdSO5

    2023

  33. [41]

    Just ask for calibration: Strategies for eliciting calibrated confidence scores from language models fine-tuned with human feedback

    Katherine Tian, Eric Mitchell, Allan Zhou, Archit Sharma, Rafael Rafailov, Huaxiu Yao, Chelsea Finn, and Christopher Manning. Just ask for calibration: Strategies for eliciting calibrated confidence scores from language models fine-tuned with human feedback. In Houda Bouamor, ...

    2023

  34. [42]

    Llama 2: Open foundation and fine-tuned chat models

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. ArXiv, 2023. URL https://arxiv.org/abs/2307.09288

    2023 arXiv

  35. [43]

    A stitch in time saves nine: Detecting and mitigating hallucinations of llms by validating low-confidence generation, 2023

    Neeraj Varshney, Wenlin Yao, Hongming Zhang, Jianshu Chen, and Dong Yu. A stitch in time saves nine: Detecting and mitigating hallucinations of llms by validating low-confidence generation, 2023. URL https://arxiv.org/abs/2307.03987

    2023 arXiv

  36. [44]

    Factuality of large language models: A survey

    Yuxia Wang, Minghan Wang, Muhammad Arslan Manzoor, Fei Liu, Georgi Nenkov Georgiev, Rocktim Jyoti Das, and Preslav Nakov. Factuality of large language models: A survey. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Proceedings of the 2024 Conference on Empirical...

    2024 doi

  37. [45]

    Redpajama: an open dataset for training large language models

    Maurice Weber, Daniel Y Fu, Quentin Gregory Anthony, Yonatan Oren, Shane Adams, Anton Alexandrov, Xiaozhong Lyu, Huu Nguyen, Xiaozhe Yao, Virginia Adams, Ben Athiwaratkun, Rahul Chalamala, Kezhen Chen, Max Ryabinin, Tri Dao, Percy Liang, Christopher Re, Irina Rish, and Ce Zhan...

    2024

  38. [46]

    Can LLM s express their uncertainty? an empirical evaluation of confidence elicitation in LLM s

    Miao Xiong, Zhiyuan Hu, Xinyang Lu, YIFEI LI, Jie Fu, Junxian He, and Bryan Hooi. Can LLM s express their uncertainty? an empirical evaluation of confidence elicitation in LLM s. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview....

    2024

  39. [47]

    Alignment for honesty

    Yuqing Yang, Ethan Chern, Xipeng Qiu, Graham Neubig, and Pengfei Liu. Alignment for honesty. ArXiv, 2023. URL https://arxiv.org/abs/2312.07000

    2023 arXiv

  40. [48]

    R -tuning: Instructing large language models to say ` I don ' t know '

    Hanning Zhang, Shizhe Diao, Yong Lin, Yi Fung, Qing Lian, Xingyao Wang, Yangyi Chen, Heng Ji, and Tong Zhang. R -tuning: Instructing large language models to say ` I don ' t know ' . In Kevin Duh, Helena Gomez, and Steven Bethard (eds.), Proceedings of the 2024 Conference of t...

    2024

  41. [49]

    Muru Zhang, Ofir Press, William Merrill, Alisa Liu, and Noah A. Smith. How language model hallucinations can snowball. In Forty-first International Conference on Machine Learning, 2024 b . URL https://openreview.net/forum?id=FPlaQyAGHu

    2024

  42. [50]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  43. [51]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  44. [52]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  45. [53]

    Lehrbuch der Geologie

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.