Pith. sign in

REVIEW 5 major objections 5 minor 40 references

From Parameters to Prompts: Understanding and Mitigating the Factuality Gap between Fine-Tuned LLMs

T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that the test-time prompt can shrink or even reverse the factuality gap caused by fine-tuning on known versus unknown knowledge.

desk verdict Solid empirical observation about the fragility of the fine-tuning factuality gap, wrapped in a graph-theoretic 'proof' that assumes its own conclusion. read the letter →

arxiv 2505.23410 v1 pith:ZGZYN5KF submitted 2025-05-29 cs.CL

Xuan Gong , Hanbo Huang , Shiyu Liang This is my paper
classification cs.CL
keywords factualitygapin-contextlearningchain-of-thoughtknowledgegraphsfine-tuningdataextractionselectionevaluationLLM
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the accuracy gap between an LLM fine-tuned on knowledge it already knows and one fine-tuned on knowledge it does not know is not fixed at training time: the prompt used at test time can shrink it, eliminate it, or even reverse it. The authors show this empirically on 8B and 7B models across question answering, open-ended generation, and a controlled toy setting, and they prove the effect on a knowledge-graph model of how fine-tuning stores facts. In that model, fine-tuning on a known fact adds many edges between embedding-similar subject and answer nodes, fine-tuning on an unknown fact adds only one, and an in-context prompt injects its own edges, so the gap in connectivity, and hence in accuracy, narrows. The practical stakes are that in-context learning can compensate for limited or low-quality fine-tuning data, and that evaluations of fine-tuning data selection that rely on few-shot or chain-of-thought prompting may be measuring the prompt rather than the data.

What carries the argument

The central object is the relation-specific knowledge graph $G_r=(V,E_r,E^{\mathrm{sim}})$: entity tokens are nodes, an explicit edge $(v_s,v_a)$ records that the model answers $a$ to $(s,r)$, and similarity edges connect tokens whose fixed embeddings are close in Euclidean distance. Supervised fine-tuning is formalized as an edge-completion process (Lemma 4.1): training on $(s,r,a)$ adds edges between the similarity neighborhoods of $s$ and $a$, so known knowledge, whose subject and answer nodes sit in dense neighborhoods, induces more new edges than unknown knowledge. The mechanism that carries the mitigation argument is prompt-induced subgraph injection: a few-shot demonstration set $\Pi=\{(s'_i,r,a'_i)\}$ forms an auxiliary graph $G_\Pi$, and a chain-of-thought reasoning chain $C=\{(s,r_i,a_i)\}$ forms a support graph $G_C$, so inference happens on the augmented graph $G^\star=G\cup G_\Pi\cup G_C$; the factuality gap shrinks because the added edges help the sparser unknown-knowledge graph more, provided the prompt overlaps more with the known graph's edges than with the unknown graph's edges.

What would settle it

Fine-tune the same base model on known and unknown splits, fix a demonstration set with high embedding overlap with unknown-knowledge subjects and low overlap with known ones, and compare the factuality gap with and without that prompt; Theorem 5.1 predicts the gap always shrinks, so observing a prompt that widens the gap, or that shrinks it less than a prompt with lower overlap, would falsify the mechanism.

Watch

Extended reading notes

Core claim

The paper's central claim is stated as Theorem 5.1: under prompt-augmented inference the factuality gap satisfies $\Delta^*_{\mathrm{fact}} < \Delta_{\mathrm{fact}}$. The authors model fine-tuning as an edge-completion process on a latent knowledge graph $G_r=(V,E_r,E^{\mathrm{sim}})$ per relation: a one-hop edge $(v_s,v_a)$ exists exactly when the model, given subject $s$ and relation $r$, predicts answer $a$. Known knowledge corresponds to high-degree neighborhoods, so fine-tuning on it adds many edges; unknown knowledge adds a single edge, giving $\Delta_{\mathrm{fact}} \propto |E_{\mathrm{kn}}|-|E_{\mathrm{unk}}| > 0$. A few-shot or chain-of-thought prompt is treated as an auxiliary graph $G_\Pi$ (or reasoning chain $G_C$) injected at inference, yielding the augmented graph $G^\star = G \cup G_\Pi \cup G_C$, and, under an overlap condition $|E_\Pi\cap E_{\mathrm{kn}}|>|E_\Pi\cap E_{\mathrm{unk}}|$, the proof shows the gap shrinks because the prompt adds more new edges to the unknown-knowledge graph than to the known-knowledge graph; when chain-of-thought covers the test set, the gap can fall to zero. Empirically this shows up as larger gains on the unknown split than on the known split, and as near-vanishing gaps when the test distribution is semantically distant from the training distribution.

Load-bearing premise

The theory assumes that a highly simplified version of a transformer, with word representations fixed during fine-tuning, and the idea that few-shot examples make the model copy the demonstrated answer pattern, describe how real large language models retrieve knowledge; if either fails for 8B-to-70B models, the proved gap-narrowing result does not apply.

Editorial extensions

If this is right

  • A model fine-tuned on only 5% of a training set can reach the accuracy of a fully fine-tuned model when evaluated with in-context prompts.
  • Carefully chosen few-shot examples and chain-of-thought reasoning narrow the factuality gap more on early-stopped and instruction-tuned models than on fully converged base models.
  • Factuality gaps persist under near in-distribution shifts but nearly disappear under strong semantic shifts, so OOD evaluation alone can hide the effect of fine-tuning data.
  • Benchmarks that evaluate fine-tuning data selection methods with few-shot prompting may overstate or mask the value of the data itself, because the prompt can dominate extraction.
  • Chain-of-thought can bypass mappings learned during fine-tuning: on the toy perturbed-entity task, CoT raised unknown-set accuracy from about 42% to 83%, nearly eliminating the gap.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not test this, but its result implies single-prompt benchmark comparisons of fine-tuned models are confounded: varying the prompt would reveal how much of the score is data and how much is prompt.
  • The graph account has a directly checkable consequence: measure the embedding overlap between a demonstration set and known versus unknown knowledge graphs; if a helpful-looking prompt does not satisfy $|E_\Pi \cap E_{\mathrm{kn}}| > |E_\Pi \cap E_{\mathrm{unk}}|$, the proof's key inequality fails.
  • A cheaper route to factuality for knowledge-intensive tasks may be test-time demonstration injection rather than data curation, at least for relational QA with available examples.
  • The vanishing-gap prediction under distribution shift suggests a diagnostic experiment the paper does not run: interpolate semantic distance and watch the gap; monotone decay supports the graph story, a sharp discontinuity would point to a different mechanism.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper studies the factuality gap between LLMs fine-tuned on known versus unknown factual knowledge, asking whether the gap can be understood, mitigated, and exploited. The authors run fine-tuning experiments on Llama-3.1-8B and Mistral-7B-v0.3 variants across QA and open-ended generation tasks, report that the gap shrinks under out-of-distribution evaluation, and show that few-shot and chain-of-thought prompting often narrow the gap. They then propose a graph-theoretic model in which fine-tuning adds edges to a knowledge graph and ICL injects an auxiliary graph, and they claim to prove that prompt-augmented inference strictly reduces the factuality gap. The paper concludes that ICL can compensate for low-quality or small fine-tuning data and that ICL-based evaluations of data-selection methods may misrepresent the true contribution of the fine-tuning data.

Significance. The target phenomenon is important: if test-time prompting can overshadow the effect of fine-tuning data on factual knowledge extraction, then both model evaluation and data-selection methodology need to be reconsidered. The paper's empirical scope is substantial, covering two model families, base and instruction variants, four datasets, and two task formats, and the graph edge-completion perspective is a useful organizing metaphor. The authors also include an explicit Limitations section, which is a strength. However, the significance is currently capped by three problems: the headline theorem is not actually proved because its key inequality is assumed; the main mitigation result is based on post-hoc prompt selection without variance estimates; and the OOD conclusion rests on a confounded comparison. If these are repaired, the empirical finding would be a solid contribution even if the theoretical claim is downgraded to a conditional formal model.

major comments (5)
  1. [Appendix A.5 / Theorem 5.1] The proof of Theorem 5.1 does not establish the claimed strict inequality. In the few-shot case, the proof reduces the augmented gap to lambda times (|Ekn| - |Eunk| - (|Ekn intersect EP| - |Eunk intersect EP|)) and then asserts that 'there exists a sufficiently large subgraph GP such that |E_Pi intersect Ekn| > |E_Pi intersect Eunk|' without deriving this inequality from Assumption A.1, the one-layer model of Section 3.2, or any definition of a 'valid' prompt. This inequality is exactly the condition needed for the conclusion Delta*_fact < Delta_fact, so the theorem is either circular or conditional on an unproven overlap assumption. The CoT half of the proof switches from the edge-count definition of the gap to a coverage-based definition, and the term 'valid' is never formally defined. Since Section 6.2's recommendation about ICL-based evaluation leans directly on this theorem, the theoretical contribution needs either a real derivation from stated assumptions or an explicit conditional restatement.
  2. [Section 5.1 / Table 3] The main mitigation result is reported after selecting, for each model and dataset, 'the prompt set yielding the highest performance on the Unknown model' among three prompt sets. This post-hoc selection biases the Unknown gains upward, and no variance estimates or multiple seeds are reported. For example, the EQ Unknown improvement of +13.3 for Llama at early stop could reflect prompt-selection noise. To support the claim that ICL narrows the factuality gap, the authors should pre-specify the prompt-selection rule, validate it on held-out data, and report run-to-run error bars.
  3. [Section 4.2 / Table 2] The claim that the factuality gap 'vanishes under strong distribution shifts' is inferred from the mmlu_ood set, which differs from the EntityQuestions training and ID test data in several confounded ways: task format (multiple choice versus free-form QA), prompt template, answer distribution, and domain. The cosine-similarity value of 0.55 therefore does not isolate distributional distance from format and domain shift. A same-format OOD set with low similarity, or a template-only perturbation of the ID set, is needed to support the specific claim that the gap decays with semantic distance rather than with format change.
  4. [Section 7 / Table 4] The Toy Example CoT prompt contains an explicit instruction to 'Ignore all the special characters in the following question' and to 'clean all special characters in the question.' The large gap reduction on perturbed entities (Unknown from 44.73 to 84.08 at early stop) is therefore largely the result of directly specifying the inverse mapping of the perturbation, not of general chain-of-thought reasoning. This makes the discussion's conclusion that CoT 'bypasses' the fine-tuning mapping an instruction-following effect, and it does not support the broader claim about reasoning prompts. The experiment should be reframed as a direct-instruction ablation, or rerun with a CoT that does not explicitly name the perturbation rule.
  5. [Section 8 Limitations / Section 5.1 Obs. 3] The Limitations section candidly notes that the framework 'does not fully explain the anomalous behavior observed on datasets such as MMLU and WikiBios,' but those are precisely the datasets where Table 3 shows few-shot or CoT sometimes increasing the factuality gap. The general conclusion that ICL 'can mitigate or even fully eliminate' the gap is therefore in tension with the admitted anomalies. The paper needs a positive characterization of when ICL helps (for example, by task format, prompt length, or fine-tuning stage), not just a caveat, before the broad claim can be accepted.
minor comments (5)
  1. [Section 5 heading] The heading contains a typo: 'Can Fatcuality Gap be Easily Mitigated?' should read 'Can Factuality Gap be Easily Mitigated?'.
  2. [Table 3 caption] The caption says underlined results are few-shot and non-underlined are few-shot CoT, but the table as rendered does not visibly distinguish the two conditions; please clarify with explicit columns or a clear legend.
  3. [Appendix A.5 / CoT proof] In the CoT part of the proof, the definition of the gap changes from the edge-count difference used in Theorem 5.1 to a coverage-based difference; both notions should be defined separately and the comparison between them made explicit.
  4. [Tables 1 and 2] The split label 'Unknow' appears instead of 'Unknown' in several places; please standardize the terminology.
  5. [Appendix D] Tables 5, 6, and 7 are not referenced in the main text and their selection criteria (best of three sets for Unknown examples) are described only after the tables; moving the selection description into Section 5.2 would improve reproducibility.

Circularity Check

2 steps flagged · score 6.0 of 10

Theorem 5.1's proof assumes the overlap inequality it needs to prove; the 'valid prompt' is effectively defined by the conclusion, so the claimed theoretical proof is circular while the empirical measurements are independent.

  1. self definitional [Theorem 5.1 and Appendix A.5]
    "Since GΠ, Gunk, Gkn ⊆ Gr, with a knowledge prompt that has enough semantic connection with the in-distribution data, there exists a sufficiently large subgraph GP such that |EΠ ∩ Ekn| > |EΠ ∩ Eunk|."

    This inequality is exactly the condition that makes Δ*fact = λ(|Ekn| − |Eunk| − (|Ekn ∩ EP| − |Eunk ∩ EP|)) smaller than Δfact. The theorem quantifies over 'a valid in-context prompt', but 'valid' is never defined. In the proof, the needed inequality is simply asserted as existence of a 'sufficiently large subgraph' with 'enough semantic connection', not derived from Lemma 4.1, Assumption A.1, or the graph-augmentation formalism. Prompts whose overlap with the known graph exceeds overlap with the unknown graph are precisely prompts that narrow the gap; assuming such a prompt exists is assuming the theorem's conclusion. The proof therefore reduces to a restatement of the desired gap-narrowing property rather than a derivation of it.

  2. self definitional [Appendix A.5, CoT case]
    "But for every test triple (s, r, a) that is explained by the CoT prompt, it is covered by both augmented graphs. Therefore, its contribution to the gap is 1−1 = 0. Any remaining gap can only come from test triples not supported by CoT."

    The CoT half assumes the very phenomenon to be proved: that the same CoT prompt 'explains' the same test triples for both the known-finetuned and unknown-finetuned graphs. The conclusion Δ*fact ≤ Δfact follows only if covered triples are correct under both augmented graphs, but coverage by an added support graph does not by itself imply equality of predictions unless one assumes the prompt overrides the fine-tuning difference identically. 'Explained by the CoT prompt' is a relabeling of 'the gap has been eliminated on those triples'. No mechanism is provided for why a single added graph makes both models output the same answer, so the CoT proof is circular rather than derived from the one-layer model.

full rationale

The empirical core of the paper is self-contained: Tables 1, 2, 3, 5, 6, and 7 report actual factuality gaps and ICL/CoT effects on Llama-3.1-8B and Mistral-7B-v0.3, and these measurements are not derived from the theory. However, the paper's headline theoretical claim — 'We prove this phenomenon theoretically' — rests on Theorem 5.1, whose proof in Appendix A.5 inserts an unproven overlap assumption |EΠ ∩ Ekn| > |EΠ ∩ Eunk|. That assumption is precisely the condition needed for Δ*fact < Δfact, so the theorem is a formal restatement of the mitigation effect rather than a derivation of it from the one-layer attention model. The CoT half is similarly assumptive: it declares that CoT-covered triples are covered by both augmented graphs, which is the gap-narrowing outcome itself. The term 'valid prompt' is never defined; if it is intended to include the overlap inequality, the theorem becomes true by definition. No self-citation chain is load-bearing here; the circularity is internal to the proof. The empirical results would stand independently even if Theorem 5.1 were removed, but the claimed proof of prompt dominance over fine-tuning data does reduce by construction to the assumed overlap condition. Score 6: partial circularity, with the central proof claim affected while the experiments remain independent.

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

The central claims rest on multiple unproven premises: a toy one-layer model, an ICL attention-routing assumption, a uniform-edge distribution, an assumption that an external embedding approximates the LLM's internal geometry, and an overlap condition in Theorem 5.1 that is effectively the conclusion. The single free parameter ε is unspecified. These premises limit the theoretical contribution to a heuristic story rather than a proof about real LLMs.

free parameters (1)
  • epsilon (similarity threshold)
    The definition of similarity edges E_sim uses a threshold ε on embedding distance (Section 4.3); no value is ever assigned, and the graph-theoretic predictions are qualitative in ε. It determines which entity pairs count as connected, so it shapes the model's predictions.
assumptions (5)
  • ad hoc to paper A one-layer transformer with fixed non-orthogonal embeddings faithfully models how fine-tuning and ICL affect factual recall in real LLMs.
    The entire theoretical section (Section 3.2, Appendix A) is developed for this toy architecture; the paper assumes the results transfer to Llama-3.1-8B, Mistral-7B, and Llama-3.3-70B without validation.
  • domain assumption Assumption A.1: few-shot prompting makes the attention mechanism guide the query to follow demonstration patterns in predicting the answer.
    Explicitly stated in Appendix A.3 and used to argue that known facts correspond to large similarity neighborhoods and to prove Theorem 4.1's supporting claim. Justified by citations to prior ICL work, not proven here.
  • ad hoc to paper Edges under a relation r are uniformly distributed over all token pairs in the knowledge graph.
    Used in the proof of Theorem 4.1 to obtain Pr((v_s,v_a) in E_r) = |E_r|/|V|^2 (Appendix A.3). A strong simplification that is not true for real knowledge graphs.
  • domain assumption Cosine similarity computed with the external all-MiniLM-L6-v2 embedding model approximates the LLM's internal semantic similarity.
    Remark 3 acknowledges this assumption; needed to make Theorem 4.2's prediction empirically testable.
  • ad hoc to paper For a CoT prompt that 'explains' a test triple, both the known-fine-tuned and unknown-fine-tuned models answer that triple correctly.
    The second half of Theorem 5.1's proof (Appendix A.5) assumes CoT coverage erases the gap for covered triples; this is equivalent to asserting the effect the theorem claims to prove.

how reviews work

0 comments
Cite this review

Pith. "Pith review of From Parameters to Prompts: Understanding and Mitigating the Factuality Gap between Fine-Tuned LLMs." pith.science (2026). https://pith.science/paper/ZGZYN5KF

@misc{pith2026250523410,
  author       = {Pith},
  title        = {Pith review of: From Parameters to Prompts: Understanding and Mitigating the Factuality Gap between Fine-Tuned LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZGZYN5KF}},
  note         = {Machine review of arXiv:2505.23410}
}
read the original abstract

Factual knowledge extraction aims to explicitly extract knowledge parameterized in pre-trained language models for application in downstream tasks. While prior work has been investigating the impact of supervised fine-tuning data on the factuality of large language models (LLMs), its mechanism remains poorly understood. We revisit this impact through systematic experiments, with a particular focus on the factuality gap that arises when fine-tuning on known versus unknown knowledge. Our findings show that this gap can be mitigated at the inference stage, either under out-of-distribution (OOD) settings or by using appropriate in-context learning (ICL) prompts (i.e., few-shot learning and Chain of Thought (CoT)). We prove this phenomenon theoretically from the perspective of knowledge graphs, showing that the test-time prompt may diminish or even overshadow the impact of fine-tuning data and play a dominant role in knowledge extraction. Ultimately, our results shed light on the interaction between finetuning data and test-time prompt, demonstrating that ICL can effectively compensate for shortcomings in fine-tuning data, and highlighting the need to reconsider the use of ICL prompting as a means to evaluate the effectiveness of fine-tuning data selection methods.

Figures

Figures reproduced from arXiv: 2505.23410 by the authors.

Figure 1
Figure 1. Overview: In-context learning (ICL) prompts [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Memorizing a known knowledge triple (s0, r, a0) generalizes to memorizing (s1, r, a1) but memorizing an unknown knowledge triple (s2, r, a2) can not generalize. isfies E[Zkn − Zunk] = ∆fact ∝ |Ekn| − |Eunk| > 0. Remark 2. Theorem 4.1 interprets the factuality gap as a direct consequence of differences in one￾hop connectivity induced by fine-tuning. Under uniform sampling assumptions, the expected gap in one-hop accu… view at source ↗
Figure 5
Figure 5. In an LLM fine-tuned on unknown knowledge [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Ablation study of prompt formulation. We use [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 6
Figure 6. Figure 6: Comparison between Llama-3.1-8B and Mistral-7B-v0.3 models fine-tuned on 5% of dataset and the whole dataset. 7 [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

40 extracted references · 14 canonical work pages

  1. [1]

    Ekin Aky \"u rek, Dale Schuurmans, Jacob Andreas, Tengyu Ma, and Denny Zhou. 2023. https://openreview.net/forum?id=0g0X4H8yN4I What learning algorithm is in-context learning? investigations with linear models . In The Eleventh International Conference on Learning Representations

    work page 2023

  2. [2]

    Zeyuan Allen-Zhu and Yuanzhi Li. 2024. https://arxiv.org/abs/2309.14316 Physics of language models: Part 3.1, knowledge storage and extraction . Preprint, arXiv:2309.14316

    arXiv 2024

  3. [3]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin...

    arXiv 2020

  4. [4]

    Banghao Chen, Zhaofeng Zhang, Nicolas Langrené, and Shengxin Zhu. 2024. https://arxiv.org/abs/2310.14735 Unleashing the potential of prompt engineering in large language models: a comprehensive review . Preprint, arXiv:2310.14735

    arXiv 2024

  5. [5]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    arXiv 2024

  6. [6]

    Zorik Gekhman, Gal Yona, Roee Aharoni, Matan Eyal, Amir Feder, Roi Reichart, and Jonathan Herzig. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.444 Does fine-tuning LLM s on new knowledge encourage hallucinations? In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 7765--7784, Miami, Florida, USA. Associati...

    doi:10.18653/v1/2024.emnlp-main.444 2024

  7. [7]

    Gaurav Rohit Ghosal, Tatsunori Hashimoto, and Aditi Raghunathan. 2024. https://openreview.net/forum?id=cPsn9AcOYh Understanding finetuning for factual knowledge extraction . In Forty-first International Conference on Machine Learning

    work page 2024

  8. [8]

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300

    arXiv 2020

Show all 40 references
  1. [9]

    Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. 2025. https://doi.org/10.1145/3703155 A survey on hallucination in large language models: Principles, taxonomy, challenges, and o...

    2025 doi

  2. [10]

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas ...

    2023 arXiv

  3. [11]

    Nitish Joshi, Javier Rando, Abulhair Saparov, Najoung Kim, and He He. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.364 Personas as a way to model truthfulness in language models . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, p...

    2024 doi

  4. [12]

    Nikhil Kandpal, Haikang Deng, Adam Roberts, Eric Wallace, and Colin Raffel. 2023. https://proceedings.mlr.press/v202/kandpal23a.html Large language models struggle to learn long-tail knowledge . In Proceedings of the 40th International Conference on Machine Learning, volume 20...

    2023

  5. [13]

    Katie Kang, Eric Wallace, Claire Tomlin, Aviral Kumar, and Sergey Levine. 2024. https://openreview.net/forum?id=5H5IQuTlMz Unfamiliar finetuning examples control how language models hallucinate . In Automated Reinforcement Learning: Exploring Meta-Learning, AutoML, and LLMs

    2024

  6. [14]

    Mehran Kazemi, Sid Mittal, and Deepak Ramachandran. 2023. https://arxiv.org/abs/2301.11293 Understanding finetuning for factual knowledge extraction from language models . Preprint, arXiv:2301.11293

    2023 arXiv

  7. [15]

    Bill Yuchen Lin, Abhilasha Ravichander, Ximing Lu, Nouha Dziri, Melanie Sclar, Khyathi Chandu, Chandra Bhagavatula, and Yejin Choi. 2024 a . https://openreview.net/forum?id=wxJ0eXwwda The unlocking spell on base LLM s: Rethinking alignment via in-context learning . In The Twel...

    2024

  8. [16]

    Sheng-Chieh Lin, Luyu Gao, Barlas Oguz, Wenhan Xiong, Jimmy Lin, Wen-tau Yih, and Xilun Chen. 2024 b . Flame: Factuality-aware alignment for large language models. arXiv preprint arXiv:2405.01525

    2024 arXiv

  9. [17]

    Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. https://doi.org/10.18653/v1/2022.acl-long.229 T ruthful QA : Measuring how models mimic human falsehoods . In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pa...

    2022 doi

  10. [18]

    Haochen Liu, Song Wang, Yaochen Zhu, Yushun Dong, and Jundong Li. 2024 a . https://doi.org/10.18653/v1/2024.findings-acl.376 Knowledge graph-enhanced large language models via path selection . In Findings of the Association for Computational Linguistics: ACL 2024, pages 6311--...

    2024 doi

  11. [19]

    Zifan Liu, Amin Karbasi, and Theodoros Rekatsinas. 2024 b . https://openreview.net/forum?id=wjbTHLUSzU TSDS : Data selection for task-specific model finetuning . In The Thirty-eighth Annual Conference on Neural Information Processing Systems

    2024

  12. [20]

    Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. https://doi.org/10.18653/v1/2023.acl-long.546 When not to trust language models: Investigating effectiveness of parametric and non-parametric memories . In Proceedings of the 6...

    2023 doi

  13. [21]

    Kevin Meng, David Bau, Alex J Andonian, and Yonatan Belinkov. 2022. https://openreview.net/forum?id=-h6WAS6eE4 Locating and editing factual associations in GPT . In Advances in Neural Information Processing Systems

    2022

  14. [22]

    Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.741 FA ct S core: Fine-grained atomic evaluation of factual precision in long form text generatio...

    2023 doi

  15. [23]

    Fabio Petroni, Aleksandra Piktus, Angela Fan, Patrick Lewis, Majid Yazdani, Nicola De Cao, James Thorne, Yacine Jernite, Vladimir Karpukhin, Jean Maillard, Vassilis Plachouras, Tim Rockt \"a schel, and Sebastian Riedel. 2021. https://doi.org/10.18653/v1/2021.naacl-main.200 KIL...

    2021 doi

  16. [24]

    Fabio Petroni, Tim Rockt \"a schel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019 a . https://doi.org/10.18653/v1/D19-1250 Language models as knowledge bases? In Proceedings of the 2019 Conference on Empirical Methods in Natural Languag...

    2019 doi

  17. [25]

    Miller, and Sebastian Riedel

    Fabio Petroni, Tim Rocktäschel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, Alexander H. Miller, and Sebastian Riedel. 2019 b . https://arxiv.org/abs/1909.01066 Language models as knowledge bases? Preprint, arXiv:1909.01066

    2019 arXiv

  18. [26]

    Nils Reimers and Iryna Gurevych. 2019. https://arxiv.org/abs/1908.10084 Sentence-bert: Sentence embeddings using siamese bert-networks . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics

    2019 arXiv

  19. [27]

    Christopher Sciavolino, Zexuan Zhong, Jinhyuk Lee, and Danqi Chen. 2021. Simple entity-centric questions challenge dense retrievers. arXiv preprint arXiv:2109.08535

    2021 arXiv

  20. [28]

    Marco Antonio Stranisci, Rossana Damiano, Enrico Mensa, Viviana Patti, Daniele Radicioni, and Tommaso Caselli. 2023. https://doi.org/10.18653/v1/2023.acl-long.691 W iki B io: a semantic resource for the intersectional analysis of biographical events . In Proceedings of the 61s...

    2023 doi

  21. [29]

    Jiabin Tang, Yuhao Yang, Wei Wei, Lei Shi, Lixin Su, Suqi Cheng, Dawei Yin, and Chao Huang. 2024. https://doi.org/10.1145/3626772.3657775 Graphgpt: Graph instruction tuning for large language models . In SIGIR, pages 491--500

    2024

  22. [30]

    Johannes Von Oswald, Eyvind Niklasson, Ettore Randazzo, Joao Sacramento, Alexander Mordvintsev, Andrey Zhmoginov, and Max Vladymyrov. 2023. https://proceedings.mlr.press/v202/von-oswald23a.html Transformers learn in-context by gradient descent . In Proceedings of the 40th Inte...

    2023

  23. [31]

    Jianing Wang, Chengyu Wang, Chuanqi Tan, Jun Huang, and Ming Gao. 2024 a . https://doi.org/10.18653/v1/2024.findings-naacl.207 Knowledgeable in-context tuning: Exploring and exploiting factual knowledge for in-context learning . In Findings of the Association for Computational...

    2024 doi

  24. [32]

    Zhihu Wang, Shiwan Zhao, Yu Wang, Heyuan Huang, Sitao Xie, Yubo Zhang, Jiaxin Shi, Zhixing Wang, Hongyan Li, and Junchi Yan. 2024 b . https://arxiv.org/abs/2408.06904 Re-task: Revisiting llm tasks from capability, skill, and knowledge perspectives . Preprint, arXiv:2408.06904

    2024 arXiv

  25. [33]

    Jason Wei, Maarten Paul Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew Mingbo Dai, and Quoc V. Le. 2022 a . https://openreview.net/forum?id=gEZrGCozdqR Finetuned language models are zero-shot learners

    2022

  26. [34]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed Chi, Quoc V Le, and Denny Zhou. 2022 b . https://proceedings.neurips.cc/paper_files/paper/2022/file/9d5609613524ecf4f15af0f7b31abca4-Paper-Conference.pdf Chain-of-thought prompting elicits reason...

    2022

  27. [35]

    Qinyuan Wu, Mohammad Aflah Khan, Soumi Das, Vedant Nanda, Bishwamittra Ghosh, Camila Kolling, Till Speicher, Laurent Bindschaedler, Krishna Gummadi, and Evimaria Terzi. 2025. https://doi.org/10.1145/3701551.3703562 Towards reliable latent knowledge estimation in llms: Zero-pro...

    2025

  28. [36]

    Mengzhou Xia, Sadhika Malladi, Suchin Gururangan, Sanjeev Arora, and Danqi Chen. 2024. LESS : Selecting influential data for targeted instruction tuning. In International Conference on Machine Learning (ICML)

    2024

  29. [37]

    Linyi Yang, Shuibai Zhang, Zhuohao Yu, Guangsheng Bao, Yidong Wang, Jindong Wang, Ruochen Xu, Wei Ye, Xing Xie, Weizhu Chen, and Yue Zhang. 2024. https://openreview.net/forum?id=bAMPOUF227 Supervised knowledge makes large language models better in-context learners . In The Twe...

    2024

  30. [38]

    Xunjian Yin, Xu Zhang, Jie Ruan, and Xiaojun Wan. 2024. https://doi.org/10.18653/v1/2024.acl-long.124 Benchmarking knowledge boundary for large language models: A different perspective on model evaluation . In Proceedings of the 62nd Annual Meeting of the Association for Compu...

    2024 doi

  31. [39]

    Tong Zhang, Peixin Qin, Yang Deng, Chen Huang, Wenqiang Lei, Junhong Liu, Dingnan Jin, Hongru Liang, and Tat-Seng Chua. 2024. https://doi.org/10.18653/v1/2024.acl-long.578 CLAMBER : A benchmark of identifying and clarifying ambiguous information needs in large language models ...

    2024 doi

  32. [40]

    Ce Zheng, Lei Li, Qingxiu Dong, Yuxuan Fan, Zhiyong Wu, Jingjing Xu, and Baobao Chang. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.296 Can we edit factual knowledge by in-context learning? In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Pro...

    2023 doi

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.