Let your LLM generate a few tokens and you will reduce the need for retrieval

Herv\'e D\'ejean

REVIEW 4 major objections 5 minor 29 references

Let your LLM generate a few tokens and you will reduce the need for retrieval

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Training an LLM to output an 'I Know' score—distilled from an LLM-as-judge and fed the first 32 tokens of the model's own answer—lets a RAG pipeline skip retrieval on more than half of queries on some datasets while matching or beating…

desk verdict Useful empirical mapping of when-to-skip retrieval, but the shared judge for labels and evaluation makes the headline retrieval-reduction claim unproven. read the letter →

arxiv 2412.11536 v1 pith:QDXS5SC4 submitted 2024-12-16 cs.CL

Hervé Déjean This is my paper · ORCID

classification cs.CL

keywords IKnowscoreLLM-as-a-judgeretrieval-augmentedgenerationwhentoretrieveparametricmemorydistillationquestionansweringconfidenceestimation

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether a large language model can be trained to know what it knows: specifically, whether it can answer a question correctly without retrieving external documents. The author's answer is yes—an LLM fine-tuned to predict an 'I Know' (IK) score, with labels produced by an LLM-as-judge, reaches around 80% accuracy on the detection task. When the first 32 tokens of the model's own answer are fed in, the classifier becomes accurate enough that a retrieval-augmented generation (RAG) pipeline can skip retrieval for more than half of queries on some datasets while keeping answer quality equal or slightly better. The claim matters because RAG's main costs—retrieval, reranking, and long prompts—are only worth paying when the parametric memory actually needs the help.

What carries the argument

The IK (I Know) score is the central object: after fine-tuning on judge-generated Yes/No labels, the score is the softmax probability of the 'Yes' token taken over only the Yes and No logits. The second mechanism is the input format—the question alone versus the question plus the first $k$ tokens of the model's own generated answer. The $k=32$ prefix is the operational sweet spot: it lifts IK accuracy by 4–5 points over question-only input and cuts the required training data to about 20K samples. The score does the work by acting as a gate: when it exceeds a chosen threshold, the retriever and reranker are skipped, so the claim stands or falls on whether this gate reliably separates queries the model can answer from those it cannot.

What would settle it

Train the IK classifier with SOLAR as judge, then score the final answers on the NQ test set with a different, stricter judge (or human raters); if the retrieval-skipping threshold that worked under SOLAR now lowers answer quality, the classifier was optimizing for judge leniency rather than for genuine parametric knowledge. Additionally, test on recent-fact questions outside the model's knowledge cutoff; if the classifier marks many as 'I know', it is not actually detecting stored knowledge.

Watch

Extended reading notes

Core claim

The paper's central discovery is that the 'when to retrieve' decision for a RAG system can be distilled into a small classifier: take the LLM, fine-tune it to output a binary 'I Know' label using silver labels produced by an LLM-as-a-judge, and feed it the first 32 tokens of the model's own generated answer. This yields an IK classifier with 80–83% accuracy and AUC up to 0.92 on KILT NQ. With a suitable threshold, the classifier lets a RAG pipeline skip retrieval on 29–50% of queries (depending on model and threshold) while achieving LLMEval scores equal to or up to 2 points above the full-RAG baseline. The author also finds the IK score can characterize datasets—TriviaQA and SCIQ are mostly answered from parametric memory, PopQA and HotpotQA mostly require retrieval, and NQ/ASQA are mixed—and that roughly 20,000 training samples suffice when the 32-token prefix is included.

Load-bearing premise

The whole pipeline rests on the assumption that the same judge model, SOLAR, can reliably label both whether the LLM knows an answer and whether a final answer is good; if that judge is lenient or biased, the classifier may be learning to match the judge's taste instead of detecting genuine knowledge.

Editorial extensions

If this is right

For datasets where the model is largely confident (e.g., TriviaQA, SCIQ), the IK classifier can cut retrieval to 7–27% of queries with no quality loss.
Adding the first 32 generated answer tokens improves IK accuracy by 4–5 points over question-only input, and reduces the required training data from roughly 40K to 20K samples.
The distillation works across model families (Llama, Mistral, Gemma, SOLAR), but only when the teacher is an LLM judge; string-based teachers like recall or substring match produce classifiers too weak to be useful.
The IK score also serves as a dataset characterization tool, revealing a U-shaped distribution for NQ and ASQA, high confidence for TriviaQA/SCIQ, and low confidence for PopQA/HotpotQA.
Skipping RAG based on IK reduces per-query latency substantially (up to 80% of RAG processing time, ignoring retrieval/reranking), since the classifier itself adds only a few milliseconds.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The evaluation pipeline uses the same model (SOLAR) as both the label generator and the final answer-scorer, so a natural next step would be to re-run the experiments with an independent, stricter judge for the final LLMEval score; if the retrieval savings shrink, the IK classifier was partly matching the judge's taste rather than detecting knowledge.
The 32-token prefix finding suggests a streaming policy: generate a short prefix, compute IK, then decide whether to continue without retrieval or to restart with retrieved documents; this would remove the extra pass and could be tested directly.
The IK score's dataset-characterization behavior could be used to pre-screen corpora or query streams for whether RAG is worth deploying at all, or to set per-query retrieval budgets instead of a single threshold.
Because the best threshold varies by dataset (0.5–0.8), a practical extension would be a calibration procedure that maps the IK score to a desired retrieval rate, rather than fixing one threshold.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Read this paper for the token-length and data-efficiency numbers, not for the headline 'we skip half of retrievals' claim, because that claim rests on using the same judge to create the training labels and to score the final answers.

What's genuinely new here is the breadth of the empirical mapping. The core recipe—distill an LLM-as-judge into a binary 'I know' classifier and feed it a few generated tokens—is already in Self-RAG and the Slim Proxy Models work, and the paper cites both. But Déjean tests it across four model families, six datasets, several teacher models, token lengths from 0 to 128, and training set sizes from 5K to 83K. The result that 32 tokens is enough, that 20K samples suffice when tokens are included, and that string-based teachers fail while larger LLM teachers mostly transfer are useful, concrete data points for anyone building a RAG system.

The paper is admirably transparent about its setup: single runs, variance under 0.005, the unfinished GPT-4o teacher row ('to do'), and no code released. That last bit is a real gap—an efficiency paper should ship the classifier—but it's not a fatal flaw.

The soft spot is the one the paper itself voices in Section 5: SOLAR generates the silver labels and also produces the LLMEval score used to measure final QA quality. The IK classifier is trained to predict SOLAR's 'Yes/No', and the policy is then scored by the same SOLAR. If SOLAR is lenient or biased, the classifier may simply be learning to match judge preferences, and the reported retrieval savings could be an artifact. The teacher ablation helps a little—Llama-70B as teacher gives similar QA results—but the evaluation metric is still SOLAR in those runs, so the circularity isn't broken. The 'best point' thresholds are also selected on the evaluation set, which inflates the apparent benefit.

That said, the central empirical findings are plausibly robust. The token-length plateau at 32 and the 20K training-set saturation are the kind of results that survive a change of metric. The paper just needs an independent QA metric (exact match, F1, or a different judge) and a fixed threshold policy to make the efficiency claim credible.

Who's this for? Engineers working on retrieval pipelines who want a lightweight decision rule; they'll get practical guidance and a few implementation details. It deserves a serious referee, though my recommendation would be major revision rather than acceptance as is.

Referee Report

4 major / 5 minor

Summary. The paper trains an 'I Know' (IK) classifier that predicts whether an LLM can answer a question without retrieval. Silver labels are produced by applying a judge LLM (SOLAR) to the model's answer, and the classifier is fine-tuned with adapters to output a Yes/No score, optionally taking the first 32 tokens of the generated answer as additional input. The IK score is then used to decide when to trigger RAG within the BERGEN framework. Experiments on six QA datasets report IK accuracy around 80%, retrieval reductions of more than 50% on several datasets, and a finding that about 20K training samples suffice when generated tokens are used.

Significance. The main strength of the paper is its concise demonstration that a lightweight classifier, trained on judge-produced labels and a few generated answer tokens, can route queries away from RAG while roughly maintaining the LLMEval score. The paper is transparent about single-run variance, points to public code, and includes useful analyses of training set size and answer-token length. If the circularity concern regarding the shared judge is adequately addressed, the method could be practically valuable for reducing RAG cost. As presented, however, the evidence does not yet separate 'learning the judge's preferences' from 'detecting parametric knowledge,' and the oracle-based threshold selection overstates the expected benefit. The significance is therefore conditional on additional validation.

major comments (4)

[Sections 3.1 and 3.3; Tables 1-2] The silver labels for IK training are produced by the SOLAR LLM-as-judge, and the final QA quality metric (LLMEval) is also computed by SOLAR. Because the IK classifier is trained to reproduce SOLAR's judgments and the retrieval policy is scored by that same judge, the reported accuracy and downstream gains mainly measure alignment with SOLAR's preferences rather than factual correctness. The Discussion (Section 5) acknowledges this concern but only offers qualitative analysis; an independent metric (e.g., exact match or F1 on NQ/TriviaQA, or a judge from a different model family) should be added to validate the central claim.
[Tables 1-5, 'Best Point' columns] The best-point threshold is selected by maximizing LLMEval on the evaluation split itself. This is an oracle threshold selection and likely overstates the achievable benefit, since in practice the threshold must be fixed before seeing the test set. Please report a threshold-selection procedure on a held-out validation set and then evaluate the chosen threshold on a separate test set, or at least quantify the sensitivity by also presenting results at a predetermined threshold such as 0.5.
[Section 4.4 and Table 4] The paper states that 'SOLAR appears to be a more lenient judge' and argues that this leniency is 'necessary' for NQ because of multi-item answers. This observation directly bears on the validity of the learned labels. The paper should quantify agreement between SOLAR and the other judges (e.g., Llama-3.1-70B, Mixtral) on a sample, and show how label disagreements propagate to the IK classifier's decisions and final QA scores. Without such analysis, the claim that the IK score detects 'whether an answer is stored in parametric memory' is not established.
[Section 5] The claim that 'the gain is mostly due to cases where non relevant documents were retrieved' is not supported by any quantitative evidence in the manuscript. Please include a breakdown of the cases where the IK policy chooses no-RAG but the RAG answer is better, and vice versa, along with judge scores or independent quality indicators for those cases. This would also help the reader evaluate whether the retrieval savings are obtained at the expense of correctness.

minor comments (5)

[Table 3] In the row for 0 tokens, the NORAG LLMEval is shown as 0.42, while all other rows show 0.62 for the same column; this appears to be a typo and should be corrected.
[Figures 2 and 3] The captions of Figures 2 and 3 appear to be swapped relative to their references in Section 4.2: Section 4.2 first refers to Figure 3 for the distribution of IK scores, whereas Figure 2 has the histogram caption, and the later reference to 'Figure 3 shows the strange behaviour of the SCIQ dataset' matches the evaluation curves. Please renumber the figures or fix the cross-references.
[Table 5] The row for '10K 32' appears to have misaligned or merged columns: '0.78 0.7 50' does not clearly separate the Best Point LLMEval, %Retr, and IK theta values. Please realign the table entries.
[Table 4] The last row of Table 4 contains the incomplete placeholder 'idem GPT-4o as Judge to do to do'; this should be completed or removed before publication.
[Section 2] The related work section would benefit from a short comparison with reference [19], which also trains a judgment model using generated answers; making the differences explicit would help the reader position the contribution.

Circularity Check

2 steps flagged · score 7.0 of 10

The headline savings are scored by the same SOLAR judge that generated the IK training labels, so the central claim partially reduces to learning the judge's preferences; the paper itself asks this question.

fitted input called prediction [Sections 3.1, 3.2, 3.3, 4.4; Table 1 caption]
"To teach the LLM what it knows (IK score), we apply the model to a dataset and evaluate its output using an LLM-as-a-judge... The score generated by the LLM-as-a-judge is used as a binary label (Yes or No). ... Evaluation is performed using the LLMEval score provided by BERGEN. ... We use as judge the SOLAR model as default judge in BERGEN."

The IK classifier is trained to reproduce SOLAR-as-judge Yes/No labels, and the final RAG/no-RAG policy is evaluated with LLMEval computed by the same SOLAR judge. The reported benefit (maintained or improved LLMEval while skipping retrieval for many queries) therefore measures agreement with the judge function that produced the training labels, not an independent check of whether the answer is in parametric memory. Section 5 explicitly asks 'Are we not just learning how to increase the judge score instead of detecting a lack of knowledge of the LLM?' and answers only with qualitative analysis. Section 4.4 further notes 'SOLAR appears to be a more lenient judge,' so matching its preferences can inflate the apparent savings.
fitted input called prediction [Table 1 caption; Tables 1-5 'Best Point' columns]
"Additionally, we identify thebest point prducing the best LLMEval, along with the corresponding retrieval percentage and IK threshold. KILT NQ dataset (dev split). LLMEval(Solar) used as QA metric."

The 'Best Point' operating point is chosen by maximizing LLMEval on the NQ dev split, and the corresponding retrieval percentage is then reported as a result. This is fitting the decision threshold to the evaluation data rather than predicting performance at a pre-specified threshold: the reported LLMEval and retrieval rate at the best point are forced by the selection criterion. The fixed IK=0.5 columns are genuine predictions, but the best-point numbers that support the strongest efficiency claims are optimized on the evaluation set.

full rationale

The central derivation chain is: SOLAR-as-judge converts generated answers into binary IK labels (Section 3.1); the IK classifier is fine-tuned to reproduce those labels (Section 3.2); the same judge's LLMEval scores the final answers selected by the classifier between RAG and no-RAG (Section 3.3). The final metric is therefore not an external test of whether the LLM 'knows' the answer; it is the same function used to generate the training signal. The paper's own Discussion poses exactly this objection, 'Are we not just learning how to increase the judge score instead of detecting a lack of knowledge of the LLM?', and answers only with qualitative analysis. This makes the headline retrieval-savings claim partially circular: a policy that mimics the judge will tend to score well on the judge. There is genuine independent content: the IK classifier's 80% accuracy is a held-out agreement rate with judge labels, the teacher-robustness experiments include non-SOLAR judges (Llama-3.1-70B, Mixtral-8x7B) and string metrics, and the response-length and training-size ablations are self-contained. The BERGEN citation [16] is self-citational for the claim that SOLAR correlates with OpenAI judges, but it is not the load-bearing element because alternative teachers are tested. The fixed-threshold (IK=0.5) results are close to true predictions, but the 'Best Point' columns select thresholds on the dev split, which inflates the reported savings. Overall, the central claim is substantially constrained by the shared judge, so the circularity score is 7 rather than 0-2.

Assumptions & free parameters 2 free parameters · 4 assumptions · 1 invented entities

The central claims depend on trusting judge-generated labels, using the same judge for evaluation, and assuming that a classifier trained on NQ transfers to other datasets. These are domain assumptions, not standard math.

free parameters (2)

IK decision threshold theta = 0.5, 0.6, 0.7, 0.8, 0.9 (per dataset)
The 'best point' threshold is selected per dataset to maximize the evaluation metric on the evaluation split, which can inflate reported retrieval reductions.
Number of generated answer tokens = 32
Chosen after scanning 0, 4, 8, 16, 32, 64, and 128 tokens; 32 is the minimum length that shows the claimed improvement.

assumptions (4)

domain assumption LLM-as-judge binary labels reliably indicate whether the main LLM knows an answer.
Section 3.1 uses judge output as silver training labels; the paper flags the controversy and Discussion questions whether it learns judge scores rather than knowledge.
domain assumption SOLAR-based LLMEval is a valid measure of final QA quality.
Section 3.3 uses LLMEval as the QA metric; because the same judge generates training labels, evaluation and training are not independent.
domain assumption The first 32 tokens of a generated answer contain enough signal to predict whether the model knows the answer.
Section 4.3 shows empirically that 32 tokens suffice; the paper generalizes this beyond the exact datasets tested.
domain assumption A classifier trained on NQ transfers to other QA datasets.
Section 4.2 evaluates the NQ-trained Mistral classifier on five other datasets and assumes the learned IK signal generalizes.

invented entities (1)

IK score
purpose: A scalar confidence used to decide whether to activate RAG; computed as softmax probability of the 'Yes' token over 'Yes' and 'No' logits.
The score is trained to imitate an LLM judge, so it has no ground truth outside that judge and no independent falsifiable handle.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Let your LLM generate a few tokens and you will reduce the need for retrieval." pith.science (2026). https://pith.science/paper/QDXS5SC4

@misc{pith2026241211536,
  author       = {Pith},
  title        = {Pith review of: Let your LLM generate a few tokens and you will reduce the need for retrieval},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QDXS5SC4}},
  note         = {Machine review of arXiv:2412.11536}
}

read the original abstract

In this paper, we investigate how efficiently large language models (LLM) can be trained to check whether an answer is already stored in their parametric memory. We distill an LLM-as-a-judge to compute the IK (I Know) score. We found that this method is particularly beneficial in the context of retrieval-assisted augmented generation (RAG), with a respectable accuracy of 80%. It enables a significant reduction (more than 50%) in the number of search and reranking steps required for certain data sets. We have also introduced the IK score, which serves as a useful tool for characterising datasets by facilitating the classification task. Interestingly, through the inclusion of response tokens as input, our results suggest that only about 20,000 training samples are required to achieve good performance. The central element of this work is the use of a teacher model - the LLM as a judge - to generate training data. We also assess the robustness of the IK classifier by evaluating it with various types of teachers, including both string-based methods and LLMs, with the latter providing better results.

Figures

Figures reproduced from arXiv: 2412.11536 by the authors.

**Figure 2.** Histogram displaying the distribution of the IK score for six datasets utilizing the Mistral-32 model. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** Evaluation conducted on six datasets, with the IK model trained using the NQ dataset. Red dotted line [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: Impact of the response length used during [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

**Figure 5.** Figure 5: Visualization of the effect of varying the num [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

29 extracted references · 18 canonical work pages

[1]

Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. 2023. Self-rag: Learn- ing to retrieve, generate, and critique through self-reflection. ArXiv, abs/2310.11511. 1, 2, 3

arXiv 2023
[2]

Amos Azaria and Tom Mitchell. 2023. The In- ternal State of an LLM Knows When It’s Lying. ArXiv:2304.13734 [cs]. 1, 2, 3

arXiv 2023
[3]

Yanda Chen, Ruiqi Zhong, Narutatsu Ri, Chen Zhao, He He, Jacob Steinhardt, Zhou Yu, and Kathleen McKeown. 2023. Do models ex- plain themselves? counterfactual simulatabil- ity of natural language explanations. ArXiv, abs/2307.08678. 1

arXiv 2023
[4]

Retrieveonlywhen it needs: Adaptive retrieval augmentation for hal- lucination mitigation in large language models

Hanxing Ding, Liang Pang, Zihao Wei, Huawei Shen, andXueqiCheng.2024. Retrieveonlywhen it needs: Adaptive retrieval augmentation for hal- lucination mitigation in large language models. ArXiv, abs/2402.10612. 2

arXiv 2024
[5]

Hui Huang, Yingqi Qu, Hongli Zhou, Jing Liu, Muyun Yang, Bing Xu, and Tiejun Zhao. 2024. On the Limitations of Fine-tuned Judge Models for LLM Evaluation. ArXiv:2403.02839 [cs]. 2

arXiv 2024
[6]

Zhengbao Jiang, Frank F Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi-Yu, Yiming Yang, Jamie Callan, and Graham Neubig. 2023. Active retrieval augmented generation.arXiv preprint arXiv:2305.06983. 2

arXiv 2023
[7]

Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. 2017. TriviaQA: A large scale distantly supervised challenge dataset for read- ing comprehension. In Proceedings of the 55th Annual Meeting of the Association for Computa- tional Linguistics (Volume 1: Long Papers), pages 1601–1611, Vancouver, Canada. Association for Computational Linguistics. 2

work page 2017
[8]

A fairly small quantity (5k) already allows for decent performances if 32 tokens from the answer are used

Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova Das- Sarma, Eli Tran-Johnson, Scott Johnston, Sheer El-Showk, Andy Jones, Nelson Elhage, Tristan Hume, Anna Chen, Yuntao Bai, Sam Bowman, Stanislav Fort, Deep Ganguli, Danny Hernandez, Josh Jacobson, Jackson Kernion, Shauna Krave...

arXiv 2022

Show all 29 references

[9]

Large language models are zero-shot reasoners

Takeshi Kojima, Shixiang Shane Gu, Machel Reid, YutakaMatsuo, andYusukeIwasawa.2022. Large language models are zero-shot reasoners. Ad- vances in neural information processing systems, 35:22199–22213. 2

2022
[10]

Tiziano Labruna, Jon Ander Campos, and Gorka Azkune. 2024. When to retrieve: Teaching llms to utilize information retrieval effectively. 1, 2

2024
[11]

Carlos Lassance, Hervé Dejean, Stéphane Clin- chant, and Nicola Tonellotto. 2024. Two-Step SPLADE: Simple, Efficient and Effective Approxima- tion of SPLADE, page 349–363. Springer Nature Switzerland. 7

2024
[12]

Patrick Lewis, Ethan Perez, Aleksandra Pik- tus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. 1

2020
[13]

Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi
[14]

Shiyu Ni, Keping Bi, Jiafeng Guo, and Xueqi Cheng. 2024. When Do LLMs Need Retrieval Augmentation? Mitigating LLMs’ Overconfidence Helps Retrieval Augmentation. In Findings of the Association for Computational Linguistics ACL 2024, pages 11375–11388, Bangkok, Thailand and virt...

2024
[15]

Smith, and Mike Lewis

Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A. Smith, and Mike Lewis. 2022. Measuring and narrowing the compositionality gap in language models. 2

2022
[16]

David Rau, Hervé Déjean, Nadezhda Chirkova, Thibault Formal, Shuai Wang, Vassilina Nikoulina, and Stéphane Clinchant. 2024. Bergen: A bench- marking library for retrieval-augmented genera- tion. 2, 3, 6

2024
[17]

Minjoon Seo, Jinhyuk Lee, Tom Kwiatkowski, Ankur Parikh, Ali Farhadi, and Hannaneh Ha- jishirzi. 2019. Real-time open-domain question answering with dense-sparse phrase index. In Proceedings of the 57th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 4...

2019
[18]

Ivan Stelmakh, Yi Luan, Bhuwan Dhingra, and Ming-Wei Chang. 2022. ASQA: Factoid questions meet long-form answers. In Proceedings of the 2022 Conference on Empirical Methods in Natu- ral Language Processing, pages 8273–8288, Abu Dhabi, UnitedArabEmirates.AssociationforCom- puta...

2022
[19]

Jiejun Tan, Zhicheng Dou, Yutao Zhu, Peidong Guo, Kun Fang, and Ji-Rong Wen. 2024. Small Models, Big Insights: Leveraging Slim Proxy Mod- els To Decide When and What to Retrieve for LLMs. ArXiv:2402.12052 [cs]. 1, 2

2024 arXiv
[20]

Jiejun Tan, Zhicheng Dou, Yutao Zhu, Peidong Guo, Kun Fang, and Ji-Rong Wen. 2024. Small models, big insights: Leveraging slim proxy mod- els to decide when and what to retrieve for llms. 1, 2, 3

2024
[21]

Aman Singh Thakur, Kartik Choudhary, Venkat Srinik Ramayapally, Sankaran Vaidyanathan, and Dieuwke Hupkes. 2024. Judging the judges: Evaluating alignment 9 Let your LLM generate a few tokens and you will reduce the need for retrieval and vulnerabilities in llms-as-judges. ArXi...

2024 arXiv
[22]

Hongru Wang, Boyang Xue, Baohang Zhou, Tian- hua Zhang, Cunxiang Wang, Guanhua Chen, Huimin Wang, and Kam-fai Wong. 2024. Self- DC: When to retrieve and When to generate? Self Divide-and-Conquer for Compositional Unknown Questions. ArXiv:2402.13514 [cs]. 1

2024 arXiv
[23]

Yile Wang, Peng Li, Maosong Sun, and Yang Liu. 2023. Self-Knowledge Guided Retrieval Augmentation for Large Language Models. ArXiv:2310.05002 [cs]. 2

2023 arXiv
[24]

Liu, and Matt Gard- ner

Johannes Welbl, Nelson F. Liu, and Matt Gard- ner. 2017. Crowdsourcing multiple choice science questions. InProceedings of the 3rd Workshop on Noisy User-generated Text, pages 94–106, Copen- hagen, Denmark. Association for Computational Linguistics. 2

2017
[25]

Miao Xiong, Zhiyuan Hu, Xinyang Lu, Yifei Li, Jie Fu, Junxian He, and Bryan Hooi. 2024. Can llms express their uncertainty? an empirical evalua- tion of confidence elicitation in llms. 1

2024
[26]

Ziyou Yan. 2024. Evaluating the effectiveness of llm-evaluators (aka llm-as-judge).eugeneyan.com. 2

2024
[27]

Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. HotpotQA: A dataset for diverse, explainable multi-hop ques- tion answering. InProceedings of the 2018 Con- ference on Empirical Methods in Natural Langu...

2018
[28]

YukunZhao,LingyongYan,WeiweiSun,Guoliang Xing, Chong Meng, Shuaiqiang Wang, Zhicong Cheng, Zhaochun Ren, and Dawei Yin. 2024. KnowingWhatLLMsDONOTKnow: ASimpleYet Effective Self-Detection Method. InProceedings of the 2024 Conference of the North American Chap- ter of the Assoc...

2024
[2023]

When not to trust language models: In- vestigating effectiveness of parametric and non- parametric memories. 1, 2

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. 2023. Self-rag: Learn- ing to retrieve, generate, and critique through self-reflection. ArXiv, abs/2310.11511. 1, 2, 3

arXiv 2023

[2] [2]

Amos Azaria and Tom Mitchell. 2023. The In- ternal State of an LLM Knows When It’s Lying. ArXiv:2304.13734 [cs]. 1, 2, 3

arXiv 2023

[3] [3]

Yanda Chen, Ruiqi Zhong, Narutatsu Ri, Chen Zhao, He He, Jacob Steinhardt, Zhou Yu, and Kathleen McKeown. 2023. Do models ex- plain themselves? counterfactual simulatabil- ity of natural language explanations. ArXiv, abs/2307.08678. 1

arXiv 2023

[4] [4]

Retrieveonlywhen it needs: Adaptive retrieval augmentation for hal- lucination mitigation in large language models

Hanxing Ding, Liang Pang, Zihao Wei, Huawei Shen, andXueqiCheng.2024. Retrieveonlywhen it needs: Adaptive retrieval augmentation for hal- lucination mitigation in large language models. ArXiv, abs/2402.10612. 2

arXiv 2024

[5] [5]

Hui Huang, Yingqi Qu, Hongli Zhou, Jing Liu, Muyun Yang, Bing Xu, and Tiejun Zhao. 2024. On the Limitations of Fine-tuned Judge Models for LLM Evaluation. ArXiv:2403.02839 [cs]. 2

arXiv 2024

[6] [6]

Zhengbao Jiang, Frank F Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi-Yu, Yiming Yang, Jamie Callan, and Graham Neubig. 2023. Active retrieval augmented generation.arXiv preprint arXiv:2305.06983. 2

arXiv 2023

[7] [7]

Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. 2017. TriviaQA: A large scale distantly supervised challenge dataset for read- ing comprehension. In Proceedings of the 55th Annual Meeting of the Association for Computa- tional Linguistics (Volume 1: Long Papers), pages 1601–1611, Vancouver, Canada. Association for Computational Linguistics. 2

work page 2017

[8] [8]

A fairly small quantity (5k) already allows for decent performances if 32 tokens from the answer are used

Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova Das- Sarma, Eli Tran-Johnson, Scott Johnston, Sheer El-Showk, Andy Jones, Nelson Elhage, Tristan Hume, Anna Chen, Yuntao Bai, Sam Bowman, Stanislav Fort, Deep Ganguli, Danny Hernandez, Josh Jacobson, Jackson Kernion, Shauna Krave...

arXiv 2022