QuCo-RAG: Quantifying Uncertainty from the Pre-training Corpus for Dynamic Retrieval-Augmented Generation

Dehai Min; Kailin Zhang; Lu Cheng; Tongtong Wu

arxiv: 2512.19134 · v2 · pith:B45OJOSQnew · submitted 2025-12-22 · 💻 cs.CL · cs.IR

QuCo-RAG: Quantifying Uncertainty from the Pre-training Corpus for Dynamic Retrieval-Augmented Generation

Dehai Min , Kailin Zhang , Tongtong Wu , Lu Cheng This is my paper

Pith reviewed 2026-05-21 17:27 UTC · model grok-4.3

classification 💻 cs.CL cs.IR

keywords dynamic RAGuncertainty quantificationpre-training corpusentity co-occurrencehallucination mitigationretrieval augmented generationmulti-hop QA

0 comments

The pith

QuCo-RAG quantifies uncertainty for dynamic RAG by measuring entity frequency and co-occurrence in the pre-training corpus rather than model logits.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Dynamic retrieval-augmented generation tries to decide during output when to fetch outside information so that large language models avoid fabricating facts. Current methods watch internal signals such as token probabilities or entropy, yet these signals are unreliable because models are often poorly calibrated and express high confidence on mistakes. QuCo-RAG replaces those signals with two objective checks against the pre-training corpus: it first flags low-frequency entities that may represent knowledge gaps, then verifies during generation whether the entities ever appeared together in the training data. Zero co-occurrence triggers retrieval. The queries run in milliseconds over trillions of tokens, and the resulting decisions improve exact-match scores on multi-hop question answering while remaining usable even when the target model’s full training data is unknown.

Core claim

QuCo-RAG claims that uncertainty for deciding retrieval can be measured directly from pre-training corpus statistics through a two-stage process: pre-generation detection of low-frequency entities that mark long-tail knowledge gaps, followed by during-generation verification of entity co-occurrence, where zero co-occurrence serves as an objective marker of hallucination risk and therefore as the trigger for retrieval.

What carries the argument

Two-stage corpus-grounded uncertainty quantification that queries the pre-training corpus for entity frequency before generation and for co-occurrence during generation to decide when to retrieve.

If this is right

Retrieval timing becomes independent of any particular model’s internal calibration or logit values.
The same corpus checks transfer to models whose pre-training data remains undisclosed, yielding exact-match gains up to 14 points.
The approach extends beyond short-form QA to long-form generation and biomedical question answering.
Corpus verification supplies a model-agnostic alternative to logit- or entropy-based dynamic RAG.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The same frequency and co-occurrence queries could be repurposed to flag uncertainty in other open-ended generation settings such as summarization or dialogue.
Maintaining a continuously updated corpus index would allow the method to handle knowledge that appears after the original pre-training cutoff.
Because the queries are fast, the technique could be embedded directly into streaming generation pipelines without noticeable latency.

Load-bearing premise

Zero co-occurrence of entities in the pre-training corpus reliably indicates hallucination risk even when the index only partially represents or does not match the target model’s actual training data.

What would settle it

Finding multiple cases in which entities have zero co-occurrence in the queried corpus yet the model produces factually correct statements without retrieval would directly test whether zero co-occurrence tracks hallucination risk.

Figures

Figures reproduced from arXiv: 2512.19134 by Dehai Min, Kailin Zhang, Lu Cheng, Tongtong Wu.

**Figure 2.** Figure 2: Overview of QuCo-RAG Framework. using pre-training corpus statistics to quantify uncertainty and trigger retrieval, enabling reliable hallucination detection and mitigation. 3 Methodology 3.1 Problem Formulation We formalize the dynamic RAG problem as follows. Let M denote an LLM, C represent an external knowledge base for retrieval (e.g., Wikipedia), and P denote the pre-training corpus used to train M. … view at source ↗

**Figure 3.** Figure 3: Efficiency-performance trade-off analysis on HotpotQA with OLMo-2-13B-Instruct. (a) EM score versus [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

**Figure 4.** Figure 4: Average runtime breakdown per question for [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗

**Figure 5.** Figure 5: Performance stratified by entity frequency [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗

**Figure 6.** Figure 6: Threshold sensitivity analysis on 2WikiMulti [PITH_FULL_IMAGE:figures/full_fig_p013_6.png] view at source ↗

**Figure 7.** Figure 7: Performance comparison of QuCo-RAG with different retrievers (Qwen3-Embedding, SGPT, and BM25) on 2WikiMultihopQA using OLMo-2-7B. A.4 Effect of Different Retrievers To verify that QuCo-RAG is robust to retriever choice, we compare BM25 with dense retrievers SGPT (Muennighoff, 2022) and Qwen3- Embedding-0.6B (Zhang et al., 2025). As shown in [PITH_FULL_IMAGE:figures/full_fig_p014_7.png] view at source ↗

read the original abstract

Dynamic Retrieval-Augmented Generation adaptively determines when to retrieve during generation to mitigate hallucinations in large language models (LLMs). However, existing methods rely on model-internal signals (e.g., logits, entropy), which are fundamentally unreliable because LLMs are typically ill-calibrated and often exhibit high confidence in erroneous outputs. We propose QuCo-RAG, which shifts from subjective confidence to objective statistics computed from pre-training data. Our method quantifies uncertainty through two stages: (1) before generation, we identify low-frequency entities indicating long-tail knowledge gaps; (2) during generation, we verify entity co-occurrence in the pre-training corpus, where zero co-occurrence often signals hallucination risk. Both stages leverage Infini-gram for millisecond-latency queries over 4 trillion tokens, triggering retrieval when uncertainty is high. Experiments on multi-hop QA benchmarks show QuCo-RAG achieves EM gains of 5--12 points over state-of-the-art baselines with OLMo-2 models, and transfers effectively to models with undisclosed pre-training data (Llama-3, Qwen2.5, GPT-4.1/5-chat), improving EM by up to 14 points. Generalization to long-form generation and biomedical QA further validates the robustness of our paradigm. These results establish corpus-grounded verification as a principled, practically model-agnostic paradigm for dynamic RAG. Our code is publicly available at https://github.com/ZhishanQ/QuCo-RAG.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

QuCo-RAG uses pre-training frequency and co-occurrence stats via Infini-gram to trigger retrieval in RAG and reports EM gains, but the proxy index for undisclosed model data leaves the uncertainty mechanism on shaky ground.

read the letter

QuCo-RAG shifts dynamic retrieval to use pre-training corpus stats instead of model internals. Before generation it flags low-frequency entities, and during generation it checks for zero co-occurrence of entity pairs via Infini-gram queries over a large index to decide when to retrieve. This approach is new in tying retrieval triggers directly to objective frequency and co-occurrence counts from the pre-training data. It does well on the practical side by achieving EM gains of 5-12 points on multi-hop QA benchmarks with OLMo-2 and up to 14 points on models with undisclosed data like Llama-3, Qwen2.5, and GPT variants. The method stays fast with millisecond queries, generalizes to long-form and biomedical QA, and comes with public code. The main soft spot is the assumption that the Infini-gram 4T token index serves as a reliable proxy for the actual pre-training corpora of the black-box models. If the indexed data diverges from what those models were trained on, zero co-occurrence in the index does not confirm the model never saw the pair, so the causal connection to hallucination risk is not fully established. The reported gains are consistent with the method helping in practice, but stronger evidence linking the proxy to the models' specific knowledge gaps would tighten this up. Details on statistical significance and baseline controls would also help confirm the results. This paper is for people developing RAG systems who want an external, corpus-grounded way to handle uncertainty rather than trusting LLM calibration. Readers working on factual reliability or multi-hop reasoning tasks would get concrete ideas and results to consider. I recommend engaging with it through peer review. The idea and experiments have enough weight to benefit from referee feedback on the proxy validation and experimental rigor.

Referee Report

2 major / 2 minor

Summary. The paper proposes QuCo-RAG, a dynamic RAG approach that quantifies uncertainty using objective pre-training corpus statistics via Infini-gram queries over 4T tokens, rather than model-internal signals. It detects low-frequency entities before generation and verifies entity co-occurrence during generation, triggering retrieval on zero co-occurrence as a hallucination risk signal. Experiments report EM gains of 5-12 points over SOTA baselines on multi-hop QA with OLMo-2 models, with transfer to undisclosed-data models (Llama-3, Qwen2.5, GPT-4.1/5-chat) yielding up to 14-point gains, plus generalization to long-form generation and biomedical QA.

Significance. If the results hold, the work introduces a model-agnostic, corpus-grounded paradigm for dynamic RAG that directly addresses LLM miscalibration by leveraging external pre-training statistics. The millisecond-latency Infini-gram queries, public code release, and cross-domain validation add practical value for improving generation reliability.

major comments (2)

[§4] §4 (Transfer experiments): The claim that QuCo-RAG transfers effectively to models with undisclosed pre-training data relies on treating the Infini-gram 4T-token index as a reliable proxy for objective co-occurrence statistics. No overlap analysis, correlation study, or verification is provided showing that zero co-occurrence in the index corresponds to actual knowledge gaps in Llama-3, Qwen2.5 or GPT models. This assumption is load-bearing for the model-agnostic and 'objective' verification claims.
[§5] §5 (Experimental results): The reported EM gains of 5--12 points (and up to 14 on transfer) are presented without statistical significance tests, exact baseline configurations, data split details, or controls isolating retrieval quality from the uncertainty signal. This makes it difficult to attribute improvements specifically to the corpus-grounded method rather than implementation or retrieval differences.

minor comments (2)

[Abstract] Abstract: The model notation 'GPT-4.1/5-chat' is unclear; specify the exact model variants and versions used in the transfer experiments.
[§3] §3: The precise definition and thresholding for 'low-frequency entities' in the pre-generation stage should be stated explicitly, including any frequency cutoffs or normalization applied to Infini-gram counts.

Simulated Author's Rebuttal

2 responses · 1 unresolved

We thank the referee for the constructive feedback. We address each major comment below and describe the revisions we will make to strengthen the claims and experimental reporting.

read point-by-point responses

Referee: [§4] §4 (Transfer experiments): The claim that QuCo-RAG transfers effectively to models with undisclosed pre-training data relies on treating the Infini-gram 4T-token index as a reliable proxy for objective co-occurrence statistics. No overlap analysis, correlation study, or verification is provided showing that zero co-occurrence in the index corresponds to actual knowledge gaps in Llama-3, Qwen2.5 or GPT models. This assumption is load-bearing for the model-agnostic and 'objective' verification claims.

Authors: We agree that the transfer results depend on the 4T-token Infini-gram index functioning as a reasonable proxy. Because the pre-training corpora of Llama-3, Qwen2.5, and the GPT models are undisclosed, a direct overlap or correlation analysis is not possible. The consistent EM gains we observe nevertheless provide empirical evidence that zero co-occurrence in this broad public corpus aligns with hallucination risk even for these models. In the revision we will add a short subsection in §4 that explicitly discusses the proxy assumption, cites any publicly available information on training-data composition, and revises the wording from 'objective' to 'corpus-grounded' to reflect the approximation. revision: yes
Referee: [§5] §5 (Experimental results): The reported EM gains of 5--12 points (and up to 14 on transfer) are presented without statistical significance tests, exact baseline configurations, data split details, or controls isolating retrieval quality from the uncertainty signal. This makes it difficult to attribute improvements specifically to the corpus-grounded method rather than implementation or retrieval differences.

Authors: We accept that the current experimental section lacks sufficient rigor for clear attribution. The revised manuscript will add (i) statistical significance tests (McNemar’s test and bootstrap confidence intervals) for all EM differences, (ii) complete baseline configurations including prompts, retrieval hyperparameters, and model versions, (iii) explicit train/validation/test split details, and (iv) new ablation controls that hold retrieval quality fixed while varying only the uncertainty trigger. These additions will isolate the contribution of the corpus-grounded signal. revision: yes

standing simulated objections not resolved

Direct overlap or correlation analysis between the Infini-gram 4T index and the undisclosed pre-training data of Llama-3, Qwen2.5, or GPT models cannot be performed.

Circularity Check

0 steps flagged

No significant circularity; derivation grounded in external corpus queries

full rationale

The paper's core derivation defines uncertainty via two stages that query an external Infini-gram index over 4T tokens for entity frequency and co-occurrence statistics. These quantities are computed independently of the target LLM's logits, parameters, or any fitted values from the present experiments. No equations or steps reduce a claimed prediction back to a fitted input by construction, nor does the method rely on load-bearing self-citations, imported uniqueness theorems, or smuggled ansatzes. The transfer results to models with undisclosed pre-training data rest on an empirical proxy assumption rather than a definitional equivalence. The reported EM gains are presented as experimental outcomes, not forced by the method's internal logic. This is a self-contained, externally grounded approach with no circular reduction.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The central claim rests on two domain assumptions about what corpus statistics reveal about model knowledge gaps and hallucination risk; no free parameters or invented entities are explicitly introduced in the abstract.

axioms (2)

domain assumption Low-frequency entities in the pre-training corpus indicate long-tail knowledge gaps that the model is likely to mishandle.
Invoked in the pre-generation stage to identify uncertainty.
domain assumption Zero co-occurrence of entities in the pre-training corpus signals elevated hallucination risk during generation.
Central to the during-generation verification stage.

pith-pipeline@v0.9.0 · 5806 in / 1399 out tokens · 141726 ms · 2026-05-21T17:27:39.735148+00:00 · methodology

discussion (0)

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

IndisputableMonolith/Cost/FunctionalEquation.lean washburn_uniqueness_aczel unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

zero co-occurrence often signals hallucination risk... cooc(h, t;P) = |{ω ∈ P : h ∈ ω ∧ t ∈ ω}| ... δi = I(min cooc(h,t;P) < τcooc)

What do these tags mean?

matches: The paper's claim is directly supported by a theorem in the formal canon.
supports: The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends: The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses: The paper appears to rely on the theorem as machinery.
contradicts: The paper's claim conflicts with a theorem or certificate in the canon.
unclear: Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

Adaptive Stopping for Multi-Turn LLM Reasoning
cs.CL 2026-04 unverdicted novelty 8.0

MiCP is the first conformal prediction method for multi-turn LLM pipelines that allocates per-turn error budgets to enable adaptive stopping with an overall coverage guarantee, shown to reduce turns and cost on RAG an...

Reference graph

Works this paper leans on

3 extracted references · 3 canonical work pages · cited by 1 Pith paper

[1]

Huiyi Chen, Jiawei Peng, Kaihua Tang, Xin Geng, and Xu Yang

Reverse question answering: Can an llm write a question so hard (or bad) that it can’t answer? In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Compu- tational Linguistics: Human Language Technologies (Volume 2: Short Papers), pages 44–64. Huiyi Chen, Jiawei Peng, Kaihua Tang, Xin Geng, and Xu Yang. 2025....

work page arXiv 2025
[2]

InFindings of the Association for Computational Linguistics: EMNLP 11 2023, pages 10303–10315, Singapore

Self-knowledge guided retrieval augmenta- tion for large language models. InFindings of the Association for Computational Linguistics: EMNLP 11 2023, pages 10303–10315, Singapore. Association for Computational Linguistics. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. Chain-of-thought p...

work page 2023
[3]

So the answer is

Knowing you don’t know: Learning when to continue search in multi-round rag through self- practicing. InProceedings of the 48th International ACM SIGIR Conference on Research and Develop- ment in Information Retrieval, pages 1305–1315. Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Ben- gio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning...

work page arXiv 2018

[1] [1]

Huiyi Chen, Jiawei Peng, Kaihua Tang, Xin Geng, and Xu Yang

Reverse question answering: Can an llm write a question so hard (or bad) that it can’t answer? In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Compu- tational Linguistics: Human Language Technologies (Volume 2: Short Papers), pages 44–64. Huiyi Chen, Jiawei Peng, Kaihua Tang, Xin Geng, and Xu Yang. 2025....

work page arXiv 2025

[2] [2]

InFindings of the Association for Computational Linguistics: EMNLP 11 2023, pages 10303–10315, Singapore

Self-knowledge guided retrieval augmenta- tion for large language models. InFindings of the Association for Computational Linguistics: EMNLP 11 2023, pages 10303–10315, Singapore. Association for Computational Linguistics. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. Chain-of-thought p...

work page 2023

[3] [3]

So the answer is

Knowing you don’t know: Learning when to continue search in multi-round rag through self- practicing. InProceedings of the 48th International ACM SIGIR Conference on Research and Develop- ment in Information Retrieval, pages 1305–1315. Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Ben- gio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning...

work page arXiv 2018