Pith. sign in

REVIEW 4 major objections 5 minor 9 cited by

Parametric Retrieval Augmented Generation

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper proposes Parametric RAG, which turns each retrieved document into a small set of low-rank weight patches and merges them into the LLM's feed-forward networks, claiming it beats in-context knowledge injection on most evaluated…

desk verdict A genuinely new RAG paradigm with open code and a solid control, but the core adapter-summing assumption is unvalidated and the evaluation is thinner than the claims. read the letter →

arxiv 2501.15915 v1 pith:5K32PKUK submitted 2025-01-27 cs.CL cs.IR

Weihang Su , Yichen Tang , Qingyao Ai , Junxi Yan , Changyue Wang , Hongning Wang , Ziyi Ye , Yujia Zhou
show 1 more author
Yiqun Liu
This is my paper · ORCID
classification cs.CLcs.IR
keywords LargeLanguageModelRetrievalAugmentedGenerationKnowledgeRepresentationParametricInformationLow-RankAdaptationFeed-ForwardNetworksRetrieve-Update-GenerateDocumentParameterization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is trying to establish that retrieval-augmented generation can be done by writing retrieved knowledge into the model's own weights rather than appending documents to the prompt. It argues that this parametric injection avoids the long-context slowdowns and reasoning degradation of in-context RAG, and that a per-document low-rank weight patch merged by a weighted sum is enough to give the model the combined knowledge of several retrieved documents. If the claim holds, RAG systems can serve many queries without paying to re-read the same documents, and the approach can be stacked on existing in-context RAG to improve accuracy further.

What carries the argument

The load-bearing object is the per-document parametric representation: for each document, the paper trains low-rank matrices A and B so that the feed-forward weight update is W' = W + AB^T, a LoRA-style adaptation attached to the LLM's feed-forward networks. At inference, the retrieved documents' representations are combined by the weighted sum of their A and B products and added to the base weights, producing a temporary updated model. This merge formula is what makes the Retrieve-Update-Generate workflow feasible: documents are parameterized once offline and only their small weight patches are loaded and summed online.

What would settle it

Train separate low-rank patches for two documents that state contradictory facts, retrieve both documents together, and check whether the merged model answers both facts correctly; if a fact is lost or the answer flips, the additive-merge assumption fails.

Watch

Extended reading notes

Core claim

The central claim is that external documents can be converted offline into compact parametric representations, low-rank weight updates to the feed-forward layers of a specific LLM, so that at query time the retrieved top-k documents are merged into one additive update and loaded into the model. The updated model then answers from its parameters alone, with the original prompt and no appended documents. Experiments across LLaMA-1B, Qwen-1.5B, and LLaMA-8B on 2WikiMultihopQA, HotpotQA, PopQA, and ComplexWebQuestions report that Parametric RAG outperforms standard RAG, DA-RAG, FLARE, and DRAGIN on most benchmarks, cuts inference time by roughly a third, and performs best when combined with in-context injection.

Load-bearing premise

The load-bearing premise is that summing independently trained per-document low-rank updates produces a model that simultaneously knows all retrieved documents, with no knowledge conflicts or forgetting.

Editorial extensions

If this is right

  • RAG systems can stop paying per-query token costs for long document contexts; only the short query is decoded.
  • The same offline document parameterization can be reused across many queries, making the method cost-effective once query volume exceeds roughly twice the document count.
  • Parametric knowledge injection combines with in-context RAG: the merged pipeline gives the best reported F1 on nearly every setting, so it can be added to existing systems without replacing them.
  • The gains over in-context baselines widen from the 1B to the 8B model, suggesting larger models internalize injected document knowledge more effectively.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the additive-merge assumption holds for adversarial document pairs, the same pipeline becomes a knowledge-caching layer: hot documents stay pre-parameterized and cold documents fall back to in-context injection.
  • A task-aware variant is suggested by the warm-up results: pre-training the adapter space on task QA pairs before parameterizing documents could specialize the merge geometry, a testable upgrade the paper leaves implicit.
  • The Retrieve-Update-Generate loop also resembles a continual-knowledge-update mechanism; testing whether the merge rule scales to hundreds of documents would show whether it can replace fine-tuning for absorbing new facts.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces Parametric RAG, a retrieval-augmented generation paradigm in which external documents are parameterized offline as per-document LoRA adapters trained on augmented document-QA data, and then injected into the FFN weights of an LLM at inference via a Retrieve-Update-Generate pipeline. The update step merges the adapters of the top-k retrieved documents by the weighted sum in Eq. (6), and the updated model generates answers without the documents in context. Experiments on 2WikiMultihopQA, HotpotQA, PopQA, and ComplexWebQuestions with LLaMA-1B, Qwen-1.5B, and LLaMA-8B compare P-RAG with Standard RAG, DA-RAG, FLARE, DRAGIN, and a combined in-context plus parametric setting. The paper reports F1 improvements in most settings and a modest inference speedup on LLaMA-8B.

Significance. If the result holds, Parametric RAG is a substantive new point in the RAG design space: external knowledge is stored in model weights rather than in the context, enabling temporary parameter updates that are compatible with in-context augmentation. The paper includes valuable controls, including the DA-RAG baseline, augmentation ablations, and a warm-up initialization study, and the authors have released code, data, and models, which is a strength. The main qualification is that the multi-document merging mechanism, which is what makes the approach scalable, is supported only by end-to-end results and not by direct evidence that the merge preserves the knowledge of each retrieved document.

major comments (4)
  1. [§3.3.2, Eq. (6)] The merge rule ΔW_merge = α Σ_j A_j B_j^T is the load-bearing component of the Retrieve-Update-Generate pipeline, because it is what allows multiple retrieved documents to be injected at once. Property (3) in §3.1 asserts that after such a merge the model grasps the combined knowledge of the documents, but the paper provides neither a derivation nor a direct experiment for this property. The end-to-end results in Table 1 cannot isolate merge fidelity from retrieval quality and the base model's own parametric knowledge. Please add a k=1 oracle to verify that a single document adapter actually stores the document's knowledge; add a sequential ablation that adds adapters one at a time and reports per-step F1; compare Eq. (6) with a LoRA trained jointly on all retrieved documents; and compare with standard adapter-fusion alternatives such as task arithmetic or TIES. Without these, the central multi-document claim is not empirically grounded.
  2. [§5.1, Table 1] The statistical markers in Table 1 are not defined. The footnote says '*' and '†' denote p<0.05 against the best method and against P-RAG, respectively, but the test procedure, the test statistic, the number of samples per cell, and whether the test is paired over the 300 questions are never stated. No error bars or variance estimates are reported anywhere, and the evaluation uses only the first 300 questions per sub-dataset. The headline claim that 'P-RAG outperforms existing RAG frameworks in most of the benchmarks and LLMs evaluated' also needs qualification: in Table 1 P-RAG is below Standard RAG on the LLaMA-1B HotpotQA total (0.1999 vs. 0.2671) and on the LLaMA-1B CWQ total (0.3482 vs. 0.3726). Please report confidence intervals or repeated runs, describe the significance test, and discuss these exceptions.
  3. [§4.2 and §5.1, DA-RAG] The DA-RAG baseline is intended to show that P-RAG's gains come from in-parameter injection rather than from document augmentation. However, DA-RAG places all augmented rewrites and QA pairs in the context, so its input is much longer than Standard RAG's. Long contexts are known to degrade LLM reasoning, so the observed P-RAG > DA-RAG gap could be explained by context-length effects rather than by the parametric injection paradigm. Add a matched control in which the same augmented content is presented to the model in a shorter or oracle-selected form, for example only the QA pairs or the original document plus QA pairs, so that the comparison isolates the injection mechanism.
  4. [§3.4.1 and §5.5, Table 4] The efficiency claim rests on Table 4, which reports P-RAG at 2.34+0.32s on 2WQA and 2.07+0.32s on CWQ, versus 3.03s and 2.82s for Standard RAG. The 0.32s merging/loading overhead is roughly 12–14% of the total P-RAG time, so the asymptotic argument in §3.4.1 that loading LoRA parameters is 'neglectable' is not supported by the measurements. Please report the efficiency comparison including the merge overhead, provide a break-even analysis showing when the overhead is amortized, and benchmark the efficiency on the other two base models as well.
minor comments (5)
  1. [§4.1] The paper says the 'Total' column is not a simple average of sub-datasets, but it does not state the pooling or weighting procedure; please specify how the totals are computed for 2WQA and HQA.
  2. [§5.3, Figure 3] Figure 3 lacks error bars and does not state how many runs each point represents; as with Table 1, the absence of variance information makes the ablation differences hard to interpret.
  3. [§5.5, Table 4] The table caption should state explicitly that the reported P-RAG and Combine Both times include the 0.32s merge/load overhead and that the speed-up column is computed relative to the full time including that overhead.
  4. [§3.4.1] The statement that loading LoRA parameters is 'neglectable' conflicts with the measured 0.32s overhead in Table 4; please reconcile the theoretical bound with the implementation measurement and state the hardware conditions.
  5. [References [29,30]] The Hugging Face model identifiers in [29,30] should include the full stable model card URLs or additional identifiers, as the 'Accessed' dates alone are not durable citation information.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central claim is an empirical benchmark evaluation on held-out questions, and no load-bearing step reduces by construction to a fitted parameter or to a self-citation chain.

full rationale

The paper's central claim is that P-RAG, which trains per-document LoRA adapters on augmented document content and merges them during inference, outperforms in-context RAG baselines. This is tested on held-out benchmark questions in Section 5.1, so the comparison is not derived from the parameters that were fit. The offline parameterization in Section 3.2 trains delta-theta on the document's own augmented text, which is the intended mechanism rather than a disguised evaluation target. The merge rule in Eq. (6) is an explicit algorithmic choice, not a quantity fitted to the evaluation data; its validity is an empirical assumption that the end-to-end experiments are designed to test, and the absence of a dedicated interference analysis is a correctness risk, not a circularity. The paper's self-citations, such as DRAGIN [45], appear as baselines and related work rather than as evidence for the central effectiveness claim. The warm-up ablation in Section 5.2 uses training-set QA pairs distinct from the test questions, so it is a standard transfer-learning ablation rather than a fitted prediction. No self-definitional step, no imported uniqueness theorem, no ansatz smuggled via citation, and no renaming of a known result as a derivation was found. The derivation chain is self-contained in the sense that the reported gains are empirical outcomes of a held-out evaluation, not consequences of the method's definition.

Assumptions & free parameters 6 free parameters · 5 assumptions · 1 invented entities

The central claim rests on two external research premises (FFN knowledge storage and augmentation benefits) and one paper-specific assumption (additive LoRA merging). The free parameters are standard hyperparameters, but the merge assumption is unvalidated and load-bearing.

free parameters (6)
  • LoRA rank r = 2
    Set by hand in Section 4.3; controls the capacity of each parametric document representation.
  • Scaling factor alpha = 32
    Set by hand in Section 4.3; controls the magnitude of the merged update in Eq. (6).
  • Number of rewrites n = 1
    Each document is rewritten once, stated in Section 4.3 as a preprocessing hyperparameter.
  • Number of QA pairs m = 3
    Each document generates three QA pairs, stated in Section 4.3.
  • Number of retrieved documents k = 3
    Top-k retrieved documents used for all experiments, stated in Section 4.2.
  • Warm-up QA pairs (ablation only) = 600
    Used in the warm-up initialization ablation of Section 5.2; not part of the main experimental configuration.
assumptions (5)
  • domain assumption Knowledge in LLMs is stored primarily in feed-forward network parameters, so parameter-level injection is fundamentally more effective than context injection.
    Invoked in Sections 1 and 2, supported by external works [1, 31, 59]. The paper does not derive this premise itself.
  • domain assumption Training on QA pairs and multiple rewrites makes the model apply knowledge rather than memorize text.
    Taken from Allen-Zhu and Li [1] and used as the basis of Document Augmentation in Section 3.2.1.
  • ad hoc to paper Summation of independently trained LoRA adapters preserves and combines each document's knowledge without destructive interference.
    Eq. (6) merges retrieved adapters by weighted sum; no theoretical or empirical interference analysis is provided.
  • domain assumption The LLM used for augmentation can produce faithful rewrites and QA pairs from the document.
    The whole parameterization pipeline depends on LLM-generated augmented data; no human evaluation of generated QA pairs is reported.
  • standard math Transformer inference time scales as O(|x|^2 h + |x| h^2).
    Used in Section 3.4.1 to argue online efficiency; standard for decoder-only models but approximate for modern attention variants.
invented entities (1)
  • Parametric representation of a document p_i
    purpose: A set of LoRA matrices A_i, B_i that encodes document d_i into the LLM's feed-forward weights.
    This is not a new physical or architectural entity; it is a LoRA adapter with a new role. Independent evidence outside the paper is not applicable.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Parametric Retrieval Augmented Generation." pith.science (2026). https://pith.science/paper/5K32PKUK

@misc{pith2026250115915,
  author       = {Pith},
  title        = {Pith review of: Parametric Retrieval Augmented Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5K32PKUK}},
  note         = {Machine review of arXiv:2501.15915}
}
read the original abstract

Retrieval-augmented generation (RAG) techniques have emerged as a promising solution to enhance the reliability of large language models (LLMs) by addressing issues like hallucinations, outdated knowledge, and domain adaptation. In particular, existing RAG methods append relevant documents retrieved from external corpus or databases to the input of LLMs to guide their generation process, which we refer to as the in-context knowledge injection method. While this approach is simple and often effective, it has inherent limitations. Firstly, increasing the context length and number of relevant documents can lead to higher computational overhead and degraded performance, especially in complex reasoning tasks. More importantly, in-context knowledge injection operates primarily at the input level, but LLMs store their internal knowledge in their parameters. This gap fundamentally limits the capacity of in-context methods. To this end, we introduce Parametric retrieval-augmented generation (Parametric RAG), a new RAG paradigm that integrates external knowledge directly into the parameters of feed-forward networks (FFN) of an LLM through document parameterization. This approach not only saves online computational costs by eliminating the need to inject multiple documents into the LLMs' input context, but also deepens the integration of external knowledge into the parametric knowledge space of the LLM. Experimental results demonstrate that Parametric RAG substantially enhances both the effectiveness and efficiency of knowledge augmentation in LLMs. Also, it can be combined with in-context RAG methods to achieve even better performance. We have open-sourced all the code, data, and models in the following anonymized GitHub link: https://github.com/oneal2000/PRAG

Figures

Figures reproduced from arXiv: 2501.15915 by the authors.

Figure 1
Figure 1. An illustration of the comparison of in-context RAG and our proposed Parametric RAG paradigms: In-context RAG [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. An illustration of how we parameterize each document [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Ablation study on the impact of the document aug [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 9 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Language Models Need Sleep: Learning to Self-Modify and Consolidate Memories

    cs.LG 2026-06 unverdicted novelty 6.0 of 10

    Sleep-time Knowledge Seeding plus Dreaming lets LLMs expand capacity, distill fragile in-context memories into stable parameters, and self-improve without human labels.

  2. AI5GTest: AI-Driven Specification-Aware Automated Testing and Validation of 5G O-RAN Components

    cs.NI 2025-06 conditional novelty 6.0 of 10

    An LLM-based framework that generates expected O-RAN and 3GPP procedural flows from standards and validates captured signaling logs against them, reporting 100% accuracy on 15 testbed instances and under an hour per t...

  3. Cartridges: Lightweight and general-purpose long context representations via self-study

    cs.CL 2025-06 conditional novelty 6.0 of 10

    A per-corpus trained KV cache, called a Cartridge, matches full-context in-context learning quality on long-document benchmarks while using up to 38.6x less serving memory.

  4. RbFT: Robust Fine-tuning for Retrieval-Augmented Generation against Retrieval Defects

    cs.CL 2025-01 conditional novelty 6.0 of 10

    Fine-tuning an LLM with defect detection and utility extraction tasks makes it more robust to noisy, irrelevant, and counterfactual documents in retrieval-augmented generation.

  5. GRIP: In-Parameter Graph Reasoning through Fine-Tuning Large Language Models

    cs.CL 2025-11 reject novelty 5.0 of 10

    An LLM can memorize a knowledge graph into LoRA weights and answer relation/reasoning queries about it without graph context, but the evaluation partly trains on the test task.

  6. PL-CA: A Parametric Legal Case Augmentation Framework

    cs.CL 2025-09 reject novelty 5.0 of 10

    PL-CA applies parametric RAG with LoRA to Chinese legal tasks and presents a 2,580-instance expert-annotated benchmark, claiming improved performance and lower context overhead than vanilla RAG.

  7. LoRA-Augmented Generation (LAG) for Knowledge-Intensive Language Tasks

    cs.CL 2025-07 conditional novelty 5.0 of 10

    LAG is a two-stage router that filters a 1,000-adapter LoRA library with Arrow and reranks with SpectR, outperforming the Arrow baseline and reaching 92.1% of its Oracle's performance on KILT tasks.

  8. MemOS: An Operating System for Memory-Augmented Generation (MAG) in Large Language Models

    cs.CL 2025-05 reject novelty 4.0 of 10

    A unified memory-operating-system design for LLMs, built around a MemCube abstraction, is presented without any experimental validation.

  9. Dynamic and Parametric Retrieval-Augmented Generation

    cs.CL 2025-06 unverdicted novelty 2.0 of 10

    A tutorial outline that categorizes recent RAG work into Dynamic RAG and Parametric RAG, and explains why both are needed.

Reference graph

Works this paper leans on

62 extracted references · 8 canonical work pages · cited by 9 Pith papers

  1. [1]

    Zeyuan Allen-Zhu and Yuanzhi Li. [n. d.]. Physics of Language Models: Part 3.1, Knowledge Storage and Extraction. In Forty-first International Conference on Machine Learning

    work page

  2. [2]

    Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. [n. d.]. Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection. In The Twelfth International Conference on Learning Representations

    work page

  3. [3]

    Ingeol Baek, Hwan Chang, Byeongjeong Kim, Jimin Lee, and Hwanhee Lee. 2024. Probing-RAG: Self-Probing to Guide Language Models in Selective Document Retrieval. arXiv preprint arXiv:2410.13339 (2024)

    arXiv 2024

  4. [4]

    Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Ruther- ford, Katie Millican, George Bm Van Den Driessche, Jean-Baptiste Lespiau, Bog- dan Damoc, Aidan Clark, et al. 2022. Improving language models by retrieving from trillions of tokens. In International conference on machine learning . PMLR, 2206–2240

    2022

  5. [5]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877–1901

    2020

  6. [6]

    Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Se- bastian Gehrmann, et al. 2022. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311 (2022)

    arXiv 2022

  7. [7]

    Guanting Dong, Hongyi Yuan, Keming Lu, Chengpeng Li, Mingfeng Xue, Dayi- heng Liu, Wei Wang, Zheng Yuan, Chang Zhou, and Jingren Zhou. 2023. How abilities in large language models are affected by supervised fine-tuning data composition. arXiv preprint arXiv:2310.05492 (2023)

    arXiv 2023

  8. [8]

    Qian Dong, Qingyao Ai, Hongning Wang, Yiding Liu, Haitao Li, Weihang Su, Yiqun Liu, Tat-Seng Chua, and Shaoping Ma. 2025. Decoupling Knowledge and Context: An Efficient and Effective Retrieval Augmented Generation Framework via Cross Attention. In Proceedings of the ACM on Web Conference 2025

    2025

Show all 62 references
  1. [9]

    Qian Dong, Yiding Liu, Qingyao Ai, Haitao Li, Shuaiqiang Wang, Yiqun Liu, Dawei Yin, and Shaoping Ma. 2023. I3 retriever: incorporating implicit interaction in pre-trained language models for passage retrieval. InProceedings of the 32nd ACM International Conference on Informat...

    2023

  2. [10]

    Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, and Jonathan Larson. 2024. From local to global: A graph rag approach to query-focused summarization. arXiv preprint arXiv:2404.16130 (2024)

    2024 arXiv

  3. [11]

    Yan Fang, Jingtao Zhan, Qingyao Ai, Jiaxin Mao, Weihang Su, Jia Chen, and Yiqun Liu. 2024. Scaling laws for dense retrieval. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval . 1339–1349

    2024

  4. [12]

    Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang. 2020. Retrieval augmented language model pre-training. In International conference on machine learning. PMLR, 3929–3938

    2020

  5. [13]

    Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. 2020. Constructing a multi-hop QA dataset for comprehensive evaluation of reasoning steps. arXiv preprint arXiv:2011.01060 (2020)

    2020 arXiv

  6. [14]

    Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. 2022. LoRA: Low-Rank Adaptation of Large Language Models. In International Conference on Learning Representations

    2022

  7. [15]

    Yuntong Hu, Zhihan Lei, Zheng Zhang, Bo Pan, Chen Ling, and Liang Zhao. 2024. GRAG: Graph Retrieval-Augmented Generation. arXiv preprint arXiv:2405.16506 (2024)

    2024 arXiv

  8. [16]

    Gautier Izacard and Edouard Grave. 2020. Leveraging passage retrieval with generative models for open domain question answering. arXiv preprint arXiv:2007.01282 (2020)

    2020 arXiv

  9. [17]

    Soyeong Jeong, Jinheon Baek, Sukmin Cho, Sung Ju Hwang, and Jong Park

  10. [18]

    Zhengbao Jiang, Luyu Gao, Jun Araki, Haibo Ding, Zhiruo Wang, Jamie Callan, and Graham Neubig. 2022. Retrieval as attention: End-to-end learning of retrieval and reading within a single transformer. arXiv preprint arXiv:2212.02027 (2022)

    2022 arXiv

  11. [19]

    Zhengbao Jiang, Frank F Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi-Yu, Yiming Yang, Jamie Callan, and Graham Neubig. 2023. Active retrieval augmented generation. arXiv preprint arXiv:2305.06983 (2023)

    2023 arXiv

  12. [20]

    Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open- domain question answering. arXiv preprint arXiv:2004.04906 (2020)

    2020 arXiv

  13. [21]

    Zixuan Ke, Weize Kong, Cheng Li, Mingyang Zhang, Qiaozhu Mei, and Michael Bendersky. 2024. Bridging the preference gap between retrievers and llms. arXiv preprint arXiv:2401.06954 (2024)

    2024 arXiv

  14. [22]

    Mosh Levy, Alon Jacoby, and Yoav Goldberg. 2024. Same task, more tokens: the impact of input length on the reasoning performance of large language models. arXiv preprint arXiv:2402.14848 (2024)

    2024 arXiv

  15. [23]

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing S...

    2020

  16. [24]

    Haitao Li, Jia Chen, Weihang Su, Qingyao Ai, and Yiqun Liu. 2023. Towards better web search performance: pre-training, fine-tuning and learning to rank. arXiv preprint arXiv:2303.04710 (2023)

    2023 arXiv

  17. [25]

    Huanshuo Liu, Hao Zhang, Zhijiang Guo, Kuicai Dong, Xiangyang Li, Yi Quan Lee, Cong Zhang, and Yong Liu. 2024. CtrlA: Adaptive Retrieval-Augmented Generation via Probe-Guided Control. arXiv preprint arXiv:2405.18727 (2024)

    2024 arXiv

  18. [26]

    Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024. Lost in the middle: How language models Parametric Retrieval Augmented Generation Conference, Under Review, use long contexts. Transactions of the Association for ...

    2024

  19. [27]

    Yixiao Ma, Yueyue Wu, Weihang Su, Qingyao Ai, and Yiqun Liu. 2023. CaseEn- coder: A Knowledge-enhanced Pre-trained Model for Legal Case Encoding.arXiv preprint arXiv:2305.05393 (2023)

    2023 arXiv

  20. [28]

    Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories. In Proceedings of the 61st Annual Meeting of the Association for Compu...

    2023 doi

  21. [29]

    Meta. 2024. Llama-3.2-1B-Instruct. https://huggingface.co/meta-llama/Llama- 3.2-1B-Instruct Accessed: 2024-09

    2024

  22. [30]

    Meta. 2024. Meta-Llama-3-8B-Instruct. https://huggingface.co/meta-llama/Meta- Llama-3-8B-Instruct Accessed: 2024-04

    2024

  23. [31]

    Neel Nanda, Senthooran Rajamanoharan, János Kramár, and Rohin Shah. 2023. Fact Finding: Attempting to Reverse-Engineer Factual Recall on the Neuron Level. https://www.lesswrong.com/posts/iGuwZTHWb6DFY3sKB/fact-finding- attempting-to-reverse-engineer-factual-recall Accessed: 2025-01-24

    2023

  24. [32]

    Boci Peng, Yun Zhu, Yongchao Liu, Xiaohe Bo, Haizhou Shi, Chuntao Hong, Yan Zhang, and Siliang Tang. 2024. Graph retrieval-augmented generation: A survey. arXiv preprint arXiv:2408.08921 (2024)

    2024 arXiv

  25. [33]

    Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, and Yoav Shoham. 2023. In-context retrieval-augmented language models. arXiv preprint arXiv:2302.00083 (2023)

    2023 arXiv

  26. [34]

    Stephen Robertson, Hugo Zaragoza, et al . 2009. The probabilistic relevance framework: BM25 and beyond. Foundations and Trends® in Information Retrieval 3, 4 (2009), 333–389

    2009

  27. [35]

    Alireza Salemi and Hamed Zamani. 2024. Towards a search engine for machines: Unified ranking for multiple retrieval-augmented large language models. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval . 741–751

    2024

  28. [36]

    Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilić, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, et al. 2022. Bloom: A 176b-parameter open-access multilingual language model. arXiv preprint arXiv:2211.05100 (2022)

    2022 arXiv

  29. [37]

    Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Rich James, Mike Lewis, Luke Zettlemoyer, and Wen-tau Yih. 2023. Replug: Retrieval-augmented black-box language models. arXiv preprint arXiv:2301.12652 (2023)

    2023 arXiv

  30. [38]

    Craig Silverstein, Hannes Marais, Monika Henzinger, and Michael Moricz. 1999. Analysis of a very large web search engine query log. SIGIR Forum 33, 1 (Sept. 1999), 6–12. https://doi.org/10.1145/331403.331405

    1999

  31. [39]

    Tim Soulo. 2023. 96.55% of Content Gets No Traffic From Google. Here’s How to Be in the Other 3.45% [New Research for 2023] . https://ahrefs.com/blog/search- traffic-study/ Accessed: 2025-01-24

    2023

  32. [40]

    Weihang Su, Qingyao Ai, Xiangsheng Li, Jia Chen, Yiqun Liu, Xiaolong Wu, and Shengluan Hou. 2023. Wikiformer: Pre-training with Structured Information of Wikipedia for Ad-hoc Retrieval. arXiv preprint arXiv:2312.10661 (2023)

    2023 arXiv

  33. [41]

    Weihang Su, Qingyao Ai, Yueyue Wu, Yixiao Ma, Haitao Li, and Yiqun Liu. 2023. Caseformer: Pre-training for Legal Case Retrieval.arXiv preprint arXiv:2311.00333 (2023)

    2023 arXiv

  34. [42]

    Weihang Su, Yiran Hu, Anzhe Xie, Qingyao Ai, Quezi Bing, Ning Zheng, Yun Liu, Weixing Shen, and Yiqun Liu. 2024. STARD: A Chinese Statute Retrieval Dataset Derived from Real-life Queries by Non-professionals. InFindings of the Association for Computational Linguistics: EMNLP 2...

    2024 doi

  35. [43]

    Weihang Su, Xiangsheng Li, Yiqun Liu, Min Zhang, and Shaoping Ma. 2023. Thuir2 at ntcir-16 session search (ss) task. arXiv preprint arXiv:2307.00250 (2023)

    2023 arXiv

  36. [44]

    Weihang Su, Yichen Tang, Qingyao Ai, Changyue Wang, Zhijing Wu, and Yiqun Liu. 2024. Mitigating entity-level hallucination in large language models. In Proceedings of the 2024 Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the...

    2024

  37. [45]

    Weihang Su, Yichen Tang, Qingyao Ai, Zhijing Wu, and Yiqun Liu. 2024. DRAGIN: Dynamic Retrieval Augmented Generation based on the Real-time Information Needs of Large Language Models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (V...

    2024 doi

  38. [46]

    Weihang Su, Changyue Wang, Qingyao Ai, Yiran Hu, Zhijing Wu, Yujia Zhou, and Yiqun Liu. 2024. Unsupervised real-time hallucination detection based on the internal states of large language models. arXiv preprint arXiv:2403.06448 (2024)

    2024 arXiv

  39. [47]

    Alon Talmor and Jonathan Berant. 2018. The Web as a Knowledge-Base for Answering Complex Questions. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) , Maril...

    2018 doi

  40. [48]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 (2023)

    2023 arXiv

  41. [49]

    Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal

  42. [50]

    Changyue Wang, Weihang Su, Qingyao Ai, and Yiqun Liu. 2024. Knowledge Editing through Chain-of-Thought. arXiv preprint arXiv:2412.17727 (2024)

    2024 arXiv

  43. [51]

    Changyue Wang, Weihang Su, Hu Yiran, Qingyao Ai, Yueyue Wu, Cheng Luo, Yiqun Liu, Min Zhang, and Shaoping Ma. 2024. LeKUBE: A Legal Knowledge Update BEnchmark. arXiv preprint arXiv:2407.14192 (2024)

    2024 arXiv

  44. [52]

    Yile Wang, Peng Li, Maosong Sun, and Yang Liu. 2023. Self-knowledge guided retrieval augmentation for large language models.arXiv preprint arXiv:2310.05002 (2023)

    2023 arXiv

  45. [53]

    Zihao Wang, Anji Liu, Haowei Lin, Jiaqi Li, Xiaojian Ma, and Yitao Liang. 2024. Rat: Retrieval augmented thoughts elicit context-aware reasoning in long-horizon generation. arXiv preprint arXiv:2403.05313 (2024)

    2024 arXiv

  46. [54]

    Tongtong Wu, Linhao Luo, Yuan-Fang Li, Shirui Pan, Thuy-Trang Vu, and Gho- lamreza Haffari. 2024. Continual learning for large language models: A survey. arXiv preprint arXiv:2402.01364 (2024)

    2024 arXiv

  47. [55]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al . 2024. Qwen2. 5 Technical Report. arXiv preprint arXiv:2412.15115 (2024)

    2024 arXiv

  48. [56]

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W Cohen, Ruslan Salakhutdinov, and Christopher D Manning. 2018. HotpotQA: A dataset for di- verse, explainable multi-hop question answering. arXiv preprint arXiv:1809.09600 (2018)

    2018 arXiv

  49. [57]

    Zijun Yao, Weijian Qi, Liangming Pan, Shulin Cao, Linmei Hu, Weichuan Liu, Lei Hou, and Juanzi Li. 2024. Seakr: Self-aware knowledge retrieval for adaptive retrieval augmented generation. arXiv preprint arXiv:2406.19215 (2024)

    2024 arXiv

  50. [58]

    Yue Yu, Wei Ping, Zihan Liu, Boxin Wang, Jiaxuan You, Chao Zhang, Mohammad Shoeybi, and Bryan Catanzaro. 2024. Rankrag: Unifying context ranking with retrieval-augmented generation in llms. arXiv preprint arXiv:2407.02485 (2024)

    2024 arXiv

  51. [59]

    Zeping Yu and Sophia Ananiadou. 2024. Neuron-Level Knowledge Attribution in Large Language Models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (Eds.). Association for Computationa...

    2024 doi

  52. [60]

    ChengXiang Zhai. 2008. Statistical language models for information retrieval. Synthesis lectures on human language technologies 1, 1 (2008), 1–141

    2008

  53. [2022]

    arXiv preprint arXiv:2212.10509 (2022)

    Interleaving retrieval with chain-of-thought reasoning for knowledge- intensive multi-step questions. arXiv preprint arXiv:2212.10509 (2022)

    2022 arXiv

  54. [2024]

    Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity. InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), Kevi...

    2024 doi

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.