Pith. sign in

REVIEW 4 major objections 5 minor 38 references

A fully parametric model can match RAG accuracy by learning to search its own weights.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 23:50 UTC pith:G67C5S42

load-bearing objection Genuinely new idea—learned parametric retrieval via RL—sold with an overclaimed headline; Table 1 only supports 'matches RAG' on 14B English, and the theory assumes what it needs to prove. the 4 major comments →

arxiv 2608.01630 v1 pith:G67C5S42 submitted 2026-08-03 cs.CL cs.AI

RING: Retrieval-Internalized Generation for Continual Large-Scale Knowledge Injection

Shicheng Xu , Liang Pang , Liyi Chen , Zihao Wei , Jingcheng Deng , Yan Gao , Yi Wu , Yao Hu
show 2 more authors
Huawei Shen Xueqi Cheng
This is my paper
classification cs.CL cs.AI
keywords retrieval-internalized generationparametric knowledge injectionmixture-of-memory expertsreinforcement learningretrieval-augmented generationcontinual knowledge injectiondual causal attentionNews-2025
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that the reason parametric knowledge injection lags behind retrieval-augmented generation is not storage but search: once a corpus is written into model weights, the hard part is knowing which weight-region holds the answer. RING addresses this by turning retrieval into a learned behavior. A sparse Mixture-of-Memory Experts keeps the original LLM intact in a Basic Expert while a duplicate Knowledge Expert memorizes a 341M-token news corpus; a Router and Search Gate are trained, first by supervised "search-then-answer" examples and then by reinforcement learning with hierarchical rewards, to select the right expert and the right memory fragment. The paper reports that RING matches or beats strong RAG pipelines in accuracy while cutting time-to-first-token by 3–19x, and frames the whole pipeline as a variational approximation to the classical RAG objective. If correct, RING removes the external retriever, vector database, and index-maintenance machinery from knowledge-intensive QA over stable corpora.

Core claim

RING's central discovery claim is that the retrieval distribution p(z|x) that RAG computes with an external embedding model and vector index can be learned inside the LLM itself. The paper implements this with a Router that picks between a Basic Expert and a Knowledge Expert at every token, and a Search Gate inside the Knowledge Expert that reweights memory projections query-dependently; the Knowledge Expert's up/down projections are treated as a differentiable key-value index over roughly 1.6 million fragments. Training is a three-stage pipeline: continued pre-training with Dual Causal Attention writes the corpus into the memory projections; supervised fine-tuning teaches the model to emit

What carries the argument

The central mechanism is the Mixture-of-Memory Experts layer: a frozen Basic Expert preserves base abilities, a Knowledge Expert (a duplicated MLP split into Memory Up, Memory Down, and a SiLU-activated Search Gate) stores the injected corpus, and a learned Router makes a top-1 expert choice per token. Dual Causal Attention (three attention masks over target block and complement) injects the corpus bidirectionally while keeping inference causal. The search-then-answer SFT template and the RL stage with hierarchical rewards (format, dense LCS-based search rewards, answer verification) turn parametric retrieval from a static lookup into a trained policy.

Load-bearing premise

The load-bearing assumption is capacity: the duplicated Knowledge Expert MLP must have enough parameters to memorize all ~1.6 million fragments with sufficient fidelity, and the Router and Search Gate must have enough capacity to represent the same retrieval distribution as an external embedding retriever.

What would settle it

Train RING on a corpus whose total token count substantially exceeds the Knowledge Expert's parameter count while holding the query distribution fixed, and measure whether retrieval-block exact-match recall and QA accuracy stay at RAG levels; if accuracy falls below a top-1 embedding-based RAG baseline, the capacity assumption fails. A cheaper probe: on held-out questions whose evidence fragment was never seen in SFT/RL, check whether the generated <retrieval> block reproduces the reference span; low exact-match rates would show the learned policy memorized SFT targets rather than generalizabl

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • External retrieval infrastructure (embedding models, vector databases, index refresh) can be dropped for knowledge-intensive QA over corpora that are stable enough to be internalized before deployment.
  • Serving cost stops growing with corpus size and scales with model FLOPs: RING's time-to-first-token is 3–19x lower than every evaluated RAG variant.
  • New knowledge can be injected continually without catastrophic forgetting: the Basic Expert and frozen memory projections keep accuracy on old knowledge above 30% at 100k injected documents.
  • Retrieval becomes optimizable: because the policy is learned by RL, evidence selection can be aligned with answer correctness rather than with embedding similarity, which the paper argues can make RING strictly better than similarity-search RAG on some distributions.
  • The ELBO framing gives a principled interpretation for why the three stages must be CPT–SFT–RL and why the Knowledge Expert can be viewed as a differentiable key-value memory.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the learned retrieval policy generalizes beyond the training corpus distribution, the same architecture could serve multiple knowledge bases by swapping or adding Knowledge Experts, turning knowledge updates into module swaps rather than retraining.
  • The paper's explicit <retrieval> block is a generated paraphrase, not a verbatim document; this suggests a natural hybrid design where RING proposes evidence and a cheap external verifier confirms it, preserving provenance without paying full RAG latency.
  • RING's PPL–accuracy coupling implies that RL makes the model less willing to guess from prior knowledge; this coupling could be used as a diagnostic to detect whether a model is actually using injected knowledge or relying on memorized training priors.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes RING, a fully parametric alternative to RAG. It replaces the external retriever with a Mixture-of-Memory-Experts layer: a Basic Expert preserves the base model, a duplicated Knowledge Expert is meant to memorize an injected corpus, and a Router plus a Search Gate perform parametric retrieval. Training proceeds in three stages: continued pre-training with a proposed Dual Causal Attention (DCA) mask to inject the corpus, SFT to teach a search-then-answer format with an explicit <retrieval> block, and RL with hierarchical rewards to optimize the router/search policy. The authors construct a bilingual post-cutoff news benchmark, News-2025, and report accuracy and time-to-first-token results on Qwen3-8B and Qwen3-14B. They also give a theoretical latent-variable analysis claiming that RING can match or approximate explicit RAG.

Significance. If the central claim were fully established, RING would be a meaningful step toward removing external retrieval infrastructure at serving time while retaining knowledge-grounded QA. The paper has clear strengths: it introduces a new post-cutoff benchmark, runs a substantial three-stage training pipeline on two model sizes, includes ablations of architecture, attention, training stage, and reward design, and attempts a formal latent-variable viewpoint. However, the evidence as presented does not support the headline claim in its current form: the paper's own Table 1 shows RING trailing the strongest RAG pipeline on three of four language/backbone settings, the memory-fidelity assumptions in the theoretical section are not empirically validated, and the reported latency metric is not an answer-latency comparison. These issues are load-bearing for the two advertised contributions of accuracy and efficiency.

major comments (4)
  1. [Abstract; §5.2, Table 1] The abstract claims RING 'matches or surpasses both search-based RAG and parametric injection baselines in accuracy and efficiency.' Table 1 contradicts that for the strongest external-search baseline: on Qwen3-8B, Qwen3-Reranker RAG scores 39.51/41.05 vs. RING's 35.08/32.36 (gaps of 4.43 and 8.69 points), and on Qwen3-14B Chinese RING trails 37.92 vs. 39.82. Only the 14B English result supports the claim. The main text later acknowledges a trade-off, but the abstract and conclusion overstate. Please reframe as a favorable accuracy-latency trade-off rather than a match, and report variance or confidence intervals across runs.
  2. [§A.4, Theorem A.4] Theorem A.4 assumes 'sufficient fidelity' of the stored memory units and 'enough capacity' of the router/Search Gate to represent the external retrieval distribution. These are exactly the properties that need to be demonstrated. Proposition A.2 gives only dff value vectors per MLP layer, far fewer than the 1.6 million memory fragments in the corpus, and no capacity argument is provided for storing 1.6M fragments across the model. The theorem is therefore a hypothesis-class inclusion conditional on the conclusion. Please add direct retrieval-fidelity measurements, e.g., whether the <retrieval> block matches the gold memory unit on held-out queries, at the CPT, SFT, and RL stages, and report recall/exact-match/LCS numbers.
  3. [§3.2, Eq. (13); Appendix A.5, Prop. A.7] The DCA objective for α=1/2 and α=1/4 lets tokens in the target block attend to the entire future complement Cα. This means L(α) can be minimized by copying from Cα rather than by storing knowledge into the Memory Down/Up weights. Proposition A.7—that conditioning on a superset lowers optimal NLL—is exactly the mathematical description of this leakage, not a proof of memorization. Since inference uses only M(1), the complement is absent and any benefit may not transfer. The paper needs a control that isolates storage from contextual copying, e.g., evaluating CPT+DCA on a retrieval task where the complement is masked at test time.
  4. [§5.3, Table 1 (TTFT)] The reported TTFT is time to the first generated token. For RING that first token is the beginning of the template sentence, not the answer; the model then generates a <retrieval> block of up to 256 tokens before the <answer> block. For RAG, TTFT includes retrieval and context processing before the first output token, so the comparison is not an answer-latency comparison. The claim of 3×–19× faster inference is therefore unsupported. Please report time-to-answer (e.g., time until the <answer> block) or total end-to-end latency for both paradigms, alongside tokens/s.
minor comments (5)
  1. [§5.4, Fig. 5a; Table 2] The text says 'SFT ∼20%' accuracy in the low-PPL region, while Table 2 reports SFT-only accuracy of 28.41%. Please reconcile these numbers and clarify what is plotted.
  2. [§5.1] The test set is described as 'held-out' documents that do not appear in SFT or RL, but the CPT set is said to include the whole document collection. Please state explicitly whether these held-out documents were seen during CPT; if so, clarify that this is intentional for the injection-and-utilization test.
  3. [Table 1] All accuracy numbers are reported as point estimates without error bars or significance tests. Given the small gaps between RING and some baselines in the 14B rows, significance testing would strengthen the comparison.
  4. [§3.4, Eq. (2)] R3_dense is called a 'dense reward' but is a binary exact-match indicator; the terminology is confusing. Consider renaming or explaining why it functions as a dense signal in the RL setting.
  5. [Appendix A.5] The mask definition uses Kα=⌊αT⌋, which for α=1/4 makes the target block one quarter of the sequence. It would help to state explicitly how variable-length sequences are handled in batching and how the three forward passes with different masks are implemented without inflating memory costs.

Circularity Check

2 steps flagged

Theorem A.4 assumes the very retrieval/memory capacity it claims to prove; the DCA 'guarantee' is a tautology about conditioning supersets.

specific steps
  1. self definitional [Appendix A.4, Theorem A.4 (also invoked in Section 4 'Theoretical Grounding')]
    "Assume that after CPT, the injected parameters Θmem store the evidence units {uz}Nz=1 with sufficient fidelity, and that the router/Search Gate parameters ϕ have enough capacity to represent the same retrieval distribution as an external retriever on the query domain of interest; i.e., for all relevant x, pϕ(z|x)≈p̃η(z|x,U) for z∈TK(x), up to arbitrarily small error. Then for any explicit top-K RAG model in Eq. (7), there exists a RING instance in Eq. (9) that matches its conditional distribution p(y|x) arbitrarily well on that domain."

    The theorem's antecedent is exactly the target capability: storing all evidence units with fidelity and realizing the external retrieval distribution with the router/Search Gate. The proof sketch then constructs pϕ to approximate p̃η and sets pθgen to match the RAG generator—this is the same content as the assumption. Thus Theorem A.4 does not derive RING's expressivity from the architecture or training; it restates the desired conclusion as a hypothesis. Section 4 cites this theorem as proof that 'parametric retrieval can theoretically match external search expressivity,' so the theoretical grounding reduces to the assumption it claims to establish.

  2. other [Appendix A.5, Proposition A.7; Section 4 'Training as Variational Inference']
    "Any training objective that allows tokens in the target block to condition on a superset of information (as DCA does for the target block) has an optimal achievable negative log-likelihood no larger than that of standard causal conditioning, because the model can always learn to ignore additional context. Consequently, DCA can provide a tighter (not looser) upper bound on the conditional entropy of the target block given its complement, improving the information available for storing bidirectional dependencies into Θmem during CPT."

    The 'tighter bound' is a tautology of the DCA mask definition (Eq. 13): conditioning the target block on the complement Cα is a superset of standard prefix conditioning, so optimal NLL cannot increase. The proposition's 'Consequently' imports the storage conclusion without proof; a tighter bound on conditional entropy says nothing about whether Θmem weights store the 1.6M fragments. Section 4's claim that this 'theoretically guarantees more efficient storage' therefore reduces to the definition of the loss, not to a property of the parametric memory. At inference only M(1) is active, so any CPT loss reduction obtained by copying from Cα does not transfer.

full rationale

The empirical comparison on News-2025 is self-contained: RING's accuracy and TTFT are measured against external RAG and parametric injection baselines, so the headline 'matches or surpasses' is not circular. The circularity lies in the theoretical apparatus. Theorem A.4 proves dominance only after assuming that Θmem stores all units with fidelity and that the router/gate realize the external retrieval distribution—the very capabilities the paper claims to establish; Corollary A.5 inherits this. Prop. A.7's 'tighter bound' is a tautology about superset conditioning and does not imply storage; for α=1/2,1/4 the model can reduce L(α) by copying from the future complement Cα rather than from weights, and at inference the complement is absent. The paper also never reports retrieval-block fidelity against gold memory units, so the mechanism 'internalized retrieval' is not directly evidenced. The Limitations section concedes that generated evidence may be paraphrased or distorted, which further weakens the retrieval-fidelity interpretation. These are partly missing-evidence/correctness concerns rather than pure circularity, but the theoretical justification partially reduces to its assumptions, warranting a score of 6.

Axiom & Free-Parameter Ledger

5 free parameters · 6 axioms · 0 invented entities

RING introduces architectural components (Knowledge Expert, Memory Up/Down, Search Gate, Dual Causal Attention) but these are modules within a neural network, not new physical or mathematical entities. The critical unproven premises are capacity and expressivity assumptions, plus the train-to-inference transfer of DCA. These are listed as axioms rather than invented entities.

free parameters (5)
  • DCA loss weights lambda1, lambda2, lambda3 = 1.0, 0.3, 0.1
    Chosen by hand; control contribution of each mask ratio to memorization loss. Reported in Section C.4.
  • RL reward weights lambda_a, lambda_b, lambda_c = 0.8, 0.8, 1.0
    Hand-set weights for dense search rewards (title LCS, content LCS, exact match). Reported in Section C.4.
  • RL total reward weights lambda_format, lambda_answer, lambda_search = 0.1, 1.0, 1.0
    Hand-set weights for composite reward; heavily favor answer correctness and search over format. Reported in Section C.4.
  • SFT generic-instruction mixing ratio = 1:1
    Ratio of retrieval-style instructions to no-retrieval instructions in SFT data. Reported in Section C.4; affects 'retrieve only when needed' behavior.
  • <retrieval> block truncation length = 256 tokens
    Retrieved fragments truncated to 256 tokens in SFT to bound generation cost. Reported in Section B.1/C.4.
axioms (6)
  • domain assumption The Knowledge Expert's memory projections have sufficient capacity to store the entire corpus (341M tokens) with high fidelity after CPT.
    Assumed in Theorem A.4 ('after CPT, the injected parameters store the evidence units with sufficient fidelity') and implicitly in the whole experimental design. No independent capacity bound is given.
  • domain assumption The Router and Search Gate can represent any relevant retrieval distribution over the memory units, matching an external retriever's top-K distribution.
    Assumed in Theorem A.4 ('router/Search Gate parameters phi have enough capacity to represent the same retrieval distribution as an external retriever'). This is the key expressivity assumption that makes the theorem an assumption, not a derivation.
  • domain assumption The generated <retrieval> block is a faithful proxy for the latent retrieval variable z.
    Section A.2 equates p_phi(z|x) with the probability of generating unit u_z inside <retrieval>. The Limitations section admits the model may compress or paraphrase, so this mapping is not exact.
  • ad hoc to paper Training with DCA's bidirectional masks transfers to standard causal inference without harmful distribution shift.
    DCA uses masks M(1/2) and M(1/4) during CPT but only M(1) at inference. No theoretical or empirical analysis of this train-inference mismatch is provided.
  • standard math Low-temperature softmax concentrates on the unique argmax (standard result).
    Used in Lemma A.3 to connect gated MLP readout to maximum inner product search. Mathematically standard.
  • standard math Optimal conditional cross-entropy monotonically improves with more conditioning information.
    Used in Prop. A.7 to argue DCA does not worsen optimal NLL. True information-theoretically, but says nothing about generalization or train-inference transfer.

pith-pipeline@v1.3.0-daily-deepseek · 19691 in / 11379 out tokens · 125579 ms · 2026-08-04T23:50:26.899836+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of RING: Retrieval-Internalized Generation for Continual Large-Scale Knowledge Injection." pith.science (2026). https://pith.science/paper/G67C5S42

@misc{pith2026260801630,
  author       = {Pith},
  title        = {Pith review of: RING: Retrieval-Internalized Generation for Continual Large-Scale Knowledge Injection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/G67C5S42}},
  note         = {Machine review of arXiv:2608.01630}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Retrieval-augmented generation (RAG) improves factuality but adds latency and engineering overhead at serving time. We propose RING (Retrieval-Internalized Generation), a holistic paradigm spanning both architecture and training that injects large-scale external knowledge into a \textit{Mixture-of-Memory Experts} and learns parametric search over this internal memory via reinforcement learning, removing the external retriever entirely. Training proceeds in three stages: continued pre-training injects new corpora into a Knowledge Expert via our novel \textit{Dual Causal Attention}; supervised fine-tuning teaches a ``search-then-answer'' pattern; and reinforcement learning with hierarchical rewards optimizes the routing-and-search policy over the parametric memory. Unlike prior parametric injection methods that pair internal memory with a fixed or rule-based retriever, RING {learns} its retrieval policy directly from task signals. We further frame RING theoretically as a search-free approximation to the classical RAG objective. To evaluate large-scale injection of genuinely {new} knowledge without test-time leakage, we further construct News-2025, a benchmark built from news strictly post-dating the base LLM's pretraining cutoff. RING matches or surpasses both search-based RAG and parametric injection baselines in accuracy and efficiency.

Figures

Figures reproduced from arXiv: 2608.01630 by Huawei Shen, Jingcheng Deng, Liang Pang, Liyi Chen, Shicheng Xu, Xueqi Cheng, Yan Gao, Yao Hu, Yi Wu, Zihao Wei.

Figure 1
Figure 1. Figure 1: Overview of RING versus Conventional RAG. Unlike traditional methods that depend on external [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Model Architecture in our RING. residual connections to simplify the expression): h˜ (ℓ+1) t = X k∈K 1 h g (ℓ) t = k i E (ℓ) k  h (ℓ) t  . (1) Parameter Separation We further separate the parameters of knowledge expert Eknw into memory Em knw and search Es knw. This ensures that subse￾quent training of search capabilities will not affect the original memory and allows for incremental addition of new memo… view at source ↗
Figure 3
Figure 3. Figure 3: Dual Causal Attention. The white areas are [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Comparison of knowledge retention strate [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: (a) We plot the linear relationship between [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

38 extracted references · 15 canonical work pages · 2 internal anchors

  1. [1]

    Proceedings of the ACM Web Conference 2026 , pages=

    Self-evolving llms via continual instruction tuning , author=. Proceedings of the ACM Web Conference 2026 , pages=

    work page 2026

  2. [2]

    arXiv preprint arXiv:2401.01286 , year=

    A comprehensive study of knowledge editing for large language models , author=. arXiv preprint arXiv:2401.01286 , year=

    Pith/arXiv arXiv

  3. [3]

    Advances in neural information processing systems , volume=

    Retrieval-augmented generation for knowledge-intensive nlp tasks , author=. Advances in neural information processing systems , volume=

  4. [4]

    Proceedings of the 16th conference of the european chapter of the association for computational linguistics: main volume , pages=

    Leveraging passage retrieval with generative models for open domain question answering , author=. Proceedings of the 16th conference of the european chapter of the association for computational linguistics: main volume , pages=

  5. [5]

    ACM Transactions on Information Systems , volume=

    From matching to generation: A survey on generative information retrieval , author=. ACM Transactions on Information Systems , volume=. 2025 , publisher=

    work page 2025

  6. [6]

    ACM Computing Surveys , volume=

    Continual learning of large language models: A comprehensive survey , author=. ACM Computing Surveys , volume=. 2025 , publisher=

    2025

  7. [7]

    arXiv preprint arXiv:2604.27998 , year=

    Latent-GRPO: Group relative policy optimization for latent reasoning , author=. arXiv preprint arXiv:2604.27998 , year=

    Pith/arXiv arXiv

  8. [8]

    Latent reasoning in llms as a vocabulary-space superposition , author=

    work page

  9. [9]

    Dynamic Rollout Editing for Reducing Overthinking in RL-Trained Reasoning Models

    Dynamic Rollout Editing for Reducing Overthinking in RL-Trained Reasoning Models , author=. arXiv preprint arXiv:2606.17890 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  10. [10]

    arXiv preprint arXiv:2604.04989 , year=

    Skillattack: Automated red teaming of agent skills through attack path refinement , author=. arXiv preprint arXiv:2604.04989 , year=

    Pith/arXiv arXiv

  11. [11]

    arXiv preprint arXiv:2210.07229 , year=

    Mass-editing memory in a transformer , author=. arXiv preprint arXiv:2210.07229 , year=

    Pith/arXiv arXiv

  12. [12]

    International Conference on Learning Representations , volume=

    Kblam: Knowledge base augmented language model , author=. International Conference on Learning Representations , volume=

    work page

  13. [13]

    Advances in Neural Information Processing Systems , volume=

    Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model? , author=. Advances in Neural Information Processing Systems , volume=

  14. [14]

    arXiv preprint arXiv:1908.01091 , year=

    Toward understanding catastrophic forgetting in continual learning , author=. arXiv preprint arXiv:1908.01091 , year=

    Pith/arXiv arXiv 1908

  15. [15]

    International Conference on Learning Representations , volume=

    Everything is editable: Extend knowledge editing to unstructured data in large language models , author=. International Conference on Learning Representations , volume=

    work page

  16. [16]

    IEEE transactions on pattern analysis and machine intelligence , volume=

    A comprehensive survey of continual learning: Theory, method and application , author=. IEEE transactions on pattern analysis and machine intelligence , volume=. 2024 , publisher=

    2024

  17. [17]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    SR-KI: Scalable and Real-Time Knowledge Integration into LLMs via Supervised Attention , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    work page

  18. [18]

    arXiv preprint arXiv:2507.18071 , year=

    Group sequence policy optimization , author=. arXiv preprint arXiv:2507.18071 , year=

    Pith/arXiv arXiv

  19. [19]

    International Conference on Learning Representations , volume=

    Towards understanding factual knowledge of large language models , author=. International Conference on Learning Representations , volume=

    work page

  20. [20]

    2023 , eprint=

    Making Large Language Models A Better Foundation For Dense Retrieval , author=. 2023 , eprint=

    2023

  21. [21]

    Proceedings of the 62nd annual meeting of the association for computational linguistics (volume 1: Long papers) , pages=

    Unsupervised information refinement training of large language models for retrieval-augmented generation , author=. Proceedings of the 62nd annual meeting of the association for computational linguistics (volume 1: Long papers) , pages=

    work page

  22. [22]

    International Conference on Learning Representations , volume=

    A theory for token-level harmonization in retrieval-augmented generation , author=. International Conference on Learning Representations , volume=

    work page

  23. [23]

    arXiv preprint arXiv:2310.07521 , year=

    Survey on factuality in large language models: Knowledge, retrieval and domain-specificity , author=. arXiv preprint arXiv:2310.07521 , year=

    Pith/arXiv arXiv

  24. [24]

    Proceedings of the ACM Web Conference 2024 , pages=

    Search-in-the-chain: Interactively enhancing large language models with search for knowledge-intensive tasks , author=. Proceedings of the ACM Web Conference 2024 , pages=

    work page 2024

  25. [25]

    Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval , pages=

    Parametric retrieval augmented generation , author=. Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval , pages=

    work page

  26. [26]

    arXiv preprint arXiv:2312.10997 , volume=

    Retrieval-augmented generation for large language models: A survey , author=. arXiv preprint arXiv:2312.10997 , volume=

    Pith/arXiv arXiv

  27. [27]

    arXiv preprint arXiv:2401.11504 , year=

    With greater text comes greater necessity: Inference-time training helps long text generation , author=. arXiv preprint arXiv:2401.11504 , year=

    Pith/arXiv arXiv

  28. [28]

    arXiv preprint arXiv:2402.03216 , volume=

    Bge m3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation , author=. arXiv preprint arXiv:2402.03216 , volume=

    Pith/arXiv arXiv

  29. [29]

    arXiv preprint arXiv:2505.12306 , year=

    Bidirectional lms are better knowledge memorizers? a benchmark for real-world knowledge injection , author=. arXiv preprint arXiv:2505.12306 , year=

    Pith/arXiv arXiv

  30. [30]

    Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , pages=

    Transformer feed-forward layers are key-value memories , author=. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , pages=

    2021

  31. [31]

    Advances in Neural Information Processing Systems , volume=

    Autoregressive search engines: Generating substrings as document identifiers , author=. Advances in Neural Information Processing Systems , volume=

    work page

  32. [32]

    Advances in neural information processing systems , volume=

    Transformer memory as a differentiable search index , author=. Advances in neural information processing systems , volume=

    work page

  33. [33]

    arXiv preprint arXiv:2506.05176 , year=

    Qwen3 embedding: Advancing text embedding and reranking through foundation models , author=. arXiv preprint arXiv:2506.05176 , year=

    Pith/arXiv arXiv

  34. [34]

    Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Precise zero-shot dense retrieval without relevance labels , author=. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  35. [35]

    AtlasKV: Augmenting LLMs with Billion-Scale Knowledge Graphs in 20GB VRAM

    AtlasKV: Augmenting LLMs with Billion-Scale Knowledge Graphs in 20GB VRAM , author=. arXiv preprint arXiv:2510.17934 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  36. [36]

    arXiv preprint arXiv:2508.01832 , year=

    Mlp memory: A retriever-pretrained memory for large language models , author=. arXiv preprint arXiv:2508.01832 , year=

    arXiv

  37. [37]

    arXiv preprint arXiv:2507.05346 , year=

    LoRA-Augmented Generation (LAG) for Knowledge-Intensive Language Tasks , author=. arXiv preprint arXiv:2507.05346 , year=

    Pith/arXiv arXiv

  38. [38]

    arXiv preprint arXiv:2503.07891 , year=

    Gemini embedding: Generalizable embeddings from gemini , author=. arXiv preprint arXiv:2503.07891 , year=

    Pith/arXiv arXiv