{"id":"fda9a8f6-223e-46ff-bda1-23e9a4f09d64","arxiv_id":"2608.01630","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A mixture-of-experts LLM trained with reinforcement learning to perform retrieval from its own parametric memory can replace external retrieval in some settings, at lower latency.","lead":"This paper proposes RING, a system that lets a large language model store external documents in its own weights and learn to search them internally, so it no longer needs a separate document retriever at serving time. It reports accuracy that is competitive with or better than retrieval-augmented generation in some settings while running several times faster.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Capacity assumption for 1.6M-fragment internal memory is untested; DCA future-context masks may let CPT copy instead of memorize, so RING's 'matches RAG' claim lacks direct retrieval-fidelity evidence.","rationale":"The reader's weakest assumption is the capacity of the duplicated Knowledge Expert MLP and the Router/Search Gate to store 1.6M fragments and emulate an external retriever. I agree this is load-bearing and conditional: Theorem A.4 states it as an assumption rather than proving it. My concern sharpens this with a specific mechanism: Dual Causal Attention's α<1 masks give target tokens access to the future complement, allowing low training loss by copying rather than memorizing, so the CPT stage may not actually internalize the corpus. The paper's own Prop. A.7 concedes that conditioning on a superset lowers achievable NLL, but that is a statement about easier training, not about storage. No reported experiment distinguishes 'stored and retrieved' from 'answer-level memorized during SFT/RL'. A retrieval-block fidelity test would settle this directly. The reader's conditional verdict remains appropriate: the central claim should not be accepted as stated until retrieval fidelity is demonstrated. I mark agreement as partial because the reader identified the capacity assumption generally, while the added DCA-leakage mechanism points to a concrete, testable failure mode not explicitly flagged in the reader's rationale.","tokens_in":19920,"tokens_out":6773,"duration_ms":87725,"concrete_test":"Measure RING's parametric retrieval fidelity directly. Take 1,000 held-out test questions and force generation of only the <retrieval> block; compare it to the gold memory unit u* via exact string match and token-level ROUGE-L/F1. Also take 1,000 random CPT fragments, prompt with the title only, and ask for verbatim content under the inference-time causal mask. If RING's retrieval-block exact-match is substantially below the RAG top-1 hit rate (e.g., <50% exact or <0.6 ROUGE-L), the capacity assumption fails and QA accuracy cannot be attributed to internalized search. As a control, ablate the DCA α=1/2,1/4 masks in CPT and repeat the same reconstruction test: if removing future-context masks does not reduce reconstruction quality, DCA's apparent QA gain is not from memory storage.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that after CPT the Knowledge Expert actually stores the 341M-token corpus (1.6M fragments) and that the Router/Search Gate reproduce the external retriever's distribution. Theorem A.4 assumes exactly this ('sufficient fidelity', 'enough capacity'), so it provides no independent support. No experiment measures retrieval fidelity: the paper reports QA accuracy, PPL, and reward components, but never whether the generated <retrieval> block matches the gold memory unit on held-out questions. The architecture has only dff value vectors per layer (Prop. A.2), far fewer than 1.6M fragments, and the sparse top-1 router selects experts per token, not a document-level latent z. Moreover, DCA (Eq. 13) makes this worse: for α=1/2,1/4, target-block tokens attend to the entire future complement Cα, so CPT can reduce L(α) by copying from Cα rather than storing content in weights. Prop. A.7's 'conditioning on a superset lowers NLL' describes exactly this leakage, not memory. At inference only M(1) is active, so the crutch disappears. If retrieval fidelity is low, RING's QA gains may come from SFT/RL memorizing answer patterns rather than from internalized retrieval, and 'matches RAG' would not hold for the claimed reason.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes RING, a fully parametric alternative to RAG. It replaces the external retriever with a Mixture-of-Memory-Experts layer: a Basic Expert preserves the base model, a duplicated Knowledge Expert is meant to memorize an injected corpus, and a Router plus a Search Gate perform parametric retrieval. Training proceeds in three stages: continued pre-training with a proposed Dual Causal Attention (DCA) mask to inject the corpus, SFT to teach a search-then-answer format with an explicit <retrieval> block, and RL with hierarchical rewards to optimize the router/search policy. The authors construct a bilingual post-cutoff news benchmark, News-2025, and report accuracy and time-to-first-token results on Qwen3-8B and Qwen3-14B. They also give a theoretical latent-variable analysis claiming that RING can match or approximate explicit RAG.","tokens_in":20320,"tokens_out":6048,"duration_ms":72881,"significance":"If the central claim were fully established, RING would be a meaningful step toward removing external retrieval infrastructure at serving time while retaining knowledge-grounded QA. The paper has clear strengths: it introduces a new post-cutoff benchmark, runs a substantial three-stage training pipeline on two model sizes, includes ablations of architecture, attention, training stage, and reward design, and attempts a formal latent-variable viewpoint. However, the evidence as presented does not support the headline claim in its current form: the paper's own Table 1 shows RING trailing the strongest RAG pipeline on three of four language/backbone settings, the memory-fidelity assumptions in the theoretical section are not empirically validated, and the reported latency metric is not an answer-latency comparison. These issues are load-bearing for the two advertised contributions of accuracy and efficiency.","major_comments":[{"comment":"The abstract claims RING 'matches or surpasses both search-based RAG and parametric injection baselines in accuracy and efficiency.' Table 1 contradicts that for the strongest external-search baseline: on Qwen3-8B, Qwen3-Reranker RAG scores 39.51/41.05 vs. RING's 35.08/32.36 (gaps of 4.43 and 8.69 points), and on Qwen3-14B Chinese RING trails 37.92 vs. 39.82. Only the 14B English result supports the claim. The main text later acknowledges a trade-off, but the abstract and conclusion overstate. Please reframe as a favorable accuracy-latency trade-off rather than a match, and report variance or confidence intervals across runs.","section":"Abstract; §5.2, Table 1"},{"comment":"Theorem A.4 assumes 'sufficient fidelity' of the stored memory units and 'enough capacity' of the router/Search Gate to represent the external retrieval distribution. These are exactly the properties that need to be demonstrated. Proposition A.2 gives only dff value vectors per MLP layer, far fewer than the 1.6 million memory fragments in the corpus, and no capacity argument is provided for storing 1.6M fragments across the model. The theorem is therefore a hypothesis-class inclusion conditional on the conclusion. Please add direct retrieval-fidelity measurements, e.g., whether the <retrieval> block matches the gold memory unit on held-out queries, at the CPT, SFT, and RL stages, and report recall/exact-match/LCS numbers.","section":"§A.4, Theorem A.4"},{"comment":"The DCA objective for α=1/2 and α=1/4 lets tokens in the target block attend to the entire future complement Cα. This means L(α) can be minimized by copying from Cα rather than by storing knowledge into the Memory Down/Up weights. Proposition A.7—that conditioning on a superset lowers optimal NLL—is exactly the mathematical description of this leakage, not a proof of memorization. Since inference uses only M(1), the complement is absent and any benefit may not transfer. The paper needs a control that isolates storage from contextual copying, e.g., evaluating CPT+DCA on a retrieval task where the complement is masked at test time.","section":"§3.2, Eq. (13); Appendix A.5, Prop. A.7"},{"comment":"The reported TTFT is time to the first generated token. For RING that first token is the beginning of the template sentence, not the answer; the model then generates a <retrieval> block of up to 256 tokens before the <answer> block. For RAG, TTFT includes retrieval and context processing before the first output token, so the comparison is not an answer-latency comparison. The claim of 3×–19× faster inference is therefore unsupported. Please report time-to-answer (e.g., time until the <answer> block) or total end-to-end latency for both paradigms, alongside tokens/s.","section":"§5.3, Table 1 (TTFT)"}],"minor_comments":[{"comment":"The text says 'SFT ∼20%' accuracy in the low-PPL region, while Table 2 reports SFT-only accuracy of 28.41%. Please reconcile these numbers and clarify what is plotted.","section":"§5.4, Fig. 5a; Table 2"},{"comment":"The test set is described as 'held-out' documents that do not appear in SFT or RL, but the CPT set is said to include the whole document collection. Please state explicitly whether these held-out documents were seen during CPT; if so, clarify that this is intentional for the injection-and-utilization test.","section":"§5.1"},{"comment":"All accuracy numbers are reported as point estimates without error bars or significance tests. Given the small gaps between RING and some baselines in the 14B rows, significance testing would strengthen the comparison.","section":"Table 1"},{"comment":"R3_dense is called a 'dense reward' but is a binary exact-match indicator; the terminology is confusing. Consider renaming or explaining why it functions as a dense signal in the RL setting.","section":"§3.4, Eq. (2)"},{"comment":"The mask definition uses Kα=⌊αT⌋, which for α=1/4 makes the target block one quarter of the sequence. It would help to state explicitly how variable-length sequences are handled in batching and how the three forward passes with different masks are implemented without inflating memory costs.","section":"Appendix A.5"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth a look, but read Table 1 before the abstract. RING's core idea—learning the parametric retrieval policy with RL over an MoE memory—is a real step beyond KBLAM, SR-KI, LAG, MLP Memory, and AtlasKV, which all use fixed or rule-based retrieval. The News-2025 benchmark is also a useful new resource: bilingual, post-cutoff, clean train/test split. The ablations are informative: RL adds ~7 points over SFT, the search reward beats answer-only reward, and DCA helps over standard causal attention. So there is substance.\n\nThe problem is the headline claim. The abstract says RING 'matches or surpasses' RAG. Table 1 says otherwise: on Qwen3-8B, the strongest RAG pipeline (Qwen3-Reranker, top-10 to top-3) beats RING by 4.43 EN and 8.69 ZH points; on 14B Chinese, RING trails by 1.90. Only 14B English (41.19 vs. 38.26) actually supports the claim. That is one out of four settings, with no error bars, no significance tests, and no human-validated judge—the evaluation uses a single LLM judge without reported agreement.\n\nThe theory is where I part company with the authors. Theorem A.4 assumes the memory stores units with sufficient fidelity and the router can represent the external retrieval distribution—essentially the conclusion. Proposition A.7 is just monotonicity of conditioning sets, and that monotonicity is exactly what lets DCA leak. For α=1/2 and 1/4, the target block attends to the entire future complement Cα, so CPT can reduce loss by copying from Cα rather than storing content in weights. At inference only the causal mask is active, so the crutch disappears. The paper never reports retrieval fidelity—whether the generated <retrieval> block matches the gold memory unit on held-out questions. Without that, the QA gains could come from SFT/RL memorizing answer patterns rather than from internalized search.\n\nStill, the idea is plausible and the method is more developed than the baselines. The authors are also honest about limitations: generated evidence isn't verbatim, provenance is harder, and the pipeline is expensive. This paper deserves a serious referee, but not as it stands. The headline needs to be corrected, retrieval fidelity needs to be measured directly, and code/data should be released. If those happen, it becomes a solid contribution to parametric knowledge injection.","headline":"Genuinely new idea—learned parametric retrieval via RL—sold with an overclaimed headline; Table 1 only supports 'matches RAG' on 14B English, and the theory assumes what it needs to prove.","tokens_in":20823,"tokens_out":3632,"would_cite":true,"duration_ms":33574,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A fully parametric model can match RAG accuracy by learning to search its own weights.","keywords":["retrieval-internalized generation","parametric knowledge injection","mixture-of-memory experts","reinforcement learning","retrieval-augmented generation","continual knowledge injection","dual causal attention","News-2025"],"falsifier":"Train RING on a corpus whose total token count substantially exceeds the Knowledge Expert's parameter count while holding the query distribution fixed, and measure whether retrieval-block exact-match recall and QA accuracy stay at RAG levels; if accuracy falls below a top-1 embedding-based RAG baseline, the capacity assumption fails. A cheaper probe: on held-out questions whose evidence fragment was never seen in SFT/RL, check whether the generated <retrieval> block reproduces the reference span; low exact-match rates would show the learned policy memorized SFT targets rather than generalizabl","tokens_in":19811,"feed_emoji":"🧠","tokens_out":5868,"duration_ms":58704,"temperature":0.7,"pith_summary":"The paper claims that the reason parametric knowledge injection lags behind retrieval-augmented generation is not storage but search: once a corpus is written into model weights, the hard part is knowing which weight-region holds the answer. RING addresses this by turning retrieval into a learned behavior. A sparse Mixture-of-Memory Experts keeps the original LLM intact in a Basic Expert while a duplicate Knowledge Expert memorizes a 341M-token news corpus; a Router and Search Gate are trained, first by supervised \"search-then-answer\" examples and then by reinforcement learning with hierarchical rewards, to select the right expert and the right memory fragment. The paper reports that RING matches or beats strong RAG pipelines in accuracy while cutting time-to-first-token by 3–19x, and frames the whole pipeline as a variational approximation to the classical RAG objective. If correct, RING removes the external retriever, vector database, and index-maintenance machinery from knowledge-intensive QA over stable corpora.","feed_headline":"Learned search inside the weights matches RAG at 3–19x lower latency","feed_subtitle":"RING stores a 341M-token corpus in its weights, learns to search them, and matches RAG accuracy at lower latency.","key_machinery":"The central mechanism is the Mixture-of-Memory Experts layer: a frozen Basic Expert preserves base abilities, a Knowledge Expert (a duplicated MLP split into Memory Up, Memory Down, and a SiLU-activated Search Gate) stores the injected corpus, and a learned Router makes a top-1 expert choice per token. Dual Causal Attention (three attention masks over target block and complement) injects the corpus bidirectionally while keeping inference causal. The search-then-answer SFT template and the RL stage with hierarchical rewards (format, dense LCS-based search rewards, answer verification) turn parametric retrieval from a static lookup into a trained policy.","core_discovery":"RING's central discovery claim is that the retrieval distribution p(z|x) that RAG computes with an external embedding model and vector index can be learned inside the LLM itself. The paper implements this with a Router that picks between a Basic Expert and a Knowledge Expert at every token, and a Search Gate inside the Knowledge Expert that reweights memory projections query-dependently; the Knowledge Expert's up/down projections are treated as a differentiable key-value index over roughly 1.6 million fragments. Training is a three-stage pipeline: continued pre-training with Dual Causal Attention writes the corpus into the memory projections; supervised fine-tuning teaches the model to emit","pith_inferences":["If the learned retrieval policy generalizes beyond the training corpus distribution, the same architecture could serve multiple knowledge bases by swapping or adding Knowledge Experts, turning knowledge updates into module swaps rather than retraining.","The paper's explicit <retrieval> block is a generated paraphrase, not a verbatim document; this suggests a natural hybrid design where RING proposes evidence and a cheap external verifier confirms it, preserving provenance without paying full RAG latency.","RING's PPL–accuracy coupling implies that RL makes the model less willing to guess from prior knowledge; this coupling could be used as a diagnostic to detect whether a model is actually using injected knowledge or relying on memorized training priors."],"forward_implications":["External retrieval infrastructure (embedding models, vector databases, index refresh) can be dropped for knowledge-intensive QA over corpora that are stable enough to be internalized before deployment.","Serving cost stops growing with corpus size and scales with model FLOPs: RING's time-to-first-token is 3–19x lower than every evaluated RAG variant.","New knowledge can be injected continually without catastrophic forgetting: the Basic Expert and frozen memory projections keep accuracy on old knowledge above 30% at 100k injected documents.","Retrieval becomes optimizable: because the policy is learned by RL, evidence selection can be aligned with answer correctness rather than with embedding similarity, which the paper argues can make RING strictly better than similarity-search RAG on some distributions.","The ELBO framing gives a principled interpretation for why the three stages must be CPT–SFT–RL and why the Knowledge Expert can be viewed as a differentiable key-value memory."],"supporting_citations":[{"why":"Defines the classical RAG formulation and the objective RING approximates in its theoretical section.","marker":"Lewis et al., 2020"},{"why":"Provides evidence that bidirectional attention improves memorization, motivating the Dual Causal Attention design.","marker":"Zhang et al., 2025b"},{"why":"Establishes feed-forward layers as key-value memories, grounding the Memory Up/Down architecture of the Knowledge Expert.","marker":"Geva et al., 2021"},{"why":"Supports the premise that RL improves sampling from a model's parameter space, motivating the RL stage.","marker":"Chen et al., 2026"},{"why":"Supplies the GSPO RL algorithm used to optimize the routing-and-search policy.","marker":"Zheng et al., 2025"},{"why":"Parametric RAG baseline that still depends on an external retriever, the contrast RING removes.","marker":"Su et al., 2025"},{"why":"KBLAM modular-injection baseline with memory linear in corpus size, outperformed by RING.","marker":"Wang et al., 2025"},{"why":"SR-KI baseline, the strongest parametric injection competitor in the comparison.","marker":"Yu et al., 2026"},{"why":"AtlasKV retriever-imitating baseline that stores compressed KV caches, closest competitor among parametric methods.","marker":"Huang et al., 2025"},{"why":"MLP Memory baseline that imitates kNN retrieval distributions, another retriever-imitating approach RING surpasses.","marker":"Wei et al., 2025"}],"fun_headline_variants":["RING learns to search its weights, matching RAG at 3–19x lower latency","No external retriever: RING internalizes search, cuts latency 3–19x","Parametric memory: RING learns retrieval, matches RAG accuracy at 3–19x speed","Drop the retriever: RING's learned search matches RAG at lower latency","RING: internalized retrieval matches RAG accuracy, 3–19x faster"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The load-bearing assumption is capacity: the duplicated Knowledge Expert MLP must have enough parameters to memorize all ~1.6 million fragments with sufficient fidelity, and the Router and Search Gate must have enough capacity to represent the same retrieval distribution as an external embedding retriever.","fun_headline_variants_meta":{"raw":{"variants":["RING learns to search its weights, matching RAG at 3–19x lower latency","No external retriever: RING internalizes search, cuts latency 3–19x","Parametric memory: RING learns retrieval, matches RAG accuracy at 3–19x speed","Drop the retriever: RING's learned search matches RAG at lower latency","RING: internalized retrieval matches RAG accuracy, 3–19x faster"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00032,"raw_usage":{"total_tokens":1651,"prompt_tokens":763,"completion_tokens":888,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":507,"completion_tokens_details":{"reasoning_tokens":770}},"tokens_in":507,"tokens_out":888,"duration_ms":9289,"temperature":1.0,"reasoning_tokens":770,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T23:50:26.899836+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train RING on a corpus whose total token count substantially exceeds the Knowledge Expert's parameter count while holding the query distribution fixed, and measure whether retrieval-block exact-match recall and QA accuracy stay at RAG levels; if accuracy falls below a top-1 embedding-based RAG baseline, the capacity assumption fails. A cheaper probe: on held-out questions whose evidence fragment was never seen in SFT/RL, check whether the generated <retrieval> block reproduces the reference span; low exact-match rates would show the learned policy memorized SFT targets rather than generalizabl","supporting_citations":[],"review_version":1}