Pith. sign in

REVIEW 4 major objections 6 minor 57 references

FedMosaic proposes that federated RAG should work by encoding documents into masked parametric adapters, and presents a framework that reportedly beats existing methods on accuracy while keeping raw documents local.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review

2026-08-03 04:17 UTC pith:R23A2N6R

load-bearing objection A genuinely new architecture for federated parametric RAG, worth engaging seriously, but the empirical support is thinner than the bold claims suggest. the 4 major comments →

arxiv 2602.05235 v2 pith:R23A2N6R submitted 2026-02-05 cs.CL

FedMosaic: Federated Retrieval-Augmented Generation via Parametric Adapters

Zhilin Liang , Yuxiang Wang , Zimu Zhou , Hainan Zhang , Boyi Liu , Yongxin Tong This is my paper
classification cs.CL
keywords federated retrieval-augmented generationparametric adaptersLoRAdocument-specific masksselective adapter aggregationlocality constraintmulti-document adaptersprivacy-preserving RAG
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Federated RAG lets a central LLM answer questions using knowledge scattered across hospitals or firms that refuse to share raw text. The paper argues that the right way to do this is parametric: encode each document into a small adapter module that merges with a frozen LLM, so only parameters ever leave the silo. To make that cheap and accurate, FedMosaic groups similar documents into one shared adapter and learns a binary mask per document that selects which rows of the adapter that document is allowed to use. During a query, silos send only relevance scores and masks; the server picks documents that are both relevant and whose masks barely overlap, then merges their masked adapters. If the paper is right, federated RAG can be both private and practical: roughly 11% more accurate than existing approaches while cutting storage by 78.8% to 86.3% and communication by 91.4%.

Core claim

At its core, FedMosaic rests on a separability hypothesis: document-specific knowledge can be captured by distinct subsets of the rows of a shared low-rank adapter. The paper trains one LoRA adapter per cluster of semantically related documents, freezes it, and then optimizes a sparse binary row mask for each document so that masking the adapter recovers that document's knowledge. At query time, silos rerank local documents and upload only scores and masks; the server selects the top candidates while penalizing mask overlap as a proxy for parameter conflict, and merges the selected adapters as a relevance-weighted sum of masked low-rank updates. The claimed result is that this two-stage desi

What carries the argument

Multi-document parametric adapters with row-wise masks: one frozen cluster-level LoRA adapter per semantic cluster, plus a learned binary mask per document that gates the low-rank update row-wise, with a rescaling factor to keep magnitudes stable. Mask overlap between candidate documents is treated as a conflict signal in a greedy selection objective that the paper shows to be NP-hard; selected adapters are merged through relevance-weighted masked summation. Mask storage is reduced by bit-packing to one byte per eight mask entries.

Load-bearing premise

The whole scheme rests on the untested hypothesis that each document's knowledge lives in a distinct, separable subset of the shared adapter's rows, so a binary row mask can isolate documents and mask overlap can stand in for harmful parameter conflict.

What would settle it

Train one cluster adapter on two semantically unrelated documents with mask training, then measure per-document answer quality after deleting the other document's masked rows. If isolating a document's rows does not preserve its answer quality materially better than random row subsets, the central hypothesis fails. A second check: test whether mask overlap actually correlates with measured negative transfer when the corresponding adapters are merged.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X LinkedIn Reddit HN

If this is right

  • Locality becomes enforceable by construction: silos share scores, masks, and adapter parameters, never document text, so privacy regulations are not violated by the retrieval protocol itself.
  • Per-document storage drops to roughly 11% to 21% of per-document adapter storage, and per-query communication becomes nearly flat as retrieval depth grows.
  • Selective aggregation stabilizes accuracy as the number of retrieved documents grows, in contrast to indiscriminate adapter averaging, which degrades after a few adapters.
  • The offline cluster adapters can serve many queries with only lightweight online selection and merging, and the gains persist when the backbone scales to a larger LLM.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the row-separability hypothesis holds beyond the tested corpora, the mask machinery gives a cheap update path: new knowledge could be added by training a mask on an existing frozen adapter instead of retraining or adding a new adapter, making federated RAG more dynamic.
  • Mask overlap is a geometric proxy for conflict; a stronger engineering variant might measure actual interference by probing merged adapters on a few validation queries before committing to a selection.
  • Uploaded masks are metadata that could leak inter-document similarity to the server; adding noise or formal privacy guarantees to masks is a natural extension the paper does not address.
  • Because the approach is built on generic low-rank adapters, it should transfer to other parameter-efficient fine-tuning schemes and to non-text modalities, provided the separability hypothesis survives those settings.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. FedMosaic proposes a federated retrieval-augmented generation framework built on parametric adapters (LoRA). To avoid transmitting raw documents, each silo clusters semantically related documents and trains one LoRA adapter per cluster. A document-specific binary mask is then learned over the frozen adapter so that each document activates a subset of adapter rows. At query time, silos upload only relevance scores and masks; the server selects k' documents under a conflict-aware criterion based on mask overlap, and aggregates the masked adapters in a relevance-weighted sum. The paper claims an average 10.9% F1 improvement over state-of-the-art baselines across four datasets, storage reductions of 78.8%–86.3%, communication reductions of 91.4%, and strong resistance to data-extraction attacks, while preserving the locality constraint.

Significance. If the central claims hold, FedMosaic would be a meaningful contribution to privacy-preserving federated RAG. The idea of clustering documents into multi-document adapters with row-wise masks is original, and the paper provides a formal NP-hardness reduction for the selection objective, a useful decomposition of storage/communication overhead, and ablation evidence that the mask and selection components help. The method's reliance on parametric adapters is a principled way to satisfy the locality constraint. However, the current evidence is not yet sufficient: the core separability hypothesis is not directly validated, the experimental results lack statistical grounding, and the overhead/privacy claims need clarification.

major comments (4)
  1. [Sec. 3.2.2, Eqs. (4)–(8), Fig. 5] The paper's central mechanism rests on the hypothesis that 'document-specific knowledge can also be captured by distinct subsets of LoRA parameters.' The only supporting evidence is Fig. 5, which compares FedMosaic with a 'w/o Mask' variant. This does not rule out alternative explanations: the improvement could come from additional per-document next-token training on the frozen adapter, from the sparsity regularizer, or from the rescale factor in Eq. (5). To make the claim load-bearing, the authors should test whether row-wise masks actually isolate document knowledge—e.g., compare against random masks, dense masks, or masks trained on a different document; measure overlap between masks of related and unrelated documents; or evaluate per-document QA accuracy under mask gating versus an oracle conflict proxy. Without such evidence, Eqs. (10)–(12) optimize an unvalidated proxy for paramete
  2. [Sec. 4.1, Table 1] All F1 numbers are from a single run on 300 subsampled Q&A instances per dataset, and key hyperparameters (c, k', lambda_ol, tau, alpha, lambda_l1) are selected empirically on the same benchmark (Sec. 4.4). No error bars, confidence intervals, or seeds are reported. Consequently, the headline 10.9% average improvement and the per-subset percentages (e.g., 10.57%, 13.08%, 10.03%) are not statistically grounded. The authors should report multiple runs with variance, and either use a held-out validation split or provide a sensitivity analysis showing that the results are robust to the hyperparameter choices.
  3. [Abstract & Sec. 4.4.1, Fig. 4] The overhead claims are not tied to a clearly defined baseline. The abstract states a 91.4% communication reduction and storage reductions of 78.8%–86.3%, but Fig. 4b reports that at k=10 the cost is reduced to 4.86% of the 'w/o clustering' variant, which corresponds to a ~95% reduction, not 91.4%. It is also unclear whether the reported storage/communication numbers include the masks, the cluster-level adapters, or the bit-packing savings from Eq. (9). Please specify the exact baseline, the parameter settings, and the components counted in each overhead figure.
  4. [Sec. 4.3.1, Table 2] The privacy analysis is narrower than the 'never sharing raw documents' claim. While FedMosaic does not transmit raw text, Table 2 reports nonzero success rates under prefix attacks (e.g., 44.00 for target prompts, 29.53 for target info), so the method is not 'almost immune' in an absolute sense. The paper should state the residual attack surface and discuss what the uploaded masks and relevance scores might reveal about the underlying documents. A formal privacy notion (e.g., DP for the adapters/masks) or an attack model on the uploaded parameters would be needed to support the privacy guarantee.
minor comments (6)
  1. [Appendix] The phrase 'by through a reduction from CLIQUE' is a typo. The reduction is otherwise clear, but the proof would benefit from stating that the objective in Eq. (11) can be written exactly as the Weighted Subgraph Selection objective with a_v = s_v/k' and b_uv = 2λ_ol·overlap(M_u,M_v)/(k'(k'-1)).
  2. [Fig. 5b] The x-axis label 'Implicit Sparsity Radio' appears to be a typo for 'Ratio.' Also clarify how the sparsity ratio is computed and whether the 'FedMosaic' points include the λ_l1=0 setting mentioned in Sec. 4.4.2.
  3. [Algorithm 2, line 16] The algorithm says silos upload 'corresponding LoRA adapters' after selection, but Sec. 3.3.2 says only scores and masks are uploaded initially. Clarify the two-stage communication protocol: what is uploaded before selection and what is uploaded after selection.
  4. [Sec. 4.3.3, Table 4] The text refers to 'AUGPR' but Table 4 lists 'AUGPE.' Use a consistent name and verify the citation.
  5. [Sec. 2.3, Fig. 2] The motivating experiments in Fig. 2 are described informally ('Results are measured on 2WikiMultihopQA Bridge dataset') and appear to be single-run curves with no error bars. Please report the setup (number of documents, epochs, adapter rank, retrieval procedure) or move these to an appendix.
  6. [Related Work] The description of C-FedRAG and FRAG as 'still transmit raw documents' is too broad; the cited methods are described in their own abstracts as preventing leakage or using encrypted search. Clarify what exactly is transmitted in those baselines when adapting them to the federated setting.

Circularity Check

0 steps flagged

No structural circularity: FedMosaic's reported gains are measured against external baselines and its central separability claim is an explicit hypothesis, not a restatement of its inputs.

full rationale

FedMosaic's derivation chain is not circular. Its inputs are external: parametric RAG [30], LoRA [16], constrained k-means clustering [4], public QA benchmarks, and standard model-merging notions. The proposed mechanisms—cluster-level adapters, row-wise binary masks, and conflict-aware selection using mask overlap—are presented as design choices motivated by an openly stated hypothesis in Sec. 3.2.2 ('We hypothesize that document-specific knowledge can also be captured by distinct subsets of LoRA parameters'), not as consequences forced by the inputs. The reported accuracy, storage, communication, and privacy numbers are evaluated against external baselines and against FedMosaic ablations; none are restatements of fitted values or of the method's own definitions. The NP-hardness appendix is an independent polynomial reduction from CLIQUE. There are no load-bearing self-citations by the present authors. The main evaluative weakness—that hyperparameters such as maximum cluster size, λ_l1, λ_ol, τ are chosen empirically on the same evaluation datasets, and mask-training epochs/learning rates are swept without a reported held-out protocol—is an in-sample-tuning/overfitting concern, not a definitional or fit-by-construction circularity. Even if the core separability hypothesis were false, the paper would fail empirically, not circularly. Therefore no significant circularity is present.

Axiom & Free-Parameter Ledger

10 free parameters · 6 axioms · 0 invented entities

The ledger shows the method pulls in standard ML machinery (LoRA, clustering, parameter averaging) but rests on a strong, only partially tested hypothesis: document-specific knowledge can be cleanly separated by row-wise binary masks over shared adapters, and mask overlap measures harmful parameter conflict. Many experimental hyperparameters are not fixed to concrete values, so the reported gains are not yet tied to a reproducible configuration.

free parameters (10)
  • maximum cluster size c = 5-10 (empirically chosen, Sec. 4.4.1)
    Constrained k-means capacity; controls storage versus intra-silo interference; selected on the evaluation datasets.
  • top-k retrieval budget k = 1, 3, 5, 10, 15 (varied in Fig. 4 and Fig. 6)
    Number of candidate documents each silo re-ranks and uploads scores/masks for; no fixed default is stated.
  • selection budget k' = varied 1-30 in Table 5; no default stated
    Number of documents selected globally for adapter aggregation in Eq. (11).
  • conflict penalty lambda_ol = not specified
    Balances relevance against mask overlap in Eq. (11); no value or tuning protocol is given.
  • relevance threshold tau = not specified
    Filters low-score candidates in Sec. 3.3.2; value absent.
  • sparsity regularization lambda_l1 = not specified; ablation sets it to 0
    Controls mask sparsity in Eq. (7); trade-off with accuracy is shown but no default is fixed.
  • sharpening factor alpha = not specified
    Sigmoid sharpening in Eq. (7); no value given.
  • LoRA rank r = not specified
    Low-rank dimension of adapters; inherited from PRAG but unreported in this paper.
  • augmentation counts n and m = not specified
    Number of rewrites and QA pairs per document in Eq. (1); affects training cost and adapter quality.
  • Dirichlet alpha for silo partition = 0.1
    Controls document distribution skew across silos in the experimental protocol; a simulation choice, not a method parameter.
axioms (6)
  • domain assumption Locality constraint is a hard requirement: no plaintext document may leave its silo
    Defines the FedRAG problem in Sec. 2.1 and motivates the entire adapter-based design; if only weak privacy were required, in-context RAG with encryption or anonymization might be acceptable.
  • domain assumption All silos and the server share the same base LLM and a homogeneous re-ranking model M_r
    Stated in Sec. 3.1 as a standard cross-silo federated learning assumption; FedMosaic's mask and aggregation compatibility depends on identical model parameterizations.
  • domain assumption A document's knowledge is captured by a distinct subset of LoRA rows, so a row-wise binary mask can isolate it within a shared adapter
    Hypothesized in Sec. 3.2.2; both multi-document adapters and conflict-aware selection (using mask overlap as conflict) rest on this. Ablation Fig. 5 gives indirect support but no proof.
  • domain assumption Weighted averaging of masked LoRA updates is a valid composition mechanism if masks are low-overlap
    Eq. (13) assumes parameter-space averaging preserves knowledge when conflicts are suppressed; cited model-merging literature is relied on, but no formal guarantee for this specific masked merge is given.
  • domain assumption LLM-generated rewrites and QA pairs in Eq. (1) faithfully represent document content and can be filtered for sensitive material
    The privacy argument in Sec. 4.3.1 depends on training from rewritten data with simple filtering prompts; if filtering fails, leakage risk changes.
  • domain assumption Constrained k-means on embedding vectors yields semantically coherent clusters that do not interfere during adapter training
    Used in Eq. (3); semantic similarity in embedding space is assumed to align with adapter non-interference, validated only indirectly by F1 ablations.

reviewed 2026-08-03 · how reviews work

0 comments
Cite this review

Pith. "Pith review of FedMosaic: Federated Retrieval-Augmented Generation via Parametric Adapters." pith.science (2026). https://pith.science/paper/R23A2N6R

@misc{pith2026260205235,
  author       = {Pith},
  title        = {Pith review of: FedMosaic: Federated Retrieval-Augmented Generation via Parametric Adapters},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/R23A2N6R}},
  note         = {Machine review of arXiv:2602.05235}
}
Share X LinkedIn Reddit HN
read the original abstract

Retrieval-Augmented Generation (RAG) enhances Large Language Models (LLMs) by grounding generation in external knowledge to improve factuality and reduce hallucinations. Yet most deployments assume a centralized corpus, which is infeasible in privacy aware domains where knowledge remains siloed. This motivates federated RAG (FedRAG), where a central LLM server collaborates with distributed silos without sharing raw documents. In context RAG violates this requirement by transmitting verbatim documents, whereas parametric RAG encodes documents into lightweight adapters that merge with a frozen LLM at inference, avoiding raw-text exchange. We adopt the parametric approach but face two unique challenges induced by FedRAG: high storage and communication from per-document adapters, and destructive aggregation caused by indiscriminately merging multiple adapters. We present FedMosaic, the first federated RAG framework built on parametric adapters. FedMosaic clusters semantically related documents into multi-document adapters with document-specific masks to reduce overhead while preserving specificity, and performs selective adapter aggregation to combine only relevance-aligned, nonconflicting adapters. Experiments show that FedMosaic achieves an average 10.9% higher accuracy than state-of-the-art methods in four categories, while lowering storage costs by 78.8% to 86.3% and communication costs by 91.4%, and never sharing raw documents.

Figures

Figures reproduced from arXiv: 2602.05235 by Boyi Liu, Hainan Zhang, Yongxin Tong, Yuxiang Wang, Zhilin Liang, Zimu Zhou.

Figure 1
Figure 1. Figure 1: Federated RAG with locality constraint. Despite rapid progress in RAG, existing methods cannot be di￾rectly extended to the federated setting because they violate the locality constraint. Conventional RAG depends on in-context in￾tegration, where retrieved passages are inserted into the LLM’s prompt [10, 14, 19, 29, 43]. In a federated environment, this re￾quires the server to fetch verbatim documents from… view at source ↗
Figure 2
Figure 2. Figure 2: Accuracy curves of parametric RAG for (a) grouped [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: FedMosaic architecture and workflow. document-specific masks that gate adapter parameters dur￾ing adapter aggregation and answer generation. This design preserves per-document specificity and mitigates intra-silo adapter interference during adapter sharing. • Selective Adapter Aggregation (Sec. 3.3). To mitigate inter￾silo adapter interference during adapter averaging, FedMo￾saic aggregates only adapters a… view at source ↗
Figure 4
Figure 4. Figure 4: Storage and communication overhead. in-context FedRAG. We consider two data extraction attacks: tar￾geted and prefix [5, 46]. We construct 300 privacy-sensitive samples following [47], and then apply retrieval-data attacks to in-context FedRAG and training-data attacks to FedMosaic. The in-context FedRAG (i.e., IC-FedRAG) adopts a minimal FRAG [1, 49] design, capturing key characteristics of MKPQA [28] and… view at source ↗
Figure 5
Figure 5. Figure 5: Impact of document mask. and, even when applied document-wise (e.g. DP-Prompt), can dam￾age fine-grained knowledge crucial for accurate RAG. Similarly, anonymization-based methods such as Sage also lose substantial information, with an average 53% degradation, indicating that key evidence for Q&A is often compromised during anonymizatio. Over￾all, the poor utility of these approaches in RAG tasks makes the… view at source ↗
Figure 6
Figure 6. Figure 6: Performance of federated RAG under varying top- [PITH_FULL_IMAGE:figures/full_fig_p009_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

57 extracted references · 4 linked inside Pith

  1. [1]

    Parker Addison, Minh-Tuan H Nguyen, Tomislav Medan, Jinali Shah, Moham- mad T Manzari, Brendan McElrone, Laksh Lalwani, Aboli More, Smita Sharma, Holger R Roth, et al. 2024. C-fedrag: A confidential federated retrieval-augmented generation system.arXiv preprint arXiv:2412.13163(2024)

    Pith/arXiv arXiv 2024

  2. [2]

    Immanuel M Bomze, Marco Budinich, Panos M Pardalos, and Marcello Pelillo

  3. [3]

    Kym M Boycott and Roberto Giugliani. 2025. The RDI–Lancet Commission on Rare Diseases: improving visibility to address health-care disparities for 400 million people.The Lancet405, 10479 (2025), 605–607

    2025

  4. [4]

    Paul S Bradley, Kristin P Bennett, and Ayhan Demiriz. 2000. Constrained k-means clustering.Microsoft Research, Redmond20 (2000)

    2000

  5. [5]

    Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang. 2022. Quantifying memorization across neural language models. InICLR

    2022

  6. [6]

    Zhe Chen, Yusheng Liao, Shuyang Jiang, Pingjie Wang, YiQiu Guo, Yanfeng Wang, and Yu Wang. 2025. Towards omni-RAG: Comprehensive retrieval-augmented generation for large language models in medical applications. InACL. 15285– 15309

    2025

  7. [7]

    Xin Cheng, Di Luo, Xiuying Chen, Lemao Liu, Dongyan Zhao, and Rui Yan. 2023. Lift yourself up: Retrieval-augmented text generation with self-memory.NeurIPS 36 (2023), 43780–43799

    2023

  8. [8]

    Congress

    U.S. Congress. 1996. Health Insurance Portability and Accountability Act of 1996. https://www.govinfo.gov/content/pkg/PLAW-104publ191/pdf/PLAW- 104publ191.pdf. Accessed: 2025-9-19

    1996

  9. [9]

    Chenxu Cui, Haihui Fan, Jinchao Zhang, Lin Shen, Bo Li, and Weiping Wang

  10. [10]

    Guanting Dong, Yutao Zhu, Chenghao Zhang, Zechen Wang, Ji-Rong Wen, and Zhicheng Dou. 2025. Understand what LLM needs: Dual preference alignment for retrieval-augmented generation. InWWW. 4206–4225

    2025

  11. [11]

    Guodong Du, Zitao Fang, Jing Li, Junlin Li, Runhua Jiang, Shuyang Yu, Yifei Guo, Yangneng Chen, Sim Kuan Goh, Ho-Kin Tang, Daojing He, Honghai Liu, and Min Zhang. 2025. Neural Parameter Search for Slimmer Fine-Tuned Models and Better Transfer. InACL. 32668–32687

    2025

  12. [12]

    Wenqi Fan, Yujuan Ding, Liangbo Ning, Shijie Wang, Hengyun Li, Dawei Yin, Tat-Seng Chua, and Qing Li. 2024. A survey on rag meeting llms: Towards retrieval-augmented large language models. InSIGKDD. 6491–6501

    2024

  13. [13]

    Rachid Guerraoui, Anne-Marie Kermarrec, Diana Petrescu, Rafael Pires, Mathis Randl, and Martijn de Vos. 2025. Efficient federated search for retrieval- augmented generation. InProceedings of the 5th Workshop on Machine Learning and Systems. 74–81

    2025

  14. [14]

    Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang

  15. [15]

    Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. 2020. Constructing a multi-hop qa dataset for comprehensive evaluation of reasoning steps. InCOLING. 6609–6625

    2020

  16. [16]

    Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. 2022. LoRA: Low-rank adaptation of large language models. InICLR

    2022

  17. [17]

    Chenyu Huang, Peng Ye, Tao Chen, Tong He, Xiangyu Yue, and Wanli Ouyang

  18. [18]

    Dahyun Lee, Yongrae Jo, Haeju Park, and Moontae Lee. 2025. Shifting from Ranking to Set Selection for Retrieval Augmented Generation. InACL. 17606– 17619

    2025

  19. [19]

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al

  20. [20]

    Jianghao Lin, Rong Shan, Chenxu Zhu, Kounianhua Du, Bo Chen, Shigang Quan, Ruiming Tang, Yong Yu, and Weinan Zhang. 2024. Rella: Retrieval-enhanced large language models for lifelong sequential behavior comprehension in recom- mendation. InWWW. 3497–3508

    2024

  21. [21]

    Zhenyi Lu, Chenghao Fan, Wei Wei, Xiaoye Qu, Dangyang Chen, and Yu Cheng

  22. [22]

    Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. When not to trust language models: Investigating effectiveness of parametric and non-parametric memories. InACL. 9802–9822

    2023

  23. [23]

    Guillermo Ortiz-Jimenez, Alessandro Favero, and Pascal Frossard. 2023. Task arithmetic in the tangent space: Improved editing of pre-trained models.NeurIPS 36 (2023), 66727–66754

    2023

  24. [24]

    Retrieval-augmented generation for knowledge-intensive nlp tasks.NeurIPS 33 (2020), 9459–9474

    2020

  25. [25]

    Stephen Robertson, Hugo Zaragoza, et al . 2009. The probabilistic relevance framework: BM25 and beyond.Foundations and Trends®in Information Retrieval 3 (2009), 333–389

    2009

  26. [26]

    Tianyi Shen, Yuxi Li, Yanlin Cao, Xin Du, Xinru Wang, Yajuan Zhang, and Yi Zhang. 2025. Rapid deployment of large language model DeepSeek in Chinese hospitals demands a regulatory response.Nature Medicine(2025), 1–6

    2025

  27. [27]

    NeurIPS37 (2024), 78905–78935

    Twin-merging: Dynamic integration of modular expertise in model merging. NeurIPS37 (2024), 78905–78935

    2024

  28. [28]

    Parshin Shojaee, Sai Sree Harsha, Dan Luo, Akash Maharaj, Tong Yu, and Yunyao Li. 2025. Federated retrieval augmented generation for multi-product question answering. InCOLING, Vol. Industry Track. 387–397

    2025

  29. [29]

    Weihang Su, Yichen Tang, Qingyao Ai, Zhijing Wu, and Yiqun Liu. 2024. DRAGIN: Dynamic retrieval augmented generation based on the real-time information needs of large language models. InACL. 12991–13013

    2024

  30. [30]

    Tao Ouyang, Guihang Hong, Kongyange Zhao, Zhi Zhou, Weigang Wu, Zhaobiao Lv, and Xu Chen. 2025. AdaRAG: Adaptive Optimization for Retrieval Augmented Generation with Multilevel Retrievers at the Edge. InINFOCOM. IEEE, 1–10

    2025

  31. [31]

    Alon Talmor and Jonathan Berant. 2018. The web as a knowledge-base for answering complex questions. InNAACL. 641–651

    2018

  32. [32]

    Yuqiao Tan, Shizhu He, Huanxuan Liao, Jun Zhao, and Kang Liu. 2025. Dynamic parametric retrieval augmented generation for test-time knowledge enhancement. arXiv preprint arXiv:2503.23895(2025)

    Pith/arXiv arXiv 2025

  33. [33]

    Teng Shi, Jun Xu, Xiao Zhang, Xiaoxue Zang, Kai Zheng, Yang Song, and Han Li

  34. [34]

    Retrieval Augmented Generation with Collaborative Filtering for Personal- ized Text Generation. InSIGIR

  35. [35]

    Shuai Wang, Ekaterina Khramtsova, Shengyao Zhuang, and Guido Zuccon. 2024. Feb4rag: Evaluating federated search in the context of retrieval augmented gen- eration. InSIGIR. 763–773

    2024

  36. [36]

    Yuhao Wang, Ruiyang Ren, Yucheng Wang, Wayne Xin Zhao, Jing Liu, Hua Wu, and Haifeng Wang. 2025. Unveiling Knowledge Utilization Mechanisms in LLM-based Retrieval-Augmented Generation. InSIGIR

    2025

  37. [37]

    Weihang Su, Yichen Tang, Qingyao Ai, Junxi Yan, Changyue Wang, Hongning Wang, Ziyi Ye, Yujia Zhou, and Yiqun Liu. 2025. Parametric retrieval augmented generation. InSIGIR. 1240–1250

    2025

  38. [38]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models.NeurIPS35 (2022), 24824–24837

    2022

  39. [39]

    Chulin Xie, Zinan Lin, Arturs Backurs, Sivakanth Gopi, Da Yu, Huseyin A Inan, Harsha Nori, Haotian Jiang, Huishuai Zhang, Yin Tat Lee, et al. 2024. Differentially private synthetic data via foundation model apis 2: Text. InICML

    2024

  40. [40]

    European Union. 2016. Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation). https://eur-lex.europa.eu/eli/reg/2016/679/oj/eng. Ac...

    2016

  41. [41]

    Saiteja Utpala, Sara Hooker, and Pin-Yu Chen. 2023. Locally differentially private document generation using zero shot prompting. InFindings of EMNLP

    2023

  42. [42]

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D Manning. 2018. HotpotQA: A dataset for diverse, explainable multi-hop question answering. InEMNLP. 2369–2380

    2018

  43. [43]

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, and et al. 2023. React: Synergizing reasoning and acting in language models. InICLR

    2023

  44. [44]

    Ziyao Wang, Zheyu Shen, Yexiao He, Guoheng Sun, Hongyi Wang, Lingjuan Lyu, and Ang Li. 2024. Flora: Federated fine-tuning large language models with heterogeneous low-rank adaptations.NeurIPS37 (2024), 22513–22533

    2024

  45. [45]

    Zichun Yu, Chenyan Xiong, Shi Yu, and Zhiyuan Liu. 2023. Augmentation- adapted retriever improves generalization of language models as generic plug-in. InACL. 2421–2436

    2023

  46. [46]

    Shenglai Zeng, Jiankun Zhang, Pengfei He, Yiding Liu, Yue Xing, Han Xu, Jie Ren, Yi Chang, Shuaiqiang Wang, Dawei Yin, et al. 2024. The good and the bad: Exploring privacy issues in retrieval-augmented generation (RAG). InFindings of ACL. 4505–4524

    2024

  47. [47]

    Jing Xu and Jingzhao Zhang. 2024. Random masking finds winning tickets for parameter efficient fine-tuning. InICML. 55501–55524

    2024

  48. [48]

    Shicheng Xu, Liang Pang, Jun Xu, Huawei Shen, and Xueqi Cheng. 2024. List- aware reranking-truncation joint model for search and retrieval-augmented generation. InWWW. 1330–1340

    2024

  49. [49]

    Dongfang Zhao. 2024. Frag: Toward federated vector database management for collaborative and secure retrieval-augmented generation.arXiv preprint arXiv:2410.13272(2024)

    Pith/arXiv arXiv 2024

  50. [50]

    Xuejiao Zhao, Siyan Liu, Su-Yin Yang, and Chunyan Miao. 2025. Medrag: Enhanc- ing retrieval-augmented generation with knowledge graph-elicited reasoning for healthcare copilot. InWWW. 4442–4457

    2025

  51. [51]

    Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, and Yongbin Li. 2024. Language models are super mario: Absorbing abilities from homologous models as a free lunch. In ICML

    2024

  52. [54]

    Shenglai Zeng, Jiankun Zhang, Pengfei He, Jie Ren, Tianqi Zheng, Hanqing Lu, Han Xu, Hui Liu, Yue Xing, and Jiliang Tang. 2024. Mitigating the privacy issues in retrieval-augmented generation (rag) via pure synthetic data.arXiv preprint arXiv:2406.14773(2024)

    Pith/arXiv arXiv 2024

  53. [55]

    Jianyi Zhang, Saeed Vahidian, Martin Kuo, Chunyuan Li, Ruiyi Zhang, Tong Yu, Guoyin Wang, and Yiran Chen. 2024. Towards building the federatedgpt: Federated instruction tuning. InICASSP. 6915–6919

    2024

  54. [1999]

    InHandbook of Combinatorial Optimization: Supplement Volume A

    The maximum clique problem. InHandbook of Combinatorial Optimization: Supplement Volume A. Springer, 1–74

  55. [2020]

    Retrieval augmented language model pre-training. InICML. 3929–3938

  56. [2024]

    Emr-merging: Tuning-free high-performance model merging.NeurIPS37 (2024), 122741–122769

    2024

  57. [2025]

    CIRAG: Retrieval-Augmented Language Model with Collective Intelligence. InSIGIR. 1316–1326

This paper was first reviewed by deepseek-v4-flash on August 3, 2026.