{"id":"70c65ea2-2956-4de9-af8c-01ccfd5a7dc0","arxiv_id":"2602.05235","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":10,"one_line_summary":"FedMosaic is a federated RAG system that encodes local documents as mask-gated LoRA adapters, clusters related documents into shared adapters, and selectively merges only relevant, low-conflict adapters at the server.","lead":"FedMosaic lets privacy-protected data silos share their knowledge for question answering by sending small trainable 'patches' (LoRA adapters) instead of raw documents. It reports higher accuracy than existing federated RAG methods while cutting storage and communication costs, which could make cross-hospital or cross-company knowledge assistants practical.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The core hypothesis that document knowledge is separable into row-wise LoRA mask subsets is unproven; the provided ablation does not rule out alternative explanations, and conflict-aware aggregation's validity rests entirely on it.","rationale":"The reader's weakest_assumption identified exactly the same load-bearing concern: the unproven hypothesis that document-specific knowledge is cleanly separable into distinct subsets of LoRA rows. I agree that this is the most critical point. The paper states the hypothesis explicitly in Sec. 3.2.2 and provides only indirect support via an ablation (Fig. 5). The ablation shows that adding masks improves accuracy, but it does not isolate the mechanism. The conflict-aware selection (Eq. 11) uses mask overlap as a proxy for harmful parameter conflicts; if separability fails, this proxy is invalid. The central claim of a locality-preserving, efficient, and accurate federated RAG framework depends on the masks being both document-specific and non-overlapping for non-conflicting documents. Since the reader's verdict is CONDITIONAL with moderate confidence, and my analysis reinforces the same condition rather than discovering a new fatal flaw, I keep the verdict unchanged. The concrete test I propose would directly falsify or support the hypothesis, and if it fails, the system's mechanism would need substantial revision. I did not identify any additional concern that would move the verdict to REJECT, because the paper is honest about the hypothesis and the issue is empirically addressable. The experimental weaknesses (no error bars, small subsamples, no code release) are secondary and already noted by the reader; they also contribute to the CONDITIONAL verdict but are not the single most load-bearing issue. My assessment is a good-faith reading: the system is plausible, but its foundation needs direct validation before the claims can be accepted.","tokens_in":18188,"tokens_out":3454,"duration_ms":37809,"concrete_test":"Directly test the separability of document-specific knowledge in a controlled setup on 2WikiMultihopQA with LLaMA3.2-1B. Train a cluster adapter on a mixture of two document sets A and B, then train masks M_A and M_B following Sec. 3.2.2. (1) Compute overlap(M_A, M_B) = <M_A, M_B>/d; if it exceeds 0.8, the masks do not select distinct subspaces. (2) Evaluate the masked adapter M_A∘B on held-out QA from B: if its F1 is close to that on A (i.e., leakage), the mask fails to isolate. (3) Compare M_A∘B against an oracle adapter trained only on A: if M_A∘B underperforms by more than a small margin, separability is incomplete. (4) Repeat with cluster sizes 5/10 and LoRA ranks r=4/8/16 to see if separability degrades as cluster size grows or rank shrinks. If masks are not distinct and non-leaky, the core hypothesis is refuted and the conflict-aware aggregation Eq. (13) lacks a valid basis.","verdict_should_be":"UNCHANGED","load_bearing_attack":"FedMosaic's central mechanism and the claimed accuracy/overhead trade-off hinge on the hypothesis stated in Sec. 3.2.2: 'document-specific knowledge can also be captured by distinct subsets of LoRA parameters.' This underlies both multi-document adapters (row-wise masks, Eq. 4) and conflict-aware selection (mask overlap as conflict proxy, Eqs. 10-12). The only direct evidence is the ablation in Sec. 4.4.2 (Fig. 5), which shows that mask training improves F1 over a no-mask variant. That result is consistent with alternative explanations: the mask may act as a sparsity regularizer, the rescale factor λ (Eq. 5) may stabilize magnitudes, or the improvement may simply come from additional training on the document's own data while the adapter is frozen. The paper does not demonstrate that a row-wise binary mask on a jointly trained cluster adapter actually isolates document-specific knowledge. Because the cluster adapter is trained on all documents in the cluster, information for a given document is likely distributed across many rows and entangled with others; a rank-r adapter (r≪d) may not have sufficient capacity to separate k documents via row selection alone. If the hypothesis is false, then intra-silo interference is not actually mitigated, and the overlap penalty in Eqs. (10)-(12) optimizes an invalid proxy for parameter conflict, so the selective aggregation story collapses. The central quantitative claims (10.9% average improvement, 78.8-86.3% storage reduction, 91.4% communication reduction) are also weakened by the absence of error bars, 300-instance subsamples, and no code release, but the mask-separability assumption is the most load-bearing because even perfect experiments would not rescue the method if this premise fails.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"FedMosaic proposes a federated retrieval-augmented generation framework built on parametric adapters (LoRA). To avoid transmitting raw documents, each silo clusters semantically related documents and trains one LoRA adapter per cluster. A document-specific binary mask is then learned over the frozen adapter so that each document activates a subset of adapter rows. At query time, silos upload only relevance scores and masks; the server selects k' documents under a conflict-aware criterion based on mask overlap, and aggregates the masked adapters in a relevance-weighted sum. The paper claims an average 10.9% F1 improvement over state-of-the-art baselines across four datasets, storage reductions of 78.8%–86.3%, communication reductions of 91.4%, and strong resistance to data-extraction attacks, while preserving the locality constraint.","tokens_in":18537,"tokens_out":4732,"duration_ms":51235,"significance":"If the central claims hold, FedMosaic would be a meaningful contribution to privacy-preserving federated RAG. The idea of clustering documents into multi-document adapters with row-wise masks is original, and the paper provides a formal NP-hardness reduction for the selection objective, a useful decomposition of storage/communication overhead, and ablation evidence that the mask and selection components help. The method's reliance on parametric adapters is a principled way to satisfy the locality constraint. However, the current evidence is not yet sufficient: the core separability hypothesis is not directly validated, the experimental results lack statistical grounding, and the overhead/privacy claims need clarification.","major_comments":[{"comment":"The paper's central mechanism rests on the hypothesis that 'document-specific knowledge can also be captured by distinct subsets of LoRA parameters.' The only supporting evidence is Fig. 5, which compares FedMosaic with a 'w/o Mask' variant. This does not rule out alternative explanations: the improvement could come from additional per-document next-token training on the frozen adapter, from the sparsity regularizer, or from the rescale factor in Eq. (5). To make the claim load-bearing, the authors should test whether row-wise masks actually isolate document knowledge—e.g., compare against random masks, dense masks, or masks trained on a different document; measure overlap between masks of related and unrelated documents; or evaluate per-document QA accuracy under mask gating versus an oracle conflict proxy. Without such evidence, Eqs. (10)–(12) optimize an unvalidated proxy for paramete","section":"Sec. 3.2.2, Eqs. (4)–(8), Fig. 5"},{"comment":"All F1 numbers are from a single run on 300 subsampled Q&A instances per dataset, and key hyperparameters (c, k', lambda_ol, tau, alpha, lambda_l1) are selected empirically on the same benchmark (Sec. 4.4). No error bars, confidence intervals, or seeds are reported. Consequently, the headline 10.9% average improvement and the per-subset percentages (e.g., 10.57%, 13.08%, 10.03%) are not statistically grounded. The authors should report multiple runs with variance, and either use a held-out validation split or provide a sensitivity analysis showing that the results are robust to the hyperparameter choices.","section":"Sec. 4.1, Table 1"},{"comment":"The overhead claims are not tied to a clearly defined baseline. The abstract states a 91.4% communication reduction and storage reductions of 78.8%–86.3%, but Fig. 4b reports that at k=10 the cost is reduced to 4.86% of the 'w/o clustering' variant, which corresponds to a ~95% reduction, not 91.4%. It is also unclear whether the reported storage/communication numbers include the masks, the cluster-level adapters, or the bit-packing savings from Eq. (9). Please specify the exact baseline, the parameter settings, and the components counted in each overhead figure.","section":"Abstract & Sec. 4.4.1, Fig. 4"},{"comment":"The privacy analysis is narrower than the 'never sharing raw documents' claim. While FedMosaic does not transmit raw text, Table 2 reports nonzero success rates under prefix attacks (e.g., 44.00 for target prompts, 29.53 for target info), so the method is not 'almost immune' in an absolute sense. The paper should state the residual attack surface and discuss what the uploaded masks and relevance scores might reveal about the underlying documents. A formal privacy notion (e.g., DP for the adapters/masks) or an attack model on the uploaded parameters would be needed to support the privacy guarantee.","section":"Sec. 4.3.1, Table 2"}],"minor_comments":[{"comment":"The phrase 'by through a reduction from CLIQUE' is a typo. The reduction is otherwise clear, but the proof would benefit from stating that the objective in Eq. (11) can be written exactly as the Weighted Subgraph Selection objective with a_v = s_v/k' and b_uv = 2λ_ol·overlap(M_u,M_v)/(k'(k'-1)).","section":"Appendix"},{"comment":"The x-axis label 'Implicit Sparsity Radio' appears to be a typo for 'Ratio.' Also clarify how the sparsity ratio is computed and whether the 'FedMosaic' points include the λ_l1=0 setting mentioned in Sec. 4.4.2.","section":"Fig. 5b"},{"comment":"The algorithm says silos upload 'corresponding LoRA adapters' after selection, but Sec. 3.3.2 says only scores and masks are uploaded initially. Clarify the two-stage communication protocol: what is uploaded before selection and what is uploaded after selection.","section":"Algorithm 2, line 16"},{"comment":"The text refers to 'AUGPR' but Table 4 lists 'AUGPE.' Use a consistent name and verify the citation.","section":"Sec. 4.3.3, Table 4"},{"comment":"The motivating experiments in Fig. 2 are described informally ('Results are measured on 2WikiMultihopQA Bridge dataset') and appear to be single-run curves with no error bars. Please report the setup (number of documents, epochs, adapter rank, retrieval procedure) or move these to an appendix.","section":"Sec. 2.3, Fig. 2"},{"comment":"The description of C-FedRAG and FRAG as 'still transmit raw documents' is too broad; the cited methods are described in their own abstracts as preventing leakage or using encrypted search. Clarify what exactly is transmitted in those baselines when adapting them to the federated setting.","section":"Related Work"}],"recommendation":"major_revision","confidential_remarks":"The paper is promising and the technical core is coherent, but the central separability hypothesis is not directly validated and the experimental evidence is under-powered. I would not reject it, but the current version is not ready for acceptance. The authors should be asked to (i) add controlled experiments that isolate the mechanism behind the masks, (ii) report error bars and a validation strategy for the many hyperparameters, and (iii) clarify the overhead and privacy claims. The 'first' claim in the abstract should also be checked against a broader literature search, as the related work does not cover all recent parametric-RAG variants."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"FedMosaic is a real contribution: it is the first federated RAG system I know of that builds on parametric adapters, and it does so with a sensible, coherent design. Clustering documents into multi-document LoRA adapters, learning per-document binary masks over adapter rows, and then selecting adapters based on both relevance and mask overlap is a fresh combination. The NP-hardness proof for the selection problem is a nice formal addition, and the overhead analysis is clearly presented. I also give credit for the broad ablation coverage, including the impact of cluster size, retrieval k, and selection k. The idea that document knowledge can be isolated via row-wise masks is explicitly stated as a hypothesis, and the paper does not overclaim it as a proven fact in the main text.\n\nNow the soft spots. The headline numbers rest on thin empirical ground: single runs without error bars, only 300 Q&A instances per dataset, and several hyperparameters (cluster size, thresholds, penalties) apparently chosen on evaluation sets. The abstract’s 10.9% average improvement is not cleanly reconciled with the per-dataset numbers in Table 1. That matters because the improvements over the strongest baselines are modest in places (e.g., 2WQA Compare vs. PRAG is about 4 points). The most load-bearing assumption is the mask-separability hypothesis. The ablation in Fig. 5 shows that mask training helps, but it does not rule out alternative explanations: the masks may act as a sparsity regularizer, or the rescale factor may stabilize magnitudes. I would like to see a direct test, e.g., comparing learned masks to random masks with matching sparsity, or probing whether a row subset truly corresponds to document identity. If the hypothesis fails, the conflict-aware aggregation loses its foundation.\n\nThere are also smaller issues: the privacy evaluation shows nonzero prefix-attack leakage, so the abstract’s “never sharing raw documents” is true as an architectural property but should not be read as a privacy guarantee. And there are typos/nomenclature errors: “Dargin” should be “DRAGIN”, “FLora” should be “FLoRA”, “AUGPE” vs. “AUGPR”, and “IC-FedRAG” is written inconsistently. These are fixable but suggest a hasty final pass.\n\nOverall, the architecture is sound and the direction is important for privacy-sensitive federated settings. The paper deserves serious peer review, but the reviewers should insist on code release, error bars, a clear description of the hyperparameter selection protocol, and a direct investigation of mask separability. The core idea is good enough that I would want to see it properly tested rather than desk-rejected.\n\nFor you: if you work on federated RAG or parameter-efficient adaptation, this is worth reading and citing as related work, but not as a definitive result yet.","headline":"A genuinely new architecture for federated parametric RAG, worth engaging seriously, but the empirical support is thinner than the bold claims suggest.","tokens_in":19137,"tokens_out":1754,"would_cite":true,"duration_ms":18324,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FedMosaic proposes that federated RAG should work by encoding documents into masked parametric adapters, and presents a framework that reportedly beats existing methods on accuracy while keeping raw documents local.","keywords":["federated retrieval-augmented generation","parametric adapters","LoRA","document-specific masks","selective adapter aggregation","locality constraint","multi-document adapters","privacy-preserving RAG"],"falsifier":"Train one cluster adapter on two semantically unrelated documents with mask training, then measure per-document answer quality after deleting the other document's masked rows. If isolating a document's rows does not preserve its answer quality materially better than random row subsets, the central hypothesis fails. A second check: test whether mask overlap actually correlates with measured negative transfer when the corresponding adapters are merged.","tokens_in":18021,"feed_emoji":"🧩","tokens_out":4329,"duration_ms":44305,"temperature":0.7,"pith_summary":"Federated RAG lets a central LLM answer questions using knowledge scattered across hospitals or firms that refuse to share raw text. The paper argues that the right way to do this is parametric: encode each document into a small adapter module that merges with a frozen LLM, so only parameters ever leave the silo. To make that cheap and accurate, FedMosaic groups similar documents into one shared adapter and learns a binary mask per document that selects which rows of the adapter that document is allowed to use. During a query, silos send only relevance scores and masks; the server picks documents that are both relevant and whose masks barely overlap, then merges their masked adapters. If the paper is right, federated RAG can be both private and practical: roughly 11% more accurate than existing approaches while cutting storage by 78.8% to 86.3% and communication by 91.4%.","feed_headline":"Federated RAG gains 10.9% accuracy with private parametric adapters","feed_subtitle":"Clustered adapters with per-document masks cut storage by up to 86% and communication by 91%.","key_machinery":"Multi-document parametric adapters with row-wise masks: one frozen cluster-level LoRA adapter per semantic cluster, plus a learned binary mask per document that gates the low-rank update row-wise, with a rescaling factor to keep magnitudes stable. Mask overlap between candidate documents is treated as a conflict signal in a greedy selection objective that the paper shows to be NP-hard; selected adapters are merged through relevance-weighted masked summation. Mask storage is reduced by bit-packing to one byte per eight mask entries.","core_discovery":"At its core, FedMosaic rests on a separability hypothesis: document-specific knowledge can be captured by distinct subsets of the rows of a shared low-rank adapter. The paper trains one LoRA adapter per cluster of semantically related documents, freezes it, and then optimizes a sparse binary row mask for each document so that masking the adapter recovers that document's knowledge. At query time, silos rerank local documents and upload only scores and masks; the server selects the top candidates while penalizing mask overlap as a proxy for parameter conflict, and merges the selected adapters as a relevance-weighted sum of masked low-rank updates. The claimed result is that this two-stage desi","pith_inferences":["If the row-separability hypothesis holds beyond the tested corpora, the mask machinery gives a cheap update path: new knowledge could be added by training a mask on an existing frozen adapter instead of retraining or adding a new adapter, making federated RAG more dynamic.","Mask overlap is a geometric proxy for conflict; a stronger engineering variant might measure actual interference by probing merged adapters on a few validation queries before committing to a selection.","Uploaded masks are metadata that could leak inter-document similarity to the server; adding noise or formal privacy guarantees to masks is a natural extension the paper does not address.","Because the approach is built on generic low-rank adapters, it should transfer to other parameter-efficient fine-tuning schemes and to non-text modalities, provided the separability hypothesis survives those settings."],"forward_implications":["Locality becomes enforceable by construction: silos share scores, masks, and adapter parameters, never document text, so privacy regulations are not violated by the retrieval protocol itself.","Per-document storage drops to roughly 11% to 21% of per-document adapter storage, and per-query communication becomes nearly flat as retrieval depth grows.","Selective aggregation stabilizes accuracy as the number of retrieved documents grows, in contrast to indiscriminate adapter averaging, which degrades after a few adapters.","The offline cluster adapters can serve many queries with only lightweight online selection and merging, and the gains persist when the backbone scales to a larger LLM."],"fun_headline_variants":["FedMosaic: 10.9% better federated RAG, 91% less comms","Sparse masks slash federated RAG storage by 86%","Parametric adapters boost federated RAG accuracy 10.9%","Cluster adapters make private RAG more accurate","Selective LoRA merging lifts federated RAG 10.9%"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The whole scheme rests on the untested hypothesis that each document's knowledge lives in a distinct, separable subset of the shared adapter's rows, so a binary row mask can isolate documents and mask overlap can stand in for harmful parameter conflict.","fun_headline_variants_meta":{"raw":{"variants":["FedMosaic: 10.9% better federated RAG, 91% less comms","Sparse masks slash federated RAG storage by 86%","Parametric adapters boost federated RAG accuracy 10.9%","Cluster adapters make private RAG more accurate","Selective LoRA merging lifts federated RAG 10.9%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000183,"raw_usage":{"total_tokens":1170,"prompt_tokens":785,"completion_tokens":385,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":529,"completion_tokens_details":{"reasoning_tokens":286}},"tokens_in":529,"tokens_out":385,"duration_ms":4810,"temperature":1.0,"reasoning_tokens":286,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T04:17:38.602829+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train one cluster adapter on two semantically unrelated documents with mask training, then measure per-document answer quality after deleting the other document's masked rows. If isolating a document's rows does not preserve its answer quality materially better than random row subsets, the central hypothesis fails. A second check: test whether mask overlap actually correlates with measured negative transfer when the corresponding adapters are merged.","supporting_citations":[],"review_version":1}