{"id":"a2a19778-1750-4d6b-9d35-dc571f3df508","arxiv_id":"2502.00859","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"FedRIR improves personalized federated learning by separating client-specific and global features with masked reconstruction and mutual information minimization, achieving up to 3.93% higher accuracy than prior methods.","lead":"This paper proposes FedRIR, a federated learning method in which each client trains a private feature extractor for its own data patterns and a shared feature extractor for common patterns, then combines both for predictions. On standard image benchmarks it reports accuracy gains of up to about 4% over prior federated methods, with communication costs similar to FedAvg.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The vCLUB bound in the IDM is not guaranteed to hold because q_theta is trained with L_id itself, so the claimed mutual-information reduction is unverified.","rationale":"The reader's weakest_assumption identifies the vCLUB bound's reliability as the core risk. My analysis sharpens this into a specific flaw: q_theta is trained with the same L_id objective, so the necessary condition for the upper bound (q_theta approximating the true conditional) is not enforced. This is a concrete, technical concern that directly affects the paper's central mechanism, not a matter of external consensus. If the bound is invalid or loose, the IDM may not reduce true MI, and the claimed information-distillation effect is not established. The empirical accuracy improvements could still be real, but the paper's stated explanation would be unsupported. The proposed test—measuring MI and comparing with a correctly fitted q—would settle this. The reader already gave a CONDITIONAL verdict; my concern reinforces that condition without moving it to REJECT, because the experimental results are plausible and the code is available for verification. Therefore the verdict remains UNCHANGED.","tokens_in":16903,"tokens_out":13783,"duration_ms":133454,"concrete_test":"On a fixed benchmark (e.g., CIFAR10 pathological, 20 clients), train FedRIR with the published code. Then: (1) extract f_cs and f_g on a held-out local test set and estimate I(f_cs;f_g) using a non-parametric estimator (e.g., KSG) over training checkpoints; (2) compute the vCLUB value L_id at the same checkpoints. If L_id is below the estimated MI (bound violation), or if the estimated MI does not decrease while L_id decreases, the IDM is not performing information distillation. Additionally, retrain a variant where q_theta is first optimized by maximizing E log q(f_g|f_cs) (conditional log-likelihood) before each F_g update, as in the original vCLUB procedure, and compare MI reduction and final accuracy. If this fitted-q variant does not outperform the published joint-training version, the claimed MI-minimization mechanism is not the source of the accuracy gain.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The Information Distillation Module rests on Eq. (7), where the vCLUB quantity is an upper bound on I(f_cs;f_g) only if q_theta(f_g|f_cs) faithfully approximates the true conditional p(f_g|f_cs). In Algorithm 1 (step 10), the module network I_i, which implements q_theta, is updated by minimizing L_id = I_vCLUB jointly with the global extractor F_g and the head H_i. This L_id contains a positive-pair log-likelihood term plus a negative marginal-pair term; minimizing it does not enforce q_theta ≈ p(f_g|f_cs). Consequently, the inequality I_vCLUB ≥ I(f_cs;f_g) is not guaranteed, and reducing L_id may not reduce the true redundancy between global and client-specific features. The paper never measures I(f_cs;f_g) after training nor validates the bound's tightness, so the central claim that IDM 'purifies' the global representation is both theoretically and empirically unsupported. The ablation gain in Table 5 could instead arise from the extra network and loss acting as a regularizer rather than from genuine information distillation.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FedRIR, a personalized federated learning method that decomposes learned representations into client-specific features, obtained by a masked denoising autoencoder (MCSL), and global features, purified by an Information Distillation Module (IDM) that penalizes a vCLUB estimate of mutual information between the two feature sets. The features are concatenated and classified by a local head; only the global extractor is communicated and aggregated with FedAvg-style weighting. The authors report accuracy improvements over eleven baselines on pathological, practical, and real-world heterogeneity benchmarks, plus scalability, stability under client dropout, and communication cost comparisons.","tokens_in":17138,"tokens_out":5684,"duration_ms":56060,"significance":"If the empirical results hold, FedRIR offers a simple and communication-efficient way to improve both personalized and global representations simultaneously, which would be a useful result for personalized federated learning. The paper's strengths include extensive benchmarking with 11 methods, ablations of both modules, a mask-ratio ablation, scalability up to 100 clients, stability under random participation, and released code; the paper reports means and standard deviations over three runs. The main weaknesses are theoretical and methodological: the vCLUB bound used in IDM is not guaranteed to hold under the training procedure, the global generalization claim is not supported by a direct global-model evaluation, and the mask ratio and loss weights are selected on the same evaluation benchmarks, which makes the magnitude of the claimed gains partly a product of test-set tuning.","major_comments":[{"comment":"The vCLUB inequality I_vCLUB >= I(f_cs; f_g) holds only when q_theta is a valid conditional distribution that equals or closely approximates p(f_g|f_cs). Training q_theta by minimizing L_id jointly with F_g and H_i does not enforce this condition, so the paper's claim that minimizing L_id reduces the true mutual information is not established. The authors should report estimates of I(f_cs; f_g) before and after training and check the bound's tightness, or explicitly reframe IDM as a heuristic regularizer. As written, the 'purer global representation' story is unsupported, and the IDM ablation gain in Table 5 could be a regularization effect.","section":"Section 3.3, Eq. (7), Algorithm 1 step 10"},{"comment":"Eq. (10) defines the overall loss at client i as L_id + L_cls, but Algorithm 1 step 7 explicitly optimizes F_cs and G_i by minimizing L_recon. This makes the reported total loss incomplete and inconsistent with the algorithm. Please correct Eq. (10) or clarify that L_recon is optimized in a separate MCSL stage and is not part of the combined loss in Eq. (10).","section":"Eq. (10) versus Algorithm 1"},{"comment":"The paper claims FedRIR simultaneously enhances global generalization and local personalization, but Table 1 reports only the accuracy of the personalized model. Comparing this number against FedAvg's global-model accuracy is apples-to-oranges, since FedAvg does not use personalized heads. The t-SNE visualizations in Figure 3 are qualitative and do not quantify global feature quality. To support the global-generalization claim, the authors should report a direct evaluation of the aggregated global extractor (e.g., the global model's accuracy on a server-side or fixed-head test set) for all methods.","section":"Section 4.4.1 and Table 1"},{"comment":"The mask ratio r=0.6 is selected via a hyperparameter sweep on the same datasets (Cifar10, Cifar100, OfficeCaltech10, etc.) that are later used for the main comparisons in Table 1, and the loss balance weights in Eq. (10) are set to 1 with no sensitivity analysis. This means the reported gains include the effect of fitting two hyperparameters to the evaluation benchmarks. A nested validation split or a sensitivity analysis with training-only selection would be needed to rule out circularity; the statement that r=0.6 is universally optimal is stronger than the evidence supports.","section":"Section 4.7.1, Figure 4, Eq. (10)"}],"minor_comments":[{"comment":"The phrase 'without transfer private data' should read 'without transferring private data'.","section":"Abstract"},{"comment":"The heading 'Global Generization Effectiveness' contains a typo; it should be 'Global Generalization Effectiveness'.","section":"Section 4.4.1 heading"},{"comment":"The term 'Variational Constrastive Log-ratio Upper Bound' contains a typo; it should be 'Variational Contrastive Log-ratio Upper Bound'.","section":"Section 3.3, Eq. (7)"},{"comment":"Step 8 freezes F_cs, but the status of G_i during the information distillation stage is not stated; please clarify whether G_i is also frozen or simply unused.","section":"Algorithm 1"},{"comment":"The caption and axis labels of Figure 4 appear as corrupted Unicode glyph sequences in the manuscript, making the mask-ratio hyperparameter analysis difficult to read; this should be fixed in the camera-ready version.","section":"Figure 4"},{"comment":"The 'Δ SOTA' row is not defined; please clarify that it denotes the improvement of FedRIR over the best baseline in each column.","section":"Table 1, 'Δ SOTA' row"},{"comment":"The terms 'Information Distillation Model' and 'Information Distillation Module' are used interchangeably; please standardize the terminology.","section":"Section 3.1.1"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is within the scope of the conference and the empirical study is extensive. The main technical gap is the unverified vCLUB bound; if the authors cannot demonstrate its validity, they should soften the information-theoretic framing. The hyperparameter selection issue, while common in this literature, should be addressed explicitly because the paper's central claim of simultaneous improvement depends on the chosen configuration. The code release and reproducibility details are commendable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Three things to know. First, the architecture is genuinely new for pFL: a masked client-specific autoencoder (MCSL) plus a vCLUB mutual-information penalty (IDM) to separate global and personalized features. Each ingredient is borrowed, but the combination is not in the cited literature. Second, the experiments are unusually thorough for a conference paper: eleven baselines, pathological/practical/real-world settings, scalability to 100 clients, client dropout, ablations, and t-SNE. The gains are consistent and the communication cost stays essentially equal to FedAvg. Third, the theory is the weakest part: the vCLUB upper bound in Eq. (7) assumes q_theta approximates the true conditional p(f_g|f_cs), but the training objective in Algorithm 1 just minimizes L_id = I_vCLUB jointly with the feature extractor. That does not enforce the approximation, so the bound can be invalid and the claim that IDM provably purifies global features is unsupported. The stress-test note has this right. But this is not fatal: the empirical results stand on their own, and the IDM ablation shows a consistent gain except on OfficeCaltech10, where it slightly hurts. The extra network and loss could be acting as a regularizer; the paper never rules that out. Other soft spots in proportion. Eq. (10) omits the reconstruction loss L_recon that Algorithm 1 optimizes; that is a one-line fix. The mask ratio r=0.6 is selected after seeing test accuracy on the benchmark datasets, which is mild circularity. Several standard deviations are exactly zero across three seeds (e.g., FedRIR 97.51 plus or minus 0.00, FedBABU 98.92 plus or minus 0.00, FedALA 95.56 plus or minus 0.00), which is suspicious and deserves a footnote. The PDF contains a corrupted section with /uni000000 tokens in the hyperparameter and t-SNE figures; likely a rendering bug, but it makes verification harder. The citation pattern is honest, and the code is linked. This is a practical pFL method that people in the field would want to know about. It deserves a serious referee, but a referee should push on the MI claim and ask for a comparison where q_theta is trained first to maximize conditional likelihood and then frozen before applying the vCLUB penalty. I would send it to review with a request for revision, not reject it. If the authors reposition IDM as an empirical regularizer instead of a guaranteed information bottleneck, the paper is essentially sound.","headline":"FedRIR is a solid, incremental pFL paper with extensive experiments and a plausible but not rigorously grounded information-theoretic story; the empirical method works, but the vCLUB justification needs rework.","tokens_in":691,"tokens_out":903,"would_cite":true,"duration_ms":49653,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FedRIR claims that masked client-specific learning plus mutual-information distillation lets a federated model improve global generalization and local personalization at the same time.","keywords":["Federated Learning","Personalized Federated Learning","Information Representation","Information Distillation","Masked Representation Learning","Mutual Information","Statistical Heterogeneity","vCLUB"],"falsifier":"Train FedRIR on a synthetic dataset where the client-specific features are the only signal for a second classification task while the global features carry the original labels; if the IDM penalty reduces true redundancy, the global-only classifier's accuracy should be unaffected, but if the penalty discards shared discriminative information, the global classifier's accuracy should drop. A more direct check is to estimate the true mutual information $I(f_{cs}; f_g)$ by histogram on a held-out set and compare FedRIR with and without IDM; if the true MI does not decrease even as the vCLUB penalty is minimized, the mechanism claimed for IDM is not what drives the improvement.","tokens_in":16667,"feed_emoji":"📱","tokens_out":7703,"duration_ms":68277,"temperature":0.7,"pith_summary":"Federated learning under heterogeneous client data has traditionally forced a choice between building one well-generalized global model and giving each client a personalized model. FedRIR argues this trade-off is not necessary: a client can maintain both a client-specific feature extractor, trained with masked reconstruction, and a global feature extractor, purified by minimizing its mutual information with the client-specific features. The paper reports that combining the two feature sets for classification improves accuracy by up to 3.93 percentage points over prior methods across pathological, practical, and real-world heterogeneity, while keeping communication cost comparable to FedAvg. If correct, the method offers a concrete recipe for simultaneously improving personalization and generalization without extra communication overhead.","feed_headline":"One trick boosts both global and local accuracy in federated learning","feed_subtitle":"Masking plus mutual-information distillation purifies global features, improving accuracy by up to 3.93%.","key_machinery":"The load-bearing mechanism is the two-stage feature separation: in the MCSL stage, a masked autoencoder (a VAE turned into a denoising autoencoder by random masking) extracts client-specific features $f_{cs}^i$ via reconstruction loss $\\|G^i(F^i_{cs}(x^i_{masked})) - x^i\\|^2$. In the ID stage, an Information Distillation Module implements the variational CLUB upper bound, a contrastive log-ratio upper bound on mutual information estimated by a jointly trained neural network $q_\\theta(f_g|f_{cs}^i)$, and minimizes it, so the global features $f_g$ are cleaned of information already carried by the client-specific branch. The classification head consumes the concatenated features. This machinery is what, in the paper's argument, lets the global aggregate become more general and the local representation stay personalized.","core_discovery":"The paper's central claim is that separating representation into a client-specific branch and a global branch, then actively de-correlating the two, lets personalized federated learning improve both objectives at once. Masked Client-Specific Learning (MCSL) treats each client's data as the input to a denoising autoencoder: random masking forces the client-specific extractor $F^i_{cs}$ to encode the client's idiosyncratic structure rather than copying the input. The Information Distillation Module (IDM) then minimizes the vCLUB upper bound of the mutual information $I(f_{cs}^i; f_g)$ between the frozen client-specific features and the global features, pushing the global extractor $F_g$ to drop client-specific redundancy. The personalized representation is the concatenation $[f_{cs}^i, f_g]$, classified by a client-specific head. The paper argues this yields a purer global model whose server-side aggregation generalizes better, while the client-specific branch retains the local nuances needed for personalization.","pith_inferences":["The paper does not analyze where the information bound is loose; a natural extension is to measure the true $I(f_{cs};f_g)$ and check whether vCLUB minimization actually reduces it, or whether the gain comes from a regularizing side effect.","The masking-ratio tuning at $r=0.6$ is dataset-dependent; an adaptive masking schedule or masking in feature space rather than input space could make the method more robust across clients with different data volumes, a testable variant the paper does not explore.","The same feature-separation architecture might transfer to non-federated multi-domain settings, e.g., domain generalization, where a shared encoder and per-domain private encoders are trained with the same de-correlation penalty."],"forward_implications":["If FedRIR is correct, a single personalized federated learning algorithm can beat the state of the art on both global generalization and local personalization, so future pFL designs need not treat the two as a zero-sum trade-off.","The recipe of mask-then-purify is transferable: any split of representation into private and shared components could benefit from an explicit mutual-information penalty between the branches.","Because FedRIR transmits only the global feature extractor, like FedAvg, its accuracy gains come without extra communication overhead, making the approach practical on bandwidth-limited edge devices.","The performance gap over baselines widens as the number of clients grows (from +4.01 points at 10 clients to +5.73 at 100), suggesting the benefit increases with federated scale and fragmentation.","In real-world domain-shift settings (OfficeCaltech10, DomainNet), FedRIR improves accuracy by 3.57 and 2.90 points over best baselines, indicating usefulness when client data come from different acquisition conditions."],"supporting_citations":[{"why":"Supplies the variational lower bound used to justify maximizing mutual information between input and client-specific features in MCSL.","marker":"[2]"},{"why":"Supplies the contrastive log-ratio upper bound (vCLUB) that the Information Distillation Module minimizes to purify global features.","marker":"[6]"},{"why":"The masked autoencoder; motivates random masking to prevent trivial reconstruction in client-specific feature learning.","marker":"[19]"},{"why":"The denoising autoencoder formulation; explains how masking turns the VAE reconstruction into a meaningful representation learner.","marker":"[37]"},{"why":"The variational autoencoder framework used for the reconstruction-based client-specific feature extractor.","marker":"[22]"},{"why":"The FedAvg aggregation rule used to average global feature extractors on the server.","marker":"[30]"},{"why":"Baseline that also separates global and personalized features, and the source of the pathological heterogeneity setup.","marker":"[45]"},{"why":"Baseline for split global/personalized features; a comparison point on global feature separation.","marker":"[5]"}],"fun_headline_variants":["Splitting features boosts both personalization and generalization in FL","FedRIR: Masking and mutual-information distillation improve FL on both axes","De-correlating local and global features lifts both accuracy goals in FL","Isolate client-specific info and purify global features for better FL"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole method relies on the assumption that minimizing the vCLUB upper bound of the mutual information between client-specific and global features really removes redundant client-specific information from the global features, and that this does not also discard shared class-discriminative information.","fun_headline_variants_meta":{"raw":{"variants":["Splitting features boosts both personalization and generalization in FL","FedRIR: Masking and mutual-information distillation improve FL on both axes","De-correlating local and global features lifts both accuracy goals in FL","Isolate client-specific info and purify global features for better FL"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00071,"raw_usage":{"total_tokens":3234,"prompt_tokens":1019,"completion_tokens":2215,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":635,"completion_tokens_details":{"reasoning_tokens":2138}},"tokens_in":635,"tokens_out":2215,"duration_ms":16437,"temperature":1.0,"reasoning_tokens":2138,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T17:27:06.693153+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train FedRIR on a synthetic dataset where the client-specific features are the only signal for a second classification task while the global features carry the original labels; if the IDM penalty reduces true redundancy, the global-only classifier's accuracy should be unaffected, but if the penalty discards shared discriminative information, the global classifier's accuracy should drop. A more direct check is to estimate the true mutual information $I(f_{cs}; f_g)$ by histogram on a held-out set and compare FedRIR with and without IDM; if the true MI does not decrease even as the vCLUB penalty is minimized, the mechanism claimed for IDM is not what drives the improvement.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the variational lower bound used to justify maximizing mutual information between input and client-specific features in MCSL."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the contrastive log-ratio upper bound (vCLUB) that the Information Distillation Module minimizes to purify global features."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The denoising autoencoder formulation; explains how masking turns the VAE reconstruction into a meaningful representation learner."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Baseline that also separates global and personalized features, and the source of the pathological heterogeneity setup."}],"review_version":1}