{"id":"b7478fb4-ea24-4369-95cb-59e8cfa6507c","arxiv_id":"2501.03223","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"An adaptive LoRA-aggregation method for federated cardiac MRI segmentation improves Dice scores and cuts bandwidth compared to full-weight FedAvg and FedPETuning.","lead":"The paper proposes Rate-My-LoRA, a federated learning method that fine-tunes only low-rank adapters and aggregates them by penalizing adapters when client validation accuracy drops. It reports improved cardiac MRI segmentation accuracy on public multi-center data with up to 94% bandwidth savings compared with full-weight fine-tuning.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported gains could be driven by an unequal 'one extra epoch' of local fine-tuning rather than the Rate-My-LoRA aggregation rule; Section 3 does not explicitly state this was applied to all baselines, and no ablation isolates it.","rationale":"The reader's weakest_assumption identifies exactly the same load-bearing concern: the 'one extra epoch' may not have been applied identically across methods. My reading of Section 3 confirms the ambiguity: the sentence 'After aggregation, models are fine-tuned on local data for one extra epoch' is placed in the general experimental settings but is not qualified as applying to all compared FL methods. Since the paper's contribution is the adaptive aggregation rule, the comparison must isolate the rule from the extra epoch. The practical impact is high: an extra epoch of local adaptation can improve Dice on cardiac MRI segmentation, especially when the aggregated model has been produced under non-IID conditions. The proposed concrete test would settle the concern by ablating the extra epoch on both sides. No other concern is as directly tied to the central claim: the method is a plausible heuristic, low-rank regularization is sensible, and the qualitative visualizations are consistent, but the confound affects the quantitative evidence that supports the headline contribution. I agree with the reader's conditional verdict; the paper should not be fully accepted until this ablation is provided or the experimental protocol is clarified.","tokens_in":8378,"tokens_out":8064,"duration_ms":75045,"concrete_test":"Re-run the full experimental comparison with a 2x2 ablation: Rate-My-LoRA with and without the 'one extra epoch' local fine-tuning, and FedPETuning/Average Weighting with and without the same extra epoch, keeping all other settings and seeds fixed. If the Dice gaps in Tables 1 and 2 shrink to within noise or reverse when the extra epoch is applied uniformly, the reported advantage is not attributable to the Rate-My-LoRA aggregation rule. Additionally, report the mean and standard deviation over at least three seeds for each condition to confirm the gaps are not within run-to-run variance.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that Rate-My-LoRA's adaptive aggregation rule improves in-client and cross-client accuracy over LoRA-based FL baselines. Section 3 (Experimental Settings) states: 'After aggregation, models are fine-tuned on local data for one extra epoch.' This sentence appears immediately after the description of Rate-My-LoRA's hyperparameters (LoRA adapter sizes and lambda) and does not explicitly say that FedPETuning, Average Weighting, or Local Only received the same extra epoch. Because one extra epoch of local fine-tuning is a known performance booster, especially under domain shift, the reported Dice gains in Tables 1 and 2 (e.g., 0.910 vs. 0.883 for FedPETuning on client 2) could be explained entirely by this extra training step rather than by Eq. (1)'s adaptive penalization. The paper provides no ablation that removes the extra epoch from Rate-My-LoRA or adds it to the baselines. Without such an ablation, the experimental comparison does not isolate the contribution of the proposed aggregation rule, so the central claim is not yet supported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Rate-My-LoRA, a federated learning method for cardiac MRI segmentation that combines low-rank adaptation (LoRA) with an adaptive aggregation rule. After local training of LoRA adapters, each client evaluates the preliminarily merged model on its local validation set and reports the accuracy; the server then applies a penalty weight (1−λ) to adapters according to Eq. (1) to mitigate data heterogeneity. Experiments on the M&Ms-2 dataset with an ACDC-pretrained U-Net report improved in-client and cross-client Dice scores over Local Only, full-weight FedAvg, and FedPETuning, while reducing communication bandwidth by up to 94%.","tokens_in":8714,"tokens_out":8971,"duration_ms":78711,"significance":"If the empirical results are trustworthy, the paper makes a useful practical contribution: a simple, communication-efficient FL scheme for heterogeneous medical image segmentation, with a plausible mechanism (validation-guided penalization) that could translate to other domains. The bandwidth savings are clearly quantified, and the paper provides qualitative visualizations as supporting evidence. However, the significance is tempered by the fragility of the experimental comparison, which hinges on an ambiguous training protocol, and by missing implementation details that would be needed for reproduction.","major_comments":[{"comment":"The sentence 'After aggregation, models are fine-tuned on local data for one extra epoch' is ambiguous: it does not explicitly state whether this extra epoch was applied to all baselines (Local Only, Average Weighting, FedPETuning) or only to Rate-My-LoRA. Because one extra epoch of local fine-tuning is a known performance booster under domain shift, the reported gains in Tables 1 and 2 (e.g., client 2 Dice 0.910 vs 0.883 for FedPETuning) could be entirely due to this extra training step rather than to the aggregation rule in Eq. (1). Please clarify the protocol and add ablations: (a) Rate-My-LoRA without the extra epoch, and (b) each baseline with the extra epoch, so that the contribution of the adaptive weighting is isolated.","section":"Section 3, Experimental Settings"},{"comment":"The adaptive weighting rule is written as w_t(c)=1−λ only if there exists a client i with a validation accuracy drop while client c's accuracy improves. The accompanying text and Fig. 4 describe a different-sounding rule: when the aggregated model causes a performance drop on client 3, the adapters of the *other* clients (1 and 2) are penalized. The equation is consistent with the figure only if the non-dropping clients simultaneously improve, which is not stated or guaranteed. As written, if multiple clients drop and only one improves, only the improving client is penalized, which may frustrate the stated goal of penalizing adapters that reduce generalizability. Please clarify the intended rule and ensure Eq. (1), the text, and Fig. 4 are mutually consistent.","section":"Section 2, Eq. (1)"},{"comment":"The aggregation formula and algorithm are underspecified. Eq. (2) writes W_t = W_{t-1} + (1/Σ|D_c|) Σ w_t(c)|D_c| B_c^t A_c^t, but it is not clear whether B_c^t A_c^t denotes the full adapter weights after local training or a per-round delta, nor how W_t relates to the frozen pretrained model W0 and the adapters from previous rounds. Algorithm 1's TrainLocal call similarly mixes the global model and all adapters without specifying which parameters are updated. Without a precise definition of these quantities, the method is not reproducible.","section":"Section 2, Eq. (2) and Algorithm 1"},{"comment":"The paper does not report standard deviations or significance tests for the Dice/VOE/HD/ASSD results. The improvements in Tables 1 and 2 are presented as point estimates from (apparently) a single run, so it is unclear whether the observed gaps, some of which are modest, are within run-to-run variability. Please report results over multiple random seeds or provide confidence intervals, especially for the central comparison against FedPETuning.","section":"Section 3, Experimental Settings"}],"minor_comments":[{"comment":"LoRA is normally defined for linear layers, but the paper says 'LoRA adapters injected into each convolution block' without describing the reformulation for convolutions. Please provide a brief explanation or cite the implementation approach used.","section":"Section 3, Experimental Settings"},{"comment":"The Hausdorff distance values for 'Local Only' (e.g., 89.897, 93.124) are an order of magnitude larger than those of other methods; please verify the units or check for a typo, as these values seem implausible for cardiac segmentation.","section":"Table 1"},{"comment":"The condition in Eq. (1) uses strict inequality (P_c^t > P_c^{t-1}); if a client's accuracy stays flat while another drops, that client's adapter is not penalized. Consider whether ties should also trigger the penalty for consistency with the described behavior.","section":"Section 2, Eq. (1)"},{"comment":"Several standard training hyperparameters are missing, including the number of communication rounds T, batch size, learning rate, optimizer, and total training epochs. Please report these to enable reproducibility.","section":"Experimental Settings"},{"comment":"The abstract states 'up to 94% communication bandwidth' while Section 3 says 'saves up to 15.5x'; please make the relationship explicit (15.5x corresponds to ~93.5% reduction) to avoid confusion.","section":"Abstract and Conclusion"}],"recommendation":"major_revision","confidential_remarks":"The paper's central claim is plausible but the experimental support is fragile due to the unablated 'extra epoch' and the ambiguous Eq. (1). The authors should be asked to provide a clear protocol statement and ablations. The manuscript appears to be a short conference-style paper; if the journal expects more thorough experimental validation, the bar for acceptance should be adjusted accordingly."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth a look if you work on federated medical imaging or parameter-efficient FL. The core idea is simple: instead of averaging LoRA adapters by dataset size, each client evaluates the merged model on its own validation set, and when one client's validation score drops, the server shrinks the contribution of the other clients' adapters by a factor (1-lambda). That is a reasonable heuristic, and the bandwidth savings from transmitting only adapters are real. The paper does a sensible thing by evaluating both in-client and cross-client accuracy on ACDC and M&Ms-2, with visual results that help sanity-check the numbers.\n\nThe novelty is modest, but honest. Performance-weighted aggregation already exists in Stripelis et al. and Xu et al.; the step here is applying it to LoRA adapters with a specific penalty rule. That is a legitimate extension, and the paper says so explicitly. The LoRA-based framing also gives a clean bandwidth story.\n\nThe biggest soft spot is the \"one extra epoch\" sentence in Section 3. It says: \"After aggregation, models are fine-tuned on local data for one extra epoch.\" It appears right after the description of the proposed method's hyperparameters and does not explicitly say FedPETuning, Average Weighting, or Local Only received the same extra local fine-tuning. If only Rate-My-LoRA got that, the reported Dice gains (e.g., 0.910 vs. 0.883 for FedPETuning on client 2) could come from that step rather than from the adaptive weighting. This is a fixable ambiguity, but it needs a clear statement or an ablation. The paper also has no error bars or significance tests; with three clients and single runs, the gaps of a few Dice points could be within run-to-run noise. And the LoRA-on-conv implementation is underspecified: it should state exactly how the low-rank matrices are applied to convolutional kernels, since that is not standard from the LLM literature. The code link is mentioned in the abstract but no actual link appears in the text.\n\nNone of these are fatal. The central heuristic is plausible, the evaluation protocol is otherwise sane, and the citation pattern is fair. The results are suggestive rather than conclusive. I would send this to peer review, with the expectation that a good referee will ask for the extra-epoch clarification, multiple seeds, and a couple of ablations. It is a useful incremental contribution for the federated medical imaging community, not a breakthrough, but it deserves proper scrutiny.","headline":"A plausible and clearly written LoRA federated aggregation heuristic, but a single ambiguous sentence about an extra epoch of local fine-tuning could confound the central comparison.","tokens_in":9129,"tokens_out":1968,"would_cite":false,"duration_ms":18225,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Rate-My-LoRA lifts cardiac MRI Dice, cuts bandwidth 94%","keywords":["federated learning","low-rank adaptation","LoRA","cardiac MRI segmentation","data heterogeneity","adaptive aggregation","parameter-efficient fine-tuning","communication efficiency"],"falsifier":"A head-to-head comparison that applies an identical one-epoch local fine-tuning step to FedPETuning and full-weight FedAvg, or removes it from Rate-My-LoRA, would directly test whether the adaptive weighting is responsible for the Dice improvement. If the gap disappears under either condition, the central claim is not supported.","tokens_in":8228,"feed_emoji":"🫀","tokens_out":6740,"duration_ms":56979,"temperature":0.7,"pith_summary":"The paper claims that its adaptive aggregation rule, Rate-My-LoRA, improves both in-client and cross-client segmentation accuracy in LoRA-based federated learning for cardiac MRI, while slashing communication cost. The method trains a low-rank adapter on each hospital's data, merges them with weights that are adjusted whenever a client's validation accuracy drops, and reports Dice scores of 0.910 (in-client) and 0.904 (cross-client), outperforming FedPETuning and full-weight FedAvg. If correct, this would let hospitals with limited bandwidth pool their data for segmentation without sharing images, counteracting the performance loss caused by non-IID imaging data.","feed_headline":"Rate-My-LoRA lifts cardiac MRI Dice, cuts bandwidth 94%","feed_subtitle":"In-client and cross-client Dice reach 0.910 and 0.904, beating FedPETuning on heterogeneous hospital data.","key_machinery":"The load-bearing object is the adaptive weight rule $w_t(c) = 1-\\lambda$ exactly when at least one client $i$ satisfies $P^i_t < P^i_{t-1}$ and client $c$ satisfies $P^c_t > P^c_{t-1}$; otherwise $w_t(c)=1$. These weights enter the aggregation $W_t = W_{t-1} + \\frac{1}{\\sum_c |D_c|} \\sum_c w_t(c) |D_c| B^c_t A^c_t$, where $B^c_t A^c_t$ is the LoRA adapter from client $c$. The diminishing $\\lambda$ schedule makes the method converge to FedAvg as $\\lambda \\to 0$. The LoRA decomposition $W = W_0 + BA$ with small rank $r$ is what keeps communication to a few megabytes per round.","core_discovery":"Rate-My-LoRA's central claim is that evaluating the merged model on each client's local validation set and using the resulting accuracy changes to penalize non-generalizing adapters yields a better aggregated model than dataset-size-based weighting. After each round, the server sends the equal-weight merged model to all clients; each client reports its validation accuracy. If any client's accuracy falls relative to the previous round, the server sets the aggregation weight of every other client's adapter to $1-\\lambda$, with $\\lambda$ decaying by 5% per round, and uses dataset-size-weighted aggregation with these adaptive weights. This is shown to raise in-client Dice from 0.840/0.883/0.873 (FedPETuning) to 0.889/0.910/0.895 across the three clients, and cross-client Dice from 0.849/0.865/0.862 to 0.882/0.894/0.904, while transmitting only the compact LoRA adapters.","pith_inferences":["A natural next step is to test whether the validation-drop trigger can be replaced by a more robust signal, such as the magnitude of the drop or a confidence interval, to avoid penalizing adapters when validation noise causes false drops.","The 5% decay of $\\lambda$ is a heuristic; one could adapt the decay schedule based on the variance of validation scores across rounds, which is not explored in the paper.","Because the method explicitly compares validation accuracy across rounds, it assumes clients have enough local validation data to produce stable measurements; in few-shot settings with tiny validation sets, the penalty trigger could become overly sensitive."],"forward_implications":["Hospitals with different MRI vendors, such as GE, Philips, and Siemens, can participate in a shared cardiac segmentation model without transferring patient images, with per-round communication of 1.8–3.6 MB rather than 28 MB.","The adaptive weighting rule can be applied to any parameter-efficient tuning method that exchanges compact adapters, not only LoRA.","The reported improvement in cross-client accuracy implies that knowledge from one hospital's data can transfer to other hospitals' imaging conditions, which is the primary goal of federated learning in medical imaging.","The method's reliance on local validation scores ties aggregation quality to the availability of a reliable validation set at each client."],"supporting_citations":[{"why":"Introduces the LoRA low-rank decomposition that the method fine-tunes and communicates.","marker":"[17]"},{"why":"The main comparison method, FedPETuning, which combines LoRA with FedAvg and dataset-size weighting.","marker":"[22]"},{"why":"The original FedAvg algorithm that serves as the baseline and the fallback when lambda decays to zero.","marker":"[9]"},{"why":"ACDC dataset used to train the base model.","marker":"[28]"},{"why":"M&Ms-2 dataset that provides the three hospital clients with different MRI vendors.","marker":"[29]"},{"why":"U-Net architecture used as the segmentation backbone.","marker":"[31]"}],"fun_headline_variants":["Adaptive LoRA weighting lifts federated cardiac MRI Dice","Rate-My-LoRA: accuracy-aware LoRA aggregation for heart MRI","Federated cardiac MRI: LoRA with client scoring boosts Dice","Smarter LoRA merging slashes bandwidth, lifts cardiac MRI Dice","Rate-My-LoRA: adaptive LoRA tuning for federated cardiac MRI"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported advantage rests on the assumption that the extra local fine-tuning epoch after aggregation, described in the experiments, was applied identically to every compared method; if only Rate-My-LoRA received that extra training, the observed gains could come from that step rather than from the adaptive weighting rule.","fun_headline_variants_meta":{"raw":{"variants":["Adaptive LoRA weighting lifts federated cardiac MRI Dice","Rate-My-LoRA: accuracy-aware LoRA aggregation for heart MRI","Federated cardiac MRI: LoRA with client scoring boosts Dice","Smarter LoRA merging slashes bandwidth, lifts cardiac MRI Dice","Rate-My-LoRA: adaptive LoRA tuning for federated cardiac MRI"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000913,"raw_usage":{"total_tokens":3933,"prompt_tokens":968,"completion_tokens":2965,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":584,"completion_tokens_details":{"reasoning_tokens":2871}},"tokens_in":584,"tokens_out":2965,"duration_ms":21246,"temperature":1.0,"reasoning_tokens":2871,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:52:18.019538+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A head-to-head comparison that applies an identical one-epoch local fine-tuning step to FedPETuning and full-weight FedAvg, or removes it from Rate-My-LoRA, would directly test whether the adaptive weighting is responsible for the Dice improvement. If the gap disappears under either condition, the central claim is not supported.","supporting_citations":[{"cited_title":"FedED: Federated learning via ensemble distillation for medical relation extraction,","cited_arxiv_id":null,"evidence_quote":"The main comparison method, FedPETuning, which combines LoRA with FedAvg and dataset-size weighting."},{"cited_title":"The prog- nostic impact of dynamic ventricular dyssynchrony in patients with idiopathic dilated cardiomyopathy and nar- row QRS,","cited_arxiv_id":null,"evidence_quote":"The original FedAvg algorithm that serves as the baseline and the fallback when lambda decays to zero."},{"cited_title":"Prox- edit: Improving tuning-free real image editing with proximal guidance,","cited_arxiv_id":null,"evidence_quote":"U-Net architecture used as the segmentation backbone."}],"review_version":1}