{"id":"79b3f433-d4df-449c-947c-8d76c4a240fe","arxiv_id":"2412.03851","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"FedMetaMed is claimed to reach 87% average F1 and 90% AUC across four dermoscopy clients, outperforming FedBN, FedAvg, and MetaFed in the reported tables.","lead":"FedMetaMed combines federated learning with meta-learning for personalized medical image classification, using Fourier-domain model aggregation on the server and knowledge distillation between paired models on each client. The paper reports large accuracy gains over existing federated learning methods on dermoscopic image datasets, but the results contain internal inconsistencies that make the headline claims difficult to verify.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table III aggregate metrics cannot be recomputed from the per-client values, so the claimed SOTA superiority is not supported by the manuscript's own data.","rationale":"The reader's verdict is REJECT with high correctness risk, and the rationale already lists several numerical inconsistencies, including MetaFed and FedBN average mismatches, the 9.79% F1 gain discrepancy, and the ablation percentage mismatches. My stress-test confirms these are the most load-bearing issues: the paper's central claim is empirical, and the primary evidence table cannot be recomputed from its own per-client numbers. The reader's explicitly named 'weakest assumption' was the spectral-semantics assumption underlying CFA, which is a different and arguably less decisive concern; even if that assumption were granted, the reported results would still be internally inconsistent. Therefore I partially agree with the reader: the same verdict is warranted, but the decisive flaw is the unreproducible arithmetic in the results tables rather than the unproven frequency-domain motivation. The proposed concrete test is a simple recomputation and cross-reconciliation that would settle whether the numerical support for the headline claim exists. Without corrected tables, raw per-client metrics, or code, the manuscript cannot be accepted as evidence of FedMetaMed's superiority.","tokens_in":10965,"tokens_out":3010,"duration_ms":79093,"concrete_test":"Recompute every average in Table III from the per-client columns and compare each against the printed Avg column. Then reconcile the two OOD F1 values: Table II gives FedMetaMed F1=75.12, while Section IV-D-b and Table IV report F1=89.50. If the recomputed averages differ from the printed averages, or if the two OOD F1 values cannot be reconciled, the claimed state-of-the-art advantage is unsupported. Ideally, obtain per-client raw output files or the training/aggregation code to reproduce the table entries.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central empirical claim is that FedMetaMed outperforms state-of-the-art FL methods on HAMK/MSK. The evidence for this claim is Table III, but its aggregate row entries are internally inconsistent. For example, MetaFed F1 per-client values 59, 70, 68, 82 average to 69.75, not the reported 76; FedBN F1 values 54.6, 72.1, 54.3, 62.7 average to 60.9, not 65.7; and FedMetaMed F1 values 82.0, 89.7, 83.7, 86.6 average to 85.5, not 87. AUC averages are similarly off: FedBN's 83.0, 96.3, 79, 81 average to 84.8, not 88, and MetaFed's 84, 82, 85, 79 average to 82.5, not 80. The text's claimed F1 gain over FedBN (9.79%) also does not match Table III, where 87 vs 65.7 is a 21.3-point difference. The OOD results are self-contradictory as well: Table II lists FedMetaMed F1=75.12, while Section IV-D-b reports F1=89.50 and Table IV also shows 89.5. These inconsistencies mean a reader cannot determine the true reported performance, so the headline superiority claim is not verifiable from the manuscript. This is an internal arithmetic failure at the core of the evidence, not a disagreement with external consensus.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FedMetaMed, a personalized federated meta-learning framework for distributed healthcare data. On the server side, Cumulative Fourier Aggregation (CFA) averages only the low-frequency components of client model parameters while retaining high-frequency components, with the low-frequency threshold increasing over training. On the client side, Collaborative Transfer Optimization (CTO) maintains a deputy model that receives the aggregated server model and transfers knowledge to a personalized local model through Retrieve, Reciprocate, and Refine steps. The authors evaluate FedMetaMed against FedAvg, FedProx, FedBN, SiloBN, IDA, FML, and MetaFed on HAMK and MSK dermoscopic image datasets with four clients, reporting superior F1, AUC, accuracy, and out-of-distribution generalization.","tokens_in":11246,"tokens_out":4197,"duration_ms":42496,"significance":"If the reported gains were reproducible, FedMetaMed would be a useful contribution to personalized federated learning for medical imaging, and the idea of frequency-selective aggregation with a client-side deputy model is conceptually interesting. The paper also includes multiple baselines, ablations, and communication-round curves, which are appropriate experimental components. However, the central empirical claim is not currently verifiable because the summary statistics in Table III do not match the per-client values, the out-of-distribution results are self-contradictory, and no code, data splits, or seeds are provided. As a result, the evidence presented does not support the stated state-of-the-art superiority.","major_comments":[{"comment":"The aggregate F1 and AUC values in Table III cannot be recomputed from the per-client entries. For example, the FedMetaMed F1 values 82.0, 89.7, 83.7, 86.6 average to 85.5, not the reported 87; the MetaFed F1 values 59, 70, 68, 82 average to 69.75, not 76; and the FedBN F1 values 54.6, 72.1, 54.3, 62.7 average to 60.9, not 65.7. The AUC averages are similarly inconsistent: FedBN's 83.0, 96.3, 79, 81 average to 84.8, not 88, and MetaFed's 84, 82, 85, 79 average to 82.5, not 80. Because the paper's headline claim that FedMetaMed outperforms state-of-the-art FL methods rests on these aggregate numbers, the central empirical result is not supported by the manuscript's own data.","section":"Table III"},{"comment":"The out-of-distribution generalization results are mutually contradictory. Section IV-D-b states that FedMetaMed achieves an F1 score of 89.50% on the unseen cohort, and Table IV lists an F1 of 89.5, but Table II reports FedMetaMed's F1 on the same out-of-distribution setting as 75.12. The text also claims 'statistically significant improvements' without reporting any statistical tests. These discrepancies make it impossible to determine the true OOD performance and undermine the claim of superior generalization.","section":"Section IV-D-b, Table II, Table IV"},{"comment":"The claimed performance margin over FedBN is not consistent with the reported table. Section IV-D says FedMetaMed improves average F1 over FedBN by 9.79%, but Table III reports 87 vs. 65.7, a difference of 21.3 percentage points; even using the corrected FedBN average of 60.9, the difference is 26.1 points. The corresponding AUC improvement is also inconsistent with the reported numbers. This suggests either the comparative figures or the summary statistics are misreported, and the authors should provide corrected tables and text.","section":"Section IV-D and Table III"},{"comment":"The main hyperparameters of the proposed method—s0=0.26, s1=0.55, lambda1=0.6, and lambda2=0.8—are fixed on the same two benchmarks used for the final evaluation. No sensitivity analysis, separate validation set, or independent dataset is provided to show that the results are not a product of tuning these threshold parameters. Additionally, Section III-A's core premise that low-frequency components of network parameters determine network capability while high-frequency components carry client-specific information is stated without empirical evidence or theoretical justification. Given that CFA's design depends entirely on this assumption, the paper needs either a spectral analysis of client weights or an ablation across thresholds to support the claim.","section":"Section IV-B and Section III-A"}],"minor_comments":[{"comment":"The method is named 'Cumulative Fourier Agglomeration' in Section III-A and Figure 1 but 'Cumulative Fourier Aggregation' elsewhere; please standardize the terminology.","section":"Throughout"},{"comment":"The implementation section reports no random seeds, number of runs, or error bars, and the hardware description includes a typo ('NIVIDIA'). Reporting mean and variance over multiple runs would strengthen the empirical claims.","section":"Section IV-B"},{"comment":"The baseline name is written as 'MetFed' in Table II but 'MetaFed' elsewhere; please make the naming consistent.","section":"Table II"},{"comment":"The statement that this is 'the first work to apply frequency-domain parameter aggregation in Federated Learning' is not substantiated and is likely inaccurate given existing frequency-domain FL methods; please temper or cite prior work.","section":"Introduction"},{"comment":"The scaling experiment in Figure 8 reports only four methods and omits several baselines used in the main comparison, such as SiloBN and MetaFed; this limits the strength of the claim about performance with larger client counts.","section":"Section IV-D-c"}],"recommendation":"reject","confidential_remarks":"The numerical inconsistencies are too extensive to be dismissed as isolated typographical errors, and they affect the core empirical claim. Even if the authors corrected the tables, the absence of code, seeds, data splits, and independent hyperparameter validation would require substantial additional experiments before the claims could be assessed. The manuscript is not ready for publication in its current form."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Real idea, poorly supported evaluation. The new thing here is the package: server-side Cumulative Fourier Aggregation (CFA) that averages only the low-frequency part of client weights and leaves high-frequency components client-specific, combined with a client-side Retrieve-Reciprocate-Refine (CTO) distillation loop that uses a deputy model to transfer global knowledge without overwriting the local model. I have not seen that exact combination in the papers they cite, and the design is coherent on its face. The writing is readable, and the method section, aside from some notation slips, is understandable.\n\nWhere it falls apart is the evidence. The aggregate numbers in Table III cannot be recomputed from the per-client rows. MetaFed's F1 row is 59/70/68/82 but the average is listed as 76, not 69.75. FedBN's 54.6/72.1/54.3/62.7 averages to 60.9, not 65.7. FedMetaMed's own row averages to 85.5, not 87. AUC averages are off in the same way. Then the text says FedMetaMed improves over FedBN by 9.79% F1, but the table implies a gap closer to 21 points. The OOD section is self-contradictory: Table II gives FedMetaMed F1=75.12, while Section IV-D-b and Table IV say 89.5. These are not rounding differences; they are inconsistent accounts of the same experiment. There are no error bars, no significance tests, no code, and no description of the OOD cohort. The spectral-semantics assumption (low frequencies are shared, high are client-specific) is asserted without evidence, and the convergence/privacy claims are sketches rather than proofs. The \"first frequency-domain aggregation in FL\" claim is also too strong; a broader literature search would likely find prior work.\n\nNone of this makes the method impossible. The design is worth examining, and the problem is real. But the paper in its current form does not support its headline claim. If I were the editor, I would desk reject this version and invite a resubmission with corrected tables, a consistent OOD story, and released code. If it is already under review, the referee should require those fixes before anything else. It is not ready for referee time as-is.\n\nLet me know if you want to talk through the CFA/CTO design separately — that part may have legs.","headline":"The method is a plausible new combination, but the paper's own tables don't support the headline results.","tokens_in":11876,"tokens_out":4035,"would_cite":false,"duration_ms":40263,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FedMetaMed claims to outperform all compared federated learning baselines on skin-lesion classification by aggregating only low-frequency weight components.","keywords":["federated learning","meta-learning","personalized medicine","frequency-domain aggregation","Fourier transform","knowledge distillation","medical imaging","out-of-distribution generalization"],"falsifier":"Run the identical FedMetaMed procedure on the same datasets and splits, but reverse the frequency mask so that high-frequency components are averaged and low-frequency components are kept per client. If the F1 improvement over FedAvg persists, the claimed spectral-semantics mechanism is not what drives the result, and the gains would need a different explanation.","tokens_in":10610,"feed_emoji":"🩺","tokens_out":10182,"duration_ms":91608,"temperature":0.7,"pith_summary":"FedMetaMed tries to fix a specific failure mode in federated learning for personalized medicine: when hospitals train on non-identical data, standard parameter averaging makes the global model worse for each site. The paper proposes two coupled mechanisms—Cumulative Fourier Aggregation on the server, which averages only the low-frequency components of client model weights, and Collaborative Transfer Optimization on each client, which transfers global knowledge without overwriting the local personalized model. On two real-world skin-lesion image datasets, the authors report an average F1 of 87% and an average AUC of 90%, outperforming every compared baseline, and an out-of-distribution cohort with 89.51% accuracy and 75.12% F1, against 54.5% F1 for the closest baseline. If these results hold, the recipe offers a privacy-preserving way to build personalized medical models from distributed data without exchanging patient records.","feed_headline":"Frequency-based aggregation tops seven medical federated baselines","feed_subtitle":"Averaging only low-frequency model weights and transferring knowledge locally lifts out-of-distribution F1 to 75%.","key_machinery":"Cumulative Fourier Aggregation (CFA) is the load-bearing server-side mechanism: it applies a fast Fourier transform to each client's convolutional and fully connected weight tensors, averages only the low-frequency components across clients according to a mask whose radius grows during training, and then inverts the transform, leaving each client's high-frequency components intact. Collaborative Transfer Optimization (CTO) is the client-side mechanism: a second 'client model' receives the server aggregate and is updated through Retrieve (restore with the personalized local model as teacher), Reciprocate (mutual learning between the two models once performance is close), and Refine (the client model teaches the personalized model). The two mechanisms together are intended to accumulate shared low-frequency knowledge while preserving client-specific high-frequency knowledge and local progress.","core_discovery":"The paper's central claim is that the performance drop seen in federated aggregation under data heterogeneity comes from two correctable errors: element-wise averaging in the parameter space, and overwriting local models with the server aggregate. FedMetaMed replaces the first with Cumulative Fourier Aggregation, which transforms each client's weights with the fast Fourier transform, averages only the low-frequency band (a central mask grown from $s_0=0.26$ to $s_1=0.55$ over training), and keeps each client's high-frequency components untouched. It replaces the second with Collaborative Transfer Optimization, a three-step Retrieve–Reciprocate–Refine procedure in which a separate client model receives the server aggregate, is first restored by distillation from the local personalized model, then exchanges knowledge with it, and finally acts as its teacher. On two real-world dermatology datasets, the authors report that this combination gives the best F1 and AUC among all compared baselines, and that an unseen out-of-distribution cohort is classified with 89.51% accuracy and 75.12% F1.","pith_inferences":["A natural extension the paper does not test: apply CFA to non-convolutional architectures (e.g., attention-based networks), where the frequency decomposition of weight matrices has no established semantic meaning, to see if the low-frequency/high-frequency split still holds.","The privacy discussion is qualitative; the paper does not analyze how much information about local data the shared high-frequency parameters can leak, so the frequency split is best read as a heuristic rather than a formal privacy guarantee.","The frequency threshold schedule ($s_0=0.26$ to $s_1=0.55$) is fixed by hand; one testable extension is whether the schedule needs to be re-tuned per dataset, architecture, or heterogeneity level for the gains to persist."],"forward_implications":["If low-frequency weight components really do carry shared knowledge, CFA could be applied as a drop-in aggregation rule for other federated learning tasks, not just medical imaging.","Because CTO never overwrites the local model, clients would not lose their accumulated local knowledge at each communication round, which could improve convergence speed and final accuracy in heterogeneous settings.","Reported out-of-distribution F1 of 75.12%, over 20 points higher than the closest baseline, suggests that models trained this way might generalize to hospitals whose patient populations differ from the training cohort.","The paper's scaling experiment from 4 to 50 clients indicates the method keeps its advantage as the number of participants grows, which would matter for real hospital networks."],"supporting_citations":[{"why":"Supplies FedAvg, the standard averaging baseline that CFA replaces and that FedMetaMed is compared against.","marker":"[14]"},{"why":"Supplies FedProx, a baseline that adds a proximal term to align local updates, used as a comparative method.","marker":"[15]"},{"why":"Supplies FedBN, a personalized FL method preserving local batch normalization statistics, the strongest personalized baseline to beat.","marker":"[17]"},{"why":"Supplies SiloBN, a siloed federated learning method with local batch normalization, another comparative baseline.","marker":"[18]"},{"why":"Supplies MetaFed, the cyclic-knowledge-distillation personalized FL method that FedMetaMed directly compares against and extends.","marker":"[23]"}],"fun_headline_variants":["Low-frequency server average beats high-frequency noise in FL","Fourier-filtered aggregation lifts federated OOD accuracy to 89%","Medical FL gets personal: meta-learning plus Fourier aggregation","FedMetaMed: less weight averaging, more knowledge transfer","Low-frequency mask in server aggregation personalizes medical FL"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire method depends on the idea that the low-frequency parts of a model's weights contain the shareable, general knowledge while the high-frequency parts are client-specific; if that split is wrong, the aggregation could throw away useful knowledge or keep noise, and the reported improvements would not carry over to other settings.","fun_headline_variants_meta":{"raw":{"variants":["Low-frequency server average beats high-frequency noise in FL","Fourier-filtered aggregation lifts federated OOD accuracy to 89%","Medical FL gets personal: meta-learning plus Fourier aggregation","FedMetaMed: less weight averaging, more knowledge transfer","Low-frequency mask in server aggregation personalizes medical FL"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000501,"raw_usage":{"total_tokens":2488,"prompt_tokens":1022,"completion_tokens":1466,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":638,"completion_tokens_details":{"reasoning_tokens":1383}},"tokens_in":638,"tokens_out":1466,"duration_ms":12959,"temperature":1.0,"reasoning_tokens":1383,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T22:00:47.897710+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the identical FedMetaMed procedure on the same datasets and splits, but reverse the frequency mask so that high-frequency components are averaged and low-frequency components are kept per client. If the F1 improvement over FedAvg persists, the claimed spectral-semantics mechanism is not what drives the result, and the gains would need a different explanation.","supporting_citations":[{"cited_title":"Siloed federated learning for multi-centric histopathology datasets,","cited_arxiv_id":null,"evidence_quote":"Supplies SiloBN, a siloed federated learning method with local batch normalization, another comparative baseline."},{"cited_title":"Metafed: Feder- ated learning among federations with cyclic knowledge distillation for personalized healthcare,","cited_arxiv_id":null,"evidence_quote":"Supplies MetaFed, the cyclic-knowledge-distillation personalized FL method that FedMetaMed directly compares against and extends."}],"review_version":1}