{"id":"0c8d3b86-59a9-4fd7-bb33-e55c950041aa","arxiv_id":"2506.17466","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"FedNAMs averages per-feature neural additive models across clients in federated learning, but the paper lacks the quantitative accuracy comparison its central claim requires.","lead":"The paper combines Neural Additive Models, which explain predictions through per-feature functions, with federated learning, where models train on distributed data. It claims this yields interpretable federated models with little accuracy loss, but reports no accuracy numbers to support that claim.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The abstract's central claim of 'minimal accuracy loss' is unsupported: Section 6 promises evaluation by accuracy and ROC-AUC, but no quantitative metrics, baseline comparison, or error bars appear anywhere in the paper.","rationale":"The reader's REJECT verdict is correct, but the most load-bearing concern is not the function-space averaging in Equations (3)–(7). Even if that averaging is accepted as a design choice, the paper's central comparative claim is untested: no accuracy numbers appear in Section 6 or Appendix B. The only quantitative tables are attribution scores, not model performance. A rejection on missing evidence is stronger and simpler than speculating about alignment, which may be unnecessary for univariate NAM shape functions. The reader's algorithm concern is worth noting—averaging per-client feature functions rather than weights is never justified theoretically—but it is secondary to the absence of any measured accuracy. My recommendation is therefore UNCHANGED: the paper should remain rejected until the promised evaluation is actually reported.","tokens_in":11743,"tokens_out":3694,"duration_ms":42911,"concrete_test":"Run the reproducibility repository (github.com/amitashnanda/FedNAM) on the three reported datasets with the paper's three-client setup, computing test accuracy and ROC-AUC for FedNAMs and a matched FedAvg DNN baseline under identical train/validation splits, client partitions, and hyperparameter search; report per-client and global metrics with standard deviations over at least three seeds. If FedNAM accuracy is absent or not within a pre-specified 'minimal loss' threshold (e.g., at least 95% of baseline accuracy) on all datasets, the central claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that FedNAMs 'deliver strong interpretability with minimal accuracy loss compared to traditional Federated Deep Neural Networks'—has no quantitative support in the manuscript. Section 6 states that 'model performance is evaluated based on classification accuracy and metrics such as the ROC-AUC score,' yet no accuracy, ROC-AUC, confidence interval, or baseline comparison is reported. Tables 1–3 list client-wise feature contributions and Captum attribution values, not predictive performance; Appendix B contains only shape-function plots and a masked MNIST image. There is no table or figure comparing FedNAMs to a Federated DNN on any dataset, so the 'minimal accuracy loss' and 'state-of-the-art' statements in the abstract and conclusion are overclaims. This is a missing-evidence problem, not merely a stylistic issue: the paper's own evaluation section promises metrics that never appear.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FedNAMs, a federated learning framework that trains Neural Additive Models (NAMs) on each client and aggregates the per-feature shape functions by averaging them in function space. The authors claim that FedNAMs provide strong interpretability, identify critical predictive features at client and global levels, and achieve state-of-the-art performance with minimal accuracy loss relative to traditional federated DNNs, based on experiments on the UCI Heart Disease, OpenML Wine, and Iris datasets. The manuscript describes the architecture, presents client-wise feature contribution tables and shape-function plots, and qualitatively compares with Captum attributions on a masked MNIST image. It also includes a reproducibility artifact link and an appendix with additional plots.","tokens_in":11899,"tokens_out":3213,"duration_ms":36019,"significance":"If the claims were substantiated, FedNAMs would be a useful contribution to interpretable federated learning: per-feature, client-level explanations on tabular benchmarks without major accuracy loss would be practically valuable, especially in finance and healthcare. The architectural idea of imposing an additive structure in a federated setting and aggregating shape functions is reasonable and worth exploring. However, the paper's central empirical claims are entirely unsupported by the presented evidence: no predictive accuracy, ROC-AUC, variance, or baseline comparison appears anywhere, and the interpretability benchmark against Captum is not quantitatively validated. The paper's current value is therefore only as a proposal sketch, not as a demonstrated method.","major_comments":[{"comment":"The evaluation section states that 'model performance is evaluated based on classification accuracy and metrics such as the ROC-AUC score,' but no accuracy, ROC-AUC, confidence interval, or baseline comparison is reported anywhere in the manuscript. Tables 1-3 list feature contribution values and Captum attributions, and Appendix B contains only shape-function plots and a masked MNIST image. The abstract's claim of 'minimal accuracy loss compared to traditional Federated Deep Neural Networks' and the conclusion's claim of 'state-of-the-art performance' are therefore unsupported by any quantitative evidence. This is a load-bearing omission: the paper's central contribution is empirical, and the promised evaluation is absent.","section":"Section 6 (Experimentation and Results)"},{"comment":"The aggregation rule averages client-specific shape functions f_i(x_k) in function space to obtain the global feature function. In contrast, standard FedAvg, given in Equation (2), averages network weights. The manuscript provides no justification that per-feature networks across clients are semantically aligned such that averaging their outputs yields a meaningful global model. Issues such as permutation of hidden units, differing bias terms, non-identifiability of additive components (the E[f_i]=0 centering constraint is mentioned in Section 1 but not enforced or discussed in the federated setting), and lack of convergence guarantees are not addressed. Without an alignment argument or an experiment comparing function-space averaging to weight-space FedAvg, the validity of the 'final aggregated feature function' is not established, and this directly affects both accuracy and interpretability claims.","section":"Equations (3)-(7)"},{"comment":"The interpretability benchmark against Captum is not quantitatively validated. Table 1 reports client-wise feature contributions on a scale of roughly 2-5, while Table 2 reports Captum attribution values on a scale of roughly -0.01 to 0.003; no correlation, rank agreement, or normalized comparison is given. The text claims that FedNAMs offer 'more detailed and feature-specific interpretability than Captum,' but no metric supports this comparative claim. Figure 6 is a qualitative image comparison. The assertion that the framework's interpretability is validated by this benchmark is therefore unsupported.","section":"Tables 1-3 and Section 6.1"},{"comment":"Table 3 is captioned as 'Client-wise feature contributions and Feature attribution values of Captum for UCI Wine dataset with reduced precision,' but the table lists heart-disease features (thalach, thal, age, ca, cp, sex, trestbps, oldpeak, fbs, restecg, exang, slope, chol), not wine features such as fixed acidity, volatile acidity, or sulfates. This internal inconsistency invalidates the wine-specific feature contribution results and undermines the paper's claims about identifying critical predictive features for the wine dataset. The source of this error needs to be corrected and the results regenerated.","section":"Table 3"},{"comment":"The abstract claims the paper studies 'various text and image classification tasks,' but the experiments use only tabular datasets (Heart Disease, Wine, Iris). The only image experiment is a qualitative MNIST illustration of masked pixels in Section 6.2, which is not a classification task with reported performance. This overstatement misrepresents the scope of the experiments and should be corrected.","section":"Abstract and Section 6.2"}],"minor_comments":[{"comment":"The Iris dataset description says the target has three classes but names only 'Iris-setosa and Iris-versicolor'; the third class (Iris-virginica) is missing. Also, the text describes Iris classification as 'multi-label classification,' but it is a multiclass problem, not multi-label.","section":"Section 5 (Datasets)"},{"comment":"There is a typo: 'model'sel's' should be 'model's'.","section":"Section 2"},{"comment":"The notation g(E[y_client1]) is unclear; it mixes client indices with a generic link function and does not define the relationship between the per-client equations and the aggregated functions in Equations (3)-(7). The superscripts in later equations also appear inconsistent (e.g., f_i^j versus f_j^i).","section":"Equations (8)-(11)"},{"comment":"The phrase 'with reduced precision' is vague; it is unclear whether this refers to numerical precision, model precision, or something else, and no explanation is given.","section":"Table 3 caption"},{"comment":"Some references do not support the claims they are cited for. For example, Zhang et al. (2024a) is cited as 'proposed FedGNN, a federated learning framework that utilizes Graph Neural Networks,' but the listed reference is 'Recent methodological advances in federated learning for healthcare,' which does not describe FedGNN. The citation for Bonawitz (2019) also lacks page numbers and appears in an unusual format.","section":"References"}],"recommendation":"reject","confidential_remarks":"The manuscript appears to be an early draft: the central evaluation section promises metrics that never appear, Table 3 contains features from the wrong dataset, and several claims go beyond the presented evidence. The missing quantitative results are not a local presentation issue; they are the substance of the paper's contribution. Even if the experiments were added, the current interpretability validation would need a redesign to compare FedNAMs with a federated DNN baseline on accuracy and with a quantitative attribution agreement metric. Given the manuscript's current state, rejection is appropriate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this paper proposes FedNAMs, a federated version of Neural Additive Models, and claims strong interpretability with minimal accuracy loss. There are no accuracy numbers anywhere in the manuscript. The claim is not supported.\n\nWhat is actually new: composing NAMs with federated averaging is, as far as the cited literature goes, new. The architecture is simple and the paper describes it clearly: each client trains a NAM, and the global model averages the per-feature shape functions. That is a reasonable baseline for interpretable federated learning. The paper also includes a reproducibility link and client-wise shape plots, which is more than many submissions in this space.\n\nThe soft spots are big. Section 6 says performance is evaluated by accuracy and ROC-AUC, but no such metrics appear. Tables 1–3 report feature contribution scores and Captum attributions, not predictive performance. There is no baseline comparison to a federated DNN or any other model. The abstract's 'minimal accuracy loss' and the conclusion's 'state-of-the-art' are overclaims on the evidence presented. The benchmark against Captum is on a different scale and no quantitative agreement is shown. Equations (3)–(7) average client shape functions in function space without any alignment or convergence argument, which is a real concern, though secondary to the missing evaluation. Several citations are also mismatched: Gu et al. 2021 is about device unavailability, not interpretable decision trees.\n\nThe paper is a plausible idea that needs a real experimental section. It is not ready for review as a full paper. I would desk reject with an invitation to resubmit once the authors provide accuracy, ROC-AUC, baselines, and error bars.","headline":"Routine composition of NAMs and FedAvg with no evaluation to back the central accuracy claim.","tokens_in":12409,"tokens_out":2390,"would_cite":false,"duration_ms":26535,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Federated learning gains per-feature explanations by training one small network per input feature and averaging the feature functions across clients.","keywords":["federated learning","neural additive models","interpretability","generalized additive models","feature attribution","explainable AI","distributed training","tabular data"],"falsifier":"Train FedNAMs on the same three benchmark datasets with a deliberately non-IID partition (e.g., each client receiving one class or one subpopulation) and compare the function-space-averaged global model's accuracy and per-feature curves against a centrally trained NAM on the pooled data; if the averaged model's accuracy or the shape of the curves diverges substantially, the aggregation rule does not hold.","tokens_in":11532,"feed_emoji":"🔍","tokens_out":4469,"duration_ms":46071,"temperature":0.7,"pith_summary":"FedNAMs places a Neural Additive Model — one small network per input feature — inside a federated learning loop, so that each client learns its own shape function for every feature while data never leaves the device. The paper's central claim is that averaging these client shape functions pointwise yields a global model that is both accurate and interpretable, with only a small accuracy trade-off compared with ordinary federated deep networks. That matters because finance and healthcare need explanations of model decisions, and federated models have so far been mostly black boxes. On the Wine, Heart Disease, and Iris benchmarks, the framework identifies the same high-importance features (volatile acidity, sulfates, chlorides; chest pain type, maximum heart rate, number of vessels; petal length and width) at both client and global level.","feed_headline":"Per-feature networks keep federated models interpretable","feed_subtitle":"Client-level shape functions reveal what drives predictions on heart, wine, and iris data with minimal accuracy loss.","key_machinery":"The load-bearing object is the per-feature shape function $f_i(x_k)$: each input feature is processed by its own small neural network (a FeatureNN with three hidden layers of 20 neurons and ExU units), and the federated global model is the pointwise average of the client shape functions (Eqs. 3-7), rather than an average of network weights as in FedAvg. This function-space averaging is what converts client-local additive models into a global additive model whose per-feature plots are directly readable.","core_discovery":"The paper introduces Federated Neural Additive Models (FedNAMs), in which the global model is not a single shared network but an additive collection of per-feature networks. Each client $i$ trains its own shape functions $f^i_k(x_k)$ for every feature $k$, and the final global feature function is defined as the unweighted average of the client functions, $f_k(x_k) = \\frac{1}{n}\\sum_i f^i_k(x_k)$. Because each feature's contribution is a separate sub-network, the model retains the glass-box property of Generalized Additive Models, and the per-client structure yields client-level explanations alongside the global explanation. The paper reports that on UCI Heart Disease, OpenML Wine, Iris, and MNIST, FedNAMs achieve strong interpretability with minimal accuracy loss compared to traditional federated DNNs, and that the identified important features match domain expectations.","pith_inferences":["If function-space averaging of independently trained shape functions is stable, FedNAMs could serve as a cheap diagnostic layer on top of existing federated deep networks, rather than a replacement architecture.","A testable consequence the paper leaves implicit is that the global explanation should approach the explanation of a centrally trained NAM on pooled data as client data becomes IID, giving a calibration check for the aggregation step.","The same aggregation rule could be applied to other additive interpretable models (e.g., spline GAMs or boosted trees) inside federated loops, provided their feature functions are comparable across clients."],"forward_implications":["Federated models on tabular data can be glass-box: each feature's marginal effect is visible as a curve, both per client and globally.","Privacy is preserved at the level of standard federated learning, since only shape-function updates (or the functions themselves) leave clients, not raw data.","Low-contributing features can be identified and potentially dropped, streamlining models in federated deployment.","Client-level variation in shape functions provides a natural diagnosis of non-IID data drift.","The approach extends NAMs' multiclass and multitask flexibility to federated settings, beyond tree-based GAMs."],"supporting_citations":[{"why":"Supplies the Neural Additive Model architecture and ExU units that FedNAMs places inside each client.","marker":"Agarwal et al., 2021"},{"why":"Supplies the federated averaging paradigm that FedNAMs adapts, and the standard it compares against.","marker":"McMahan et al., 2017"},{"why":"Supplies the Generalized Additive Model formulation whose feature functions NAMs and FedNAMs inherit.","marker":"Hastie, 2017"},{"why":"Provides LIME, a local explainability baseline that the paper contrasts with its global per-feature approach.","marker":"Ribeiro et al., 2016"},{"why":"Provides SHAP feature attribution, the baseline used for benchmarking feature importance.","marker":"Lundberg & Lee, 2017"}],"fun_headline_variants":["Federated models get a glass-box upgrade with per-feature networks","Per-feature shapes reveal client and global predictors in federated learning","FedNAMs: federated learning with per-feature interpretability","Client-specific shape functions make federated models explainable","Federated additive models pinpoint predictive features across clients"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper assumes that client-specific feature networks are comparable enough that averaging their per-feature outputs yields a valid global model, without aligning the networks or proving that the average preserves the accuracy and zero-centering of the individual shape functions.","fun_headline_variants_meta":{"raw":{"variants":["Federated models get a glass-box upgrade with per-feature networks","Per-feature shapes reveal client and global predictors in federated learning","FedNAMs: federated learning with per-feature interpretability","Client-specific shape functions make federated models explainable","Federated additive models pinpoint predictive features across clients"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00105,"raw_usage":{"total_tokens":4443,"prompt_tokens":1009,"completion_tokens":3434,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":625,"completion_tokens_details":{"reasoning_tokens":3348}},"tokens_in":625,"tokens_out":3434,"duration_ms":25723,"temperature":1.0,"reasoning_tokens":3348,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T23:29:55.420561+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train FedNAMs on the same three benchmark datasets with a deliberately non-IID partition (e.g., each client receiving one class or one subpopulation) and compare the function-space-averaged global model's accuracy and per-feature curves against a centrally trained NAM on the pooled data; if the averaged model's accuracy or the shape of the curves diverges substantially, the aggregation rule does not hold.","supporting_citations":[{"cited_title":"Generalized additive models","cited_arxiv_id":null,"evidence_quote":"Supplies the Generalized Additive Model formulation whose feature functions NAMs and FedNAMs inherit."}],"review_version":1}