{"id":"1ad030a7-7ae6-4821-b799-5cd5334cc562","arxiv_id":"2411.14070","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"An empirical study of design choices for asynchronous federated learning on the non-IID Extrasensory HAR dataset, with an open-source Flower extension.","lead":"This paper studies how to move human activity recognition from centralized training to federated learning when data is unevenly spread across users. It finds that global data scaling and SGD with momentum work better, and releases an open-source extension for asynchronous federated learning.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Equation 2 as printed is not FedAsync: with Δ_i defined as client gradients, x_t+Δ_i is dimensionally invalid, and the rule never subtracts the old global model. The AFL extension and all AFL results hinge on this equation (or on code that contradicts it).","rationale":"I read the paper as an empirical design-decision study for non-IID HAR with an open-source AFL tool. The most load-bearing condition for the central claim is that the AFL implementation actually follows the FedAsync update rule it states. Equation 2 fails that condition as written. If the code is checked and correct, this is a typo; then the remaining main limitation is external validity, exactly as the reader said. If the code mirrors Equation 2, the AFL experiments are not trustworthy. This is more specific than the reader's generalizability concern, so I partially agree. The reader's points about no error bars, single dataset, and missing baselines remain valid but secondary. I keep the reader's CONDITIONAL verdict, now with a sharper condition: verify the implemented update rule and correct Equation 2 if the code is FedAsync-correct.","tokens_in":15200,"tokens_out":7241,"duration_ms":65259,"concrete_test":"Inspect the server-side aggregate method in the public GitHub repository (github.com/r-gg/flower-async-fork) and run a one-step synthetic merge: initialize global x_t = e1, client sends x_i = e2 (equivalently Δ_i = e2 - e1), with α_MR(n_i/N) = 0.5. If the implementation produces x_{t+1} = 0.5 e1 + 0.5 e2, it implements FedAsync and Equation 2 is a typo. If it produces x_{t+1} = e1 + 0.5(e1 + e2) = 1.5 e1 + 0.5 e2, or any form matching the printed Equation 2, the implementation is not FedAsync and the AFL experiments in Section 5 should be rerun with the corrected rule. Re-checking the repository is feasible since the code is open source.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central software contribution is the claim in Section 3 that the Flower extension 'enables AFL' and follows the FedAsync update rule stated as Equation 2. Equation 2 is x_{t+1} = x_t + α_MR (n_i/N)(x_t + Δ_i), with Δ_i defined as 'client i's gradients sent to the server for aggregation.' Under that definition, x_t + Δ_i is dimensionally invalid (parameter vector plus gradient vector), and no term subtracts the old global model. The FedAsync convex-combination update is x_{t+1} = (1 - α)x_t + α x_i = x_t + α(x_i - x_t). If Δ_i is intended to be the client model delta x_i - x_t, the correct expression is x_t + α_MR(n_i/N)Δ_i, not x_t + α_MR(n_i/N)(x_t+Δ_i). If Δ_i is intended to be the full client model x_i, the printed rule reduces to x_t + β x_i, which adds the client model to the current global model every update and would diverge. Thus either Equation 2 is a typographical slip and the actual implemented rule is not specified in the paper, or the implemented AFL is not FedAsync and the convergence reported in Section 5.3 is unexplained by the stated algorithm. This is load-bearing because the paper's central practical deliverable and the validity of the AFL experiments depend on this update rule.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a methodology for transitioning from centralized learning to synchronous and asynchronous federated learning for human activity recognition under non-IID data. Using the Extrasensory dataset, the authors evaluate the impact of data augmentation, feature scaling, optimizer choice, learning rate, and batch size on model performance in centralized, synchronous federated, and asynchronous federated settings. They report that moderate Gaussian-noise augmentation, global feature scaling, and SGD with momentum perform best, and they release an open-source extension of the Flower framework intended to enable asynchronous federated learning. The paper positions these results as practical guidance for IoT/HAR deployments and emphasizes the persistence of feature skew after global standardization.","tokens_in":15508,"tokens_out":8165,"duration_ms":71950,"significance":"If the empirical findings are reliable, the paper offers a useful, reproducible blueprint for IoT/HAR practitioners facing non-IID federated data, and the open-source Flower extension is a concrete community resource. The main strengths are the use of a realistic, naturally non-IID dataset, a clearly described preprocessing pipeline, and public code. However, the strength of the empirical conclusions is limited by the use of a single dataset and a single run per configuration, and the central update equations of the paper are stated incorrectly, so the exact algorithms used in the experiments are not specified by the manuscript. The authors explicitly acknowledge the single-dataset limitation in Section 7, which is commendable, but the methodological issues with the aggregation formulas and test-set tuning need to be addressed before the claims can be fully trusted.","major_comments":[{"comment":"The synchronous update rule is misstated. With x_i^{t+1} defined as client i's updated model after local training, the standard FedAvg update is x_{t+1} = sum_{i in S} (n_i/N) x_i^{t+1}, not x_{t+1} = x_t + sum_{i in S} (n_i/N) x_i^{t+1}. As printed, the server would add full client models to the current global model every round rather than replacing the global model with their weighted average. Please correct the equation or explicitly define x_i^{t+1} as a model delta relative to x_t.","section":"Section 2.2, Eq. (1)"},{"comment":"The asynchronous update rule is dimensionally inconsistent and does not reproduce FedAsync. If Delta_i denotes client gradients, then x_t + Delta_i mixes parameter and gradient spaces; if Delta_i denotes the client model delta x_i - x_t, the extra x_t inside the parentheses is spurious; if Delta_i denotes the full client model, the rule accumulates models and would diverge. The FedAsync convex-combination update should be x_{t+1} = x_t + alpha_MR (n_i/N) Delta_i with Delta_i = x_i - x_t. Because Section 3 states that the server updates the global model according to Equation 2, the actual AFL algorithm implemented in the Flower extension and used in Section 5 is not specified by the paper.","section":"Section 2.2, Eq. (2)"},{"comment":"Hyperparameters (batch size, learning rate, mixing ratio) are selected using the centralized test set: Section 4.4 states that models are evaluated on the test set during tuning, and Section 5.1 uses the centralized test set on the server for final evaluation. Using the same test set for both model selection and final evaluation introduces optimistic bias into all reported numbers in Tables 1 and 2. Please reserve a separate validation split for tuning and report final metrics on a truly held-out test set.","section":"Sections 4.4 and 5.1"},{"comment":"The experimental results are single runs with no error bars, repeated seeds, or significance tests. Consequently, several headline claims are not statistically supported: for example, in Table 2 the SFL balanced accuracy for global versus local scaling is identical (0.61 vs. 0.61) while the text claims global scaling improves all metrics, and the SFL macro-F1 difference is only 0.02. Please provide uncertainty quantification (e.g., multiple seeds with confidence intervals) or soften the comparative claims accordingly.","section":"Section 5, Table 2"}],"minor_comments":[{"comment":"The abstract says \"multi-label classification,\" but the experiments use six mutually exclusive labels, which is multi-class classification as clarified in Section 4.4; please align the abstract's terminology with the body.","section":"Abstract"},{"comment":"There is a typo: \"waling\" should be \"walking.\"","section":"Section 4.2.1"},{"comment":"References [33] and [34] are the same survey (Xu et al., \"Asynchronous federated learning on heterogeneous devices: A survey\"); please merge them or remove the duplicate.","section":"References"},{"comment":"The claim that global scaling improves \"all four tracked metrics\" is not fully supported by Table 2, which omits the Sitting F1 score and shows equal SFL balanced accuracy for global and local scaling; please specify that the claim refers to the AFL curves in Figure 4 or add the missing metric.","section":"Section 5.2.3"},{"comment":"The statement that \"Flower does not support AFL\" should be qualified with respect to the Flower version used, since later versions of the framework may include asynchronous strategies.","section":"Section 3"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a conference-style empirical study with a valuable software artifact, but the incorrect statement of the aggregation equations is a serious issue because the experiments are not reproducible from the paper as written. The test-set tuning and lack of repeated runs further weaken the quantitative claims. I would not accept the paper in its current form, but I believe a careful revision that corrects Equations (1) and (2), re-runs or re-reports the evaluation with proper validation and uncertainty quantification, and tightens the claims is within scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, the thing to know: this is a useful empirical paper for the FL-for-IoT community, but the central update rule (Eq. 2) is wrong as printed, and that needs to be fixed before the AFL results can be taken at face value.\n\nWhat's genuinely new: a systematic comparison of design decisions—data augmentation, feature scaling, optimizer, batch size and learning rate—for synchronous and asynchronous FL on the Extrasensory dataset, plus an open-source Flower extension that implements AFL. The methodology is clearly written, the takeaways are plausible (global scaling helps, moderate Gaussian augmentation helps, SGD-m beats Adam in stateless AFL, server delays hurt), and Section 7 honestly acknowledges the single-dataset limitation. That is concrete value for practitioners moving from centralized to federated HAR.\n\nThe soft spots are what the reader's report flags: no repeated runs or error bars, so the rankings in Tables 1 and 2 could sit within noise; one dataset only; and no comparison against existing AFL baselines (FedAsync, ASO-Fed) despite citing them. The title also overclaims: there is no adaptive mechanism here, just design recommendations, with adaptivity deferred to future work.\n\nBut the biggest issue is Equation 2, which the stress-test note catches and which I confirmed on reading the paper. With Δ_i defined as client gradients, x_t + Δ_i in the equation is not a valid model update, and the rule never subtracts the old global model. FedAsync should be x_t + α(x_i − x_t), or equivalently x_t + α Δ_i if Δ_i is the client delta. As printed, the equation adds a scaled copy of the global model to itself, which would diverge. The text says the implementation 'follows' Equation 2, so either the equation is a typo and the code does something else—in which case the paper misdescribes its own flagship contribution—or the code really does this update, and the reported convergence is unexplained. The authors need to provide the actual update rule and align the code.\n\nIf it's a typo, the paper is a modest but useful empirical study; if not, the AFL experiments are suspect. Either way, this deserves a serious referee and a revision, not a desk reject. The engineering and dataset analysis are worth preserving. I'd bring it to a reading group only if someone is actively building FL for IoT, and I wouldn't cite it in my own work until the equation is corrected.","headline":"Useful empirical FL-for-IoT paper with a load-bearing error in its core update equation—worth a revision, not a desk reject.","tokens_in":16016,"tokens_out":3451,"would_cite":false,"duration_ms":31520,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"For human activity recognition under federated learning with severely skewed device data, this paper claims the winning design choices are global feature scaling, SGD with momentum, and moderate minority-class augmentation, and it ships…","keywords":["asynchronous federated learning","non-IID data","human activity recognition","IoT","data augmentation","feature scaling","optimizer selection","Flower framework"],"falsifier":"Run the same ablation matrix (global vs local scaling, none vs base vs balanced augmentation, SGD-m vs ADAM) on a second naturally non-IID HAR dataset, such as the Heterogeneity Human Activity Recognition dataset or a client-partitioned UCI-HAR, using the same test protocol; if global scaling does not beat local scaling or ADAM beats SGD-m on the minority class, the paper's design-decisions ranking fails.","tokens_in":15010,"feed_emoji":"🏃","tokens_out":9804,"duration_ms":85742,"temperature":0.7,"pith_summary":"This paper is trying to turn the move from centralized to federated learning for human activity recognition into a set of concrete, testable design choices, and to identify which choices actually work when the data is severely uneven across devices. Using a real dataset of 60 people's phone-sensor readings, it claims that globally standardizing features before training, using SGD with momentum rather than ADAM, and adding a moderate amount of Gaussian-noise augmentation to rare activity labels all improve model performance, in both synchronous and asynchronous federated learning. If the claims hold, practitioners have a direct, no-frills recipe for federated HAR on non-IID IoT data, plus an open-source extension of the Flower framework that lets them run asynchronous federated learning themselves. The main caveat, stated by the authors themselves, is that everything rests on experiments with a single dataset.","feed_headline":"Global scaling beats local scaling in federated activity recognition","feed_subtitle":"Real-phone-sensor ablations show global scaling and SGD-m beat local scaling and ADAM in non-IID federated learning.","key_machinery":"The load-bearing mechanism is the asynchronous aggregation rule of Equation 2, $x_{t+1}=x_t+\\alpha_{MR}\\frac{n_i}{N}(x_t+\\Delta_i)$, implemented in an open-source Flower extension, together with the paper's methodology of evaluating each design decision (scaling, augmentation, optimizer, batch size, learning rate) under fixed update counts and from central and distributed vantage points on the Extrasensory dataset.","core_discovery":"On the paper's own terms, the central discovery is that the apparently messy problem of non-IID federated learning for HAR responds to a small set of preprocessing and optimizer choices, and that these choices interact with the asynchronous setting in a specific way. Equation 2 defines the asynchronous server update: when client $i$ sends gradient $\\Delta_i$, the global model becomes $x_{t+1}=x_t+\\alpha_{MR}\\frac{n_i}{N}(x_t+\\Delta_i)$, with mixing ratio $\\alpha_{MR}$ and client sample share $n_i/N$. Running this update on the Extrasensory dataset, the paper finds that global feature scaling (sharing client means and standard deviations with the server to form one global standardization) improves all four tracked metrics over client-local scaling; that SGD with momentum outperforms ADAM, which suffers because its per-client optimizer state is reset every round; that moderate Gaussian-noise augmentation (\"base\") beats both no augmentation and fully balanced augmentation; and that feature skew persists for minority classes even after global standardization. The paper also finds that adding server-side delay degrades model quality, and it provides an open-source Flower extension that implements the asynchronous update.","pith_inferences":["Because the augmentation intensities (20x for running, 8x for cycling) were fixed rather than swept, the claim that \"base augmentation is best\" may be specific to these multiplicities; sweeping augmentation strength on other datasets is a natural next test.","The ADAM-versus-SGD-m result likely depends on the very small local epoch count; if the paper's suggested fixes (server-side ADAM or shared state) remove the state-reset problem, the ranking could change, and the paper itself leaves this open.","The persistence of feature skew after global standardization suggests that simple feature-wise scaling cannot eliminate inter-client distribution shift; techniques that calibrate per-client feature representations (e.g., per-client normalization layers) are an untested extension of the paper's finding.","The single-dataset evaluation makes transferability the key open question; repeating the same ablation matrix on at least one more naturally non-IID HAR dataset would turn the paper's blueprint into a general result."],"forward_implications":["A practitioner transitioning a HAR system to federated learning should standardize features with global statistics computed across clients rather than per-client statistics; this is the single data-side choice with the clearest performance gain in the paper's ablations.","Use SGD with momentum, not ADAM, unless the optimizer state is shared or applied server-side; ADAM's per-round state reset makes it weaker and noisier on rare classes in stateless federated optimization.","Treat data augmentation as a tunable knob: moderate oversampling of rare classes with small Gaussian noise helps, but forcing class balance can backfire by biasing training toward replicated samples.","For evaluation, use a fair test set assembled from all clients; leaving clients out can omit entire classes and hide the true model quality in highly skewed data.","In asynchronous FL deployments, server-side latency matters even when the server does no training; measurable delay reduces model quality independent of client update count."],"supporting_citations":[{"why":"Supplies the Extrasensory dataset, the real-world in-the-wild HAR data whose natural label, quantity, and feature skew drives every experiment.","marker":"[29]"},{"why":"Provides the Flower framework that the paper extends for asynchronous FL and whose ray simulation hosts the evaluation.","marker":"[2]"},{"why":"Defines FedAsync, the asynchronous federated aggregation whose update rule Equation 2 follows.","marker":"[32]"},{"why":"Along with FedAsync, one of the asynchronous FL designs the paper's Flower extension is based on.","marker":"[6]"},{"why":"Provides the privacy-preserving asynchronous FL mechanism whose server-side merging informs the paper's update rule.","marker":"[15]"},{"why":"Gives the model architecture and balanced/macro metrics for the Extrasensory task that the paper reuses as its baseline and evaluation.","marker":"[30]"},{"why":"Defines ADAM, the optimizer the paper compares against SGD with momentum in the stateless federated setting.","marker":"[11]"}],"fun_headline_variants":["SGD-m outperforms ADAM in federated activity recognition","Feature skew persists after global scaling in federated HAR","Global scaling and SGD-m key for federated activity recognition","Async federated learning: global scaling beats local for HAR"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The ranking of design decisions measured on the Extrasensory dataset, with its particular client split, 64-16-20 local split, mean imputation, and six mutually exclusive labels, carries over to other HAR and IoT federated learning deployments.","fun_headline_variants_meta":{"raw":{"variants":["SGD-m outperforms ADAM in federated activity recognition","Feature skew persists after global scaling in federated HAR","Global scaling and SGD-m key for federated activity recognition","Async federated learning: global scaling beats local for HAR"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000397,"raw_usage":{"total_tokens":2109,"prompt_tokens":1006,"completion_tokens":1103,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":622,"completion_tokens_details":{"reasoning_tokens":1036}},"tokens_in":622,"tokens_out":1103,"duration_ms":8996,"temperature":1.0,"reasoning_tokens":1036,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T15:33:12.752365+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same ablation matrix (global vs local scaling, none vs base vs balanced augmentation, SGD-m vs ADAM) on a second naturally non-IID HAR dataset, such as the Heterogeneity Human Activity Recognition dataset or a client-partitioned UCI-HAR, using the same test protocol; if global scaling does not beat local scaling or ADAM beats SGD-m on the minority class, the paper's design-decisions ranking fails.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Along with FedAsync, one of the asynchronous FL designs the paper's Flower extension is based on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the privacy-preserving asynchronous FL mechanism whose server-side merging informs the paper's update rule."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Gives the model architecture and balanced/macro metrics for the Extrasensory task that the paper reuses as its baseline and evaluation."}],"review_version":1}