{"id":"938fc966-c478-42ae-a4a5-3918e64717be","arxiv_id":"2501.19279","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"S-VOTE selects clients by cosine similarity of model weights and lets low-vote clients sometimes skip training, reducing communication and energy while improving non-IID accuracy in some settings.","lead":"S-VOTE is a new method for devices in a decentralized federated learning network to pick which neighbors to train with, using how similar their model weights are. It claims to cut communication and energy use while improving accuracy under uneven data splits.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Under-specified threshold τ in Eq. (4) makes S-VOTE's central selection rule irreproducible; the reported gains cannot be attributed to a well-defined algorithm.","rationale":"The central contribution is the selection rule in Eq. (4); every claimed benefit flows through it. The paper gives τ as an input in Algorithm 1 but never sets it. The reader's weakest_assumption correctly identifies this. My stress-test supports that: without τ, the experiments are under-determined and cannot be distinguished from threshold tuning. I also note supporting concerns: Table II shows S-VOTE is not uniformly better than SCAFFOLD/FedProx (e.g., EMNIST α=0.5 F@20: SCAFFOLD/FedProx 0.84 vs S-VOTE 0.77; CIFAR10 α=0.5 F@20: SCAFFOLD 0.70 vs S-VOTE 0.65), and no significance tests are reported. However, the paper hedges 'in some configurations' and discloses these cases in the text, so the accuracy claim is not internally false. The resource claims are compared only to FedAvg, which is a stated choice. The under-specified τ is the most load-bearing because it blocks reproduction and causal attribution of the entire mechanism. The reader asked for exact τ and code; my concrete test is the minimal check that would settle it. If the authors disclose τ and reproduce results, conditional acceptance is appropriate.","tokens_in":11785,"tokens_out":9283,"duration_ms":94982,"concrete_test":"Obtain the τ value(s) used from the authors or, failing that, re-run the reported MNIST/FashionMNIST configurations with τ ∈ {−1, 0, 0.5, 1, 1.5, 2} and all other settings held fixed. If no fixed τ reproduces the F1/communication/energy entries in Tables II-IV, or if the F1 spread across τ exceeds the claimed 9–17% improvement margin, then the headline results depend on an unstated threshold and the central claim fails as written.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Algorithm 1 lists τ as an input to the selection rule, but the paper never reports the value(s) of τ used in any experiment. Eq. (4) selects client i iff cosine_sim(w_l, w_i) ≥ μ + τ·σ; with τ=0 the threshold is the mean similarity, with τ=1 one standard deviation above the mean, and with large τ almost no neighbors are selected. The composition of the aggregation set — and hence which clients receive enough votes to train, how often the p-probability conditional training triggers, and the measured bytes sent/received in Table IV — is therefore a function of an invisible hyperparameter. Section IV-A specifies V_min = N/2 and the Dirichlet α values, but is silent on τ. This is not a cosmetic omission: if τ was tuned per configuration, the 'in some configurations' accuracy gains in the abstract may reflect threshold search rather than the mechanism; if a fixed τ was used, it should be stated for reproducibility. Without τ, an independent implementation cannot reproduce Table II-IV nor determine whether the similarity-voting step is the cause of the claimed improvements.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents S-VOTE, a decentralized federated learning client-selection mechanism in which each client selects aggregation partners via cosine similarity of model weight vectors (Eq. 3), applies a threshold μ + τ·σ (Eq. 4), and votes for selected peers; clients that receive fewer than Vmin votes perform probabilistic local training. The authors evaluate S-VOTE on MNIST, FashionMNIST, EMNIST, and CIFAR-10 under Dirichlet α∈{0.1,0.5}, comparing against FedAvg, SCAFFOLD, and FedProx. They report F1 scores, elapsed time, energy consumption, and byte-transmission costs, claiming up to 21% lower communication costs, 4-6% faster convergence, 9-17% higher accuracy, and 14-24% lower energy consumption.","tokens_in":12054,"tokens_out":5245,"duration_ms":46432,"significance":"The proposed mechanism is simple and plausible: using model-similarity to restrict aggregation is a reasonable heuristic for non-IID DFL, and the conditional-training rule is intuitive. The paper evaluates on four datasets, two topologies, and two heterogeneity levels, using real network communications and an energy-monitoring module, which is a strength. If the algorithm is fully specified and the gains survive a more systematic comparison, this would be a useful contribution to resource-efficient DFL. However, the current manuscript is not yet reproducible because the key threshold τ is unreported, and the empirical support is selective with several configurations where a baseline outperforms S-VOTE by a wide margin.","major_comments":[{"comment":"Eq. (4) defines the selection rule with a threshold μ + τ·σ, and Algorithm 1 lists τ as an input, but Section IV-A never states the value of τ used in any of the experiments. Since τ controls how many neighbors are selected (τ=0 selects about half, large τ selects almost none), the reported F1 scores, convergence times, byte counts, and energy figures in Tables II-IV are functions of an unspecified hyperparameter. The paper must report τ for every configuration or provide a sensitivity analysis; without this, the central selection rule cannot be reproduced and the reported gains cannot be attributed to a well-defined algorithm.","section":"III-D / Algorithm 1"},{"comment":"The claim of 9-17% higher accuracy is not representative of Table II. On MNIST with α=0.1, FedProx achieves 0.97-0.98 while S-VOTE achieves 0.86-0.94 across the four topologies; on EMNIST and CIFAR-10, SCAFFOLD beats S-VOTE in most configurations, e.g., EMNIST α=0.5 fully-connected 20 clients: 0.84 vs 0.77, and CIFAR-10 α=0.5 fully-connected 20 clients: 0.70 vs 0.65. Also, the reported standard deviations (±0.2-0.3) are comparable to or larger than the differences in most configurations. The paper should report how often S-VOTE wins, loses, and ties per configuration, and quantify statistical significance, rather than highlighting only favorable cases.","section":"IV-B.1 / Table II"},{"comment":"Resource-efficiency comparisons are made only against FedAvg, while the abstract says 'compared to baseline methods'. SCAFFOLD and FedProx are excluded from the time/energy/communication tables because they 'introduce additional computational overhead' (Section IV-B.2), but this is not a sufficient justification when the abstract makes general claims. Furthermore, Table III reports elapsed time reduction (5-6%), not 'faster convergence' in terms of rounds, so the 4-6% faster convergence claim is not directly supported. The energy and communication gains also do not account for the extra cost of computing pairwise cosine similarities and voting messages.","section":"IV-B.2 / Tables III and IV"},{"comment":"Line 25 states that each client shares its updated model with all neighbors, yet Section IV-B.2 attributes the byte-count reductions to clients 'not waiting for updates from all their neighbors'. If models are always broadcast to all neighbors, the number of bytes sent per client should be similar to FedAvg; the reductions in Table IV require that non-selected or non-training clients either do not send or send less often. The paper should clarify the exact communication protocol and reconcile Algorithm 1 with Table IV.","section":"Algorithm 1, Steps 3-6"}],"minor_comments":[{"comment":"The dataset name 'CIFACR10' is a typo and should be 'CIFAR10'.","section":"Table I"},{"comment":"The norms of the weight vectors are written as |wi| and |wj|; they should be double bars, e.g., ||wi|| and ||wj||.","section":"Section III-C, Eq. (3)"},{"comment":"The title contains 'V oting' with an unusual space; the same spacing issue appears in the Algorithm 1 heading. The text should be 'Voting'.","section":"Title and Algorithm 1"},{"comment":"The statement that Vmin = N/2 'optimally stabilizes training' is not accompanied by a sensitivity analysis or comparison with other Vmin values, so this claim is not supported by the reported experiments.","section":"Section IV-A"},{"comment":"Reference [7] contains 'F oundations' with an unusual space, and reference [4] is an arXiv preprint rather than a peer-reviewed publication; the authors should verify the formatting of all references.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The missing τ is likely fixable by reporting the actual values used, which would restore reproducibility. However, the selective performance reporting and the inconsistency in the communication protocol require substantive revision. The paper's contribution is plausible, but the current version does not meet the standard for acceptance; a careful revision with full hyperparameter disclosure, rigorous statistical comparison, and a clarified protocol could change that."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The short version: S-VOTE is a coherent, incremental contribution to decentralized FL client selection, and it is worth engaging on the merits, but the paper as written under-specifies its central threshold τ and compares resource savings only against FedAvg. Both gaps are fixable and should gate acceptance.\n\nWhat is actually new: the combination of cosine-similarity-based neighbor selection with a voting threshold and an adaptive random-training skip rule. Each ingredient appears in prior work, but this particular package is new, and the paper positions itself honestly against FedeRiCo, AFIND+, and others. The experimental setup spans four datasets, two topologies, and two non-IID levels, and the mechanism for giving underutilized clients a rising probability of local training is a sensible way to address participation imbalance.\n\nThe load-bearing problem is τ in Eq. (4). Algorithm 1 lists it as an input, the text calls it a \"predefined value,\" and the evaluation never says what it was set to. This matters because the threshold determines which neighbors get aggregated, which clients receive enough votes, and how often the p-trigger fires. Without τ, the measured byte counts and energy numbers in Tables III–IV are not reproducible, and one cannot tell whether the reported gains come from the mechanism or from a tuned parameter. The paper should report τ per configuration and include a sensitivity sweep.\n\nSecond, the resource comparisons (time, energy, bytes) are only against FedAvg. The paper justifies this by saying SCAFFOLD and FedProx add overhead, but those methods appear in the performance tables, so a reader reasonably expects the resource comparison to cover them too. Measuring actual bytes and energy for all methods would settle whether S-VOTE's savings survive against stronger baselines.\n\nThird, the performance claims are selective. The abstract says \"9-17% higher accuracy in some configurations,\" and Table II shows plenty of cases where SCAFFOLD or FedProx beats S-VOTE (e.g., EMNIST alpha=0.1, CIFAR10). Standard deviations are large relative to the reported differences, and there are no significance tests. The conclusion's mention of future work on adaptive thresholding is an implicit admission that the fixed threshold is a limitation.\n\nNone of this kills the idea. The mechanism is plausible and the paper is clearly written. It is a workshop-to-conference-level contribution that needs revision, not desk rejection. I would send it to peer review, with the clear expectation that the authors disclose τ, add sensitivity analysis, and broaden the resource comparison. A serious referee will catch the same issues, but the paper is worth that referee's time.","headline":"A plausible DFL client-selection heuristic with a coherent idea, but an unreported core hyperparameter and selective baselines keep it from being reproducible as written.","tokens_in":12548,"tokens_out":2155,"would_cite":false,"duration_ms":22191,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"S-VOTE selects aggregation partners by model-weight similarity, claiming up to 21% lower communication costs, 4-6% faster convergence, 9-17% higher accuracy in some configurations, and 14-24% energy savings in decentralized federated…","keywords":["decentralized federated learning","client selection","voting mechanism","model similarity","non-IID data","communication efficiency","energy consumption"],"falsifier":"Measure the correlation between pairwise cosine similarity of model weight vectors and a direct measure of label-distribution similarity (e.g., Jensen-Shannon divergence of class proportions) across the Dirichlet(0.1) and Dirichlet(0.5) splits used in the paper; if the correlation is weak or negative in some regimes, S-VOTE would be selecting the wrong neighbors and its accuracy improvements would not replicate.","tokens_in":11621,"feed_emoji":"🗳️","tokens_out":4005,"duration_ms":38340,"temperature":0.7,"pith_summary":"This paper proposes S-VOTE, a client selection mechanism for decentralized federated learning (DFL) where no central server coordinates training. Each client compares its own model weight vector with those of its neighbors using cosine similarity, keeps only the neighbors whose similarity exceeds a threshold (the mean plus a multiple of the standard deviation), and votes for those selected peers. Clients receiving enough votes train normally; under-voted clients train only occasionally, with the probability increasing over time. The paper argues that this similarity-based voting improves model accuracy by 9-17%, speeds convergence by 4-6%, cuts communication costs by up to 21%, and reduces energy consumption by 14-24% in non-IID data settings.","feed_headline":"Similarity voting cuts decentralized FL costs by 21%","feed_subtitle":"Peers vote for neighbors whose model weights match theirs, saving energy and speeding convergence on non-IID data.","key_machinery":"The central mechanism is the combination of cosine similarity between model weight vectors (Eq. 3) with a statistical threshold selector (Eq. 4) that keeps only clients whose similarity to the local model exceeds mu + tau * sigma, where mu and sigma are the mean and standard deviation of similarities to all neighbors. A voting rule (Eq. 5) then gates local training: a client with at least Vmin = N/2 votes trains normally, while under-voted clients fall back to random training with a gradually increasing probability (Eq. 6). This machinery selects a per-client subset of aggregation partners at each round, reducing the number of model transmissions and the energy spent on communication and aggregation.","core_discovery":"S-VOTE establishes that a purely local, similarity-based voting rule can replace full-neighbor aggregation in DFL without sacrificing model quality, while substantially reducing resource usage. After an initial FedAvg phase and a short divergence phase, each client computes the cosine similarity between its own model weight vector and those of all neighbors, selects the neighbors whose similarity is at least the mean plus tau times the standard deviation of all pairwise similarities, and aggregates only the models of those selected clients. The voting step then decides which clients continue training: a client trains normally if it receives at least N/2 votes (or has at most two neighbors), otherwise it trains randomly with probability p that starts at 0.1 and increases by 0.1 each round it remains under-voted. The paper claims this mechanism enhances generalization to unseen data and stabilizes training in non-IID environments, as evidenced by experiments on MNIST, FashionMNIST, EMNIST, and CIFAR-10 with Dirichlet-split data.","pith_inferences":["Because the paper never reports the value of tau used to set the selection threshold, the exact operating point of the mechanism is underdetermined; a practitioner would need to tune tau per federation, and the claimed gains may be sensitive to this choice.","The cosine-similarity assumption could be tested directly: if two clients have similar model weights but very different label distributions (e.g., disjoint classes that happen to produce similar weight directions), S-VOTE would select the wrong aggregation partners, so the mechanism's success hinges on the empirical correlation between weight similarity and data-distribution similarity.","The energy savings reported likely depend on the communication cost model; in settings where local computation dominates over communication, the 14-24% reduction would shrink, while in bandwidth-constrained networks it might be larger.","The voting and threshold ideas could be transplanted to centralized federated learning, where a server could apply the same similarity filter to select participating clients, eliminating the need for a local voting step while retaining the communication savings."],"forward_implications":["Deploying S-VOTE in a peer-to-peer federation reduces the number of model exchanges per round, directly lowering communication bandwidth and energy consumption without requiring a central coordinator.","In highly non-IID settings (Dirichlet alpha = 0.1), S-VOTE improves F1 scores by up to 37 points over FedAvg in some configurations, suggesting that similarity-based selection mitigates the negative effects of data heterogeneity.","The conditional training rule keeps underutilized clients from dropping out entirely, allowing rare data distributions to contribute occasionally and thereby improving generalization to unseen classes.","The method's gains are consistent across fully connected and random topologies, indicating it scales to partial connectivity which is more realistic in decentralized deployments."],"supporting_citations":[{"why":"Provides the foundational similarity-based client selection method (FedeRiCo) that S-VOTE builds on.","marker":"[1]"},{"why":"Introduces a collaboration-graph and greedy-selection approach for personalized DFL, which S-VOTE compares against as a voting-based alternative.","marker":"[3]"},{"why":"Presents coreset-based neighbor selection in DFL, a competing sampling strategy that S-VOTE differentiates from by using model similarity.","marker":"[5]"},{"why":"Documents the non-IID data challenge in decentralized learning that S-VOTE aims to mitigate.","marker":"[9]"},{"why":"Surveys client selection principles and motivates the need for selection mechanisms in FL, framing why S-VOTE matters.","marker":"[13]"},{"why":"Describes the Nebula/Fedstellar framework used to run the experiments, providing the testbed for evaluating S-VOTE.","marker":"[17]"},{"why":"Supplies the energy consumption monitoring module used to measure S-VOTE's 14-24% energy savings.","marker":"[18]"}],"fun_headline_variants":["Vote similar peers to cut DFL costs by 21%","Similarity voting speeds DFL and saves energy","S-VOTE: link with similar models, pay less","Peer similarity voting trims DFL resource use","Vote on model similarity for efficient DFL"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole mechanism rests on the assumption that the cosine similarity between two clients' model weight vectors reliably reflects how similar their local data distributions are, so that picking the neighbors above the mean-plus-tau threshold actually picks the most useful aggregation partners.","fun_headline_variants_meta":{"raw":{"variants":["Vote similar peers to cut DFL costs by 21%","Similarity voting speeds DFL and saves energy","S-VOTE: link with similar models, pay less","Peer similarity voting trims DFL resource use","Vote on model similarity for efficient DFL"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000326,"raw_usage":{"total_tokens":1820,"prompt_tokens":933,"completion_tokens":887,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":549,"completion_tokens_details":{"reasoning_tokens":810}},"tokens_in":549,"tokens_out":887,"duration_ms":8831,"temperature":1.0,"reasoning_tokens":810,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T20:41:49.985992+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the correlation between pairwise cosine similarity of model weight vectors and a direct measure of label-distribution similarity (e.g., Jensen-Shannon divergence of class proportions) across the Dirichlet(0.1) and Dirichlet(0.5) splits used in the paper; if the correlation is weak or negative in some regimes, S-VOTE would be selecting the wrong neighbors and its accuracy improvements would not replicate.","supporting_citations":[{"cited_title":"Find Your Friends: Personalized Federated Learning with the Right Collaborators","cited_arxiv_id":"2210.06597","evidence_quote":"Provides the foundational similarity-based client selection method (FedeRiCo) that S-VOTE builds on."},{"cited_title":"Smart Sampling: Helping from Friendly Neighbors for Decentralized Federated Learning","cited_arxiv_id":"2407.04460","evidence_quote":"Presents coreset-based neighbor selection in DFL, a competing sampling strategy that S-VOTE differentiates from by using model similarity."},{"cited_title":"The non-iid data quagmire of decentralized ma- chine learning,","cited_arxiv_id":null,"evidence_quote":"Documents the non-IID data challenge in decentralized learning that S-VOTE aims to mitigate."},{"cited_title":"Fedstellar: A platform for decentralized federated learning,","cited_arxiv_id":null,"evidence_quote":"Describes the Nebula/Fedstellar framework used to run the experiments, providing the testbed for evaluating S-VOTE."},{"cited_title":"Design and implementation of environmental sustainability module for decentralized federated learning,","cited_arxiv_id":null,"evidence_quote":"Supplies the energy consumption monitoring module used to measure S-VOTE's 14-24% energy savings."}],"review_version":1}