{"id":"ef30a151-6520-4393-b5f4-4b53d6a99ba2","arxiv_id":"2412.11207","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"ProFe reduces decentralized federated learning communication by 40-50% through teacher-student distillation, prototype sharing, and 16-bit quantization, with slight accuracy gains in non-IID settings and about 20% more training time.","lead":"ProFe is a new algorithm that cuts communication cost in decentralized federated learning by roughly 40 to 50 percent by combining knowledge distillation, prototypes, and quantization. Smart generalists should care because communication, not compute, is often the main bottleneck when many edge devices train models together without a central server.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Equation (4) scales global prototypes down by 1/|N^j| after instance weighting, corrupting the nearest-prototype rule in non-IID settings; the implementation must be checked.","rationale":"The reader's weakest assumption identifies Eq. (4) as unverified, and my reading confirms that this is the most load-bearing technical flaw. The central claim is that ProFe reduces communication cost by 40-50% while maintaining or improving accuracy, and the prototype mechanism is one of the two main techniques (alongside KD) claimed to preserve accuracy under non-IID data. If Eq. (4) is wrong as written, the described method is not a correctly normalized aggregation, and the Euclidean prototype classifier in Eq. (5) can be severely biased for classes with different client support. This does not necessarily invalidate the empirical results if the code implements the correct weighted average, but it makes the paper's formal description unreliable and its performance claims dependent on an undocumented correction. The ambiguity about the FedAvg baseline model size is also relevant but secondary, because even an unfair baseline would not undermine the internal correctness of the proposed method as much as an incorrect core equation. I agree with the reader's conditional verdict: the paper is plausible and empirically rich, but the prototype aggregation must be verified against the code. The concrete test is straightforward and decisive: compare the released implementation with Eq. (4) and run a controlled two-client case.","tokens_in":8560,"tokens_out":6100,"duration_ms":57428,"concrete_test":"Clone the linked GitHub branch (github.com/CyberDataLab/nebula, feature/communication-optimization) and locate the prototype aggregation code. Compare the implemented formula to Eq. (4). Then construct a minimal non-IID case with two clients and two classes: client A holds only class 0 (100 samples), client B holds only class 1 (100 samples). Run one communication round and print the global prototypes C^(0) and C^(1). If their norms are half of the respective local prototype norms, the equation is literally implemented; if they equal the local prototypes, the text is a typo. Additionally, re-run the CIFAR10 non-IID 20% experiment with the corrected instance-weighted average and report the F1 curve; a shift of more than 2 points would indicate that the published performance claim depends on the prototype-scaling behavior.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The global prototype formula in Eq. (4) is internally inconsistent. It defines C^(j) = (1/|N^j|) * Σ_{i∈N^j} (|D_{i,j}| / N_j) * C_i^(j), where N_j is stated to be the total number of instances of class j across all nodes. Since Σ_{i∈N^j} |D_{i,j}| / N_j = 1, the factor 1/|N^j| is an extra divisor by the number of clients that know class j. This shrinks each global prototype by a class-dependent factor. In non-IID configurations where label sets are disjoint or partial (Section IV, non-IID 60%, 40%, 20%, Dirichlet), classes known by different numbers of clients receive different scale factors. The Euclidean nearest-prototype rule in Eq. (5) is not invariant under per-class scaling, so predictions become biased against classes known by many clients, and the MSE loss in Eq. (6) is mis-scaled per class. This is the core mechanism that the paper claims improves performance in non-IID settings, yet the formula as written cannot be used to produce a valid average prototype. The manuscript also conflates N^j (a set) and N_j (a count), so the error is not merely notational: either the released code implements a corrected weighted average and Eq. (4) is a reproducible-typo, or the code follows the paper and the reported F1 results in Fig. 2 are difficult to reconcile with the stated algorithm.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes ProFe, a decentralized federated learning algorithm that combines knowledge distillation from a local teacher to a smaller student, prototype aggregation, and 16-bit quantization to reduce communication cost. The evaluation uses MNIST, CIFAR10, and CIFAR100 with 20 nodes and five data distributions, measuring average node F1, network bytes sent/received, and wall-clock training time against FedAvg, FedProto, FedGPD, and FML. The headline claims are a 40-50% communication reduction with maintained or improved model performance and approximately 19% training-time overhead.","tokens_in":8923,"tokens_out":5506,"duration_ms":51947,"significance":"If the implementation matches the description, the paper offers a practical communication-efficiency recipe for decentralized federated learning and includes direct communication measurements over real Docker-based node-to-node traffic, which is a strength. The comparison across several literature baselines and the public code availability are also positive features. However, two load-bearing issues must be resolved before the central claim can be accepted: the global prototype aggregation in Eq. (4) appears improperly normalized, and the FedAvg baseline architecture is unspecified, making the reported communication savings difficult to attribute to the proposed algorithm.","major_comments":[{"comment":"The global prototype formula is not a correctly weighted average. The weights |D_{i,j}|/N_j sum to 1 by definition, so the additional factor 1/|N^j| introduces a class-dependent scale factor equal to 1 divided by the number of clients that know class j. Because Eq. (5) compares Euclidean distances to these prototypes and Eq. (6) anchors the representation loss to them, the scale factor biases predictions against classes known by many clients in the partial-label non-IID settings of Section IV. Please correct Eq. (4) to a properly normalized weighted average, or show from the released code that the implemented aggregation differs from the formula as written. The notation N^j versus N_j is also inconsistent, so the formula cannot be disambiguated by notation alone.","section":"Section III-B, Eq. (4)"},{"comment":"The FedAvg baseline architecture and communication details are not specified. ProFe transmits a smaller student model (half-channel CNN for MNIST, ResNet8 for CIFAR10, ResNet18 for CIFAR100) under 16-bit quantization, while the FedAvg model size, architecture, and bit width are not given. The reported 40-50% communication reduction could therefore reflect the smaller model and quantization choices rather than the proposed distillation and prototype mechanism. The paper should report the FedAvg architecture, parameter count, bytes per update, and bit width so that the comparison isolates the algorithmic contribution.","section":"Section IV-B, Table II"},{"comment":"The approximately 19% training-time overhead for CIFAR10 and CIFAR100 is reported as a single aggregate value, with no breakdown among teacher training, student training, prototype computation, and communication. Since ProFe trains two networks per node, the near-zero overhead on MNIST and the exact 19% figures across datasets are hard to interpret without such a breakdown. Adding a time decomposition, or at least separate measurements for the main components, would support the communication-versus-training-time trade-off claimed in the paper.","section":"Section IV-B, Table III"},{"comment":"The hyperparameters controlling the losses in Eqs. (2), (8), and (9) are not reported: the initial distillation weight beta, the decay limit beta_limit, the prototype and distillation weights beta_s and alpha_s, the temperature T, and the quantization step Delta of Section III-D. Because the performance and communication trade-off depends directly on these values, the experimental results are not reproducible without them. Please add a hyperparameter table and, ideally, a brief sensitivity study.","section":"Section IV"}],"minor_comments":[{"comment":"The abstract states a communication reduction of 'up to ~40-50%' while the introduction states '~50–60%'; the two numbers should be reconciled.","section":"Abstract and Section I"},{"comment":"The text says validation used the Nebula framework and cites reference [12], but [12] is the Fedstellar paper and the NEBULA repository is reference [13]; the citation should be corrected.","section":"Section IV"},{"comment":"The F1 curves are shown without error bars or confidence bands, although Table II reports standard deviations; adding uncertainty information would strengthen the claim that performance is maintained.","section":"Figure 2"},{"comment":"Phrases such as '+95% F1-Score' are likely proportions (0.95), but the percentage notation is ambiguous; please use standard decimal notation for F1 values.","section":"Section IV-A"},{"comment":"The quantization function is described, but the manuscript does not state how Delta is chosen per tensor or whether gradients and prototypes are quantized in the same way; please clarify the quantization scope.","section":"Section III-D"}],"recommendation":"major_revision","confidential_remarks":"The paper is an empirical systems contribution with direct communication measurements, which fits an applied journal. The main risk is that the comparison is not apples-to-apples because the FedAvg baseline is underspecified, and the prototype aggregation formula appears to contain a normalization error. If the authors can correct or justify Eq. (4) and fully specify the baseline, the central claims may become defensible; I would not recommend rejection based on the formula alone because the code is available for verification."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The genuinely new thing here is the combination: knowledge distillation, prototype learning, and quantization in a decentralized topology, with real Docker-based communications measured directly rather than simulated. That measurement setup is a real strength. The F1 comparisons across MNIST, CIFAR10, and CIFAR100 with five data distributions are broad, and the claim is honestly scope-limited: roughly 40–50% communication savings, about 20% training time overhead, accuracy roughly on par with FedAvg and better than FedProto and FML in non-IID settings. I believe the empirical work is mostly solid, and the related work is cited fairly.\n\nThe soft spots are not fatal, but they are real. First, Eq. (4) is wrong as written. The global prototype is defined as (1/|N^j|) * sum_{i in N^j} (|D_{i,j}|/N_j) * C_i^(j). Since sum_i |D_{i,j}|/N_j = 1, the extra 1/|N^j| factor scales each class prototype down by the number of clients that know that class. The nearest-prototype rule in Eq. (5) and the MSE loss in Eq. (6) are not invariant to per-class scaling, so in non-IID settings with partial label overlap, this biases predictions. Either the code implements a corrected weighted average and Eq. (4) is a typo, or the code follows the paper and the reported F1 curves are suspect. This needs to be checked before I would trust the central non-IID claim.\n\nSecond, the FedAvg baseline architecture is not specified. If FedAvg transmits teacher-size models while ProFe transmits student models, part of the 40–50% savings is just model-size reduction, not the algorithm. That has to be stated. Third, the abstract says ≈40–50% while the introduction says ≈50–60%; minor but embarrassing. Fourth, key hyperparameters (initial KD weight, beta_limit, alpha_s, beta_s, temperature) are missing, so reproduction is impossible without the code. The communication table only shows IID, but the authors give a reasonable justification that the proportional savings are distribution-independent; I would still want one non-IID row.\n\nThe citation pattern is clean and the code is linked. This is an engineering composition paper, not a conceptual leap, and it does not overclaim. I would send it to peer review with a request for major revision: fix Eq. (4), specify the FedAvg model, add the missing hyperparameters, and reconcile the savings numbers. If the code fixes the prototype formula, this is a useful contribution for DFL practitioners. If it does not, the paper's main non-IID results are in doubt.\n\nFor a reading group: maybe, mainly to discuss the prototype aggregation bug. I would not cite it in my own work until the code and equation are checked.","headline":"ProFe is a useful empirical composition of KD, prototypes, and quantization for DFL, but the prototype aggregation in Eq. (4) as written can't be right, and the FedAvg baseline architecture is unspecified; check the code before relying on the 40–50% number.","tokens_in":9447,"tokens_out":3054,"would_cite":false,"duration_ms":30965,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"ProFe claims a 40–50% communication cut in decentralized federated learning by exchanging distilled student models and class prototypes instead of full models.","keywords":["Decentralized Federated Learning","Communication Optimization","Knowledge Distillation","Prototype Learning","Quantization","Non-IID data","Model compression"],"falsifier":"A direct test is to rerun the CIFAR10 non-IID 20% experiment with clients whose local label sets are disjoint, and compare nearest-prototype accuracy against predictions made by the student's own classifier head. If the nearest-prototype rule does not correctly classify classes a client never saw, the prototype mechanism's claimed benefit fails; a second check is to recompute global prototypes with a properly count-normalized average and see whether accuracy changes.","tokens_in":8435,"feed_emoji":"📉","tokens_out":6879,"duration_ms":57779,"temperature":0.7,"pith_summary":"ProFe is a decentralized federated learning algorithm that combines three techniques — knowledge distillation, prototype learning, and 16-bit quantization — to shrink the amount of data nodes exchange each round. The paper's central claim is that this combination cuts communication costs by roughly 40–50% compared with FedAvg and other literature baselines on MNIST, CIFAR10, and CIFAR100, while maintaining or improving model F1-score, especially under non-IID data splits. The price is an 18–20% increase in training time on the harder datasets, which the authors frame as a deliberate trade-off for bandwidth-constrained settings. If true, the result matters because communication overhead is the main bottleneck in decentralized federated learning, and ProFe would offer a way to keep accuracy without a central server.","feed_headline":"Federated learning traffic cut 40–50% with ProFe","feed_subtitle":"A decentralized scheme keeps accuracy up by distilling big teachers into small students and sharing class prototypes.","key_machinery":"The central object is the ProFe training loop: each node keeps a large teacher model and a small student model, and the student is trained with cross-entropy, a prototype-matching mean-squared-error loss, a knowledge-distillation KL divergence, and an additional mean-squared-error loss between the student's and teacher's intermediate representations. The teacher is trained with cross-entropy and the prototype-matching loss. The artifacts exchanged between nodes are the student model and per-class prototypes — averages of intermediate representations taken before the final prediction layer — and both are quantized to 16 bits before transmission to halve the byte count. A schedule called Professor Importance Decay halves the distillation weight after each federated round and zeroes it below a threshold, so the student eventually stops imitating the teacher.","core_discovery":"ProFe's central discovery is that a decentralized federated system can exchange only small student models and class prototypes instead of full models, and still match or beat a full-model baseline. On MNIST, CIFAR10, and CIFAR100, with 20 nodes in a fully connected topology, average bytes sent and received drop by roughly 40–50% relative to FedAvg, while average node F1 stays within the same range or improves in non-IID settings; the cost is an 18–20% rise in training time on the harder datasets. The mechanism is a joint loss that trains a small student to imitate a large local teacher's softened logits and intermediate representations, while also pulling the student's representations toward global class prototypes computed from other nodes. Classification during testing switches to the nearest global prototype, which is how the system claims to handle classes a client never observed locally.","pith_inferences":["Implicit in the design but not tested: the 16-bit quantization alone accounts for the 50% part of the saving, so lower-precision quantization (8-bit) or quantization-aware training is a natural way to push the reduction further.","The paper reports communication costs only under IID; it assumes the ratio of savings transfers to non-IID settings because byte counts are distribution-independent. A direct measurement under the Dirichlet splits used for accuracy would test that assumption.","Because Eq. (4) divides the count-weighted prototype sum by the number of clients rather than by the total count, re-normalizing it could change the global prototypes; re-running the experiments with the corrected average would isolate whether the current formula helps or hurts the reported accuracy.","The teacher–student representation-matching term in Eq. (8) is an extra constraint; ablating it would show how much of ProFe's accuracy retention comes from distillation versus prototype regularization."],"forward_implications":["In a fully connected decentralized topology, replacing full model exchange with a distilled student plus quantized prototypes cuts average bytes sent and received by roughly 40–50% relative to FedAvg on MNIST, CIFAR10, and CIFAR100.","Model quality, measured by average node F1-score, stays close to FedAvg and beats FedProto and FML in most non-IID settings, so the communication saving does not come at the cost of accuracy in the tested configurations.","Because only the student model and class prototypes are exchanged, the method supports heterogeneous local model sizes; each node can keep its large teacher locally for distillation.","The 16-bit quantization step halves the byte size of every transmitted model and prototype, which is the direct source of the 50% part of the saving.","The added 18–20% training time on CIFAR10 and CIFAR100 is the cost of running both teacher and student plus prototype losses, a trade-off that matters when bandwidth is scarcer than compute."],"supporting_citations":[{"why":"Provides the FedAvg baseline that defines the communication cost and accuracy reference point for all comparisons.","marker":"[1]"},{"why":"Supplies the teacher–student knowledge distillation scheme that ProFe adapts to the decentralized setting.","marker":"[4]"},{"why":"Federated mutual learning baseline that ProFe compares against in the experiments.","marker":"[8]"},{"why":"Defines the prototype-sharing and aggregation approach that ProFe builds on and also uses as a literature baseline.","marker":"[9]"},{"why":"Combines prototypes and distillation and serves as another baseline for accuracy and communication comparisons.","marker":"[10]"},{"why":"Demonstrates pruning and quantization for communication reduction, motivating the quantization component of ProFe.","marker":"[11]"},{"why":"Provides the simulation environment used to run the decentralized experiments with real communications between nodes.","marker":"[12]"}],"fun_headline_variants":["ProFe cuts decentralized federated communication by 40-50%","ProFe: 40-50% less traffic for decentralized federated learning","ProFe uses distillation and prototypes to cut federated bandwidth","Decentralized federated learning: ProFe trims communication 40-50%","ProFe: distill, quantize, share prototypes - cut traffic 40-50%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method's correctness rests on the assumption that the class summaries (prototypes) exchanged between nodes remain accurate when averaged by the paper's formula, and that classifying a point by the closest summary works even for classes a node never saw in its local data.","fun_headline_variants_meta":{"raw":{"variants":["ProFe cuts decentralized federated communication by 40-50%","ProFe: 40-50% less traffic for decentralized federated learning","ProFe uses distillation and prototypes to cut federated bandwidth","Decentralized federated learning: ProFe trims communication 40-50%","ProFe: distill, quantize, share prototypes - cut traffic 40-50%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000929,"raw_usage":{"total_tokens":3950,"prompt_tokens":888,"completion_tokens":3062,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":504,"completion_tokens_details":{"reasoning_tokens":2962}},"tokens_in":504,"tokens_out":3062,"duration_ms":21048,"temperature":1.0,"reasoning_tokens":2962,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T15:10:58.239763+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A direct test is to rerun the CIFAR10 non-IID 20% experiment with clients whose local label sets are disjoint, and compare nearest-prototype accuracy against predictions made by the student's own classifier head. If the nearest-prototype rule does not correctly classify classes a client never saw, the prototype mechanism's claimed benefit fails; a second check is to recompute global prototypes with a properly count-normalized average and see whether accuracy changes.","supporting_citations":[{"cited_title":"Communication-efficient learning of deep networks from decentralized data,","cited_arxiv_id":null,"evidence_quote":"Provides the FedAvg baseline that defines the communication cost and accuracy reference point for all comparisons."},{"cited_title":"Communication-efficient federated learning via knowl- edge distillation,","cited_arxiv_id":null,"evidence_quote":"Supplies the teacher–student knowledge distillation scheme that ProFe adapts to the decentralized setting."},{"cited_title":"Federated mutual learning: a collaborative machine learning method for heterogeneous data, mod- els, and objectives,","cited_arxiv_id":null,"evidence_quote":"Federated mutual learning baseline that ProFe compares against in the experiments."},{"cited_title":"Fedproto: Federated prototype learning across heterogeneous clients,","cited_arxiv_id":null,"evidence_quote":"Defines the prototype-sharing and aggregation approach that ProFe builds on and also uses as a literature baseline."},{"cited_title":"Global prototype distillation for hetero- geneous federated learning,","cited_arxiv_id":null,"evidence_quote":"Combines prototypes and distillation and serves as another baseline for accuracy and communication comparisons."},{"cited_title":"Ac- celerating federated learning for iot in big data analytics with pruning, quantization and selective updating,","cited_arxiv_id":null,"evidence_quote":"Demonstrates pruning and quantization for communication reduction, motivating the quantization component of ProFe."}],"review_version":1}