{"id":"f0dcbbcd-978c-4de1-bee5-67f08b4e511b","arxiv_id":"2504.17520","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"MCE-PL lets each agent learn a personalized binary mask on a shared fixed random network, transmitting only masks, and the paper's theoretical DSLTH proof is intended to justify this design.","lead":"This paper proposes MCE-PL, a decentralized learning algorithm in which all agents share one fixed random neural network and exchange only personalized binary pruning masks, cutting communication to one bit per parameter. The authors also formulate and attempt to prove a distributed strong lottery ticket hypothesis stating that such masks can extract high-accuracy local models from the shared random network.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 1's heterogeneity lower bound (Eq. 26) does not follow from the reverse triangle inequality; the correct bound is max(0, alpha_l - epsilon1 - epsilon2), so the proof of DSLTH's personalized-heterogeneity claim collapses.","rationale":"The reader's verdict is REJECT, and this stress-test does not move it. The strongest claim, DSLTH, has two components: that locally pruned subnetworks perform well, and that they are heterogeneous across agents. The first component is plausibly supported by Lemma 1 if its overparameterization and membership assumptions hold. The second, heterogeneity, is supported only by Eq. (26), which is derived by an invalid combination of upper bounds on a - b and a lower bound on a. The reverse triangle inequality gives a different, weaker, and sometimes trivial bound. Since this theorem is advertised as the theoretical foundation of MCE-PL, the central theoretical claim is unproven. The paper's own scope note that Theorem 1 excludes system heterogeneity and general architectures further narrows the proof, but the Eq. (26) gap is already decisive. The empirical results are suggestive, and the availability of code is a positive, but single-run accuracy numbers with learning rates selected by test performance and an unspecified Fil threshold are not enough to carry the DSLTH claim. The concern is technical and specific, not an ad hominem or a disagreement with the field consensus; it is an internal derivation error in the proof.","tokens_in":992,"tokens_out":867,"duration_ms":47607,"concrete_test":"Independently re-derive Eq. (26) from (A.2)-(A.5). Using ||a - b|| <= s = epsilon1 + epsilon2 and ||a|| >= alpha_l, the reverse triangle inequality yields inf ||b|| >= max(0, alpha_l - s), not the three-term minimum of absolute values stated in Eq. (26). Instantiate alpha_l = 1, alpha_u = 10, epsilon1 = epsilon2 = 1: the correct bound is 0 while Eq. (26) asserts 1, exposing the invalid step. If the re-derivation instead reproduces Eq. (26), the concern is settled; otherwise Theorem 1 must be revised and the DSLTH heterogeneity claim currently lacks proof.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing flaw is the derivation of the heterogeneity lower bound in Theorem 1, Eq. (26), from Assumption 3 in Appendix A. With a = f1 - f2 and b = gm1 - gm2, the proof establishes sup ||a - b|| <= epsilon1 + epsilon2 and inf ||a|| >= alpha_l. The reverse triangle inequality gives, for every x, ||b(x)|| >= ||a(x)|| - ||a(x) - b(x)|| >= alpha_l - (epsilon1 + epsilon2), so inf_x ||b(x)|| >= max(0, alpha_l - epsilon1 - epsilon2). Eq. (26)'s min{|epsilon1 + epsilon2 - alpha_l|, |epsilon1 + epsilon2 - alpha_u|, |alpha_l|} is not implied; for example, alpha_l = 1, alpha_u = 10, epsilon1 = epsilon2 = 1 yields a claimed lower bound of 1 while the correct lower bound is 0. This matters because the heterogeneous-subnetworks half of DSLTH rests entirely on this inequality; without it, Theorem 1 only shows both agents can find accurate pruned networks, not that personalized heterogeneous masks exist, let alone that MCE-PL's specific update rules find them. Assumption 2, requiring f1, f2 in F with ||v_l||_1 <= 1, is also unsupported for trained CNNs, but the Eq. (26) gap alone invalidates the proof.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MCE-PL, a decentralized learning method in which agents keep a shared randomly initialized weight tensor fixed and communicate only personalized binary pruning masks. A real-valued mask tensor is updated by backpropagation, thresholded to a binary mask, and aggregated through an intermediate aggregation tensor with a personalized fine-tuning step. The authors state a distributed strong lottery ticket hypothesis (DSLTH) and claim a proof under a restricted CNN setting (Theorem 1), based on applying a centralized SLTH lemma to each of two agents and then bounding the correlation and heterogeneity between the resulting pruned subnetworks via Assumptions 2 and 3. Experiments on CIFAR-10 with AlexNet compare accuracy and communication cost under data and node heterogeneity with several baselines.","tokens_in":20692,"tokens_out":12059,"duration_ms":123820,"significance":"The paper addresses a real and timely problem, personalized decentralized learning under both statistical and system heterogeneity, and the idea of transmitting only binary masks is well motivated and clearly described. The authors provide code, compare against multiple baselines, and are transparent about some limitations, including the fact that Theorem 1 does not cover system heterogeneity. If DSLTH could be established, it would be a valuable distributed analogue of SLTH, and the intermediate-aggregation-tensor design is a sensible engineering contribution. However, the theoretical result as stated is not valid: the assumptions are mutually inconsistent for the stated no-bias CNN class, the derivation of Eq. (26) in the proof is incorrect, and the theorem is an existence statement not connected to the actual MCE-PL updates. The empirical study may support a systems-oriented paper, but not the theoretical claims made here.","major_comments":[{"comment":"Assumptions 2 and 3 are mutually inconsistent for the function class F defined in Section IV. The networks in F are defined in (20) as no-bias convolutional ReLU networks, so for the zero input 0 in the domain [0,1]^{D x D x O0}, every f in F satisfies f(0)=0. Hence for any f1,f2 in F, inf_{x in X} ||f1(x)-f2(x)||_max = 0. Assumption 3 can therefore hold only with alpha_l = 0; with alpha_l = 0 the right-hand side of (26) collapses to 0, so Theorem 1 cannot establish any positive heterogeneity between the pruned subnetworks. The proof's use of a positive alpha_l is not justified and contradicts Assumption 2.","section":"Section IV, Assumptions 2 and 3"},{"comment":"Even if a positive alpha_l were granted, the displayed lower bound in (26) does not follow from the stated inequalities. From sup ||(f1-f2)-(gm1-gm2)|| <= epsilon1+epsilon2 and inf ||f1-f2|| >= alpha_l, the reverse triangle inequality gives pointwise ||gm1-gm2|| >= ||f1-f2|| - ||(f1-f2)-(gm1-gm2)||, and hence inf ||gm1-gm2|| >= max(0, alpha_l - epsilon1 - epsilon2). The expression min{|epsilon1+epsilon2-alpha_l|, |epsilon1+epsilon2-alpha_u|, |alpha_l|} is not implied. For example, with alpha_l=0.1, alpha_u=1, and epsilon1+epsilon2=0.15, the paper's bound is 0.05 while the valid bound is 0. Since the heterogeneity half of DSLTH rests entirely on (26), this gap invalidates the proof as written.","section":"Appendix A, Eq. (26)"},{"comment":"Theorem 1 is an existence result and does not analyze the MCE-PL algorithm. The theorem guarantees, via the external SLTH lemma, that some binary masks approximate f1 and f2; it says nothing about whether the specific gradient-based update (14), the aggregation rules (16)-(19), or the personalized fine-tuning step (17) converge to such masks or preserve their guarantees. The claim that Theorem 1 establishes the theoretical foundation of MCE-PL therefore overstates what is proved. The authors acknowledge that node heterogeneity is not covered by the theorem, but the title and abstract nevertheless present a joint theory for data and node heterogeneity.","section":"Sections III-C and IV, Theorem 1 versus MCE-PL"},{"comment":"Assumption 2 is not justified for the trained models considered in this work. Requiring f1,f2 in F means that every convolution weight tensor has entries in [-1,1] and ||v_l||_1 <= 1; trained AlexNet models in the experiments are not shown to satisfy this, and scaling weights into F changes the function because the network is nonlinear. The sentence 'Assumption 2 can be satisfied through proper initialization and training process' is asserted without a construction or proof, so Lemma 1 cannot be invoked for the models of interest.","section":"Section IV, Assumption 2"}],"minor_comments":[{"comment":"The notation nabla f_i / nabla v_i is nonstandard and dimensionally unclear; gradients should be written as gradients with respect to the relevant tensors, and the replacement of nabla m_i / nabla z_i by sign(z_i) mixes tensor calculus with a heuristic approximation that should be explained more carefully.","section":"Section II-B, Eq. (4)"},{"comment":"The sentence 'the analytical results in IV cannot be practically applied here' is an honest caveat, but it also means the experiments do not test the assumptions of Theorem 1; this should be stated more prominently in the main text, not only in the simulation section.","section":"Section V-A"},{"comment":"There are several typographical and reference issues: 'We defer to future works' should be 'We refer to future work', and reference [14] appears to duplicate [13] with a different first author.","section":"Throughout"},{"comment":"The numerical comparisons are reported without standard deviations or multiple seeds; given the small differences between some methods, the conclusions would be more robust if the experiments reported variability across runs.","section":"Section V-B, Tables I and II"}],"recommendation":"reject","confidential_remarks":"The main advertised contribution is the theoretical DSLTH proof, and that proof has a load-bearing error: the derived heterogeneity lower bound is invalid, and Assumptions 2 and 3 are inconsistent for the no-bias CNN class. Because the theory is the stated foundation of the method, I cannot recommend acceptance. If the authors resubmit an empirical systems paper that drops or substantially repositions the theoretical claims, the mask-aggregation idea and the communication-cost comparisons might be publishable. The availability of code is a positive feature."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The thing to know about this paper is that the algorithm is worth a look, but the theory does not hold up as written. MCE-PL fixes a shared random network and has each agent learn only a personalized binary mask, communicating masks rather than weights. That is a legitimate extension of FedMask, LotteryFL, and Dis-PFL, and the aggregation tensor plus per-iteration fine-tuning is a sensible response to the node-heterogeneity problem. The experiments cover data and node heterogeneity, compare against sensible baselines, and show real communication savings; the authors also ship code. Credit where due: the empirical story is the strongest part of the paper.\n\nThe soft spot is Theorem 1. The heterogeneity lower bound in Eq. (26) does not follow from the stated inequalities. From the reverse triangle inequality you get inf ||gm1 - gm2|| >= max(0, alpha_l - eps1 - eps2), not the min-of-absolute-values expression printed. With alpha_l = 1, eps1 = eps2 = 1, the paper claims a lower bound of 1 while the actual bound is 0. This is not a small typo; it is the only step in the theorem that establishes that the two pruned subnetworks are actually different. Without it, DSLTH reduces to two independent applications of the SLTH plus an upper bound on their distance, and the name 'distributed strong lottery ticket hypothesis' overclaims what is proven. Assumptions 2 and 3 are also asserted for trained CNNs without evidence, so even the per-node approximation part rests on shaky ground.\n\nThe experiments are useful but not bulletproof: learning rates are selected using test accuracy, the Fil threshold is unspecified, and results appear to be single runs without variance or multiple seeds. These are fixable issues, but they make the empirical claims less solid than the tables suggest.\n\nMy take: the paper deserves a serious referee, but not acceptance in its current form. The algorithm is plausible and the empirical direction is valuable. The authors should either repair or remove the flawed lower bound—if the theory stays, it needs a correct proof, and if it goes, the paper can still stand as an empirical methods contribution. They also need to tighten the experiments. I would send it to review with the expectation of major revision, and I would not cite the DSLTH theorem until it is fixed.","headline":"The MCE-PL algorithm is a plausible and useful extension of mask-based personalized FL, but the proof of its central DSLTH theorem contains a real inequality error that leaves the heterogeneity claim unproven.","tokens_in":21184,"tokens_out":2744,"would_cite":false,"duration_ms":27159,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that heterogeneous subnetworks inside one randomly initialized network can fit each agent's local data, so decentralized agents need exchange only binary masks.","keywords":["distributed strong lottery ticket hypothesis","personalized learning","data heterogeneity","node heterogeneity","binary mask pruning","communication efficiency","decentralized learning","structured sparsity"],"falsifier":"Train two AlexNet agents on disjoint CIFAR-10 label subsets, fix a random shared initialization $w$, and run MCE-PL's mask search at retention ratios 0.1, 0.3, and 0.5; also record the trained weights' layerwise $\\ell^1$ norms and the sup-norm output distance between $f_1$ and $f_2$ on a shared test set. If, for any agent, no binary mask reaches the fully-trained accuracy within a chosen tolerance, or if $\\|v_l\\|_1 > 1$ or the output distance violates the assumed $\\alpha$-bounds, then the DSLTH as stated is not supported in that setting.","tokens_in":20055,"feed_emoji":"🎟️","tokens_out":9003,"duration_ms":78276,"temperature":0.7,"pith_summary":"The paper proposes a distributed strong lottery ticket hypothesis: in a network of agents with different data distributions and different computational capacities, one randomly initialized over-parameterized neural network contains a separate sparse subnetwork for each agent that fits that agent's local data well. If the hypothesis holds, every agent can keep the same global real-valued weights $w$ fixed and personalize only by learning a binary mask $m_i$, with the local model written as the Hadamard product $v_i = w \\odot m_i$. The authors build MCE-PL on this foundation: agents update real-valued mask tensors by backpropagation, apply group-sparsity regularization and a filter pruning rule to obtain structured binary masks, aggregate neighboring masks through an intermediate aggregation tensor, and add a personalized fine-tuning step. On CIFAR-10 with AlexNet, the method matches or beats baseline algorithms (decentralized SGD, FedProx, LotteryFL, Dis-PFL) while transmitting far fewer bits. The theoretical proof of the hypothesis is narrower than the general claim, covering only pairs of nodes and a restricted convolutional architecture under assumptions about the trained models.","feed_headline":"Agents can personalize one shared network by trading only 1-bit masks","feed_subtitle":"Each agent fits its own binary mask over fixed random weights, so updates can be 1-bit masks.","key_machinery":"The load-bearing object is the shared random over-parameterized CNN $g$ with fixed weights $w$, together with the mask parameterization $v_i = w \\odot m_i$, where $\\odot$ is the Hadamard product and $m_i$ is a per-agent binary tensor. Because binary tensors are not directly differentiable, each agent maintains a real-valued mask tensor $z_i$, updates it with a sign-based gradient approximation, and thresholds it layerwise by the agent's retention ratio $r_i$ to produce $m_i$. Structured sparsity comes from a group-Lasso penalty on output filters plus an extra rule that zeros any filter with too few nonzero entries. The aggregation mechanism is an intermediate tensor $y_i = z_i + \\mathrm{mean}[\\mathrm{abs}(\\tilde{z}_i)]\\, \\mathrm{sign}(\\tilde{z}_i) \\odot \\frac{1}{|\\mathcal{N}_i|}\\sum_{j\\in\\mathcal{N}_i} m_j$, which injects neighbor mask information into the local real-valued tensor at an adaptively chosen amplitude; Theorem 1 is obtained by instantiating Lemma 1, the SLTH for CNNs, separately for each node and combining the approximation and triangle-inequality bounds.","core_discovery":"The central claim, DSLTH, is stated informally in Section III-A and proved as Theorem 1 for an arbitrary pair of independently trained local models $f_1$ and $f_2$. Theorem 1 shows that with probability at least $(1-\\delta)^2$ there exist binary mask tensors $m_1$ and $m_2$ such that the pruned versions of one random $2L$-layer CNN $g$ approximate $f_1$ and $f_2$ with accuracy $\\varepsilon_1$ and $\\varepsilon_2$, respectively; moreover, the two pruned networks' outputs are correlated (distance at most $\\varepsilon_1 + \\varepsilon_2 + \\alpha_u$) yet still heterogeneous (distance at least $\\min\\{|\\varepsilon_1 + \\varepsilon_2 - \\alpha_l|, |\\varepsilon_1 + \\varepsilon_2 - \\alpha_u|, |\\alpha_l|\\}$). The quantities $\\alpha_u$ and $\\alpha_l$ come from Assumption 3, which bounds the output distance of the two trained models. The authors read these bounds as establishing that personalization can be achieved purely by choosing masks in one shared random network, with inter-agent similarity and local distinctiveness both guaranteed. They also state explicitly that the theorem does not cover system heterogeneity and applies only to a restricted CNN form without pooling, stride, or bias.","pith_inferences":["The pairwise proof structure suggests a natural extension to arbitrary graphs: if each edge is treated as a pair $(i,j)$, the correlation bound (25) is exactly the quantity an averaging-based consensus analysis would need, so DSLTH may be convertible into a convergence guarantee for mask aggregation.","The weakest spot is measurable: for a trained AlexNet on CIFAR-10 one can compute the layerwise $\\ell^1$ norms $\\|v_l\\|_1$ and the output-distance bounds $\\alpha_u, \\alpha_l$; if these contradict Assumptions 2 and 3, Theorem 1's hypotheses fail in the paper's own experimental setting even if the empirical DSLTH holds.","Mask-based updates use only the sign of $z_i$ for gradient approximation; one could test whether including gradient magnitude or a straight-through estimator with temperature changes mask quality and convergence, which the paper does not explore.","If the hypothesis extends to heterogeneous depths and dynamically estimated retention ratios, the framework could directly address the open problems the paper names: agents with different network depths and unknown capacity constraints."],"forward_implications":["Communication per round drops to one bit per retained parameter: agents exchange binary masks only, not 32-bit floats, so total cost is set by the sparsity ratios $r_i$ and the graph degree.","Node heterogeneity is expressed as differing retention ratios $r_i$, so each agent's model size scales with its own capacity instead of forcing all agents to a common architecture.","The aggregation-tensor step avoids the intersection-fusion failure mode in which the shared model collapses toward the most pruned agent, preserving the diversity of retained filters across agents.","Personalized fine-tuning each iteration steers the aggregated mask toward the local data distribution, which the experiments link to faster convergence and higher accuracy than independent mask training.","Because Theorem 1 gives both an upper and a lower bound on the distance between pruned subnetworks, it provides quantitative guarantees that the learned personalized models remain correlated yet distinct."],"supporting_citations":[{"why":"Supplies Lemma 1, the SLTH for CNNs, from which Theorem 1's mask-existence and approximation bounds are derived.","marker":"[41]"},{"why":"Introduces the FedMask idea of learning personalized binary masks while keeping real-valued parameters fixed; MCE-PL adopts and extends this core mechanism.","marker":"[33]"},{"why":"Provides the observation that mask entries indicate which weights would move toward zero, motivating binary-mask updating and aggregation.","marker":"[36]"},{"why":"LotteryFL, the main lottery-ticket-based personalized FL baseline against which MCE-PL is tested.","marker":"[15]"},{"why":"Dis-PFL, the decentralized sparse-training baseline that transmits both real-valued parameters and masks, compared for accuracy and communication cost.","marker":"[24]"},{"why":"Proves the lottery ticket hypothesis for dense networks by pruning, part of the lineage the DSLTH extends to distributed settings.","marker":"[37]"}],"fun_headline_variants":["DSLTH: one net, many masks, zero weight updates","Prune to personalize: 1-bit masks on fixed weights","Masks only: each agent prunes a shared net for its own task","One fixed net, personalized pruning, 1-bit comms"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The proof's load-bearing premise is that the two fully trained personal models $f_1$ and $f_2$ lie in the restricted class $\\mathcal{F}$ with $\\|v_l\\|_1 \\le 1$ and that their output distance is bounded by $\\alpha_u$ and $\\alpha_l$ (Assumptions 2 and 3); no evidence is given that realistic trained CNNs satisfy these, so if they fail, Theorem 1 — and with it the theoretical foundation for MCE-PL — does not apply.","fun_headline_variants_meta":{"raw":{"variants":["DSLTH: one net, many masks, zero weight updates","Prune to personalize: 1-bit masks on fixed weights","Masks only: each agent prunes a shared net for its own task","One fixed net, personalized pruning, 1-bit comms"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001017,"raw_usage":{"total_tokens":4327,"prompt_tokens":1016,"completion_tokens":3311,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":632,"completion_tokens_details":{"reasoning_tokens":3236}},"tokens_in":632,"tokens_out":3311,"duration_ms":22605,"temperature":1.0,"reasoning_tokens":3236,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T10:39:11.791942+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train two AlexNet agents on disjoint CIFAR-10 label subsets, fix a random shared initialization $w$, and run MCE-PL's mask search at retention ratios 0.1, 0.3, and 0.5; also record the trained weights' layerwise $\\ell^1$ norms and the sup-norm output distance between $f_1$ and $f_2$ on a shared test set. If, for any agent, no binary mask reaches the fully-trained accuracy within a chosen tolerance, or if $\\|v_l\\|_1 > 1$ or the output distance violates the assumed $\\alpha$-bounds, then the DSLTH as stated is not supported in that setting.","supporting_citations":[{"cited_title":"Proving the stro ng lottery ticket hypothesis for convolutional neural networ ks,","cited_arxiv_id":null,"evidence_quote":"Supplies Lemma 1, the SLTH for CNNs, from which Theorem 1's mask-existence and approximation bounds are derived."},{"cited_title":"Fedmas k: Joint computation and communication-efficient personaliz ed fed- erated learning via heterogeneous masking,","cited_arxiv_id":null,"evidence_quote":"Introduces the FedMask idea of learning personalized binary masks while keeping real-valued parameters fixed; MCE-PL adopts and extends this core mechanism."},{"cited_title":"Deconstructing lottery tickets: Zeros, signs, and the supermask,","cited_arxiv_id":null,"evidence_quote":"Provides the observation that mask entries indicate which weights would move toward zero, motivating binary-mask updating and aggregation."},{"cited_title":"Dispfl: Towards communication-efficient personalized federated learning via de- centralized sparse training,","cited_arxiv_id":null,"evidence_quote":"Dis-PFL, the decentralized sparse-training baseline that transmits both real-valued parameters and masks, compared for accuracy and communication cost."},{"cited_title":"Proving the lottery ticket hypothesis: Pruning is all you n eed,","cited_arxiv_id":null,"evidence_quote":"Proves the lottery ticket hypothesis for dense networks by pruning, part of the lineage the DSLTH extends to distributed settings."}],"review_version":1}