{"id":"600e674f-ac7e-42f0-aac0-c016e22b6f2b","arxiv_id":"2505.06651","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Dyn-D2P dynamically adjusts DP noise and gradient clipping in decentralized learning, with a 1/sqrt(n) utility rate on top of an unquantified clipping bias.","lead":"This paper proposes Dyn-D2P, a decentralized learning method that shrinks the differential-privacy noise over training by decaying the gradient clipping bound and growing the per-step privacy budget under a fixed total budget. It reports accuracy gains over fixed-noise baselines and gives a utility bound that scales as one over the square root of the number of nodes, aside from a clipping-bias term.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Proposition 4's privacy bound uses e^{μ_k^2}−1 < 2μ_k^2 without stating or verifying μ_k≤1; for the exponential schedules of Corollary 1 this condition can fail, leaving the DP guarantee and the utility bound built on (10) unsupported.","rationale":"The reader's weakest assumption identifies exactly the same load-bearing gap: Proposition 4's proof uses e^{μ_k^2}−1 < 2μ_k^2 without the required per-step condition μ_k≤1, and Corollary 1 inherits this. My independent check confirms the concern is real for the stated general proposition, and it can be demonstrated numerically in parameter regimes consistent with the paper's schedules (notably the n=1 centralized case in Remark 2). The issue is fixable—either by using exact composition, by adding and verifying the μ_k≤1 condition, or by restricting the theorem to schedules for which it provably holds—so I do not move the verdict; the manuscript still needs a conditional revision. The clipping-bias term T2 is a secondary concern about the strength of the utility bound, but the privacy-accounting gap is more fundamental because it threatens the validity of the DP guarantee itself.","tokens_in":25761,"tokens_out":22240,"duration_ms":227132,"concrete_test":"Fix the Corollary 1 schedules with a concrete instance, e.g., J=1250, μ_tot=0.1, ρ_c=2, ρ_μ=10, and K=nJ^2μ_tot^2. Compute μ_k = C_k/(σ̃σ̃_k) using (10), and then compute the exact GDP cost (1/J)√(Σ_{k=0}^{K−1}(e^{μ_k^2}−1)). If for any k we have μ_k>1 or the exact cost exceeds μ_tot, Proposition 4's noise scale is insufficient. Run the same check for the centralized reduction n=1, where K=J^2μ_tot^2; there μ_{K−1} is about 1.5 for ρ_μ=10, so the claimed privacy guarantee fails as stated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The main utility claim (Corollary 1) is obtained by substituting the noise scale of Proposition 4 into Theorem 1. In Appendix C, Proposition 4 bounds the exact GDP composition cost by (1/J)√(2Σμ_k^2), using the inequality e^x−1 < 2x, which is valid only for x≤1 (i.e., for every per-step budget μ_k = C_k/(σ̃σ̃_k) ≤ 1). The paper neither states this condition nor verifies it for the exponential schedules in Corollary 1. For C_k = Θ((ρ_c)^{−k/K}) and σ̃_k = Θ((ρ_c ρ_μ)^{−k/K}), the per-step budget under (10) is μ_k = (Jμ_tot)ρ_μ^{k/K} / √(2Σ_{s=0}^{K−1}ρ_μ^{2s/K}). If K = nJ^2μ_tot^2 as in Corollary 1, this is roughly ρ_μ^{k/K} / √(n(ρ_μ^2−1)/ln ρ_μ) up to constants; for n=1, ρ_μ=10, it exceeds 1 at late iterations, and for arbitrary sequences or smaller n it can be much larger. When μ_k>1, e^{μ_k^2}−1 > 2μ_k^2, so the actual composition cost exceeds the quantity the proof bounds, and the noise scale (10) may not provide the claimed (ε,δ)-DP guarantee. Algorithm 1's exact accounting in (8) avoids this issue, but Proposition 4 and Corollary 1—the paper's stated privacy/utility theorem—do not.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Dyn-D2P, a differentially private decentralized learning algorithm for non-convex stochastic optimization over time-varying directed networks. The algorithm combines Push-Sum consensus with per-iteration gradient clipping and Gaussian noise, using a decaying clipping bound C_k and an increasing per-step GDP privacy budget μ_k so that the injected noise level decreases over training. The authors provide a general utility result (Theorem 1) for arbitrary clipping and noise schedules, then specialize to exponential schedules in Corollary 1, obtaining a bound of the form O(1/((1-q)^2 √(n) J μ_tot)) plus a bias term T2 from clipping. Experiments on CIFAR-10 and FashionMNIST show that Dyn-D2P outperforms fixed-noise decentralized DP baselines, especially under strong privacy. The paper claims to be the first to give a utility guarantee for decentralized non-convex DP optimization with dynamic clipping bounds and noise levels, with an explicit network dependence.","tokens_in":26172,"tokens_out":12033,"duration_ms":114701,"significance":"If the central results are correct, this is a meaningful contribution: it extends dynamic clipping/noise scheduling from centralized DP-SGD to the decentralized setting, provides explicit network-dependent rates, and demonstrates empirical gains. The 1/√n scaling in Corollary 1 is a useful qualitative result, and the general Theorem 1 covers arbitrary schedules. The paper also includes two ablation variants (Dyn[C]-D2P and Dyn[μ]-D2P) and a fixed-noise baseline, and the experiments support the main qualitative claims. The proofs follow standard descent-plus-consensus arguments and are largely coherent. However, the privacy proof of Proposition 4 contains a domain condition that is neither stated nor verified for the schedules used in Corollary 1; this is a load-bearing gap that must be addressed before the utility guarantee can be considered fully supported.","major_comments":[{"comment":"The proof of Proposition 4 uses the inequality e^x − 1 < 2x for 0 ≤ x ≤ 1, with x = μ_k^2, but the statement of Proposition 4, the description of Algorithm 2, and Corollary 1 do not state or verify that every per-step GDP parameter μ_k = C_k/(σ̃ σ̃_k) satisfies μ_k ≤ 1 (or at least μ_k^2 below the validity threshold of the inequality). For the exponential schedules of Corollary 1, with C_k = Θ((ρ_c)^{-k/K}), σ̃_k = Θ((ρ_c ρ_μ)^{-k/K}), γ = 1/(√n J μ_tot), and K = n J^2 μ_tot^2, the per-step budget satisfies μ_k = (J μ_tot) ρ_μ^{k/K} / √(2Σ_{s=0}^{K-1} ρ_μ^{2s/K}) up to constants. For n = 1 and ρ_μ = 10, this exceeds 1 at later iterations (approximately 1.52 at k = K−1), so e^{μ_k^2} − 1 > 2μ_k^2 for those steps and the composition bound in (10) may not hold. Consequently, the (ε,δ)-DP guarantee claimed in Proposition 4 and the utility bound in Corollary 1 built on (10) are not established as stated. The authors should either add the condition μ_k ≤ 1 for all k and verify it for the schedules used in Corollary 1, or replace the relaxed inequality with the exact composition formula (as already done in (8) for Algorithm 1).","section":"Appendix C, Proposition 4"},{"comment":"The headline utility bound in Corollary 1 is O(1/((1-q)^2 √n J μ_tot)) + T2, where the bias term T2 is left entirely unquantified. Since T2 = 2E[(1/K)Σ Λ‖∇f(x̄_k)‖ (1/n)Σ P_k^i(C_k)] is nonnegative and can be of constant order, the stated 1/√n scaling does not by itself provide a convergence guarantee to a stationary point unless the clipping probabilities P_k^i(C_k) are controlled. The authors should either prove a bound on T2 under an explicit condition on the clipping schedule (e.g., a bound on the probability of clipping as a function of C_k), or clearly state that the result is a bias-variance trade-off and that the 1/√n scaling applies only to the non-bias terms. Without this, the claim of a \"provable utility guarantee\" is weaker than the notation suggests.","section":"Corollary 1, Eq. (12)"}],"minor_comments":[{"comment":"In Algorithm 2, the noise is drawn as N_i^k ∼ N(0, σ̃^2 I_d) and then multiplied by σ̃_k in the update, so the actual injected noise has variance σ̃^2 σ̃_k^2. The notation is consistent with the text \"σ_k = σ̃·σ̃_k\", but reading line 5 alone is confusing because σ̃^2 is not the full variance; please clarify by writing N_i^k ∼ N(0, I_d) and scaling by σ̃ σ̃_k in the update, or by defining the noise in one place.","section":"Algorithm 2, lines 5–6"},{"comment":"The abstract and conclusion state the contribution as the 'first model utility analysis for differentially private decentralized non-convex optimization with dynamic gradient clipping bounds and noise levels.' Given that the proposition underlying the privacy guarantee currently requires an unstated condition, the authors should either temper this claim until the condition is verified or explicitly state the condition in the theorem statements.","section":"Abstract and Section 6"},{"comment":"The inequality e^x − 1 < 2x is stated for '0 ≤ x ≤ 1', but the proof does not explain why μ_k^2 falls in this range. Even when the condition μ_k ≤ 1 is added, the authors should also justify it for the schedules in Algorithm 1, where μ_k is computed via the exact composition formula (8) rather than the relaxed bound (10).","section":"Appendix C, step (a)"},{"comment":"There are several typos: 'unblanceness' should be 'unbalancedness'; 'Cauchy-Swarchz' should be 'Cauchy-Schwarz'; and in the pseudocode of Algorithm 1, 'Calculates the clipping bound by: C_k = C_0·(ρ_c)^{-k/K}' uses a superscript minus sign that renders awkwardly. A careful proofreading pass is recommended.","section":"Throughout"},{"comment":"The Θ notation in Corollary 1 hides constants that depend on ρ_c and ρ_μ, and the step from (35) to (36) uses the fact that (1/K)(ρ_μ^2−1)/(ρ_μ^{2/K}−1) is O(1) for fixed ρ_μ. This is correct, but the dependence on ρ_μ and ρ_c is not made explicit in the final O(·) statement; the authors should state whether the hidden constants remain uniformly bounded over the range of ρ_μ, ρ_c used in the experiments.","section":"Appendix B, Eq. (35)–(36)"}],"recommendation":"major_revision","confidential_remarks":"The central algorithmic idea and the descent-plus-consensus analysis are sound and the experimental study is useful. The blocking issue is the unverified μ_k ≤ 1 condition in Proposition 4, which directly affects Corollary 1's privacy and utility claims; this is fixable by adding the condition and verifying it for the exponential schedules, or by using exact composition. The second major comment on the unquantified bias term T2 is also worth addressing. The paper fits the journal's scope and, after revision, should be reconsidered."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: this paper does something genuinely new—the first utility analysis for decentralized non-convex DP optimization with dynamic clipping bounds and noise levels—but the headline privacy theorem has a condition gap that can undermine the DP guarantee for exactly the exponential schedules the paper proposes. The empirical story is encouraging, though the comparison is narrower than it should be.\n\nThe new piece is the combination of push-sum, dynamic clipping, and an increasing per-step GDP budget, analyzed in a time-varying directed network. The bound in Corollary 1 gives explicit dependence on network connectivity and a 1/sqrt(n) scaling, up to a bias term T2. That result is absent from the fixed-noise decentralized works and from the centralized dynamic-noise analysis. The proof follows the standard descent-plus-consensus template, and I did not find an internal contradiction. The experiments show large accuracy gains over the fixed-noise Const-D2P baseline at strong privacy, and the hyperparameter sensitivity analysis is useful.\n\nThe soft spots, in proportion:\n\nFirst and most important, Proposition 4 is stated without a condition that its proof requires. The proof uses e^x - 1 < 2x, valid only for x <= 1, i.e., mu_k <= 1 at every step. The proposition does not state that, and Algorithm 1's exponential schedule does not guarantee it. For small n, high rho_mu, and the K chosen in Corollary 1, late iterations can have mu_k > 1; the stress-test example with n=1, rho_mu=10 is concrete. So the privacy claim for Algorithm 2 is unsupported as written. This is fixable—use the exact composition in (8) or state and enforce mu_k <= 1—but it is a real gap.\n\nSecond, T2 is unquantified. The authors openly call it a bias term, but it prevents the utility bound from being a full guarantee; without a bound on the clipping probability the O(1/sqrt(n)) rate is conditional. A reviewer should ask for a bound under a standard assumption such as sub-Gaussian gradients.\n\nThird, experiments choose hyperparameters per algorithm via grid search for best accuracy, which favors the proposed method; there are no variance bars, and no head-to-head comparison against the actual prior DP decentralized algorithms (Yu, Xu, Li and Chi). The gains over Const-D2P are real, but 'superiority' needs a broader baseline.\n\nWho it's for: people working in private or decentralized learning, especially those combining DP with gossip. It deserves a serious referee; the mechanism is plausible and the utility analysis is a genuine first, but the privacy gap and unquantified bias need repair before acceptance.\n\nMy recommendation: send it to peer review. The authors have a clear path to fix the issues.","headline":"Genuinely new utility analysis for dynamic clipping/noise in decentralized DP, but the privacy theorem silently requires mu_k <= 1 and the utility bound leaves T2 unquantified; worth peer review with repair requests.","tokens_in":26672,"tokens_out":3745,"would_cite":false,"duration_ms":36629,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that an exponentially decaying clipping bound and an exponentially growing per-step privacy budget let a decentralized learning system shrink its injected noise over time while holding a fixed total privacy budget, and…","keywords":["differential privacy","decentralized learning","dynamic noise schedule","gradient clipping","Gaussian differential privacy","non-convex optimization","utility-privacy trade-off","time-varying directed graphs"],"falsifier":"For a concrete case, choose $J$, $\\rho_\\mu$, $K$, and $\\mu_{\\mathrm{tot}}$ as in the paper's experiments, solve equation (8) for $\\mu_0$, and compute $\\max_k \\mu_0(\\rho_\\mu)^{k/K}$; if the maximum exceeds $1$, the proof of Proposition 4 does not apply at that schedule, so the noise scale from (10) may not deliver the claimed privacy guarantee.","tokens_in":25564,"feed_emoji":"🔒","tokens_out":9162,"duration_ms":77775,"temperature":0.7,"pith_summary":"The paper is trying to show that fixed-strength noise in decentralized private learning is wasteful: once training gradients shrink below a constant clipping bound, the algorithm is adding more noise than the privacy budget requires. It proposes Dyn-D2P, which decays the clipping bound and grows the per-step privacy budget as training progresses, so the Gaussian noise decreases while the total privacy guarantee over all steps stays fixed. The central theoretical result is a utility bound for non-convex decentralized optimization under this dynamic schedule, $O(1/((1-q)^2\\sqrt{n}J\\mu_{\\mathrm{tot}}))$ plus a bias term, presented as the first such analysis for dynamic clipping and noise levels. If correct, this matters because it says the accuracy cost of privacy improves explicitly with the number of nodes and with faster network information propagation.","feed_headline":"First utility bound for dynamic-noise private decentralized learning","feed_subtitle":"Dynamic clipping and noise schedules shrink error at rate 1/√n, up to a clipping bias.","key_machinery":"The carrying mechanism is the noise schedule: at iteration $k$ each node clips its stochastic gradient at $C_k=C_0(\\rho_c)^{-k/K}$, assigns itself a per-step Gaussian-DP budget $\\mu_k=\\mu_0(\\rho_\\mu)^{k/K}$, and adds Gaussian noise with standard deviation $\\sigma_k=C_k/\\mu_k$, so noise decays at the combined rate $(\\rho_c\\rho_\\mu)^{-k/K}$. The privacy accounting is the GDP composition formula $\\mu_{\\mathrm{tot}}=(1/J)\\sqrt{\\sum_{k=0}^{K-1}(e^{\\mu_k^2}-1)}$, and Proposition 4 turns that into a single noise-scale condition $\\tilde{\\sigma}$. The utility analysis then writes the error as a privacy-noise term $T_1$ plus a clipping-bias term $T_2$, with consensus error controlled by the Push-Sum averaging protocol's propagation constant $q$.","core_discovery":"For its general Algorithm 2 with arbitrary clipping bounds $C_k$ and noise multipliers $\\tilde{\\sigma}_k$, the paper proves (Theorem 1) an upper bound on the average squared gradient norm at the node-average model, with the error split into a privacy-noise term $T_1$ and a clipping-bias term $T_2$. Choosing $C_k=\\Theta((\\rho_c)^{-k/K})$ and $\\tilde{\\sigma}_k=\\Theta((\\rho_c\\rho_\\mu)^{-k/K})$, together with step size $\\gamma=1/(\\sqrt{n}J\\mu_{\\mathrm{tot}})$ and $\\gamma K=\\sqrt{n}J\\mu_{\\mathrm{tot}}$, the bound becomes $O(1/((1-q)^2\\sqrt{n}J\\mu_{\\mathrm{tot}}))+T_2$ whenever $J\\mu_{\\mathrm{tot}}>\\sqrt{n}$. This is the paper's claimed first model-utility guarantee for differentially private decentralized non-convex optimization with dynamic gradient clipping and dynamic noise, and it makes the network dependence explicit through the information-propagation constant $q$.","pith_inferences":["The privacy proof's inequality $e^{\\mu_k^2}-1<2\\mu_k^2$ is only valid for $\\mu_k\\le 1$, so any concrete schedule from Algorithm 1 should be checked against that condition; without that check, the claimed DP guarantee for large $J$ is unverified.","Because Theorem 1 allows arbitrary $C_k$ and $\\tilde{\\sigma}_k$ sequences, it provides a ready template for evaluating adaptive clipping heuristics that set the bound from current gradient norms instead of an exponential schedule.","A sharper treatment of the clipping-bias term $T_2$ would be needed to obtain a fully vanishing error; in the strong-privacy regime the bias may dominate, and the paper's own experiments show the largest gains at small $\\epsilon$ without isolating how much of those gains come from reduced noise versus bias changes."],"forward_implications":["Under the Corollary 1 scheduling, the utility error floor decreases as $1/\\sqrt{n}$, so adding nodes genuinely improves accuracy under a fixed privacy budget.","The explicit factor $1/(1-q)^2$ means faster-mixing network topologies are provably better for private decentralized training with the same total privacy cost.","Letting $n=1$ recovers the centralized dynamic-DP rate $O(1/(J\\mu_{\\mathrm{tot}}))$, showing the decentralized analysis contains the single-node result as a special case.","The bias term $T_2$ persists because clipping makes gradients biased, and the paper's design keeps the clipping probability roughly constant as gradient norms decay, rather than eliminating the bias."],"supporting_citations":[{"why":"defines the µ-GDP mechanism whose noise rule σ=S/µ sets the per-step noise variance.","marker":"[Dong et al., 2019]"},{"why":"gives the µ-GDP to (epsilon,delta)-DP conversion used to set mu_tot from the target privacy budget.","marker":"[Bu et al., 2020]"},{"why":"supplies the GDP composition theorem for varying per-step budgets and the centralized dynamic-DP utility analysis this paper extends.","marker":"[Du et al., 2021]"},{"why":"provides the Push-Sum consensus lemma that bounds de-biasing error in terms of q.","marker":"[Assran et al., 2019]"},{"why":"underlies the Push-Sum averaging scheme used to handle directed time-varying graphs.","marker":"[Kempe et al., 2003]"},{"why":"is one of the fixed-noise decentralized DP baselines against which Dyn-D2P is compared in utility and experiments.","marker":"[Yu et al., 2021]"},{"why":"is the recent fixed-noise decentralized DP work whose lack of dynamic-noise utility analysis motivates the paper's claim of first.","marker":"[Li and Chi, 2025]"}],"fun_headline_variants":["First utility bound for dynamic-DP decentralized learning","Dynamic clipping/noise yields provable gain in decentralized DP","Adaptive DP noise: first non-convex decentralized utility bound","Private decentralized training speed-up: 1/√n rate proven","Dyn-D²P: beating static noise with a provable utility edge"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that every per-step privacy budget satisfies $\\mu_k \\le 1$, because the composition bound's inequality $e^{\\mu_k^2}-1<2\\mu_k^2$ only holds in that range, and the paper never states or verifies this condition for its exponentially growing schedules.","fun_headline_variants_meta":{"raw":{"variants":["First utility bound for dynamic-DP decentralized learning","Dynamic clipping/noise yields provable gain in decentralized DP","Adaptive DP noise: first non-convex decentralized utility bound","Private decentralized training speed-up: 1/√n rate proven","Dyn-D²P: beating static noise with a provable utility edge"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000182,"raw_usage":{"total_tokens":1334,"prompt_tokens":993,"completion_tokens":341,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":609,"completion_tokens_details":{"reasoning_tokens":255}},"tokens_in":609,"tokens_out":341,"duration_ms":4263,"temperature":1.0,"reasoning_tokens":255,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:37:26.083779+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"For a concrete case, choose $J$, $\\rho_\\mu$, $K$, and $\\mu_{\\mathrm{tot}}$ as in the paper's experiments, solve equation (8) for $\\mu_0$, and compute $\\max_k \\mu_0(\\rho_\\mu)^{k/K}$; if the maximum exceeds $1$, the proof of Proposition 4 does not apply at that schedule, so the noise scale from (10) may not deliver the claimed privacy guarantee.","supporting_citations":[{"cited_title":"Stochastic gradient push for distributed deep learning","cited_arxiv_id":null,"evidence_quote":"provides the Push-Sum consensus lemma that bounds de-biasing error in terms of q."},{"cited_title":"Gossip-based computation of aggregate information","cited_arxiv_id":null,"evidence_quote":"underlies the Push-Sum averaging scheme used to handle directed time-varying graphs."},{"cited_title":"Convergence and privacy of decentralized nonconvex optimization with gradient clipping and communication compression","cited_arxiv_id":null,"evidence_quote":"is the recent fixed-noise decentralized DP work whose lack of dynamic-noise utility analysis motivates the paper's claim of first."}],"review_version":1}