REVIEW 2 major objections 2 minor 45 references
A dynamic stochastic quantizer lets distributed online stochastic optimization converge almost surely to the optimum while guaranteeing (0,δ^i)-local differential privacy on directed graphs even after infinite iterations.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · grok-4.3
2026-06-29 05:54 UTC pith:EXQNW3RE
load-bearing objection Claims a.s. convergence plus (0,δ)-LDP over infinite steps on directed graphs via dynamic quantization, but the per-step privacy decay needed for composition may conflict with the error decay needed for convergence. the 2 major comments →
Local Differential Privacy via Dynamic Quantization in Distributed Online Stochastic Optimization
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
By employing an elaborately designed dynamic stochastic quantizer to mask exchanged information, the proposed algorithm converges almost surely to the optimal solution and achieves (0,δ^i)-local differential privacy for each agent i even when the number of iterations tends to infinity. The algorithm is fully distributed and applicable to directed graphs. This is the first work on distributed online stochastic optimization that simultaneously achieves exact convergence and rigorous local differential privacy over a directed graph by exploiting quantization effects.
What carries the argument
The dynamic stochastic quantizer, which adjusts its stochastic levels over time to mask information sufficiently for privacy while preserving the almost-sure convergence of the distributed updates.
Load-bearing premise
An elaborately designed dynamic stochastic quantizer exists that can mask the exchanged values enough to meet the local differential privacy bound without preventing the iterates from converging almost surely.
What would settle it
A concrete counter-example or long-horizon simulation on a directed graph in which the proposed quantizer either leaks more than δ^i probability mass for some agent or the sequence of iterates fails to converge almost surely to the claimed optimum.
If this is right
- The iterates converge almost surely to the optimal solution.
- Each agent satisfies (0,δ^i)-local differential privacy for any finite or infinite number of iterations.
- The algorithm runs in a fully distributed manner on arbitrary directed graphs.
- The same quantizer-based approach works for streaming data tasks such as online classification and recognition.
Where Pith is reading between the lines
- The quantization schedule could be adapted to trade communication bits against privacy level in other directed-network problems without introducing separate privacy primitives.
- Real-time applications such as distributed sensor fusion on asymmetric topologies become feasible when both convergence and ongoing privacy are required.
- The method suggests examining whether similar quantization dynamics can protect privacy in non-stochastic or non-convex distributed settings.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes a distributed online stochastic optimization algorithm that employs a dynamic stochastic quantizer to mask information exchanged over a directed communication graph. It claims that the algorithm converges almost surely to the optimal solution while simultaneously guaranteeing (0, δ^i)-local differential privacy for each agent i, even as the iteration count tends to infinity. The approach is asserted to be fully distributed and the first to achieve both exact convergence and rigorous LDP on directed graphs by exploiting quantization effects; numerical experiments on three classification datasets are provided to illustrate performance.
Significance. If the central claims hold, the result would be significant for privacy-preserving distributed learning: it would establish the first rigorous combination of almost-sure convergence and non-vanishing (0,δ)-LDP over infinite horizons on directed graphs without requiring a central coordinator or sacrificing exact optimality.
major comments (2)
- [Abstract and theoretical analysis] Abstract and §3–§4 (theoretical analysis): the claim of (0,δ^i)-LDP holding for the infinite sequence of quantized messages requires that the per-step privacy losses satisfy ∑δ_t < ∞. Standard almost-sure convergence arguments for online stochastic gradient methods on directed graphs, however, require the quantization noise variance to obey ∑α_t²σ_t² < ∞. No explicit parameter regime or design rule for the dynamic quantizer is shown to reconcile these two summability conditions simultaneously under the stated assumptions on the objective, noise, and graph.
- [§4 (privacy analysis)] §4 (privacy analysis): the (0,δ^i) guarantee is stated to hold “even when the number of iterations tends to infinity.” Under standard composition, this requires the dynamic quantizer to drive δ_t → 0 sufficiently fast; the manuscript does not exhibit the explicit decay schedule of the quantizer parameters that achieves both the privacy sum and the convergence sum on a directed graph.
minor comments (2)
- [§2–§3] Notation for the dynamic quantizer (e.g., the dependence of the quantization levels or noise variance on iteration index t) should be introduced earlier and kept consistent between the algorithm statement and the convergence/privacy proofs.
- [Numerical experiments] The numerical experiments section would benefit from an explicit statement of the directed-graph topology used and how the step-size and quantizer schedules were chosen to satisfy the theoretical conditions.
Simulated Author's Rebuttal
We thank the referee for the careful reading and for highlighting the need to explicitly reconcile the summability conditions for privacy and convergence. We address each major comment below.
read point-by-point responses
-
Referee: [Abstract and theoretical analysis] Abstract and §3–§4 (theoretical analysis): the claim of (0,δ^i)-LDP holding for the infinite sequence of quantized messages requires that the per-step privacy losses satisfy ∑δ_t < ∞. Standard almost-sure convergence arguments for online stochastic gradient methods on directed graphs, however, require the quantization noise variance to obey ∑α_t²σ_t² < ∞. No explicit parameter regime or design rule for the dynamic quantizer is shown to reconcile these two summability conditions simultaneously under the stated assumptions on the objective, noise, and graph.
Authors: The manuscript parameterizes the dynamic quantizer so that its noise variance can be chosen to satisfy both requirements simultaneously: the convergence condition depends on the product α_t²σ_t² while the per-step privacy loss δ_t depends on the sensitivity divided by the effective noise scale. Because the quantization levels and dynamic range are free design parameters, one can select σ_t² = O(α_t / t^ε) for ε>0 (or an analogous schedule) to make ∑α_t²σ_t² < ∞ while ensuring ∑δ_t < ∞. We agree that this joint regime was not stated explicitly and will add a dedicated remark (or short subsection) in the revised §4 that lists the admissible decay rates under the paper’s standing assumptions on the objective, noise, and graph. revision: yes
-
Referee: [§4 (privacy analysis)] §4 (privacy analysis): the (0,δ^i) guarantee is stated to hold “even when the number of iterations tends to infinity.” Under standard composition, this requires the dynamic quantizer to drive δ_t → 0 sufficiently fast; the manuscript does not exhibit the explicit decay schedule of the quantizer parameters that achieves both the privacy sum and the convergence sum on a directed graph.
Authors: The privacy analysis already invokes the infinite-horizon composition theorem and relies on the quantizer becoming progressively finer. The explicit schedule is obtained by letting the quantization step size decrease at a polynomial rate compatible with the step-size sequence α_t used for convergence on the directed graph. We will insert the concrete decay rule (e.g., δ_t = O(1/t^{1+ε})) together with the corresponding choice of quantization parameters into the revised version of §4 so that both summability conditions are visibly satisfied. revision: yes
Circularity Check
No circularity: claims rest on analysis of a novel dynamic quantizer construction
full rationale
The paper proposes a new distributed online stochastic optimization algorithm that incorporates an elaborately designed dynamic stochastic quantizer. It then states that theoretical analysis establishes both almost-sure convergence to the optimum and (0,δ^i)-LDP for each agent even as the iteration count tends to infinity, on directed graphs. No quoted step, equation, or self-citation in the provided abstract reduces the privacy or convergence result to a fitted parameter, a renaming of a known pattern, or a prior result whose only justification is the authors' own earlier work. The derivation is presented as a direct proof for the constructed algorithm rather than a tautology or composition of self-referential definitions. This is the normal case of an independent construction whose correctness is left to the reader to verify externally.
Axiom & Free-Parameter Ledger
read the original abstract
Distributed online stochastic optimization has received extensive attention in large-scale distributed learning and other related fields due to its unique advantage in processing streaming data. However, information exchange through the communication network during the optimization process may lead to privacy leakage. To address this issue, this paper proposes a locally differentially private distributed online stochastic optimization algorithm that employs an elaborately designed dynamic stochastic quantizer to mask the exchanged information prior to communication. Theoretical analysis shows that the proposed algorithm not only converges almost surely to the optimal solution but also achieves $(0,\delta^i)$-local differential privacy for each agent $i$ even when the number of iterations tends to infinity. Furthermore, the algorithm is fully distributed and applicable to scenarios where the interaction network among agents is a directed graph. To the best of our knowledge, this is the first work on distributed online stochastic optimization that simultaneously achieves exact convergence and rigorous local differential privacy over a directed graph by exploiting quantization effects. Numerical experiments of distributed online training on the mushroom classification dataset, handwritten digits recognition dataset, and brain-computer interface dataset verify the effectiveness of the proposed method.
Figures
Reference graph
Works this paper leans on
-
[1]
Optimal distributed control of A C microgrids with coordinated voltage regulation and reacti ve power sharing,
S. M. Mohiuddin, J. Qi, “Optimal distributed control of A C microgrids with coordinated voltage regulation and reacti ve power sharing,” IEEE Trans. Smart Grid , 2022, 13(3): 1789- 1800
2022
-
[2]
Distributed lear ning in wireless sensor networks,
J. B. Predd, S. B. Kulkarni, H. V. Poor, “Distributed lear ning in wireless sensor networks,” IEEE Signal Process. Mag. , 2006, 23(4): 56-69
2006
-
[3]
Distributed learning for wireless communications: Methods, applicati ons and challenges,
L. Qian, P. Yang, M. Xiao, O. Dobre, et al, “Distributed learning for wireless communications: Methods, applicati ons and challenges,” IEEE J. Sel. Top. Signal Process. , 2022, 16(3): 326-342
2022
-
[4]
Scalable distributed algorithms for mu lti- robot near-optimal motion planning,
G. Zhao, M. Zhu, “Scalable distributed algorithms for mu lti- robot near-optimal motion planning,” Automatica, 2022, 140: 110241
2022
-
[5]
On distributed convex optimizatio n under inequality and equality constraints,
M. Zhu, S. Martinez, “On distributed convex optimizatio n under inequality and equality constraints,” IEEE Trans. Autom. Control, 2011, 57(1): 151-164
2011
-
[6]
Convergence in high probability of distributed stochastic gradient desce nt algorithms,
K. Lu, H. W ang, H. Zhang, L. W ang, “Convergence in high probability of distributed stochastic gradient desce nt algorithms,” IEEE Trans. Autom. Control, 2023, 69(4): 2189- 2204
2023
-
[7]
Distributed variable samplesize stochastic optimization with fixed step-sizes,
J. Lei, P. Yi, J. Chen, Y. Hong, “Distributed variable samplesize stochastic optimization with fixed step-sizes, ” IEEE Trans. Autom. Control , 2022, 67(10): 5630-5637
2022
-
[8]
Distributed stochast ic constrained optimization with constant step-sizes via sad dle- point dynamics,
Y. Huang, S. Cui, X. Zeng, Z. Meng, “Distributed stochast ic constrained optimization with constant step-sizes via sad dle- point dynamics,” Automatica, 2026, 183: 112575
2026
-
[9]
Confidence region for distributed stoch astic optimization problem via stochastic gradient tracking method,
S. Zhao, Y. Liu, “Confidence region for distributed stoch astic optimization problem via stochastic gradient tracking method,” Automatica, 2024, 159: 111352
2024
-
[10]
Zeroth- order algorithms for stochastic distributed nonconvex optimization,
X. Yi, S. Zhang, T. Yang, K. H. Johansson, “Zeroth- order algorithms for stochastic distributed nonconvex optimization,” Automatica, 2022, 142: 110353. 14
2022
-
[11]
Stochastic dual averag ing for decentralized online optimization on time-varying communication graphs,
S. Lee, A. Nedi´ c, M. Raginsky, “Stochastic dual averag ing for decentralized online optimization on time-varying communication graphs,” IEEE Trans. Autom. Control, 2017, 62(12): 6407-6414
2017
-
[12]
Online stochastic optimization of networked distributed energy resources,
X. Zhou, E. Dall ´Anese, L. Chen, “Online stochastic optimization of networked distributed energy resources,” IEEE Trans. Autom. Control , 2019, 65(6): 2387-2401
2019
-
[13]
Online stochastic optimiz ation with time-varying distributions,
X. Cao, J. Zhang, H. V. Poor, “Online stochastic optimiz ation with time-varying distributions,” IEEE Trans. Autom. Control, 2020, 66(4): 1840-1847
2020
-
[14]
Online distributed nonconvex optimization with stochastic objective functions: High probability bound analysis of dynamic regrets,
H. Xu, K. Lu, Y. W ang, “Online distributed nonconvex optimization with stochastic objective functions: High probability bound analysis of dynamic regrets,” Automatica, 2024, 170: 111863
2024
-
[15]
Online distributed optimizati on with clipped stochastic gradients: High probability bound of regrets,
Y. Yang, K. Lu, L. W ang, “Online distributed optimizati on with clipped stochastic gradients: High probability bound of regrets,” Automatica, 2025, 182: 112525
2025
-
[16]
Deep leakage from gradients,
L. Zhu, Z. Liu, S. Han, “Deep leakage from gradients,” in Proc. Int. Conf. Adv. Neural Inf. Process. Syst. , 2019, 32: 17-31
2019
-
[17]
Privacy preserving distributed optimiz ation using homomorphic encryption,
Y. Lu, M. Zhu, “Privacy preserving distributed optimiz ation using homomorphic encryption,” Automatica, 2018, 96: 314- 325
2018
-
[18]
Privacy preservation in distributed subgradient optimization algorithms,
Y. Lou, L. Yu, S. W ang, P. Yi, “Privacy preservation in distributed subgradient optimization algorithms,” IEEE Trans. Cybern., 2017, 48(7): 2154-2165
2017
-
[19]
Private optimization on network s,
S. Gade, N. H. Vaidya, “Private optimization on network s,” in Proc. Amer. Control Conf. , 2018: 1402-1409
2018
-
[20]
The algorithmic foundations of differential privacy,
C. Dwork, A. Roth, “The algorithmic foundations of differential privacy,” Found. Trends Theor. Comput. Sci. , 2014, 9(3-4): 211-487
2014
-
[21]
Distributed empirical risk minimization with differential privacy,
C. Liu, K. H. Johansson, Y. Shi, “Distributed empirical risk minimization with differential privacy,” Automatica, 2024, 162: 111514
2024
-
[22]
Differential priva cy in distributed optimization with gradient tracking,
L. Huang, J. W u, D. Shi, S. Dey, et al, “Differential priva cy in distributed optimization with gradient tracking,” IEEE Trans. Autom. Control , 2024, 69(9): 5727-5742
2024
-
[23]
Privacy-preserv ing decentralized dual averaging for online optimization over directed networks,
Q. L¨ u, K. Zhang, S. Deng, Y. Li, et al, “Privacy-preserv ing decentralized dual averaging for online optimization over directed networks,” IEEE Trans. Ind. Cyber-Phys. Syst. , 2023, 1: 79-91
2023
-
[24]
Privacy-preserved distributed learning with zeroth-ord er optimization,
C. Gratton, N. K. D. Venkategowda, R. Arablouei, S. W ern er, “Privacy-preserved distributed learning with zeroth-ord er optimization,” IEEE Trans. Inf. Forensics Secur. , 2021, 17: 265-279
2021
-
[25]
Tailoring gradient methods for differentially private distributed optimization,
Y. W ang, A. Nedi´ c, “Tailoring gradient methods for differentially private distributed optimization,” IEEE Trans. Autom. Control, 2023, 69(2): 872-887
2023
-
[26]
Robust constrained consensus and inequality-constrained distributed optimization wit h guaranteed differential privacy and accurate convergence,
Y. W ang, A. Nedi´ c, “Robust constrained consensus and inequality-constrained distributed optimization wit h guaranteed differential privacy and accurate convergence, ” IEEE Trans. Autom. Control , 2024, 69(11): 7463-7478
2024
-
[27]
J. Chen, J. W ang, J. Zhang, “Differentially private grad ient- tracking-based distributed stochastic optimization over directed graphs,” IEEE Trans. Autom. Control , 2026, DOI: 10.1109/TAC.2026.3678834
-
[28]
Locally differentially private distr ibuted online learning with guaranteed optimality,
Z. Chen, Y. W ang, “Locally differentially private distr ibuted online learning with guaranteed optimality,” IEEE Trans. Autom. Control, 2024, 70(4): 2521-2536
2024
-
[29]
Quantization enabled privacy protection in decentralized stochastic optimization,
Y. W ang, T. Ba¸ sar, “Quantization enabled privacy protection in decentralized stochastic optimization,” IEEE Trans. Autom. Control, 2022, 68(7): 4038-4052
2022
-
[30]
Quantization enabled differential privacy in bandit games with cooperati ve players,
Y. Lin, K. Liu, I. Bistritz, Q. Ma, et al, “Quantization enabled differential privacy in bandit games with cooperati ve players,” IEEE Trans. Autom. Control , 2025, 70(11): 7771- 7778
2025
-
[31]
Local differential privacy for decentralized online stochastic optimization with guaran teed optimality and convergence speed,
Z. Chen, Y. W ang, “Local differential privacy for decentralized online stochastic optimization with guaran teed optimality and convergence speed,” IEEE Trans. Autom. Control, 2024, 70(7): 4238-4253
2024
-
[32]
A sharp estimat e on the transient time of distributed stochastic gradient desc ent,
S. Pu, A. Olshevsky, I. C. Paschalidis, “A sharp estimat e on the transient time of distributed stochastic gradient desc ent,” IEEE Trans. Autom. Control , 2021, 67(11): 5900-5915
2021
-
[33]
A stoc hastic newton algorithm for distributed convex optimization,
B. Bullins, K. Patel, O. Shamir, N. Srebro, et al, “A stoc hastic newton algorithm for distributed convex optimization,” in Proc. Int. Conf. Adv. Neural Inf. Process. Syst , 2021, 34: 26818-26830
2021
-
[34]
Push-pull gradient meth ods for distributed optimization in networks,
S. Pu, W. Shi, J. Xu, A. Nedi´ c, “Push-pull gradient meth ods for distributed optimization in networks,” IEEE Trans. Autom. Control, 2020, 66(1): 1-16
2020
-
[35]
Convergence and privacy of decentralize d nonconvex optimization with gradient clipping and communication compression,
B. Li, Y. Chi, “Convergence and privacy of decentralize d nonconvex optimization with gradient clipping and communication compression,” IEEE J. Sel. Top. Signal Process., 2025, 19(1): 273-282
2025
-
[36]
Gradient-tracking-based distrib uted optimization with guaranteed optimality under noisy information sharing,
Y. W ang, T. Ba¸ sar, “Gradient-tracking-based distrib uted optimization with guaranteed optimality under noisy information sharing,” IEEE Trans. Autom. Control , 2022, 68(8): 4796-4811
2022
-
[37]
UCI machine learning repository,
A. Asuncion, D. Newman, “UCI machine learning repository,” Sch. Inf. Comput. Sci., Univ. California, Irvine, CA, USA, 2007. [Online]. Available: http: //archive.ics.uci.edu/ml
2007
-
[38]
Gradient-ba sed learning applied to document recognition,
Y. Lucun, L. Bottou, Y. Bengio, P. Haffner, “Gradient-ba sed learning applied to document recognition,” Proc. IEEE, 2002, 86(11): 2278-2324
2002
-
[39]
iDLG: Improved Deep Leakage from Gradients
B. Zhao, K. R. Mopuri, H. Bilen, “iDLG: Improved deep leakage from gradients,” 2020, arXiv:2001.02610
work page Pith review arXiv 2020
-
[40]
Can decentralized algorithms outperform centralized algorit hms? a case study for decentralized parallel stochastic gradien t descent,
X. Lian, C. Zhang, H. Zhang, C.-J. Hsieh, et al, “Can decentralized algorithms outperform centralized algorit hms? a case study for decentralized parallel stochastic gradien t descent,” in Advances in Neural Information Processing Systems, 2017: 5330-5340
2017
-
[41]
EEGNet: A compact convolutional neural network for EEG-based brain-computer interfaces,
V. J. Lawhern, A. J. Solon, N. R. W aytowich, S. M. Gordon, et al, “EEGNet: A compact convolutional neural network for EEG-based brain-computer interfaces,” J. Neural Eng. , 2018, 15(5): 056013
2018
-
[42]
Review of the BCI competition IV,
M. Tangermann, K. R. M¨ uller, A. Aertsen, N. Birbaumer, et al, “Review of the BCI competition IV,” Front. Neurosci., 2012, 6: 55
2012
-
[43]
User identity protection in eeg-based brain-computer interfaces,
L. Meng, X. Jiang, J. Huang, W. Li, et al, “User identity protection in eeg-based brain-computer interfaces,” IEEE Trans. Neural Syst. Rehabil. Eng. , 2023, 31: 3576-3586
2023
-
[44]
Fderated motor imagery classification for privacy-preserving brain-comp uter interfaces,
T. Jia, L. Meng, S. Li, J. Liu, et al, “Fderated motor imagery classification for privacy-preserving brain-comp uter interfaces,” IEEE Trans. Neural Syst. Rehabil. Eng. , 2024, 32: 3442-3451
2024
-
[45]
Transfer learning for brain-computer interfaces: A Euclidean space data alignment approach,
H. He, D. W u, “Transfer learning for brain-computer interfaces: A Euclidean space data alignment approach,” IEEE Trans. Biomed. Eng. , 2019, 67(2): 399-410. 15
2019
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.