REVIEW 2 major objections 4 minor 33 references
Differentially Private Decentralized Dataset Synthesis Through Randomized Mixing with Correlated Noise
T0 review · 2 major / 4 minor · reviewed 2026-08-04 · deepseek-v4-flash
Pith's one-line read The paper claims that CAPE-assisted Federated DP-CDA releases an (ε,δ)-differentially private synthetic dataset in a federated setting, with utility matching centralized DP-CDA (MNIST 0.795 at ε=10), by adding correlated zero-sum noise at e
desk verdict The federated CAPE-DP-CDA proposal has a load-bearing privacy flaw: the proof uses the global subsampling rate for local subsampling and ignores the server's view, so the claimed epsilon does not follow. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the two-component noise structure: each client adds g_s ~ N(0, τ_g^2) independently and e_s ~ N(0, τ_e^2) jointly with the other clients under the constraint Σ_s e_s = 0. The zero-sum identity means that when the server forms the average (1/S)Σ_s, the e terms vanish and the variance of the aggregate equals the centralized case if τ_g^2 = τ_pool^2. The privacy proof then treats the random selection of l samples from each class as subsampling and applies a Rényi differential privacy amplification bound with probability p=lK/N, then composes over T synthetic samples.
What would settle it
Re-run the Rényi amplification calculation with p replaced by S·lK/N for the paper's settings (e.g., S=10, K=10, N=60,000, l=4) and compare the resulting ε with the paper's ε=10; if the inflated ε exceeds the claimed budget, the guarantee as stated is false. A second direct check: inspect the per-client transmitted values at the server before averaging and test whether a membership-inference adversary can distinguish a client's records using the individual noisy vectors, since the correlated component does not cancel in that view.
Extended reading notes
Core claim
The central claim is that the CAPE-assisted Federated DP-CDA mechanism releases a synthetic dataset D̃ that is (ε,δ)-differentially private, with ε = min_α T ε'(α) − log δ/(α−1), where ε'(α) is a subsampled Rényi divergence bound using the per-class sampling probability p=lK/N. Empirically, the synthetic data trains a small CNN to 78%+ accuracy on MNIST at ε=10, matching the centralized DP-CDA baseline, while conventional federated DP-CDA (independent noise only) falls to about 66%. The discovery is that the zero-sum correlated noise lets each client satisfy local differential privacy while the server-side average cancels the correlated component, removing the √S noise penalty that otherwise
Load-bearing premise
The proof assumes a local record's chance of entering a mix is the global per-class probability lK/N, but because a client holds only N/(SK) examples per class, the true chance is S·lK/N; a larger sampling probability means weaker subsampling amplification, so the reported ε may be understated.
Editorial extensions
If this is right
- If the guarantee holds, federated synthetic data can match centralized DP data utility, so downstream models can be trained on the synthetic release instead of on the raw distributed data.
- The √S noise inflation that forces conventional federated DP mechanisms to degrade with more clients disappears, making the method's accuracy roughly independent of S (as the paper's client-scaling experiments show).
- Because the released synthetic dataset satisfies DP, the synthetic data can be reused for many downstream tasks without consuming additional privacy budget beyond the original ε.
- The parameter l serves as a trade-off dial: small l preserves more sample detail but raises sensitivity; the experiments identify l≈4 as a sweet spot for MNIST and FashionMNIST.
- The computational cost O(T·S·l·d_x) grows linearly in the number of clients, so scaling to larger federations does not add a separate algorithmic overhead.
Reading between the lines
- The subsampling amplification in Theorem 1 uses p = lK/N, but each client samples l records from a local class set of size N/(SK); the true per-record inclusion probability is S·lK/N, so the numerical ε values are only as tight as that substitution is valid.
- A practical prerequisite the paper does not cost out: generating the zero-sum correlated noise across clients requires either a trusted coordinator or a secure multi-party random generation step; the privacy argument assumes the server never sees the individual e_s values before aggregation.
- If the cancellation is exact, the aggregate noise covariance should equal the centralized case, which could be verified by measuring sample covariance over repeated runs with the same τ_g; this would empirically separate the privacy-accounting question from the utility-cancellation question.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes CAPE Assisted Federated DP-CDA, a protocol for decentralized differentially private synthetic data generation. Each client z-score normalizes and clips its local data, then for each class randomly selects l samples, averages them, and adds two noise vectors: an independent Gaussian g_s and a correlated zero-sum Gaussian e_s. The server averages the received noisy synthetic samples, canceling the correlated noise, and releases the aggregate. The central theoretical claim is Theorem 1, which states that the released dataset is (ε,δ)-DP with ε = min_α T ε'(α) − log δ/(α−1), where ε'(α) is the RDP subsampling amplification bound applied with sampling probability p = lK/N. Experiments on MNIST and FashionMNIST report accuracy close to centralized DP-CDA (e.g., 78.07% on MNIST at ε=10), attributed to the cancellation of correlated noise.
Significance. If the privacy guarantee in Theorem 1 were correct, the paper would provide a useful decentralized protocol that achieves centralized-level utility for synthetic data. The experimental setup is clear, and the paper is explicit that the utility match is achieved by intentionally setting the local noise so that the aggregate variance equals the centralized case (Section 4, 'We intentionally set the local noise...'). This transparency is a strength, but it also means the empirical comparison is not an independent validation; everything rests on the privacy accountant. The paper ships no code and the proof is the main technical contribution. Unfortunately, the proof has two load-bearing gaps: the subsampling probability is wrong by a factor of S, and the composition ignores the server's observation of all local messages. These are not presentation issues; they invalidate the reported ε values.
major comments (2)
- [Theorem 1, Algorithm 1 Step 4] Theorem 1 sets p = lK/N and applies the subsampling amplification bound of [23] as if l points were sampled from the entire global dataset of size N. In Algorithm 1, however, each client samples l points from its local class-specific set X_{s,k}. Under the paper's own uniform partition assumption (N_s = N/S, K classes), |X_{s,k}| = N/(S K), so the inclusion probability of a fixed record is q = l / (N/(S K)) = S·lK/N = S p, not p. The RDP amplification bound is strongly increasing in p (roughly quadratically for small p), so the reported ε is optimistic by a factor on the order of S^2. Since the noise τ_g in the experiments is calibrated to satisfy the claimed ε (Appendix A), correcting q either requires a much larger ε or a larger τ_g, which would erase the accuracy advantage in Table 1. This is a load-bearing error in the central privacy claim.
- [Theorem 1 proof, Algorithm 1 Steps 8–11] The proof composes only over T synthetic samples, but the protocol transmits S local outputs for each t and class, and the honest-but-curious server observes all of them. The CAPE zero-sum property cancels e_s only in the aggregate average; the individual messages received by the server still contain e_s and are not covered by an analysis that 'reduces to analyzing the local noise component with variance τ_g^2'. A correct analysis for the released dataset in this threat model must either compose over S·T local mechanisms (in which case the relevant local noise variance is τ_e^2 + τ_g^2, not τ_g^2) or assume secure aggregation and analyze the aggregate mechanism directly. The manuscript does neither, so the stated (ε,δ) guarantee for the server's view is unsupported.
minor comments (4)
- [Algorithm 1, line 2] The input preprocessing step says 'Preprocess local dataset D_i' but the input is D_s; the subscript is inconsistent with the rest of the algorithm.
- [Theorem 1 statement] The expression 'ε = min_{α∈{3,4,...}} T ε′(α) − log δ/(α−1)' is ambiguous; it should be 'min_α (T ε′(α) − log δ/(α−1))' (or equivalently with log(1/δ)). As written, it could be read as (min_α T ε′(α)) − log δ/(α−1), which is not a valid conversion from RDP.
- [Appendix A, variance calculation] The formula for τ_pool is written as τ_pool = (1/N) sqrt(2 log(1.25/δ)), omitting the ε dependence from the Gaussian mechanism (Definition 3). If the intention is sensitivity 1 and ε=1, that should be stated; otherwise the expression is dimensionally and information-theoretically incomplete.
- [Table 2] The entry 'FashionMnist' should be 'FashionMNIST' for consistency. Also, the comparison with [19] would benefit from stating whether the centralized DP-CDA numbers are taken from [19] or reproduced; the current text does not clarify this.
Circularity Check
Utility match with centralized DP-CDA is engineered by setting the local noise variance to match the centralized variance, making the headline 'comparable utility' an input rather than an independent prediction.
-
fitted input called prediction
[Section 4, 'Performance on Test Partitions of Real Datasets in CAPE Assisted Federated DP-CDA']
"Recall that the correlated noise is canceled out during the final data aggregation at the central aggregator for our proposed scheme. We intentionally set the local noise to ensure that the noise variance of the ¯x(k) t and ¯yone-hot(k) t match that of the pooled-data or centralized scenarios. As a result, the same test accuracy (%) can be achieved as in the centralized DP-CDA [19] through our proposed CAPE Assisted Federated DP-CDA. We refer the reader to [20] for a formal analysis of this."
The paper's headline empirical claim—that CAPE-assisted federated DP-CDA achieves utility comparable to centralized DP-CDA—is not an independent finding. The local noise τ_g is chosen (fitted) so that, after the zero-sum CAPE noise cancels in the server-side average, the aggregate noise variance equals the centralized DP-CDA noise variance (Appendix A sets the CAPE estimator variance equal to the pooled variance). The paper then explicitly states 'As a result, the same test accuracy (%) can be achieved.' Thus the compared quantity (accuracy) is asserted to follow from the parameter-setting step; it is a design target enforced by construction rather than a prediction validated against the centralized baseline. The formal analysis is then delegated to the authors' own prior CAPE paper [20].
full rationale
The privacy theorem (Theorem 1) is a reapplication of standard RDP composition and subsampling amplification (Wang et al. [23]) to the DP-CDA mixing operation, with the CAPE zero-sum noise cancellation used to justify dropping the correlated term. That chain is not circular in itself, though it is inherited largely from the authors' prior DP-CDA work [19] and contains a serious sampling-probability error (p should reflect the local class-subset size, not the global N) that is a correctness risk rather than circularity. The main circular element is the utility claim: the paper explicitly sets the local noise so that the aggregated noise variance matches the centralized DP-CDA baseline and then presents the resulting equality in test accuracy as its central empirical contribution. Because the parameter is calibrated to the baseline before the comparison, the 'match' is built into the setup, not derived or independently predicted. This warrants a moderate circularity score. Self-citations to [19] and [20] are frequent but not independently problematic; the CAPE protocol is from a peer-reviewed external source and the DP-CDA analysis is at least partially re-derived in the text.
Assumptions & free parameters
free parameters (3)
- clipping threshold c
- local noise standard deviation τ_g
- correlated noise standard deviation τ_e
assumptions (3)
- domain assumption Each client's local dataset contains all classes with sufficient samples per class
- domain assumption The correlated noise terms e_s are unknown to the server/adversary
- ad hoc to paper Subsampling amplification applies with sampling probability p = lK/N
Cite this review
Pith. "Pith review of Differentially Private Decentralized Dataset Synthesis Through Randomized Mixing with Correlated Noise." pith.science (2026). https://pith.science/paper/7HCEAAC6
@misc{pith2026250910385,
author = {Pith},
title = {Pith review of: Differentially Private Decentralized Dataset Synthesis Through Randomized Mixing with Correlated Noise},
year = {2026},
howpublished = {\url{https://pith.science/paper/7HCEAAC6}},
note = {Machine review of arXiv:2509.10385}
}
read the original abstract
In this work, we explore differentially private synthetic data generation in a decentralized-data setting by building on the recently proposed Differentially Private Class-Centric Data Aggregation (DP-CDA). DP-CDA synthesizes data in a centralized setting by mixing multiple randomly-selected samples from the same class and injecting carefully calibrated Gaussian noise, ensuring ({\epsilon}, {\delta})-differential privacy. When deployed in a decentralized or federated setting, where each client holds only a small partition of the data, DP-CDA faces new challenges. The limited sample size per client increases the sensitivity of local computations, requiring higher noise injection to maintain the differential privacy guarantee. This, in turn, leads to a noticeable degradation in the utility compared to the centralized setting. To mitigate this issue, we integrate the Correlation-Assisted Private Estimation (CAPE) protocol into the federated DP-CDA framework and propose CAPE Assisted Federated DP-CDA algorithm. CAPE enables limited collaboration among the clients by allowing them to generate jointly distributed (anti-correlated) noise that cancels out in aggregate, while preserving privacy at the individual level. This technique significantly improves the privacy-utility trade-off in the federated setting. Extensive experiments on MNIST and FashionMNIST datasets demonstrate that the proposed CAPE Assisted Federated DP-CDA approach can achieve utility comparable to its centralized counterpart under some parameter regime, while maintaining rigorous differential privacy guarantees.
Figures
Reference graph
Works this paper leans on
-
[20]
A correlated noise-assisted decentralized differentially private estimation protocol, and its application to fmri source separation,
H. Imtiaz, J. Mohammadi, R. Silva, B. Baker, S. M. Plis, A. D. Sarwate, and V . D. Calhoun, “A correlated noise-assisted decentralized differentially private estimation protocol, and its application to fmri source separation,” IEEE Transactions on Signal Processing, vol. 69, pp. 6355–6370, 2021. 8 APREPRINT- SEPTEMBER15, 2025
2021
-
[23]
Subsampled rényi differential privacy and analytical moments accountant,
Y .-X. Wang, B. Balle, and S. P. Kasiviswanathan, “Subsampled rényi differential privacy and analytical moments accountant,” inThe 22nd international conference on artificial intelligence and statistics, pp. 1226–1235, PMLR, 2019
2019
-
[1]
Membership inference attacks against machine learning models,
R. Shokri, M. Stronati, C. Song, and V . Shmatikov, “Membership inference attacks against machine learning models,” in2017 IEEE symposium on security and privacy (SP), pp. 3–18, IEEE, 2017
2017
-
[2]
Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning,
M. Nasr, R. Shokri, and A. Houmansadr, “Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning,” in2019 IEEE symposium on security and privacy (SP), pp. 739–753, IEEE, 2019
2019
-
[3]
The secret sharer: Evaluating and testing unintended memorization in neural networks,
N. Carlini, C. Liu, Ú. Erlingsson, J. Kos, and D. Song, “The secret sharer: Evaluating and testing unintended memorization in neural networks,” in28th USENIX security symposium (USENIX security 19), pp. 267–284, 2019
2019
-
[4]
Property inference attacks on fully connected neural networks using permutation invariant representations,
K. Ganju, Q. Wang, W. Yang, C. A. Gunter, and N. Borisov, “Property inference attacks on fully connected neural networks using permutation invariant representations,” inProceedings of the 2018 ACM SIGSAC conference on computer and communications security, pp. 619–633, 2018
2018
-
[5]
Federated learning in non-iid settings aided by differentially private synthetic data,
H. Chen and H. Vikalo, “Federated learning in non-iid settings aided by differentially private synthetic data,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5027–5036, 2023
2023
-
[6]
Gen- erative models for effective ml on private, decentralized datasets,
S. Augenstein, H. B. McMahan, D. Ramage, S. Ramaswamy, P. Kairouz, M. Chen, R. Mathews,et al., “Gen- erative models for effective ml on private, decentralized datasets,”2020 International Conference on Learning Representations (ICLR), 2020
2020
Show all 33 references
-
[7]
Federated learning: Challenges, methods, and future directions,
T. Li, A. K. Sahu, A. Talwalkar, and V . Smith, “Federated learning: Challenges, methods, and future directions,” IEEE signal processing magazine, vol. 37, no. 3, pp. 50–60, 2020
2020
-
[8]
Federated machine learning: Concept and applications,
Q. Yang, Y . Liu, T. Chen, and Y . Tong, “Federated machine learning: Concept and applications,”ACM Transactions on Intelligent Systems and Technology (TIST), vol. 10, no. 2, pp. 1–19, 2019
2019
-
[9]
Communication-efficient learning of deep networks from decentralized data,
B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” inArtificial intelligence and statistics, pp. 1273–1282, PMLR, 2017
2017
-
[10]
Federated learning: Strategies for improving communication efficiency,
J. Kone ˇcn`y, H. B. McMahan, F. X. Yu, P. Richtárik, A. T. Suresh, and D. Bacon, “Federated learning: Strategies for improving communication efficiency,”arXiv preprint arXiv:1610.05492, 2016
2016 arXiv
-
[11]
Inverting gradients-how easy is it to break privacy in federated learning?,
J. Geiping, H. Bauermeister, H. Dröge, and M. Moeller, “Inverting gradients-how easy is it to break privacy in federated learning?,”Advances in neural information processing systems, vol. 33, pp. 16937–16947, 2020
2020
-
[12]
Handling privacy-sensitive medical data with federated learning: challenges and future directions,
O. Aouedi, A. Sacco, K. Piamrat, and G. Marchetto, “Handling privacy-sensitive medical data with federated learning: challenges and future directions,”IEEE journal of biomedical and health informatics, vol. 27, no. 2, pp. 790–803, 2022
2022
-
[13]
Cafe: Catastrophic data leakage in vertical federated learning,
X. Jin, P.-Y . Chen, C.-Y . Hsu, C.-M. Yu, and T. Chen, “Cafe: Catastrophic data leakage in vertical federated learning,”Advances in neural information processing systems, vol. 34, pp. 994–1006, 2021
2021
-
[14]
Tapfed: Threshold secure aggregation for privacy-preserving federated learning,
R. Xu, B. Li, C. Li, J. B. Joshi, S. Ma, and J. Li, “Tapfed: Threshold secure aggregation for privacy-preserving federated learning,”IEEE Transactions on Dependable and Secure Computing, vol. 21, no. 5, pp. 4309–4323, 2024
2024
-
[15]
Privacy-preserving federated learning via functional encryption, revisited,
Y . Chang, K. Zhang, J. Gong, and H. Qian, “Privacy-preserving federated learning via functional encryption, revisited,”IEEE Transactions on Information Forensics and Security, vol. 18, pp. 1855–1869, 2023
2023
-
[16]
Differentially private federated learning: An information- theoretic perspective,
S. Asoodeh, W.-N. Chen, F. P. Calmon, and A. Özgür, “Differentially private federated learning: An information- theoretic perspective,” in2021 IEEE international symposium on information theory (ISIT), pp. 344–349, IEEE, 2021
2021
-
[17]
Fedv: Privacy-preserving federated learning over vertically partitioned data,
R. Xu, N. Baracaldo, Y . Zhou, A. Anwar, J. Joshi, and H. Ludwig, “Fedv: Privacy-preserving federated learning over vertically partitioned data,” inProceedings of the 14th ACM workshop on artificial intelligence and security, pp. 181–192, 2021
2021
-
[18]
Differentially private federated learning: A client level perspective,
R. C. Geyer, T. Klein, and M. Nabi, “Differentially private federated learning: A client level perspective,”arXiv preprint arXiv:1712.07557, 2017
2017 arXiv
-
[19]
Dp-cda: An algorithm for enhanced privacy preservation in dataset synthesis through randomized mixing,
U. Saha, T. M. Tonoy, and H. Imtiaz, “Dp-cda: An algorithm for enhanced privacy preservation in dataset synthesis through randomized mixing,”arXiv preprint arXiv:2411.16121, 2024
2024 arXiv
-
[21]
Privacy-preserving non-negative matrix factorization with outliers,
S. Saha and H. Imtiaz, “Privacy-preserving non-negative matrix factorization with outliers,”ACM Transactions on Knowledge Discovery from Data, vol. 18, no. 3, pp. 1–26, 2024
2024
-
[22]
Approximating functions with approximate privacy for applications in signal estimation and learning,
N. Tasnim, J. Mohammadi, A. D. Sarwate, and H. Imtiaz, “Approximating functions with approximate privacy for applications in signal estimation and learning,”Entropy, vol. 25, no. 5, p. 825, 2023
2023
-
[24]
Gradient-based learning applied to document recognition,
Y . LeCun, L. Bottou, Y . Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324, 2002
2002
-
[25]
Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms,
H. Xiao, K. Rasul, and R. V ollgraf, “Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms,”arXiv preprint arXiv:1708.07747, 2017
2017 arXiv
-
[26]
Dppro: Differentially private high-dimensional data release via random projection,
C. Xu, J. Ren, Y . Zhang, Z. Qin, and K. Ren, “Dppro: Differentially private high-dimensional data release via random projection,”IEEE Transactions on Information Forensics and Security, vol. 12, no. 12, pp. 3081–3093, 2017
2017
-
[27]
Privacy-preserving data mining,
R. Agrawal and R. Srikant, “Privacy-preserving data mining,” inProceedings of the 2000 ACM SIGMOD international conference on Management of data, pp. 439–450, 2000
2000
-
[28]
Synthesizing differentially private datasets using random mixing,
K. Lee, H. Kim, K. Lee, C. Suh, and K. Ramchandran, “Synthesizing differentially private datasets using random mixing,” in2019 IEEE International Symposium on Information Theory (ISIT), pp. 542–546, IEEE, 2019
2019
-
[29]
Calibrating noise to sensitivity in private data analysis,
C. Dwork, F. McSherry, K. Nissim, and A. Smith, “Calibrating noise to sensitivity in private data analysis,” in Theory of Cryptography: Third Theory of Cryptography Conference, TCC 2006, New York, NY, USA, March 4-7,
2006
-
[30]
The algorithmic foundations of differential privacy,
C. Dwork, A. Roth,et al., “The algorithmic foundations of differential privacy,”Foundations and Trends® in Theoretical Computer Science, vol. 9, no. 3–4, pp. 211–407, 2014
2014
-
[31]
Mechanism design via differential privacy,
F. McSherry and K. Talwar, “Mechanism design via differential privacy,” in48th Annual IEEE Symposium on Foundations of Computer Science (FOCS’07), pp. 94–103, IEEE, 2007
2007
-
[32]
Rényi differential privacy,
I. Mironov, “Rényi differential privacy,” in2017 IEEE 30th computer security foundations symposium (CSF), pp. 263–275, IEEE, 2017. 9 APREPRINT- SEPTEMBER15, 2025 A Relevant Definitions, Theorems, and Proofs In this section, we present some definitions and theorems relevant to ...
2017
-
[2006]
265–284, Springer, 2006
Proceedings 3, pp. 265–284, Springer, 2006
2006
Reviewed August 4, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.