REVIEW 3 major objections 6 minor 21 references
Steps Adaptive Decay DPSGD: Enhancing Performance on Imbalanced Datasets with Differential Privacy with HAM10000
T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read A step-decay noise schedule lifts differentially private training on imbalanced skin images.
desk verdict A plausible schedule tweak with standard DP accounting, but the 1% accuracy claim rests on test-set tuning and a mismatched abstract; it needs a proper experiments pass before it earns trust. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the $\sigma$ clip estimation procedure (Algorithm 1): total training epochs $T$ are split into steps $D_0,\dots,D_n$ with $D_i\cdot\gamma=D_{i-1}$, and within each step the noise multiplier and clipping threshold are held constant while across steps $\sigma_i=\sigma_{i+1}\cdot\beta$ and $C_i=C_{i+1}\cdot a$. This produces a low-noise, high-clipping early phase and a high-noise, low-clipping late phase. The privacy accounting uses RDP composition (Proposition 1) to sum per-step RDP guarantees and converts the total to $(\epsilon,\delta)$-DP via Lemma 1, so the adaptive schedule still provides a formal privacy guarantee.
What would settle it
Re-run the HAM10000 comparison with hyperparameters selected on a held-out validation split (or via nested cross-validation) and report the test accuracies; if SAD-DPSGD no longer beats Auto-DPSGD-S at $\epsilon=3$, the central claim is not supported.
Extended reading notes
Core claim
The central claim is that the schedule of the noise multiplier and clipping threshold, not just their final values, controls how well differentially private training survives class imbalance. SAD-DPSGD divides training into $n$ steps with geometrically decreasing lengths, sets the noise multiplier to increase by a factor $\beta$ and the clipping threshold to decay by a factor $a$ across steps, and uses Rényi differential privacy composition to account for the changing noise. The result, the paper asserts, is that the model avoids the early suboptimal solutions caused by aggressive clipping of minority-class gradients and achieves higher accuracy than Auto-DPSGD and DPSGD at every privacy budget tested.
Load-bearing premise
The reported gain over Auto-DPSGD is not an artifact of choosing the decay parameters ($\beta,\gamma,n$) after seeing the test-set results; the paper does not describe a separate validation split.
Editorial extensions
If this is right
- SAD-DPSGD is a drop-in training schedule change: it requires no extra data, no change to model architecture, and no post-processing, only a decaying noise and clipping schedule.
- On HAM10000, SAD-DPSGD improves over Auto-DPSGD-S by 1.00, 0.99, and 0.80 percentage points at $\epsilon=3,8,16$ respectively with $\delta=10^{-3}$.
- The method lifts both majority- and minority-class accuracy relative to Auto-DPSGD-S, indicating that the benefit is not simply a majority-class artifact.
- The proposed parameter ranges ($\beta=0.8$, $\gamma=0.9$, $n=3$) give a starting point for applying the schedule to other imbalanced datasets.
Reading between the lines
- If the schedule effect is real, it suggests a 'privacy curriculum': spending privacy budget early, when minority-class gradients are most informative, may be more valuable than spreading it uniformly.
- A testable extension is to benchmark SAD-DPSGD on other imbalanced datasets (e.g., iNaturalist or UTKFace) with a fixed, pre-registered hyperparameter rule; currently the optimal $\beta,\gamma,n$ are chosen on HAM10000.
- The mechanism may interact with class-weighted loss or data augmentation; combining a step-decay privacy schedule with those data-side fixes could compound the gain.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes SAD-DPSGD, a variant of DP-SGD in which the noise multiplier and the clipping threshold decay according to a pre-determined, geometrically non-uniform step schedule. The motivation is that on small imbalanced datasets, uniform schedules cause minority-class gradients to be clipped too aggressively early in training. The authors provide an RDP-based privacy accounting for the varying noise schedule, then compare SAD-DPSGD against DPSGD and two Auto-DPSGD variants on HAM10000, reporting accuracy gains of about one percentage point at epsilon = 3, 8, and 16. The central claim is that this simple schedule change improves DP accuracy on imbalanced medical image data without additional data or architectural changes.
Significance. If the central claim held, SAD-DPSGD would be a practically valuable, drop-in modification to DP-SGD that improves accuracy on small imbalanced datasets while preserving a formal privacy guarantee. The RDP composition framework used for the pre-determined schedule is the standard and appropriate tool, and the paper reasonably documents a hyperparameter exploration across several baseline algorithms. However, the empirical evidence as presented is not sufficient to support the claim: the reported gains are small, hyperparameters are selected on the test set, results are single runs, and the abstract's headline number misstates the comparison. The privacy accounting also has a load-bearing gap concerning step multiplicity. These issues are addressable in revision but currently undermine the paper's main conclusions.
major comments (3)
- [Section IV.B / Algorithm 1] The RDP composition in Section IV.B sums rdp_t only over the n steps, but each step D_i contains D_i training iterations and the Gaussian mechanism is applied once per iteration. Since the per-iteration RDP is alpha/(2*sigma_i^2), the correct budget for a step is D_i * alpha/(2*sigma_i^2), and the total is sum_i D_i * alpha/(2*sigma_i^2). As written, the reported epsilon = 3, 8, 16 undercount privacy by roughly a factor of T/n, which is not a conservative error: it makes SAD-DPSGD appear more private than it actually is. Please correct the accounting and re-run the experiments with adjusted noise schedules, or clarify that each D_i denotes a single iteration and revise Algorithm 1 accordingly.
- [Section V.B / Table II] Hyperparameters beta, gamma, and n are selected by accuracy on the HAM10000 test set (Figs. 6-7 and Table I), and the same test set is then used to report the final comparison in Table II. No held-out validation split, cross-validation, or multiple-seed statistics are described. With reported advantages of only 0.80-1.00 percentage points over Auto-DPSGD-S, the observed gap is within the range that test-set selection among the swept hyperparameters could produce. Please provide an out-of-sample evaluation protocol and report mean plus/minus standard deviation over multiple runs.
- [Abstract / Section V.C and Table II] The abstract claims a 2.15% improvement under epsilon = 3.0, but Table II shows SAD-DPSGD at 80.68% versus Auto-DPSGD-S at 79.68%, a 1.00 percentage-point gain; the 2.15% figure is the gap to DPSGD (78.53%). The abstract should state the comparison correctly, and the same correction applies to any summary of the results.
minor comments (6)
- [Algorithm 2] The input parameter names 'clipping decay parameter beta, noise multiplier decay parameter gamma' are inconsistent with Algorithm 1, where beta is the sigma decay parameter and gamma is the step decay parameter; the call sigma clip estimation(T, t, gamma, beta, a, sigma_n, C_n) matches Algorithm 1, so the input list should be corrected.
- [Algorithm 2, line 6] C_j is undefined; the clipping threshold should be C_t, consistent with the noise scale used in line 8.
- [Algorithm 2, line 2] 'with probability |B⊔|/N' should read 'with probability B/N'.
- [Section V.A] The mean noise multiplier formula mu = (1/n) sum s_i*sigma_i should be normalized by sum s_i = T, not by n; otherwise the reported means are not the average noise multiplier over training.
- [Section V.A.3] 'Figure 6 shows the results' for the effect of n should refer to Figure 5; Figure 6 is the beta-performance plot.
- [References and in-text citations] The text contains raw LaTeX citation commands (e.g., \cite{19}, \cite{8}) and at least one cited work (DPAdaMod-AGC) does not appear in the reference list; the bibliography and citation commands should be cleaned up.
Circularity Check
No significant circularity: the privacy-accounting derivation is standard RDP composition from external results, and the empirical comparisons are measurements rather than predictions derived from fitted parameters.
full rationale
The paper's derivation chain is the privacy-budget argument in Section IV.B: each step uses the Gaussian mechanism with sensitivity C_t and noise scale C_t * sigma_t, giving (alpha, alpha/(2*sigma_t^2))-RDP via Definition 5; Proposition 1 from Mironov's RDP work justifies summing the per-step RDP values; and Lemma 1 converts the total to (epsilon, delta)-DP. This chain uses standard external results and does not feed the accuracy numbers back into the privacy computation; it is self-contained and independent of the empirical comparisons. There are no load-bearing self-citations by the present authors, no imported uniqueness theorem, and no ansatz smuggled in via citation. The empirical claim in Section V.C is an external measurement on HAM10000, not a fitted parameter renamed as a prediction. The main validity concern is that beta, gamma, and n are selected in Section V.B by maximizing accuracy on HAM10000 and then the same dataset is used for the Table II comparison, with no described validation split or error bars; this is a real selection-bias and experimental-design concern, but it is not circularity of the derivation. Section VI likewise concedes, as a limitation, that optimal hyperparameters may vary across datasets. The abstract's 2.15% improvement figure is internally inconsistent with Table II, where the gain over Auto-DPSGD-S is 1.00 percentage point and 2.15 percentage points is the gain over plain DPSGD; this is an accuracy/consistency error, not a circular step. Overall, the privacy derivation is first-principles and the empirical result is an external measurement, so no significant circularity is present.
Assumptions & free parameters
free parameters (6)
- beta (noise multiplier decay parameter) =
0.8
- gamma (step decay parameter) =
0.9
- n (number of steps) =
3
- a (clipping threshold decay parameter) =
not reported
- sigma_n (final noise multiplier) =
not reported
- C_n (final clipping threshold) =
not reported
assumptions (5)
- standard math Per-step RDP composition via Proposition 1 (Mironov 2017) with sum of per-step RDP values.
- standard math Gaussian mechanism with sensitivity C_t and noise std sigma_t C_t satisfies (alpha, alpha/(2 sigma_t^2))-RDP.
- domain assumption The clipping threshold C_t bounds the l2 sensitivity of each per-sample gradient at step t.
- domain assumption RDP analysis without subsampling amplification provides a valid upper bound on privacy loss for the Poisson-sampled batches.
- ad hoc to paper The non-uniform step schedule (D_i gamma = D_{i-1}) and geometric decay of sigma and C improve learning on imbalanced data.
Cite this review
Pith. "Pith review of Steps Adaptive Decay DPSGD: Enhancing Performance on Imbalanced Datasets with Differential Privacy with HAM10000." pith.science (2026). https://pith.science/paper/ZW2XBIAF
@misc{pith2026250706619,
author = {Pith},
title = {Pith review of: Steps Adaptive Decay DPSGD: Enhancing Performance on Imbalanced Datasets with Differential Privacy with HAM10000},
year = {2026},
howpublished = {\url{https://pith.science/paper/ZW2XBIAF}},
note = {Machine review of arXiv:2507.06619}
}
abstract
When applying machine learning to medical image classification, data leakage is a critical issue. Previous methods, such as adding noise to gradients for differential privacy, work well on large datasets like MNIST and CIFAR-100, but fail on small, imbalanced medical datasets like HAM10000. This is because the imbalanced distribution causes gradients from minority classes to be clipped and lose crucial information, while majority classes dominate. This leads the model to fall into suboptimal solutions early. To address this, we propose SAD-DPSGD, which uses a linear decaying mechanism for noise and clipping thresholds. By allocating more privacy budget and using higher clipping thresholds in the initial training phases, the model avoids suboptimal solutions and enhances performance. Experiments show that SAD-DPSGD outperforms Auto-DPSGD on HAM10000, improving accuracy by 2.15% under $\epsilon = 3.0$ , $\delta = 10^{-3}$.
Figures
Figures from the paper (5 more)
Reference graph
Works this paper leans on
-
[1]
Neither private nor fair: Impact of data imbalance on utility and fairness in differential privacy,
T. Farrand, F. Mireshghallah, S. Singh, and A. Trask, “Neither private nor fair: Impact of data imbalance on utility and fairness in differential privacy,” in Proceedings of the 2020 workshop on privacy-preserving machine learning in practice , 2020, pp. 15–19
work page 2020
-
[2]
Reconciling privacy and accuracy in ai for medical imaging,
A. Ziller, T. T. Mueller, S. Stieger, L. F. Feiner, J. Brandt, R. Braren, D. Rueckert, and G. Kaissis, “Reconciling privacy and accuracy in ai for medical imaging,” Nature Machine Intelligence , vol. 6, no. 7, pp. 764–774, 2024
work page 2024
-
[3]
The ham10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions,
P. Tschandl, C. Rosendahl, and H. Kittler, “The ham10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions,” Scientific data, vol. 5, no. 1, pp. 1–9, 2018
2018
-
[4]
Differential Privacy Under Class Imbalance: Methods and Empirical Insights
L. Rosenblatt, Y . Lut, E. Turok, M. Avella-Medina, and R. Cummings, “Differential privacy under class imbalance: Methods and empirical insights,” arXiv preprint arXiv:2411.05733 , 2024
work page Pith review arXiv 2024
-
[5]
Adaptive privacy preserving deep learning algorithms for medical data,
X. Zhang, J. Ding, M. Wu, S. T. Wong, H. Van Nguyen, and M. Pan, “Adaptive privacy preserving deep learning algorithms for medical data,” in Proceedings of the IEEE/CVF winter conference on applications of computer vision, 2021, pp. 1169–1178
work page 2021
-
[6]
S. Chen and W. Liang, “A study on adaptive gradient clipping algorithms for differential privacy: Enhancing cyber security and trust,” in 2024 IEEE Cyber Science and Technology Congress (CyberSciTech) . IEEE, 2024, pp. 100–106
work page 2024
-
[7]
Deep learning with differential privacy,
M. Abadi, A. Chu, I. Goodfellow, H. B. McMahan, I. Mironov, K. Talwar, and L. Zhang, “Deep learning with differential privacy,” in Proceedings of the 2016 ACM SIGSAC conference on computer and communications security, 2016, pp. 308–318
2016
-
[8]
Learning rate adaptation for differentially private learning,
A. Koskela and A. Honkela, “Learning rate adaptation for differentially private learning,” in International Conference on Artificial Intelligence and Statistics. PMLR, 2020, pp. 2465–2475
work page 2020
Show all 21 references
-
[9]
Dynamic differential- privacy preserving sgd,
J. Du, S. Li, X. Chen, S. Chen, and M. Hong, “Dynamic differential- privacy preserving sgd,” arXiv preprint arXiv:2111.00173 , 2021
2021 arXiv
-
[10]
On the convergence and calibration of deep learning with differential privacy,
Z. Bu, H. Wang, Z. Dai, and Q. Long, “On the convergence and calibration of deep learning with differential privacy,” Transactions on machine learning research, vol. 2023, pp. https–openreview, 2023
2023
-
[11]
Disparate im- pact in differential privacy from gradient misalignment,
M. S. Esipova, A. A. Ghomi, Y . Luo, and J. C. Cresswell, “Disparate im- pact in differential privacy from gradient misalignment,” arXiv preprint arXiv:2206.07737, 2022
2022 arXiv
-
[12]
Dpadamod agc: Adaptive gradient clipping-based differential privacy,
J. Zhang, W. Yang, Y . Zhang, H. Zheng, and T. Zhang, “Dpadamod agc: Adaptive gradient clipping-based differential privacy,” in 2024 27th International Conference on Computer Supported Cooperative Work in Design (CSCWD). IEEE, 2024, pp. 950–955
2024
-
[13]
Differential privacy has disparate impact on model accuracy,
E. Bagdasaryan, O. Poursaeed, and V . Shmatikov, “Differential privacy has disparate impact on model accuracy,” Advances in neural informa- tion processing systems , vol. 32, 2019
2019
-
[14]
Diversity in faces,
M. Merler, N. Ratha, R. S. Feris, and J. R. Smith, “Diversity in faces,” arXiv preprint arXiv:1901.10436 , 2019
1901 arXiv
-
[15]
Privacy in deep learning: A survey,
F. Mireshghallah, M. Taram, P. Vepakomma, A. Singh, R. Raskar, and H. Esmaeilzadeh, “Privacy in deep learning: A survey,” arXiv preprint arXiv:2004.12254, 2020
2004 arXiv
-
[16]
Demographic dialectal variation in social media: A case study of african-american english,
S. L. Blodgett, L. Green, and B. O’Connor, “Demographic dialectal variation in social media: A case study of african-american english,” arXiv preprint arXiv:1608.08868 , 2016
2016 arXiv
-
[17]
The inaturalist species classifi- cation and detection dataset,
G. Van Horn, O. Mac Aodha, Y . Song, Y . Cui, C. Sun, A. Shepard, H. Adam, P. Perona, and S. Belongie, “The inaturalist species classifi- cation and detection dataset,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2018, pp. 8769–8778
2018
-
[18]
Differential privacy: A survey of results,
C. Dwork, “Differential privacy: A survey of results,” in International conference on theory and applications of models of computation . Springer, 2008, pp. 1–19
2008
-
[19]
Calibrating noise to sensitivity in private data analysis,
C. Dwork, F. McSherry, K. Nissim, and A. Smith, “Calibrating noise to sensitivity in private data analysis,” in Theory of Cryptography: Third Theory of Cryptography Conference, TCC 2006, New York, NY, USA, March 4-7, 2006. Proceedings 3 . Springer, 2006, pp. 265–284
2006
-
[20]
R ´enyi divergence and kullback-leibler divergence,
T. Van Erven and P. Harremos, “R ´enyi divergence and kullback-leibler divergence,” IEEE Transactions on Information Theory , vol. 60, no. 7, pp. 3797–3820, 2014
2014
-
[21]
R ´enyi differential privacy,
I. Mironov, “R ´enyi differential privacy,” in 2017 IEEE 30th computer security foundations symposium (CSF) . IEEE, 2017, pp. 263–275
2017
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.