{"id":"0d07d91e-d8cc-400f-9b0c-04346d942439","arxiv_id":"2509.03341","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"Under identical differential privacy budgets, GANs show less membership inference leakage than diffusion models in the paper's MNIST experiments, a gap the authors attribute to training stability.","lead":"This paper compares membership inference leakage in GANs versus diffusion models when both are trained under differential privacy, and reports that GANs leak less. It backs the claim with a stability-based theory and a single MNIST experiment.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The theoretical gap is unsupported: Lemma 2's coupled-noise expected bound is used as a pointwise uniform-stability bound, and the independent-noise term dropped in Eq. 85 may dominate at fixed epsilon.","rationale":"Good-faith reading: the paper asks a real question and its empirical setup is reasonable; the MNIST results show a small AUC gap in the predicted direction at epsilon=5 and epsilon=1, though without error bars. The load-bearing issue is theoretical: the proof of the headline gap is an upper-bound sandwich. Even granting Lemmas 3 and 4, a larger upper bound on beta does not imply a larger actual beta or larger ADV; the omitted independent-noise term in Eq. 85 is exactly the term relevant to real DP-SGD, and it depends on sigma values that the paper does not report per architecture. The expectation-versus-pointwise mismatch is a separate but compounding gap: Theorem 1's margin argument needs a worst-case stability constant, but Lemma 2 supplies only an expectation. A revised manuscript would need either a high-probability stability lemma or a direct bound on ADV from expected stability, plus an explicit check of the O(alpha sigma sqrt(T)) term. I therefore keep the reader's reject verdict.","tokens_in":16696,"tokens_out":6827,"duration_ms":67113,"concrete_test":"Run the actual DP-SGD configurations from Appendix E on neighboring MNIST splits with independent noise draws, estimate beta_GAN and beta_Diff as E|loss_D - loss_D\\i|, and compare these measured values with the two terms in Eq. 85 (C/m sum alpha_t vs alpha sigma sqrt(T)) at epsilon = 10, 5, 1. If the measured ratio is not much greater than 1, or if the independent-noise term dominates, the Section 3.4 conclusion fails. Also check whether the pointwise margin probability in Theorem 1 is controlled by the expected stability constant; if not, a high-probability stability lemma is required.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central inference in Section 3.4 is an upper-bound comparison, not a proof of an actual gap. Lemma 2 (Eq. 16) bounds beta as sup_{z,i} E|...| under coupled DP-SGD runs, so the DP noise sigma cancels; Appendix D, Eq. 85 acknowledges that independent noise, the regime of real deployments and of the reported experiments, adds a term O(alpha sigma sqrt(T)). The paper never shows that this term is negligible, and because the DP accountant sets different sigma for the two architectures (with diffusion's K=32 noise-multiplicity changing the per-step sensitivity), the ordering beta_Diff >> beta_GAN is not established. Independently, Lemma 2 is an expected-stability statement, while Theorem 1 and Lemma 1 require a pointwise uniform bound |s_fD(x) - s_fD\\i(x)| <= L_s beta for every x; expected closeness does not control the threshold-margin event used in the proof of Theorem 1. Thus the chain from DP-SGD to ADV_MIA has a gap that precedes the GAN/diffusion comparison.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper studies membership inference leakage in differentially private GANs and diffusion models. The authors propose a theoretical framework based on uniform stability: they connect attack-score stability to loss stability (Properties 1-2, Lemma 1), bound membership advantage by 2Q L_s β for threshold attacks (Theorem 1), and derive DP-SGD stability bounds for GAN discriminators and diffusion denoisers (Lemmas 2-4). They conclude that the diffusion model's weighted multi-pass denoising objective yields β_Diff >> β_GAN and hence a larger upper bound on the MIA advantage, explaining an empirically observed privacy gap. The empirical section compares shadow-model MIAs on MNIST for DP-GANs and DP-diffusion models at ε ∈ {1,5,10,∞}, reporting attack metrics and FID.","tokens_in":16960,"tokens_out":7025,"duration_ms":64724,"significance":"If the central claim were established, the paper would be significant: it would show that model architecture and training dynamics, not just the DP budget, shape membership leakage, and it would provide a concrete stability-based explanation. The paper also contributes a reproducible shadow-model attack pipeline for private generative models and a clean MNIST comparison. However, the theoretical argument as presented does not support the claim: the comparison of upper bounds is not a proof of a gap in actual leakage, and several technical steps (pointwise versus expected stability, coupled versus independent noise, unmeasured constants) are unresolved. The empirical results are restricted to MNIST and confounded by large FID differences between the two families.","major_comments":[{"comment":"The inference from larger upper bounds to larger actual leakage is invalid. Eq. (38) and the following paragraph compare upper bounds on β and then on ADV_MIA, concluding that the 'upper bound on ADVDiff MIA is significantly larger than that of ADVGAN MIA' and that this 'provides a theoretical explanation of greater membership leakage.' A larger upper bound does not imply a larger actual value; without lower bounds on the adversary's success or a direct comparison of the leakage quantities, the empirical gap is not explained by these inequalities. This is load-bearing because the paper's central claim is that the theory explains the observed leakage gap.","section":"Section 3.4, after Eq. (38)"},{"comment":"There is a mismatch between the stability notion used in Lemma 2 and the one required by Theorem 1. Lemma 2 bounds the supremum over z,i of the expected absolute loss difference, i.e., an expected-stability quantity of the form sup_{z,i} E|ℓ(f_D,z) - ℓ(f_{D\\i},z)|. In contrast, Definition 3 and Lemma 1 require a pointwise uniform bound |ℓ(f_D,z) - ℓ(f_{D\\i},z)| ≤ β for all z. The proof of Theorem 1 uses the pointwise version in the margin argument (Eqs. (75)-(77)): the indicator functions can differ only when the score lies within L_s β of τ, which requires a deterministic bound on |s_{f_D}(x) - s_{f_{D\\i}}(x)|. The expectation bound from Lemma 2 does not control this event, so the chain from Lemma 2 to Theorem 1 is broken.","section":"Lemma 2 (Eq. 16) and Theorem 1 (Eq. 15)"},{"comment":"The coupled-randomness assumption in Lemma 2 makes the DP noise scale σ cancel, and the main text states that the bound reflects the sensitivity of the algorithm rather than the effect of noise. Appendix D, Eq. (85), acknowledges that with independent noise, which is what real DP-SGD and the paper's own experiments use, the parameter divergence acquires an additional random-walk term O(α σ √T). Because the σ needed to achieve a fixed (ε,δ) differs between the GAN and the diffusion model (e.g., due to the noise multiplicity K=32 in Eq. (5)), the omitted term may dominate and could reverse the claimed ordering β_Diff >> β_GAN. The paper provides no argument or measurement showing that the uncoupled term is negligible.","section":"Appendix D, Eq. (85)"},{"comment":"The proof of Theorem 1 contains an unjustified distributional step. The first equality identifies the member probability with E_{xi}[I{s(xi)≤τ}] and the non-member probability with E_x[I{s'(x)≤τ}], where s' is the score of the model trained on D\\i. The subsequent inequality replaces both expectations by an expectation over x∼P, but the member xi is drawn from the empirical distribution of the training set D, not from P, and the non-member probability in the definition of ADVMIA is evaluated with the same model f_D, not with f_{D\\i}. The bound in Eq. (74) is therefore not justified as written; a correct proof or a precise reference for this coupling step is needed.","section":"Appendix B, proof of Theorem 1, Eqs. (72)-(77)"},{"comment":"The quantitative comparison relies on several unmeasured constants: the shared Lipschitz constant L, the prediction-error bound B, the average EDM weight λ̄, the score-density bound Q, and the loss interval [a,b]. In particular, Theorem 1's bound is 2Q L_s β, and Q is not shown to be comparable between GANs and diffusion models; if the diffusion score density is more concentrated, the advantage bound could be smaller despite a larger β. Without measuring or bounding these quantities, the conclusion β_Diff >> β_GAN in Eq. (38) is not established.","section":"Section 3.4 and Theorem 1"}],"minor_comments":[{"comment":"The phrase 'in under comparable conditions' should read 'under comparable conditions'.","section":"Section 1"},{"comment":"For ε=∞, the GAN AUC is 0.74 and the diffusion AUC is 0.75, so the claim of a 'marked privacy robustness gap' is not supported by the AUC at the non-private setting; the differences at ε=1 are also small (GAN AUC 0.49, diffusion AUC 0.52). The paper does not report confidence intervals, which would help assess whether the reported gaps are meaningful.","section":"Table 1"},{"comment":"The FID gap between GANs and diffusion models is large at every ε (e.g., 183.2 vs. 72.9 at ε=1). If the GAN outputs are of much lower quality, the attack score distributions may be trivially different for reasons unrelated to membership leakage, confounding the empirical privacy comparison. This should be discussed as a limitation.","section":"Table 2"},{"comment":"The bound in Eq. (16) contains a factor 2 that does not appear in the proof of Lemma 2: Eq. (21) gives LC/m Σ α_t, while Eq. (16) states 2LC/m Σ α_t. Please reconcile this constant.","section":"Lemma 2"}],"recommendation":"reject","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper asks a real question — under the same DP budget, do GANs or diffusion models leak more membership information? — and the empirical trend it reports may well be real. But the theory does not prove the headline claim. Section 3.4 compares upper bounds on membership advantage, and a larger upper bound does not imply a larger actual value. Worse, Lemma 2's stability bound is an expected bound under coupled randomness; Theorem 1 needs a pointwise uniform bound, and the independent-noise term that Eq. 85 drops can dominate at a fixed epsilon. So beta_Diff >> beta_GAN is not established.\n\nWhat is genuinely new: this is the first work I know that studies MIA leakage specifically for DP-trained GANs versus diffusion models. The stability framing — weighted multi-pass denoising objectives amplify per-sample sensitivity — is a plausible mechanism and worth testing. The empirical setup (shadow models, consistent privacy accounting) is sensible, and the observation that diffusion retains non-trivial leakage at epsilon=1 while GANs drop to near-random is interesting, even if it's just MNIST and no error bars are given.\n\nThe soft spots are load-bearing, not cosmetic. The proof of the gap is a comparison of upper bounds with unmeasured constants (Q, B, L, lambda_bar) assumed comparable. The bounded-density assumption is plausible but untested. The empirical section is thin: one dataset, small AUC differences, no variance estimates. To the authors' credit, they admit the coupled-noise limitation in Appendix D, even though they wave it away. The paper is not sloppy in the usual sense; the logic is clear, just overreaching.\n\nWho gets value from this? Practitioners choosing between private GANs and private diffusion get a warning that epsilon alone doesn't cap membership risk. Theorists get a setup with a potential fix: prove or disprove the stability gap with actual pointwise bounds, or measure the constants.\n\nRecommendation: this deserves a serious referee, but with expected major revision. If the empirical trend holds on more datasets and the theory is reframed as a heuristic or fixed with a real bound, it would be a decent paper. As written, the central claim overstates what is established.","headline":"A plausible empirical observation wrapped in an overreaching theoretical claim; the stability-gap comparison is not established as written.","tokens_in":17464,"tokens_out":3208,"would_cite":false,"duration_ms":29136,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68P27","68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"DP-diffusion models leak more membership information than DP-GANs under identical privacy budgets, because their denoising objective is less uniformly stable.","keywords":["membership inference attacks","differential privacy","generative adversarial networks","diffusion models","uniform stability","DP-SGD","privacy leakage","attacker advantage"],"falsifier":"Train both architectures with DP-SGD under the same $\\varepsilon$ but with independent Gaussian noise (the standard setting), measure the actual MIA advantage on a fixed dataset, and compute the uncoupled bound of Eq. (85) with the actual $T$, $\\alpha$, and $\\sigma$. If the diffusion advantage does not exceed the GAN advantage once the noise term $\\alpha\\sigma\\sqrt{T}$ is accounted for, the paper's central claim is refuted.","tokens_in":16518,"feed_emoji":"🔒","tokens_out":6727,"duration_ms":55015,"temperature":0.7,"pith_summary":"The paper sets out to explain a privacy puzzle: when GANs and diffusion models are both trained with differential privacy, why does one leak more about its training data than the other? It claims that the answer is structural, not a matter of tuning. Diffusion models are trained on a noise-weighted denoising objective in which low-noise terms carry very large weights, and this makes their outputs far less stable when a single training example changes; GANs, whose discriminator is the only private component and trains on a logistic loss, stay much more stable. The authors translate stability into an upper bound on membership-inference advantage, and show experimentally on MNIST that the gap persists across privacy budgets, with diffusion models retaining non-trivial leakage even at $\\varepsilon=1$ while GAN attacks fall to near-random. If the claim is right, the privacy budget $\\varepsilon$ alone is insufficient to compare private generative models; architecture choice is a first-order privacy lever.","feed_headline":"DP diffusion models leak more membership data than DP GANs","feed_subtitle":"Same privacy budget, different real risk: low-noise denoising weights undermine stability.","key_machinery":"The load-bearing object is the uniform-stability coefficient $\\beta$ of the DP-SGD-trained model, bounding how much a per-sample loss can change when one training point is removed. Lemma 2 gives $\\beta \\le \\frac{2LC}{m}\\sum_t \\alpha_t$ under coupled randomness; Lemmas 3 and 4 instantiate it for GANs and diffusion models, with the diffusion bound carrying an extra factor $\\bar{\\lambda}$, the average of the EDM noise-weighting function. Theorem 1 then converts score stability into a membership-advantage bound $\\mathrm{ADV}_{\\mathrm{MIA}} \\le 2QL_s\\beta$, where $Q$ is the maximum score density and $L_s$ the Lipschitz constant between attack score and training loss.","core_discovery":"The central claim is that, at the same DP-SGD budget, the expected uniform stability of a diffusion denoiser is far weaker than that of a GAN discriminator, written $\\beta_{\\mathrm{Diff}} \\gg \\beta_{\\mathrm{GAN}}$, and that attacker advantage on any threshold membership-inference attack is bounded by $2QL_s\\beta$. The paper derives this ordering from a general stability bound $\\beta \\le \\frac{2LC}{m}\\sum_{t=1}^{T}\\alpha_t$: the diffusion loss has a large Lipschitz constant $L_D \\approx 2\\bar{\\lambda}LB$ because the EDM weighting $\\lambda(\\sigma)=\\frac{\\sigma^2+\\sigma_{\\mathrm{data}}^2}{(\\sigma\\sigma_{\\mathrm{data}})^2}$ blows up at small noise levels, and diffusion models run far more DP-SGD steps than GAN discriminators ($T_D \\gg T_G$). The same privacy parameter therefore corresponds to different real privacy risks across architectures.","pith_inferences":["The stability ordering relies on coupling DP-SGD noise between neighboring runs; with independent noise, an extra term of order $\\alpha\\sigma\\sqrt{T}$ appears, and at the large noise scales needed for small $\\varepsilon$ this term could dominate and reverse the ordering. Testing the bound under uncoupled noise is a direct extension.","The bound contains $Q$ and $L_s$, which are score- and architecture-dependent; the empirical gap might shrink or widen with different attack scores (e.g., loss-based vs. logit-based) or different architectures.","A natural testable extension is to run the same MIA pipeline on larger image datasets and at smaller $\\varepsilon$; the theory predicts the diffusion-vs-GAN gap persists as long as the low-noise EDM weights remain large.","The privacy-accounting implication is that architectures with high loss Lipschitz constants may need either more noise or fewer steps to reach the same true risk, suggesting that per-architecture noise calibration could replace fixed $\\varepsilon$ budgets."],"forward_implications":["The same reported $(\\varepsilon,\\delta)$ budget does not imply the same membership-inference exposure across generative architectures.","DP-GANs, whose only private component is the discriminator, keep MIA advantage near random at moderate budgets ($\\varepsilon=10$), while DP-diffusion models retain measurable leakage even at $\\varepsilon=1$.","Privacy evaluation of generative models should report an empirical MIA metric alongside $\\varepsilon$, because $\\varepsilon$ alone can be misleading.","There is a fidelity-privacy trade-off: the better sample quality of diffusion models comes with higher membership leakage under DP."],"supporting_citations":[{"why":"Supplies the definition of uniform stability that underlies the paper's privacy-leakage analysis.","marker":"Bousquet and Elisseeff 2002"},{"why":"Supplies the SGD stability bound that Lemma 2 extends to DP-SGD with per-sample clipping.","marker":"Hardt, Recht, and Singer 2016"},{"why":"Supplies DP-SGD and the moment accountant used to enforce and track differential privacy in both model families.","marker":"Abadi et al. 2016"},{"why":"Supplies the EDM formulation and the noise-weighting function $\\lambda(\\sigma)$ that drives the diffusion instability argument.","marker":"Karras et al. 2022"},{"why":"Supplies the noise-multiplicity diffusion training objective used in both the stability bound and the experiments.","marker":"Dockhorn et al. 2023"},{"why":"Supplies the attacker-advantage definition and the stability-to-leakage link that Theorem 1 extends to score-based threshold attacks.","marker":"Yeom et al. 2018"},{"why":"Supplies the empirical observation that non-private diffusion models leak more than GANs and the denoising-loss attack score used for diffusion models.","marker":"Carlini et al. 2023"},{"why":"Supplies the private-GAN training recipe and the discriminator/generator update balancing used in the experiments.","marker":"Bie, Kamath, and Zhang 2023"}],"fun_headline_variants":["Diffusion Leaks More Under DP Than GANs","GANs Win DP Privacy: Diffusion Leaks","DP MIA: Diffusion Worse Than GANs","Diffusion's Noise Weakens DP Stability","Stability Gap: GANs Safer Under DP"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's main ordering $\\beta_{\\mathrm{Diff}} \\gg \\beta_{\\mathrm{GAN}}$ uses the coupled-noise stability bound of Lemma 2, in which the DP noise cancels between neighboring runs; the uncoupled-noise term of order $\\alpha\\sigma\\sqrt{T}$ noted in Appendix D is not included in the comparison, and if that term dominates at realistic noise scales the claimed ordering need not hold.","fun_headline_variants_meta":{"raw":{"variants":["Diffusion Leaks More Under DP Than GANs","GANs Win DP Privacy: Diffusion Leaks","DP MIA: Diffusion Worse Than GANs","Diffusion's Noise Weakens DP Stability","Stability Gap: GANs Safer Under DP"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000333,"raw_usage":{"total_tokens":1828,"prompt_tokens":898,"completion_tokens":930,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":514,"completion_tokens_details":{"reasoning_tokens":855}},"tokens_in":514,"tokens_out":930,"duration_ms":8980,"temperature":1.0,"reasoning_tokens":855,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T16:30:44.688359+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train both architectures with DP-SGD under the same $\\varepsilon$ but with independent Gaussian noise (the standard setting), measure the actual MIA advantage on a fixed dataset, and compute the uncoupled bound of Eq. (85) with the actual $T$, $\\alpha$, and $\\sigma$. If the diffusion advantage does not exceed the GAN advantage once the noise term $\\alpha\\sigma\\sqrt{T}$ is accounted for, the paper's central claim is refuted.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the definition of uniform stability that underlies the paper's privacy-leakage analysis."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the noise-multiplicity diffusion training objective used in both the stability bound and the experiments."}],"review_version":2}