{"id":"2f8ce81b-6136-4b37-a3d9-668d06c6f4da","arxiv_id":"2509.06264","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"PLRV-O replaces Gaussian noise in DP-SGD with a randomized-scale Laplace distribution and claims large accuracy gains at epsilon under 1, but the privacy accounting likely underestimates the true privacy loss due to shared subsampling across model parameters.","lead":"This paper proposes a new way to add noise during private deep learning training, using Laplace-type noise with a randomly chosen scale from a Gamma distribution, and claims much higher accuracy at strong privacy levels. The key open question is whether the mathematical privacy accounting is actually valid.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 4.9's multivariate PLRV moments bound is invalid: summing per-coordinate univariate MAFs under-estimates the true joint MAF because scale and subsampling are shared across coordinates.","rationale":"The reader's weakest assumption identifies the same failure: the multivariate PLRV accountant sums per-coordinate subsampled univariate bounds although the scale b and the batch subsampling are common to all coordinates, making the per-coordinate privacy losses positively correlated. I checked the mechanism definitions and Theorem 4.9 and find the concern load-bearing. The exact expression conditional on b factors over coordinates, but the unconditional expectation of the product cannot be replaced by the product of unconditional expectations; positive association gives the opposite inequality, so Eq. (19) is an under-estimate, not an upper bound. This is not an 'outside consensus' disagreement; it is an internal proof gap. The empirical audits and public code are useful, but they provide only lower bounds on ε and cannot establish the DP guarantee asserted. The reported utility gains at claimed ε values are therefore not currently substantiated. The paper should be revised with either a corrected multivariate accountant (e.g., numerical PLD composition over the actual shared-scale mechanism) or a mechanism whose coordinates are genuinely independent; until then, the central claim fails. No change to the reader's REJECT verdict is needed.","tokens_in":39461,"tokens_out":9220,"duration_ms":95159,"concrete_test":"Numerically compute the exact MAF for a toy PLRV mechanism with n=2, C=1, u~Gamma(k=5, θ=0.1), ζ=0.05. Let p0(x,y)=∫ f(b)(2b)^{-2} exp(-(|x|+|y|)/b) db and p1(x,y)=(1-ζ)p0(x,y)+ζ∫ f(b)(2b)^{-2} exp(-(|x-1|+|y|)/b) db. For λ=1,...,9 compute α_true(λ)=log E_{p0}[(p1/p0)^{λ+1}] = log ∫ p0 (p1/p0)^{λ+1} dx dy via high-resolution quadrature. Compare with Eq. (19) evaluated at the same λ with x_1=1, x_2=√2−1 and M_u(t)=(1−tθ)^{-k} for t<1/θ. If α_true(λ) > Eq. (19) for any λ, the proposed bound is not an upper bound.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central privacy guarantee rests on Eq. (19) in Theorem 4.9, which bounds the multivariate PLRV mechanism's moments accountant by the sum of the univariate bounds from Theorem 4.8. This is not a valid upper bound. For one DP-SGD iteration, the true privacy loss is log(μ1/μ0) evaluated on the full output vector, where μ0 and μ1 are densities of the mechanism on adjacent datasets. These densities are mixtures over the single random scale b and over a single shared subsampling draw. Conditional on b, coordinates are independent, so the exact MAF is log E_b[∏_i E_i(b)] (with the subsampling event also shared). The paper instead uses Σ_i log E_b[E_i(b)] = log ∏_i E_b[E_i(b)]. Since the factors E_i(b) are comonotone functions of the shared b and the shared inclusion indicator, E_b[∏_i E_i(b)] ≥ ∏_i E_b[E_i(b)]; equality holds only if b (and the batch draw) were independent across coordinates, which is not the mechanism in Definitions 4.6/4.7. The 'composition over coordinates' gloss in §4.1 would require conditional per-coordinate MAFs, not marginal univariate ones. Consequently Eq. (19) under-estimates privacy loss, and all reported ε values—including the headline 94.03% at ε≈0.5 and 92.20% at ε≈0.2—are not supported by the proof. The privacy audit in §6.4 gives only empirical lower bounds and cannot certify the claimed guarantee.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces PLRV-O, a framework for DP-SGD in which the noise distribution is a multivariate Laplace mechanism whose scale parameter is itself randomized (Definition 4.6). The scale is drawn from a Gamma distribution, giving two tunable parameters (k, theta) in addition to the clipping threshold C. The authors derive a moments accountant for this mechanism (Theorem 4.9), apply majorization to support ℓ2 clipping, and then optimize the parameters with a constrained nonlinear solver. The empirical sections report large utility improvements at very small ε: e.g., 94.03% CIFAR-10 ViT accuracy at ε≈0.5 and 92.20% SST-2 RoBERTa-large accuracy at ε≈0.2, compared to Gaussian baselines. The central claim is that PLRV-O achieves these results under valid (ε, δ)-DP guarantees with a tight accountant.","tokens_in":39874,"tokens_out":5455,"duration_ms":70452,"significance":"If the privacy accounting were sound, this would be a significant contribution: it proposes a genuinely non-Gaussian DP-SGD noise family, gives a closed-form moments accountant, and demonstrates substantial utility gains at strong privacy levels across both vision and language tasks. The paper includes source code and extensive experiments, and the attempt to optimize the privacy loss random variable itself is an interesting angle. However, the central privacy bound is not a valid upper bound on the true joint moments accountant. Since the headline ε values and all comparisons depend on that bound, the main scientific claim is currently unsupported.","major_comments":[{"comment":"Eq. (19) bounds the multivariate PLRV mechanism's moments accountant by the sum of per-coordinate univariate moment bounds. This is not a valid upper bound, because the PLRV mechanism in Definitions 4.6–4.7 draws a single scale b for the whole vector, and the DP-SGD subsampling event is shared across all coordinates. For one iteration, after the binomial expansion the true subsampled moment contains a term of the form E_b[∏_i F_i(b)] (where the product is over coordinates and F_i depends on the coordinate's majorized sensitivity x_i and the same random b), while the proof uses ∏_i E_b[F_i(b)]. The functions F_i(b) are comonotone (decreasing) in b, so E_b[∏_i F_i(b)] ≥ ∏_i E_b[F_i(b)]. Thus Eq. (19) underestimates the privacy loss. The 'composition over coordinates' argument in §4.1 would require releasing coordinates with independent randomness; it does not apply to a one-shot vector-val","section":"Theorem 4.5, Eq. (16)"},{"comment":"The same flaw already appears in Theorem 4.5 for the standard multivariate Laplace mechanism: Eq. (15) sets the total MAF to Σ_i α_{g_i}(λ), and Eq. (16) bounds it by summing per-coordinate univariate subsampled Laplace bounds. Conditional on the scale b, coordinates are independent, but the batch-inclusion event is common to all coordinates, so the mixture induced by subsampling does not factor across coordinates. Even for fixed b, the joint moment is log E_{(I,z)}[(1−ζ+ζ r(z))^{λ+1}] with r(z) = exp(Σ_i (|z_i| − |z_i−g_i|)/b), not the sum of per-coordinate log-moments. Consequently the resulting bound is not an upper bound. Since Theorem 4.9 inherits this structure from Theorem 4.5, the central privacy guarantee of the paper is unsupported.","section":null},{"comment":"The privacy audit reports only empirical lower bounds on ε under specific attacks (ClipBKD), with a finite number of trials. Such an audit cannot certify the claimed (ε, δ)-DP guarantees. The validation of the headline privacy numbers must come from the moments accountant, which is exactly the step that fails. The audit therefore does not mitigate the flaw in Theorem 4.9.","section":"§6.4"}],"minor_comments":[{"comment":"The phrase 'Mironov et al. [42] and Sander et al. [42, 51]' appears to have a citation error: reference [42] is listed twice. Please correct.","section":"§1"},{"comment":"The proof refers to 'Proof B.2 (Theorem 3.2)', but there is no Theorem 3.2 in the paper; the intended reference is likely Theorem 4.2. Similar internal cross-reference issues appear elsewhere.","section":"Appendix B.2"},{"comment":"The algorithm header includes 'φ2' but the algorithm signature uses 'φ1'; this appears to be an artifact. Also, line 3 computes T = ⌈E/q⌉ while the text earlier uses T = ⌈E·N/B⌉; please reconcile the notation.","section":"Algorithm 3"},{"comment":"The y-axis in the audit plots is labeled 'Estimate' but the caption says 'empirical ε'. Clarify whether these are lower bounds on ε and how the confidence level 0.01 maps to the displayed values.","section":"Figures 10–11"}],"recommendation":"reject","confidential_remarks":"The central privacy bound (Theorem 4.9) is not a valid upper bound under the mechanism's shared random scale and shared subsampling. This is a load-bearing error, not a presentation issue: all reported ε values and utility comparisons depend on it. A fix would require a genuinely joint multivariate accounting analysis or a change to the mechanism itself, and the empirical claims would need to be redone. The paper is otherwise interesting and well-scoped, but this flaw prevents acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea here is real: replacing the single-scale Gaussian with a randomized-scale Laplace distribution gives DP-SGD a genuinely new lever for trading distortion against privacy. The majorization trick to extend Laplace-style accounting to l2 clipping is also interesting, and the evaluation is thorough—vision and language models, fine-tuning and training, plus a released codebase. I want that part on the record before the bad news.\n\nThe bad news is load-bearing. Theorem 4.9 sums per-coordinate univariate moments accountant bounds to bound the multivariate PLRV mechanism. That step does not work. In the actual mechanism, one scale b is drawn per step and the minibatch subsampling event is shared across all coordinates. Conditional on b the coordinates are independent, so the vector moment for one iteration is E_b[∏_i m_i(b)], where m_i(b) is the coordinate's univariate moment. The paper instead uses log ∏_i E_b[m_i(b)]. Since the m_i(b) are comonotone functions of the same b, Jensen gives E_b[∏ m_i(b)] ≥ ∏ E_b[m_i(b)]; the inequality goes the wrong way. The proposed sum is a lower bound on the true privacy loss, not an upper bound. That makes all the reported epsilons—CIFAR-10 at 0.5, SST-2 at 0.2, the whole Table 2/6/7 story—optimistic to an unknown degree. The empirical audit in Section 6.4 measures lower bounds on epsilon and cannot certify the claim either.\n\nThe univariate Theorem 4.8 might be fine; the failure is in the multivariate composition step. The authors need either to derive a valid vector bound that accounts for the shared scale and subsampling draw, or redesign the mechanism so coordinates are genuinely independent (e.g., draw b per coordinate—though that would wreak havoc on the privacy accounting too). Majorization alone cannot fix the correlation problem, since the worst-case gradient vector would still share one b.\n\nThe optimization framework has heuristic bits (Cmax = 2 Cmin, the distortion cap of 10) but those are secondary. The main issue is mathematical. This is a promising mechanism in search of a correct proof.\n\nWho is this for? Someone working on new noise distributions for DP-SGD will find the construction and experiments worth reading, with caution. As a claimed privacy guarantee, the paper is not acceptable in current form.\n\nI would send it to peer review—the novelty and the subtlety of the flaw deserve a careful referee report—but my recommendation to the authors would be that the multivariate accounting must be fixed or the paper should not stand.","headline":"The mechanism is clever and the experiments are impressive, but the multivariate privacy bound in Theorem 4.9 is not valid, so the headline epsilon values are unsupported.","tokens_in":40355,"tokens_out":3369,"would_cite":false,"duration_ms":43038,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that replacing the Gaussian mechanism in DP-SGD with a two-parameter randomized-scale Laplace noise can improve model accuracy in the high-privacy regime, reporting gains of roughly 10 to 40 accuracy points at epsilon aroun","keywords":["differential privacy","DP-SGD","privacy loss random variable","randomized-scale Laplace mechanism","moments accountant","subsampling amplification","majorization","private fine-tuning"],"falsifier":"Run one iteration of the actual PLRV-O algorithm (shared batch mask and a single sampled scale b for all coordinates) on a small model, compute the full multivariate privacy-loss distribution numerically by exact or PLD-style accounting, and compare the resulting epsilon with the sum-of-coordinate bound; if the direct epsilon is larger than the reported bound, the privacy guarantee is understated.","tokens_in":1660,"feed_emoji":"🔒","tokens_out":6163,"duration_ms":157637,"temperature":0.7,"pith_summary":"The paper's central claim is that the standard Gaussian noise in DP-SGD is an unnecessarily restrictive choice, and that a two-parameter family of Laplace noises with randomized scale can be both privacy-accounted and tuned to the task. PLRV-O builds this family by drawing the inverse scale from a gamma distribution, and derives a moments accountant for it that works with l2 clipping via a majorization bound. The payoff is a claimed separation of privacy cost from distortion: for the same reported epsilon, the optimized noise can tolerate larger clipping thresholds and preserves more signal. Reported results include 94.03% accuracy at epsilon about 0.5 on CIFAR-10 ViT fine-tuning versus 83.93% for Gaussian, and 92.20% at epsilon about 0.2 on SST-2 RoBERTa-large versus 50.25%.","feed_headline":"Laplace-family noise beats Gaussian at strong privacy","feed_subtitle":"Reports double-digit accuracy gains at epsilon ~0.2–0.5 on vision and language models.","key_machinery":"Two interlocking pieces. The first is the PLRV noise itself: a multivariate Laplace mechanism whose scale b is a random draw from a seed density, so the noise density is a mixture of Laplace densities. The second is a privacy accountant for it: after l2 clipping, the coordinate-wise subsampled moments are Schur-convex, and the majorization set x_i = C(sqrt(i)-sqrt(i-1)) dominates the clipped gradient magnitudes in weak majorization order; therefore the total moments bound is the sum of the univariate bounds, with each univariate bound expressed through the MGF M_u(t)=(1-t theta)^{-k} for gamma seeds. The separation of k (which mostly controls entropy and utility) and theta (which mostly cont","core_discovery":"PLRV-O claims that the privacy loss random variable, not the noise distribution itself, is the right target for mechanism design in DP-SGD. It introduces a search space of randomized-scale Laplace mechanisms, where the inverse scale u=1/b follows a gamma distribution with shape k and scale theta. For this family, Theorem 4.9 bounds the subsampled multivariate moments accountant by a sum of per-coordinate terms evaluated on the majorization set x_i = C(sqrt(i)-sqrt(i-1)), with each term depending on the moment generating function of u; because that MGF is (1 - t theta)^{-k}, the account is closed-form and can support thousands of moment orders. This turns noise design into a constrained optim","pith_inferences":["The central reported epsilons inherit a risk: because one scale b is sampled per update and shared by all parameters, the per-coordinate independence assumption in the moments accountant may understate true privacy loss; a numerical PLD audit of the exact multivariate mechanism would settle this.","If the multivariate bound fails, the strong-accuracy results would need to be re-read at their true (larger) epsilon; the Gaussian comparisons might still favor PLRV-O, but the margin would shrink.","The optimization objective C(k-1)theta is a signal-to-noise surrogate, not an accuracy guarantee; the paper does not prove that maximizing this objective maximizes test accuracy, so the reported gains are empirical rather than optimality-based.","A natural next step is to test the same randomized-scale construction with other distortion measures beyond l1 error, such as l2 or per-coordinate heterogeneous costs, since the accountant only needs the MGF of the reciprocal scale."],"forward_implications":["At strictly small epsilon, PLRV-O reports accuracy gains over Gaussian DP-SGD large enough to change deployment choices: e.g., 94.03% vs 83.93% on CIFAR-10 ViT at epsilon about 0.5, and 92.20% vs 50.25% on SST-2 RoBERTa-large at epsilon about 0.2.","Because the accountant runs per coordinate and sums via a majorization set, the method avoids the sqrt(n) noise inflation that made Laplace noise impractical for deep nets, so it applies to models with tens of millions of parameters.","The same noise mechanism can be dropped into other DP algorithms: the paper reports consistent gains when PLRV-O noise replaces Gaussian noise in DP-FTRL on MNIST and CIFAR-10.","The framework supports tailoring noise to task properties such as model size, number of steps, batch sampling rate, and clipping threshold; the reported runs also show faster convergence at the same epsilon.","Any seed distribution with a known moment generating function can be used in the same accounting, so the gamma family is an instance rather than an endpoint."],"supporting_citations":[{"why":"Supplies the DP-SGD formulation, clipping, and moments accountant that PLRV-O generalizes and compares against.","marker":"[1]"},{"why":"Provides the subsampled Gaussian moments accountant and binomial-expansion bound that PLRV-O adapts to Laplace and PLRV noise.","marker":"[42]"},{"why":"Gives the tight conversion from moment bounds to the (epsilon, delta) guarantee used to report all privacy budgets.","marker":"[2]"},{"why":"Provides the strong-DP large-language-model baseline and the optimizer/clipping pipeline used in the fine-tuning comparison.","marker":"[39]"},{"why":"Supplies the majorization-theory machinery used to construct the coordinate-wise bound x_i = C(sqrt(i)-sqrt(i-1)).","marker":"[41]"},{"why":"Supplies the Schur-convexity criterion that lets the per-coordinate moments be summed under the majorization set.","marker":"[52]"}],"fun_headline_variants":["Optimizing privacy loss beats Gaussian in DP-SGD","PLRV-O: new DP noise family lifts accuracy at low epsilon","Gamma-scaled Laplace noise outperforms Gaussian in DP","DP-SGD gains 10+ accuracy by reshaping privacy loss","Privacy loss random variable design improves DP training"],"cache_read_input_tokens":41984,"weakest_assumption_plain":"The whole privacy accounting assumes that the privacy loss of one training step can be bounded by the sum of per-coordinate privacy-loss bounds, which requires the subsampling events and the random noise scale to be independent across coordinates; in the actual algorithm they are shared, so correlated coordinate losses could make the true loss exceed the bound.","fun_headline_variants_meta":{"raw":{"variants":["Optimizing privacy loss beats Gaussian in DP-SGD","PLRV-O: new DP noise family lifts accuracy at low epsilon","Gamma-scaled Laplace noise outperforms Gaussian in DP","DP-SGD gains 10+ accuracy by reshaping privacy loss","Privacy loss random variable design improves DP training"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000646,"raw_usage":{"total_tokens":2842,"prompt_tokens":816,"completion_tokens":2026,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":560,"completion_tokens_details":{"reasoning_tokens":1960}},"tokens_in":560,"tokens_out":2026,"duration_ms":17954,"temperature":1.0,"reasoning_tokens":1960,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T23:54:41.977876+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run one iteration of the actual PLRV-O algorithm (shared batch mask and a single sampled scale b for all coordinates) on a small model, compute the full multivariate privacy-loss distribution numerically by exact or PLD-style accounting, and compare the resulting epsilon with the sum-of-coordinate bound; if the direct epsilon is larger than the reported bound, the privacy guarantee is understated.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the DP-SGD formulation, clipping, and moments accountant that PLRV-O generalizes and compares against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the subsampled Gaussian moments accountant and binomial-expansion bound that PLRV-O adapts to Laplace and PLRV noise."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Gives the tight conversion from moment bounds to the (epsilon, delta) guarantee used to report all privacy budgets."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the strong-DP large-language-model baseline and the optimizer/clipping pipeline used in the fine-tuning comparison."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the majorization-theory machinery used to construct the coordinate-wise bound x_i = C(sqrt(i)-sqrt(i-1))."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Schur-convexity criterion that lets the per-coordinate moments be summed under the majorization set."}],"review_version":1}