{"id":"7bbe3f9f-3b18-4149-adba-a95cc554cba6","arxiv_id":"2607.19580","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"Perturbing softmax outputs with the Dirichlet mechanism during training yields input-private, label-public classifiers that beat prior differentially private training accuracy on five image benchmarks.","lead":"This paper trains image classifiers so that the training images are protected by differential privacy while the labels stay public: at each training step the model's probability output is randomly perturbed with a Dirichlet distribution. On five image benchmarks it reports accuracy well above previous private-training methods at the same privacy budgets.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 10's RDP-to-DP conversion has a δ=10^-5 floor ≈3.9, so the printed theorem cannot certify any ε<3.9; Table 7's ε=0.5–3 r* values and the reported ε=1 result are underivable from the theorem as stated.","rationale":"The paper's central claim is that Algorithm 1 provides (ε,δ)-DP for training inputs at ε values as low as 0.5–1 while retaining high accuracy. Theorem 10 is the only formal certificate for those ε labels. Its final term, taken verbatim from Lemma 13, has a positive floor at δ=10^-5: minimizing (log(λ−1)−logδ+λlogλ)/(λ−1) over λ>1 gives ≈3.9. Consequently the theorem cannot certify any ε<3.9, regardless of r, q, or T. Since Table 7 and the abstract claim ε=0.5, 1, 2, 3, the printed theorem is internally inconsistent with the paper's own r* table. This is a load-bearing formal defect, not a nitpick: it invalidates the privacy labels unless corrected.\n\nI agree with the reader's assessment that this is likely a typo in the conversion formula. A standard RDP-to-DP conversion ε_RDP + log(1/δ)/(λ−1) has no such floor and can plausibly reproduce the reported r* values for ε=1 with λ near the upper limit 1+α/r. The released code makes the issue checkable. If the code uses a standard conversion, the reported accuracies can stand with a corrected theorem; if the code uses the printed formula, the empirical ε labels are not supported.\n\nI considered whether the gradient computation in Algorithm 1 is a larger defect: as written, −log p̃_y with p̃ sampled independently of θ has zero gradient. However, this is standardly resolved with a reparameterized Dirichlet sample (e.g., `rsample`), and the released code likely uses it; the expected-gradient identity in Theorem 11 is plausible under that estimator. This is worth checking but is secondary to the accounting error. No other internal inconsistency reached the same severity.","tokens_in":24566,"tokens_out":17975,"duration_ms":152784,"concrete_test":"Download the released repository and locate the routine that computes r* via Eq. (1). For CIFAR-10 at ε=1, take α=3, q=0.005, T=20,000, δ=10^-5, and evaluate h(r)=min_λ ε_o(λ,δ;r) over integer λ∈[2,1+α/r), first with the printed Lemma 13 conversion term and then with the standard RDP-to-DP term log(1/δ)/(λ−1). If the standard formula reproduces the reported r*=0.235 while the printed formula cannot (because h(r)>3.9), the theorem's conversion term is the sole defect and a corrected theorem restores the ε=1 certificate; if neither formula reproduces the table, the reported privacy accounting is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Lemma 13 and Theorem 10 append the conversion term C(λ) = [log(λ−1) − log δ + λ log λ]/(λ−1). With δ=10^-5, C(λ) is minimized over λ>1 at roughly 3.9 (e.g., λ≈13–20), so ε_o(λ,δ;r) > 3.9 for every admissible λ. Therefore h(r) = min_λ ε_o(λ,δ;r) > 3.9, and the constraint h(r) ≤ ε in Eq. (1) has no solution for ε < 3.9. Yet Table 7 lists feasible r* values for ε=0.5, 1, 2, 3, and the abstract reports 82.96% accuracy at ε=1. The reported r* values cannot be derived from the printed theorem; they must come from a different (presumably standard) RDP-to-DP conversion, such as ε = ε_RDP + log(1/δ)/(λ−1). Since every headline privacy budget depends on Theorem 10, this is load-bearing: either the theorem contains a typo that must be corrected, or the ε labels on the empirical results are invalid. The empirical accuracy claims are not the issue; the formal privacy certificate is.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a differentially private training framework for neural network classifiers in which training inputs are sensitive while labels are public. At each training step, softmax outputs are randomized with the Dirichlet mechanism, and the paper derives an RDP-based privacy guarantee under Poisson subsampling (Theorem 10), proves that the expected private gradient is a scalar multiple of the clean cross-entropy gradient (Theorem 11), and uses that result to calibrate learning rates. Experiments on CIFAR-10, MNIST, DermaMNIST, FashionMNIST, and SVHN report large improvements over prior work, e.g., 88.17% accuracy at ε=4 and 82.96% at ε=1 on CIFAR-10 with δ=10^-5.","tokens_in":24855,"tokens_out":15977,"duration_ms":144722,"significance":"The idea of applying the Dirichlet mechanism at the softmax layer and analyzing it under Poisson subsampling is novel and potentially valuable for the input-private/public-label setting. Theorem 11 is a clean, explicitly derived result with practical implications for learning-rate selection; the code is promised, and the empirical gains over Monir and Ghinita (2024) are noteworthy. However, the formal privacy guarantee as printed is invalid because of an incorrect RDP-to-DP conversion, and the advertised ε labels for the strongest privacy regimes are not currently supported. The significance of the empirical results can only be assessed after this accounting error is corrected.","major_comments":[{"comment":"The RDP-to-DP conversion is invalid. For δ=10^-5, C(λ)=[log(λ−1)−logδ+λlogλ]/(λ−1) is minimized at approximately 3.9 (λ≈13–20), so ε_o(λ,δ;r) > 3.9 before the T-term is even added. In particular, a (2,0)-RDP mechanism has identical output distributions and is (0,0)-DP, but the printed formula would certify it only at ε≈12.9 for δ=10^-5. Consequently Theorem 10 cannot certify any ε<3.9, Eq. (1) has no solution for the ε=0.5–3 rows of Table 7, and the ε=1 CIFAR-10 headline is unsupported. Replace Lemma 13 with the standard conversion (e.g., ε̂+log(1/δ)/(λ−1)) and recompute all r* values and reported ε labels.","section":"Lemma 13 / Theorem 10 / Eq. (1) / Table 7"}],"minor_comments":[{"comment":"Under Poisson sampling, the minibatch B_t can be empty, making the gradient expression 1/|B_t| undefined. Please specify a convention, e.g., skip the update when B_t is empty.","section":"Algorithm 1, line 7"},{"comment":"The phrase \"NLL 1 loss\" appears to be a typo; it should be \"NLL loss\" or \"NLL.\"","section":"Section 3.2"},{"comment":"Please report the λ that achieves h(r) and the resulting numerical value of ε_o for each r* in Table 7. Without this, the minimization behind Eq. (1) cannot be reproduced.","section":"Theorem 10 / Table 7"},{"comment":"The left panel appears to use a logarithmic x-axis, but the axis label reads \"Learning rate γ\" without indicating log scale. Add an axis label such as \"log γ\" or explicitly annotate the log scale.","section":"Figure 2"}],"recommendation":"major_revision","confidential_remarks":"The central problem is a likely typo in the RDP-to-DP conversion, but it is load-bearing: the paper's headline ε=1 claim and the ε=0.5–3 rows of Table 7 cannot be derived from Theorem 10 as printed. I would require the authors to correct Lemma 13/Theorem 10, recompute all r* values and privacy budgets under the corrected formula, and clearly state whether the experiments used the printed formula or a different one. The empirical methodology and Theorem 11 are otherwise promising, so a major revision rather than rejection seems appropriate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know right away. The training idea is genuinely new and the empirical gains are real; but the main privacy theorem, as printed, is internally inconsistent with the paper's own ε values, and that has to be fixed before the headline claims can be accepted.\n\nThe new bit is using Ponnoprat's Dirichlet mechanism on softmax outputs for input-private, label-public training, with a subsampled RDP analysis. That combination is original: closest prior work (Monir & Ghinita) perturbs penultimate-layer features with Gaussian noise. Theorem 11, the gradient-moment result, is derived cleanly in the paper; I checked the exponential-family computation and the DCT argument and they hold. The empirical section is also solid: five benchmarks, fair comparisons to the nearest prior work and to DP-SGD (with the caveat that DP-SGD protects labels too, which the authors state), and the released code makes things checkable. At face value the CIFAR-10 accuracy gain at ε=4 is large.\n\nThe soft spot is load-bearing. Lemma 13 and Theorem 10 state the RDP-to-DP conversion as ε = ε̂ + [log(λ−1) − log δ + λ log λ]/(λ−1). With δ=1e-5, the additive term is minimized over λ≥2 at about 3.9. So the theorem as written cannot certify any ε<3.9, yet Table 7 lists r* values for ε=0.5,1,2,3 and the abstract reports 82.96% at ε=1. Those numbers cannot come from the printed formula. They must come from a different conversion, presumably the standard ε̂ + log(1/δ)/(λ-1). That means the empirical results are probably fine, but the formal certificate is wrong as stated—and it is the central contribution. The authors need to correct the theorem and the lemma, verify the code uses the corrected formula, and re-run the parameter-selection step.\n\nMinor: the ε=∞ baselines are below standard because of Poisson sampling and no momentum/weight decay; the authors acknowledge this, but it makes the SOTA framing less clean.\n\nFor a reading group, this is a useful example of strong results with a broken theorem — good to discuss how to review. It deserves a serious referee round, but with a major-revision requirement on the conversion. I wouldn't cite it in its current form.","headline":"A genuinely new and strong empirical training scheme, but the printed RDP-to-DP conversion in Theorem 10 cannot certify the headline ε values; fix the theorem before citing.","tokens_in":25497,"tokens_out":5812,"would_cite":false,"duration_ms":46730,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that randomizing softmax outputs with the Dirichlet mechanism trains classifiers to the highest reported accuracy for input-private, label-public learning, while preserving the direction of the expected gradient.","keywords":["differential privacy","Rényi differential privacy","Dirichlet mechanism","softmax layer","public labels","sensitive inputs","deep learning","Poisson subsampling"],"falsifier":"Recompute r* from the parameter-selection rule using Theorem 10's printed RDP-to-DP conversion for δ=10^-5. If the minimum over admissible λ of ε_o(λ, δ; r, T, q) stays above 1 for every r<α—equivalently, if no λ makes the printed bound certify ε≤1—then the reported ε=1 runs do not satisfy the privacy budget the paper claims.","tokens_in":24328,"feed_emoji":"🛡️","tokens_out":6269,"duration_ms":53731,"temperature":0.7,"pith_summary":"The paper proposes training classifiers with differential privacy for the training inputs while treating labels as public, by replacing each softmax output with a sample from a Dirichlet distribution. The central claims are that repeated Poisson-subsampled uses of this Dirichlet mechanism admit a tight Rényi differential privacy bound, and that on average the privatized gradient equals the clean cross-entropy gradient multiplied by a scalar factor, so privacy rescales the update but does not change its direction. Empirically the authors report better accuracy than previously published methods on five image benchmarks, including 88.17% on CIFAR-10 at ε=4 and 82.96% at ε=1 with δ=10^-5. A caveat is that the printed RDP-to-DP conversion formula appears too weak to certify any ε below about 3.9 at δ=10^-5, so the small-ε budget labels rest on a conversion step that is not the one displayed in the paper.","feed_headline":"At ε=4, a Dirichlet-private softmax hits 88.17% on CIFAR-10","feed_subtitle":"Inputs stay private while labels stay public; expected gradients keep their direction.","key_machinery":"The Dirichlet mechanism is the load-bearing object: for an input x, it samples ρ from Dir(r·softmax(f_θ(x)) + α·1), a distribution over the unit simplex, and uses ρ in place of the softmax vector when computing the negative log-likelihood loss. The unit-simplex sensitivity bound (ℓ2^2 ≤2, ℓ∞ ≤1) sets the privacy scale independently of the network's internals; the RDP analysis of Poisson-subsampled repeated use turns that per-sample perturbation into an end-of-training privacy budget; and the attenuation identity κ(z;r,α)=r s_y(z)ψ'(r s_y(z)+α) connects the mechanism's parameters to optimization by showing the expected gradient is a rescaling of the clean gradient.","core_discovery":"The central discovery is that the mapping induced by any neural network with a softmax head—inputs to softmax probabilities—has sensitivity bounded by the unit simplex regardless of architecture, and that randomizing these probabilities with the Dirichlet distribution enforces Rényi differential privacy. With fixed offset and scaling parameters α and r, one Dirichlet sample is (λ, λr^2ψ'(α−(λ−1)r))-RDP; combining this with Poisson subsampling and composing over T steps yields the paper's overall privacy bound. The same mechanism preserves the direction of the expected gradient: E[∇L(ρ(z), y)] = κ(z;r,α)∇L_CE(z), where κ(z;r,α)=r s_y(z)ψ'(r s_y(z)+α). Because backpropagation only sees the pri","pith_inferences":["If the effective-learning-rate collapse observed with SGD across different α values generalizes to other optimizers and architectures, it offers a practical tuning rule: choose α large enough to control variance, then set γ using γ_eff = γκ(z;r,α) at a typical s_y value such as 0.9.","The input-private, label-public setting composes naturally with label-DP mechanisms; combining the two would privatize both sides of the data at a fraction of DP-SGD's utility cost, though the paper does not test this combination.","The small-ε results should be re-generated with a corrected RDP-to-DP conversion; if the printed formula is retained, the practical privacy-accuracy frontier may shift to larger ε values than the tables suggest.","Because the noise variance scales with the number of classes rather than layer width, the method should be especially favorable on problems with few classes and large networks—a testable prediction for class-imbalanced or high-resolution few-class benchmarks."],"forward_implications":["Input-private training can be done without per-sample gradient clipping; the extra computation is roughly the cost of drawing one Dirichlet sample per class per example.","The privacy-utility tradeoff improves with smaller Poisson sampling rates; the paper reports accuracy rising as the sampling rate falls from 0.04 to 0.001 under (4,10^-5)-DP.","Because the expected gradient direction is preserved, learning rates can be rescaled through κ(z;r,α) to produce nearly matching accuracy across a wide range of α values.","At ε=1 on CIFAR-10, the reported accuracy of 82.96% exceeds the accuracy of the closest prior input-private method at its weakest tested budget, which was about 79% at ε≈4.46.","On DermaMNIST, private-input training stays within about five percentage points of the non-private baseline at ε=1, suggesting the method can preserve utility on medical images."],"fun_headline_variants":["Dirichlet privacy on softmax outputs: 88.17% at ε=4 on CIFAR-10","Public labels, private inputs: new SOTA in DP training","Input-only DP gains 9.8 points on CIFAR-10 at ε=4","At ε=4, 88.17%: the Dirichlet mechanism beats prior DP","Softmax randomized: private inputs, superior accuracy"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that the RDP-to-DP conversion formula printed in the paper is the one used to select r*; as printed that formula adds at least about 3.9 at δ=10^-5, so it cannot certify ε below about 3.9, and the ε=0.5–3 results in the tables have no certificate unless a different formula was actually used.","fun_headline_variants_meta":{"raw":{"variants":["Dirichlet privacy on softmax outputs: 88.17% at ε=4 on CIFAR-10","Public labels, private inputs: new SOTA in DP training","Input-only DP gains 9.8 points on CIFAR-10 at ε=4","At ε=4, 88.17%: the Dirichlet mechanism beats prior DP","Softmax randomized: private inputs, superior accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000918,"raw_usage":{"total_tokens":3830,"prompt_tokens":850,"completion_tokens":2980,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":594,"completion_tokens_details":{"reasoning_tokens":2873}},"tokens_in":594,"tokens_out":2980,"duration_ms":18751,"temperature":1.0,"reasoning_tokens":2873,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T12:21:52.548801+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute r* from the parameter-selection rule using Theorem 10's printed RDP-to-DP conversion for δ=10^-5. If the minimum over admissible λ of ε_o(λ, δ; r, T, q) stays above 1 for every r<α—equivalently, if no λ makes the printed bound certify ε≤1—then the reported ε=1 runs do not satisfy the privacy budget the paper claims.","supporting_citations":[],"review_version":1}