{"id":"66d80787-3d62-4970-96e9-3dae13891c41","arxiv_id":"2511.17380","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"NPPR is the worst-case probability of correct classification over all perturbation distributions inside a budget; a Gaussian mixture model is fit to approximate it, giving conservative robustness estimates.","lead":"This paper proposes a robustness metric that does not assume a fixed noise distribution: it searches over all perturbation distributions within a budget and reports the lowest probability of correct classification. It is a conservative, safety-oriented bound, but in the unrestricted case it reduces to standard adversarial robustness.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The exact NPPR infimum collapses to AR for any DNN, so the reported estimator is not approximating the defined metric.","rationale":"The reader's CONDITIONAL verdict is based on the estimator gap: the GMM+surrogate may not approximate the true infimum. I agree that gap exists and is unbounded, but the more load-bearing problem is that the exact metric defined in Def. 3 is degenerate for deep networks: the infimum over all supported distributions equals AR whenever an adversarial perturbation exists, because adversarial regions are open. This is not a matter of missing convergence analysis; it follows from the paper's own Prop. 1, Eq. 6 plus the geometry of DNN decision regions. The strict-inequality branch (Eq. 7) requires the adversarial set to have measure zero, which is incompatible with the open regions produced by continuous logits. Therefore the central conceptual contribution—a distributionally robust PR metric strictly between AR and PR—is not supported, and the experimental numbers are values of a restricted optimization, not estimates of the claimed metric. Because the central claim is mathematically undercut, the appropriate verdict moves from CONDITIONAL to REJECT, although the empirical GMM/adversarial-search pipeline might still be useful as a heuristic attack. The concrete test with a narrow Gaussian centered at a found adversarial perturbation would settle the degeneracy directly and is easy to run.","tokens_in":17505,"tokens_out":12888,"duration_ms":139061,"concrete_test":"Take one CIFAR-10 test image for which PGD-20 finds an adversarial perturbation ε_a inside the L∞ ball of radius 16/255 (shrink the PGD perturbation by 0.8 if it lies on the boundary, and verify it still misclassifies). Define a narrow Gaussian perturbation distribution centered at ε_a with covariance σ²I, truncated to the budget B, and estimate E[1_{h(x+ε)=y}] by Monte Carlo for σ = 1e-1, 1e-2, 1e-3 of the pixel scale. If this estimate tends to 0 as σ→0, then the exact continuous NPPR for that input is 0, while Table 3 reports bG_NPPR ≈ 88.32 for the same model/setting. This directly demonstrates that the reported estimator is not approximating the defined infimum.","verdict_should_be":"REJECT","load_bearing_attack":"The central claim that NPPR is a meaningful intermediate robustness metric is undercut by the definition itself. Def. 3 sets S_NPPR = inf_{ω∈P_ε} E_ω[1_{h(x+ε)=y}] with P_ε all distributions supported in B. Prop. 1, Eq. 6 already concedes that if P_ε is unrestricted then G_NPPR = G_AR. Since Def. 3 imposes no restriction beyond support in B, the equality is the default, not an extreme case.\n\nRestricting to continuous distributions does not fix it. For a standard DNN, the set of adversarial perturbations {ε : f_y(x+ε) < max_{j≠y} f_j(x+ε)} is open whenever it is nonempty, by continuity of the logits. Hence any PGD-found adversarial perturbation has an open neighbourhood of adversarial perturbations, and B has nonempty interior, so there is an interior adversarial point with an open ball fully inside B. A continuous distribution supported on a tiny ball around that point has expected correctness arbitrarily close to 0, so the infimum is 0. Thus S_NPPR(x,y)=0 whenever an adversarial example exists, exactly matching S_AR.\n\nConsequently the advertised 'strict inequality' branch of Prop. 1 (Eq. 7) relies on the adversarial set having measure zero, which is impossible for the open adversarial regions of real neural classifiers. The empirical values in Table 3 (bG_NPPR ≈ 88% vs bG_AR_PGD ≈ 9% for ResNet18/CIFAR-10) therefore cannot be estimates of the Def. 3 infimum; they are values of a restricted GMM/surrogate optimization. The paper's own theorem shows the true NPPR is essentially AR, so the metric does not provide the claimed distributionally robust middle ground.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Non-Parametric Probabilistic Robustness (NPPR), defined as the infimum over all perturbation distributions supported in a norm ball of the expected correctness probability (Def. 3, Eq. 3). It claims theoretical inequalities among adversarial robustness (AR), probabilistic robustness (PR), and NPPR (Prop. 1), and proposes a GMM-based estimator with MLP heads and bicubic up-sampling (Sec. 4). Experiments on CIFAR-10/100 and Tiny ImageNet across several architectures report NPPR values intermediate between AR and PR, which the authors interpret as a more conservative, distribution-free robustness metric.","tokens_in":17923,"tokens_out":4133,"duration_ms":39692,"significance":"The paper identifies a legitimate limitation of existing PR formulations—their reliance on a fixed, known perturbation distribution—and proposes a natural response: optimize the distribution to obtain a conservative estimate. The experimental pipeline is extensive, and the distribution visualizations (Fig. 5) are informative. However, the formal metric as defined collapses to AR for any classifier with continuous decision regions: the infimum in Def. 3 is attained (or approached) by distributions supported on adversarial regions, making NPPR equal to AR. The proposed estimator minimizes a surrogate loss over a restricted GMM family, and no result guarantees that its value is a conservative estimate of the defined infimum. Consequently, the central claim that NPPR is a meaningful intermediate metric is not supported by the theory or the experiments.","major_comments":[{"comment":"The infimum in Def. 3 over all distributions supported in B collapses to AR. For a classifier with continuous logits, if an adversarial perturbation exists in the interior of B, the set of adversarial perturbations contains an open neighborhood, which has positive Lebesgue measure. A continuous distribution supported on a small ball inside that neighborhood gives expected correctness 0, so S_NPPR(x,y)=S_AR(x,y) whenever an adversarial example exists. The strict-inequality branch in Eq. (7) assumes the adversarial set has measure zero, which is false for the open adversarial regions of real DNNs. Thus NPPR is not an intermediate metric; it is AR in disguise. The empirical values in Table 3 (e.g., 88.32% NPPR vs. 9.27% AR for ResNet18/CIFAR-10) therefore cannot be estimates of the Def. 3 infimum.","section":"Sec. 3, Def. 3 and Prop. 1 (Eqs. 3, 6–7)"},{"comment":"The estimator minimizes a C&W softplus surrogate over a GMM parameterization, which is a strict subset of P_ε, and no bound relates the surrogate objective to the 0–1 indicator or the GMM family to the full P_ε. Since the GMM family is a subset of P_ε, the true NPPR is ≤ the obtained GMM value; the GMM value is an upper bound on the true NPPR, not a conservative lower estimate. The large gap between reported NPPR and AR (Table 3) is consistent with the estimator not approximating Def. 3 at all. A consistency or approximation theorem is needed but absent.","section":"Sec. 4, Eqs. (9)–(11) and Table 3"},{"comment":"The proof of the unconditional-case reverse inequality is flawed. In Eqs. (33)–(35), the authors lower-bound E_ω[E_D[...]] by inf_ε E_D[...], but G_AR in the unconditional case is defined as inf_ε E_D[...] only in Prop. 2 (Eq. 40); in Prop. 1, G_AR is not this quantity. The definitions are inconsistent. Moreover, the proof of strict inequality (Eq. 25) asserts that if adversarial examples have measure zero under all continuous distributions, then all continuous ω yield expectation 1; this is false when the adversarial set has positive Lebesgue measure (which it does for DNNs). The theoretical chain G_AR < G_NPPR < G_PR is therefore not established for real classifiers.","section":"Appendix 7, Proof of Prop. 1 (Eqs. 23–35)"}],"minor_comments":[{"comment":"The term 'non-parametric' is misleading since the proposed estimator is a Gaussian Mixture Model, a parametric family. The paper should clarify the distinction or qualify the term.","section":"Title / Abstract"},{"comment":"For WRN50 on CIFAR-10, the standard deviation for bGPR_Uniform is reported as 30.91, which is implausibly large compared to neighboring entries and likely a typographical error. Please verify.","section":"Sec. 5.3, Table 3"},{"comment":"Typographical errors: 'exsit' should be 'exist'; Eq. (17) and surrounding text should consistently use the indicator 1_{h(x+ε)=y} rather than switching between correctness and error indicators.","section":"Appendix 7, Proof 1"},{"comment":"The softplus formulation is described as 'lower-bounds the loss at zero'; consider clarifying the sign convention, as the margin term is typically negative for adversarial examples.","section":"Sec. 4, Eq. (11)"},{"comment":"The caption mentions 'training dynamics of the NPPR' but it is unclear whether the plotted curve is the surrogate loss or the estimated 0–1 value. Please specify.","section":"Sec. 5.1, Fig. 5"}],"recommendation":"reject","confidential_remarks":"The core definitional problem is fatal to the current manuscript. The paper may be salvageable if the authors redefine NPPR as a restricted distributional robustness metric (e.g., with entropy, smoothness, or tail constraints on the perturbation distribution) and prove a proper approximation guarantee for their GMM estimator. As it stands, the empirical claims do not support the formal metric, and the theory is either tautological or incorrect for standard neural classifiers. I would not block a future submission along these lines, but the present version does not meet the bar for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the take: the paper's headline metric, NPPR as defined in Def. 3, is not a middle ground between AR and PR. It collapses to AR for any network with an adversarial example, which is every network they test. The GMM estimator is a new piece of engineering, but it does not estimate the defined NPPR.\n\nWhat's genuinely useful: the authors identify a real gap—PR assessment assumes a known perturbation distribution, which is often unrealistic. The proposal to learn a conservative distribution is sensible. The four dependency settings, the GMM+MLP+upsampling pipeline, and the cross-dataset/model evaluation are thorough. The paper is clearly written and the experiments are reproducible in spirit. If the goal is to give safety engineers a way to stress-test PR with a learned adversarial noise distribution, there is something here.\n\nThe problem is the formal definition. Def. 3 sets P_ε to all distributions supported in B. Prop. 1 already concedes G_NPPR = G_AR when Dirac measures are allowed. The stress-test note goes further: for a continuous classifier, the set of adversarial perturbations is open whenever nonempty. So even restricting to continuous distributions, you can put a continuous distribution on a tiny ball inside the adversarial region and get expected correctness arbitrarily close to 0. The infimum is 0 for every input with an adversarial example. The strict inequality branch in Prop. 1 requires the adversarial set to have measure zero, which is impossible for any classifier with an interior adversarial point—basically all the ResNets and VGGs tested. So the exact NPPR metric is not a meaningful intermediate; it is the same worst-case number as AR.\n\nThe empirical bG_NPPR values around 88–98%, far above the AR values around 2–9%, cannot be estimates of this infimum. They are the result of a restricted GMM search with a C&W surrogate and a low-dimensional latent space. The paper never bounds the gap between the fitted GMM family and P_ε, and never shows the estimator tracks the true NPPR. The 'non-parametric' label is also inflated: a fixed GMM family is parametric; they are learning the parameters.\n\nSo the paper is a useful practical exploration of a conservative PR estimator, but the theoretical framework does not support the central claim. I would not cite the NPPR metric as a distribution-free lower bound. The estimator itself might be useful as a heuristic for adversarial distribution search, but it needs to be reframed and analyzed as such.\n\nRecommendation: if it comes through your desk, I'd send it to peer review conditional on heavy revision—the experiments are valuable and the problem is real—but I'd require the authors to either fix the definition (e.g., restrict P_ε to something that doesn't collapse) or drop the claim that bG_NPPR estimates NPPR. As is, the mathematical center does not hold.","headline":"The paper's headline metric collapses to adversarial robustness for any typical DNN, so the theory does not support the empirical claims—though the GMM estimator is a useful engineering contribution.","tokens_in":18411,"tokens_out":3553,"would_cite":false,"duration_ms":35360,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","68T45"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper defines non-parametric probabilistic robustness (NPPR) as an infimum over all perturbation distributions within a budget, proves it lies between adversarial and fixed-distribution probabilistic robustness, and introduces a GMM-bas","keywords":["non-parametric probabilistic robustness","adversarial robustness","probabilistic robustness","Gaussian mixture model","distributional uncertainty","conservative risk estimation","neural network robustness","perturbation distribution"],"falsifier":"For a fixed image and perturbation budget, enumerate or search a dense family of candidate perturbation distributions (e.g., many Gaussian and mixture distributions with varied means/covariances inside the budget) and compute the model's correct-prediction rate under each. If any of these rates falls below the paper's NPPR estimate for that image, the estimator is not conservative and the Proposition 1 guarantee does not transfer to the estimate.","tokens_in":1292,"feed_emoji":"🛡️","tokens_out":1913,"duration_ms":54152,"temperature":0.7,"pith_summary":"The paper argues that existing probabilistic robustness (PR) metrics are unrealistic because they assume the perturbation noise follows a known, fixed distribution—an assumption rarely true in practice. It introduces NPPR, which takes the worst case over all perturbation distributions within a given budget: the lowest probability that the classifier keeps the correct label. The theoretical core is a sandwich inequality: adversarial robustness ≤ NPPR ≤ PR for any fixed perturbation distribution, making NPPR a distribution-free conservative safety estimate. To compute it, the authors fit a Gaussian mixture model, with MLP heads that let the distribution depend on the input, the label, both, or neither, and bicubic upsampling to handle high-resolution images. Across CIFAR-10/100 and Tiny ImageNet with several architectures, the resulting estimates are consistently lower—more conservative—than those from Gaussian or uniform noise assumptions.","feed_headline":"Noise-agnostic robustness metric yields more conservative scores","feed_subtitle":"It learns the perturbation distribution from data and bounds safety between worst-case and average-case attacks.","key_machinery":"The central object is Definition 3's infimum over the admissible perturbation family P_epsilon, paired with a concrete estimator: MLP heads produce the GMM's mixture weights, means, and covariances (with dependency structures ranging from independent to fully input-and-label dependent), Gumbel–Softmax makes categorical sampling differentiable, a scaled tanh mapping keeps perturbations inside the L-infinity budget, and bicubic upsampling maps latent perturbations to image resolution. The C&W-style softplus margin loss relaxes the 0–1 indicator so the objective can be optimized by gradient descent.","core_discovery":"The central claim is that robustness evaluation need not commit to a noise model: for each input–label pair, NPPR is defined as the infimum, over all perturbation distributions supported within the budget, of the probability that the classifier's prediction remains unchanged. Proposition 1 establishes that this quantity sits between adversarial robustness (the deterministic worst case) and any fixed-distribution probabilistic robustness, so NPPR is a distribution-free conservative estimator. The accompanying estimator optimizes a GMM-based perturbation generator—conditioned on the input, the label, both, or neither—using a C&W-style margin loss as a surrogate for the hard indicator, and obta","pith_inferences":["The proof's equality case uses Dirac delta distributions, so the meaningful content of NPPR lies in the choice of the admissible family P_epsilon; the paper's GMM estimator restricts this family to mixtures, meaning its outputs are best read as practical upper bounds on the true NPPR rather than guaranteed lower bounds.","The observed ordering—joint dependence giving the lowest NPPR, independence the highest—suggests a simple stress-testing recipe for deployment: use the joint-dependent estimator as the most pessimistic configuration when safety margins are tight.","The same formulation could transfer to text, speech, or other modalities where distributional assumptions are even less settled, provided the perturbation generator can be reparameterized in that space.","A fairer comparison of conservativeness would pit NPPR against a wide set of alternative learned distributions (e.g., adversarial-perturbation empirics or variational posteriors), not just fixed Gaussian/uniform noise."],"forward_implications":["If NPPR is estimated faithfully, it provides a lower bound on robustness that holds regardless of the true (unknown) perturbation distribution.","The inequality G_AR ≤ G_NPPR ≤ G_PR means that any PR evaluation based on a specified noise distribution can overestimate safety; NPPR corrects that by construction.","Input- and label-dependent perturbation distributions yield lower NPPR than input-independent ones, so ignoring such dependencies understates risk.","In the unrestricted distribution family, NPPR collapses to adversarial robustness, making NPPR a smooth bridge between average-case and worst-case robustness assessment.","The estimator is practical on standard image benchmarks, producing up to roughly 40% more conservative PR estimates than fixed Gaussian or uniform baselines without requiring a predefined noise model."],"fun_headline_variants":["Learn noise distribution from data for conservative robustness","Robustness metric learns perturbation distribution, stays conservative","Between worst-case and average-case: learnable noise for robustness","No fixed noise model: new robustness metric is conservative"],"cache_read_input_tokens":19584,"weakest_assumption_plain":"The load-bearing premise is that the GMM-plus-MLP estimator optimizes closely enough to the true infimum over all admissible perturbation distributions that its output remains a conservative estimate; the paper proves the inequality only for the exact infimum, not for the fitted approximation.","fun_headline_variants_meta":{"raw":{"variants":["Learn noise distribution from data for conservative robustness","Robustness metric learns perturbation distribution, stays conservative","Between worst-case and average-case: learnable noise for robustness","No fixed noise model: new robustness metric is conservative"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000632,"raw_usage":{"total_tokens":2747,"prompt_tokens":732,"completion_tokens":2015,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":476,"completion_tokens_details":{"reasoning_tokens":1952}},"tokens_in":476,"tokens_out":2015,"duration_ms":14993,"temperature":1.0,"reasoning_tokens":1952,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T20:55:46.621257+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"For a fixed image and perturbation budget, enumerate or search a dense family of candidate perturbation distributions (e.g., many Gaussian and mixture distributions with varied means/covariances inside the budget) and compute the model's correct-prediction rate under each. If any of these rates falls below the paper's NPPR estimate for that image, the estimator is not conservative and the Proposition 1 guarantee does not transfer to the estimate.","supporting_citations":[],"review_version":1}