{"id":"a041d7d2-c296-45c0-b77d-62304c06ff64","arxiv_id":"2507.21483","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"NCCR, the change in neuron activations under random perturbation, is introduced as a robustness metric and used to flag adversarial examples and backdoor triggers.","lead":"The paper proposes NCCR, a metric that measures how much a network's neuron activations change when a small random perturbation is added to an input. The authors use NCCR to rank model robustness and to detect adversarial and backdoored inputs, reporting high accuracy on image and speaker datasets.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper's own JSMA results contradict the universal claim that adversarial examples are always less robust: NCCR-based detectors trained on FGSM/PGD miss JSMA examples (AUROC ~0.90–0.92 on CIFAR-10/ImageNet), so the attack-independent detection claim does not hold.","rationale":"The reader's weakest-assumption analysis identifies the same load-bearing point: the entire detection framework depends on adversarial examples being less robust than clean examples under the chosen random perturbation. I agree, and the paper's own tables provide direct evidence that this premise is not universal. The JSMA results are the clearest internal contradiction: classifiers trained on FGSM/PGD/Combine, which are the very attacks the method is designed to generalize from, drop to AUROC values around 0.90–0.94 on JSMA. This is not an implementation artifact or a missing baseline; it is reported in Tables 3 and 4 and acknowledged in the text. The paper explains the drop by noting that JSMA is an ℓ0 attack, whereas the random perturbation used for NCCR is an ℓ2-style perturbation. That explanation confirms the concern: the method does not measure a universal property of adversarial inputs; it measures a norm-dependent sensitivity that some attacks exploit and others do not. The backdoor experiments further weaken the universality claim, since §6.1 explicitly says BadNets poisoned examples can be more robust than clean examples under small perturbations, forcing the authors to use a much larger perturbation to reverse the ordering. This means the abstract's 'adversarial examples are always less robust' is contradicted by the paper's own data. In a revised version, the authors would need to either restrict the claim to attacks whose perturbation norm matches the NCCR random perturbation, or provide evidence that NCCR separates adversarial examples across attack norms. Without that, the near-perfect detection accuracy is an artifact of the specific attacks tested, not a general result. The undefined NCCR formula is also a serious reproducibility problem, but the JSMA evidence is the single most load-bearing flaw because it directly attacks the premise on which the detection claim is built.","tokens_in":12885,"tokens_out":5665,"duration_ms":76489,"concrete_test":"Re-run §5.1 exactly with the Env2 CIFAR-10 model: train the NCCR classifier on clean images and on FGSM+PGD+Combine adversarial examples, then evaluate AUROC on JSMA examples generated with Foolbox defaults. If the result reproduces the reported value near 0.9054 (Table 4) while FGSM/PGD stay near 0.99, the 'adversarial examples are always less robust' premise is false for JSMA, and the detector is attack-dependent rather than universal. Additionally, plot the NCCR distributions for clean vs. JSMA examples; overlap of the distributions would directly show the premise fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central detection claim rests on the premise, stated in §5 and the abstract, that any adversarial example has lower robustness than any clean example under the random perturbation used to compute NCCR. The experiments in §5.1 already falsify this premise for JSMA, an ℓ0 attack. In Table 3 (Env1, CIFAR-10), classifiers trained only on FGSM, PGD, or Combine obtain AUROC 0.8931, 0.9002, and 0.9103 on JSMA, while the same classifiers achieve 0.99+ on FGSM, PGD, C&W, and DeepFool. Table 4 shows the same pattern (e.g., 0.9054 on CIFAR-10, 0.9341–0.9415 on ImageNet). The paper itself writes that performance is 'generally modest' because JSMA modifies few pixels, while NCCR uses random noise. This is not a minor limitation: it shows that the robustness deficit is not a property of adversarial inputs as such but of the norm/scale match between the attack and the random perturbation. The claimed near-perfect, attack-independent detection therefore fails for a standard attack family. The backdoor section similarly concedes that BadNets poisoned examples can be more robust than clean ones (§6.1) and requires a special large perturbation to invert the ordering, so 'always less robust' is not even true across the paper's own data. Consequently, the central claim overgeneralizes from the attacks where random noise happens to separate the distributions.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces a metric called Neuron Coverage Change Rate (NCCR), defined in prose as the change in outputs of selected neurons when an input is perturbed with random noise. It claims that NCCR can rank the robustness of neural networks, detect adversarial examples because 'adversarial examples are always less robust' than clean ones, and detect backdoor triggers. Experiments are reported on MNIST, CIFAR-10, ImageNet, GTSRB, and LibriSpeech, with AUROC and F1 comparisons against several baselines.","tokens_in":13235,"tokens_out":4434,"duration_ms":49632,"significance":"The NCCR idea is attractive because a single random-perturbation measurement could in principle yield a fast, attack-independent robustness estimate and a cheap detector. The paper has strengths: it spans image and speaker domains, compares against multiple baselines, and uses public datasets and tools. However, the central claim is not supported. The paper never gives a precise definition of NCCR, and its own JSMA results and backdoor analysis directly contradict the 'always less robust' premise. The detection method is a supervised classifier trained on known adversarial and clean examples, so it cannot establish the universal robustness-ordering property by itself. As framed, the contribution overgeneralizes from the attack families where random noise happens to separate the distributions.","major_comments":[{"comment":"The definition of NCCR is given only in prose: 'we add a random perturbation ε to x' and 'the degree of change in F(x′) relative to F(x) is called NCCR.' No formula is provided for F(x), no neuron subset is specified, no perturbation distribution is given, and no statement explains how the scalar NCCR is computed from the vector-valued activations. The experiments later state that the l2-norm was used, but the dimensionality, normalization, and aggregation over the ten perturbation sets are not described. This makes the metric and all reported numeric results non-reproducible.","section":"Section 3.2"},{"comment":"The paper's own data falsify the claim that adversarial examples are always less robust under the random perturbation used for NCCR. In Table 3 (Env1, CIFAR-10), classifiers trained on FGSM, PGD, or Combine obtain AUROC 0.8931, 0.9002, and 0.9103 on JSMA, while the same classifiers exceed 0.99 on FGSM, PGD, C&W, and DeepFool. Table 4 shows the same pattern (e.g., 0.9054 on CIFAR-10 and 0.9341–0.9415 on ImageNet). The text concedes that performance is 'generally modest' because JSMA is an l0 attack while the NCCR perturbation is random l2 noise. This shows that the robustness deficit is not a property of adversarial inputs as such but of the match between the attack's norm and the random perturbation. The claimed attack-independent, near-perfect detection therefore fails for a standard attack family.","section":"Section 5.1, Tables 3 and 4"},{"comment":"The backdoor analysis directly contradicts the universal premise stated in the abstract. The paper concedes that BadNets poisoned examples 'can be very far from the decision boundary' and 'are significantly more robust than clean examples' under small perturbations, and that the ordering inverts only when ε=256 is used, which 'completely destroys' the image content. This is an ad hoc assumption that a sufficiently large perturbation will destroy any trigger, with no justification for the required perturbation magnitude. At ε=256 the threat model is no longer the imperceptible-trigger backdoor described earlier, so the detection result is not an instance of the paper's stated 'always less robust' principle.","section":"Section 6.1, Figure 4"},{"comment":"The detection pipeline trains a classifier D on NCCR values of pre-generated adversarial and clean examples. This is a supervised classifier fitted to known attack families, not an independent demonstration that a robustness deficit is universal or attack-independent. Moreover, Baseline 3 [39] is exactly the robustness-ordering detection principle on which the paper relies, so the experiments compare the new NCCR measure against a baseline of the same family rather than validating the 'always less robust' premise. The sentence in the abstract that detection works 'as adversarial examples are always less robust' is therefore an unsupported premise rather than a conclusion established by the experiments.","section":"Section 5, detection methodology"}],"minor_comments":[{"comment":"A formal definition would greatly help reproducibility, for example NCCR(x; ε, S) = Eδ~N(0,ε)[ ||FS(x+δ) − FS(x)||p ] with S the chosen neuron subset and p the norm; the current prose leaves all of these choices implicit.","section":"Section 3.2"},{"comment":"There are several typos: 'PDG' should be 'PGD', 'CIAR-10' should be 'CIFAR-10', 'F AKEBOB' appears in Table 5, and 'Exexamples' appears in the Figure 4 caption.","section":"Tables 1, 2, 5 and Figure 4"},{"comment":"The caption labels (a) ε=8 and (b) ε=256, but the text says 'From 4b it is clear that the NCCR of the poisoned example is lower ... when ε is very small' and '4a shows that the NCCR ... is significantly higher ... when ε is large enough.' These references appear to be swapped.","section":"Figure 4"},{"comment":"The caption says the baseline is a row, but the baselines are shown as columns (BL1/BL2 and BL3); the presentation should be clarified so the reader can see which baseline result is the best in each group.","section":"Tables 3 and 4"},{"comment":"The 'To Do' section explicitly lists unfinished items: showing experimental results in more detail, reporting time cost, and discussing the effect difference of the two speaker-recognition models. These items should be completed before the paper can be considered complete, especially since the speaker-recognition results are claimed as a contribution.","section":"Section 8"}],"recommendation":"reject","confidential_remarks":"The manuscript appears to be an early preprint: Section 8 lists unfinished items, captions and table references are inconsistent, and the central claim is contradicted by the paper's own Table 3/4 results and by the backdoor analysis in Section 6.1. The NCCR measure may be salvageable as a restricted, norm-matched detector, but not in its current form as an attack-independent universal robustness metric and detector."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The NCCR idea is genuinely new and worth taking seriously: measuring how neuron activations change under random perturbation is a fast, attack-agnostic heuristic for both network and input robustness, and the authors test it across image, speaker, and backdoor settings. The empirical breadth is real, and comparisons against three baselines in image detection and two in speaker detection give a reasonable sense of where it stands. The backdoor detection via clustering over NCCR values is also a neat extension.\n\nThe soft spots, though, are not minor. The paper's central claim—that adversarial examples are always less robust, stated in the abstract and again in Section 5—is falsified by its own Table 3: classifiers trained on FGSM, PGD, or a combination get AUROC around 0.89–0.91 on JSMA, well below the 0.99+ they get on other attacks. The paper concedes JSMA is 'generally modest' because it modifies few pixels, but that concession undermines the attack-independent detection premise. The backdoor section makes the same admission: BadNets poisoned examples can be more robust than clean ones, and the method only works by cranking epsilon to 256 to destroy the trigger. So 'always less robust' is not true across the paper's own data.\n\nThe definition of NCCR is also imprecise. The text says 'the degree of change in F(x') relative to F(x)' and mentions an l2-norm, but there is no formula, no statement of which neurons are included, and no detail on how the perturbation magnitude interacts with the input scale. This makes the experiments hard to reproduce, especially since no code is provided. The classifier input dimensionality for the detection pipeline is never specified, and the results are reported as AUROC without error bars or statistical tests, even though the box plots in Figure 2 clearly show overlapping distributions.\n\nThe detection framework is also more modest than advertised: it is a supervised classifier trained on NCCR values of known attacks, so it inherits the assumption that unseen attacks share the same robustness deficit. When that assumption fails, as it does for JSMA, detection degrades. That is not a fatal flaw for the metric itself, but it should be presented as a limitation.\n\nIn short, this is an incomplete draft with a promising kernel. The metric could be useful after major revision: a precise definition, a corrected claim that does not overgeneralize, and evaluation across a wider range of attack norms. The paper also has a 'To Do' section admitting missing experiments, which is honest but confirms it is not ready.\n\nWho is this for? Someone working on lightweight adversarial detection or robustness measurement would get value from the core idea, but not from the paper as it stands. I would send it to review—the novelty and breadth deserve a referee's time—but I would expect heavy revision before it is acceptable.","headline":"A plausible robustness metric in search of a precise definition, whose own JSMA results contradict the paper's universal 'adversarial examples are always less robust' claim.","tokens_in":13732,"tokens_out":2021,"would_cite":false,"duration_ms":25630,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A network's robustness—and whether an input is adversarial—can be read from how much its neuron activations change under random perturbation.","keywords":["NCCR","neuron coverage change rate","adversarial example detection","robustness evaluation","backdoor attack detection","random perturbation","speaker recognition","image recognition"],"falsifier":"Generate adversarial examples with a strong attack such as C&W at high confidence and measure their NCCR at the paper's perturbation size; if a non-negligible fraction of them has NCCR below the upper range of clean examples, the premise that adversarial examples are always less robust is false and the detector cannot be universal.","tokens_in":12686,"feed_emoji":"🎯","tokens_out":6457,"duration_ms":70919,"temperature":0.7,"pith_summary":"The paper introduces a metric, the neuron cover change rate (NCCR), that measures how much a network's internal neuron activations shift when a small random perturbation is added to an input. The authors claim that a network with a low NCCR is genuinely robust, and that this single random-perturbation measurement ranks models without needing to run attacks. They further claim that adversarial examples are always less robust than clean examples, so an input with an NCCR far above the clean baseline is almost certainly adversarial. On image and speaker-recognition benchmarks the resulting detector reaches near-perfect AUROC, and a strong-perturbation variant separates backdoored inputs. If correct, this gives a fast, attack-independent robustness score and detector.","feed_headline":"Neuron-change score catches adversarial inputs almost every time","feed_subtitle":"Low neuron cover change marks stable networks; adversarial and backdoored inputs score high.","key_machinery":"The central object is the neuron cover change rate, a scalar built from the activation pattern of the network: record $F(x)$, the outputs of all neurons for input $x$; add small random noise $\\epsilon$ to get $x'$; then NCCR is the $\\ell^2$-norm distance between $F(x')$ and $F(x)$. It replaces static neuron coverage with a perturbation-sensitivity measure, and it carries the entire argument: low NCCR for a model means stability, high NCCR for an input means the input sits in a fragile region, and comparing an input's NCCR to a clean-example baseline is what turns the scalar into a detector. For backdoor detection, the same scalar is computed with a deliberately large perturbation so that any trigger is destroyed, which inverts the ordering and makes poisoned examples stand out.","core_discovery":"The central claim is that robustness is readable from internal activation change under random noise: for a trained network $F$ and input $x$, add a small random perturbation $\\epsilon$ to get $x'$, and define NCCR as the change in the set of neuron outputs $F(x)$ versus $F(x')$, measured with the $\\ell^2$ norm. A robust network keeps neuron activations nearly fixed, so its NCCR is low; a fragile network or a fragile input yields high NCCR. The paper reports that on MNIST, CIFAR-10, ImageNet, and LibriSpeech, adversarially trained models show lower NCCR than naturally trained ones, adversarial examples show higher NCCR than clean examples (with AUROC close to 1.0 against FGSM, PGD, C&W, DeepFool, JSMA, and speaker-domain attacks), and backdoored images, once perturbed strongly enough to destroy the trigger, separate cleanly from benign images. The authors conclude that NCCR simultaneously evaluates model robustness, flags adversarial inputs, and detects backdoor triggers.","pith_inferences":["If NCCR's separation holds under adaptive attacks, it could serve as a cheap model-agnostic filter deployed before inference, an extension the paper does not test.","The paper's observation that same-label inputs have similar neuron activations suggests NCCR might also detect out-of-distribution or mislabeled samples, though the paper does not claim this.","Because NCCR depends on perturbation size, a natural extension is an epsilon-sweep profile rather than a single scalar, which could reveal how robustness changes across perturbation scales.","The near-100% detection accuracy may degrade on attacks specifically optimized to keep examples far from decision boundaries, a regime the paper does not evaluate."],"forward_implications":["Model robustness can be ranked without running any attack: a single random-perturbation NCCR separates naturally trained from adversarially trained networks.","A low-cost classifier on NCCR values detects FGSM, PGD, C&W, DeepFool, and JSMA adversarial examples with AUROC close to 1.0 on MNIST, CIFAR-10, and ImageNet.","The same measurement transfers to speaker recognition, detecting PGD, C&W, and FAKEBOB attacks on LibriSpeech better than the two compared baselines.","Using a large perturbation so that triggers are destroyed, NCCR clustering separates BadNets, Blended, SIG, and ReFool backdoored inputs with F1 scores near 1.0."],"supporting_citations":[{"why":"Defines neuron coverage, the base activation-ratio concept that NCCR modifies and argues is inadequate.","marker":"[26]"},{"why":"Supplies the premise that adversarial examples lie near decision boundaries and therefore have lower robustness than clean examples.","marker":"[39]"},{"why":"Provides the Madry adversarially trained models used to validate that lower NCCR tracks higher robustness.","marker":"[21]"},{"why":"Defines FGSM, one of the attacks used to generate adversarial examples for training and evaluating the NCCR detector.","marker":"[11]"},{"why":"Defines the C&W optimization-based attack, a strong evaluation baseline for adversarial detection.","marker":"[2]"},{"why":"Introduces the L-BFGS attack and the adversarial-example problem that this work targets.","marker":"[31]"},{"why":"STRIP, the baseline backdoor detector whose F1 scores NCCR is compared against.","marker":"[9]"},{"why":"Activation clustering, the baseline clustering-based backdoor detector NCCR is compared against.","marker":"[3]"},{"why":"Baseline 1, label change rate under model mutation, one comparator for adversarial-example detection.","marker":"[34]"},{"why":"Baseline 2, crossing-layer input validation, another comparator for adversarial-example detection.","marker":"[33]"}],"fun_headline_variants":["Neuron cover change rate flags adversarial and backdoored inputs","Watch neurons jitter: low NCCR signals robust model and clean data","NCCR: one metric to assess robustness, detect attacks, and find backdoors","Robustness score from neuron changes catches adversarial and backdoored inputs","Neuron activation shift under noise reveals model fragility and poisoned data"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole detector rests on the premise that every adversarial example lies closer to a decision boundary than every clean example and therefore is always less robust under random perturbation.","fun_headline_variants_meta":{"raw":{"variants":["Neuron cover change rate flags adversarial and backdoored inputs","Watch neurons jitter: low NCCR signals robust model and clean data","NCCR: one metric to assess robustness, detect attacks, and find backdoors","Robustness score from neuron changes catches adversarial and backdoored inputs","Neuron activation shift under noise reveals model fragility and poisoned data"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000722,"raw_usage":{"total_tokens":3236,"prompt_tokens":940,"completion_tokens":2296,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":556,"completion_tokens_details":{"reasoning_tokens":2201}},"tokens_in":556,"tokens_out":2296,"duration_ms":20374,"temperature":1.0,"reasoning_tokens":2201,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T12:41:33.302991+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Generate adversarial examples with a strong attack such as C&W at high confidence and measure their NCCR at the paper's perturbation size; if a non-negligible fraction of them has NCCR below the upper range of clean examples, the premise that adversarial examples are always less robust is false and the detector cannot be universal.","supporting_citations":[{"cited_title":"Deepxplore: Automated whitebox testing of deep learning systems","cited_arxiv_id":null,"evidence_quote":"Defines neuron coverage, the base activation-ratio concept that NCCR modifies and argues is inadequate."},{"cited_title":"Attack as defense: Characterizing adversarial examples using robustness","cited_arxiv_id":null,"evidence_quote":"Supplies the premise that adversarial examples lie near decision boundaries and therefore have lower robustness than clean examples."},{"cited_title":"Towards evaluating the robustness of neural networks","cited_arxiv_id":null,"evidence_quote":"Defines the C&W optimization-based attack, a strong evaluation baseline for adversarial detection."},{"cited_title":"Strip: A defence against trojan attacks on deep neural networks","cited_arxiv_id":null,"evidence_quote":"STRIP, the baseline backdoor detector whose F1 scores NCCR is compared against."},{"cited_title":"Adversarial sample detection for deep neural network through model mutation testing","cited_arxiv_id":null,"evidence_quote":"Baseline 1, label change rate under model mutation, one comparator for adversarial-example detection."},{"cited_title":"Dissector: Input validation for deep learning applications by crossing-layer dissection","cited_arxiv_id":null,"evidence_quote":"Baseline 2, crossing-layer input validation, another comparator for adversarial-example detection."}],"review_version":1}