{"id":"098ff4ae-c3a3-49fb-a453-50355cc7f59b","arxiv_id":"2509.05318","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"Backdoor text samples show smaller log-probability changes under mask-filling perturbations than clean samples, which enables zero-shot backdoor detection without the poisoned model.","lead":"This paper proposes a way to spot poisoned text samples that carry hidden triggers for language-model backdoor attacks. It reports that these samples change less than clean samples under random word replacements, and uses that difference to flag them without access to the poisoned model.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Backdoor-specific claim is untested: NETE likely detects any low-likelihood/unnatural text, not trigger-bearing samples.","rationale":"The reader's weakest assumption is exactly the load-bearing concern I identify: the paper never tests whether non-malicious low-likelihood or unnatural text produces the same small perturbation discrepancy as backdoor samples. Since the detection statistic is identical to probability curvature used in machine-text detection, and since the paper's own experiments show the method detects adversarial examples and jailbreak prompts (both unnatural but not backdoor-triggered), the phenomenon is very plausibly a generic OOD/unnatural-text signal. This does not refute the empirical AUROC numbers, but it invalidates the central interpretive claim of backdoor specificity. The reader already assigned CONDITIONAL with high correctness risk; my analysis supports that verdict rather than moving it further. The concrete control experiment would settle the issue. I also note the Eq. (4) vs Algorithm 1 decision-rule inconsistency, but the OOD confound is more fundamental to the claimed contribution.","tokens_in":31123,"tokens_out":2542,"duration_ms":33739,"concrete_test":"Construct a control set of non-backdoor but unnatural texts matched to each trigger type: (a) clean sentences with random rare words inserted, (b) style-transferred clean sentences (e.g., poetry) never used in poisoning, (c) randomly shuffled clean sentences, and (d) DetectGPT-style machine-generated text. Run NETE with the same GPT-2 scorer, T5 mask-fill, and k=50 on each control versus clean, and on backdoor versus clean. If control-vs-clean AUROC is close to backdoor-vs-clean AUROC (within ~0.1), the method is a generic unnatural-text detector, not a backdoor detector; if control AUROC stays near 0.5, the backdoor-specific claim survives.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that backdoor samples exhibit smaller perturbation discrepancy d(x,p,q)=log p(x)-E_{x~q}log p(x~) than clean samples—is supported only by backdoor-vs-clean comparisons (Sec. 4.2, Tables 4–5). No control set of non-malicious but unusual or low-likelihood text is included. This matters because d is exactly the DetectGPT probability curvature used to separate machine-generated from human text, so it already tracks general log-probability geometry rather than backdoor triggers. The paper's own extension results—detecting Textbugger/Textfooler adversarial examples (Sec. 6.4, Fig. 8) and jailbreak samples with AUROC 0.91—demonstrate that NETE flags broad classes of OOD or unnatural text. Without a control for benign unnatural text, the observed separation likely reduces to out-of-distribution detection, and the claimed backdoor-specific phenomenon is unsupported. A related internal inconsistency compounds this: Equ. (4) says d<=ε => Backdoor, while Algorithm 1 uses a normalized statistic d_hat/σ_x > ε => Clean; the operational decision rule is therefore ambiguous, making the empirical numbers hard to interpret independently of the chosen rule.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes NETE, a zero-shot black-box detector for backdoor samples in NLP. The method computes a perturbation discrepancy d(x,pθ,q) = log pθ(x) − E_{x̃∼q(·|x)} log pθ(x̃), where perturbations are generated by a T5 mask-filling function, and classifies an input as backdoor when this quantity (or a normalized version) falls below a threshold ε. The central claim is an observed 'anomalous perturbation discrepancy consistency': backdoor samples exhibit smaller perturbation discrepancy than clean samples. Experiments cover word-level, sentence-level, syntactic, and style triggers on YELP, OLID, and COVID, plus LLM backdoor attacks, multi-trigger combinations, adversarial examples, and jailbreak attacks. The reported AUROC values generally exceed those of zero-shot baselines such as Log, Rank, Entropy, and ONION.","tokens_in":31465,"tokens_out":6668,"duration_ms":79098,"significance":"If the central claim holds, NETE would be a practical defense: it needs only an off-the-shelf scoring LM and a mask-filling perturbation function, with no access to the poisoned model, clean data, or extra training. The experimental breadth is a genuine strength: four classical attack types, five LLM backdoor scenarios, multi-trigger settings, and extensions to adversarial and jailbreak inputs, plus ablations over perturbation count and model size. A code URL is provided. However, the significance currently rests on two unproven points: (i) the perturbation-discrepancy gap is specific to backdoor samples rather than a general property of low-likelihood or unnatural text, and (ii) the method is truly zero-shot despite threshold calibration on known backdoor samples. The theoretical derivation in Sec. 5.3 also does not support the 'curvature' interpretation as written. These issues are load-bearing for the paper's main claims.","major_comments":[{"comment":"The central phenomenon (backdoor samples have smaller perturbation discrepancy than clean samples) is only tested against clean, in-distribution text. The statistic d(x,pθ,q) is exactly the DetectGPT probability-curvature statistic [44], which separates machine-generated from human text; the paper's own Sec. 6.4 shows the same statistic flags adversarial examples (Fig. 8) and jailbreak samples. Without a control set of non-malicious but unnatural/low-likelihood text (e.g., out-of-domain human text, random word substitutions, or machine-generated text), the observed separation may be a general out-of-distribution effect rather than a backdoor-specific property. This is load-bearing for the abstract and title; a control experiment is needed.","section":"Sec. 4.2, Tables 4–5; Sec. 6.4"},{"comment":"The derivation equating perturbation discrepancy to the negative Hessian trace is mathematically invalid. The symmetry step requires f(z)=f(−z), which is a condition on the function, not on the noise distribution; Hutchinson's estimator only requires E[z]=0 and E[zz^T]=I. Thus replacing f(x−z) by f(x+z) is unjustified. Moreover, mask-filling perturbations are discrete token substitutions and do not correspond to x+hz in a vector space, so the finite-difference approximation in Eq. (7) does not apply. The curvature interpretation should be either rigorously derived or explicitly labeled as intuition, not used as the theoretical foundation of the method.","section":"Sec. 5.3, Eq. (9)"},{"comment":"The threshold ε in Algorithm 1 is set using the mean perturbation discrepancy of 200 word-level backdoor samples. This contradicts the claimed zero-shot setting with 'no prior knowledge' and no extra data: the defender needs known backdoor samples to calibrate the detector. The numbers reported in the threshold analysis (0.60, 0.86, 0.84) are therefore not zero-shot results. The AUROC tables are threshold-independent, but the operational detector requires ε; without an unsupervised or fixed-choice rule, the practical zero-shot claim is not supported. Please report a threshold sweep and/or a principled unsupervised threshold.","section":"Sec. 6.3, Threshold analysis"},{"comment":"The formal decision rule in Eq. (4) uses the raw discrepancy d(x,pθ,q) with threshold ε, while Algorithm 1 uses the normalized statistic d̂/√σ̂_x² and returns Clean when it exceeds ε. These are different statistics, and the two rules can disagree. This ambiguity makes it unclear what threshold values in Sec. 6.3 refer to and hampers reproducibility. Please unify the notation and specify the exact decision statistic used for every reported number.","section":"Sec. 5.2, Eq. (4) vs. Algorithm 1"}],"minor_comments":[{"comment":"Equation (1) is not a well-formed function definition: it displays two branches without stating the conditioning, and d(x,pθ) is later overloaded with a different definition in Eq. (3). Please clarify the notation and distinguish the two quantities.","section":"Sec. 4.1, Eq. (1)"},{"comment":"The title and Sec. 4 use 'perturbation discrepancy consistency', but Eq. (3) and Algorithm 1 do not directly measure consistency across perturbations; they measure the mean discrepancy and normalize by the standard deviation. The relationship between these quantities and 'consistency' should be clarified.","section":"Sec. 5.2"},{"comment":"The manuscript contains numerous rendering artifacts: '/uni0000' glyph placeholders, unreadable labels in Fig. 1 and Figs. 4–9, and Table 2 appears to have empty cells. The final version needs to be regenerated and proofread.","section":"General / Figures"},{"comment":"The sentence about using an 'autoencoder' is unclear: T5 is an encoder-decoder transformer, not an autoencoder, and the claim that BERT or GPT cannot capture curvature effects is not supported. Please revise or remove.","section":"Sec. 5.2"},{"comment":"No seeds or repeated runs are reported. Several AUROC differences in Tables 4–5 are only 0.01–0.02; without variance estimates, it is hard to judge whether the improvements over the best baseline are significant.","section":"Sec. 6.1"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for a security or ML venue. The decisive issue for me is not the theory but the missing control: the reported phenomenon may simply be OOD detection. If the authors can demonstrate that backdoor samples separate from benign unnatural text, and if the threshold issue is reframed honestly, the contribution is solid. The current draft overstates both 'zero-shot' and 'backdoor-specific'."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know this paper for one reason: it proposes a genuinely practical zero-shot black-box backdoor-sample detector needing only a scoring LM and a mask-filling model. That setting is useful and the paper is honest about wanting to cover style triggers that perplexity and STRIP miss. It also releases code and evaluates against a broad set of attacks, including BadChain, BadEdit, and multi-trigger combos. That breadth is real credit.\n\nThe soft spot is the one the stress test flags: the claimed phenomenon is that backdoor samples have smaller perturbation discrepancy than clean samples, but every experiment is backdoor-vs-clean. No control set of non-malicious but low-likelihood or unnatural text. Given the statistic is exactly DetectGPT's curvature, the paper's own extension results—detecting Textfooler, Textbugger, and jailbreak samples—cut against backdoor specificity. Those are not backdoor triggers; they are just unusual text. So the central claim that this detects backdoors as such is unsupported. A benign OOD control is the single fix that would settle it, and its absence is the reason I would not call the finding established.\n\nOther issues are real but smaller. The Sec. 5.3 derivation has the symmetry mistake you noted: f(z)=f(-z) is assumed instead of symmetry of the noise distribution, and the mapping from discrete token perturbations to continuous Hutchinson vectors is hand-waved. Equ. (4) and Algorithm 1 disagree on the decision direction and on whether the statistic is normalized. The threshold in Sec. 6.3 is fitted to the mean discrepancy of 200 known word-level backdoor samples; that is a free parameter tuned on the test distribution, so the reported AUROCs flatter the method. No error bars anywhere. These are all addressable, but together they mean the empirical numbers are not yet independently interpretable.\n\nThe paper is not incoherent; the thinking is clear and the related work is engaged. The authors acknowledge the time limitation and the difficulty of restoring transformed samples. I just would not trust the backdoor-specific framing yet.\n\nBottom line: worth reading for the practical setting and as a potential OOD detector, but the backdoor claim needs that control. I would send it to review—a good referee can force the OOD experiment and the decision-rule cleanup. But I would not cite it as evidence of a backdoor-specific phenomenon in its current form.","headline":"The paper banks on a backdoor-specific perturbation-discrepancy phenomenon, but never tests it against benign unnatural text; as is, NETE looks like a general out-of-distribution detector in disguise.","tokens_in":31871,"tokens_out":900,"would_cite":false,"duration_ms":15430,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that backdoor samples in language models can be detected zero-shot by their smaller perturbation discrepancy under random mask-filling rewrites, and supports the claim with a curvature-based detector evaluated across word,","keywords":["backdoor attacks","backdoor sample detection","pre-trained language models","perturbation discrepancy","curvature","zero-shot black-box defense","mask-filling perturbation","jailbreak and adversarial detection"],"falsifier":"Take a set of clean but deliberately unnatural sentences—ungrammatical, rare-word, or machine-generated but non-malicious—score them with the same language model and mask-filling rewrites, and measure the normalized perturbation discrepancy. If these controls fall below the backdoor threshold as often as actual poisoned samples, the claimed trigger-specific phenomenon is not what drives detection.","tokens_in":31012,"feed_emoji":"🛡️","tokens_out":8409,"duration_ms":94016,"temperature":0.7,"pith_summary":"The paper tries to establish that poisoned text can be detected without access to the poisoned model, clean reference data, or extra training. Its central claim is that backdoor samples show a smaller and more stable drop in log-probability under random mask-filling rewrites than clean samples do, a property the authors call perturbation discrepancy consistency. The proposed method, NETE, estimates this discrepancy across about fifty perturbations and flags any input whose normalized score falls below a threshold. If the claim holds, the same black-box detector can clean third-party datasets before training and filter trigger-carrying inputs before inference, using only an off-the-shelf language model and a mask-filler. Experiments on word, sentence, syntactic, and style triggers, plus several large-language-model attack families, report higher AUROC than existing zero-shot baselines.","feed_headline":"Poisoned text barely flinches under random rewrites","feed_subtitle":"A zero-shot detector flags backdoor samples by their small log-probability drop after mask-filling perturbations.","key_machinery":"The load-bearing object is the perturbation discrepancy d(x,pθ,q) = log pθ(x) − E_{x∼q(·|x)} log pθ(x): the average drop in log-probability when random mask-filling rewrites of x are scored by an off-the-shelf language model. The paper interprets this discrepancy as curvature—more precisely, the negative Hessian trace of the scoring function at x, obtained through a stochastic trace estimator and a finite-difference approximation—so that small change under perturbation becomes low curvature. The detection pipeline couples this statistic with a mask-filling perturbation function q that randomly masks spans of length up to two until 10 percent of words are altered, then fills the gaps with a p","core_discovery":"The paper's central claim is that backdoor samples are anomalous in how their log-probability changes under random local perturbation. Define the perturbation discrepancy d(x,pθ,q) = log pθ(x) − E_{x∼q(·|x)} log pθ(x), where q generates mask-filling rewrites of x. The authors report that for backdoor samples this discrepancy is systematically smaller and less variable than for clean samples, across word, sentence, syntactic, style, and several LLM-based triggers. They identify this quantity with the negative trace of the Hessian (curvature) of the scoring model at the sample, via a stochastic trace estimator and finite-difference approximation. NETE estimates the discrepancy over k random pe","pith_inferences":["A testable extension the paper does not run: compare the same statistic on clean but unusual text (ungrammatical, rare-word, or machine-generated benign sentences) to see whether the signal is specific to backdoor triggers or is a general low-likelihood, out-of-distribution cue.","The curvature interpretation suggests NETE-type screening may transfer to other domains where a likelihood-scoring model and a local perturbation function exist, such as image or speech classifiers, though the mask-filling mechanism would need a domain analogue.","The threshold is set from known word-level backdoor samples and applied to unknown attacks; a direct follow-up would measure how threshold calibration transfers across trigger types and datasets without any poisoned examples at all."],"forward_implications":["A defender who can only query a pretrained model as a black box can filter backdoor samples without clean data, poisoned-model access, or retraining.","The same detection procedure applies at two points in the lifecycle: purging poisoned items from third-party datasets before training, and blocking trigger inputs before inference.","Style-based triggers, which are weakly correlated with backdoor behavior and evade methods that rely on trigger-behavior association, become detectable because they still lower perturbation discrepancy.","Detector cost can be kept low: performance saturates near 50 perturbations and is largely insensitive to the size of the scoring and mask-filling models.","The same perturbation-discrepancy signal also flags adversarial examples and mutation-based jailbreak samples, not just poison-style backdoors."],"supporting_citations":[{"why":"Supplies the log-probability curvature metric that the perturbation-discrepancy measure adapts for backdoor detection.","marker":"[44]"},{"why":"Supplies the stochastic trace estimator used to equate perturbation discrepancy with Hessian trace.","marker":"[46]"},{"why":"Supplies the mask-filling perturbation strategy on which the automated rewrite function is based.","marker":"[29]"},{"why":"Supplies the off-the-shelf language model used to score original and perturbed samples in the main experiments.","marker":"[42]"},{"why":"Supplies the text-to-text mask-filling model used to generate the random perturbed samples.","marker":"[6]"},{"why":"Provides the random-substitution detection baseline whose failure on style triggers motivates the curvature-based approach.","marker":"[28]"},{"why":"Provides the closest zero-shot black-box correction baseline that NETE is compared against.","marker":"[14]"},{"why":"Defines the style-trigger backdoor attack family that NETE targets, where trigger-behavior correlation is weakest.","marker":"[7]"}],"fun_headline_variants":["Backdoor text resists rewrite wobble","Zero-shot backdoor detector spies tiny log-prob shifts","Poisoned samples flinch less under mask-fill perturbations","Log-prob curvature unmasked: backdoors stand out","Mask-filling tells backdoor samples apart"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The method assumes that the small, stable perturbation discrepancy of backdoor samples is caused by the trigger itself, not by the mere fact that triggered text is unusual or low-probability under the scoring model; if any non-malicious unnatural text behaves the same way, the detector flags benign outliers too.","fun_headline_variants_meta":{"raw":{"variants":["Backdoor text resists rewrite wobble","Zero-shot backdoor detector spies tiny log-prob shifts","Poisoned samples flinch less under mask-fill perturbations","Log-prob curvature unmasked: backdoors stand out","Mask-filling tells backdoor samples apart"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000153,"raw_usage":{"total_tokens":1067,"prompt_tokens":790,"completion_tokens":277,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":534,"completion_tokens_details":{"reasoning_tokens":201}},"tokens_in":534,"tokens_out":277,"duration_ms":4033,"temperature":1.0,"reasoning_tokens":201,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T13:39:12.659313+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a set of clean but deliberately unnatural sentences—ungrammatical, rare-word, or machine-generated but non-malicious—score them with the same language model and mask-filling rewrites, and measure the normalized perturbation discrepancy. If these controls fall below the backdoor threshold as often as actual poisoned samples, the claimed trigger-specific phenomenon is not what drives detection.","supporting_citations":[{"cited_title":"Mitchell, Y","cited_arxiv_id":null,"evidence_quote":"Supplies the log-probability curvature metric that the perturbation-discrepancy measure adapts for backdoor detection."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the stochastic trace estimator used to equate perturbation discrepancy with Hessian trace."},{"cited_title":"Devlin, M","cited_arxiv_id":null,"evidence_quote":"Supplies the mask-filling perturbation strategy on which the automated rewrite function is based."},{"cited_title":"Radford, J","cited_arxiv_id":null,"evidence_quote":"Supplies the off-the-shelf language model used to score original and perturbed samples in the main experiments."},{"cited_title":"Raffel, N","cited_arxiv_id":null,"evidence_quote":"Supplies the text-to-text mask-filling model used to generate the random perturbed samples."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the random-substitution detection baseline whose failure on style triggers motivates the curvature-based approach."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the closest zero-shot black-box correction baseline that NETE is compared against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the style-trigger backdoor attack family that NETE targets, where trigger-behavior correlation is weakest."}],"review_version":1}