{"id":"667c166f-fcc4-4ff4-b072-2d4229f7679a","arxiv_id":"2608.08027","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"BASIS uses two linear probes on prefill attention to separately detect injection presence and predict whether the injection would actually compromise the model, refusing only when both fire.","lead":"This paper introduces BASIS, a defense that reads an LLM's attention patterns during the prefill pass and only blocks a prompt when it predicts the model would actually follow an injected instruction, instead of blocking every prompt that contains an injection. It reports much lower over-refusal on safe injection-containing inputs while keeping injection detection near-perfect across six models and four tasks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Breach probe generalization to unseen attacks and templates is unvalidated; the paper's own adaptive GCG result shows ASRdef rising to 24%, so the central selectivity claim holds only within the training distribution.","rationale":"The reader identified the breach probe's generalization to unseen attack types and instruction templates as the weakest assumption. My stress-test concurs and sharpens the concern with two concrete anchors: the absence of any cross-attack or cross-template evaluation for the breach probe (unlike the existence probe in Section V-A2), and the paper's own adaptive GCG result in Table VI, where only 50 optimized samples reduce Breach Recall from 94.0% to 73.3% and raise ASRdef to 24.0%. That result demonstrates that the linear breach probe is not robust even to modest adversarial distribution shift, which directly undermines the unqualified abstract claim that BASIS 'refuses only when the model would actually be compromised.' I considered alternative concerns, such as under-specified threshold selection and the BASIS/RAPID naming inconsistency. Threshold selection is a legitimate methodological gap, but it is addressable and does not threaten the architecture's core idea; the naming issue is cosmetic. The breach-probe generalization problem is more load-bearing because the entire selective-defense benefit depends on the breach probe's accuracy in deployment, where attackers will choose novel prompts and templates. The proposed concrete test would settle the concern by measuring ASRdef and FPR-S on a held-out attack type and a held-out instruction template, and by scaling the adaptive attack well beyond 50 samples. The reader's CONDITIONAL verdict remains appropriate: the paper presents a plausible mechanism and strong same-distribution results, but the central claim is conditional on breach-probe generalization that has not been demonstrated. The verdict should remain UNCHANGED because my concern reinforces the reader's conditional status rather than moving it to ACCEPT or REJECT.","tokens_in":34235,"tokens_out":5054,"duration_ms":61038,"concrete_test":"Train the breach probe on six non-adaptive attack types and instruction levels L0-L4 only, then evaluate on (a) a held-out novel attack type not in the training set, such as encoded or obfuscated injections, and (b) a new instruction template L6 that is structurally distinct from L0-L5. Measure ASRdef and FPR-S at the fixed thresholds used in Table III. Separately, rerun the GCG adaptive attack against the breach probe with 500 optimized samples and multiple restarts, not 50. If ASRdef exceeds 10% or Breach Recall falls below 90% on any of these held-out conditions, the claim that BASIS refuses only when the model would actually be compromised is not supported outside the training distribution.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that BASIS rejects an input only when the injection would actually compromise the model, enabling selective defense without over-refusal. This hinges on the breach probe (Section IV-C) correctly predicting y_breach for inputs outside its training distribution. The paper validates the existence probe across six attack types (Section V-A2), but no analogous cross-attack or cross-template evaluation is reported for the breach probe. The breach probe is trained on attack samples constructed from six fixed templates and evaluated only on held-out samples from those same templates; novel attack formats, novel instruction templates, and novel target-string distributions are untested. This is load-bearing because the cascade's pass decision for injection-containing inputs rests entirely on p_breach. The adaptive attack in Table VI makes the risk concrete: with only 50 GCG samples optimized against the breach probe, Breach Recall drops from 94.0% to 73.3% and ASRdef rises to 24.0%. Since an attacker can adapt, the advertised property 'refuses only when the model would actually be compromised' is not robust to distribution shift. A further mechanistic concern is that the breach probe may be learning instruction-level or template-level cues rather than per-sample breach: Table III shows that |S_safe| changes drastically across L0-L5 (322 to 500), and the attention feature rho includes instruction-side attention, so the probe could exploit template identity. If so, its apparent accuracy on the same-distribution test set would not transfer to deployment with a new prompt template or a new attack style. The paper acknowledges the adaptive weakness as future work, but the abstract and conclusion state the selectivity result without this caveat, making the generalization gap the key condition the central claim depends on.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes BASIS, a prompt-injection defense that separates detection of an injected instruction from prediction of whether the target model will actually follow it. BASIS extracts a per-head Attention Competition Ratio rho from the target model's prefill pass, trains two elastic-net logistic probes (an existence probe on clean+attack samples, a breach probe on attack samples only), and uses a cascaded decision rule that rejects an input only when both probes exceed thresholds. An offline Instruction Robustness Score (IRS) aggregates breach-probe predictions over attack samples to rank instruction templates. Experiments cover four tasks, six instruction levels, and six open-source LLMs, reporting near-perfect injection detection, large reductions in FPR-S relative to detection-only baselines, a strong IRS--ASR correlation, and an adaptive-attack evaluation with GCG and AutoDAN.","tokens_in":34527,"tokens_out":5045,"duration_ms":55342,"significance":"If the central claim holds, BASIS addresses a real and under-studied problem: over-refusal on injection-containing inputs that the model would in fact resist. The distinction between injection presence and injection effectiveness is well motivated, and the design is attractive because the online decision adds no extra LLM inference beyond the existing prefill pass. The paper deserves credit for a broad evaluation grid (4 tasks x 6 models x 6 instruction levels), a cross-attack transfer matrix for the existence probe, sample-size sensitivity analysis, and a direct comparison of dual-probe versus unified-probe architectures. The main weakness is that the breach probe, which carries the entire selectivity burden, is only evaluated on held-out samples from the same attack distribution used for training; the adaptive-attack results show that this burden is not yet robust. The IRS validation is also performed on the same attack distribution. With additional generalization experiments and a clearer threshold-selection protocol, the contribution could be solid.","major_comments":[{"comment":"The breach probe's generalization across attack types and instruction templates is never tested. Section V-A2 reports a 6x6 transfer matrix for the existence probe, with AUROC=1.000 and low log-loss in every cell, but no analogous transfer experiment is reported for the breach probe. The cascade's pass decision for injection-containing inputs rests entirely on p_breach, so if the breach probe does not transfer to unseen attack formats or unseen template phrasings, the advertised property that BASIS 'refuses only when the model would actually be compromised' is unsupported. This is not a hypothetical concern: Table VI shows that a 50-sample GCG optimization against the breach probe drops Breach Recall from 94.0% to 73.3% and raises ASRdef to 24.0%. I recommend adding a breach-probe transfer matrix across attack types (or, more convincingly, evaluating on a held-out set of novel attack templates) and reporting FPR-S and ASRdef for those held-out conditions.","section":"Section V-A2 and Section IV-C"},{"comment":"The operating point at which Table III reports FPR-S and ASRdef is not reproducible because the threshold selection procedure is not described. The text says that BASIS-Full is shown as a curve obtained by fixing tau_p and sweeping tau_b, but it does not state the value of tau_p, how tau_b is chosen for the reported point, or whether thresholds are selected on a validation set or on the test set. Since FPR-S and ASRdef are threshold-dependent, the headline numbers such as FPR-S=6.2-12.4% and ASRdef=1.2-3.8% at L0-L1 could reflect favorable test-set-based threshold tuning. The authors should specify the threshold-selection rule (e.g., choose tau_p and tau_b on a validation split, or report full ROC-style trade-off curves with confidence intervals) so that the reported operating point is meaningful and reproducible.","section":"Section V-B2 and Figure 3"},{"comment":"The IRS--ASRraw correlation is computed on the same attack distribution used to train the breach probe: IRS is aggregated from p_breach on the corresponding test attack set, and ASRraw is measured on that same set. While the test set is held out from training, it is still drawn from the same six attack templates and the same instruction-level configurations, so the Spearman rho=-0.99 largely reflects in-distribution calibration of the breach probe. The claim that IRS is 'a reliable proxy for pre-deployment instruction robustness' therefore overreaches unless IRS is validated under distribution shift, for example by computing IRS on held-out attack types or on novel instruction templates not seen in breach-probe training.","section":"Section V-C1 and Figure 4"},{"comment":"The adaptive-attack results substantially weaken the central selectivity claim, and the presentation should reflect this. For GCG optimized to evade the breach probe, ASRdef rises to 24.0% and Breach Recall falls to 73.3%; this means that the 'refuse only when actually compromised' property fails under a modest adaptive attack (50 samples). The manuscript acknowledges this as a limitation, but the abstract and conclusion still state the selectivity property without this caveat. I recommend either tempering the central claim to in-distribution or non-adaptive settings, or adding a more substantial adaptive-defense component (e.g., adversarial training of the breach probe, or a randomization/ensemble mechanism) and reporting results under a larger adaptive budget.","section":"Table VI and Section V-B4"}],"minor_comments":[{"comment":"The over-refusal tables and Figure 13 in Appendix B-C use the name 'RAPID' instead of 'BASIS' (e.g., RAPID-Full, RAPID-Existence), which appears to be a leftover from an earlier version of the manuscript and should be made consistent.","section":"Appendix B-C"},{"comment":"The cross-reference 'Section IV-D0a' is malformed; it should refer to the IRS definition section by its proper label, e.g., 'Section IV-D' or 'Section IV-D(a)'.","section":"Section IV-D"},{"comment":"The caption states that BASIS-Full is shown as a curve obtained by fixing tau_p and sweeping tau_b, but neither tau_p nor the selection criterion for the reported point is given; this is part of the reproducibility issue raised in the major comments.","section":"Figure 3 caption"},{"comment":"The column 'N (breach/safe)' for the non-adaptive Mixed row reads '50 / 450' while the text says the non-adaptive baseline draws 500 mixed-attack samples; please clarify whether the reference set is 500 samples or 50 samples with a 10% raw ASR.","section":"Table VI"},{"comment":"AUROC and F1 values of 1.000 are reported without confidence intervals or any measure of variance. Given the sample sizes, it would be helpful to report bootstrapped intervals or at least the number of test samples per cell to confirm that saturation is not an artifact of a small test set.","section":"Section V-A1 and Tables I-II"},{"comment":"The equations for FPR-S and ASRdef are referenced as 'Eqs. (III-B)' in Section V-B1, which is not a valid equation label; please use the actual equation numbers.","section":"Section III-B"}],"recommendation":"major_revision","confidential_remarks":"The manuscript appears to be a rebranded version of a system previously called 'RAPID', based on the leftover naming in Appendix B-C. The editor may want to verify that the paper has not been published under the earlier name. The main technical risk is the unvalidated transfer of the breach probe; the adaptive-attack numbers already show a concrete failure mode, so this is not merely a hypothetical concern."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the short version: this is a real contribution to prompt-injection defense, not a repackaging of Attention Tracker. The new idea is to separate 'does the input contain an injection' from 'would the target model actually follow it,' and only reject when both hold. That is a genuinely different target than prior detection work, and the FPR-S metric gives the field a clean way to measure the over-refusal cost. The two-probe cascade is simple and the offline IRS is a useful pre-deployment tool if the probe generalizes.\n\nThe evaluation is broad and honestly run: four tasks, six models, six instruction levels, six non-adaptive attack types plus adaptive GCG and AutoDAN, with ablations on regularization, training size, and feature choice. The existence probe's cross-attack transfer matrix is a nice piece of evidence. The mechanism analysis showing that breach prediction uses many more heads than existence detection is also plausible and worth reporting. The adaptive-attack results are presented without spin: GCG targeting the breach probe drops Breach Recall to 73.3% and raises ASRdef to 24%, and the paper says this is future work.\n\nThe soft spots are real, and they cluster around the breach probe. First, there is no cross-attack or cross-template generalization test for the breach probe, unlike the existence probe. That is load-bearing, because the cascade's pass decision for injection-containing inputs rests entirely on p_breach. If the breach probe has learned instruction-level or template-level cues, it will not transfer to a new deployment prompt. Second, the IRS calibration is in-sample: the robustness score is computed on the same attack distribution used to train the breach probe, so the Spearman -0.99 says less than it appears. Third, threshold selection for the reported operating points is not described, which matters for a defense you would deploy. Also, no code or data is released, which is a problem for a security paper making defense claims. Minor: the paper switches names between BASIS and RAPID in the main text and appendices, and the abstract's 'refuses only when the model would actually be compromised' overstates what the adaptive-attack results support.\n\nWho is this for? Researchers working on LLM application security, especially prompt-injection defense. The concept of breach prediction is worth engaging with and building on, even if this instantiation has a generalization gap. I would send it to peer review, but the referees should push for a breach-probe transfer study, a described threshold-selection procedure, and ideally a code release. The core idea is sound; the current claims are too broad.","headline":"A genuinely useful separation of injection presence from breach prediction, with solid breadth but a load-bearing generalization gap in the breach probe.","tokens_in":35107,"tokens_out":2564,"would_cite":true,"duration_ms":26901,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A prompt-injection defense should refuse only when the target model would actually follow the injected instruction, and that is predictable from prefill attention alone.","keywords":["prompt injection","LLM security","breach prediction","over-refusal","attention competition ratio","linear probing","instruction robustness","adaptive attacks"],"falsifier":"Run the trained breach probe against an attack family built from templates and optimization objectives absent from training, such as multilingual injections, role-play hijacks, or a GCG suffix optimized directly against the probe; the central claim fails if Breach Recall drops well below the reported 94.0% while over-refusal rises, because the cascade would then either let breaches through or over-refuse again.","tokens_in":34040,"feed_emoji":"🛡️","tokens_out":6155,"duration_ms":64755,"temperature":0.7,"pith_summary":"The paper argues that prompt-injection defenses should ask not only whether an input contains an injected instruction, but whether the target model would actually obey it. It introduces BASIS, a defense that reads a model's prefill attention and trains two sparse linear probes: one detects injection presence, the other predicts whether the injection will breach the model. The online cascade rejects an input only when both probes fire. Across six open-source LLMs and four tasks, BASIS keeps injection detection near perfect while cutting rejection of injection-containing but harmless inputs from near 100% to as low as 0% on strong instruction templates. It also shows the breach probe doubles as a pre-deployment score for how robust a given instruction template is on a given model.","feed_headline":"Selective injection defense cuts needless rejections to near zero","feed_subtitle":"Two attention probes decide whether a model would actually follow an injected instruction before refusing.","key_machinery":"The load-bearing object is the Attention Competition Ratio per layer-head pair, $\\rho_{l,h}=S^{(l,h)}_{\\mathrm{ins}}/(S^{(l,h)}_{\\mathrm{ins}}+S^{(l,h)}_{\\mathrm{data}}+\\epsilon)$, the share of the last token's attention that a head pays to instruction tokens versus data tokens. Two sparse elastic-net logistic probes take the vector of all layer-head ratios as features: the existence probe is trained on clean-versus-attack labels, and the breach probe only on attack samples whose labels come from an offline decoding pass. The cascade decision Reject iff $p_{\\mathrm{exist}}>\\tau_p$ and $p_{\\mathrm{breach}}>\\tau_b$ is the intersection of two linear half-spaces in feature space, which can represent rejection regions a single linear probe cannot. The same breach probe is reused offline to compute the Instruction Robustness Score.","core_discovery":"The central claim is that injection awareness and instruction hijacking are distinct, internally readable phenomena: whether an injected instruction is present is a different question from whether the model will follow it. BASIS trains two elastic-net logistic probes on the same per-head attention competition features. The existence probe separates clean inputs from attacked inputs, and the breach probe, trained only on attack samples with labels obtained from one offline decoding pass, separates attacks the model resists from attacks it follows. The cascade rejects exactly when both probes exceed their thresholds, so an input containing an injection the model would resist is passed rather than refused. In the reported evaluation this preserves AUROC and F1 of 1.000 for injection detection across tasks and models while reducing over-refusal on non-breaching attack inputs to between 0% and about 12% on the weakest templates, with residual attack success after defense typically at or below a few percent. The same breach probe, aggregated offline, gives an Instruction Robustness Score that tracks empirical attack success with a Spearman correlation of $-0.99$ across 144 model-task-template configurations.","pith_inferences":["A natural extension the paper does not test is cross-model transfer: since probes are trained per model-task pair, one could check whether an existence or breach probe trained on one model family predicts breach on another model without retraining.","The same separation of 'is the attack present' from 'will it succeed' could be applied to jailbreak defense or content-policy filters, which also currently over-refuse on benign prompts that merely mention attacks.","Because the features are taken from the last prefill token, extending them to a few generated tokens or to multiple positions in the context might catch injections that only take effect after the response begins.","A stress test worth running is to phrase injections as legitimate task continuations or as part of the data's own format; if the breach probe's calibration shifts, the threshold $\\tau_b$ would need per-template tuning in deployment."],"forward_implications":["Deploying the cascade instead of a detection-only defense would let applications answer attack-containing inputs the model can resist; on sentiment classification with Qwen3-8B, the over-refusal rate on safe attack inputs drops from 100% to 12.4% at the weakest template and to 0% at the strongest, while residual attack success stays at or below 3.8%.","The Instruction Robustness Score offers a pre-deployment way to compare instruction templates for a fixed model; the reported near-monotone correlation with raw attack success gives developers a concrete signal for choosing templates before serving traffic.","The two-probe design keeps decisions inspectable: a small set of early-layer heads flags injection presence, while a broader set of middle-to-deep heads predicts breach, so failures can be traced to specific attention heads.","An attacker who can optimize against the breach probe degrades the defense, with Breach Recall falling from 94.0% to 73.3% and residual attack success reaching 24.0% in the GCG adaptive case, so the claimed selectivity is bounded by the probe's robustness to adaptive optimization."],"supporting_citations":[{"why":"Supplies the formalization of prompt injection attacks and defenses and the taxonomy that BASIS extends with breach prediction.","marker":"[2]"},{"why":"Provides the prefill attention signal and a baseline showing attention patterns reveal injection, which BASIS repurposes into per-head competition features.","marker":"[10]"},{"why":"GCG is the gradient-based adaptive attack used to test the breach probe under white-box optimization against the defense.","marker":"[15]"},{"why":"AutoDAN is the evolutionary adaptive attack used alongside GCG to evaluate evasion of the existence and breach probes.","marker":"[16]"},{"why":"Elastic-net regularization is what produces the sparse, interpretable probe weights that let BASIS select a small set of attention heads.","marker":"[22]"},{"why":"The sandwich defense technique defines instruction level L4, where the largest robustness gains appear in the evaluation.","marker":"[5]"},{"why":"The instruction and security declaration technique defines level L5, the most robust instruction templates in the evaluation.","marker":"[6]"}],"fun_headline_variants":["Attention probes refuse only when injection would actually breach","Selective defense: stop rejecting input that won't be hijacked","Breach-aware gating cuts over-refusal while keeping perfect detection","Two probes know if injection will follow before you refuse"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the breach probe, trained on attack samples from a fixed set of templates, continues to predict compromise on the unseen attack styles and prompt phrasings that appear in deployment.","fun_headline_variants_meta":{"raw":{"variants":["Attention probes refuse only when injection would actually breach","Selective defense: stop rejecting input that won't be hijacked","Breach-aware gating cuts over-refusal while keeping perfect detection","Two probes know if injection will follow before you refuse"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000565,"raw_usage":{"total_tokens":2699,"prompt_tokens":983,"completion_tokens":1716,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":599,"completion_tokens_details":{"reasoning_tokens":1660}},"tokens_in":599,"tokens_out":1716,"duration_ms":12304,"temperature":1.0,"reasoning_tokens":1660,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T00:32:01.230793+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the trained breach probe against an attack family built from templates and optimization objectives absent from training, such as multilingual injections, role-play hijacks, or a GCG suffix optimized directly against the probe; the central claim fails if Breach Recall drops well below the reported 94.0% while over-refusal rises, because the cascade would then either let breaches through or over-refuse again.","supporting_citations":[{"cited_title":"Autodan: Generating stealthy jailbreak prompts on aligned large language models,","cited_arxiv_id":null,"evidence_quote":"AutoDAN is the evolutionary adaptive attack used alongside GCG to evaluate evasion of the existence and breach probes."},{"cited_title":"Regularization and variable selection via the elastic net,","cited_arxiv_id":null,"evidence_quote":"Elastic-net regularization is what produces the sparse, interpretable probe weights that let BASIS select a small set of attention heads."},{"cited_title":"Sandwich defense,","cited_arxiv_id":null,"evidence_quote":"The sandwich defense technique defines instruction level L4, where the largest robustness gains appear in the evaluation."},{"cited_title":"Instruction defense,","cited_arxiv_id":null,"evidence_quote":"The instruction and security declaration technique defines level L5, the most robust instruction templates in the evaluation."}],"review_version":1}