{"id":"ba84d001-7fec-44e4-bd09-98ce0029b4af","arxiv_id":"2501.10639","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"LATPC combines variance-based selection of refusal features for adversarial training with an inference-time embedding calibrator, reducing jailbreak success while curbing over-refusal across several LLMs.","lead":"The authors propose LATPC, a two-step defense that first adversarially trains an LLM to refuse harmful requests even when safety-critical latent features are removed, then adjusts embeddings at inference to reduce over-refusals. The method lowers jailbreak success on several attacks and models, but the calibration weakens safety on Mistral and the evaluation lacks error bars.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Post-aware calibration changes ASR on truly harmful queries, contradicting its pseudo-harmful-only trigger; the safety-utility balance may be driven by an uncharacterized second intervention.","rationale":"The reader's CONDITIONAL verdict is appropriate, but my stress-test identifies a different load-bearing weakness than the one highlighted in the reader's weakest_assumption. The variance-direction contradiction (§4.2 says select lowest variance; §5.6 says highest variability) is real and should be fixed, yet it is a description mismatch: if the code selects high-variance dimensions, the implemented method may still work, and the central empirical claim can survive a corrected narrative. The PAC inconsistency is more serious because it strikes at the attribution of the reported safety-utility balance. Section 4.4 explicitly states PAC adjusts only pseudo-harmful queries and preserves refusal behavior for truly harmful inputs. If that is true, ASR on HarmBench and AdvBench should be unchanged by PAC. Table 4 shows a 54-point ASR drop on Llama3 under Adaptive Attack when PAC is enabled, and a 15.7-point ASR increase on Mistral. These results cannot both be explained by a mechanism that only touches benign but over-refused queries; they indicate PAC is acting as a second safety-affecting intervention whose decision boundary was trained only on OR-Bench versus harmless data. The paper treats PAC as a utility-recovery add-on, but the evidence suggests it is entangled with safety, especially because the same table shows PAC also moves over-refusal rates substantially (e.g., Llama3 ORR 36.2 with PAC versus 52.3 without, Mistral 45.0 versus 83.1). This does not make the method fraudulent or worthless; it makes the central claim under-specified. The concrete check I propose would settle the issue cleanly: ablate PAC across all attacks and models, and measure how often true harmful queries trigger calibration. If harmful triggers are frequent, the safety results need to be re-attributed and the calibrator needs its own robustness evaluation. If they are rare or absent, the ASR deltas in Table 4 are an unexplained artifact that must be corrected before the balance claim is accepted. Either way, the condition is addressable with the public code, so a CONDITIONAL verdict rather than REJECT or ACCEPT is the right call.","tokens_in":24525,"tokens_out":10121,"duration_ms":111736,"concrete_test":"Run the released code with PAC enabled versus disabled across all five attack types and all four models, extending Table 4 which covers only Adaptive Attack on three models, and log the fraction of HarmBench and AdvBench harmful queries that satisfy the PAC trigger P_h > P0 at each calibrated layer. If that fraction is near zero, Table 4's Llama3 ASR delta (67.9 to 13.8) cannot be reproduced and the evaluation is inconsistent; if it is large, the pseudo-harmful-only guarantee in §4.4 is false, and PAC must be evaluated as a safety intervention in its own right, including a check for overlap between OR-Bench training data and the test attack prompts.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In §4.4, PAC is defined as an inference-time adjustment triggered only when a linear classifier (Eq. 10) labels a query as 'pseudo-harmful' (benign but over-refused), with the explicit statement that it 'preserves the safety behavior for truly harmful inputs.' Under that trigger, PAC cannot change ASR on HarmBench or AdvBench harmful queries. Yet Table 4 reports PAC lowers Llama3-8B Adaptive-Attack ASR from 67.9% to 13.8% (Δ=54.1) and raises Mistral-7B ASR from 10.7% to 26.4% (Δ=-15.7). If harmful queries are not adjusted, the Llama3 ASR delta should be roughly zero; if they are adjusted, the paper's pseudo-harmful-only guarantee is false. Either way, the reported safety numbers after PAC cannot be attributed solely to latent adversarial training, and the claimed 'superior safety-utility balance' is not characterized. The paper offers no ablation of PAC across all five attacks (only Adaptive Attack for three models), no fraction of harmful queries that trigger calibration, and no out-of-distribution test of the calibrator. This is more load-bearing than the low- versus high-variance wording issue: it affects whether the central claim reflects the proposed training method or a second, unvalidated intervention.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes LATPC, a two-step defense against jailbreak attacks. In the first step, safety-critical latent dimensions are identified by computing per-dimension variance of hidden-state differences between harmful and harmless queries, and a mask-based refusal-feature-removal attack is applied during LoRA-based adversarial training. In the second step, a linear classifier on hidden states detects 'pseudo-harmful' queries at inference and adjusts their latent representations toward the harmless direction to reduce over-refusals. The paper reports experiments on Llama3-8B, Mistral-7B, Gemma-7B, and Qwen2-7B across GCG, PAIR, HumanJailbreak, GPTFUZZER, and Adaptive Attack, with ASR, MMLU, TriviaQA, and OR-Bench over-refusal metrics, claiming a superior safety-utility balance over eight baselines.","tokens_in":24805,"tokens_out":5610,"duration_ms":50911,"significance":"If the results were fully decomposable, the paper would make a useful contribution: it provides public code, evaluates on multiple models and attack families, compares against several strong baselines, and includes latent-space analyses (PCA and cosine-similarity) that support the intuition that jailbreak attacks move harmful queries toward harmless representations. The proposed inference-time calibration is lightweight. However, the manuscript currently does not establish that the reported safety gains come from the adversarial training component as opposed to the calibration component, and the over-refusal evaluation partly overlaps with the calibration training data. These issues must be resolved before the central claim can be accepted.","major_comments":[{"comment":"The PAC trigger is defined so that the adjustment is applied only when P_h(H_q) > P0, with the text stating that this 'preserves the safety behavior for truly harmful inputs.' Under that trigger, PAC cannot change ASR on harmful queries such as Adaptive Attack. Yet Table 4 reports PAC changes ASR on Llama3-8B from 67.9% to 13.8% and on Mistral-7B from 10.7% to 26.4%. If harmful queries do not trigger calibration, these ASR deltas should be approximately zero; if they do trigger it, the paper's pseudo-harmful-only guarantee is false. In either case, the safety improvement attributed to LATPC in Tables 1-3 is not attributable solely to latent adversarial training, because the reported 'with PAC' numbers include a second, uncharacterized intervention.","section":"§4.4, Eq. (10)-(12); Table 4"},{"comment":"Section 4.2 sorts the per-dimension variances in ascending order and selects the Top-k dimensions with the lowest variance, justifying this by 'stability across instances.' Section 5.6, however, describes the variance-based method as selecting dimensions with the 'highest variability across samples,' and Fig. 9(b) is captioned as showing that the variance-based approach maintains a balanced positive/negative ratio. These are opposite selection criteria. If the actual implementation selects high-variance dimensions, the formal description in Eq. (4)-(5) is wrong; if it selects low-variance dimensions, the mechanistic story in Sec. 5.6 and Fig. 9 is wrong. This ambiguity directly affects which latent directions are attacked during adversarial training and must be resolved with a precise statement and, ideally, a reproducibility script.","section":"§4.2, Eq. (4)-(5); §5.6, Fig. 9"},{"comment":"The calibrator is trained on pseudo-harmful queries from OR-Bench, with 13 samples drawn from each safety category, and over-refusal is later evaluated on 130 pseudo-harmful queries from OR-Bench. The paper only states that the calibration samples avoid overlap with the adversarial training set; it does not state that the evaluation OR-Bench queries are disjoint from the calibration training queries. With 13 samples per category and 130 evaluation queries, it is possible that the calibration set and evaluation set are exactly the same. If so, the reported ORR reductions (e.g., Table 4) are an in-distribution result and the over-refusal claim is circular. A held-out split and an out-of-distribution calibration test are needed.","section":"§5.1 (calibration training; over-refusal evaluation)"},{"comment":"The PAC ablation is reported only under Adaptive Attack and only for three models. No PAC-only ASR results are given for GCG, PAIR, HumanJailbreak, or GPTFUZZER, and no statistic is reported for the fraction of harmful or pseudo-harmful queries that actually trigger the calibration (i.e., the fraction with P_h > P0). Consequently, the reader cannot determine whether the safety improvements in Tables 1-3 are driven by the adversarial training or by PAC, nor whether PAC's effect on the Mistral-7B ASR (a 15.7-point increase under Adaptive Attack) generalizes to other attacks. At minimum, the authors should provide a full attack-wise ablation with and without PAC, plus trigger-rate statistics on both harmful and benign queries.","section":"Table 4; §5.4"}],"minor_comments":[{"comment":"The phrase 'three commonly used three evaluation datasets' contains a duplicated word and should read 'three commonly used evaluation datasets.'","section":"§5.1"},{"comment":"The sentence beginning 'Expression-level strategies' is duplicated immediately after the token-level description; one occurrence should be removed.","section":"§2"},{"comment":"The entry 'LA TPC (Ours)' contains an erroneous space; it should be 'LATPC (Ours).'","section":"Tables 2 and A.3"},{"comment":"The column header 'Adavbench' is misspelled; it should be 'AdvBench.'","section":"Table A.2"},{"comment":"The Mistral-7B sentence 'from 94.3% to 26.4% under the strongest attack (i.e., Adaptive Attack)' is repeated almost verbatim in the same paragraph and should be stated once.","section":"§5.3"},{"comment":"The indicator script in Eq. (12) is visually similar to the probability symbol P; using a distinct notation such as 1[condition] would improve readability.","section":"Eq. (12)"}],"recommendation":"major_revision","confidential_remarks":"The paper's central claim is interesting, but the current evidence conflates two interventions. I would like the authors to provide the PAC-only and adversarial-training-only ablations on all attacks, and to confirm the OR-Bench split. If the PAC safety effect is real, the paper's framing needs revision; if it is not, the ASR tables may overstate the training component. Also, the authors should double-check the variance direction inconsistency before asking reviewers to assess the mechanism."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the short version: the paper is a worthwhile extension of latent adversarial training, but the ablation table contradicts the mechanism it describes. PAC is supposed to trigger only on pseudo-harmful queries, yet Table 4 shows PAC changing ASR on Adaptive Attack for Llama3-8B (67.9→13.8) and Mistral (10.7→26.4). If the trigger is truly exclusive, those numbers should be unchanged. Either the classifier is firing on harmful queries, or the table is reporting more than PAC. That's a load-bearing problem, not a wording issue.\n\nWhat the paper does well: it has a clear idea—use variance across harmful-harmless differences to find safety-critical directions, then adversarially train on a refusal-feature-removal attack, then calibrate embeddings at inference to reduce over-refusals. The evaluation is broad: five attacks, four 7B models, compared against ReFAT, LAT, Targeted-LAT, and others. Code is public. That counts for something.\n\nThe soft spots are real. The variance direction flip—Sec 4.2 says lowest variance, Sec 5.6 says highest—is a red flag even if it's just a wording slip. The calibrator is trained on OR-Bench pseudo-harmful queries and evaluated on 130 OR-Bench queries; the paper says it avoids overlap with the adversarial training set, but not with the eval set. The Mistral regression under PAC is acknowledged but not analyzed: if PAC is misclassifying harmful queries as pseudo-harmful, you'd expect exactly that. No error bars, single seed, so some of the small ASR gaps between baselines are noise. I'd also want to see the false-positive rate of the PAC classifier on genuinely harmful queries.\n\nWho is this for? People building latent-space defenses for open-weight models. It deserves a serious referee—the method is plausible and the experiments are extensive—but it should not be accepted until the PAC trigger is clarified, the ASR ablation is re-run with a proper held-out set, and the variance description is fixed. If those revisions come back clean, I'd cite it.","headline":"Plausible latent-space defense with a calibration ablation that contradicts its own trigger—send to review, but the central claim needs to be re-established.","tokens_in":25358,"tokens_out":3733,"would_cite":false,"duration_ms":38336,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"LATPC claims that training against removal of carefully selected refusal features in the latent space blocks disguise-based jailbreak attacks, and that a closed-form embedding calibration at inference fixes the resulting over-refusal.","keywords":["large language models","jailbreak attacks","adversarial training","latent space","refusal features","over-refusal","post-aware calibration","LoRA fine-tuning"],"falsifier":"Take a held-out set of paired harmful and harmless queries, compute difference vectors at the chosen layer, and sort dimensions by variance exactly as in Eq. (4). Then measure jailbreak attack success after ablating only the selected top-k dimensions versus ablating a random set of the same size; if random ablation performs as well, the variance criterion is not doing the causal work. A direct check of whether the selected set is the lowest-variance or highest-variance third of dimensions would also settle the paper's internal contradiction.","tokens_in":24312,"feed_emoji":"🛡️","tokens_out":6830,"duration_ms":63898,"temperature":0.7,"pith_summary":"This paper proposes LATPC, a two-stage defense against jailbreak attacks that disguise harmful queries as harmless ones. The first stage identifies the latent-space dimensions most tied to refusal by contrasting harmful and harmless inputs, then adversarially trains the model with LoRA against a simulated removal of those specific refusal features. The second stage applies a lightweight embedding-level calibration at inference that nudges borderline pseudo-harmful queries back toward the benign feature space, reducing over-refusal. Experiments across five jailbreak attack families and several 7B models report lower attack success rates than the compared defenses while keeping MMLU, TriviaQA, and over-refusal metrics comparable or better than strong baselines.","feed_headline":"Latent-space defense slashes jailbreak attack success rates","feed_subtitle":"Training against refusal-feature removal plus a calibration step keeps models useful while blocking five attack families.","key_machinery":"The core mechanism is a mask-based refusal feature removal attack: a prototype difference vector at a chosen layer, the average of harmful-minus-harmless hidden states, is multiplied element-wise by a mask that keeps only the selected safety-critical dimensions, and the masked vector is subtracted from harmful queries during training with strength λ. The dimension selection uses variance of per-dimension differences. The second mechanism is post-aware calibration: a layer-wise linear classifier separates pseudo-harmful from harmless embeddings, and when a query is classified as pseudo-harmful above a threshold, the embedding is shifted by the minimal step δ along the classifier weight direction to bring the sigmoid probability below the threshold. Truly harmful inputs, whose probabilities stay above the threshold, are not altered.","core_discovery":"LATPC is built on the observation that jailbreak attacks succeed by shifting harmful queries' latent representations close to benign ones, erasing refusal signals. The central claim is that refusal signals concentrate in a small set of safety-critical latent dimensions, which can be identified by variance analysis over difference vectors between paired harmful and harmless queries. Training the model, via LoRA, to keep refusing even when those dimensions are removed makes it robust to unseen disguise-based attacks; then a post-aware calibration step, a closed-form shift along the weight direction of a trained linear classifier at selected layers, restores utility for queries that merely look harmful. The authors report attack success rates of 0% for HumanJailbreaks and GPTFUZZER on Llama3-8B, strong reductions on other families and models, and marked over-refusal reductions without meaningful MMLU or TriviaQA loss. The method requires white-box access to model internals, a limitation the authors explicitly acknowledge.","pith_inferences":["The variance criterion is described as selecting lowest-variance dimensions in the methodology but as highest-variability dimensions in the later analysis section; a controlled comparison of the two rules would clarify which one actually targets refusal behavior, and if either works the method may owe its success to a broader mechanism than advertised.","Because calibration is a closed-form shift to the decision boundary of a linear classifier, it could in principle be applied to any white-box model whose layers separate pseudo-harmful from harmless embeddings linearly, without retraining per layer, an extension the paper does not test.","The paper's evidence that jailbreak attacks create latent overlap with benign queries implies the defense may generalize to unseen attack styles that produce the same overlap, which could be tested by evaluating against new expression-level attacks not among the five used here.","The acknowledged white-box dependence suggests a testable extension: approximate the calibration signal from output logits alone, which would allow parts of the idea to be adapted to API-level defenses without internal activations."],"forward_implications":["On HarmBench with Llama3-8B, HumanJailbreaks and GPTFUZZER attack success rates drop to 0%, and purely harmful queries reach 0% as well.","Average attack success rate on AdvBench drops from 34.9% for the vanilla model to 15.5% for LATPC, the lowest among the eight compared defenses.","Post-aware calibration reduces over-refusal by 16.1 percentage points on Llama3-8B while leaving MMLU and TriviaQA essentially unchanged.","On Mistral-7B the Adaptive Attack success rate drops from 94.3% to 26.4%, and on Gemma-7B the PAIR success rate drops from 68.6% to 40.3%.","The calibration step adds negligible inference overhead compared with ensemble-based defenses, since it is a single closed-form embedding adjustment."],"supporting_citations":[{"why":"Supplies the paired harmful/harmless dataset and the finding that refusal behavior is mediated by a single latent direction, which LATPC builds on for difference extraction.","marker":"(Arditi et al., 2024)"},{"why":"Provides the variance-based method for selecting safety-critical latent dimensions that LATPC adopts for its identification mask.","marker":"(Li et al., 2025b)"},{"why":"The ReFAT baseline and prior refusal-feature adversarial training that LATPC extends with fine-grained dimension selection and inference-time calibration.","marker":"(Yu et al., 2025)"},{"why":"Supplies the observation of linear separability between harmful and benign embeddings and the concept activation vector that motivates the post-aware calibration.","marker":"(Xu et al., 2024a)"},{"why":"The latent adversarial training framework that provides the base objective for LATPC's training phase.","marker":"(Casper et al., 2024)"},{"why":"Targeted-LAT provides the layer-selection and LoRA configuration that LATPC uses in its adversarial training.","marker":"(Sheshadri et al., 2024)"},{"why":"Circuit Breakers is a main baseline and also the source of the adversarial training dataset used in the paper.","marker":"(Zou et al., 2024)"},{"why":"Adaptive Attack is the strongest evaluated attack, used as a key stress test for the defense.","marker":"(Andriushchenko et al., 2024)"}],"fun_headline_variants":["Latent-space training blocks jailbreaks, cuts over-refusal","Zero jailbreak success on two attacks with latent-space defense","Refusal-dimension adversarial training and calibration fix over-defensiveness","Train on refusal-removal attacks to defend without over-rejecting","LATPC: adversarial training in latent space, calibration for utility"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The defense assumes that the latent dimensions chosen by variance analysis are the ones that genuinely carry refusal behavior, so that removing exactly those dimensions simulates real jailbreak attacks; if the variance criterion instead selects stable but causally irrelevant directions, the adversarial training defends the wrong features and the reported transfer could collapse, and the paper itself describes the selection as lowest-variance in the methodology while its later analysis says the variance-based method picks the highest-variability dimensions.","fun_headline_variants_meta":{"raw":{"variants":["Latent-space training blocks jailbreaks, cuts over-refusal","Zero jailbreak success on two attacks with latent-space defense","Refusal-dimension adversarial training and calibration fix over-defensiveness","Train on refusal-removal attacks to defend without over-rejecting","LATPC: adversarial training in latent space, calibration for utility"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000257,"raw_usage":{"total_tokens":1576,"prompt_tokens":939,"completion_tokens":637,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":555,"completion_tokens_details":{"reasoning_tokens":548}},"tokens_in":555,"tokens_out":637,"duration_ms":6726,"temperature":1.0,"reasoning_tokens":548,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T19:01:29.642170+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a held-out set of paired harmful and harmless queries, compute difference vectors at the chosen layer, and sort dimensions by variance exactly as in Eq. (4). Then measure jailbreak attack success after ablating only the selected top-k dimensions versus ablating a random set of the same size; if random ablation performs as well, the variance criterion is not doing the causal work. A direct check of whether the selected set is the lowest-variance or highest-variance third of dimensions would also settle the paper's internal contradiction.","supporting_citations":[{"cited_title":", author Phan, L","cited_arxiv_id":null,"evidence_quote":"Circuit Breakers is a main baseline and also the source of the adversarial training dataset used in the paper."}],"review_version":1}