{"id":"391bfc94-3c50-4ab3-85b9-13c2ca31daf4","arxiv_id":"2505.24445","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A polytope learned in LLM representation space can detect unsafe regions and steer outputs back to safety at inference time, reducing jailbreak success across several models.","lead":"The paper learns a Safety Polytope in an LLM's internal representation space, with each facet acting as a safety constraint, then steers activations at inference time to avoid unsafe outputs. It matters because it offers a post-hoc, interpretable safety layer that preserves model capabilities while defending against jailbreak attacks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The polytope is trained from sentence-level labels applied to every token (Appendix C.1) but used as a per-token gate; if token-level safety is not separable by a convex set, the steering results may rest on this unvalidated label transfer.","rationale":"The paper's central claim has real empirical support: on Held-Out Adaptive Attack, Llama2-7B ASR drops from 60% to 2% and Ministral-8B from 100% to 14%, and the MLP baseline with higher classification accuracy gives substantially worse defense, which suggests the geometric structure matters. However, the training data construction is the least secure part of the argument. Sentence-level labels are assigned to every token, while the deployed mechanism checks and steers each token independently. For the method to work as claimed, token-level hidden states must be separable into safe and unsafe regions by a convex polytope in the 16,384-dimensional concept-encoder space, and the per-token labels must be accurate enough for the CPM loss to learn such a boundary. The paper does not test this directly: it reports aggregate ASR and MMLU, but does not analyze which tokens are steered, whether benign prefixes of harmful responses are modified, or whether safe prompts occasionally trigger steering. The Qwen2 Adaptive Attack failure (100% ASR after steering) and the acknowledged incoherence on Ministral-8B are consistent with the polytope encoding response-level artifacts rather than robust token-level safety. This is an addressable empirical gap, not a fatal flaw, so the reader's CONDITIONAL verdict remains appropriate. The concrete test isolates the label-transfer assumption from the rest of the pipeline; if last-token-only training works equally well, the assumption could be dropped or corrected without changing the conclusions.","tokens_in":29229,"tokens_out":8869,"duration_ms":115010,"concrete_test":"Run the same HarmBench training and evaluation pipeline but train the polytope on the last-token representation of each response only (the token at <eos>), instead of all tokens, and compare ASR on Adaptive Attack and the nine HarmBench attacks plus MMLU against the paper's all-token variant. If the last-token-only variant preserves the ASR reduction, the all-token labeling assumption is not load-bearing; if ASR degrades, the reported defense depends on a label transfer that has no per-token ground truth.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing assumption is not just that safety is linearly separable; it is that sentence-level labels transfer to every token in a way that makes a convex polytope in EC(h) space a correct per-token gate. Appendix C.1 states: 'If an output sentence is marked as unsafe, we label all token representations in the sentence as unsafe, and vice versa.' Algorithm 1 then applies the polytope constraint to each generated token before decoding. This creates a training/label mismatch: early tokens of a harmful response (e.g., 'The', 'following', 'essay') are labeled unsafe even though their hidden states may be indistinguishable from safe tokens, while benign sentences containing harmful words are labeled safe. Because the concept encoder and CPM are trained end-to-end on these labels, the facet boundaries can absorb the noise; but nothing in the paper checks that the learned safe set corresponds to token-level safety rather than to response-level artifacts (e.g., the style of attacked prompts) or that it is convex in the relevant region. The failure pattern is consistent with this concern: Qwen2-1.5B, the model with the simplest/lowest-dimensional representations, shows no reduction on the held-out Adaptive Attack (100% ASR in Table 6), and the paper itself notes Ministral-8B often produces semantically incoherent outputs after steering.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SaP (Safety Polytope), a post-hoc geometric safety mechanism for LLMs. A concept encoder maps layer-20 activations into a sparse feature space, a Convex Polytope Machine learns a polytope of safe features from safe/unsafe labels, and Algorithm 1 steers each generated token's activation back into the polytope before decoding. Experiments on Llama2-7B, Ministral-8B, and Qwen2-1.5B report large HarmBench attack-success-rate reductions (e.g., Llama2-7B from 12.92% to 0.26%) with roughly unchanged MMLU, plus interpretability analyses of the learned facets on BeaverTails and a PAC-style sample-complexity appendix.","tokens_in":29541,"tokens_out":7988,"duration_ms":103388,"significance":"If the results hold, SaP is a valuable inference-time defense: it preserves model weights, gives interpretable safety facets, and is model-agnostic. Strengths of the manuscript include public code, five-seed evaluation for ASR, detailed per-attack tables, a genuinely held-out Adaptive Attack evaluation for two of the three models, and an explicit attempt to address polysemanticity with a concept encoder. However, the main generalization claim is weakened by the fact that the reported average ASR includes attack families used in training, and the single fully held-out attack family still defeats Qwen2-1.5B completely (100% ASR), which the main text does not flag. The unvalidated transfer of sentence-level labels to per-token training data is a further correctness risk for the core steering mechanism.","major_comments":[{"comment":"The headline average ASR is partly in-distribution. The polytope is trained on 80% of the attack strings from the top-3 most effective attack methods for each model, and the reported average in Figure 2 is then computed over all nine attack algorithms, which include those same trained attack families. Only Adaptive Attack is fully held out. For example, Llama2-7B's per-attack table (Table 2) shows AdaptiveAttack at 2.00%, while the pooled average is 0.26%. The paper should report separate in-distribution and held-out averages, or exclude the training families from the headline number, so that the claimed generalization to unseen attack methods is actually measured.","section":"§4.1, Tables 2–6, Figure 2"},{"comment":"Qwen2-1.5B remains at 100% ASR under Adaptive Attack even after SaP steering, identical to the original model, yet the main text reports only the favorable pooled average (27.57% to 11.81%) and §4.2 states that Qwen2 shows strong baseline robustness without the concept encoder. This is a load-bearing omission: one of the three evaluated models receives no adaptive-attack defense. The authors should report this failure explicitly, temper the generalization claim accordingly, and provide an explanation or further analysis.","section":"§4.1, Table 6"},{"comment":"The training data are labeled at sentence level and transferred to every token: Appendix C.1 states that if an output sentence is marked unsafe, all token representations in the sentence are labeled unsafe, and vice versa. Algorithm 1 then applies the polytope constraint to each generated token before decoding. This label-transfer assumption is unvalidated: early tokens of a harmful response may be labeled unsafe even when their hidden states are indistinguishable from safe tokens, and benign sentences containing harmful words are labeled safe. Since the concept encoder and polytope are trained end-to-end on these labels, the learned safe set may reflect response-level artifacts rather than token-level safety. The paper should validate token-level label correctness or ablate against a last-token-only or sentence-level steering baseline.","section":"Appendix C.1, Algorithm 1"},{"comment":"The capability-preservation claim is supported only by single-point MMLU numbers in Table 8, with no standard deviations or number of seeds, despite the figure caption claiming that SaP consistently retains original MMLU accuracy. The same is true for MT-Bench in Table 9, which also shows a non-negligible drop for Qwen2-1.5B (6.03 to 5.49). The authors should report means and standard deviations over the same five seeds used for ASR, and discuss the MT-Bench decrease.","section":"§4.1, Table 8, Table 9"},{"comment":"The Lagrangian steering loss in Appendix B appears to have a sign inconsistency. With [x]_- = min(0, x), the term λ_safe Σ [ϕ^T E_C(h) − ξ]_- is negative for already-satisfied constraints, so minimizing it moves the activation further into the safe side rather than leaving it unchanged. This contradicts the claim in §3 that original generation remains unchanged when the representation is already inside the polytope. If the intended penalty is max(0, −(ϕ^T E_C(h) − ξ)) or a squared penalty, the definition should be corrected; otherwise the method's behavior on safe tokens, especially with Qwen's λ_safe = 5000, is not what the text describes.","section":"Appendix B, Eq. (B.1)"}],"minor_comments":[{"comment":"The text mentions In-Context Learning (ICL) as a baseline in the results discussion, but ICL is not introduced in the baseline list in §4.1; Table 8 includes ICL results. Please clarify where ICL is described and how it is configured.","section":"§4.1"},{"comment":"The number of attack algorithms used to form the average is inconsistent: §4.1 and the Figure 2 caption say nine attacks, while Appendix C.2 says Figure 2 aggregates over seven attack algorithms. Please unify the count and state explicitly which attacks enter each reported average.","section":"Figure 2, Appendix C.2"},{"comment":"Algorithm 1's comment says the original token generation remains 'unchained'; this should read 'unchanged'.","section":"Algorithm 1"},{"comment":"In the CMDP definition, the text writes 'find a stationary policy π(x | a)'; the policy should be written π(a | x) to match the autoregressive LLM policy described immediately afterward.","section":"§2"},{"comment":"The Limitations section admits that Ministral-8B can induce semantically incoherent outputs and that the theoretical guarantees depend on strong, unverified assumptions. These caveats are important and should be connected more directly to the main-text claims, ideally with quantitative evidence about how often incoherent outputs occur.","section":"§6"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the scope of the venue and the code release and per-attack tables are assets. The main issues are evaluative framing rather than fundamental novelty: the pooled ASR includes training attack families, the Qwen Adaptive Attack failure is not flagged in the main text, and the sentence-to-token label transfer is unvalidated. These are fixable within a revision, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: this is a real contribution — a post-hoc, model-agnostic safety layer built by learning a convex polytope in the model's activation space and steering outputs back inside it. The Llama2-7B result (ASR 12.92% to 0.26% on the in-distribution average) and the Ministral-8B result are strong, the interpretability analysis (facet specialization, KL divergence) is genuinely interesting, and the code is public. The authors also honestly list limitations: they note Ministral can produce incoherent outputs and that the theory is not connected to the algorithm.\n\nThe main soft spots are real but fixable. First, the evaluation is partly in-distribution: the polytope is trained on the top-3 attack families and the headline average includes those families. The held-out Adaptive Attack is the cleaner test, and there the method works on Llama2 (60% to 2%) and Ministral (100% to 14%) but fails completely on Qwen2-1.5B (100% to 100%). That failure is buried in Table 6, not flagged in the main text — that's an honest reporting gap. Second, the stress-test concern lands: Appendix C.1 labels every token in an unsafe sentence as unsafe, but Algorithm 1 uses the polytope as a per-token gate. Early tokens of a harmful response are labeled unsafe, while benign sentences containing harmful words are labeled safe. The learned facets may be capturing response-level artifacts (e.g., the style of attacked prompts) rather than token-level safety. Nothing in the paper checks that the safe set corresponds to token-level safety or that convexity holds in the relevant region. The Qwen2 failure pattern is consistent with this concern, though not conclusive. Third, MMLU numbers lack error bars; the differences are small, but given the 5-seed protocol elsewhere, this is an easy fix.\n\nThe theory appendix (E) is essentially a restatement of existing sample-complexity bounds and doesn't justify the CMDP-to-polytope leap. The paper's own limitation section admits this.\n\nVerdict: the central idea holds up conditionally. The method deserves serious refereeing. A strong revision would validate token-level labels (e.g., by comparing per-token annotations or analyzing which positions trigger steering), report held-out attacks in the main text, and add error bars to MMLU. I'd bring it to a reading group and I'd cite it.","headline":"A solid, genuinely new inference-time safety method whose headline results are real but whose token-level label-transfer assumption needs scrutiny before the numbers are taken at face value.","tokens_in":30049,"tokens_out":2236,"would_cite":true,"duration_ms":24351,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SaP learns a polytope of safety constraints inside an LLM's representation space and steers generated tokens back into it, cutting adversarial attack success on Llama2-7B from 12.92% to 0.26% while leaving MMLU accuracy essentially…","keywords":["large language models","safety constraints","adversarial attacks","representation space","polytope learning","concept encoder","inference-time steering","interpretability"],"falsifier":"Collect human-rated harmful outputs that SaP labels safe and compute, for each, whether every facet inequality $\\phi^\\top E_C(h) \\le \\tilde\\xi$ is satisfied at the layer-20 activation. If any harmful output lies entirely inside the learned polytope, the safe set is not contained in the polytope and the geometric safety model is refuted.","tokens_in":29006,"feed_emoji":"🛡️","tokens_out":11939,"duration_ms":123011,"temperature":0.7,"pith_summary":"The paper argues that safety in large language models can be represented explicitly as geometry: a polytope, formed by intersecting linear halfspace constraints, inside the model's own representation space. It proposes SaP, which learns the polytope's facets from binary safe/unsafe labels using a Convex Polytope Machine, then enforces the constraints at generation time by steering layer-20 activations back inside the boundary. In experiments, this lowers the average adversarial attack success rate on Llama2-7B from 12.92% to 0.26% while MMLU accuracy stays nearly flat (45.8% vs. 45.7%). The approach matters because it offers post-hoc, weight-free safety control and an explicit, inspectable account of which semantic concepts each safety facet detects.","feed_headline":"Polytope steering cuts Llama2 jailbreak success from 12.92% to 0.26%","feed_subtitle":"Learning safety as a geometric region in hidden states blocks attacks while keeping MMLU accuracy flat.","key_machinery":"The Safety Polytope, $\\tilde{Q} = \\{ \\tilde f \\in \\mathbb{R}^d \\mid \\phi^\\top \\tilde f \\le \\tilde\\xi \\}$, is a set of $K$ linear halfspace facets learned from binary labels with the Convex Polytope Machine loss. The facets operate on features produced by a ReLU-activated 'concept encoder' that projects activations into a sparse 16,384-dimensional space, and they do triple duty: they classify each token as inside or outside the safe set, they define interpretable semantic concepts (e.g., a facet for kidnapping, a facet for sexual content), and they serve as the constraint in the steering step, which solves $\\min_h \\|\\bar\\pi_l(x) - h\\|_1$ subject to $\\phi^\\top E_C(h) \\le \\tilde\\xi$ during decoding.","core_discovery":"The central claim is that safety constraints are learnable geometric objects in LLM representation space. Given only binary safe/unsafe sentence labels, SaP learns $K$ linear facets with thresholds defining the polytope $\\tilde{Q} = \\{ \\tilde f \\mid \\phi^\\top \\tilde f \\le \\tilde\\xi \\}$ on sparse features extracted from layer-20 activations by a concept encoder. At inference time, SafeFlow checks each token's encoded activation against the polytope and, when the activation lies outside, solves a small constrained optimization to move it back inside before the next token is decoded. The paper reports attack success rate reductions from 12.92% to 0.26% for Llama2-7B, from 55.77% to 3.25% for Ministral-8B, and from 27.57% to 11.81% for Qwen2-1.5B, with MMLU accuracy essentially unchanged, and shows that individual facets specialize to notions such as kidnapping, sexual content, and bullying.","pith_inferences":["Beyond this paper, the same polytope construction could be retrained on other labels—toxicity, privacy, or factual reliability—since the geometric method itself is agnostic to the meaning of the safe/unsafe split.","The AdaptiveAttack results differ sharply across models (near-perfect defense for Llama2, 14% ASR for Ministral, 100% for Qwen2), which a reader could take as evidence that the load-bearing variable is how linearly separable safety happens to be in each model's hidden states.","A natural next experiment the authors did not run is transferring facets between models of different sizes or training runs, which would test whether the learned geometric safety structure is universal or model-specific.","The theory appendix bounds sample complexity for polytopes that are perfectly separable with margin, so extending SaP to non-separable or noisy label settings would require a different theoretical treatment."],"forward_implications":["Safety enforcement can be applied post-hoc to an already deployed model by editing layer activations during decoding, without retraining or fine-tuning weights.","Facets trained only on binary safe/unsafe labels align with human semantic categories, so a flagged generation carries an explanation of which safety concept it violated.","The linear structure of the facets scales to gradient-based optimization over prompts, leading to near-zero attack success rates against GCG, AutoPrompt, and other gradient-based jailbreaks while preserving MMLU performance.","Defense quality improves with the number of facets up to a point (near-perfect Llama2 defense at 20 facets, diminishing returns beyond roughly 30–40), so polytope capacity should be tuned per model.","The same machinery generalizes across three model families with different hidden-state dimensionalities, suggesting the geometric approach is architecture-agnostic."],"supporting_citations":[{"why":"Establishes that safety constraints in constrained MDPs can be learned from demonstrations as linear constraints, which is the conceptual basis for the polytope formulation.","marker":"Lindner et al. (2024)"},{"why":"Supplies the Convex Polytope Machine loss used to learn polytope facets at high dimensionality.","marker":"Kantchelian et al. (2014)"},{"why":"Provides the linear representation hypothesis that high-level concepts appear as linear directions in LLM feature space, grounding the geometric model.","marker":"Park et al. (2023)"},{"why":"Provides the HarmBench benchmark and the nine attack methods against which SaP's defense is measured.","marker":"Mazeika et al. (2024)"},{"why":"Provides the BeaverTails dataset of 330k annotated sentences in 14 safety categories used to evaluate facet interpretability.","marker":"Ji et al. (2023)"},{"why":"Motivates the constrained-optimization method for keeping an agent's actions inside a safe set during execution, adapted here to activation steering.","marker":"Dalal et al. (2018)"},{"why":"Provides GCG, one of the strongest gradient-based jailbreak attacks used for training and evaluation of the polytope.","marker":"Zou et al. (2023)"},{"why":"Provides the held-out Adaptive Attack on JailBreakBench used to test whether the defense survives a white-box adversary.","marker":"Andriushchenko et al. (2024)"},{"why":"Gives the fat-shattering dimension of polytopes with margin, from which the paper derives its sample-complexity bound in the appendix.","marker":"Gottlieb et al. (2021)"},{"why":"Provides the MMLU benchmark used to verify that the steering preserves standard task accuracy.","marker":"Hendrycks et al. (2020)"}],"fun_headline_variants":["Polytope safety cuts Llama2 jailbreaks from 12.92% to 0.26%","Safety polytope blocks jailbreaks, preserves accuracy","Learn safety as a polytope in LLM hidden space","Safety polytope cuts jailbreak success across models","Post-hoc polytope steering blocks attacks and preserves performance"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that safety is linearly separable in the layer-20 representation space after the concept encoder, so that a polytope of halfspace constraints can separate safe from unsafe behavior; if safety is not separable there, the learned facets will misclassify and steering will push activations in the wrong direction.","fun_headline_variants_meta":{"raw":{"variants":["Polytope safety cuts Llama2 jailbreaks from 12.92% to 0.26%","Safety polytope blocks jailbreaks, preserves accuracy","Learn safety as a polytope in LLM hidden space","Safety polytope cuts jailbreak success across models","Post-hoc polytope steering blocks attacks and preserves performance"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001793,"raw_usage":{"total_tokens":7054,"prompt_tokens":925,"completion_tokens":6129,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":541,"completion_tokens_details":{"reasoning_tokens":6038}},"tokens_in":541,"tokens_out":6129,"duration_ms":48945,"temperature":1.0,"reasoning_tokens":6038,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T12:21:50.788690+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Collect human-rated harmful outputs that SaP labels safe and compute, for each, whether every facet inequality $\\phi^\\top E_C(h) \\le \\tilde\\xi$ is satisfied at the layer-20 activation. If any harmful output lies entirely inside the learned polytope, the safe set is not contained in the polytope and the geometric safety model is refuted.","supporting_citations":[],"review_version":1}