{"id":"5ea3850f-e4a7-46e6-ab3b-188dec3a4010","arxiv_id":"2412.18171","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Token Highlighter identifies jailbreak-critical tokens using gradients of an affirmation loss and weakens them by shrinking their embeddings, lowering attack success on LLaMA-2 and Vicuna.","lead":"This paper introduces a defense called Token Highlighter that finds the words in a prompt pushing a chatbot toward a harmful answer and shrinks their influence before the chatbot responds. It is a low-cost, interpretable way to block jailbreak attacks on aligned language models.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline ASR/Win-Rate figures lack error bars and are based on in-sample β selection; a textual/table inconsistency in the Gradient Cuff comparison further undermines the claimed superiority, so the central empirical claim is not yet robustly established.","rationale":"The reader's weakest_assumption is about the affirmation-loss mechanism, but the paper already provides some support for that mechanism: Random Soft Removal is substantially worse (ASR 0.550 vs 0.142 on Vicuna), and varying the affirmation sentence yields stable results (Table A2). Thus the mechanism, while theoretically limited, is empirically grounded in these experiments. The more load-bearing gap is the reliability of the headline numbers: no error bars, a single seed, and β chosen on the very evaluation set used to claim success. This is compounded by the internal inconsistency in the Gradient Cuff comparison, which is exactly the kind of issue that must be resolved before the 'state-of-the-art' claim can be taken at face value. These problems are fixable with a proper held-out validation protocol and repeated runs, so a conditional accept remains appropriate rather than outright rejection. If the proposed test shows the effect survives on a held-out set, the central claim would be substantially strengthened.","tokens_in":15810,"tokens_out":6735,"duration_ms":59702,"concrete_test":"Hold out 50 of the 100 AdvBench behaviors as a test set and choose β on the remaining 50 (per model). Run the full evaluation with 5 random seeds on the held-out set, reporting mean and 95% CI for ASR and Win Rate. Also recompute the Gradient Cuff comparison using Table A5's value (0.408) and recheck the claimed 20.7% improvement. If the held-out Vicuna ASR exceeds 0.20 or the CI contains 0.243, the headline effectiveness claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that Token Highlighter reduces ASR to 0.142 on Vicuna-7B while keeping Win Rate at 0.698—rests on a single evaluation run with seed 100 and hyperparameters β=0.3 (Vicuna) and β=0.5 (LLaMA-2) chosen after inspecting the same 100 malicious queries and AlpacaEval set (Section 4.1). With 100 examples, the binomial standard error of ASR 0.142 is ~3.5 percentage points; without repeated seeds or confidence intervals, the difference from several baselines (e.g., PPL ASR 0.585, Gradient Cuff ASR 0.408 per Table A5) may not be significant. The paper's own comparison is inconsistent: Section 4.2 states Gradient Cuff 'can only decrease the ASR to 0.243' and computes TH's advantage as '0.588 vs 0.487', both derived from 0.243, yet Table A5 reports Gradient Cuff ASR 0.408 on Vicuna (reduction 0.322). Either the text or the table is wrong, so the claimed 20.7% outperformance over the best baseline is not verifiable. The utility claim (Win Rate 0.698 vs undefended 0.772) is likewise a single-point estimate. These issues directly affect whether the method actually maintains 'competent performance' and surpasses existing defenses.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Token Highlighter, a jailbreak defense for LLMs. The method computes an Affirmation Loss with respect to a fixed affirmative sentence, ranks tokens by the gradient norm of that loss, selects the top alpha fraction of tokens as a Critical Set, and then shrinks the embeddings of those tokens by a factor beta (Soft Removal) before generation. The authors evaluate on Vicuna-7B-V1.5 and LLaMA-2-7B-Chat against six jailbreak attacks (GCG, AutoDAN, PAIR, TAP, Manyshot, AIM) and report AlpacaEval Win Rate as the utility metric. They report that Token Highlighter reduces average ASR from 0.730 to 0.142 on Vicuna while keeping Win Rate at 0.698, and from 0.115 to 0.018 on LLaMA-2, with only one extra forward-backward pass for token selection.","tokens_in":16088,"tokens_out":4600,"duration_ms":40689,"significance":"If the reported results are robust, Token Highlighter is a useful contribution: it is lightweight, interpretable, and shows a large ASR reduction compared with no defense. The paper includes several commendable control experiments: Random Soft Removal, adaptive attacks, running-time comparison, and a sensitivity check over different affirmation sentences in Appendix A.2. However, the empirical claims are weakened by load-bearing issues: an internal inconsistency in the main comparison with Gradient Cuff, the absence of error bars or multiple seeds, and in-sample selection of the beta hyperparameter on the same evaluation sets. These issues prevent the paper from currently supporting the stated state-of-the-art claim.","major_comments":[{"comment":"The comparison with Gradient Cuff is internally inconsistent. The text states that Gradient Cuff 'can only decrease the ASR to 0.243' and then computes Token Highlighter's advantage as '0.588 vs 0.487', but Table A5 reports Gradient Cuff's average ASR on Vicuna-7B-V1.5 as 0.408, giving an ASR reduction of 0.730 - 0.408 = 0.322, not 0.487. The numbers 0.243 and 0.487 do not appear anywhere in the complete results table. Because this comparison is the basis for the claim that Token Highlighter outperforms the best baseline, the discrepancy must be resolved and all derived percentages recomputed from the corrected table.","section":"Section 4.2 vs Table A5"},{"comment":"All experiments are run with a single random seed (seed=100) and no error bars are reported. With 100 harmful instructions per attack, the binomial standard error for an ASR of 0.142 is about 3.5 percentage points, so the reported difference between Token Highlighter (0.142) and Semantic Smoothing (0.132) in Table A5 is not statistically meaningful. Furthermore, beta is chosen per model (0.3 for Vicuna, 0.5 for LLaMA-2) after inspecting the same 100 malicious queries and the AlpacaEval set, so the headline ASR and Win Rate numbers are in-sample selected values, not out-of-sample predictions. The paper should report means and confidence intervals over multiple seeds or attack subsamples and use a validation split or nested selection procedure for alpha and beta.","section":"Section 4.1 and Figure 2/Table A5"},{"comment":"The claim that Token Highlighter achieves 'the lowest ASR among all the methods that can keep a high Win Rate' depends on an undefined notion of 'high Win Rate'. Table A5 shows that Semantic Smoothing has a lower average ASR (0.132 vs 0.142) but a Win Rate of only 0.301, while Gradient Cuff has a higher Win Rate (0.738) but a higher ASR (0.408). Without specifying a minimum acceptable Win Rate or performing a Pareto-frontier analysis over the ASR-utility trade-off, the relative superiority over these baselines is not rigorously established.","section":"Section 4.2 and Figure 2"}],"minor_comments":[{"comment":"The notation 'argtop-nα' is not defined, and when αn is not an integer it is unclear how the number of selected tokens is rounded. Please clarify this in the text.","section":"Equation (4)"},{"comment":"The abstract says Token Highlighter 'only needs to query the protected LLM once', but the method actually requires one forward-backward pass for token selection and then a separate forward pass to generate the final response. Section 1's phrasing 'one extra query' is clearer and should be used consistently.","section":"Abstract and Section 3.3"},{"comment":"The sentence 'Random Soft Removal can also keep the utility almost unchanged... exploring the effect on the values of β and α in soft removal may be more crucial than which tokens are softly removed' is inconsistent with Table A5, where Random Soft Removal has average ASR 0.550 versus Token Highlighter's 0.142 on Vicuna. As written, it undercuts the paper's own evidence for the Critical Set construction and should be rephrased.","section":"Section 4.2"},{"comment":"There are minor typographical issues: 'horizon axis' should be 'horizontal axis' in the Figure 2 caption, and Table A3's header 'Tempalte' should be 'Template'.","section":"Figure 2 and Table A3"},{"comment":"The method requires white-box access to the model's embedding layer and gradients and is only evaluated on open-weight models. This is a practical limitation for API-only deployment and should be stated explicitly in the paper.","section":"Scope limitations"}],"recommendation":"major_revision","confidential_remarks":"The inconsistency between Section 4.2 and Table A5 is the most serious issue because it directly affects the headline claim of outperforming the best baseline. The single-seed, in-sample hyperparameter selection also needs to be addressed with proper uncertainty quantification or at least an explicit acknowledgement of the limitation. These are fixable within the manuscript's scope, so major revision rather than rejection seems appropriate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a solid, genuinely new defense idea—use gradients of an affirmation loss to find jailbreak-critical tokens, then shrink those embeddings instead of deleting them. The paper does a lot of things right. The Random Soft Removal ablation is the right experiment to show the attribution matters. They include adaptive attacks, running time, multiple affirmations, and a detailed appendix. If I needed a cheap, interpretable defense baseline, I'd use this.\n\nThe problems are all in the empirical reporting. The big one is an internal inconsistency. Section 4.2 says Gradient Cuff only lowers ASR to 0.243 on Vicuna and computes TH's advantage from that number. Table A5 reports Gradient Cuff ASR as 0.408 on the same model. Both cannot be right. The claimed 20.7% outperformance is not verifiable until this is fixed. That's a load-bearing number, though not for the whole method: TH's ASR (0.142) is still below both candidates.\n\nSecond, there are no error bars, one seed, and beta is chosen per model on the same attack set. With 100 malicious queries, the binomial standard error on ASR 0.142 is about 3.5 points, so the gap to some baselines is within noise. This is fixable but undermines the 'state-of-the-art' phrasing.\n\nThird, on LLaMA-2 every defense gets ASR near zero, so the interesting evidence is only on Vicuna. That's not a flaw, but it means the claim of broad generality is weaker than the abstract suggests.\n\nThe mechanism assumption—that jailbreak-critical tokens are exactly the top gradient norm with respect to one fixed affirmation sentence—is pragmatic and they test three affirmations, which helps. It obviously won't work without gradient access, and that's a scope limit worth stating clearly.\n\nBottom line: the core idea is worth engaging with and the experiments are extensive, but the paper needs a revision that resolves the inconsistency, adds multiple seeds or confidence intervals, and documents the hyperparameter selection honestly. I'd send this to peer review, and I'd give it a conditional accept after those fixes.","headline":"A plausible, cheap jailbreak defense whose headline numbers are undercut by an unresolved text/table inconsistency and single-seed evaluation.","tokens_in":16629,"tokens_out":2582,"would_cite":true,"duration_ms":23568,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Token Highlighter blocks jailbreak prompts by shrinking the tokens that steer a model to comply.","keywords":["jailbreak attacks","large language models","defense","Affirmation Loss","Soft Removal","gradient-based token selection","LLM interpretability","AlpacaEval"],"falsifier":"Build a harmful prompt that does not raise the probability of 'Sure, I'd like to help you with this.' above baseline; if Token Highlighter then lets the attack through after Soft Removal, the central assumption is refuted.","tokens_in":15589,"feed_emoji":"🛡️","tokens_out":7253,"duration_ms":59322,"temperature":0.7,"pith_summary":"Token Highlighter is a proposed defense against jailbreak attacks on aligned large language models. It claims that the tokens in a malicious user query that most strongly push the model toward a fixed affirmative reply — 'Sure, I'd like to help you with this.' — are the tokens that carry the jailbreak. By scaling down the embeddings of these tokens before response generation, the model can refuse the query while still answering benign prompts. Experiments on two aligned LLMs and six attack methods report the average attack success rate drops from 0.730 to 0.142 on Vicuna-7B-V1.5 and to 0.018 on LLaMA-2-7B-Chat, with only one extra model query and interpretable highlighted tokens.","feed_headline":"Token Highlighter cuts jailbreak success from 73% to 14%","feed_subtitle":"A one-query, gradient-based method finds the tokens that push a model to comply and shrinks them, keeping benign answers intact.","key_machinery":"The key mechanism is the pairing of Affirmation Loss with Soft Removal. Affirmation Loss is the negative log-likelihood of a fixed affirmation prefix under the model conditioned on the user query; its per-token gradient norms, called Influence, mark which tokens most steer the model toward affirmative completion. Soft Removal then scales down the embeddings of the top-α tokens by β, diluting their contribution to the hidden states while preserving the query's surface form.","core_discovery":"The paper introduces Affirmation Loss, the negative log-probability of a fixed affirmation sentence 'Sure, I'd like to help you with this.' given the user query, as a measure of the model's willingness to answer. It computes the gradient of this loss with respect to each token embedding, ranks tokens by the norm of these gradients, and selects the top α fraction as 'jailbreak-critical'. It then applies Soft Removal, multiplying the embeddings of those tokens by a factor β between 0 and 1, so their influence on generation is diluted without being deleted. The authors report that this combination beats existing detector-, smoothing-, and prompt-engineering-based defenses on the ASR-versus-utility trade-off.","pith_inferences":["One could test whether the same Affirmation-Loss gradients also expose tokens that are critical for refusal, turning the method into a general token-attribution tool rather than a jailbreak-specific defense.","For models without gradient access, a surrogate model or a black-box estimate of the affirmation probability might approximate the Influence metric, though the paper does not explore this.","Because the selection depends on a fixed affirmation string, attacks that deliberately avoid the 'Sure' preamble or use a different compliance phrasing might evade the step; searching over a set of affirmation templates or learning the affirmation from data would be a natural extension."],"forward_implications":["If the claim holds, any attack that works by steering the model into an affirmative preamble can be neutralized without modifying the surface text of the query.","The defense's cost is one extra forward-backward pass, so it can be deployed at inference time on open models with white-box access.","Because the highlighted tokens align with the attacker's inserted suffix or rewritten role-play instructions, the method doubles as a post-hoc explanation tool for refusals.","Combining Token Highlighter with a prompt-engineering defense like Self Reminder further lowers the attack success rate at a tunable cost to the win rate."],"supporting_citations":[{"why":"Supplies the GCG attack and the technique of using gradients of an affirmation loss for token optimization, which Token Highlighter repurposes for token selection.","marker":"[28]"},{"why":"Establishes that many successful jailbreaks produce an affirmative 'Sure' preamble, motivating the definition of Affirmation Loss.","marker":"[22]"},{"why":"Provides further evidence that safety-trained models are prone to beginning harmful responses with affirmations, supporting the choice of the fixed affirmation sentence.","marker":"[26]"},{"why":"Defines Erase Check, the hard-removal baseline that Soft Removal is designed to improve upon by avoiding utility collapse.","marker":"[11]"},{"why":"Shows that paraphrasing or perturbing the query (semantic smoothing) degrades utility, motivating the embedding-shrinking approach of Soft Removal.","marker":"[9]"}],"fun_headline_variants":["Token Highlighter: Locate and defuse jailbreak-critical tokens","Affirmation Loss gradient reveals jailbreak tokens for soft removal","One-query defense shrinks jailbreak tokens, cuts success to 14%","Gradient-based Token Highlighter blocks jailbreak prompts","Shrinking jailbreak-trigger tokens keeps benign answers intact"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that the tokens with the largest gradient norms of the fixed affirmation sentence 'Sure, I'd like to help you with this.' are exactly the tokens that carry the jailbreak, and that these gradients are available; a jailbreak that succeeds without raising the probability of that specific prefix, or a model served only through an API without gradients, would undermine the token-selection step.","fun_headline_variants_meta":{"raw":{"variants":["Token Highlighter: Locate and defuse jailbreak-critical tokens","Affirmation Loss gradient reveals jailbreak tokens for soft removal","One-query defense shrinks jailbreak tokens, cuts success to 14%","Gradient-based Token Highlighter blocks jailbreak prompts","Shrinking jailbreak-trigger tokens keeps benign answers intact"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00107,"raw_usage":{"total_tokens":4488,"prompt_tokens":960,"completion_tokens":3528,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":576,"completion_tokens_details":{"reasoning_tokens":3447}},"tokens_in":576,"tokens_out":3528,"duration_ms":24650,"temperature":1.0,"reasoning_tokens":3447,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T04:58:07.891749+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Build a harmful prompt that does not raise the probability of 'Sure, I'd like to help you with this.' above baseline; if Token Highlighter then lets the attack through after Soft Removal, the central assumption is refuted.","supporting_citations":[],"review_version":1}