{"id":"314c5af2-9483-4308-9639-f94f87ea71a0","arxiv_id":"2507.03236","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Bit-flip jailbreak attacks succeed easily on FP16 language models, are slowed but not stopped by FP8/INT8 quantization, and FP16-induced jailbreaks often survive post-attack quantization to 8-bit formats.","lead":"This paper tests whether jailbreaking language models by flipping bits in their weights still works after models are quantized to lower precision. It finds FP16 models are easily attacked, FP8 and INT8 are more resistant, and jailbreaks made on FP16 often survive re-quantization to 8-bit.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper's measured ASRs rest on an untested physical premise: bit flips are only simulated, yet the threat model (Section 3) requires Rowhammer-class injection to flip exact bits in deployed memory; if ECC or HBM constraints prevent this, the reported attack viability is overstated.","rationale":"In good faith, the paper provides a useful simulation-based measurement: FP8 and INT8 appear more resistant than FP16 under a fixed bit-flip budget, and FP16-induced jailbreaks transfer through 8-bit quantization. These findings are plausible and the relative-resilience claim could hold even if the physical layer is imperfect. However, the paper's title and threat model frame the work as a fault-injection attack, and the only evidence offered is software bit-flip simulation. The weakest condition for the central claim to be valid as an attack paper is that the simulated bit flips can actually be induced on real deployed memory. The paper itself defers this to future work (Section 7), so the condition is unverified. Other issues, such as the abstract's numerical bounds conflicting with Table 2 (e.g., INT8 Llama-3.2-3B ASR 51.6% at 25 flips) and the use of best-of learning rates and best-of-judge criteria, are real but secondary: they affect the accuracy of specific quantitative claims, not the qualitative trend. The physical-realizability concern is the most load-bearing because, if it lands, the central claim as a fault-injection attack is not established; if it does not land, the simulation results remain a valid measurement of bit-flip sensitivity. The reader's CONDITIONAL verdict already reflects this, so I do not recommend moving the verdict. A concrete hardware experiment or memory-system analysis would settle the question and is the appropriate next step.","tokens_in":13928,"tokens_out":12004,"duration_ms":148698,"concrete_test":"Run a physical fault-injection experiment on the target hardware class (e.g., A100/HBM, or commodity DDR4/DDR5 if the model is served from host memory): using a Rowhammer-class technique with user-level privileges, attempt to flip the exact bit positions selected by Algorithm 1 on a deployed quantized Llama-3.2-3B model, and measure the success rate of the intended flips after ECC correction. Alternatively, perform a memory-system analysis (ECC scheme, row-buffer conflicts, address mapping) and compare the reachable bit positions against the top-ranked bits from the simulation; if the reachable set does not include the selected bits, the reported ASRs are not practically achievable.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central quantitative results in Tables 1-3 are produced by software-simulated bit flips in memory-backed tensors (Section 4, Algorithms 1-2), and the threat model (Section 3) explicitly assumes an adversary co-located on the host can use Rowhammer-class fault injection to flip the exact bits selected by the simulation. However, Section 7 states that 'Future work should include physical fault injection,' and no hardware experiment, ECC analysis, or address-mapping evaluation is provided. The paper's own experiments run on A100 GPUs with HBM and ECC; it is not established that a user-level CPU process can flip arbitrary chosen bits in GPU weight tensors. If ECC corrects single-bit errors, or HBM row/address mapping prevents the selected bit positions from being reached, then the ASR values in Tables 1-3 characterize an idealized bit-flip adversary, not the fault-injection attacker named in the title. This concern is structurally distinct from the relative-resilience finding (the simulation ranking may still hold), but it determines whether the paper's central claim about practical attack difficulty is valid. The paper is transparent about the limitation, yet the gap is load-bearing because the entire framing and the abstract's claims ('direct parameter manipulation jailbreaks') depend on physical realizability.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper investigates whether targeted parameter perturbations (bit flips and single-weight updates) can jailbreak aligned large language models under different weight-only quantization schemes. The authors propose a progressive greedy bit-flip search and a word-level gradient-update attack, evaluate them on Llama-3.2-3B, Llama-3-8B, and Phi-4-mini in FP16, FP8, INT8, and INT4 formats, and measure Attack Success Rate on HarmBench with three automated judges. The main empirical findings are that FP16 models are readily jailbroken, FP8 is the most resilient format, INT8 is intermediate, INT4 is substantially less robust, and that jailbreaks induced in FP16 models largely survive subsequent FP8/INT8 quantization but are strongly disrupted by INT4 quantization.","tokens_in":14194,"tokens_out":10056,"duration_ms":110192,"significance":"The paper addresses a timely and practically relevant question: how weight-only quantization affects the cost of parameter-manipulation jailbreaks. Its strengths include evaluation on an external benchmark (HarmBench) with three independent automated judges, sensitivity analyses over the attack optimization dataset (Section 6.4), and a post-attack quantization experiment (Section 6.6). If the results are confirmed, the qualitative conclusion that FP8 deployment raises the number of required bit flips without removing the threat is a useful input for deployment decisions. The quantitative headline numbers in the abstract are, however, not always consistent with the reported tables, and the physical realizability of the assumed fault-injection mechanism is not demonstrated; these issues are detailed below.","major_comments":[{"comment":"The abstract states that within 25 perturbations FP8 and INT8 models exhibit ASRs below 20% and 50% respectively, and that FP8 models maintain ASR below 65% at 150 bit-flips; the paper's own tables contradict these bounds. Table 1 (word-level, iteration 25) shows Phi-4-mini FP8 at 20.8% and Phi-4-mini INT8 at 50.3%, while Table 2 (bit-level, iteration 25) shows Llama-3.2-3B INT8 at 51.6%. Table 2 (iteration 150) shows Phi-4-mini FP8 at 66.7%, and Table 3 also reports 66.7% for Phi-4-mini at iteration 146. These load-bearing quantitative claims must be corrected or replaced with per-model ranges (e.g., 'generally below') before publication. The abstract should also distinguish bit-flip perturbations from weight-update perturbations, since the two attack types have very different costs and the current wording conflates them.","section":"Abstract vs. Tables 1-3"},{"comment":"The threat model requires the adversary to flip exact, simulation-selected bits in deployed memory via Rowhammer-class fault injection, but all experiments are software simulations of bit flips on A100 GPUs with ECC, and Section 7 explicitly defers physical fault injection to future work. No evidence is given that arbitrary bit positions can be reached in HBM/ECC-protected systems. The reported ASR values are therefore upper bounds for an idealized bit-flip adversary, and the title/abstract's 'fault injection attacks' framing overstates what is demonstrated. The authors should either add a hardware-feasibility analysis or explicitly and prominently reframe the results as simulation-based, with a discussion of the conditions under which they would transfer to physical fault injection.","section":"Section 3 vs. Section 7"}],"minor_comments":[{"comment":"The model name 'Phi-4-min-' in the table header should be 'Phi-4-mini'.","section":"Table 3"},{"comment":"The symbol n is used both for the dataset size in the outer sum and for the length of the query in the definition of s_{k-1}; please use separate symbols for these two quantities.","section":"Section 4.1, Eq. (2)"},{"comment":"The text refers to 'AdvBench' in Section 5.2 and to 'AdvBench-Completions' in Section 6.4.1; clarify the relationship between these datasets.","section":"Sections 5.2 and 6.4.1"},{"comment":"The main ASR tables appear to report a single run per configuration; given the sample-selection sensitivity documented in Section 6.4, reporting mean/range across the four sample sets for at least the headline configurations would strengthen the quantitative claims.","section":"Tables 1-3"},{"comment":"The threshold-and-peak selection rule used to produce Table 3 means that the reported iteration numbers are not directly comparable across configurations when no judge reaches the 70% threshold; this caveat should be stated more prominently.","section":"Section 6.3"},{"comment":"The perturbation-location analysis is based on the single highest-ASR run for each configuration; a sentence should note that these distributions are illustrative rather than averaged over multiple runs.","section":"Section 6.5"},{"comment":"The sentence 'The primary objective this study was model destruction' appears to be missing a word ('of'); please correct the typo.","section":"Section 2"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for a security venue and the qualitative trend (FP8 is more resilient than INT8/FP16, with transferability through post-attack quantization) is plausible and useful. The abstract's numerical inconsistencies are easy to fix but currently undermine the headline claims. The lack of physical fault-injection experiments is a common limitation in this literature, but because the title and abstract frame the work as fault-injection attacks, the authors should be asked to temper the framing or add a hardware-feasibility discussion. The novelty over PrisonBreak is incremental, but the systematic comparison across quantization formats and the transferability experiment are valuable contributions."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague — quick take. This is the first systematic measurement of bit-flip jailbreak attacks across FP16/FP8/INT8/INT4 on current aligned LMs, and the transfer result is the most useful part: jailbreaks carved into FP16 weights survive re-quantization to 8-bit formats almost intact, while INT4 wipes out much of the effect. That is a concrete finding for anyone deploying quantized models.\n\nThe paper is honest and does real work. It uses HarmBench plus three independent judges, runs a sensitivity analysis on the attack dataset (sample selection and size), and compares a progressive bit-level search against a word-level update. The tables largely support the qualitative claim that FP8 is the most resistant, INT8 intermediate, and INT4 surprisingly weak.\n\nNow the soft spots, in order of actual softness.\n\nFirst, the abstract does not match the paper's own tables. It claims FP8 stays below 65% ASR at 150 flips, but Table 2 shows Phi-4-mini FP8 at 66.7%. It claims INT8 stays below 50% at 25 flips, but Table 2 shows Llama-3.2-3B INT8 at 51.6%. These are not rounding issues; the headline numbers need correcting.\n\nSecond, the reported ASRs are best-of selections: max over three learning rates for the word-level attack, and the first iteration where any of three judges crosses 70% for the cross-judge comparison. That inflates attack numbers and lowers the confidence you'd want for a measurement paper. A representative or average presentation would be more honest.\n\nThird, all bit flips are simulated. The threat model talks about Rowhammer-class injection but the experiments never leave software, and the conclusion explicitly punts physical fault injection to future work. That is consistent with the BFA literature, but the title says 'through fault injection attacks,' which overstates what was demonstrated. The paper should either bound the claim to simulated bit flips or add a hardware feasibility analysis, including ECC and HBM constraints.\n\nFourth, no code or data is released; for a paper whose main contribution is a measurement, that hurts reproducibility.\n\nNone of this breaks the central qualitative trend, and I think the transfer result will hold up. But the internal numeric contradictions are exactly the kind of thing that needs a real revision before the paper is dependable. I'd send it to a serious venue rather than desk-reject, with the numbers fixed and the threat model scoped properly. It's a solid candidate, not a finished one.","headline":"Useful first measurement of how weight-only quantization changes bit-flip jailbreak difficulty, with a valuable transfer result — but the reported numbers don't all match the abstract and the threat model stays entirely simulated.","tokens_in":14770,"tokens_out":2904,"would_cite":true,"duration_ms":33859,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FP8 and INT8 weight storage materially raises the bit-flip budget required to jailbreak aligned models, yet FP16-created jailbreaks carry over into 8-bit deployments with nearly no loss.","keywords":["jailbreak","bit-flip attack","fault injection","quantization","FP8","weight-only quantization","large language model safety","attack success rate"],"falsifier":"Run the same progressive bit-search attack on a hardware fault-injection testbed for one evaluated model and check whether the specific bit indices chosen by the simulator can actually be flipped at their assumed memory addresses; if the physically achievable flips land on different weights or get corrected by ECC, the simulated ASRs overstate the practical attack surface.","tokens_in":13734,"feed_emoji":"🔓","tokens_out":7539,"duration_ms":80139,"temperature":0.7,"pith_summary":"The paper is trying to establish that the precision format in which an aligned language model is stored changes how hard it is to jailbreak the model by flipping individual bits of its weights. Its experiments on three instruction-tuned models show that FP16 models are readily compromised (attack success above 80% within 25 flips), while under the same budget FP8 models stay below 20% and INT8 below 50%, with FP8 also staying below 65% even at 150 flips. It further claims that this resilience is not a cure: a jailbreak inserted into an FP16 model survives later quantization to FP8 or INT8 with almost no drop in success, so an attacker can compromise at full precision and let the deployment pipeline carry the jailbreak into an 8-bit model. The implied stakes are practical: quantized deployment raises the cost of direct fault-injection attacks on the deployed artifact but does not, by itself, remove the vulnerability.","feed_headline":"FP8 resists bit-flip jailbreaks until 150 flips","feed_subtitle":"Jailbreaks made in FP16 survive re-quantization to FP8 and INT8, so 8-bit deployment is not a clean fix.","key_machinery":"The load-bearing machinery is a progressive, gradient-guided bit-flip search that replaces heuristic bit targets with a direct step-size scan. At each iteration it computes the jailbreak loss and its gradient on dequantized FP16 weights, selects candidate weights by gradient magnitude, and for each candidate flips the bit that produces the largest loss decrease, scanning bits in order of beneficial step size and stopping as soon as the loss stops improving. The objective function is an exponentially windowed cross-entropy over target harmful completions, and gradients through quantized values are handled with a straight-through estimator. A word-level attack that updates a single weight by gradient descent serves as a comparison point for how much of the effect is due to bit-level discreteness.","core_discovery":"On the paper's own terms, the central discovery is that weight-only quantization is a major modifier of bit-flip jailbreak efficacy, not a uniform defense. FP8 (E4M3) is the most resistant format tested: across Llama-3.2-3B, Llama-3-8B, and Phi-4-mini, a progressive bit-level attack that jails FP16 models with 25 flips reaches under 20% ASR on FP8 at that budget and under 65% at 150 flips. INT8 is intermediate, and INT4 is surprisingly weak, with ASR trajectories that track or exceed FP16 in several cases. The paper also reports that attacks on FP16 and INT4 concentrate on attention value projections, while FP8 and INT8 attacks concentrate on MLP down-projections, and that transferring FP16-created jailbreaks into FP8 or INT8 loses less than 5% ASR whereas INT4 loses an average of 35%.","pith_inferences":["If the physical-fault premise holds, a practical implication the paper leaves implicit is that attackers should prefer compromising the model before quantization: the cheap FP16 attack carries into the deployed 8-bit artifact, while attacking the quantized artifact directly is several times more expensive.","The format-dependent layer targets suggest a testable defensive extension: protecting or checksumming attention value projections in FP16/INT4 and MLP down-projections in FP8/INT8 could force attackers to spend more flips, though the paper does not test defenses.","The transfer result also implies that quantization should be verified after the fact as a security control, since the paper's numbers show it does not act as a randomness source that erases prior perturbations in 8-bit formats.","A broader inference is that the FP8/INT8 concentration on down-projection layers resembles the behavior of rounding-based smoothing, which could be studied by ablating per-layer weights to see which projections carry the jailbreak signal."],"forward_implications":["Under a 25-perturbation budget, FP8 models keep attack success below 20% and INT8 below 50%, whereas FP16 models exceed 80%, so direct fault-injection jailbreaking of 8-bit deployments requires a substantially larger budget.","Pushing the budget to 150 flips raises FP8 success to at most 65%, so FP8 should be treated as increasing attack cost, not as eliminating the threat.","INT4 offers little robustness benefit for bit-level attacks and is not a safe alternative to 8-bit formats on its own.","Jailbreaks engineered in FP16 transfer to FP8 and INT8 post-quantization with less than 5% ASR difference, meaning quantizing a compromised model does not sanitize it; INT4 is the exception and cuts transferred ASR by roughly 35% on average.","Attack success varies with the optimization dataset's sample selection and size, so reported ASRs for a fixed 32-sample set are estimates of a range, not fixed properties."],"supporting_citations":[{"why":"Establishes the DRAM disturbance mechanism that motivates physical bit-flip attacks in the threat model.","marker":"[9]"},{"why":"Supplies the progressive gradient-guided bit-search methodology that this paper adapts to jailbreaking.","marker":"[10]"},{"why":"Provides the jailbreak loss function and the FP16 bit-flip jailbreak baseline whose success rates the quantized results are compared against.","marker":"[13]"},{"why":"Supplies the harmful-query dataset used to build the attack optimization set.","marker":"[15]"},{"why":"Supplies the HarmBench test set and judge used to measure attack success rate.","marker":"[18]"},{"why":"Supplies the straight-through estimator used to compute gradients through quantized weights.","marker":"[16]"}],"fun_headline_variants":["FP8 resists bit-flip jailbreaks until 150 flips, INT4 crumbles","Quantization reshapes jailbreak defense: FP8 solid, INT4 vulnerable","Bit-flip jailbreaks on LMs: FP8 stands, INT4 fails","FP16 jailbreaks transfer to FP8/INT8 but not INT4","FP8 hardens LMs against fault injection; INT4 exposes flaw"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The attack's practical reach rests on the assumption that a real adversary can physically flip exactly the bits the software simulator selects; the paper only simulates flips in software and defers physical injection, so memory hardware like ECC, row constraints, or address mapping could make the demonstrated success rates harder to achieve in practice.","fun_headline_variants_meta":{"raw":{"variants":["FP8 resists bit-flip jailbreaks until 150 flips, INT4 crumbles","Quantization reshapes jailbreak defense: FP8 solid, INT4 vulnerable","Bit-flip jailbreaks on LMs: FP8 stands, INT4 fails","FP16 jailbreaks transfer to FP8/INT8 but not INT4","FP8 hardens LMs against fault injection; INT4 exposes flaw"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000399,"raw_usage":{"total_tokens":2152,"prompt_tokens":1079,"completion_tokens":1073,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":695,"completion_tokens_details":{"reasoning_tokens":968}},"tokens_in":695,"tokens_out":1073,"duration_ms":11377,"temperature":1.0,"reasoning_tokens":968,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T20:15:25.571542+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same progressive bit-search attack on a hardware fault-injection testbed for one evaluated model and check whether the specific bit indices chosen by the simulator can actually be flipped at their assumed memory addresses; if the physically achievable flips land on different weights or get corrected by ECC, the simulated ASRs overstate the practical attack surface.","supporting_citations":[{"cited_title":"Flipping bits in memory without accessing them: An experimental study of dram disturbance errors.ACM SIGARCH Computer Architecture News, 42(3):361–372, 2014","cited_arxiv_id":null,"evidence_quote":"Establishes the DRAM disturbance mechanism that motivates physical bit-flip attacks in the threat model."},{"cited_title":"Bit-flip attack: Crushing neural network with progressive bit search","cited_arxiv_id":null,"evidence_quote":"Supplies the progressive gradient-guided bit-search methodology that this paper adapts to jailbreaking."}],"review_version":1}