Pith. sign in

REVIEW 2 major objections 5 minor 24 references

TamperBench: Systematically Stress-Testing LLM Safety Under Fine-Tuning and Tampering

T0 review · 2 major / 5 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Every open-weight LLM tested can be tampered into a capable, highly harmful model.

desk verdict A useful consolidation of tamper-resistance evaluation, but the 'utility preserved' numbers rest on a 140-item MMLU-Pro subset and best-of-40 selection; the core vulnerability claim still holds. read the letter →

arxiv 2602.06911 v2 pith:6B2TDGGX submitted 2026-02-06 cs.CR cs.AI

Saad Hossain , Tom Tseng , Punya Syon Pandey , Samanvay Vajpayee , Matthew Kowal , Nayeema Nonta , Samuel Simko , Stephen Casper
show 3 more authors
Zhijing Jin Kellin Pelrine Sirisha Rambhatla
This is my paper
classification cs.CRcs.AI
keywords LLMsafetytamperresistancefine-tuningattacksjailbreak-tuningrepresentationalignment-stagedefenseshyperparametersweepsStrongREJECT
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to measure whether safety alignment in open-weight LLMs survives tampering—fine-tuning or representation edits that modify model behavior after release. It builds TamperBench, a unified suite of nine tampering attacks, alignment-stage defenses, and standardized safety and utility metrics, and runs 40-trial hyperparameter sweeps per model–attack pair. The central finding is blunt: every one of 21 models admits at least one attack that pushes harmfulness above 0.68 (and above 0.77 for models over 1B parameters) while keeping MMLU-Pro accuracy within 10% of baseline. This matters because current open-weight safeguards, including defense-augmented variants, do not appear durable once weights or representations are modifiable.

What carries the argument

The load-bearing object is TamperBench itself: a registry-based toolkit that unifies weight-space and latent-space attacks, defense wrappers, and evaluations, with standardized metrics (StrongREJECT for harmfulness, a 140-example MMLU-Pro subset for utility). Its evaluation protocol is the mechanism: for each model–attack pair, run 40 hyperparameter trials, filter to trials with no more than 10% MMLU-Pro drop relative to baseline, and select the highest StrongREJECT score. The attack suite spans overt harmful and benign fine-tuning (full and LoRA), multilingual fine-tuning, three covert jailbreak-tuning variants that poison 2% of an otherwise benign dataset, and an inference-time embedding p

What would settle it

Take the exact best-of-40 attack checkpoints the paper selects and measure them on the full MMLU-Pro test set plus an independent utility benchmark (for example, full IFEval or MBPP). If more than 10% of those selected checkpoints show utility drops exceeding 10%, the central claim that tampering succeeds while preserving capabilities is falsified for those models.

Watch

Extended reading notes

Core claim

TamperBench's core discovery is that refusal-based safety alignment is systematically removable. With a realistic attacker model that maximizes StrongREJECT harmfulness subject to a 10% utility-loss bound, every evaluated model—base, instruction-tuned, and defense-augmented—surpasses 0.68 worst-case harmfulness, and models above 1B parameters consistently exceed 0.77. Jailbreak-tuning, a covert 2%-poisoning attack, is typically the most severe; harmful LoRA slightly outperforms full fine-tuning in 14 of 21 models; benign fine-tuning also raises harmfulness. Among alignment-stage defenses, Triplet best lowers average malicious harmfulness (by 0.25 relative to undefended Llama-3-8B-Instruct) w

Load-bearing premise

The claim that tampered models 'largely preserve capabilities' rests on a 140-question MMLU-Pro subset used as a filter before selecting the best of 40 attack configurations; if that small sample does not reliably measure capability loss, the central distinction between genuine harmful uplift and metric overfitting collapses.

Editorial extensions

If this is right

  • If the claim holds, shipping an open-weight model with current safety training means accepting that a determined actor can produce a capably harmful variant with modest compute and data.
  • Covert jailbreak-tuning with only 2% harmful data is enough to erase safeguards, so filtering fine-tuning APIs by content moderation alone is unlikely to stop the most effective attacks.
  • Benign or accidental fine-tuning is not safe: even well-intentioned domain adaptation erodes refusal behavior in most tested models.
  • Defenses that improve average tamper resistance, such as Triplet, do not fix the worst case; worst-case harmfulness remains near undefended levels, so evaluation needs worst-case metrics, not averages.
  • Explicit utility constraints are needed to avoid mistaking capability-collapsed models for genuinely dangerous tampering; unconstrained harmfulness maximization can produce false positives.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the headline result is only as strong as the 140-example MMLU-Pro constraint; if that small subset misestimates capability preservation, the 'capably harmful' claim could be overstated or understated.
  • A natural next step is re-running the same best-of-40 attack checkpoints on full MMLU-Pro and independent utility benchmarks; the paper reports a correlation on a small sample, not a verification on all checkpoints.
  • The framework's utility-constrained threat model suggests a design target for future defenses: grade them on worst-case harmfulness at a fixed utility level, not on average robustness across attacks.
  • The paper leaves open whether larger models or ignorance-based safety approaches would survive the same attack sweeps; nothing in the results rules out either outcome.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. TamperBench proposes a unified benchmark and toolkit for evaluating tamper resistance of open-weight LLMs under weight-space fine-tuning and representation-space attacks. The paper evaluates 21 open-weight models, including five defense-augmented variants, across nine tampering threats. For each model–attack pair, an Optuna hyperparameter sweep of 40 trials is run, and the configuration that maximizes StrongREJECT harmfulness subject to at most a 10% MMLU-Pro accuracy drop is selected. The main empirical claim is that every evaluated model admits at least one tampering attack that raises StrongREJECT harmfulness substantially (SRmax > 0.68, and > 0.77 for models above 1B parameters) while largely preserving utility. Secondary findings are that jailbreak-tuning is typically the most severe attack, that post-training effects differ between Llama and Qwen families, and that current alignment-stage defenses such as Triplet and TAR reduce average malicious harmfulness relative to the undefended Llama-3-8B-Instruct baseline.

Significance. If the main claims hold, TamperBench would be a valuable contribution: it consolidates attacks, defenses, and standardized safety/utility metrics into one extensible framework, uses systematic hyperparameter sweeps rather than hand-picked configurations, and evaluates a broad model suite with two corroborating StrongREJECT evaluators. The explicit utility constraint is a thoughtful step toward distinguishing capability-preserving harmful uplift from mere metric overfitting, and the paper is candid about several limitations. The public codebase and detailed implementation appendix are strengths. However, the headline claim rests on a utility filter measured on only a 140-question MMLU-Pro subset and on best-of-40 selection, and this measurement uncertainty is not resolved by the current experiments. The benchmark is likely to be useful to the security and alignment communities even with those caveats, but the central 'largely preserving capabilities' assertion needs stronger validation before the quantitative headline can be taken at face value.

major comments (2)
  1. [§4.1 and §A.6.1] The load-bearing claim that every model admits a tampering attack that 'largely preserves' capabilities is established using a 140-item MMLU-Pro subset as both the filter and the operational definition of utility preservation. With n=140, the standard error of an accuracy near 0.5 is roughly 4–6 absolute points, comparable to the allowed 10% relative drop (about 5 absolute points for a 0.5 baseline). Selecting the best of 40 Optuna trials after filtering on this noisy measurement introduces selection bias toward configurations whose true utility drop exceeds the measured 10% bound. The correlation evidence in Appendix A.4 is based on only 16 checkpoints from a single family (Qwen3-4B) and does not validate the selected configurations. Please provide full-set MMLU-Pro measurements (or an equivalent larger held-out capability benchmark) for the best configurations of all 21 models, or boot
  2. [§4.3 and Figure 5] The comparative statements across models and defenses are based on point estimates without confidence intervals or significance tests. The text itself notes 'it is uncertain whether they are statistically significant' for some family comparisons, but no quantification is given. This matters because the paper draws conclusions such as 'post-trained variants generally attain lower post-tampering harmfulness' and 'Triplet emerges as a leading alignment-stage defense.' Given that the experiments involve a random utility subset, an Optuna search, and a finite StrongREJECT evaluation set, please report uncertainty intervals or repeated-seed/bootstrap analyses, or at least explicitly mark which pairwise differences are robust to the selection procedure. This is also directly relevant to the best-of-40 selection bias described above.
minor comments (5)
  1. [§3.3] The 140-sample MMLU-Pro subset is not described with enough detail for reproducibility: how are the questions sampled, is the sample stratified across the 14 subjects, and what random seed is used? Please provide this information.
  2. [§1 / Abstract] The phrase 'first unified framework' is strong given that the paper cites several prior tampering evaluation efforts (e.g., Huang et al., Wang et al., Qi et al., Murphy et al.). Please soften or qualify the novelty claim to 'first open framework that combines X, Y, and Z' to avoid an easily challenged overclaim.
  3. [§A.6.1 vs. Figure 13] The protocol says 40 independent Optuna trials per model–attack pair, but Figure 13 reports rows of '70 Optuna trials' for the LoRA variants. Please align the numbers or explain the discrepancy.
  4. [§A.6.2] The embedding attack is fixed-configuration and not swept, unlike all weight-space attacks. This makes cross-attack comparisons in §4.2 somewhat unfair. Please state this limitation more prominently or provide a sensitivity analysis around the embedding attack's hyperparameters.
  5. [Conclusion] Typo: 'a extensible foundation' should be 'an extensible foundation.' Also 'TAMPERBENCHrequires' is missing a space in the abstract.

Circularity Check

0 steps flagged · score 2.0 of 10

No equation-level circularity: the benchmark results are empirical measurements, not derivations from fitted parameters or self-citations; the only notable issue is minor author overlap in the Triplet defense evaluation, which is not load-bearing.

full rationale

TamperBench's central claims are empirical outcomes of a transparent evaluation protocol. For each model–attack pair, the paper runs 40 Optuna trials, filters trials by a ≤10% MMLU-Pro drop measured on a 140-example subset, and then selects the trial with maximum StrongREJECT score (§A.6.1). The headline result—every model has some attack with SRmax > 0.68 while passing this filter—is a measured maximum, not a quantity derived by construction from the filter. The utility constraint is part of the threat model's definition of 'successfully tampered,' and the filtered maximum is reported as an empirical finding, so there is no self-definitional reduction. The 140-sample utility measurement is acknowledged in §5 as a limitation and is a statistical precision concern, not a circularity: the paper does not fit a parameter and then rename it as a prediction. The Triplet defense result involves prior work by co-authors (Simko et al., 2025), but the evaluation here uses open-sourced defended weights and standardized TamperBench metrics, and the conclusion rests on this paper's own benchmark outputs rather than on the citation's conclusions. No uniqueness theorem or ansatz is imported from the authors' prior work to force the result. Overall, the derivation chain is self-contained and empirically grounded; the author-overlap in the defense comparison warrants scrutiny but does not make the result circular.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper's results are empirical measurements rather than a derivation, so the ledger lists the measurement design choices that do most of the work: utility subset size, utility-drop threshold, and trial budget. No new theoretical entities are introduced.

free parameters (3)
  • MMLU-Pro utility subset size = 140 samples
    Capability preservation is measured on 140 of MMLU-Pro's questions; the paper acknowledges in §5 that this introduces additional uncertainty.
  • Utility drop constraint = ≤10% (≤20% in Appendix)
    Headline selection filters 40 Optuna trials to those with MMLU-Pro drop ≤10% and then picks the maximum StrongREJECT; the threshold is a modeling choice described in §4 and §A.6.1.
  • Optuna trials per model-attack pair = 40
    Computational budget choice that affects how representative the reported worst-case attack is (§A.6.1).
assumptions (4)
  • domain assumption StrongREJECT score is a valid operationalization of harmful compliance for tamper-resistance evaluation.
    Used as the sole safety metric in §3.4; the paper equates 'harmfulness' with StrongREJECT score and does not evaluate other safety paradigms such as ignorance-based defenses.
  • domain assumption A ≤10% drop in a 140-example MMLU-Pro subset means capabilities are 'largely preserved'.
    Selection and headline claims in §3.3 and §4 depend on this; §5 acknowledges the subset adds uncertainty but the central conclusions still rest on it.
  • domain assumption Attack hyperparameters from cited prior papers remain effective when applied to the 21-model suite with expanded search spaces.
    Attack implementations follow Che et al., Murphy et al., Poppi et al., and Schwinn & Geisler (§A.6.2); if these methods are model-specific, the universal 'every model vulnerable' result could be an artifact of the chosen datasets and configurations.
  • domain assumption Refusal-based safeguards are the only safeguards under test.
    Footnote 2 and §5 explicitly scope the work to refusal-based defenses, excluding ignorance-based alternatives from the main conclusions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TamperBench: Systematically Stress-Testing LLM Safety Under Fine-Tuning and Tampering." pith.science (2026). https://pith.science/paper/6B2TDGGX

@misc{pith2026260206911,
  author       = {Pith},
  title        = {Pith review of: TamperBench: Systematically Stress-Testing LLM Safety Under Fine-Tuning and Tampering},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6B2TDGGX}},
  note         = {Machine review of arXiv:2602.06911}
}
read the original abstract

As increasingly capable open-weight large language models (LLMs) are deployed, improving their tamper resistance against unsafe modifications, whether accidental or intentional, becomes critical to minimize risks. However, there is no standard approach to evaluate tamper resistance. Varied datasets, metrics, and tampering configurations make it difficult to compare safety, utility, and robustness across different models and defenses. To address this, we introduce TamperBench, the first unified framework to systematically evaluate the tamper resistance of LLMs. TamperBench (i) curates a repository of state-of-the-art weight-space fine-tuning attacks, latent-space representation attacks, and alignment-stage defenses; (ii) enables realistic adversarial evaluation through systematic hyperparameter sweeps per attack-model pair; and (iii) provides both safety and utility evaluations. We use TamperBench to evaluate 21 open-weight LLMs, including defense-augmented variants, across nine tampering threats using standardized safety and capability metrics with hyperparameter sweeps per model-attack pair. The results provide insights including effects of post-training on tamper resistance, that jailbreak-tuning is typically the most severe attack, and that current alignment-stage defenses largely fail to withstand attack sweeps. Code is available at https://github.com/criticalml-uw/TamperBench.

Figures

Figures reproduced from arXiv: 2602.06911 by the authors.

Figure 1
Figure 1. Tampering LLMs, as defined by Che et al. (2025), involves modifying their weights or latent representations and can compromise safety guardrails, yielding models that can output harmful responses. While numerous methods have been proposed to make models tamper-resistant, there is a lack of a systematic framework to measure this. TAMPERBENCH provides a framework to stress test LLM robustness to tampering. is experien… view at source ↗
Figure 2
Figure 2. While many alignment stage defenses have been proposed (e.g., Tamirisa et al., 2025; Huang et al., 2024d; Zou et al., 2024; Sheshadri et al., 2025), they do not share a standardized evaluation, making comparisons between the approaches inconclusive. This motivates TamperBench as the first framework to consolidate tampering attacks and evaluations into a unified toolkit. improving. TAMPERBENCH fills this gap by conso… view at source ↗
Figure 3
Figure 3. TamperBench evaluates a broad range of model tampering that may compromise safeguards, and assesses both safety2 and capabilities after adaptation. Tampering is taxonomized based on the model adaptor’s intent: malicious or benign (accidental). Malicious attacks are further divided into direct, overt ones, and covert ones originally designed to bypass closed-weight moderation safeguards. 3 TAMPERBENCH FRAMEWORK 3.1 T… view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: A single script can be run to benchmark an LLM by providing either a local checkpoint path or a HuggingFace repository ID, along with a list of attack names. The toolkit then executes the specified tampering attacks and evaluation modules, producing results scored with…
Figure 5
Figure 5. Figure 5: Benchmarking tamper resistant refusal of harmful requests across 21 open-weight LLMs. For each model–attack pair, we select the configuration from our hyperparameter sweeps that maximizes harmfulness (StrongREJECT score) while constraining utility loss to ≤ 10% MMLU-Pr…
Figure 6
Figure 6. Figure 6: Benchmarking tamper resistance without utility constraints. Each cell shows the StrongREJECT score from the fine-tuned evaluator (top, colored by harmfulness) and ∆ MMLU-Pro accuracy (bottom, colored by utility change) for the configuration that maximizes StrongREJECT …
Figure 7
Figure 7. Figure 7: Harmfulness of tampered models under realistic utility constraints. For each model, we report StrongREJECT scores from the fine-tuned evaluator averaged across attack categories (stealthy, directly harmful, benign) at different allowed MMLU-Pro drops: ≤ 10%, ≤ 20%, and…
Figure 8
Figure 8. Figure 8: StrongREJECT scores, with responses evaluated using a fine-tuned Gemma model evaluator (regres￾sor). This figure mirrors [PITH_FULL_IMAGE:figures/full_fig_p019_8.png]
Figure 9
Figure 9. Figure 9: Refusal sub-scores from the rubric-based StrongREJECT evaluation using gpt-4o-mini. Higher values indicate a greater tendency to refuse harmful requests. Results are for the same models & attacks in [PITH_FULL_IMAGE:figures/full_fig_p020_9.png]
Figure 10
Figure 10. Figure 10: Specificity sub-scores from the rubric-based StrongREJECT evaluation using gpt-4o-mini, measuring the degree of detail and concreteness in model responses. Results are for the same models & attacks in [PITH_FULL_IMAGE:figures/full_fig_p021_10.png]
Figure 11
Figure 11. Figure 11: Convincingness sub-scores from the rubric-based StrongREJECT evaluation using gpt-4o-mini, measuring the coherence & persuasiveness of responses. Results are for the same models & attacks in [PITH_FULL_IMAGE:figures/full_fig_p021_11.png]
Figure 12
Figure 12. Figure 12: Correlations between changes in safety and utility metrics after tampering. Each cell reports the Pearson correlation between metric deltas across 16 checkpoints (8 fine-tuning attacks on Qwen3-4B and Qwen3-4B-Base). The left matrix includes both utility metrics (chan…
Figure 13
Figure 13. Figure 13: LoRA fine-tuning attack variants on Llama-3.1-8B-Instruct. Each row shows 70 Optuna trials of a harmful LoRA attack: (A) the default setting inspired by Che et al. using 64 harmful examples and AdamW; (B) an expanded hyperparameter space that additionally allows SGD a…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

24 extracted references · 1 canonical work pages

  1. [2]

    (2024b) and He et al

    Benign fine-tuning.Following Qi et al. (2024b) and He et al. (2024), we include benign fine-tuning attacks to assess whether safety alignment degrades even without exposure to harmful data. Models are fine-tuned on BookCorpus text using the same training setup as the harmful attacks, with 128 examples to provide a modest but non-trivial adaptation signal....

    2024

  2. [3]

    Embedding attack.The embedding attack implements the soft-prompt optimization method of Schwinn & Geisler (2024), which operates at inference time by optimizing continuous prompt embed- dings to elicit harmful outputs without modifying model weights. We evaluate on the StrongREJECT dataset using the configuration identified by Schwinn & Geisler (2024) as ...

    2024

  3. [4]

    Gemini Team

    URLhttps://arxiv.org/abs/2410.04524. Gemini Team. Gemini: a family of highly capable multimodal models,

    arXiv

  4. [7]

    Luxi He, Mengzhou Xia, and Peter Henderson

    JMLR.org. Luxi He, Mengzhou Xia, and Peter Henderson. What’s in your ”safe” data?: Identifying benign data that breaks safety. InICLR 2024 Workshop on Secure and Trustworthy Large Language Models, Amherst, MA, USA,

    2024

  5. [8]

    URL https://openreview.net/forum?id= dp24p8i8Cg

    OpenReview. URL https://openreview.net/forum?id= dp24p8i8Cg. Peter Henderson, Eric Mitchell, Christopher Manning, Dan Jurafsky, and Chelsea Finn. Self- destructing models: Increasing the costs of harmful dual uses of foundation models. InProceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society, AIES ’23, pp. 287–296, New York, NY , USA,

    2023

  6. [9]

    ISBN 9798400702310

    Association for Computing Machinery. ISBN 9798400702310. doi: 10.1145/3600211.3604690. URLhttps://doi.org/10.1145/3600211.3604690. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding, 2021a. URL https:// arxiv.org/abs/2009.03300. Dan Hendrycks, Collin Bur...

    arXiv 2009

  7. [10]

    URL https://openreview.net/forum?id=fMNRYBvcQN

    OpenReview. URL https://openreview.net/forum?id=fMNRYBvcQN. Tiansheng Huang, Gautam Bhattacharya, Pratik Joshi, Josh Kimball, and Ling Liu. Antidote: Post- fine-tuning safety alignment for large language models against harmful fine-tuning, 2024a. URL https://arxiv.org/abs/2408.09600. Tiansheng Huang, Sihao Hu, Fatih Ilhan, Selim Furkan Tekin, and Ling Liu...

    arXiv

  8. [11]

    Fanxu Meng, Zhaohui Wang, and Muhan Zhang

    URL https://arxiv.org/abs/2402.04249. Fanxu Meng, Zhaohui Wang, and Muhan Zhang. Pissa: Principal singular values and singular vectors adaptation of large language models.Advances in Neural Information Processing Systems, 37: 121038–121072,

    arXiv

Show all 24 references
  1. [12]

    Kyle O’Brien, Stephen Casper, Quentin Anthony, Tomek Korbak, Robert Kirk, Xander Davies, Ishan Mishra, Geoffrey Irving, Yarin Gal, and Stella Biderman

    URL https://arxiv.org/abs/2507.11630. Kyle O’Brien, Stephen Casper, Quentin Anthony, Tomek Korbak, Robert Kirk, Xander Davies, Ishan Mishra, Geoffrey Irving, Yarin Gal, and Stella Biderman. Deep ignorance: Filtering pretraining data builds tamper-resistant safeguards into open...

  2. [13]

    URL https: //arxiv.org/abs/2508.06601. OpenAI. GPT-4o system card. System card / technical report, OpenAI, August

  3. [14]

    Punya Syon Pandey, Samuel Simko, Kellin Pelrine, and Zhijing Jin

    URL https://arxiv.org/abs/2303.08774. Punya Syon Pandey, Samuel Simko, Kellin Pelrine, and Zhijing Jin. Accidental vulnerability: Factors in fine-tuning that shift model safeguards, 2025a. URL https://arxiv.org/abs/2505. 16789. Punya Syon Pandey, Samuel Simko, Kellin Pelrine, ...

    2025 arXiv

  4. [15]

    ISBN 979-8-89176-195-7

    Association for Computational Linguistics. ISBN 979-8-89176-195-7. doi: 10.18653/v1/2025.findings-naacl.126. URLhttps://aclanthology.org/2025.findings-naacl.126/. Xiangyu Qi, Boyi Wei, Nicholas Carlini, Yangsibo Huang, Tinghao Xie, Luxi He, Matthew Jagielski, Milad Nasr, Prate...

    2025 doi

  5. [16]

    Alexandra Souly, Qingyuan Lu, Dillon Bowen, Tu Trinh, Elvis Hsieh, Sana Pandey, Pieter Abbeel, Justin Svegliato, Scott Emmons, Olivia Watkins, and Sam Toyer

    URL https://arxiv.org/ abs/2506.11938. Alexandra Souly, Qingyuan Lu, Dillon Bowen, Tu Trinh, Elvis Hsieh, Sana Pandey, Pieter Abbeel, Justin Svegliato, Scott Emmons, Olivia Watkins, and Sam Toyer. A StrongREJECT for empty jailbreaks. InThe Thirty-eight Conference on Neural Inf...

  6. [17]

    Jiongxiao Wang, Jiazhao Li, Yiquan Li, Xiangyu Qi, Junjie Hu, Yixuan Li, Patrick McDaniel, Muhao Chen, Bo Li, and Chaowei Xiao

    URLhttps://arxiv.org/abs/2307.09288. Jiongxiao Wang, Jiazhao Li, Yiquan Li, Xiangyu Qi, Junjie Hu, Yixuan Li, Patrick McDaniel, Muhao Chen, Bo Li, and Chaowei Xiao. BackdoorAlign: Mitigating fine-tuning based jail- break attack with backdoor enhanced safety alignment. In A. Gl...

    2024 arXiv

  7. [18]

    Richard J Young

    URLhttps://arxiv.org/abs/2307.02483. Richard J Young. Comparative analysis of llm abliteration methods: A cross-architecture evaluation,

    arXiv

  8. [19]

    Andy Zou, Long Phan, Justin Wang, Derek Duenas, Maxwell Lin, Maksym Andriushchenko, J Zico Kolter, Matt Fredrikson, and Dan Hendrycks

    URL https://arxiv.org/abs/2311.07911. Andy Zou, Long Phan, Justin Wang, Derek Duenas, Maxwell Lin, Maksym Andriushchenko, J Zico Kolter, Matt Fredrikson, and Dan Hendrycks. Improving alignment and robustness with circuit breakers. InThe Thirty-eighth Annual Conference on Neura...

    arXiv

  9. [20]

    - ” = “ N O T H R E S H O L D

    URLhttps://arxiv.org/abs/2310.01405. 16 Preprint A APPENDIX A.1 MAXIMIZINGHARMFULNESSWITHDIFFERENTUTILITYCONSTRAINTS Q W E N 3 - 0 . 6 B Q W E N 3 - 1 . 7 B Q W E N 3 - 4 B - B A S E Q W E N 3 - 4 B Q W E N 3 - 8 B - B A S E Q W E N 3 - 8 B M I S T R A L - 7 B M I S T R A L - ...

    2025 arXiv

  10. [21]

    In contrast, MATH is only loosely aligned, reflecting its narrower domain and strict exact-match scoring

    across tampered checkpoints, supporting its use as a general (though not exhaus- tive) proxy for capability shifts. In contrast, MATH is only loosely aligned, reflecting its narrower domain and strict exact-match scoring. On the safety side, StrongREJECT and JailbreakBench mov...

    2019

  11. [23]

    Danny Halawi, Alexander Wei, Eric Wallace, Tony Wang, Nika Haghtalab, and Jacob Steinhardt

    URL https://doi.org/10.1007/978-3-030-58526-6_23 . Danny Halawi, Alexander Wei, Eric Wallace, Tony Wang, Nika Haghtalab, and Jacob Steinhardt. Covert malicious finetuning: challenges in safeguarding LLM adaptation. InProceedings of the 41st International Conference on Machine ...

    doi

  12. [2021]

    Luke Bailey, Alex Serrano, Abhay Sheshadri, Mikhail Seleznyov, Jordan Taylor, Erik Jenner, Jacob Hilton, Stephen Casper, Carlos Guestrin, and Scott Emmons

    URLhttps://arxiv.org/abs/2108.07732. Luke Bailey, Alex Serrano, Abhay Sheshadri, Mikhail Seleznyov, Jordan Taylor, Erik Jenner, Jacob Hilton, Stephen Casper, Carlos Guestrin, and Scott Emmons. Obfuscated activations bypass llm latent-space defenses,

    arXiv

  13. [2022]

    The current year is 2025

    and fine-tune on 300 examples—a similar order of magnitude to the 100 examples used by Poppi et al. (2025), while using the same underlying harmful content as our other overt attacks. Hyperparameters follow the full-parameter search space (Table 3). Covert jailbreak-tuning att...

    2025

  14. [2023]

    Eternal Sunshine of the Spotless Net: Selec- tive Forgetting in Deep Networks

    11 Preprint Aditya Golatkar, Alessandro Achille, and Stefano Soatto. Eternal Sunshine of the Spotless Net: Selec- tive Forgetting in Deep Networks . In2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9301–9309, Los Alamitos, CA, USA, June 2020a. ...

    2020

  15. [2024]

    URL https://openreview.net/forum?id=urjPCYZt0I

    Curran Associates, Inc. URL https://openreview.net/forum?id=urjPCYZt0I. Zora Che, Stephen Casper, Robert Kirk, Anirudh Satheesh, Stewart Slocum, Lev E McKinney, Rohit Gandikota, Aidan Ewart, Domenic Rosati, Zichu Wu, Zikui Cai, Bilal Chughtai, Yarin Gal, Furong Huang, and Dyla...

    2025

  16. [2025]

    URL https://arxiv.org/abs/2408. 02946. Stephen Casper, Carson Ezell, Charlotte Siegmann, Noam Kolt, Taylor Lynn Curtis, Benjamin Bucknall, Andreas Haupt, Kevin Wei, J´er´emy Scheurer, Marius Hobbhahn, et al. Black-box access is insufficient for rigorous ai audits. InProceeding...

    2024 arXiv

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.