Pith. sign in

REVIEW 3 major objections 6 minor 39 references

When safety models are merged, blunt refusal can overwrite fine-grained harm recognition, collapsing classification accuracy to at most 12.9% while jailbreak refusal stays above 81%.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-07-31 23:34 UTC pith:7SFZPTLF

load-bearing objection The descriptive asymmetry is real and cleanly shown; the magnitude explanation is plausible but under-supported — needs a norm-matched merge. the 3 major comments →

arxiv 2607.27240 v1 pith:7SFZPTLF submitted 2026-07-26 cs.AI

Asymmetric Collapse in Model Merging: When Refusal Over- writes Recognition

Aarnav Choudhary , Matheus Fonseca Rocha , Jiwon Seo , Vasu Sharma , Maheep Chaudhary This is my paper
classification cs.AI
keywords model mergingtask vectorssafety alignmentjailbreak refusalharm classificationmagnitude imbalanceattention headsasymmetric transfer
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to show that standard model merging, when combining two safety fine-tunes, does not preserve complementary safety behaviors: a coarse refusal behavior systematically survives while a fine-grained harm-level classification capability is largely destroyed. Across four common merging methods, merged models retained 81–85% of jailbreak refusal rates but at most 12.9% of classification accuracy. The authors argue this happens not because the two learned updates point in opposite directions—they are nearly orthogonal—but because the refusal update is consistently 4–5 times larger in magnitude, so magnitude-sensitive merge operations favor refusal. A reader should care because safety pipelines that merge models may silently produce over-refusing, under-recognizing systems, and this failure may extend to any fine-grained judgment skill paired with a coarser behavioral objective.

Core claim

The paper reports a consistent asymmetric transfer in model merging: for two safety-oriented fine-tunes of a 1-billion-parameter instruction-tuned model, one trained to classify prompts into four graded harm levels (CARES) and one trained to refuse jailbreak attempts (WildJailbreak), all four tested merging methods preserve refusal behavior (81.1–85.3% attack resistance) while classification accuracy collapses to at most 12.9% (often near 0.6–0.8%). The mechanism proposed is magnitude imbalance: the refusal fine-tune's task vector (the weight difference from the base model) has 4–5× larger per-layer L2 norms than the classification fine-tune's task vector, and the two are nearly orthogonal (

What carries the argument

The central object is the task vector: the element-wise difference between a fine-tuned model's weights and the base model's weights. The paper measures per-layer L2 norms of these vectors and uses them to explain why magnitude-sensitive merge operations—trimming small updates, electing sign by magnitude, and magnitude-weighted averaging—favor the refusal task. A second mechanism is the attention-head necessity mask, a binary mask over all 104 attention heads that identifies the minimal head set needed to preserve each task's performance; comparing head survival after merging reveals that heads can remain structurally present but functionally corrupted.

Load-bearing premise

The claim that magnitude imbalance causes the collapse rests on a single rerun in which the refusal model was retrained on a smaller dataset, which simultaneously shrank the magnitude gap and made the refusal model weaker by itself, so the two changes are not cleanly separated.

What would settle it

Run the same four merges after normalizing the two full-data task vectors to equal per-layer L2 norms (or per-layer norm-matching) without retraining. If classification accuracy remains at or below chance while refusal stays high, then magnitude imbalance is not the driver and the mechanism claim fails; if classification recovers substantially, the magnitude account is confirmed.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Safety-oriented model merging can yield models that refuse broadly but cannot distinguish levels of harm, undermining graded safety judgments such as medical triage or content moderation.
  • Among the four standard methods, TIES and DARE-TIES can drive classification accuracy to near zero (0.6–0.8%), showing that supposedly interference-reducing methods can still amplify a scale imbalance.
  • Aggregate weight-space statistics, such as overlap in necessary attention heads, overstate functional preservation; merged weights may retain the right heads yet produce wrong outputs.
  • If the magnitude account is correct, normalizing task vectors to equal per-layer norms before merging should restore at least partial classification accuracy—a directly testable design change for future merge recipes.
  • The same asymmetry may apply beyond safety: any fine-grained recognition capability merged with a coarser behavioral objective that produces larger updates is at risk of being overwritten.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A plausible general principle extends from this case: in any merge, the behavior with the larger-magnitude task vector may dominate, regardless of which behavior is more desirable—so 'coarse and strong' can silently displace 'fine and weak'.
  • A norm-matched merge, which the paper explicitly leaves for future work, is the cleanest direct test of the mechanism; if it preserves classification, then magnitude is causal, and if not, the asymmetry may stem from refusal strength or task difficulty.
  • The head-analysis finding suggests that methods that aim to preserve task-specific heads (e.g., by location or attribution) should also verify functional preservation, not just structural masks, since survival rates here are misleading.
  • The balanced-data rerun implies dataset scale is not the sole cause for most methods, but DARE-TIES' inversion under balanced data indicates that stochastic sparsification can flip which task wins when magnitudes are closer—an opening for adaptive merge strategies.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper studies whether standard model merging can preserve two complementary safety behaviors simultaneously. Using Gemma-3-1B-IT, the authors fine-tune one model on CARES harm-level classification and another on WildJailbreak adversarial refusal, then merge them with Linear, SLERP, TIES, and DARE-TIES. Across all four methods, jailbreak refusal transfers substantially (81.1–85.3% attack resistance) while CARES four-class classification accuracy collapses to at most 12.9%. The authors propose a mechanism: the refusal task vector has 4–5× larger per-layer L2 norms than the classification task vector, and magnitude-sensitive merge operations therefore favor refusal updates. This is supported by near-orthogonality of the task vectors, a balanced-data rerun with a 9k WildJailbreak subsample, and attention-head necessity masks. The paper concludes that model merging can collapse safety recognition into broad refusal when safety-related task vectors differ substantially in scale.

Significance. The descriptive finding is valuable and likely to be of interest to the model-merging and safety communities. The controlled setup, use of external benchmarks (CARES, WildJailbreak), four merging methods, and the additional head-level analysis are strengths. The paper is also unusually candid about its limitations, explicitly noting that the balanced-data experiment covaries magnitude reduction with weaker refusal strength and that a norm-matched merge is future work. If the causal magnitude claim were fully supported, the paper would be a useful cautionary result for safety-aware merging. Even without the causal claim, the descriptive asymmetry—refusal overwrites fine-grained recognition across standard merging methods—is a solid empirical contribution. However, the abstract and conclusion currently assert a stronger causal statement than the evidence supports.

major comments (3)
  1. [Data size normalization / Appendix A, Table 2] The balanced-data experiment is the main evidence that magnitude asymmetry is causal, but it changes two variables at once: the WJB task-vector magnitude gap shrinks by roughly 40×, and the refusal specialist becomes weaker in isolation (88.0% vs. 77.0% attack resistance). The paper acknowledges this caveat, but the abstract and conclusion nonetheless assert that collapse occurs because task vectors 'differ substantially in scale.' A norm-matched merge of the full-data models is needed to separate magnitude from refusal strength. In addition, the paper does not report per-layer L2 norms for the balanced WJB vector, so it is not shown that the magnitude gap was actually eliminated. Without this, the causal mechanism is underdetermined; the descriptive asymmetry is well established, but the claim that magnitude is the proximate cause is not.
  2. [Table 1 and Section 3] All reported numbers are single-run point estimates with no seeds, confidence intervals, or repeated evaluations. This is particularly important for DARE-TIES, whose stochastic sparsification produces a dramatic inversion in the balanced run (CARES accuracy 0.8% to 20.0%; attack resistance 85.3% to 29.2%). One run is insufficient to establish the stability of either the headline asymmetry or the DARE-TIES inversion that is used as causal support. The word 'significantly' in the abstract also implies statistical testing that is not reported. At minimum, repeated runs with different seeds for fine-tuning, merging, and DARE sparsification are needed.
  3. [Appendix B: Attention Head Identification] The head-necessity masks are optimized with a (1+λ) evolutionary strategy, and only one mask is reported per model. The manuscript states that exact head positions are 'one valid solution rather than canonical,' yet the head-survival analysis draws conclusions from these single solutions. More seriously, the ±3% tolerance used during mask optimization is applied to each model's own baseline; for merged models with near-zero CARES accuracy (e.g., 0.6%), a tolerance of ±3% is far larger than the baseline and may make the mask underconstrained. This could distort the reported survival fractions (e.g., 'CARES heads remain but no longer classify'). The head-level evidence is not load-bearing for the main descriptive claim, but it is used to support the mechanism and should be made robust or explicitly framed as illustrative.
minor comments (6)
  1. [Appendix B, first paragraph] Typo: 'MethodGemma-3-1B-IT uses...' is missing a space or period after 'Method'.
  2. [Figure 4 caption] Typo: 'T op' should be 'Top'.
  3. [Appendix A] The 9k WildJailbreak subsample procedure is not specified. Was it a random subsample, stratified by attack category, or a fixed seed? This matters for reproducibility and for interpreting DARE-TIES's inversion.
  4. [Section 3, Mechanistic analysis] The per-layer L2 norm measurements and the global cosine similarity are described in prose but no table or figure is provided. A small table reporting per-block norms for both fine-tunes (and, ideally, for the balanced WJB model) would make the magnitude claim easier to evaluate.
  5. [Section 3, first paragraph] The sentence 'No model successfully retains harm recognition' is stated in the full-data context, but the balanced DARE-TIES result achieves 20.0% CARES accuracy. Clarify that the sentence refers to the full-data merges only.
  6. [Table 2] The abbreviation 'Atk.' is used in Table 2 while Table 1 uses 'Atk. Resist.'; keep the notation consistent.

Circularity Check

0 steps flagged

No circularity: the core asymmetry is externally measured; the magnitude explanation is confounded but not circular.

full rationale

The paper's central claim—that refusal transfers more than classification accuracy across Linear, SLERP, TIES, and DARE-TIES—is an empirical result measured against external benchmarks (CARES accuracy and WildJailbreak attack resistance). No parameter is fitted to the outcome and then renamed as a prediction; the merge methods and evaluation metrics are fixed in advance. The mechanistic explanation (4–5× larger per-layer task-vector magnitudes for the refusal fine-tune) is post hoc, but it is tested with a data-size normalization rerun and an independent attention-head necessity analysis. The main weakness is that the balanced rerun changes two variables at once—reducing the magnitude gap by ~40× while also producing a weaker refusal specialist (77.0% vs. 88.0% resistance)—so the causal role of magnitude is underdetermined. This is a confound and a correctness/validity concern, not circularity: the paper explicitly acknowledges the co-variation and states that a norm-matched merge is left to future work. There are no load-bearing self-citations, no imported uniqueness theorems, no ansatz smuggled in via citation, and no equation that is equivalent to the target result by construction. The paper is self-contained against external benchmarks, so the circularity score is 0.

Axiom & Free-Parameter Ledger

1 free parameters · 4 axioms · 0 invented entities

The central claim rests on standard-but-unverified assumptions about metric adequacy, fine-tuning faithfulness, and mergekit correctness, plus one hand-chosen merge weight. No new entities are postulated.

free parameters (1)
  • merge weight α = 0.5
    Equal weighting for all four methods; the paper notes sweeps are left to future work. The observed collapse could shift at other α, so the headline numbers are tied to this hand-chosen value.
axioms (4)
  • domain assumption CARES accuracy, WJB attack resistance, and benign compliance are adequate measures of the two safety behaviors
    The paper uses these three metrics to define retention and collapse; if the evaluation sets or metrics are not representative, the asymmetry could be an artifact of metric sensitivity. See Section 2 and Table 1.
  • domain assumption Full fine-tuning on public datasets with the stated hyperparameters produces specialists whose task vectors faithfully represent learned behaviors
    Section 2 'Fine-tuning procedure' states that full fine-tuning ensures the task vector is a complete and faithful representation; this is assumed, with no check on alternative data order or seed.
  • domain assumption The mergekit implementations of Linear, SLERP, TIES, and DARE-TIES are correct and standard
    Results depend on mergekit behavior, including the 0.9995 collinearity threshold for SLERP. No version or configuration details are verified.
  • domain assumption Attention-head necessity masks found by the evolutionary strategy are valid reflections of task-relevant heads
    Appendix B: one mask per model with a ±3% tolerance and 128-prompt calibration set; masks are a single solution, not canonical, so head-survival statistics may be unstable.

pith-pipeline@v1.3.0-alltime-deepseek · 6373 in / 11511 out tokens · 112406 ms · 2026-07-31T23:34:50.875503+00:00 · methodology

0 comments
read the original abstract

Model merging is often used to combine capabilities from separately fine-tuned models without additional training, but it is unclear whether standard merging methods preserve multiple safety-relevant behaviors simultaneously. We study this question through a controlled case study using two Gemma-3-1B-IT finetunes on two complementary safety objectives: CARES harm-level classification and WildJailbreak adversarial refusal. We merge the two fine-tunes using Linear, SLERP, TIES, and DARE-TIES, and evaluate the merged models on classification accuracy, attack resistance, and benign compliance. Across all four methods, attack resistance transfers significantly more than classification accuracy: merged models retain 81-85% jailbreak refusal rates while CARES accuracy falls to at most 12.9%. Weight-space measurements suggest that this asymmetry is not caused by strongly opposing task-vector directions: the two task vectors are nearly orthogonal (cosine similarity 0.011). Instead, the refusal fine-tune induces consistently larger per-layer task-vector magnitudes, causing magnitude-sensitive methods to favor refusal updates. These results show that standard model merging can collapse safety recognition into broad refusal when safety-relevant task vectors differ substantially in scale.

Figures

Figures reproduced from arXiv: 2607.27240 by Aarnav Choudhary, Jiwon Seo, Maheep Chaudhary, Matheus Fonseca Rocha, Vasu Sharma.

Figure 1
Figure 1. Figure 1: Experimental pipeline: data sources, fine-tuned models, merging methods, and [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Attention-head necessity (full finetuned) (a) Partition of the 104 heads by source role. (b) Survival of each role’s heads after merging; bar height is the surviving fraction (role base n in legend) and the number above each bar is the surviving head count. WJB-only heads survive at markedly higher rates than CARES-only heads under Linear and TIES [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Attention-head necessity (9k subsample) Same layout as [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Per-head necessity map Each cell is one attention head, ordered by (layer, head); the top row of each panel is the source-role assignment and the remaining rows are the four merges. Faded = head dropped by the merge; teal = head recruited from positions neither source required. Top: full finetunes. Bottom: 9k subsample. Heads lost under merging are spread across depth, and every method recruits a handful o… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

39 extracted references · 1 canonical work pages

  1. [4]

    Arcee's mergekit: A toolkit for merging large language models, 2024

    Charles Goddard, Shamane Siriwardhana, Malikeh Ehghaghi, Luke Meyers, Vladimir Karpukhin, Brian Benedict, Mark McQuade, and Jacob Solawetz. Arcee's mergekit: A toolkit for merging large language models, 2024

    2024

  2. [5]

    Model merging and safety alignment: One bad model spoils the bunch

    Hasan Abed Al Kader Hammoud, Umberto Michieli, Fabio Pizzati, Philip Torr, Adel Bibi, Bernard Ghanem, and Mete Ozay. Model merging and safety alignment: One bad model spoils the bunch. In Findings of the Association for Computational Linguistics: EMNLP 2024, pp.\ 13033--13046. Association for Computational Linguistics, 2024

    2024

  3. [6]

    Editing models with task arithmetic

    Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. Editing models with task arithmetic. In International Conference on Learning Representations (ICLR), 2023

    2023

  4. [7]

    Wildteaming at scale: From in-the-wild jailbreaks to (adversarially) safer language models

    Liwei Jiang, Kavel Rao, Seungju Han, Allyson Ettinger, Faeze Brahman, Sachin Kumar, Niloofar Mireshghallah, Ximing Lu, Maarten Sap, Yejin Choi, and Nouha Dziri. Wildteaming at scale: From in-the-wild jailbreaks to (adversarially) safer language models. In Thirty-eighth Conference on Neural Information Processing Systems (NeurIPS), 2024

    2024

  5. [8]

    LED -merging: Mitigating safety-utility conflicts in model merging with location-election-disjoint

    Qianli Ma, Dongrui Liu, Qian Chen, Linfeng Zhang, and Jing Shao. LED -merging: Mitigating safety-utility conflicts in model merging with location-election-disjoint. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL), pp.\ 21749--21767. Association for Computational Linguistics, 2025

    2025

  6. [9]

    Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, and Ludwig Schmidt

    Mitchell Wortsman, Gabriel Ilharco, Samir Yitzhak Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari S. Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, and Ludwig Schmidt. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time, 2022. URL https://arxiv.org/abs/2203.05482

    Pith/arXiv arXiv 2022

  7. [10]

    TIES -merging: Resolving interference when merging models

    Prateek Yadav, Derek Tam, Leshem Choshen, Colin Raffel, and Mohit Bansal. TIES -merging: Resolving interference when merging models. In Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS), 2023

    2023

  8. [11]

    A safety realignment framework via subspace-oriented model fusion for large language models

    Xin Yi, Shunfan Zheng, Linlin Wang, Xiaoling Wang, and Liang He. A safety realignment framework via subspace-oriented model fusion for large language models. In Knowledge-Based Systems, 2024

    2024

  9. [12]

    Language models are super mario: Absorbing abilities from homologous models as a free lunch, 2024

    Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, and Yongbin Li. Language models are super mario: Absorbing abilities from homologous models as a free lunch, 2024. URL https://arxiv.org/abs/2311.03099

    Pith/arXiv arXiv 2024

  10. [13]

    Representation engineering: A top-down approach to AI transparency

    Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, et al. Representation engineering: A top-down approach to AI transparency. arXiv preprint arXiv:2310.01405, 2024

    Pith/arXiv arXiv 2024

  11. [14]

    LED-Merging: Mitigating Safety-Utility Conflicts in Model Merging with Location-Election-Disjoint , url=

    Ma, Qianli and Liu, Dongrui and Chen, Qian and Zhang, Linfeng and Shao, Jing , year=. LED-Merging: Mitigating Safety-Utility Conflicts in Model Merging with Location-Election-Disjoint , url=. doi:10.18653/v1/2025.acl-long.1055 , booktitle=

    work page doi:10.18653/v1/2025.acl-long.1055 2025

  12. [15]

    2025 , eprint=

    CARES: Comprehensive Evaluation of Safety and Adversarial Robustness in Medical LLMs , author=. 2025 , eprint=

    2025

  13. [16]

    2024 , eprint =

    Model Swarms: Collaborative Search to Adapt LLM Experts via Swarm Intelligence , author =. 2024 , eprint =. doi:10.48550/arXiv.2410.11163 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2410.11163 2024

  14. [17]

    2024 , eprint=

    RUPBench: Benchmarking Reasoning Under Perturbations for Robustness Evaluation in Large Language Models , author=. 2024 , eprint=

    2024

  15. [18]

    2024 , eprint=

    Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunities , author=. 2024 , eprint=

    2024

  16. [19]

    2025 , eprint=

    Nature-Inspired Population-Based Evolution of Large Language Models , author=. 2025 , eprint=

    2025

  17. [20]

    2024 , eprint=

    FuseChat: Knowledge Fusion of Chat Models , author=. 2024 , eprint=

    2024

  18. [21]

    2022 , eprint=

    Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time , author=. 2022 , eprint=

    2022

  19. [22]

    2023 , eprint=

    TIES-Merging: Resolving Interference When Merging Models , author=. 2023 , eprint=

    2023

  20. [23]

    2024 , eprint=

    Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch , author=. 2024 , eprint=

    2024

  21. [24]

    2025 , eprint=

    Evolutionary Optimization of Model Merging Recipes , author=. 2025 , eprint=. doi:https://doi.org/10.1038/s42256-024-00975-8 , url=

    doi:10.1038/s42256-024-00975-8 2025

  22. [25]

    Remarkable Robustness of

    Vedang Lad and Jin Hwa Lee and Wes Gurnee and Max Tegmark , booktitle=. Remarkable Robustness of. 2025 , url=

    2025

  23. [26]

    Findings of ACL , year=

    Data Diversity Matters for Robust Instruction Tuning , author=. Findings of ACL , year=

  24. [27]

    UniReps: 3rd Edition of the Workshop on Unifying Representations in Neural Models , year=

    Bridging Large Gaps in Neural Network Representations with Model Stitching , author=. UniReps: 3rd Edition of the Workshop on Unifying Representations in Neural Models , year=

  25. [28]

    International Conference on Learning Representations (ICLR) , year=

    Editing Models with Task Arithmetic , author=. International Conference on Learning Representations (ICLR) , year=

  26. [29]

    Yadav, Prateek and Tam, Derek and Choshen, Leshem and Raffel, Colin and Bansal, Mohit , booktitle=

  27. [30]

    ACM SIGGRAPH Computer Graphics , volume=

    Animating Rotation with Quaternion Curves , author=. ACM SIGGRAPH Computer Graphics , volume=. 1985 , publisher=

    1985

  28. [31]

    Knowledge-Based Systems , year=

    A Safety Realignment Framework via Subspace-Oriented Model Fusion for Large Language Models , author=. Knowledge-Based Systems , year=

  29. [32]

    Proceedings of the 2024 ACM SIGSAC Conference on Computer and Communications Security (CCS) , year=

    BadMerging: Backdoor Attacks Against Model Merging , author=. Proceedings of the 2024 ACM SIGSAC Conference on Computer and Communications Security (CCS) , year=

    2024

  30. [33]

    Thirty-eighth Conference on Neural Information Processing Systems (NeurIPS) , year=

    WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Models , author=. Thirty-eighth Conference on Neural Information Processing Systems (NeurIPS) , year=

  31. [34]

    Proceedings of the 41st International Conference on Machine Learning (ICML) , pages=

    HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal , author=. Proceedings of the 41st International Conference on Machine Learning (ICML) , pages=. 2024 , volume=

    2024

  32. [35]

    Findings of the Association for Computational Linguistics: EMNLP 2024 , pages=

    Model Merging and Safety Alignment: One Bad Model Spoils the Bunch , author=. Findings of the Association for Computational Linguistics: EMNLP 2024 , pages=. 2024 , publisher=

    2024

  33. [36]

    2025 , publisher=

    Ma, Qianli and Liu, Dongrui and Chen, Qian and Zhang, Linfeng and Shao, Jing , booktitle=. 2025 , publisher=

    2025

  34. [37]

    arXiv preprint arXiv:2505.11413 , year=

    CARES: Comprehensive Evaluation of Safety and Adversarial Robustness in Medical LLMs , author=. arXiv preprint arXiv:2505.11413 , year=

    Pith/arXiv arXiv

  35. [38]

    arXiv preprint arXiv:2204.05862 , year=

    Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback , author=. arXiv preprint arXiv:2204.05862 , year=

    Pith/arXiv arXiv

  36. [39]

    arXiv preprint arXiv:2308.14132 , year=

    Detecting Language Model Attacks with Perplexity , author=. arXiv preprint arXiv:2308.14132 , year=

    Pith/arXiv arXiv

  37. [40]

    Representation Engineering: A Top-Down Approach to

    Zou, Andy and Phan, Long and Chen, Sarah and Campbell, James and Guo, Phillip and Ren, Richard and Pan, Alexander and Yin, Xuwang and Mazeika, Mantas and Dombrowski, Ann-Kathrin and others , journal=. Representation Engineering: A Top-Down Approach to

  38. [41]

    2024 , eprint =

    Arcee's MergeKit: A Toolkit for Merging Large Language Models , author =. 2024 , eprint =

    2024

  39. [42]

    Evaluating Open-Source Sparse Autoencoders on Disentangling Factual Knowledge in

    Chaudhary, Maheep and Geiger, Atticus , journal=. Evaluating Open-Source Sparse Autoencoders on Disentangling Factual Knowledge in