REVIEW 4 cited by
Model Merging and Safety Alignment: One Bad Model Spoils the Bunch
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Merging Large Language Models (LLMs) is a cost-effective technique for combining multiple expert LLMs into a single versatile model, retaining the expertise of the original ones. However, current approaches often overlook the importance of safety alignment during merging, leading to highly misaligned models. This work investigates the effects of model merging on alignment. We evaluate several popular model merging techniques, demonstrating that existing methods do not only transfer domain expertise but also propagate misalignment. We propose a simple two-step approach to address this problem: (i) generating synthetic safety and domain-specific data, and (ii) incorporating these generated data into the optimization process of existing data-aware model merging techniques. This allows us to treat alignment as a skill that can be maximized in the resulting merged LLM. Our experiments illustrate the effectiveness of integrating alignment-related data during merging, resulting in models that excel in both domain expertise and alignment.
Forward citations
Cited by 4 Pith papers
-
Consiglieres in the Shadow: Understanding the Use of Uncensored Large Language Models in Cybercrimes
Using a knowledge graph of model and dataset relationships, this paper identifies more than 11,000 uncensored LLMs on Hugging Face and documents their use in cybercrime services.
-
When Skills Meet Safety: Benchmarking and Characterizing the Adaptive Jailbreak Robustness of Skill-Merged LLMs
Static refusal tests overstate the safety of skill-merged LLMs: models with identical static safety differ sharply under adaptive attack, and a task-vector overlap with a safety subspace flags only same-recipe abliter...
-
On Almost Surely Safe Alignment of Large Language Models at Inference-Time
An inference-time beam-search method with a safety-state tracker and latent critic enforces a user-supplied safety cost model, with an almost-sure guarantee only relative to that model.
-
If You Can't Use Them, Recycle Them: Optimizing Merging at Scale Mitigates Performance Tradeoffs
Optimizing linear merge weights on 16 104B generalist checkpoints via CMA-ES reduces task tradeoffs and beats individual models and simple merge baselines.
Discussion (0). Continue with ORCID to comment.