Model Merging by Uncertainty-Based Gradient Matching
read the original abstract
Models trained on different datasets can be merged by a weighted-averaging of their parameters, but why does it work and when can it fail? Here, we connect the inaccuracy of weighted-averaging to mismatches in the gradients and propose a new uncertainty-based scheme to improve the performance by reducing the mismatch. The connection also reveals implicit assumptions in other schemes such as averaging, task arithmetic, and Fisher-weighted averaging. Our new method gives consistent improvements for large language models and vision transformers, both in terms of performance and robustness to hyperparameters. Code available here.
This paper has not been read by Pith yet.
Forward citations
Cited by 5 Pith papers
-
Model Merging as Probabilistic Inference in Fine-Tuning Parameter Space
Model merging is cast as PoE inference with EBM experts, revealing Gaussian assumptions in prior work and proposing convergent Cauchy experts that improve empirical performance.
-
Quantifying the Agreement Between Data-Influence and Data-Similarity to Understand LLM Behavior
Data-similarity and data-influence produce significantly overlapping rankings of training documents for LLM outputs, with asymmetry allowing a favorable cost-accuracy trade-off.
-
Unlocking the Potential of Continual Model Merging: An ODE Perspective
Introduces ODE-M, an ODE-based merging method for continual model merging that follows low-loss connecting paths to mitigate catastrophic forgetting.
-
Unlocking the Potential of Continual Model Merging: An ODE Perspective
ODE-M traces low-loss connecting paths via time-dependent velocity fields and barrier constraints to improve controllability and reduce forgetting in continual model merging.
-
Unlocking the Potential of Continual Model Merging: An ODE Perspective
ODE-M formulates continual model merging as a barrier-aware ODE trajectory in parameter space, using first-order feedback and a utility-aware schedule to balance retained knowledge and new task performance.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.