COALA applies convex optimization reformulations of neural networks to direct preference optimization, claiming single-GPU training with ~18% of DPO's TFLOPs and competitive performance on multiple datasets and models up to 8B parameters.
Convergence Analysis and Design of Multi-block ADMM via Switched Control Theory
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
We consider three challenges in multi-block Alternating Direction Method of Multipliers (ADMM): building convergence conditions for ADMM with any block (variable) sequence, finding available block sequences to be fit for ADMM, and designing useful parameter controllers for ADMM with unfixed parameters. To address these challenges, we develop a switched control framework for studying multi-block ADMM. First, since ADMM recursively and alternately updates the block-variables, it is converted into a discrete-time switched dynamical system. Second, we study exponential stability and stabilizability of the switched system for linear convergence analysis and design of ADMM by employing switched Lyapunov functions. Moreover, linear matrix inequalities conditions are proposed to ensure convergence of ADMM under arbitrary sequence, to find convergent sequences, and to design the fixed parameters. These conditions are checked and solved by employing semidefinite programming. Numerical experiments further verify the effectiveness of our proposed theories.
fields
cs.LG 1years
2026 1verdicts
UNVERDICTED 1representative citing papers
citing papers explorer
-
Convex Optimization for Alignment and Preference Learning on a Single GPU
COALA applies convex optimization reformulations of neural networks to direct preference optimization, claiming single-GPU training with ~18% of DPO's TFLOPs and competitive performance on multiple datasets and models up to 8B parameters.