NLHF achieves the minimax-optimal worst-case average-utility distortion (1/2+o(1))β, while RLHF and DPO can suffer distortion up to e^{Ω(β)} or unbounded under certain comparison sampling.
On the identifiability of mixtures of ranking models
1 Pith paper cite this work. Polarity classification is still indexing.
abstract
Mixtures of ranking models are standard tools for ranking problems. However, even the fundamental question of parameter identifiability is not fully understood: the identifiability of a mixture model with two Bradley-Terry-Luce (BTL) components has remained open. In this work, we show that popular mixtures of ranking models with two components (BTL, multinomial logistic models with slates of size 3, or Plackett-Luce) are generically identifiable, i.e., the ground-truth parameters can be identified except when they are from a pathological subset of measure zero. We provide a framework for verifying the number of solutions in a general family of polynomial systems using algebraic geometry, and apply it to these mixtures of ranking models to establish generic identifiability. The framework can be applied more broadly to other learning models and may be of independent interest.
fields
cs.LG 1years
2025 1verdicts
ACCEPT 1representative citing papers
citing papers explorer
-
Distortion of AI Alignment: Does Preference Optimization Optimize for Preferences?
NLHF achieves the minimax-optimal worst-case average-utility distortion (1/2+o(1))β, while RLHF and DPO can suffer distortion up to e^{Ω(β)} or unbounded under certain comparison sampling.