Pre-training provides a geometric warm start in a single-index model that enables weak-to-strong generalization up to a supervisor-limited bound, with empirical phase-transition evidence in LLMs.
arXiv preprint arXiv:2512.21917 , year=
2 Pith papers cite this work. Polarity classification is still indexing.
2
Pith papers citing it
abstract
Policy alignment to preference data typically assumes a known link function between observed preferences and latent rewards (e.g., Bradley-Terry model / logistic link). Misspecification of this link can bias inferred rewards and misalign learned policies. We study policy alignment under an unknown and unrestricted link function. We formulate an $f$-divergence-constrained reward maximization problem and show that realizability in a policy class induces a semiparametric single-index binary choice model, where a scalar policy-induced index captures all dependence on demonstrations and the remaining preference distribution is unrestricted. Rather than impose identifiability of structural parameters of such a model and estimate them, as in econometrics, we develop methods that directly learn policies, with the reward function implicit, analyzing error to the optimal policy and allowing for unidentifiable and nonparametric indices. We prove link-agnostic convergence guarantees in terms of generic function complexity measures and validate the methods and theory empirically. Code is available at https://github.com/causalml/spo/.
fields
cs.LG 2years
2026 2verdicts
UNVERDICTED 2representative citing papers
Position paper mapping causal inference opportunities across the LLM development pipeline from pretraining to evaluation to address confounding and non-stationarity.
citing papers explorer
-
On the Blessing of Pre-training in Weak-to-Strong Generalization
Pre-training provides a geometric warm start in a single-index model that enables weak-to-strong generalization up to a supervisor-limited bound, with empirical phase-transition evidence in LLMs.
-
Causal methods for LLM development and evaluation
Position paper mapping causal inference opportunities across the LLM development pipeline from pretraining to evaluation to address confounding and non-stationarity.