Under linear-model assumptions, offline RLHF and DPO both reduce to logistic regression, and privatizing labels before corruption (LTC) carries an extra c(ε) factor in the error bounds compared to corrupting before privatizing (CTL).
Thus, by Lemma H.2, we have with probability at least 1 − δ, 1 n nX i=1 ηixi 2 ≤ C · σ · r 1 + ln(1/δ) n , for some universal constant C >0
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
citation-role summary
method 1
citation-polarity summary
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1roles
method 1polarities
use method 1representative citing papers
citing papers explorer
-
A Unified Theoretical Analysis of Private and Robust Offline Alignment: from RLHF to DPO
Under linear-model assumptions, offline RLHF and DPO both reduce to logistic regression, and privatizing labels before corruption (LTC) carries an extra c(ε) factor in the error bounds compared to corrupting before privatizing (CTL).