Pith. sign in

REVIEW 4 major objections 4 minor 2 cited by

On Weak-to-Strong Generalization and f-Divergence

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read f-divergence losses inherit cross-entropy's ceiling in weak-to-strong generalization — yet several generalize better and resist noisy labels in practice.

desk verdict Useful empirical comparison of f-divergence losses for weak-to-strong generalization, but the main limitation theorem is not valid as stated without an extra margin condition. read the letter →

arxiv 2506.03109 v1 pith:OBG4A5KR submitted 2025-06-03 cs.LG

Wei Yao , Gengze Xu , Huayi Tang , Wenkai Yang , Donglin Di , Ziqiao Wang , Yong Liu This is my paper · ORCID
classification cs.LG
keywords weak-to-stronggeneralizationf-divergencelossfunctiondesignsuperalignmentlabelnoiseKLdivergencesamplecomplexityinformation-theoreticbounds
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Superalignment wants strong models trained on weak labels to surpass their teachers, and this paper asks whether the loss function helps. It introduces f-divergence, a family of distribution-distance measures that includes KL, reverse KL, Jensen-Shannon, Jeffreys, Hellinger, Pearson chi-squared, and total variation, as a unified loss framework for weak-to-strong generalization. The paper proves that every f-divergence loss obeys the same ceiling already known for cross-entropy: the strong model's distance from the true labeling function is bounded by the weak teacher's quality plus the square root of the student-teacher disagreement, with finite-sample and information-theoretic refinements. It also proves that, under confidence-enhancing regularization, different f-divergence losses are equivalent up to a transformation of the regularizer. Empirically, several f-divergence losses generalize better than cross-entropy and tolerate label noise better, with Hellinger distance strongest at moderate noise and Pearson chi-squared and JS at 50% noise; the paper itself notes its finite-sample bounds are asymptotic and its experiments cover classification-style tasks.

What carries the argument

The carrying object is the f-divergence $D_f(P\|Q)=\mathbb{E}_Q[f(dP/dQ)]$, a distance between probability distributions built from a convex function $f$ with $f(1)=0$; it specializes to KL, reverse KL, JS, Jeffreys, Hellinger, chi-squared, and TV. Model disagreement is the expected f-divergence between outputs, $R_f(g,h)=\mathbb{E}_{x\sim P}[D_f(g(x)\|h(x))]$, and the paper uses it directly as a loss. Theorem 4.1's proof is carried by the TV-versus-f-divergence inequality $D_{TV}=O(\sqrt{D_f})$, the mean value theorem applied to $f$, and Jensen's inequality; Theorem 4.2 is carried by a mutual-information bound of order $\sqrt{I(\theta;Z)/n}$; Theorem 4.3 is carried by a Gateaux differential (a directional derivative in function space) combined with Lagrange multipliers and normalization functions, showing that the minimizer for one f-divergence loss equals the minimizer for another with a transformed regularizer.

What would settle it

Run a two-class weak-to-strong experiment with a known weak teacher and track $|R_f(G_{sw},G_\star)-R_f(G_w,G_\star)|$ against $\sqrt{R_f(G_{sw},G_w)}$ across training checkpoints; for KL divergence, drive softmax outputs toward 0 and 1 so that $D_{KL}$ and $f'(x)=\ln x+1$ grow without bound. If the ratio diverges as confidence saturates, the boundedness assumption fails for the very losses the paper recommends.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central claim is Theorem 4.1: for any strong model $G_{sw}$, weak teacher $G_w$, and ground-truth labeling function $G_\star$, $|R_f(G_{sw},G_\star)-R_f(G_w,G_\star)|=O(\sqrt{R_f(G_{sw},G_w)})$. In words, the strong student's excess error over the weak teacher is controlled by the square root of how much the student and teacher disagree, so pushing the W2SG training error to zero pins the strong model to the weak model. The paper extends this ceiling to empirical risk minimization with sample-complexity terms of order $(C_{F_s}/n)^{1/4}$ and $(\log(1/\delta)/n)^{1/4}$, and to a stochastic version involving the mutual information $I(\theta;Z)$ between learned parameters and training data. A further equivalence theorem says that minimizing any one f-divergence loss with an auxiliary confidence-enhancing regularizer equals minimizing any other f-divergence loss with an appropriately transformed regularizer. Empirically, the paper claims several f-divergence losses improve generalization and label-noise tolerance over cross-entropy without additional weak models or complex procedures.

Load-bearing premise

The main ceiling theorem is proved under an unstated assumption: the divergence $D_f$ and its derivative $f'$ stay finite for every possible model output, which is false for KL divergence when a softmax probability approaches zero.

Editorial extensions

If this is right

  • With infinite data and unconstrained optimization, a strong student trained with any f-divergence loss converges to the weak teacher; maintaining a controlled disagreement (e.g., early stopping) is therefore a theoretical requirement, not just an implementation detail.
  • The finite-sample corollary quantifies this ceiling: excess risk is at most $\sqrt{\hat R_f}$ plus terms of order $(C_{F_s}/n)^{1/4}$ and $(\log(1/\delta)/n)^{1/4}$, so the bound remains meaningful when data are limited.
  • The equivalence theorem says confidence-enhancing regularization transfers across divergence choices, so the practical choice of loss matters through the transformed regularizer and its hyperparameter rather than through a fundamentally different objective.
  • Empirically, reverse KL and Jeffreys divergences outperform cross-entropy in clean W2SG; Hellinger distance performs best at 10%–40% label noise, sometimes by up to 3 percentage points; Pearson chi-squared and JS give consistent gains at 50% noise.
  • Because the f-divergence loss is a drop-in replacement for cross-entropy, it delivers these gains without extra weak models, iterative pipelines, or added computational overhead.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The theorem's proof requires $D_f$ and $f'$ to be bounded on the simplex of possible softmax outputs; for KL and reverse KL these quantities diverge near zero probabilities, so the strict $O(\sqrt{R_f})$ guarantee may hold only under output clipping or smoothing, which would be worth testing explicitly.
  • The equivalence result suggests that observed differences between losses in practice may come from implicit regularization (boundedness, mode-seeking behavior) rather than from the divergence itself, which would explain Hellinger's noise tolerance.
  • One testable extension: monitor $\sqrt{R_f(G_{sw},G_w)}$ during training as an online stopping signal, since the theory predicts the strong model's gain over the weak teacher is largest when this quantity is moderate rather than zero.
  • The framework is developed for classification and reward modeling; transferring it to generative token-level objectives would test whether the same ceiling and noise tolerance hold when the output space is a sequence of simplexes.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes f-divergence as a loss-function family for weak-to-strong generalization (W2SG). It states a fundamental limitation bound for any f-divergence loss, extends the bound to empirical-risk and information-theoretic settings, claims an equivalence among f-divergence losses under auxiliary confidence regularization, and reports GPT-2 reward-modeling experiments comparing six f-divergences with cross-entropy under label noise.

Significance. The question is timely, and the f-divergence unification is a natural contribution that could be useful to the W2SG community. The empirical study is also valuable: it systematically compares a range of divergences on two datasets and several weak-to-strong pairs, and it extends earlier observations about reverse KL to a broader loss family. If the theoretical statements can be repaired, the paper would provide a coherent explanation of why the strong model's quality is tied to the weak supervisor and the disagreement between them. As written, however, the main theorems overreach: the central bound is not proved for the very losses the paper recommends, and the regularization-equivalence result is essentially definitional.

major comments (4)
  1. [§4.1, Theorem 4.1 and Appendix B.1] Theorem 4.1 is stated under only the softmax condition 0 < y_i < 1, but its proof requires sup_ξ |f'(ξ)| to be bounded and asserts that 'D_f is also bounded.' For KL (f(x)=x ln x), reverse KL, squared Hellinger, Jeffreys, and Pearson χ², f' is unbounded on the feasible ratio set, so the proof step in Appendix B.1 does not go through. A concrete binary example with G*=(e^{-1/ε²}, 1-e^{-1/ε²}), Gw=(ε, 1-ε), and Gsw=(ε+ε³, 1-ε-ε³) gives D_KL(Gsw,Gw)≈ε⁵/2 while |D_KL(Gsw,G*)-D_KL(Gw,G*)|≈ε, so the ratio to √D_KL diverges as ε^{-3/2}; this contradicts Eq. (6) for KL under the stated assumptions. Corollary 4.1 and Theorem 4.2 inherit the gap through Lemma B.2. Please either add an explicit margin condition (e.g., all output components of G*, Gw, and Gsw are at least η>0) and prove the results under it, or restrict the theorems to divergences whose derivative is bounded on the relevant ratio set.
  2. [§4.2, Theorem 4.3 and Appendix B.4] The equivalence in Theorem 4.3 is essentially definitional: the transformation v_i in Eq. (24) is defined using the normalization functions N_{Q,i}(α) and N'_{Q,i}(α) of the f1 and f2 problems, so the minimizers coincide by construction. The theorem therefore does not show that different f-divergence losses are equivalent under a fixed confidence-enhancing regularization; it only shows that each problem can be reparameterized by a problem-specific transformation. This is substantially weaker than the abstract's 'equivalence' claim and should be reframed, or replaced by a theorem with an explicit transformation that does not depend on the unknown solution of the f1 problem.
  3. [Appendix B.3, proof of Theorem B.1] The subgaussian parameter is computed incorrectly. An average of n i.i.d. variables that are each σ-subgaussian is σ/√n-subgaussian, not σ√n-subgaussian; with R=σ√n in Lemma A.3, inequality (18) would scale as O(√(n I(θ;Z))), not O(√(I(θ;Z)/n)). The final O((I/n)^{1/4}) in Theorem 4.2 can be obtained with the correct scaling R=σ/√n, but the proof as written needs to be corrected.
  4. [§5, Figures 1–2 and Table 2] The experimental comparisons are reported without error bars, standard deviations, or the number of seeds. Claims such as 'Hellinger distance sometimes outperforms the runner-up loss by as much as 3%' and 'χ² and JS outperform CE by at least 1% at 50% noise' cannot be distinguished from run-to-run variation without repeated trials. Please add multiple-seed results with error bars and, where possible, significance testing, or soften the claims to qualitative trends.
minor comments (4)
  1. [§4.1, paragraph after Theorem 4.1] The sentence 'the strong model's generalization error remains fundamentally bounded by Rf(Gsw,Gw) − O(√Rf(Gsw,Gw))' does not directly follow from Eq. (6), which bounds the absolute difference |Rf(Gsw,G*) − Rf(Gw,G*)| by O(√Rf(Gsw,Gw)); please clarify the intended inequality and wording.
  2. [Appendix B.4, proof of Theorem 4.3] In Eqs. (19)–(20), the outer sums are indexed by j while the integrands are indexed by i; this appears to be a typo and should be unified.
  3. [§5.1, label noise paragraph] The label-noise procedure is described as inverting soft labels for 'a portion' of training samples, but the paper does not state how that portion is selected or whether the selection is random across seeds; please clarify.
  4. [Figure 1 caption] The caption should state explicitly that the KL curve is omitted because it overlaps with the CE curve; currently that information appears only in the main text.

Circularity Check

1 steps flagged · score 6.0 of 10

Theorem 4.3's 'equivalence of different f-divergence losses' reduces to the definition of the transformation v_i; the Theorem 4.1 limitation bound is independent.

  1. self definitional [Theorem 4.3, Appendix B.4, Eq. (24)]
    "Define the transformation function vi(x) = − 1 α f′2 ((f′1)−1 (−αx − nNQ,i(α))) − n α N′Q,i(α), for any i = 1, · · ·, n. (24) Substitute v(x) into ˚gi(θ) and we have ˚gi(θ) = (f′2)−1 ( f′2 ((f′1)−1 (−αLi(θ) − nNQ,i(α))) ) = (f′1)−1 (−αLi(θ) − nNQ,i(α)) = ˜gi(θ)."

    The claimed equivalence of the two minimization problems is enforced by the definition of vi. The transform is built from the normalization functions NQ,i and N′Q,i of the two problems, and is chosen so that the f2 stationary solution ˚gi coincides with the f1 stationary solution ˜gi by substitution. Hence the equality of minima is not an independently derived property of f-divergence losses; it is a definitional identity. The theorem would be true for any strictly convex differentiable pair with a suitably chosen transform, so the abstract's 'theoretically discover the equivalence' overstates an existence-by-construction statement. The Theorem 4.1 bound does not share this issue.

full rationale

The central limitation bound (Theorem 4.1, Eq. 6) is not circular: it is derived from Gilardoni's TV-vs-f-divergence inequality, the mean value theorem, and Jensen's inequality, with no fitted parameter or self-citational load. Its proof does, however, assert without proof that |f'(ξi)| and Df are bounded on the model simplex (Appendix B.1), which fails for KL, reverse KL, Hellinger, Jeffreys, and Pearson χ² under only 0<yi<1; Corollary 4.1 and Theorem 4.2 inherit this gap through Lemma B.2. That is a correctness/missing-assumption issue, not circularity. The circular component is Theorem 4.3: the transformation vi in Eq. (24) is defined from the normalization functions of the two f-divergence problems so that the minimizers coincide, making the 'equivalence of different f-divergence losses' true by construction. Self-citations to Yao et al. are contextual and not load-bearing. Overall, one of the two headline theoretical claims reduces to its own construction, while the other is independent; empirical claims are external benchmarks and not fitted predictions.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The main theorems rely on standard f-divergence inequalities and an unstated boundedness assumption on D_f and f'. The equivalence result in Theorem 4.3 depends on normalization functions and a transformation constructed per sample after solving the problem. No new physical entities are introduced.

free parameters (3)
  • transformation function v_i(x) = defined in Eq. (24) via N_Q,i and N'_Q,i
    Constructed after the optimal solution to force equivalence in Theorem 4.3; no independent specification is given.
  • regularization coefficient beta = 0.5
    Fixed hyperparameter from Burns et al. (2023); hand-chosen and affects all empirical results.
  • hardening threshold t = not reported
    Threshold for the hardened predictions in the auxiliary confidence loss; the value is never stated.
assumptions (5)
  • domain assumption D_f and f' are bounded on the model output simplex
    Used in the proof of Theorem 4.1 ('D_f is also bounded', '|f'(xi_i)| is bounded') without being stated in the theorem; fails for KL when probabilities approach zero.
  • standard math D_TV(P,Q) = O(sqrt(D_f(P||Q))) (Gilardoni 2010)
    Used to convert TV distance to f-divergence in the proof of Theorem 4.1 and Corollary 4.1.
  • standard math Rademacher complexity of the loss class scales as sqrt(C_Fs/n) and Lipschitz contraction applies
    Used in Lemma B.1 and Corollary 4.1 for uniform convergence of the ERM bound.
  • standard math Mutual-information generalization bounds (Xu-Raginsky 2017; Wang-Mao 2023) hold for the f-divergence loss
    Used in Theorem 4.2 to bound the stochastic generalization error.
  • ad hoc to paper Existence of normalization functions N_Q,i and N'_Q,i in Theorem 4.3
    The proof assumes the integral constraints can be solved per sample for the given f1, f2, alpha, and data; no conditions for existence are stated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of On Weak-to-Strong Generalization and f-Divergence." pith.science (2026). https://pith.science/paper/OBG4A5KR

@misc{pith2026250603109,
  author       = {Pith},
  title        = {Pith review of: On Weak-to-Strong Generalization and f-Divergence},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OBG4A5KR}},
  note         = {Machine review of arXiv:2506.03109}
}
abstract

Weak-to-strong generalization (W2SG) has emerged as a promising paradigm for stimulating the capabilities of strong pre-trained models by leveraging supervision from weaker supervisors. To improve the performance of the strong model, existing methods often require additional weak models or complex procedures, leading to substantial computational and memory overhead. Motivated by the effectiveness of $f$-divergence loss in various machine learning domains, we introduce $f$-divergence as an information-theoretic loss function framework in W2SG. Our theoretical analysis reveals fundamental limitations and equivalence of different $f$-divergence losses in W2SG, supported by sample complexity bounds and information-theoretic insights. We empirically demonstrate that $f$-divergence loss, which generalizes widely-used metrics like KL divergence, effectively improves generalization and noise tolerance of the strong model in practice.

Figures

Figures reproduced from arXiv: 2506.03109 by the authors.

Figure 1
Figure 1. Results of GPT-2-series across varying noise levels on CAI-Harmless and HH-RLHF. “GPT-2-Base to [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Performance comparison of six f-divergence variants with confidence-enhancing regularization on CAI-Harmless and HH-RLHF, using CE as baseline. controlled noise by inverting the soft labels of a portion of the training samples. Specifically, for each selected sample, the original soft label y is replaced with its complement 1 − y. We compare model performance under five noise levels (0.1, 0.2, 0.3, 0.4, and 0.5) wit… view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Weak-to-Strong Generalization via Direct On-Policy Distillation

    cs.LG 2026-07 conditional novelty 7.0 of 10

    Transferring the log-ratio of a small model's pre-RL and post-RL checkpoints provides a dense implicit reward that improves stronger student models at a fraction of the cost of direct RL.

  2. Weak-to-Strong On-Policy Distillation

    cs.LG 2026-07 conditional novelty 6.0 of 10

    A strong LLM is improved by distilling from the logit difference of two weaker models instead of from a stronger teacher.

Reference graph

Works this paper leans on

68 extracted references · 43 canonical work pages · cited by 2 Pith papers

  1. [1]

    Aakriti Agrawal, Mucong Ding, Zora Che, Chenghao Deng, Anirudh Satheesh, John Langford, and Furong Huang. 2024. Ensemw2s: Can an ensemble of llms be leveraged to obtain a stronger llm? arXiv preprint arXiv:2410.04571

    arXiv 2024

  2. [2]

    Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. 2022 a . Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862

    arXiv 2022

  3. [3]

    Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. 2022 b . Constitutional ai: Harmlessness from ai feedback. arXiv preprint arXiv:2212.08073

    arXiv 2022

  4. [4]

    Peter L Bartlett and Shahar Mendelson. 2002. Rademacher and gaussian complexities: Risk bounds and structural results. Journal of Machine Learning Research, 3:463--482

    work page 2002

  5. [5]

    Yuheng Bu, Shaofeng Zou, and Venugopal V Veeravalli. 2020. Tightening mutual information-based bounds on generalization error. IEEE Journal on Selected Areas in Information Theory, 1(1):121--130

    work page 2020

  6. [6]

    Collin Burns, Pavel Izmailov, Jan Hendrik Kirchner, Bowen Baker, Leo Gao, Leopold Aschenbrenner, Yining Chen, Adrien Ecoffet, Manas Joglekar, Jan Leike, et al. 2023. Weak-to-strong generalization: Eliciting strong capabilities with weak supervision. arXiv preprint arXiv:2312.09390

    arXiv 2023

  7. [7]

    Moses Charikar, Chirag Pabbaraju, and Kirankumar Shiragur. 2024. Quantifying the gain in weak-to-strong generalization. Advances in neural information processing systems

    work page 2024

  8. [8]

    Ziyun Cui, Ziyang Zhang, Guangzhi Sun, Wen Wu, and Chao Zhang. 2025. Bayesian weaks-to-strong from text classification to generation. In The Thirteenth International Conference on Learning Representations

    work page 2025

Show all 68 references
  1. [9]

    Francisco Daunas, I \ n aki Esnaola, Samir M Perlaza, and Gholamali Aminian. 2025. Generalization error of f -divergence stabilized algorithms via duality. arXiv preprint arXiv:2502.14544

    2025 arXiv

  2. [10]

    Francisco Daunas, I \ n aki Esnaola, Samir M Perlaza, and H Vincent Poor. 2024. Equivalence of empirical risk minimization to regularization on the family of f-divergences. In 2024 IEEE International Symposium on Information Theory, pages 759--764. IEEE

    2024

  3. [11]

    Seyed Kamyar Seyed Ghasemipour, Richard Zemel, and Shixiang Gu. 2020. A divergence minimization perspective on imitation learning methods. In Conference on robot learning, pages 1259--1277. PMLR

    2020

  4. [12]

    Gustavo L Gilardoni. 2010. On pinsker's and vajda's type inequalities for csisz \'a r's f -divergences. IEEE Transactions on Information Theory, 56(11):5377--5386

    2010

  5. [13]

    Dongyoung Go, Tomasz Korbak, Germ\` a n Kruszewski, Jos Rozen, Nahyeon Ryu, and Marc Dymetman. 2023. Aligning language models with preferences through f-divergence minimization. In Proceedings of the 40th International Conference on Machine Learning, pages 11546--11583

    2023

  6. [14]

    Yuxian Gu, Li Dong, Furu Wei, and Minlie Huang. 2024. Minillm: Knowledge distillation of large language models. In International Conference on Learning Representations

    2024

  7. [15]

    Jianyuan Guo, Hanting Chen, Chengcheng Wang, Kai Han, Chang Xu, and Yunhe Wang. 2024. Vision superalignment: Weak-to-strong generalization for vision foundation models. arXiv preprint arXiv:2402.03749

    2024 arXiv

  8. [16]

    Muhammed Emrullah Ildiz, Halil Alperen Gozeten, Ege Onur Taga, Marco Mondelli, and Samet Oymak. 2025. High-dimensional analysis of knowledge distillation: Weak-to-strong generalization and scaling laws. In The Thirteenth International Conference on Learning Representations

    2025

  9. [17]

    Liyiming Ke, Sanjiban Choudhury, Matt Barnes, Wen Sun, Gilwoo Lee, and Siddhartha Srinivasa. 2021. Imitation learning as f-divergence minimization. In Algorithmic Foundations of Robotics XIV: Proceedings of the Fourteenth Workshop on the Algorithmic Foundations of Robotics 14,...

    2021

  10. [18]

    HyunJin Kim, Xiaoyuan Yi, Jing Yao, Muhua Huang, JinYeong Bak, James Evans, and Xing Xie. 2025. Research on superalignment should advance now with parallel optimization of competence and conformity. arXiv preprint arXiv:2503.07660

    2025

  11. [19]

    HyunJin Kim, Xiaoyuan Yi, Jing Yao, Jianxun Lian, Muhua Huang, Shitong Duan, JinYeong Bak, and Xing Xie. 2024. The road to artificial superintelligence: A comprehensive survey of superalignment. arXiv preprint arXiv:2412.16468

    2024 arXiv

  12. [20]

    Hao Lang, Fei Huang, and Yongbin Li. 2025. Debate helps weak-to-strong generalization. arXiv preprint arXiv:2501.13124

    2025 arXiv

  13. [21]

    Hunter Lang, David Sontag, and Aravindan Vijayaraghavan. 2024. Theoretical analysis of weak-to-strong generalization. In The Thirty-eighth Annual Conference on Neural Information Processing Systems

    2024

  14. [22]

    Michel Ledoux and Michel Talagrand. 2013. Probability in Banach Spaces: isoperimetry and processes. Springer Science & Business Media

    2013

  15. [23]

    Cheuk Ting Li and Farzan Farnia. 2023. Mode-seeking divergences: theory and applications to gans. In International Conference on Artificial Intelligence and Statistics, pages 8321--8350. PMLR

    2023

  16. [24]

    Cheuk Ting Li, Jingwei Zhang, and Farzan Farnia. 2024. On convergence in wasserstein distance and f-divergence minimization problems. In International Conference on Artificial Intelligence and Statistics, pages 2062--2070. PMLR

    2024

  17. [25]

    Friedrich Liese and Igor Vajda. 2006. On divergences and informations in statistics and information theory. IEEE Transactions on Information Theory, 52(10):4394--4412

    2006

  18. [26]

    Yuejiang Liu and Alexandre Alahi. 2024. Co-supervised learning: Improving weak-to-strong generalization with hierarchical mixture of experts. arXiv preprint arXiv:2402.15505

    2024 arXiv

  19. [27]

    Yougang Lyu, Lingyong Yan, Zihan Wang, Dawei Yin, Pengjie Ren, Maarten de Rijke, and Zhaochun Ren. 2025. Macpo: Weak-to-strong alignment via multi-agent contrastive preference optimization. In The Thirteenth International Conference on Learning Representations

    2025

  20. [28]

    Marko Medvedev, Kaifeng Lyu, Dingli Yu, Sanjeev Arora, Zhiyuan Li, and Nathan Srebro. 2025. Weak-to-strong generalization even in random feature networks, provably. arXiv preprint arXiv:2503.02877

    2025

  21. [29]

    Abhijeet Mulgund and Chirag Pabbaraju. 2025. Relating misfit to gain in weak-to-strong generalization beyond the squared loss. arXiv preprint arXiv:2501.19105

    2025 arXiv

  22. [30]

    Nicola Novello and Andrea M Tonello. 2024. f-divergence based classification: Beyond the use of cross-entropy. In Forty-first International Conference on Machine Learning

    2024

  23. [31]

    Sebastian Nowozin, Botond Cseke, and Ryota Tomioka. 2016. f-gan: Training generative neural samplers using variational divergence minimization. Advances in neural information processing systems, 29

    2016

  24. [32]

    OpenAI. 2023. https://openai.com/index/introducing-superalignment/ Introducing superalignment

    2023

  25. [33]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 3...

    2022

  26. [34]

    Martin Pawelczyk, Lillian Sun, Zhenting Qi, Aounon Kumar, and Himabindu Lakkaraju. 2024. Generalizing trust: Weak-to-strong trustworthiness in language models. arXiv preprint arXiv:2501.00418

    2024 arXiv

  27. [35]

    Yury Polyanskiy and Yihong Wu. 2025. Information theory: From coding to learning. Cambridge university press

    2025

  28. [36]

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9

    2019

  29. [37]

    Alfr \'e d R \'e nyi. 1961. On measures of entropy and information. In Proceedings of the fourth Berkeley symposium on mathematical statistics and probability, volume 1: contributions to the theory of statistics, volume 4, pages 547--562. University of California Press

    1961

  30. [38]

    Daniel Russo and James Zou. 2016. Controlling bias in adaptive data analysis using information theory. In Artificial Intelligence and Statistics, pages 1232--1240

    2016

  31. [39]

    Jitao Sang, Yuhang Wang, Jing Zhang, Yanxu Zhu, Chao Kong, Junhong Ye, Shuyu Wei, and Jinlin Xiao. 2024. Improving weak-to-strong generalization with scalable oversight and ensemble learning. arXiv preprint arXiv:2402.00667

    2024 arXiv

  32. [40]

    Igal Sason and Sergio Verd \'u . 2016. f-divergence inequalities. IEEE Transactions on Information Theory, 62(11):5973--6006

    2016

  33. [41]

    Changho Shin, John Cooper, and Frederic Sala. 2024. Weak-to-strong generalization through the data-centric lens. arXiv preprint arXiv:2412.03881

    2024 arXiv

  34. [42]

    Ravid Shwartz-Ziv and Naftali Tishby. 2017. Opening the black box of deep neural networks via information. arXiv preprint arXiv:1703.00810

    2017 arXiv

  35. [43]

    Seamus Somerstep, Felipe Maia Polo, Moulinath Banerjee, Ya'acov Ritov, Mikhail Yurochkin, and Yuekai Sun. 2024. A statistical framework for weak-to-strong generalization. arXiv preprint arXiv:2405.16236

    2024 arXiv

  36. [44]

    Seamus Somerstep, Felipe Maia Polo, Moulinath Banerjee, Yaacov Ritov, Mikhail Yurochkin, and Yuekai Sun. 2025. A transfer learning framework for weak to strong generalization. In The Thirteenth International Conference on Learning Representations

    2025

  37. [45]

    Huayi Tang and Yong Liu. 2024. Information-theoretic generalization bounds for transductive learning and its applications. Journal of Machine Learning Research, 25(407):1--69

    2024

  38. [46]

    Naftali Tishby and Noga Zaslavsky. 2015. Deep learning and the information bottleneck principle. In 2015 ieee information theory workshop (itw), pages 1--5. Ieee

    2015

  39. [47]

    Vladimir N Vapnik. 1999. An overview of statistical learning theory. IEEE transactions on neural networks, 10(5):988--999

    1999

  40. [48]

    Neng Wan, Dapeng Li, and Naira Hovakimyan. 2020. F-divergence variational inference. Advances in neural information processing systems, 33:17370--17379

    2020

  41. [49]

    Chaoqi Wang, Yibo Jiang, Chenghao Yang, Han Liu, and Yuxin Chen. 2024. Beyond reverse kl: Generalizing direct preference optimization with diverse divergence constraints. In International Conference on Learning Representations

    2024

  42. [50]

    Ziqiao Wang and Yongyi Mao. 2023. Information-theoretic analysis of unsupervised domain adaptation. In International Conference on Learning Representations

    2023

  43. [51]

    Ziqiao Wang and Yongyi Mao. 2024. On f-divergence principled domain adaptation: An improved framework. In The Thirty-eighth Annual Conference on Neural Information Processing Systems

    2024

  44. [52]

    Jiaheng Wei and Yang Liu. 2021. When optimizing f-divergence is robust with label noise. In International Conference on Learning Representations

    2021

  45. [53]

    Yuqiao Wen, Zichao Li, Wenyu Du, and Lili Mou. 2023. f-divergence minimization for sequence-level knowledge distillation. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10817--10834

    2023

  46. [54]

    David X Wu and Anant Sahai. 2025. Provable weak-to-strong generalization via benign overfitting. In The Thirteenth International Conference on Learning Representations

    2025

  47. [55]

    Aolin Xu and Maxim Raginsky. 2017. Information-theoretic analysis of generalization capability of learning algorithms. Advances in neural information processing systems, 30

    2017

  48. [56]

    Gengze Xu, Wei Yao, Ziqiao Wang, and Yong Liu. 2025 a . On the emergence of weak-to-strong generalization: A bias-variance perspective. arXiv preprint arXiv:2505.24313

    2025

  49. [57]

    Minkai Xu, Jiaqi Han, Mingjian Jiang, Yuxuan Song, and Stefano Ermon. 2025 b . f-po: Generalizing preference optimization with f-divergence minimization. In The 28th International Conference on Artificial Intelligence and Statistics

    2025

  50. [58]

    Yihao Xue, Jiping Li, and Baharan Mirzasoleiman. 2025. Representations shape weak-to-strong generalization: Theoretical insights and empirical predictions. arXiv preprint arXiv:2502.00620

    2025 arXiv

  51. [59]

    Wenkai Yang, Shiqi Shen, Guangyao Shen, Wei Yao, Yong Liu, Zhi Gong, Yankai Lin, and Ji-Rong Wen. 2025. Super (ficial)-alignment: Strong models may deceive weak models in weak-to-strong generalization. In The Thirteenth International Conference on Learning Representations

    2025

  52. [60]

    Yuqing Yang, Yan Ma, and Pengfei Liu. 2024. Weak-to-strong reasoning. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 8350--8367

    2024

  53. [61]

    Wei Yao, Wenkai Yang, Ziqiao Wang, Yankai Lin, and Yong Liu. 2025 a . Revisiting weak-to-strong generalization in theory and practice: Reverse kl vs. forward kl. arXiv preprint arXiv:2502.11107

    2025 arXiv

  54. [62]

    Wei Yao, Wenkai Yang, Ziqiao Wang, Yankai Lin, and Yong Liu. 2025 b . Understanding the capabilities and limitations of weak-to-strong generalization. arXiv preprint arXiv:2502.01458

    2025 arXiv

  55. [63]

    Yaowen Ye, Cassidy Laidlaw, and Jacob Steinhardt. 2025. Iterative label refinement matters more than preference optimization under weak supervision. In The Thirteenth International Conference on Learning Representations

    2025

  56. [64]

    Lantao Yu, Yang Song, Jiaming Song, and Stefano Ermon. 2020. Training deep energy-based models with f-divergence minimization. In International Conference on Machine Learning, pages 10957--10967. PMLR

    2020

  57. [65]

    Meiyu Zhong and Ravi Tandon. 2023. Learning fair classifiers via min-max f-divergence regularization. In 2023 59th Annual Allerton Conference on Communication, Control, and Computing (Allerton), pages 1--8. IEEE

    2023

  58. [66]

    Yucheng Zhou, Jianbing Shen, and Yu Cheng. 2025. Weak to strong generalization for large language models with multi-capabilities. In The Thirteenth International Conference on Learning Representations

    2025

  59. [67]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  60. [68]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.