{"id":"78e7072b-fb9e-4949-b73a-00bc2b9e02b2","arxiv_id":"2501.19105","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"For convex and approximately convex model classes, the loss gain in weak-to-strong learning is at least the KL misfit between strong and weak models, plus an error term that vanishes as k grows.","lead":"This paper extends the theory of weak-to-strong generalization from squared-loss regression to general Bregman divergences, covering classification losses like cross-entropy. It proves the loss reduction of a strong model trained on weak labels is at least its KL misfit to the weak model, up to an error that shrinks as the number of model components grows.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 4.3's proof in Appendix A.2.2 contains an unjustified pointwise-vs-expectation bound; the O(sqrt(c/k)) error term is not established as stated until the proof is repaired or the regularization assumption is strengthened.","rationale":"The paper has two credible concerns: (1) the proof gap in Appendix A.2.2 and (2) the violation of the realizability assumption on real datasets, which the authors themselves document in Section 5.5. The proof gap is the more immediate load-bearing issue because it affects whether Theorem 4.3 is proven even under its stated assumptions. The reader's rationale flagged the same proof step, but the reader's weakest_assumption field instead emphasizes realizability; hence partial agreement. The theorem is likely repairable: in the finite-support setting, Assumption 2 plus finiteness of the support implies a pointwise uniform bound on M(x), so a corrected proof can recover O(sqrt(c/k)) after adding a 1/min_x P_X(x) factor to the constant. The realizability limitation is real, honestly disclosed, and empirically demonstrated, but it is a scope restriction rather than an internal inconsistency. The convex results (Theorem 4.1, Corollary 4.2) are a straightforward Bregman-geometric extension and appear sound. The reverse-KL training recipe is novel and the experiments are consistent with the qualitative misfit-gain trend. A conditional accept is therefore appropriate: the authors should fix the proof step, state the strengthened regularization or finite-support repair explicitly, and clarify the constant's dependence on the support size and the number of classes.","tokens_in":23882,"tokens_out":18218,"duration_ms":193261,"concrete_test":"Independently re-derive the key estimate in the proof of Theorem A.2 by replacing the displayed step with the pointwise bound 1/min_i(fs_i(x), gproj_i(x)) <= M(x) := max_i sup_{f in F} 1/f_i(x), then applying Jensen's inequality to move the square root outside the expectation. Check whether the resulting error term is bounded, under Assumption 2 alone, by a constant times sqrt(c/k), or whether it requires the extra finite-support factor 1/min_x P_X(x) (equivalently, requires E[M(X)^2] < infinity in the Appendix A.3 setting). If the corrected derivation needs a factor not implied by Assumption 2, the proof in the paper is incomplete as written and the regularization condition in Theorem 4.3 must be strengthened.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central non-convex result, Theorem 4.3 / Theorem A.2, rests on the proof step after Lemma A.7 in Appendix A.2.2. The paper bounds\n\n|EX[DKL(g||gproj)] - EX[DKL(g||fs)]| <= EX[ sqrt(2 DKL(fs||gproj)) / min_i(fs_i, gproj_i) ]\n <= EX[ max_i rho_i * sqrt(c/(4k) max_i rho_i + epsilon) ],\n\nwith rho_i := EX[sup_{f in F} 1/f_i(X)]. This is not justified as written. First, the only available bound on DKL(fs||gproj) is an expectation bound, E[DKL(fs||gproj)] <= (1/(8k)) sum_i rho_i + epsilon; it cannot be inserted pointwise inside the expectation and under the square root. Second, 1/min_i(...) is a pointwise random variable; the stated assumption controls first moments of sup 1/f_i, not pointwise values of the inverse probability, and it certainly does not imply the constant bound max_i rho_i used in the display. A correct treatment would bound 1/min_i(...) pointwise by M(X) := max_i sup_{f in F} 1/f_i(X) and apply Jensen/Cauchy-Schwarz, yielding a factor sqrt(E[M(X)^2]) or, in the finite-support setting, a factor 1/min_x P_X(x) that is absent from the proof. In the infinite-support extension (Appendix A.3), Assumption 2 does not imply E[M(X)^2] < infinity, so the stated argument does not generalize. The theorem may be repairable by strengthening regularization to boundedness of M(X) or its second moment, but the advertised error guarantee is not proven by the current text.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper extends the misfit-gain characterization of weak-to-strong generalization from squared-loss regression to losses induced by Bregman divergences, with a focus on classification via the KL/cross-entropy loss. For a convex strong class, Theorem 4.1 establishes that a near-minimizer of the Bregman misfit to the weak model achieves a target loss no larger than the weak model's loss minus the misfit, up to an arbitrarily small error. Corollary 4.2 specializes this to cross-entropy. For non-convex strong classes, Theorem 4.3 instead considers convex combinations of k functions from the strong class and claims the same gain-misfit inequality up to an O(sqrt(c/k)) error. The paper reports synthetic and real-data experiments supporting the qualitative and, to a degree, quantitative relationship, while Section 5.5 acknowledges that the realizability assumption fails on real datasets and only recovers quantitative agreement by evaluating losses against the best strong model rather than the ground-truth target.","tokens_in":24155,"tokens_out":4934,"duration_ms":56254,"significance":"If all claims were fully established, this would be a valuable conceptual generalization of the squared-loss theory in Charikar et al. (2024): it identifies the Bregman Pythagorean inequality as the core mechanism, extends the characterization to cross-entropy, and provides a concrete algorithmic recipe for weak-to-strong training with non-convex strong classes. The multi-class formulation and the Jensen-gap machinery are natural and potentially reusable. However, the paper's headline non-convex result, Theorem 4.3, rests on a proof step in Appendix A.2.2 that does not follow from the stated assumptions, so the advertised O(sqrt(c/k)) guarantee is not currently proven. The authors also deserve credit for explicitly testing and reporting the failure of realizability on real data, and for separating that issue from the structural inequality rather than hiding it.","major_comments":[{"comment":"The statement of Theorem 4.1 assumes only that F is a convex set of functions, but the proof invokes Fact 3.4 to obtain a unique Bregman projection onto F ∘ h_s. That fact requires the projected set to be closed as well as convex. Without closedness, existence of the projection and the equality between inf and min used in the proof are not automatic. The authors later acknowledge that uniqueness of the exact minimizer requires closedness, but the theorem statement and proof should either add a closedness hypothesis or prove attainment under the stated assumptions.","section":"Section 4, Theorem 4.1 and Fact 3.4"},{"comment":"The central bound on |E[DKL(g||g_proj)] - E[DKL(g||f_s)]| is not justified. The proof uses the display E[ sqrt(2 DKL(f_s||g_proj)) / min_i(f_s,i, g_proj,i) ] <= E[ max_i rho_i * sqrt(c/(4k) max_i rho_i + eps) ]. This requires a pointwise upper bound on DKL(f_s||g_proj), but only the expectation E[DKL(f_s||g_proj)] is bounded, via the Jensen-gap argument. It also treats 1/min_i as bounded by max_i rho_i, whereas the stated regularization assumption controls only first moments of sup_{f in F} 1/f_i(X), not pointwise values. A correct argument would need a pointwise bound on M(X) := max_i sup_f 1/f_i(X), or a second-moment condition, and would then produce a factor such as E[M(X)^2]^{1/2} rather than the stated quantity. As written, the O(sqrt(c/k)) error term in Theorem 4.3 is not established.","section":"Appendix A.2.2, proof of Theorem A.2 after Lemma A.7"},{"comment":"The claimed generalization to infinite-support input distributions inherits the same defect. Assumption 2 in Theorem A.2 controls first moments of sup 1/f_i(X); the repaired proof described above would require E[M(X)^2] < infinity, which is not implied by the stated condition. Either the infinite-support claim should be made conditional on an additional second-moment or boundedness assumption, or a separate argument must be supplied.","section":"Appendix A.3"},{"comment":"The authors verify that realizability, Assumption 1 of Theorem 4.3, does not hold in their real-data experiments, and they recover quantitative agreement only by evaluating the weak and strong losses with respect to the best strong model rather than the ground-truth target. This should be stated more prominently as a limitation: Figures 2 and 3a-c do not test the quantitative inequality (3) for the actual target function, and the abstract's claim of experimental support should be qualified accordingly.","section":"Section 5.5 and Theorem 4.3, Assumption 1"}],"minor_comments":[{"comment":"The text says 'Theorem 4.2' when referring to the corollary being proved; the same mislabeling appears in the paragraph after Corollary 4.2 ('Theorem 4.2 still applies to coF').","section":"Section 4, proof sketch of Corollary 4.2"},{"comment":"The notation 'a := (supp∈S(1/pi))^c_{i=1}' should use 'sup' rather than 'supp', and the definition of a_i as the coordinatewise supremum should be written explicitly.","section":"Appendix A.2.2, Lemma A.5"},{"comment":"The Regularization assumption displays max( E[ sup_{f in F} 1/f(X) ], E[ sup_{f in F} 1/f(X)) ] ), where the second term should presumably be E[ sup 1/(1-f(X)) ] for binary classification; the parenthesis error makes the two displayed terms identical.","section":"Appendix A.2.2, Theorem A.2 statement"},{"comment":"The square-root symbol is rendered as a placeholder '...' in several places (e.g., O(... c/k), O(... c/k)); the intended expression O(sqrt(c/k)) should be typeset correctly.","section":"Throughout, equations (10), (11), and surrounding text"},{"comment":"The word 'optmial' should be 'optimal'.","section":"Appendix A.2.2, final paragraph"}],"recommendation":"major_revision","confidential_remarks":"The proof gap in Appendix A.2.2 is the main obstacle to acceptance. The paper's core convex result appears sound, and the non-convex result is likely repairable by strengthening the regularization assumption, but the current text overclaims the O(sqrt(c/k)) guarantee. The realizability caveat in Section 5.5 should also be reflected in the paper's framing of its empirical validation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my take on Mulgund and Pabbaraju. The paper is worth engaging, but don't take Theorem 4.3's error bound at face value yet.\n\nThe convex-case result (Theorem 4.1 / Corollary 4.2) is a clean, fairly direct generalization of Charikar et al. to Bregman divergences, using the standard Pythagorean inequality. That part is sound, modulo a missing closedness assumption for exact projection. The genuinely new piece is the non-convex treatment: instead of requiring a convex strong class, they project onto convex combinations of k elements and get a misfit-gain inequality with a vanishing error. The reverse-KL recipe (optimizing DKL(student||teacher) rather than the usual cross-entropy) is interesting, and the experiments show the qualitative relationship holds. I also give them credit for being honest about the realizability failure: Section 5.5 explicitly verifies that the real-data target is not in the strong class, and only recovers the inequality by re-evaluating against the best strong model instead of ground truth. That is a limitation, not a hidden flaw. The related work is also positioned fairly, and the Zeevi-Meir connection is apt.\n\nThe soft spot is real, though. The stress-test note is correct: in Appendix A.2.2, the proof of the O(sqrt(c/k)) term crosses from an expectation bound on DKL(fs||gproj) to a pointwise bound inside the expectation. That step is not justified, and the 1/min_i(...) factor is random and cannot be treated as the constant max_i rho_i. A repaired proof needs either pointwise control of the inverse probabilities (e.g., boundedness of max_i sup_f 1/f_i(X)) or a second-moment bound, and it will likely have a different dependence on c and the support size. So the main advertised guarantee is not proven as stated in the current version.\n\nThat said, the flaw is repairable and the empirical recipe shouldn't be discarded. The paper's central intuition—that gain is lower-bounded by KL misfit—is plausible, and the experiments support it qualitatively, even if they don't quantitatively validate the exact bound because of the realizability workaround.\n\nBottom line: I would send this to a serious referee, but with the clear instruction that the proof of Theorem 4.3 must be fixed or the theorem restated under stronger assumptions. The convex results alone justify the paper; the non-convex result, if repaired, would be a solid contribution.","headline":"A real extension of the misfit-gain framework to classification, with a genuinely new k-convex recipe—but the main theorem's proof has a gap that needs repair before the advertised O(sqrt(c/k)) bound is claimed.","tokens_in":24762,"tokens_out":2765,"would_cite":true,"duration_ms":26445,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68Q32","62B10"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper proves a general misfit–gain law for weak-to-strong learning: the student's gain over the weak teacher is at least their misfit for any Bregman-divergence loss, and for nonconvex classes this holds up to a vanishing error when…","keywords":["weak-to-strong generalization","Bregman divergence","cross-entropy","KL divergence","misfit-gain inequality","convex combinations","Jensen gap","classification"],"falsifier":"Direct test on the theorem: build a synthetic classification problem where realizability and the regularization condition hold, optimize the reversed-KL objective over $\\operatorname{co}_k F$ to near-global optimality, and estimate all three expectations in Inequality (3) on fresh data; a violation larger than the $O(\\sqrt{c/k})$ term would refute Theorem 4.3. A complementary test for the practical claim: on a real dataset where a strong model trained on true labels still has nontrivial loss, compute the same three terms for the true target; the authors report that the inequality fails in this setting, so any claimed bound for the true target must confront this failure.","tokens_in":23593,"feed_emoji":"📈","tokens_out":14033,"duration_ms":120374,"temperature":0.7,"pith_summary":"This paper extends a quantitative law of weak-to-strong generalization from squared-loss regression to any loss that is a Bregman divergence, which includes the cross-entropy used in classification. The law states that when the strong model class is convex and the target is realizable, the strong student's expected loss on the true target is at most the weak teacher's expected loss minus the divergence between student and teacher outputs: gain is at least misfit. Because practical strong classes are usually not convex, the paper proves the same inequality for a student that is a convex combination of $k$ strong functions, with an extra error term $O(\\sqrt{c/k})$ that vanishes as $k$ grows. It also proposes a concrete recipe, minimizing the reversed KL divergence from the strong output to the weak labels over $k$ logistic heads, and reports that loss gain tracks KL misfit on NLP and vision benchmarks. The paper's own experiments show the guarantee depends on the target being representable in the strong class, which fails on real data and is restored only by redefining the target as the best strong model.","feed_headline":"Strong models gain at least their disagreement with weak teachers","feed_subtitle":"A Bregman-divergence proof extends the misfit–gain law from squared-loss regression to classification","key_machinery":"The machinery is the Bregman divergence $D_\\psi$ generated by a strictly convex $C^1$ function $\\psi$, together with the generalized Pythagorean inequality for forward Bregman projections onto a closed convex set: for $z$ and any $x$ in the convex set $W$, $D_\\psi(x,z) \\geq D_\\psi(x,P_W(z)) + D_\\psi(P_W(z),z)$. Applied to the expected divergence $\\mathbb{E}[D_\\psi(\\cdot, f_w \\circ h_w)]$ over the convex class $F \\circ h_s$, this is what turns near-minimal misfit on weak labels into a lower bound on loss gain. For the nonconvex case, the load-bearing device is the Jensen approximation gap of the generator, here the negative Shannon entropy, which lets a $k$-fold convex combination approximate any point of the convex hull in Bregman cost with error $O(\\sum_i \\rho_i/k)$, plus a Pinsker-type log-ratio bound; combining these gives the $O(\\sqrt{c/k})$ slack in Theorem 4.3.","core_discovery":"The central discovery is a misfit-gain inequality that holds beyond squared loss. Theorem 4.1 shows that if the strong class $F$ is convex and the target $g$ lies in $F \\circ h_s$, then any student $f_s$ that nearly minimizes the expected Bregman divergence to the weak labels satisfies $\\mathbb{E}[D_\\psi(g(X), f_s(h_s(X)))] \\leq \\mathbb{E}[D_\\psi(g(X), f_w(h_w(X)))] - \\mathbb{E}[D_\\psi(f_s(h_s(X)), f_w(h_w(X)))] + \\epsilon$. Because binary cross-entropy is KL divergence plus an entropy constant, Corollary 4.2 converts this into a classification guarantee: training with the KL objective in the reverse direction provably reduces expected cross-entropy by at least the KL misfit. For nonconvex $F$, Theorem 4.3 proves the same inequality holds up to $O(\\sqrt{c/k})$ when the student is a convex combination of $k$ functions from $F$, under a mild regularization condition; the slack comes from a Jensen-gap bound on the negative Shannon entropy generator.","pith_inferences":["Editorial inference: since the paper's own Section 5.5 shows realizability fails on real data, the practically relevant statement is about the best achievable strong model rather than the ground-truth target; a version of Theorem 4.3 stated for unrealizable targets would be needed to certify gains on actual benchmarks.","Editorial inference: the reversed-KL recipe is a projection in the probability simplex rather than in logit space; testing whether optimizing in the dual logit space with the same $k$-combination changes how quickly gain approaches misfit would separate the geometry of the loss from the optimizer's behavior.","Editorial inference: the Jensen-gap argument is specialized to KL in the nonconvex theorem; extending Lemma A.4 to other Bregman generators, such as the Itakura-Saito divergence, would show whether the $O(\\sqrt{c/k})$ rate is a property of cross-entropy or an artifact of this proof technique.","Editorial inference: no accuracy-level theorem is claimed, but the experiments consistently show accuracy moving with loss; a testable extension is that the misfit–gain gap predicts accuracy improvement whenever the loss–accuracy curve is locally linear."],"forward_implications":["For any regression or classification loss that is a Bregman divergence, a convex strong class yields a provable gain of at least the student–teacher misfit; the earlier squared-loss result is the special case $\\psi(x)=x^2/2$.","For classification, minimizing the KL divergence with the strong output in the first argument and the weak labels in the second, over a convex combination of $k$ logistic heads, provably lowers cross-entropy on the true target once $k$ is large enough.","The quantities in the inequality are expectations that can be estimated from hold-out data, so the bound can be used to predict how much gain to expect from a given weak teacher before training the strong model.","No assumption is placed on the weak model; the requirements fall on the strong class, convexity or a mild regularization condition, and on realizability of the target.","Empirically, $k=100$ is enough for gain and misfit to track each other closely on NLP and vision benchmarks, and increasing $k$ shrinks the difference as the error term predicts."],"supporting_citations":[{"why":"Supplies the squared-loss misfit-gain theorem and the geometric setup that the paper generalizes to Bregman divergences.","marker":"[12]"},{"why":"Introduces Bregman divergences, the family of losses that replaces squared loss in the main theorems.","marker":"[8]"},{"why":"Gives the generalized Pythagorean inequality for forward Bregman projections that carries the convex-class proof of Theorem 4.1.","marker":"[17]"},{"why":"Supplies the probabilistic-method and Jensen-gap technique used to control approximation by convex combinations in Theorem 4.3.","marker":"[57]"},{"why":"Provides the weak-to-strong generalization paradigm and the NLP and vision datasets used in the experiments.","marker":"[9]"},{"why":"Defines the KL divergence used to connect cross-entropy to the Bregman machinery in Corollary 4.2.","marker":"[34]"}],"fun_headline_variants":["Misfit to gain: A Bregman law for weak-to-strong learning","Beyond squared loss: Misfit predicts strong model gain","Weak teachers, strong students: Misfit-gain law extended to classification","Bregman divergences unify weak-to-strong gain characterization","Disagreement predicts gain: Bregman proof for weak-to-strong"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is realizability: the true target function must be exactly representable by some function in the strong model class, and the paper's own experiments (Section 5.5) show this fails on real datasets; the misfit–gain law is restored only when the target is redefined as the best strong model.","fun_headline_variants_meta":{"raw":{"variants":["Misfit to gain: A Bregman law for weak-to-strong learning","Beyond squared loss: Misfit predicts strong model gain","Weak teachers, strong students: Misfit-gain law extended to classification","Bregman divergences unify weak-to-strong gain characterization","Disagreement predicts gain: Bregman proof for weak-to-strong"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000727,"raw_usage":{"total_tokens":3287,"prompt_tokens":1004,"completion_tokens":2283,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":620,"completion_tokens_details":{"reasoning_tokens":2186}},"tokens_in":620,"tokens_out":2283,"duration_ms":14961,"temperature":1.0,"reasoning_tokens":2186,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T21:17:37.195040+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Direct test on the theorem: build a synthetic classification problem where realizability and the regularization condition hold, optimize the reversed-KL objective over $\\operatorname{co}_k F$ to near-global optimality, and estimate all three expectations in Inequality (3) on fresh data; a violation larger than the $O(\\sqrt{c/k})$ term would refute Theorem 4.3. A complementary test for the practical claim: on a real dataset where a strong model trained on true labels still has nontrivial loss, compute the same three terms for the true target; the authors report that the inequality fails in this setting, so any claimed bound for the true target must confront this failure.","supporting_citations":[{"cited_title":"The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming","cited_arxiv_id":null,"evidence_quote":"Introduces Bregman divergences, the family of losses that replaces squared loss in the main theorems."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Gives the generalized Pythagorean inequality for forward Bregman projections that carries the convex-class proof of Theorem 4.1."},{"cited_title":"Zeevi and Ronny Meir","cited_arxiv_id":null,"evidence_quote":"Supplies the probabilistic-method and Jensen-gap technique used to control approximation by convex combinations in Theorem 4.3."}],"review_version":1}