{"id":"81ee775f-be13-4c36-89ba-c44e2a70e6ca","arxiv_id":"2501.07754","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"BOLT, a loss derived from an f-divergence bound on Bayes error, matches or slightly beats cross-entropy on MNIST, Fashion-MNIST, CIFAR-10, and IMDb.","lead":"This paper introduces a new loss function, BOLT, built from an upper bound on the Bayes error rate, and tests it on image and text classification tasks. The authors claim that minimizing this loss can push a classifier toward the minimum achievable error, and report small accuracy gains over cross-entropy on several benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Section 4 uses softmax probabilities h_{θ,i}∈[0,1] as the h_i of Theorem 2, whose domain is (-1,0]; the inequality ε_bys ≤ L_θ therefore is not established for the implemented BOLT loss, so the multi-class experiments do not verify the claimed Bayes-optimal training.","rationale":"I agree with the reader's verdict and with the identified weakest assumption. The paper's central claim is that minimizing L_θ drives a classifier to the Bayes error because L_θ is an upper bound on ε_bys. That implication requires the network output functions h_{θ,i} to belong to the class H of functions mapping X into (-1,0] used in Theorem 2. The paper instead defines h_{θ,i} as categorical probabilities in [0,1], normalized by (17). For such outputs, the variational f-divergence bound used to prove Theorem 1 cannot be applied, because the Fenchel conjugate of the hinge loss is finite only on (-1,0]; outside that interval the lower bound on the divergence, and hence the upper bound on the Bayes error, does not follow. The binary toy example does not rescue the multi-class implementation, since the toy explicitly transforms the sigmoid output into (-1,0] before computing the BOLT loss. The multi-class experiments do not state any such transform. I also considered whether Theorem 2 itself might be false; a direct analysis of the pointwise inequality implied by the theorem did not yield a counterexample for m=3 or m=4, so the decisive, clearly established problem is the domain mismatch between the theorem and the implemented loss. This is sufficient to reject the paper's central claim, because even a correct Theorem 2 would not justify the loss actually trained in the experiments.","tokens_in":8566,"tokens_out":19520,"duration_ms":172551,"concrete_test":"Run the Section 5.1 Gaussian binary experiment twice: once with the output transformed to (-1,0] as described, and once with a plain sigmoid output in [0,1] used directly in the BOLT loss, matching how Section 4 defines h_{θ,i} for the multi-class loss. Record the empirical BOLT loss L_θ and the known Bayes error Q(|μ1-μ2|/2) for a separation such as μ1-μ2=1, where the Bayes error is approximately 0.309. If the [0,1]-output version yields L_θ < ε_bys for any separation, the central inequality ε_bys ≤ L_θ is violated and the implemented loss is not the bound from Theorem 2.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing step is Section 4's identification of the trained network's output functions h_{θ,i}(x) with the functions h_i of Theorem 2. Theorem 2 is stated for any functions h_i : X -> (-1,0], and its derivation uses the Fenchel conjugate of the hinge loss, f*_hng(t)=t on (-1,0] and +∞ otherwise; outside this domain the variational lower bound on the f-divergence is not valid. In the implemented loss, however, h_{θ,i}(x) is explicitly described as a categorical probability, e.g. a softmax output in [0,1], with normalization (17). Consequently, the substitution f*_hng(h_{θ,i})=h_{θ,i} used in the binary derivation is not legitimate, and the quantity L_θ = E[1 - E_λ(F_θ(x))] is not shown to be an upper bound on the Bayes error. The failure is not merely cosmetic: in the binary case, if one takes h∈[0,1] and sets h=1 on the region where P_C1 dominates and h=0 elsewhere, the quantity 1/2 - 1/2(E_{P_C1}[h] - E_{P_C2}[h]) can fall below the true Bayes error for overlapping Gaussians. Thus minimizing the implemented BOLT loss does not enforce the claimed bound. The multi-class loss also depends on an arbitrary label ordering; no argument establishes that the resulting training objective is tight or Bayes-optimal. The binary toy experiment is consistent with Theorem 1 only because it explicitly transforms the output to (-1,0]; the multi-class experiments on CIFAR-10, MNIST, Fashion-MNIST, and IMDb do not perform such a transform and therefore do not validate the theorem-to-loss link.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an upper bound on the Bayes error rate based on f-divergence variational representations, and uses it to define a loss function called BOLT. The authors claim that minimizing BOLT drives a trained neural network to the Bayes optimal error rate. The paper states a binary bound (Theorem 1), extends it by recursion to a multi-class bound (Theorem 2), and defines BOLT as the empirical version of that bound. Experiments compare BOLT with cross-entropy on a binary Gaussian toy example, CIFAR-10, MNIST, Fashion-MNIST, and IMDb. The paper reports that BOLT matches the Bayes error in the toy example and slightly outperforms cross-entropy on the real datasets.","tokens_in":9002,"tokens_out":15971,"duration_ms":144636,"significance":"If the central claim were established, the paper would make a useful contribution: a loss function directly tied to the Bayes error rate, with empirical sampling from a model's outputs, would be a principled alternative to cross-entropy. The f-divergence variational formulation is a legitimate independent tool, and the binary toy experiment is a nice sanity check because the network output is explicitly transformed to the interval (-1,0] required by Theorem 1. The paper is also honest in deferring the multi-class proof to an extended version. However, the current manuscript does not provide the technical support needed for the advertised claims: the multi-class theorem is unproved, the implemented loss does not satisfy the theorem's hypotheses, and the experiments do not compare against Bayes error or the bound itself. The contribution therefore remains a promising but unsubstantiated idea.","major_comments":[{"comment":"Theorem 2 is the theoretical foundation for the multi-class BOLT loss, but its proof is omitted. The text says only 'The bound in the binary case is extended to m-class classification by treating it as a sequence of binary tasks. Details are presented in the extended version.' This is not acceptable for a central result: the formula for Eλ in (16) and the claimed inequality (15) cannot be verified from the material in the paper. A full proof, or a precise statement with a complete derivation, is required before the multi-class claims can be assessed.","section":"Section 3.2, Theorem 2"},{"comment":"There is a mismatch between the hypothesis of Theorem 2 and the outputs used in the implemented BOLT loss. Theorem 2 requires functions h_i : X -> (-1,0], and the proof of Theorem 1 relies on the Fenchel conjugate of the hinge loss being f*_hng(t)=t on (-1,0] and +∞ otherwise. In Section 4, however, the model output Fθ(x)=[hθ,1(x),...,hθ,m(x)] is described as a categorical distribution (e.g., softmax probabilities), so each hθ,i takes values in [0,1] and satisfies the normalization (17). The multi-class experiments in Section 5.2 do not transform these outputs into (-1,0]. Consequently, the inequality ε_bys ≤ Lθ is not established for the loss that is actually minimized, and the central claim that BOLT training drives the model to the Bayes error rate is unsupported.","section":"Section 4, Eq. (17); Section 5.2"},{"comment":"The paper asserts 'From Theorem 2, we know that ε_bys ≤ minθ Lθ. The right-hand side can closely approach to the Bayes error rate,' but no tightness or attainability result is proved for the multi-class case. Theorem 2 is only an upper bound for arbitrary admissible functions; it does not state that the infimum over the parameterized family {Fθ} approaches that bound. Moreover, the expression Eλ in (16) depends on a specific ordering of the labels, and no argument is given that the optimal h_i under this ordering recover the MAP classifier or that the resulting minimal Lθ is close to ε_bys. For the binary case the argument works because maximizing E1[h]-E2[h] over h∈[0,1] recovers the Bayes classifier, but for m>2 the analogous statement needs a proof and is not supplied.","section":"Section 4, paragraph after Eq. (17)"},{"comment":"The experiments do not validate the main claim of achieving Bayes optimal accuracy. Table 1 reports only test accuracy for BOLT and cross-entropy; it does not report the Bayes error rate, an estimate of it, or the value of Lθ during or after training. The CIFAR-10 and IMDb improvements (1.34% and 1.05%) are presented as evidence, but without a comparison to ε_bys or to the theoretical bound, these results only show that BOLT sometimes beats cross-entropy on the chosen tasks. The binary toy experiment in Figure 1 is the only place where the model's output is explicitly transformed to (-1,0], and it is the only experiment that actually checks closeness to ε_bys.","section":"Section 5, Table 1"}],"minor_comments":[{"comment":"The final sentence of the proof is incomplete: 'noting that for any h ∈ H, we have f∗hng(h(X))' should explicitly state that f∗hng(t)=t for t∈(-1,0], so that the expression reduces to E_{P_C1}[h(X)] - E_{P_C2}[h(X)].","section":"Section 3.2, proof of Theorem 1"},{"comment":"The loss ℓBOLT(Fθ(x), λ) uses the term hθ,λ-1, but hθ,0 is not defined in Section 4; the reader must infer hθ,0 = -1 from Theorem 2, and this should be stated explicitly in the loss definition.","section":"Section 4, definition of ℓBOLT"},{"comment":"The experimental section does not report random seeds or the number of independent runs; without variance estimates it is unclear whether differences such as 91.79% vs. 91.39% on Fashion-MNIST are statistically meaningful.","section":"Section 5.2, Table 1"},{"comment":"Equation (5) is introduced but never used in the subsequent derivations; either connect it to the f-divergence argument or remove it.","section":"Section 2, Eq. (5)"},{"comment":"The symbol P_Ci is used both for the conditional probability distribution and for its density, which creates ambiguity in equations such as (9) and (10); a clear distinction between a measure and its density would improve readability.","section":"Notation throughout"},{"comment":"Figure 1 shows no error bars or confidence intervals, and the statement that the two curves are 'nearly indistinguishable' is not quantified; reporting the maximum or mean absolute deviation between the BOLT error and ε_bys would make the claim more precise.","section":"Section 5.1, Figure 1"}],"recommendation":"reject","confidential_remarks":"The manuscript appears to be an early preprint whose central multi-class theorem is deferred to an extended version, and the implemented loss does not satisfy the theorem's stated range condition. These are load-bearing gaps rather than presentation issues. I see no reason to doubt the authors' good faith, but the paper's main claim cannot be accepted in its current form. If a complete proof of Theorem 2 is supplied and the loss is redefined or re-derived so that the inequality ε_bys ≤ Lθ holds for the actual model outputs, a resubmission could be reconsidered."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nThe headline: the binary bound in Theorem 1 is correct and the toy experiment is a genuine check. The multi-class story, however, does not hold together as written. Theorem 2 is stated without proof, deferred to an 'extended version', and the BOLT loss as implemented uses softmax probabilities in [0,1] where the theorem requires maps to (-1,0]. That breaks the chain from theory to experiments.\n\nWhat's new and good: The derivation of the binary bound is a clean combination of the f-divergence variational form and the Bayes-error/divergence identity, and it is presented clearly. BOLT is a new loss, and the idea of training by minimizing a bound on the Bayes error is worth exploring. The toy experiment is properly done: it maps the sigmoid output to (-1,0] and shows the learned error tracks the Bayes error. The CIFAR-10 improvement over cross-entropy (93.29% vs 91.95%) is a modest but real-looking gain.\n\nWhere the soft spots are: The missing proof of Theorem 2 is the obvious one. A main theorem without proof cannot carry the paper. But more troubling is the mismatch the stress-test note highlights. The theorem's domain is (-1,0] because the Fenchel conjugate of the hinge loss is +∞ outside that range. Section 4 explicitly describes the model output as a categorical distribution (e.g., softmax), with h_{θ,i} in [0,1] and a sum-to-one constraint. Substituting these into the bound is not legitimate. The inequality ε_bys ≤ L_θ is not established, and the multi-class experiments do not validate the theorem-to-loss link. The label-ordering issue is also unaddressed. These are not cosmetic issues; they strike at the central claim of Bayes-optimal training.\n\nWho this is for: a reader interested in loss-function design might appreciate the binary result and the idea, but the paper as a whole is not a reliable reference. I would not cite it for the multi-class claim.\n\nRecommendation: As it stands, I would not send this to peer review. The missing proof alone is enough for a desk reject, and the domain mismatch compounds it. If the authors come back with the extended version's proof and adjust the loss to use outputs in (-1,0] (or prove the bound for [0,1] outputs), it would be worth a serious look. For now, it's a promising draft, not a finished paper.","headline":"A correct binary bound and a promising loss idea, but the multi-class theorem is unproven and the implemented loss violates the theorem's domain conditions.","tokens_in":9510,"tokens_out":3847,"would_cite":false,"duration_ms":35784,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","62H30","62F15","68T05","62B10"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that the Bayes error of m-class classification can be upper-bounded by a quantity sampled from any classifier, and that minimizing that quantity as the BOLT loss drives a neural network to the Bayes-optimal error rate.","keywords":["Bayes error rate","f-divergence","classification","BOLT loss","neural network training","variational divergence bound","cross-entropy comparison"],"falsifier":"Take a dataset with a known Bayes error, train a network with a standard softmax head by minimizing BOLT, then compute the empirical value of $1 - \\frac{1}{m}\\sum_{\\lambda=1}^m E_\\lambda$ from the trained outputs and compare it with the model's test error; if test error is larger than this computed value, the claimed upper bound is violated for the implemented loss. A complementary check is to retrain the same model under several random permutations of the class labels: if final test accuracy changes substantially, the objective depends on arbitrary label ordering rather than being a universal Bayes-optimal loss.","tokens_in":8400,"feed_emoji":"🎯","tokens_out":9362,"duration_ms":77447,"temperature":0.7,"pith_summary":"The paper's central claim is that for an m-class classification problem with uniform class priors, the Bayes error rate admits an upper bound that can be evaluated by sampling the outputs of any parameterized classifier, no knowledge of the true data distribution required. The paper derives this bound via the f-divergence machinery, expressing the Bayes error as a sum of class-pair divergences and then bounding each divergence variationally through the hinge loss. This bound is then reinterpreted as a loss, the Bayes optimal learning threshold (BOLT), and the paper argues that minimizing it on a training set pushes the model's error toward the minimum achievable error. If the claim is right, BOLT gives a training objective that directly targets generalization at the Bayes limit rather than the log-likelihood proxy used by cross-entropy. Experiments on MNIST, Fashion-MNIST, CIFAR-10, and IMDb report BOLT matching or exceeding cross-entropy test accuracy, with the largest gains on CIFAR-10 and IMDb.","feed_headline":"New loss beats cross-entropy while targeting the Bayes error","feed_subtitle":"Sampling-based BOLT loss minimizes an upper bound on the minimum achievable error, matching or beating cross-entropy in tests.","key_machinery":"The machinery is the variational representation of the f-divergence, $D_f(P \\| Q) \\geq \\sup_{h\\in\\mathcal{H}} [ \\mathbb{E}_{x\\sim P} h(x) - \\mathbb{E}_{x\\sim Q} f^*(h(x)) ]$, applied with $f$ equal to the hinge loss $f_{\\mathrm{hng}}(u)=\\max\\{0,1-u\\}$, whose Fenchel conjugate is $f^*_{\\mathrm{hng}}(t)=t$ on $(-1,0]$ and $+\\infty$ elsewhere. Using this representation on the class-pair divergences that sum to the Bayes error turns the Bayes error into a supremum over bounded functions $h_i$, and Theorem 2 turns the multi-class case into a sum of class-conditional expectations of those functions. BOLT is the empirical loss obtained by replacing those expectations with sample averages over the network's output functions $h_{\\theta,i}(x)$. The bound is what carries the argument: if the loss is minimized, the model's error is driven toward the Bayes limit.","core_discovery":"The core discovery is Theorem 2: for uniform priors, $\\varepsilon_{\\mathrm{bys}} \\leq 1 - \\frac{1}{m}\\sum_{\\lambda=1}^{m} E_\\lambda$, where $E_\\lambda = \\mathbb{E}_{x\\sim P_{C_\\lambda}}\\left\\{ \\sum_{i=\\lambda}^{m-1} h_i(x) - h_{\\lambda-1}(x) \\right\\}$ for arbitrary measurable functions $h_i : X \\to (-1,0]$ with $h_0 = -1$. The paper's step is to notice that the right-hand side is an expectation over class-conditional data, so it can be approximated by averaging samples drawn from a trained model's outputs. Taking the model outputs $h_{\\theta,i}(x)$ to be the functions $h_i$, the paper defines the BOLT loss as the empirical version of this bound and claims that stochastic minimization of this loss makes a classifier achieve the Bayes error rate. This is the sense in which the bound is 'universal': the same loss construction applies to any parameterized classifier and any m-class task with balanced classes.","pith_inferences":["The paper does not test invariance to class-label permutations; since Theorem 2's $E_\\lambda$ terms depend on the chosen ordering of $h_i$, an easy extension is to measure BOLT's accuracy variance under random label shuffles. Large variance would mean the loss is not as universal as its derivation suggests.","The toy experiment uses a sigmoid output shifted into $(-1,0]$, satisfying the theorem, but the deep-network experiments use standard probability outputs; aligning the output head with the theorem's range might further close the gap to the Bayes error.","BOLT could be used as an auxiliary regularizer alongside cross-entropy rather than a replacement; the paper only evaluates BOLT alone, but the bound's Bayes-error interpretation makes it a natural penalty term."],"forward_implications":["On uniform-prior classification tasks, training with BOLT instead of cross-entropy should yield models whose test error approaches the Bayes error rate rather than merely minimizing negative log-likelihood.","Because the bound is sample-based and differentiable, BOLT can be plugged into standard mini-batch SGD for any neural architecture that outputs class probabilities.","On the paper's reported benchmarks, BOLT reaches 93.29% test accuracy on CIFAR-10 against 91.95% for cross-entropy, and 94.56% versus 93.51% on IMDb, while matching cross-entropy on MNIST.","BOLT provides a principled way to measure generalization: the gap between a trained model's error and the BOLT loss value is a computable proxy for distance to the Bayes limit."],"supporting_citations":[{"why":"Supplies the expression of Bayes error as a sum of class-conditional divergences and the binary hinge-loss identity from which the bound is derived.","marker":"Noshad et al. [2019]"},{"why":"Provides the variational lower bound on f-divergence (Lemma 1) that lets the bound be evaluated by sampling from model outputs.","marker":"Nowozin et al. [2016]"},{"why":"Cited source for the f-divergence variational bound and its Fenchel-conjugate formulation.","marker":"Liese and Miescke [2006]"},{"why":"Cited source for divergence theory and the conjugate representation used in the bound.","marker":"Liese and Vajda [2006]"},{"why":"Supports the toy-experiment claim that a feedforward network can represent the optimal h function, so BOLT training can approach the Bayes error.","marker":"Hornik [1991]"}],"fun_headline_variants":["BOLT loss achieves Bayes error, beats cross-entropy","Universal BOLT loss targets Bayes error, tops CE","Sampling-based BOLT loss hits Bayes error bound","Train any classifier to Bayes optimal with BOLT","New loss BOLT: Bayes optimal, cross-entropy beater"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument assumes the network's output functions $h_{\\theta,i}$ satisfy the same conditions as the theorem's $h_i$, which must map into $(-1,0]$; a standard softmax output lies in $[0,1]$, so the claimed inequality $\\varepsilon_{\\mathrm{bys}} \\leq L_\\theta$ may not hold for the loss actually implemented.","fun_headline_variants_meta":{"raw":{"variants":["BOLT loss achieves Bayes error, beats cross-entropy","Universal BOLT loss targets Bayes error, tops CE","Sampling-based BOLT loss hits Bayes error bound","Train any classifier to Bayes optimal with BOLT","New loss BOLT: Bayes optimal, cross-entropy beater"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000325,"raw_usage":{"total_tokens":1800,"prompt_tokens":904,"completion_tokens":896,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":520,"completion_tokens_details":{"reasoning_tokens":812}},"tokens_in":520,"tokens_out":896,"duration_ms":8644,"temperature":1.0,"reasoning_tokens":812,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:36:37.094867+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a dataset with a known Bayes error, train a network with a standard softmax head by minimizing BOLT, then compute the empirical value of $1 - \\frac{1}{m}\\sum_{\\lambda=1}^m E_\\lambda$ from the trained outputs and compare it with the model's test error; if test error is larger than this computed value, the claimed upper bound is violated for the implemented loss. A complementary check is to retrain the same model under several random permutations of the class labels: if final test accuracy changes substantially, the objective depends on arbitrary label ordering rather than being a universal Bayes-optimal loss.","supporting_citations":[{"cited_title":"f-gan: Training generative neural samplers using variational divergence minimization","cited_arxiv_id":null,"evidence_quote":"Provides the variational lower bound on f-divergence (Lemma 1) that lets the bound be evaluated by sampling from model outputs."},{"cited_title":"Approximation capabilities of multilayer feedforward networks","cited_arxiv_id":null,"evidence_quote":"Supports the toy-experiment claim that a feedforward network can represent the optimal h function, so BOLT training can approach the Bayes error."}],"review_version":1}