{"id":"11067127-77c1-4220-a7fe-b24384224330","arxiv_id":"2508.14077","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Under flexible models and clean labels, the outputs of label-smoothed classifiers trace the optimal trade-off curve of the information bottleneck problem.","lead":"This paper argues that label smoothing, a common trick of blending each training label with a little uniform noise, secretly solves an information bottleneck: the smoothed model outputs ignore irrelevant details and keep only what predicts the label. If the theory holds, a widely used regularization trick gets a principled interpretation as efficient representation learning.","discovery_kind":"unification","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The main theorem is unverifiable from the supplied text; the decisive unresolved assumption is that finite-capacity trained outputs actually attain the IB-optimal distribution.","rationale":"The reader's weakest_assumption correctly names 'sufficient model flexibility' as the central premise, and I agree that finite capacity is the main place where the practical claim could fail. I only partially agree because, given the unreadable body, the more immediate blocker is that the exactness of the equivalence cannot be checked at all: the paper could be correct, circular, or approximate, and the supplied text provides no way to distinguish these. The abstract itself is coherent and the claim is plausible, but the strongest version of the claim (label-smoothed outputs are exactly the IB-optimal family) requires a derivation that is not inspectable here. Since neither I nor the reader can verify the proof or the experiments, the UNVERDICTED verdict is the appropriate outcome, and my stress-test does not move it. If a readable version confirms the derivation and shows capacity-matched experiments approach the frontier, the verdict could become ACCEPT or CONDITIONAL; if the mapping is constructed rather than derived, it should be REJECT.","tokens_in":11709,"tokens_out":4330,"duration_ms":50405,"concrete_test":"Obtain the readable LaTeX/HTML source and independently re-derive the main theorem: for a fixed input distribution and smoothing coefficient α, minimize the label-smoothed cross-entropy over the full family of conditional distributions p(y|x), and verify that the resulting closed form equals the IB-optimal distribution for a corresponding trade-off parameter β with a monotone mapping α(β) that is independent of p(x). If the derivation is valid, additionally run a capacity-controlled experiment (e.g., a synthetic two-dimensional distribution with a known IB frontier) where model width/depth is varied; check whether the label-smoothed outputs approach the frontier as capacity grows, and report the divergence. This would settle whether the finite-capacity gap is a practical limitation or only a formal assumption.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that, as smoothing strength varies, the label-smoothed output distribution exactly traces the information-bottleneck optimal family. For this to be true, two things must hold: (i) there is a closed-form mapping from the smoothing coefficient to the IB trade-off parameter such that the global minimizer of label-smoothed cross-entropy equals the IB-optimal conditional distribution under the abstract's assumptions, and (ii) the networks trained in the experiments are flexible enough to realize those minimizers. The abstract's 'sufficient model flexibility' assumption is doing the load-bearing work: real neural networks have finite width, depth, and training time, so the empirically obtained output need not equal the IB optimum, and the experimental component is the only evidence for that regime. Because the supplied full text is mojibake, no theorem, derivation, or experimental protocol can be inspected, so it is impossible to tell whether the equivalence is derived or is built in by defining the IB parameter through the smoothing coefficient, and whether 'explores' means exact equality or loose approximation. The no-conflicting-labels assumption also excludes exactly the noisy-label regime where label smoothing is commonly applied, so the practical scope is narrower than the title suggests unless the paper explicitly extends the result.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper claims that, under sufficient model flexibility and no conflicting labels for the same input, the output distribution of a model trained with label smoothing explores the optimal solution of the information bottleneck (IB) as the smoothing strength varies. On this basis, the paper interprets label smoothing as a simple, practical implementation of the IB principle and reports experiments showing that label smoothing is insensitive to input factors that carry no information about the target, or no additional information beyond other variables. The abstract is legible, but the full text is severely corrupted: the body consists of mojibake characters, and no theorem, derivation, experimental protocol, figure, or table can be inspected.","tokens_in":11721,"tokens_out":2630,"duration_ms":30691,"significance":"If the claimed equivalence were established rigorously, the paper would provide a clean conceptual bridge between label smoothing and information-theoretic regularization, making the IB objective accessible through a one-line modification of standard training. The reported insensitivity property is also a falsifiable prediction with practical value. These strengths are real but conditional: the manuscript as supplied contains no inspectable proof or experimental evidence, and the stated assumptions (sufficient model flexibility, no conflicting labels) substantially narrow the regime in which the central claim is supposed to hold. The contribution therefore cannot currently be assessed beyond its abstract-level promise.","major_comments":[{"comment":"The body of the manuscript is unreadable: it is a sequence of replacement characters and garbled text, with no legible definitions, theorem statements, derivations, or experimental results. The central claim is a mathematical equivalence between label-smoothed outputs and IB-optimal distributions, and that claim cannot be verified from the supplied text. This is a load-bearing issue, not a cosmetic one. Please provide a readable version with the full derivation, the formal statement of the theorem, and the experimental details.","section":"Full text (all sections after the Abstract)"},{"comment":"The abstract does not state the mapping between the label-smoothing coefficient ε and the IB trade-off parameter β. The IB objective is L = I(X;Z) - β I(Z;Y) (or similar), so some relationship between β and ε is required for the claimed equivalence. If β is defined implicitly through ε so that the two objectives coincide by construction, then the claim that label smoothing 'explores' the IB optimal family risks being circular. Please give the explicit mapping and show that it is not an identity imposed by definition.","section":"Abstract and Theorem (unreadable body)"},{"comment":"The assumption of sufficient model flexibility is doing the load-bearing work: the theoretical equivalence requires that the trained model can represent the IB-optimal output distribution for every smoothing strength. Real neural networks have finite capacity and finite training time, so the empirically obtained output generally will not equal the theoretical minimizer. The experimental section is the only possible evidence for the finite-capacity regime, but it cannot be inspected in the corrupted text. Please state the theorem's exact representational condition and provide the experiments that validate the finite-capacity case.","section":"Abstract, assumption of sufficient model flexibility"},{"comment":"The assumption of no conflicting labels for the same input excludes exactly the noisy-label setting in which label smoothing is commonly applied in practice. The paper's title and framing suggest broad applicability, but the stated assumption narrows the scope considerably. Please clarify whether the equivalence or the insensitivity property holds when the dataset contains label noise, or, if not, state the limitation prominently.","section":"Abstract, assumption of no conflicting labels"}],"minor_comments":[{"comment":"The header contains 'arXiv:2508.14074v1 [cs.LG] 12 Aug 2025', which does not match the manuscript number 2508.14077 cited in this review; please correct the arXiv identifier.","section":"Header"},{"comment":"The word 'explores' is imprecise: please specify whether the label-smoothed output equals the IB-optimal distribution exactly, converges to it in the limit, or approximates it in a particular metric.","section":"Abstract"},{"comment":"The phrase 'no conflicting labels for the same input' needs a formal definition, including how it interacts with stochastic or soft labels in the label-smoothing construction.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":"To the editor: the uploaded full text appears to be an encoding-corrupted version of the manuscript. This is likely a submission error rather than a scientific defect, but it makes review impossible. I recommend asking the authors to upload a readable PDF and, if needed, to address the assumptions and beta-mapping concerns above in the revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this paper's abstract makes one crisp claim. Under enough model flexibility and no conflicting labels, the label-smoothed output distribution explores the optimal solution of the information bottleneck as smoothing strength varies. That would be a genuinely useful unification: label smoothing is everywhere, and an exact IB reading would give it a principled home and make the insensitivity-to-irrelevant-features claim a testable consequence. The abstract is well written and internally coherent.\n\nWhat is actually new is the claimed exact equivalence, if it is derived rather than assumed. We already know label smoothing is entropy regularization and that IB optimal conditionals have a Gibbs form. The new step would be showing that the global minimizer of the label-smoothed objective equals that Gibbs form with the IB trade-off parameter a function of the smoothing coefficient. The abstract also promises experiments on insensitivity to target-irrelevant factors, which is a concrete and falsifiable bonus.\n\nNow the soft spots, in proportion. I cannot inspect the main text: the body in front of me is mojibake, so no theorem, proof, or experiment protocol is legible. That forces an abstract-only judgment. Two assumptions are load-bearing. \"Sufficient model flexibility\" is doing real work: actual finite-capacity networks will not reach the IB optimum, so the experiments are the only evidence for that regime, and I cannot see them. \"No conflicting labels\" excludes the noisy-label setting where practitioners often reach for label smoothing; the title's generality may exceed the stated scope unless the paper explicitly extends the result. There is also the structural risk that the IB parameter is matched to the smoothing coefficient by construction, making the equivalence partly definitional. I cannot check that from the abstract. These are open questions, not demonstrated flaws; a readable version would settle them.\n\nIf the body is as clean as the abstract, this is a solid within-subfield paper, not a paradigm shift, and it deserves a serious referee. My own verdict is unverified, not skeptical. Get a readable copy, check whether the minimizer equality is derived, and check whether the experiments actually trace the frontier on finite networks.\n\nRecommendation: yes, send it to peer review. The claim is important enough and the abstract is strong enough that desk rejection would be wrong; a referee can verify the rest. If the derivation collapses, it will fail naturally in review.","headline":"A clean, plausible abstract-level claim that label smoothing traces the IB frontier, but the supplied body is unreadable so the derivation stays unverified.","tokens_in":12400,"tokens_out":2157,"would_cite":false,"duration_ms":23533,"reading_group":"maybe","serious_thinker":"unclear","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","94A17"],"pacs":[],"model":"deepseek-v4-flash","headline":"Label smoothing, as its smoothing strength varies, provably traces the information-bottleneck-optimal family of output distributions, making the technique a practical implementation of the information bottleneck.","keywords":["label smoothing","information bottleneck","mutual information","cross-entropy training","representation learning","trade-off curve","feature selection","model output distribution"],"falsifier":"Construct a dataset whose information-plane frontier is computable in closed form, such as a deterministic label plus an independent nuisance feature, train a high-capacity network under label smoothing at several strengths $\\alpha$, and measure the mutual information of the learned output features with the input and with the label. If the measured points do not lie on the predicted IB frontier, or if the trained output departs measurably from $q_\\alpha$ in a regime where capacity should suffice, the claimed equivalence is refuted.","tokens_in":11303,"feed_emoji":"🧠","tokens_out":4999,"duration_ms":53010,"temperature":0.7,"pith_summary":"This paper tries to show that label smoothing—the common trick of replacing one-hot training targets with a mixture of the true label and a uniform distribution—is not just a regularizer. Under the premises that the model is flexible enough to fit any output distribution and that no input carries conflicting labels, the paper argues that the distribution a label-smoothed model outputs is exactly the one a certain information bottleneck problem would pick. Varying the smoothing strength then sweeps along the trade-off between compressing the input and keeping label information. If true, label smoothing becomes a cheap, practical way to run information-bottleneck-style representation learning, and it should inherit IB's known insensitivity to irrelevant or redundant features; the paper adds experiments showing exactly that insensitivity.","feed_headline":"Label smoothing is an information bottleneck in practice","feed_subtitle":"Varying the smoothing strength moves a trained model's outputs along the optimal compression–prediction trade-off.","key_machinery":"The central object is the smoothed conditional target distribution\n$$q_\\$\\alpha$(y\\mid x)=(1-\\$\\alpha$)\\,\\delta_{y^*}(y)+\\frac{\\$\\alpha$}{K},$$\nused as the cross-entropy training target. Because a sufficiently flexible softmax model can interpolate any target, the trained output becomes $q_\\alpha$, and the paper's derivation identifies this output family with the solution set of an information bottleneck optimization, where $\\alpha$ acts as the Lagrange multiplier balancing compression of $x$ against prediction of $y$. This identification is what turns a training heuristic into a principled IB method.","core_discovery":"This paper's central claim is that label smoothing is a pragmatic information bottleneck. When a model is trained with cross-entropy against the smoothed target\n$$q_\\$\\alpha$(y\\mid x)=(1-\\$\\alpha$)\\,\\delta_{y^*}(y)+\\frac{\\$\\alpha$}{K},$$\nsufficient flexibility lets the model's output match $q_\\alpha$ for every input, and the paper derives that this output family is exactly the family a suitably formulated information bottleneck problem selects as $\\alpha$ varies. The strength $\\alpha$ therefore plays the role of the Lagrange multiplier that trades compression of the input representation against preservation of label information. Equivalently, each label-smoothed network is already doing IB-optimal compression on its output layer, with no auxiliary network or variational approximation needed, under the stated assumptions of sufficient model flexibility and no conflicting labels for the same input. The paper also demonstrates experimentally that label smoothing suppresses features that carry no information about the target, or no additional information once another feature is known, which is the behavioral signature of an information bottleneck solution.","pith_inferences":["If the claimed equivalence holds, the trained network's output entropy becomes an observable proxy for the IB trade-off point selected by $\\alpha$, so practitioners could choose $\\alpha$ by targeting a desired compression level rather than by grid search on validation accuracy.","The 'no conflicting labels' premise carves out exactly the noisy-label regime where smoothing is often recommended; extending the argument there would require modeling conflicts as part of the label distribution instead of excluding them, which the paper does not attempt.","The same mechanism suggests a broader design rule: any training target formed as a convex mixture of a hard label and a fixed reference distribution induces an IB-style trade-off, so teacher-smoothed distillation objectives might trace the same frontier with the teacher playing the role of the reference distribution."],"forward_implications":["Label smoothing can serve as a drop-in information bottleneck objective, requiring no auxiliary networks, variational bounds, or separate information-theoretic losses.","The smoothing strength $\\alpha$ is not merely a regularization knob; it selects a specific point on the IB trade-off curve between input compression and label preservation.","Models trained with label smoothing should ignore features that carry no target information, whether alone or conditioned on other features, matching the behavioral signature of IB-optimal representations.","The reported calibration and robustness effects of label smoothing can be read as consequences of IB-style compression rather than as unrelated side effects."],"supporting_citations":[],"fun_headline_variants":["Label smoothing is a pragmatic info bottleneck in practice","Label smoothing implements an information bottleneck","How label smoothing compresses without a variational net","Label smoothing's alpha tunes the IB trade-off"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire equivalence depends on the trained model being flexible enough to output exactly the smoothed target distribution for every input and every smoothing strength, and on no input having two different labels; real networks only approximate the first condition, so the theoretical claim may not hold exactly for them.","fun_headline_variants_meta":{"raw":{"variants":["Label smoothing is a pragmatic info bottleneck in practice","Label smoothing implements an information bottleneck","How label smoothing compresses without a variational net","Label smoothing's alpha tunes the IB trade-off"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000798,"raw_usage":{"total_tokens":3454,"prompt_tokens":833,"completion_tokens":2621,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":449,"completion_tokens_details":{"reasoning_tokens":2565}},"tokens_in":449,"tokens_out":2621,"duration_ms":20403,"temperature":1.0,"reasoning_tokens":2565,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T17:31:45.014433+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Construct a dataset whose information-plane frontier is computable in closed form, such as a deterministic label plus an independent nuisance feature, train a high-capacity network under label smoothing at several strengths $\\alpha$, and measure the mutual information of the learned output features with the input and with the label. If the measured points do not lie on the predicted IB frontier, or if the trained output departs measurably from $q_\\alpha$ in a regime where capacity should suffice, the claimed equivalence is refuted.","supporting_citations":[],"review_version":2}