{"id":"d6a8c5d2-15e0-4f91-a1ec-bd9f03005a04","arxiv_id":"2607.04733","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A fine-tuning objective that preserves the base model's local preference among non-label tokens improves pass@1 accuracy and retains more pretrained capability than standard SFT.","lead":"Researchers propose a new way to fine-tune language models that tries to preserve what the pretrained model already knows about alternative word choices, instead of forcing it to focus only on the labeled token. The method improves accuracy and reduces forgetting across math, code, and general-knowledge benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Fixed Kmax=10 truncation is not justified by the paper's own entropy analysis: plateau-k states at k=12 and k=15 are documented, so LP-SFT cannot preserve the full local structure it claims to protect.","rationale":"The reader's weakest assumption—that Kmax=10 captures the protective local structure across all domains and scales—is the most load-bearing concern because it directly targets the mechanism claimed in the abstract and conclusion. The paper's entropy analysis is internally inconsistent with the design choice: it highlights plateau states at k=12 and k=15 in Appendix B, yet §5.1 sets Kmax=10 and justifies this as covering 'the prominent higher-order regime around k=10'. The method therefore explicitly does not preserve the higher-order entropy-compressed states that the same analysis shows to be widespread (structural ambiguity in IDs, dates, URLs). The paper itself admits this truncation limitation in §5.2. This is not a disagreement with consensus; it is a mismatch between the motivating analysis and the proposed objective. The concern is testable by measuring the frequency of k>10 plateau states and by ablating Kmax. The paper otherwise presents a coherent empirical study with meaningful gains over strong baselines, and the ablation (Table 17) shows the two design components help; the issue is that the fixed support size may be too small for the claimed mechanism. The reader's CONDITIONAL verdict remains appropriate: the central claim is plausible but not fully established until the truncation is shown to be harmless or adequately motivated. I do not see a reason to change the verdict, so I set verdict_should_be to UNCHANGED.","tokens_in":18758,"tokens_out":6465,"duration_ms":78604,"concrete_test":"On the three training datasets (UltraFeedback, Magicoder, NuminaMath) for each base model, compute the proportion of target positions where the base distribution has N1>10 and R>0.95 (using the K=30 normalization of §3), i.e., high-order plateau states beyond Kmax. Also record how often |At|<3 (target in top-10 leaving few non-target alternatives). If this proportion is non-negligible (>2–5%), rerun LP-SFT on Qwen3-4B-UltraFeedback with Kmax=20 and Kmax=30 (same seed, µ=1) and compare the Table 2-style average. If the larger Kmax changes the average by more than about 1 point or improves pass@k on math/code, the Kmax=10 choice is not adequately covering the structure and the headline claim is conditional on an arbitrary truncation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central mechanism of LP-SFT is preservation of the base model's 'local preference structure' among plausible alternatives. But the preservation set is hard-truncated at Kmax=10 (§5.1), and the paper's own analysis shows that high-order plateau states extend well beyond this cutoff. Appendix B reports near-uniform plateau states at k=12 (N1=11.998, R=0.9981, Table 12) and k=15 (N1=14.966, R=0.9969, Table 13), and Figure 1 shows ridges at N1=10 and beyond. Setting Kmax=10 therefore excludes exactly the high-order structural-ambiguity states the paper argues are important (random IDs, URLs, hashes, timestamps). The limitation is acknowledged in §5.2: the local set 'may not cover the entire plateau region, especially when the effective support of the base distribution exceeds Kmax.' Since the objective then preserves only the first 10 base alternatives (minus the target token), it does not actually protect the full multimodal entropy structure that motivates the method. The claimed benefit could be an artifact of regularizing over a convenient, cheap subset rather than preserving the structure the paper identifies. If high-order plateaus contribute to retention or diversity, LP-SFT's benefit should shrink or invert on domains where those states are frequent, and the fixed Kmax=10 choice is not robust across domains/scales.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes LP-SFT, a supervised fine-tuning objective that adds a local KL-regularization term to cross-entropy. The term is computed over a frozen base model's top-K candidate set after removing the supervised target token, with locally normalized probabilities, intended to preserve the base model's relative preferences among non-label alternatives. The motivation is an empirical analysis of next-token distributions reporting a multimodal 'plateau-k' entropy structure, including higher-order plateaus around k=10, 12, and 15. Experiments fine-tune Qwen3-4B, Qwen3-14B, and Llama-3.1-8B on UltraFeedback, Magicoder-OSS-Instruct-75K, and NuminaMath-CoT, and evaluate on MATH-500, AIME 2021-26, MBPP+, HumanEval+, and MMLU. The paper reports that LP-SFT improves aggregate pass@1/pass@k and MMLU retention over vanilla CE, DFT, EAFT, GEM, and ASFT, with ablations supporting the target-token-removal and local-normalization design choices.","tokens_in":19104,"tokens_out":4967,"duration_ms":52219,"significance":"If the empirical claims are robust, LP-SFT is a practical, low-overhead regularization scheme for supervised fine-tuning that mitigates catastrophic forgetting and preserves sampling diversity without full-vocabulary anchoring. The paper's strengths include evaluation across multiple model families and scales, multiple training distributions, and several recent baselines; a careful ablation of the two design choices; a computationally efficient offline base-model cache; and released code. The central weakness is that all comparisons are single-seed point estimates with no variance reporting, and the fixed truncation at Kmax=10 is not justified by the paper's own entropy analysis, which documents plateau states beyond that cutoff. The motivation-to-method link is therefore not fully established.","major_comments":[{"comment":"The method hard-truncates the preservation set at Kmax=10, but the paper's own analysis documents plateau-k states at k=12 (Table 12: N1=11.998, R=0.9981) and k=15 (Table 13: N1=14.966, R=0.9969), and Figure 1 shows ridges beyond N1=10. The text in §5.2 explicitly states that the local set 'may not cover the entire plateau region, especially when the effective support of the base distribution exceeds Kmax.' Since high-order plateaus are argued to represent structural ambiguity (random IDs, URLs, digits, timestamps), domains rich in such tokens are exactly where LP-SFT's protection is incomplete. The claim that LP-SFT preserves the multimodal entropy structure identified in Section 4 is therefore only partially realized. Please either (a) provide evidence that k>10 plateaus are negligible for retention/diversity, or (b) include a sensitivity analysis over Kmax (e.g., 10, 15, 20) showing t","section":"§5.1, §5.2, Appendix B"},{"comment":"All experiments use a single fixed seed (1234) and report point estimates without error bars, confidence intervals, or multiple runs. Several headline differences are numerically small; for example, Table 2 (Llama-3.1-8B, UltraFeedback) shows LP-SFT Avg 33.91 vs. CE 33.20, and Table 16 (Qwen3-4B, NuminaMath) shows Avg 61.25 vs. CE 60.35. The central claim that LP-SFT 'improves overall performance' is not statistically supported. Please run at least 3 seeds for the main comparisons (or provide bootstrap intervals over evaluation samples) and report mean±std, so that the reader can judge the significance of the reported gains.","section":"§6.1, Appendix C.1, Tables 1, 2, 15, 16"},{"comment":"The abstract and conclusion state that LP-SFT improves overall performance across mixed-domain and single-domain experiments, but the results for Llama-3.1-8B show that CE is stronger on several math metrics (Table 2: MATH-500 pass@1 10.76 vs. 9.41; AIME pass@1 0.23 vs. 0.12), and the overall advantage is small. The discussion acknowledges this in prose, but the general claim should be qualified or the paper should characterize the conditions under which local preservation helps versus hurts. One concrete step is to analyze the base model's entropy structure for Llama-3.1-8B and relate it to the observed performance pattern; currently the reader cannot tell whether the method's benefit depends on the quality of the base distribution's plateau structure.","section":"§6.3, Tables 2 and 16"}],"minor_comments":[{"comment":"The title has typographical spacing errors: 'LOCAL-PRESERVINGSUPERVISEDFINE-TUNING' and 'VIAMULTIMODALENTROPYSTRUCTURE' should be separated into words.","section":"Abstract/Title"},{"comment":"The notation 'K=K max=10' is confusing because Section 3 uses K=30 for the entropy-analysis truncation. Please use distinct symbols, e.g., K_ent for analysis and K_max for the preservation set, and state the default values explicitly.","section":"§5.1"},{"comment":"Training-time ratios exclude the offline base-model precomputation pass. While the text explains that this pass is amortized, the first-run end-to-end time should also be reported, since Stage 1 is a full forward pass over the training set and may be nontrivial for large models or large datasets.","section":"Appendix E.3, Table 18"},{"comment":"The plateau-k definition uses fixed thresholds N1 in [k-0.3, k+0.3] and R>0.95. This is a reasonable heuristic, but the paper should state how sensitive the observed conclusions in Figures 1-2 are to these thresholds, or at least acknowledge that the thresholds are chosen post hoc.","section":"§4.2, Eq. (12)"}],"recommendation":"major_revision","confidential_remarks":"The entropy-structure motivation relies on reference [19], which is authored by overlapping researchers. The paper's contribution relative to [19] is the extension to higher-order plateaus and the use of the effective-support ratio R; that is defensible, but the authors should ensure the novelty is framed carefully and that [19] is not the only basis for the central motivation. The lack of variance reporting and the fixed Kmax choice are the main technical blockers; both are addressable within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"LP-SFT is a genuinely useful, low-cost trick: take the frozen base model's top-10 non-target tokens at each position, normalize both base and trained distributions over that local set, and add a KL term that preserves relative preferences. Target-token removal makes sense, and the offline caching keeps the overhead to about 1.07x CE. The empirical story is consistent across three backbones and three training sets, with some striking wins on MMLU retention, and the ablations support both design choices. The Llama-3.1-8B math results are weaker, which the authors acknowledge — that's honest and actually strengthens the credibility of the main claim.\n\nThe soft spots are real but not fatal. Everything is single-seed, no error bars, so the reported gains over CE could be partly noise. The AIME 2021–26 set is collected but not released, which hurts reproducibility. And the entropy-analysis motivation is partly circular: Kmax=10 is chosen from the same plateau analysis used to justify the method, while Appendix B shows near-uniform plateaus at k=12 and k=15. The paper does explicitly concede in §5.2 that the local set may miss structure when effective support exceeds Kmax, and the stress-test concern about high-order plateaus is a legitimate gap. But this is a weakness in the framing, not in the central held-out comparison. The regularizer works even if the precise mechanism is fuzzier than the title claims. What's missing is an ablation varying Kmax and an analysis of how much high-order plateau structure actually matters on these benchmarks.\n\nWho should read this: anyone working on fine-tuning, catastrophic forgetting, or sampling diversity. It deserves a serious referee and a conditional accept after the authors add multiple seeds, release the AIME problems, and show Kmax sensitivity. I'd probably cite it if I worked in this area.","headline":"A cheap, effective SFT regularizer with consistent gains, but the entropy-based motivation is softer than advertised and the experiments need error bars before I'd trust the aggregate numbers.","tokens_in":19578,"tokens_out":2084,"would_cite":true,"duration_ms":25790,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"LP-SFT shows that supervised fine-tuning can reduce catastrophic forgetting and preserve generation diversity by adding a locally normalized KL term over the frozen base model's top-10 non-target token alternatives, alongside standard cross","keywords":["supervised fine-tuning","catastrophic forgetting","multimodal entropy structure","local preservation","KL divergence","top-k support","pass@k diversity","language models"],"falsifier":"Measure the effective support size N1 of a base model's next-token distributions at positions where fine-tuning causes the largest accuracy drops; if most of those positions have N1 well above 10 (say, >20), then LP-SFT with Kmax = 10 should fail to preserve them, and a version with larger or adaptive K should outperform it. A direct experiment is to fine-tune with Kmax = 10 versus Kmax = 30 or 100 and compare pass@1, pass@k, and MMLU; a substantial gain from larger K would indicate that the fixed 10-token truncation is the bottleneck.","tokens_in":18665,"feed_emoji":"🧠","tokens_out":5703,"duration_ms":60320,"temperature":0.7,"pith_summary":"Standard supervised fine-tuning (SFT) improves a model on a target task but often degrades existing abilities and shrinks output diversity, because cross-entropy trains only the observed label token and leaves unconstrained how probability mass is redistributed over other plausible tokens. This paper shows that pretrained models encode a regular multimodal entropy structure: next-token distributions contain near-uniform plateaus at k = 1, 2, ..., 10 plausible alternatives, reflecting logical branching at low order and structural ambiguity (digits, dates, IDs) at high order. The paper proposes LP-SFT, which adds to cross-entropy a locally normalized KL divergence over the frozen base model's top-10 non-target candidates, preserving relative preferences among those alternatives without constraining their absolute probability mass. Across mixed-domain and single-domain fine-tuning experiments, LP-SFT outperforms vanilla SFT and recent SFT-enhancement baselines, achieving the best balance between pass@1 accuracy and pass@k diversity while retaining general knowledge. If correct, local preservation is a cheap and effective way to mitigate catastrophic forgetting and maintain sampling-accessible diversity during fine-tuning.","feed_headline":"New fine-tuning loss preserves old skills while adding new ones","feed_subtitle":"Adding a locally normalized KL term over non-label top-10 tokens keeps general knowledge and sampling diversity intact.","key_machinery":"The central mechanism is the multimodal entropy structure of next-token distributions, quantified by the effective support ratio R = N2/N1, where N1 = exp(H1) is the Shannon effective support and N2 = exp(H2) the Rényi-2 effective support over a normalized top-30 window. Peaks in the (N1, R) plane near integer N1 with R close to 1 identify plateau-k states — positions where uncertainty is spread nearly uniformly across k plausible alternatives. LP-SFT uses this structure to build its preservation set: at each position it takes the frozen base model's top-10 tokens (Kmax = 10, chosen to cover low-order plateaus and the prominent k ≈ 10 regime), removes the supervised target token, and applies","core_discovery":"The paper's central claim is that the harmful side effects of supervised fine-tuning arise from overwriting the pretrained model's local preference structure, and that this structure can be preserved at almost no extra computational cost. The authors first demonstrate that base models exhibit next-token entropy peaks at H ≈ ln k for k = 1 through 10 and beyond, meaning token positions fall into discrete uncertainty regimes: some are nearly deterministic, others admit a small set of roughly equally plausible continuations. Standard cross-entropy, rewarding only the observed target token, suppresses these alternatives and distorts the pretrained structure, which the authors identify as a mecha","pith_inferences":["Because LP-SFT's preservation relies on the frozen base distribution being a good structural reference, the analysis suggests that fine-tuning data containing many structurally ambiguous positions (digits, dates, IDs) will benefit most; these are exactly the positions where cross-entropy injects arbitrary supervision.","The plateau-k framework could be used as a diagnostic: measuring N1 and R before fine-tuning might allow one to predict which domains will suffer the most forgetting under vanilla SFT and to choose Kmax accordingly, rather than fixing it at 10.","A testable extension is to let Kmax adapt per position to the base distribution's observed effective support; this should improve retention on models or domains where the support exceeds 10, as hinted by the paper's own weaker-backbone math results.","The local KL term suggests a broader principle: when adapting a model, preserve the relative ordering of plausible alternatives, not just the top-1. This principle could transfer to preference optimization or RLHF, where reference-policy structure is often regularized globally rather than locally."],"forward_implications":["If LP-SFT is right, then standard SFT's catastrophic forgetting is largely a failure to preserve local preference structure, and a lightweight top-10 constraint can recover much of the benefit of full-vocabulary anchoring at a fraction of the computational cost.","General knowledge (as measured by MMLU) degrades far less under LP-SFT than under vanilla cross-entropy or other SFT variants, narrowing the average gap to the frozen base model from double digits to about one point in the mixed-domain setting.","Pass@k diversity is maintained or improved while pass@1 accuracy also improves, suggesting that preserving non-label alternatives does not come at the expense of single-sample correctness.","LP-SFT is relatively insensitive to the preservation weight μ, unlike full-vocabulary KL anchoring which is reported to be sensitive to its KL weight; this makes the method easier to deploy without hyperparameter tuning.","The method's benefit depends on the quality of the base distribution: for weaker backbones on math tasks, vanilla cross-entropy can remain stronger on some metrics, implying that local preservation is most helpful when the pretrained distribution is a reliable structural reference."],"fun_headline_variants":["Fine-tuning loss preserves pretrained diversity","LP-SFT: keep old skills while learning new tasks","Protect entropy, fine-tune without forgetting","New SFT objective shields prior knowledge","Local KL term stops fine-tuning skill loss"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that a fixed top-10 truncation of the frozen base distribution captures the alternatives whose preservation protects pre-trained capabilities; if the effective local support is much larger or concentrated above k = 10, the preservation term omits what matters and the method's benefit could shrink or even reverse.","fun_headline_variants_meta":{"raw":{"variants":["Fine-tuning loss preserves pretrained diversity","LP-SFT: keep old skills while learning new tasks","Protect entropy, fine-tune without forgetting","New SFT objective shields prior knowledge","Local KL term stops fine-tuning skill loss"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000123,"raw_usage":{"total_tokens":958,"prompt_tokens":787,"completion_tokens":171,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":531,"completion_tokens_details":{"reasoning_tokens":113}},"tokens_in":531,"tokens_out":171,"duration_ms":3054,"temperature":1.0,"reasoning_tokens":113,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T08:32:53.885955+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the effective support size N1 of a base model's next-token distributions at positions where fine-tuning causes the largest accuracy drops; if most of those positions have N1 well above 10 (say, >20), then LP-SFT with Kmax = 10 should fail to preserve them, and a version with larger or adaptive K should outperform it. A direct experiment is to fine-tune with Kmax = 10 versus Kmax = 30 or 100 and compare pass@1, pass@k, and MMLU; a substantial gain from larger K would indicate that the fixed 10-token truncation is the bottleneck.","supporting_citations":[],"review_version":2}