{"id":"251f9812-2d68-4cac-a7a8-1895e21dd1c2","arxiv_id":"2506.06215","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A training and sampling method that lets autoregressive LLMs resample earlier tokens in a small window, improving reasoning and coding benchmark scores by about 10% relative after a 100B-token fine-tuning.","lead":"Autoregressive language models usually generate text left to right and can never fix a wrong token. This paper proposes Resample-Previous-Tokens (RPT), which revisits and replaces nearby tokens during generation, and reports about 10% relative gains on coding and reasoning benchmarks after a short fine-tuning.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The evidence for Eq. (20) is not evidence: the PTP cross-entropy gap can be an entropy effect, Table 2 measures distance to a one-hot rather than to the true conditional, and κ in Eq. (18) is never measured, so ρ<1 is unestablished.","rationale":"I agree with the reader's identification of Eq. (20) as the critical assumption, but I would sharpen it: the offered evidence is not merely incomplete, it is the wrong measurement. Lower CE on PTP is confounded by lower conditional entropy, and the table labeled 'TV distance' actually measures confidence in the observed token. The κ term in Eq. (18) is also unmeasured, so even a true PTP-vs-NTP error gap would not by itself give ρ<1. The correct response is not rejection: the empirical method could still work, and the benchmark gains, although noisy and partly present at k=0, are plausible enough to warrant further study. The theoretical support, however, is weaker than the paper presents, and the central 'corrector sampling' mechanism is not isolated from the fine-tuning effect. The verdict should remain CONDITIONAL pending a test that actually measures ρ, plus release of code/checkpoint and a multi-seed fixed-temperature evaluation that separates k=0 gains from k>0 sampling gains.","tokens_in":17079,"tokens_out":13242,"duration_ms":118129,"concrete_test":"On a held-out corpus, use a much larger reference model (or ensemble) as a proxy for the true conditionals π_i and π_{i|i+1}. Estimate the three error norms in Eq. (18) on a sample of contexts, and estimate κ from the ergodicity coefficient of the empirical RPT transition kernel (restricting the state space to the model's top-k vocabulary to make the computation tractable). If the resulting ρ is ≥ 1 for typical contexts, Theorem 1's premise fails and the CE curves in Fig. 3 cannot support RPT's theoretical advantage. An even more direct check: recompute Table 2 as actual TV distance to the reference-model conditional rather than 1 − p̂(x_i); if the PTP-vs-NTP ordering reverses, Eq. (20) is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing condition for the paper's theoretical mechanism is ρ<1 in Eq. (18), which the paper reduces to the claim in Eq. (20) that the previous-token prediction error ‖ε_{i|i+1}‖_∞ is smaller than the NTP errors ‖ε_i‖_1 and ‖ε_{i+1|i}‖_∞. This premise is not actually tested. Figure 3 only shows lower PTP cross-entropy than NTP cross-entropy. Those are different prediction tasks: H(π_i | x_{<i}, x_{i+1}) ≤ H(π_i | x_{<i}) always, so lower CE is expected even if the learned conditionals have equal or larger KL/TV error. The paper's assumption that the relevant entropies are 'near zero' is not credible for real token distributions, so Pinsker cannot convert the CE gap into a TV gap. Table 2's 'empirical TV distance' is defined as 1 − p̂^{(k)}(x_i | x_{<i}) against the observed validation token, i.e., distance to a one-hot distribution, not to the true conditional p(x_i | x_{<i}); it therefore does not measure the quantity in Eq. (20). In addition, ρ contains the Markov-chain condition number κ, which is never estimated; and Theorem 1 only compares an upper bound on RPT error to a 'tight' NTP bound. None of the paper's evidence establishes that RPT's actual sampling error is lower, and the benchmark tables do not isolate the sampling effect because k=0 (NTP from the RPT-finetuned model) already shows most of the gains over AR-F.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Resample-Previous-Tokens (RPT), a sampling method that, after standard autoregressive generation, revisits tokens in a small window and resamples them using a learned previous-token-prediction (PTP) conditional. The PTP conditionals are trained by randomly permuting adjacent tokens during a short fine-tuning phase, using a relative positional embedding to encode the permutation. The authors present a first-order perturbation analysis of the two-token case, define an RPT factor ρ (Eq. 18), and state Theorem 1: if ρ < 1, RPT's error bound is lower than a tight NTP error bound. They support the premise ρ < 1 with cross-entropy curves (Figure 3), an empirical 'TV distance' table (Table 2), and a synthetic experiment (Figure 4). On nine coding/reasoning benchmarks, fine-tuning an 8B model for 100B tokens with the RPT objective and then sampling with RPT yields an average relative improvement of roughly 10% over the fully AR-trained baseline, though one benchmark (PHP) regresses.","tokens_in":17462,"tokens_out":14232,"duration_ms":122288,"significance":"If the empirical claims are upheld, RPT would be a low-cost, easy-to-integrate alternative sampling scheme for autoregressive LLMs, with a plausible mechanism (reducing error accumulation through local correction) and a conditional theoretical guarantee. Strengths of the paper include a controlled synthetic experiment that directly tests the mechanism, a clear training procedure, and a large-scale fine-tuning study. However, the load-bearing theoretical premise—that PTP error is smaller than NTP error—is not directly verified by the provided evidence, and the benchmark results are more mixed and more confounded with fine-tuning and decoding-heuristic effects than the abstract suggests. The paper is of interest to the community but needs substantial additional evidence before its central claims can be accepted as stated.","major_comments":[{"comment":"The paper's central theoretical premise, Eq. (20), is that the previous-token-prediction error ‖ε_{i|i+1}‖∞ is smaller than the next-token-prediction errors ‖ε_i‖1 and ‖ε_{i+1|i}‖∞. The only direct empirical support offered is Figure 3, which compares cross-entropies. This comparison is not informative for the error norms in Eq. (20): because conditioning on a future token reduces the entropy of the target distribution (H(X_i | X_{<i}, X_{i+1}) ≤ H(X_i | X_{<i})), the PTP cross-entropy is expected to be lower even when the learned conditionals have equal or larger KL divergence to the ground truth. The 'near-zero entropy' assumption used to convert the cross-entropy gap into a TV gap via Pinsker is not justified for natural-language token distributions and is not tested. The authors should provide a direct measurement of the errors in Eq. (14), for example by evaluating on a corpus with known ground-truth conditionals, or by reporting estimated TV/KL distances to estimated or curated targets.","section":"Section 3, Eq. (20), Figure 3"},{"comment":"The 'empirical total variation distance' in Section 5.3 is defined as ε^(k)(x_i) = 1 − p̂^(k)(x_i | x_{<i}) on validation tokens. This is the complement of the predicted probability of the observed token, not the total variation distance between p̂^(k)(· | x_{<i}) and the true conditional p(· | x_{<i}). It equals TV only in the special case where the ground-truth conditional is deterministic. Since the paper does not establish approximate determinism, Table 2 does not measure the quantity in Eq. (20) and cannot be used to support the claim that RPT sampling reduces error relative to the true conditional. The table also reports the RPT-updated conditional after k iterations rather than the PTP conditional itself, so it does not quantify ε_{i|i+1}.","section":"Section 5.3, Table 2"},{"comment":"The RPT factor ρ in Eq. (18) contains the Markov-chain condition number κ, defined after Eq. (31) as (1 − Λ(P))^{-1}. The paper never estimates κ or bounds it for the models used in the experiments. Even if the PTP error norm were shown to be smaller than the NTP error norm, the product κ ‖ε_{i|i+1}‖∞ could still exceed ‖ε_i‖1, making ρ > 1. Theorem 1 is therefore a conditional statement whose condition is unverified; as stated, it is essentially a restatement of the definition of ρ. The authors should either estimate κ for the learned chain, report the actual RPT-to-NTP error ratio in a setting where ground truth is known (as in the synthetic experiment) at a more realistic scale, or else clearly label the theory as purely conditional and not part of the empirical evidence.","section":"Section 3, Eq. (18), Theorem 1"},{"comment":"Table 1 does not support the abstract's '~10% relative improvements' as a property of RPT sampling. PHP is worse under RPT than AR-F (25.5 vs 28.5 at k=1, a -10.5% relative regression), and the row k=0 (NTP sampling from the RPT-finetuned model) already captures most of the improvement over AR-F on several benchmarks. For example, C++ reaches 31.0 at k=0 and stays at 31.0 at k=1; HumanEval+ goes from 27.4 at k=0 to 28.6 at k=1.5; Java goes from 38.3 at k=0 to 41.1 at k=1. The incremental effect of the resampling iterations is therefore small and benchmark-dependent. The paper should report the k=0-to-k>0 deltas explicitly and temper the claim of 'consistent and significant improvement'.","section":"Section 5.4, Table 1"},{"comment":"The main reported RPT results use greedy decoding in the PTP step and a confidence threshold η=0.9 (Section 5.1), neither of which is part of the Markov-chain process analyzed in Section 3 nor of the standard NTP baseline. The ablation in Appendix B.2 shows that without these heuristics, RPT sampling does not consistently beat NTP: for example, Java drops from 37.9 (AR-F) to 34.8 (RPT, k=1, no heuristics). The gains in Table 1 are therefore partly attributable to decoding heuristics rather than to the resampling mechanism, and the NTP baseline is not reported with the same greedy/confidence heuristics. The paper should either apply the same heuristics to the NTP baselines or present the heuristic-free comparison as the primary test of the method.","section":"Section 5.1 and Appendix B.2"}],"minor_comments":[{"comment":"The abstract claims '~10% relative improvements' and Section 5.4 claims 'consistent and significant improvement', but Table 1 includes a negative result on PHP and no statistical significance testing is reported; please clarify the aggregation rule and report per-task deltas with confidence intervals or multiple runs.","section":"Abstract and Section 5.4"},{"comment":"Figure 3 and the surrounding text refer to a 1.5B parameter model, while the main experiments use an 8B model; please state whether the cross-entropy curves are representative of the 8B model and add the corresponding curves for the fine-tuned model.","section":"Section 3, Figure 3"},{"comment":"The notation '(w<i+w, xi)' in Eq. (7) appears to be a typo for the set (x_{<i+w} \\ {x_i}); please correct and define the notation clearly.","section":"Equation (7)"},{"comment":"The definitions of the permutation σ and target sequence τ in Eqs. (9)-(11) are difficult to parse; a worked example with explicit indices (beyond Figure 2) would improve accessibility.","section":"Equations (9)-(11)"},{"comment":"The synthetic experiment uses a jointly uniform random distribution with V=20; this is useful as a mechanism check but does not validate Eq. (20) for language-like distributions, and the paper should say so explicitly.","section":"Section 3, Synthetic example"},{"comment":"The selection of the best temperature in {0.0,0.05,0.1} for each method is an oracle procedure; it should be stated prominently in the main text, and the temperature sensitivity plots in Appendix C should be discussed in relation to the reported improvements.","section":"Table 1 and Appendix C"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is likely of interest to the machine-learning community, and the core idea is sufficiently novel. The main risk is overclaiming: the theoretical contribution is conditional, the key empirical premise is not directly tested, and the benchmark results are more mixed than the abstract suggests. The revision should require direct evidence for the PTP-error premise, a clearer separation of fine-tuning, sampling, and heuristic effects, and a more measured presentation of the results. I do not see a novelty disclosure issue; the related work is adequately cited."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know before reading: the practical recipe is genuinely new and plausible, and the theory section is weaker than the rest of the paper.\n\nWhat the paper does well. The idea is to fine-tune a pretrained autoregressive model with local permutations of token order, teaching it previous-token-prediction (PTP) conditionals, then at inference resample a window of past tokens. This specific combination—local window swaps, relative positional embeddings, and a lightweight fine-tuning stage on a standard AR model—is not in the cited insertion/diffusion work. The training scheme is clean, they run a synthetic experiment that shows the mechanism works when PTP error is lower, and they do a proper large-scale fine-tune with a sensible baseline. On most benchmarks the fine-tuned model with NTP sampling (k=0) already matches or beats the fully trained AR model, which is a nice sanity check.\n\nWhere it gets soft. The load-bearing theoretical claim (Theorem 1, Eq. 18–20) reduces to an assumption that PTP error is smaller than NTP error in L∞/L1 norm. The evidence for that is indirect. Figure 3 compares cross-entropies, but conditioning on a future token always lowers entropy, so a lower PTP cross-entropy is expected even if the learned conditionals are no more accurate in total variation. The paper's 'near-zero entropy' assumption does not hold for real token distributions. Table 2's 'empirical TV distance' is 1 − p̂(x_i | x_<i) against the observed validation token, which is a confidence score, not a distance to the true conditional. κ in Eq. 18 is never measured. So ρ<1 is unestablished and the theorem does not do the advertised work. The empirical claim is also thinner than the abstract: no error bars, best-of-three temperature selection, gains mostly 2–5% relative rather than 10%, and one benchmark (PHP) regresses at k=1. 'Preserving speed' is misleading; RPT costs extra iterations.\n\nBottom line. This is an empirical methods paper with a heuristic justification, not a proven theory. The recipe is worth testing and the fine-tuning results are suggestive. I'd send it to peer review, asking for multi-seed runs, fixed temperatures, released code, and a more careful error analysis. The stress-test's critique of the theory is accurate; the method still deserves attention.","headline":"A clean fine-tuning recipe for local token resampling that shows modest, uneven benchmark gains; the theory does not establish the claimed guarantee.","tokens_in":18002,"tokens_out":3011,"would_cite":true,"duration_ms":30888,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that allowing an autoregressive model to resample a short window of already-written tokens during decoding reduces sampling error and yields about ten percent relative gains on reasoning and coding benchmarks after a…","keywords":["resample previous tokens","corrector sampling","autoregressive language models","previous-token prediction","error accumulation","code generation benchmarks","reasoning benchmarks","Markov chain perturbation"],"falsifier":"Take a held-out corpus of the kind the paper evaluates on, compute the empirical PTP and NTP conditional cross-entropies after RPT fine-tuning, and estimate the RPT factor $\\rho$ from Equation (18); if $\\rho \\ge 1$ or the PTP cross-entropy is not below the NTP cross-entropy, the central premise is violated. A direct behavioral check is to compare total-variation distance to validation tokens for $k=0$ and $k=1$ RPT sampling: the paper predicts a decrease on each dataset, so a dataset where $k=1$ raises the TV distance would refute the claim for that setting.","tokens_in":16886,"feed_emoji":"🔄","tokens_out":10605,"duration_ms":95523,"temperature":0.7,"pith_summary":"Autoregressive language models are trained to predict the next token and, at inference, commit to each token from left to right with no chance to undo it. This paper proposes Resample-Previous-Tokens (RPT), a sampling procedure that repeatedly revisits a short window of already-generated tokens and replaces a token using a conditional that also sees the following token. The paper argues that this backward-looking prediction is systematically easier than next-token prediction, and proves that whenever that error gap holds, RPT's sampling-error bound is smaller than standard sampling's bound. To obtain the needed conditionals, the method fine-tunes an existing autoregressive model on permuted token orders with a small positional-embedding change. After fine-tuning an 8-billion-parameter model for only 100 billion extra tokens, RPT sampling yields roughly 10% relative improvements on coding and reasoning benchmarks over standard sampling.","feed_headline":"Letting an LLM revise its tokens lifts code and math scores ~10%","feed_subtitle":"A short fine-tuning adds backward prediction; resampling previous tokens then beats standard sampling.","key_machinery":"The load-bearing object is the RPT Markov chain on token pairs, with transition kernel $p(x_i', x_{i+1}'\\mid x_i, x_{i+1}) = \\pi(x_i'\\mid x_{i+1}) \\pi(x_{i+1}'\\mid x_i')$, whose stationary distribution is the joint that RPT sampling approximates; the comparison with NTP is made through perturbation bounds on that stationary distribution. The identity that carries the argument is the RPT factor $\\rho$ from Theorem 1: RPT's error bound beats NTP's exactly when $\\rho<1$. Training supplies the needed previous-token conditionals by randomly permuting each sequence (moving a token $w-1$ places right with probability $q$, on a random subset), feeding the model the permuted order plus input/target position indices, and training with the same cross-entropy loss against both next-token and previous-token targets; a learned relative positional embedding of size $w+1$ tells the model whether the target lies in the past or future.","core_discovery":"The paper's central claim is that the fixed left-to-right order of autoregressive sampling is a source of error that can be reduced without changing the model's next-token-prediction quality. RPT alternates two conditional samples inside a window: given the future token $x_{i+1}$, sample the earlier token $x_i$ from $\\hat p(x_i\\mid x_{<i}, x_{i+1})$, then resample $x_{i+1}$ from the standard next-token conditional, and iterate. The theoretical result, Theorem 1, says that in an asymptotic-error analysis RPT has a lower total-variation error bound than NTP whenever the RPT factor $\\rho = \\kappa(\\|\\epsilon_{i|i+1}\\|_\\infty + \\|\\epsilon_{i+1|i}\\|_\\infty) / (\\|\\epsilon_i\\|_1 + \\|\\epsilon_{i+1|i}\\|_\\infty)$ is below $1$, where $\\kappa$ is a Markov-chain condition number; the key empirical premise, supported by cross-entropy curves and a synthetic experiment, is that the previous-token error $\\|\\epsilon_{i|i+1}\\|_\\infty$ is markedly smaller than the next-token errors. The practical result is that fine-tuning a pretrained 8B autoregressive model for 100B tokens with the RPT objective produces consistent improvements on nine coding and reasoning benchmarks, around 10% relative over the fully trained standard-sampling baseline, while retaining KV-caching speed.","pith_inferences":["If the RPT mechanism generalizes beyond the benchmarks tested, the same fine-tuning recipe could be applied to instruction-tuned or RL-tuned models; the paper only demonstrates the effect on a plain autoregressive checkpoint, so this remains an extension.","A testable boundary is high-entropy text: in domains where the next token is nearly unpredictable, the previous-token prediction advantage may shrink or reverse, which would make the RPT factor exceed 1; running RPT on open-ended generation would probe that boundary.","Because the headline gains are reported as an oracle over three temperatures, a deployment-focused study with a single fixed temperature and a fixed iteration count would clarify the practical magnitude of the improvement.","The Markov-chain formulation suggests RPT can be seen as a small denoising loop that could be reused as a test-time corrector on top of other samplers; the paper does not explore that composition."],"forward_implications":["An existing pretrained autoregressive model can be upgraded with a short fine-tuning pass, and the resulting RPT sampler outperforms standard NTP sampling from the same model on all nine reported benchmarks.","Because the fine-tuned model's NTP loss stays close to the pretrained model's loss (about 0.02 higher at convergence), RPT training does not sacrifice the model's original next-token abilities.","The theory and loss curves indicate that using more future context helps: in the w=3 training objective, PTP-2 has lower cross-entropy than PTP-1, and both are below NTP.","In the synthetic experiment RPT already beats NTP when the previous-token error is only 75% of the next-token error, so the method does not require a near-perfect backward predictor."],"supporting_citations":[{"why":"Supplies the Markov-chain perturbation bound that converts the RPT kernel error into a bound on the stationary-distribution error.","marker":"Cho and Meyer, 2001"},{"why":"Provides the Markov-chain background and the ergodicity coefficient behind the conditional number used in the RPT factor.","marker":"Seneta, 2006"},{"why":"Gives optimal condition-number bounds for Markov chains, tightening the constant that appears in the RPT error bound.","marker":"Kirkland et al., 2008"},{"why":"Documents the failure modes of deterministic decoding that motivate keeping corrector sampling close to standard NTP sampling.","marker":"Holtzman et al., 2020"},{"why":"Supplies the oracle-temperature evaluation protocol used to compare methods fairly in the benchmark tables.","marker":"Gloeckle et al., 2024"},{"why":"Provides the HumanEval benchmark used to measure code-completion improvement.","marker":"Chen et al., 2021"},{"why":"Provides the MBPP benchmark used to measure few-shot code-generation improvement.","marker":"Austin et al., 2021"},{"why":"Provides the GSM8K benchmark used to measure mathematical-reasoning improvement.","marker":"Cobbe et al., 2021"},{"why":"Provides the MultiPL-E harness and translated benchmarks used for the C++, C#, PHP, Bash, Java, and TypeScript evaluations.","marker":"Ben Allal et al., 2022"}],"fun_headline_variants":["Revising tokens during sampling boosts LLM reasoning ~10%","LLM sampling that revises past tokens lifts code/math 10%","RPT: let LLMs resample previous tokens for ~10% better scores","Backward token resampling helps LLMs, ~10% gain on benchmarks","Sampling with token revision improves LLM coding and math ~10%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method works only if predicting a token while already seeing the next token is genuinely easier than ordinary next-token prediction; all of the benchmark gains depend on that empirical gap, which the theory assumes rather than proves.","fun_headline_variants_meta":{"raw":{"variants":["Revising tokens during sampling boosts LLM reasoning ~10%","LLM sampling that revises past tokens lifts code/math 10%","RPT: let LLMs resample previous tokens for ~10% better scores","Backward token resampling helps LLMs, ~10% gain on benchmarks","Sampling with token revision improves LLM coding and math ~10%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000788,"raw_usage":{"total_tokens":3474,"prompt_tokens":946,"completion_tokens":2528,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":562,"completion_tokens_details":{"reasoning_tokens":2444}},"tokens_in":562,"tokens_out":2528,"duration_ms":19235,"temperature":1.0,"reasoning_tokens":2444,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T05:58:34.502239+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a held-out corpus of the kind the paper evaluates on, compute the empirical PTP and NTP conditional cross-entropies after RPT fine-tuning, and estimate the RPT factor $\\rho$ from Equation (18); if $\\rho \\ge 1$ or the PTP cross-entropy is not below the NTP cross-entropy, the central premise is violated. A direct behavioral check is to compare total-variation distance to validation tokens for $k=0$ and $k=1$ RPT sampling: the paper predicts a decrease on each dataset, so a dataset where $k=1$ raises the TV distance would refute the claim for that setting.","supporting_citations":[],"review_version":1}