{"id":"27ca58dc-4315-40e5-bf9f-c5b9c7ba6970","arxiv_id":"2507.12759","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"ThinkLogit blends logits from a small reasoning guider into a frozen 32B model, improving math pass@1 by up to 29% without training the large model.","lead":"This paper shows that a small, already-reasoning AI model can steer a much larger frozen model to think longer and solve more math problems by blending their word-scores during text generation. The trick could make advanced reasoning cheaper because the big model never needs retraining.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Missing direction control: the paper never tests whether the contrastive logit delta, rather than expert-logit ensembling or perturbation size, drives the gains; the central transfer claim is therefore unsupported.","rationale":"I read the paper in good faith. The method is simple and the reported gains are consistent across four datasets, and the budget-forcing ablation is a genuine attempt to rule out the trivial length explanation. The DPO variant explicitly acknowledges distribution mismatch, which lends some support to the transfer mechanism. However, the central claim depends on the delta's direction being meaningful in the 32B model's logit space. This is an empirical assumption that the paper never tests directly. The strongest evidence would come from a control that destroys the direction information while preserving the norm. If such a control performs as well, the paper's mechanism would be indistinguishable from generic logit perturbation or expert ensembling. Given that the reader already conditioned acceptance on additional evidence, my concern does not change the verdict; it sharpens the specific condition that should be met. I am not accusing the authors of anything; this is a standard missing-control issue.","tokens_in":15362,"tokens_out":14002,"duration_ms":162973,"concrete_test":"On AIME2025 and AMC23, rerun the ThinkLogit protocol (alpha=1, warm-up 100 tokens, same sampling) with the delta replaced by (i) -[logits(S_star) - logits(S)] and (ii) a same-norm vector with vocabulary entries randomly permuted. If either control matches the reported pass@1 (19.2 on AIME2025, 62.2 on AMC23), the direction is not the active ingredient; if both heavily degrade, the direction is necessary and the central transfer claim is supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that logit arithmetic elicits long reasoning rests on the assumption that delta = logits(S_star) - logits(S) is a meaningful direction in the target model's output space (Section 3). The paper provides no direct test of this. The budget-forcing ablation rules out length alone, and the alpha sweep (Table 2) only varies magnitude; neither isolates the semantic content or direction of delta. Notably, the alpha sweep is non-monotonic: on AMC23, both alpha=0.5 (36.6) and alpha=1.5 (41.9) fall below the 57.2 baseline, so only the single value alpha=1 works, raising the possibility that the effect is a narrow perturbation effect rather than a robust directional transfer. No control replaces delta with a same-norm random or negative vector, or omits the subtraction to test whether simple ensembling with the guider suffices. Without such a control, the reported gains could be attributed to the delta's norm, to the injection of math-specialized logits, or to a generic entropy increase, rather than to the long-reasoning direction the paper claims.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes THINKLOGIT, a decoding-time method that adds a scaled difference between the logits of a small long-CoT reasoning model (S*) and its base model (S) to the logits of a frozen large target model L, with the goal of eliciting long chain-of-thought reasoning without training L. A variant, THINKLOGIT-DPO, first fine-tunes the guider with Direct Preference Optimization on preference pairs sampled from both the target and guider models. Experiments on AIME2024, AIME2025, AMC23, and a MATH-hard subset with Qwen2.5-32B as target report average pass@1 improvements of 26% and 29% over the base model for the two variants, plus a 13% improvement when the guider is an RFT-trained 1.5B model. The paper includes ablations on guidance strength, warm-up, budget forcing, and DPO data construction.","tokens_in":15606,"tokens_out":6555,"duration_ms":67554,"significance":"If the reported results hold, the paper demonstrates a cheap and practical way to recover a substantial share of long-reasoning gains from full fine-tuning by transferring a contrastive logit direction from a small reasoning model to a larger frozen model. The strengths are the simplicity of the method, the public code release, the consistent direction of the gains across all four datasets for the SFT guider, and the budget-forcing control, which credibly rules out the trivial explanation that the gains come from output length alone. The DPO ablations comparing pair types and SFT alternatives are informative. However, as detailed below, the central transfer claim needs a direction control, and the headline numbers currently lack statistical support because alpha is selected on the test sets and no confidence intervals are reported. With those additions the contribution would be solid.","major_comments":[{"comment":"The paper's central transfer claim is not tested against any direction control. The method is equivalent to adding alpha times the contrastive vector Δ = ℓ(S*) − ℓ(S) to the target logits, and the interpretation that this 'encodes the probability shift that turns a short-CoT model into a long-CoT one' presupposes that this direction transfers across architectures. The ablations in Tables 2 and 3 vary alpha, warm-up, and DPO data, but none removes or inverts the subtraction, replaces Δ with a same-norm random vector, or compares against simply adding ℓ(S*) without subtracting ℓ(S). The non-monotonic alpha sweep in Table 2 (on AMC23, alpha=0.5 gives 36.6 and alpha=1.5 gives 41.9, both below the 57.2 baseline) makes it especially important to show that the effect is due to the semantic direction rather than to a narrow perturbation magnitude. Adding these controls is necessary to support the abstract's claim that logit arithmetic elicits long reasoning.","section":"Section 3 (definition of ˜ℓ_{t+1})"},{"comment":"The guidance strength alpha=1 is selected on the evaluation sets themselves. The ablation in Table 2 is reported on AIME2025 and AMC23, which also appear in the main results in Table 1, and no validation split or nested selection procedure is described. The headline improvement of 26%/29% therefore partly reflects test-set selection of alpha. Please report how alpha and T_warmup were chosen, or perform selection on a hold-out set and report the corresponding test results.","section":"Section 4.1 / Table 2"},{"comment":"No error bars, confidence intervals, or significance tests are reported, despite the fact that AIME2024 and AIME2025 contain only 30 problems each and AMC23 contains 40 problems. A single pass@1 difference of a few problems (e.g., AIME2024: 14.6 vs 22.5) can be within sampling noise, especially with 8 samples per problem. The paper should report bootstrap or exact binomial confidence intervals, and ideally multiple decoding seeds, before claiming that the 26%/29% relative improvement is robust.","section":"Table 1"}],"minor_comments":[{"comment":"The 'Average' column for pass@8 omits MATH-hard, so the pass@8 averages are over three datasets rather than four; the text should state this explicitly to avoid implying a four-dataset average.","section":"Table 1"},{"comment":"Section 4.1 says 10K preference pairs are randomly selected from a total of 50K, while Appendix A.2 reports 11,974 + 43,209 = 55,183 pairs; these numbers should be reconciled.","section":"Section 4.1 / Appendix A.2"},{"comment":"The relationship between lambda and the description 'directly concatenating datasets as DPO training data' could be clarified; if lambda = |D1|/(|D1|+|D2|), the weighted sum equals the average over the union, but this equivalence is not stated.","section":"Equation (1)"},{"comment":"The related-work discussion says THINKLOGIT 'sidesteps these constraints entirely' compared with representation-engineering methods, but THINKLOGIT still requires white-box logit access to the base and guider small models; the sentence should be softened.","section":"Section 2"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a useful empirical paper that applies established logit arithmetic (DExperts, proxy tuning) to a new target -- eliciting long chain-of-thought in a frozen large model with a small guider. The main new ingredient is DPO-aligning the guider to the target, and the results are consistent across four math benchmarks. The budget-forcing ablation is a good control: lengthening outputs alone hurts, so the gains are not just about length. I also like the RFT transfer result. The paper is honest about its lineage.\n\nThe soft spots are real but not fatal. The stress-test note is right: there is no control that isolates the contrastive direction. Adding logits(S*) without subtracting logits(S) -- or adding a random/negative delta with the same norm -- would tell you whether the effect comes from the delta direction or just from injecting math-specialized logits. The alpha sweep is non-monotonic (on AMC23, both 0.5 and 1.5 underperform the base), which makes the alpha=1 result look a bit fragile. And alpha is tuned on AIME2025 and AMC23, which are then reported in the main table -- that's a selection effect, even if modest. The transfer of the delta from a 1.5B model to a 32B model also remains an assumption; the DPO variant partially addresses distribution mismatch, but the paper never directly tests cross-model transfer.\n\nAlso missing: error bars or significance tests, and a comparison to other decoding-time methods like contrastive decoding or DoLa. The 'SFT on target' baseline is weak (LoRA, 10K examples), so the claim that direct fine-tuning fails needs a stronger check.\n\nOn balance, the central empirical claim -- a 1.5B guider improves a frozen 32B model's math pass@1 by roughly 26-29% -- holds up as a phenomenon. Whether the mechanism is truly directional transfer is not established. This is a solid submission for a main conference with revisions. I'd send it to a serious referee, primarily to ask for the missing controls and statistical rigor.","headline":"A cheap, practical way to elicit long CoT from a frozen large model using a small guider; worth a referee, but the direction control is missing and alpha tuning on test sets is a real weakness.","tokens_in":16146,"tokens_out":2948,"would_cite":true,"duration_ms":32070,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A frozen 32B language model can be steered into long chain-of-thought reasoning by adding the logit difference between a small reasoning model and its base.","keywords":["logit arithmetic","chain-of-thought reasoning","decoding-time steering","frozen language models","mathematical reasoning","preference optimization","small-to-large model transfer","reasoning distillation"],"falsifier":"Apply the identical ThinkLogit setup to a 32B target from a different pretrained family whose tokenizer does not overlap with the 1.5B guider's, after aligning the vocabularies by projection; if the 26% relative pass@1 gain disappears or turns negative, the claim that the small-model delta transfers to arbitrary large models is false. A cheaper control: add a sign-reversed delta ($-\\alpha$ instead of $+\\alpha$) or a random delta of the same magnitude; if either also improves accuracy, the specific long-vs-short direction is not what carries the gain.","tokens_in":15198,"feed_emoji":"🧮","tokens_out":7597,"duration_ms":71595,"temperature":0.7,"pith_summary":"The paper asks whether a large language model can be made to reason at length—backtracking, checking, and correcting itself—without any training of that model. It proposes ThinkLogit, a decoding-time method that adds, at every token step, the difference between the logits of a small reasoning model and the logits of its small non-reasoning base to the logits of a frozen large target. On four mathematical benchmarks this raises the target's pass@1 (the fraction of single-attempt answers that are correct) by 26% relative, and by 29% when the small guider is first aligned to the target with preference optimization (ThinkLogit-DPO). The same recipe transfers reasoning skills learned by reinforcement learning in a 1.5B model, improving the 32B target by 13%. The point is that long chain-of-thought behavior can be treated as a steerable direction in score space rather than something that must be trained into each large model.","feed_headline":"Small model's logits unlock long reasoning in a frozen 32B model","feed_subtitle":"Adding a 1.5B model's long-vs-short logit difference lifts math pass@1 by 26% with no training of the 32B target.","key_machinery":"The mechanism is logit arithmetic on the pre-softmax score vectors: at each step after a 100-token warm-up, the target's logits are replaced by $\\tilde{\\ell}_{t+1} = \\ell^{(L)}_{t+1} + \\alpha(\\ell^{(S^\\star)}_{t+1} - \\ell^{(S)}_{t+1})$, with $\\alpha = 1$. The delta $\\ell^{(S^\\star)} - \\ell^{(S)}$ is the \"become a long reasoner\" direction computed from a small model pair, and the paper's premise is that this direction is portable to a larger model. ThinkLogit-DPO is a second mechanism: it first trains the small guider with DPO on two kinds of preference pairs—cases where the large model's short answer is right and the small model's long one is wrong, and the reverse—so the guider learns to preserve the target's strengths while fixing its errors, and then applies the same logit arithmetic with the improved guider.","core_discovery":"The central claim is that long chain-of-thought reasoning is not a capacity that has to be installed in a large model by fine-tuning; it can be elicited by steering the frozen model's next-token scores. Concretely, the paper shows that the vector difference between a small long-CoT model's logits and its short-CoT base model's logits—the shift that makes the small model generate long, self-correcting reasoning—can be added to a much larger model's logits at decoding time, transferring the behavior with no weight updates. The paper further claims that this transfer is not an artifact of longer outputs: a budget-forcing control that simply makes the large model write more tokens lowers accuracy, while the logit-guided model improves it. On AIME2024, AIME2025, AMC23, and a hard subset of MATH500, ThinkLogit raises the frozen Qwen2.5-32B's average pass@1 from 32.7 to 41.2, and ThinkLogit-DPO to 42.2; with an RL-trained 1.5B guider the gain is from 32.7 to 36.8.","pith_inferences":["Not in the paper: if the same logit-delta direction transfers across model families and tokenizers, the recipe could be applied to targets whose weights are frozen or inaccessible, but only when the target exposes logits; API models that hide logits would need a different interface.","Not in the paper: the shared Qwen ancestry of the target and guiders may make the score-space alignment easier than in cross-family pairs; testing with a target from a different pretrained family would bound the transfer claim.","Not in the paper: an adaptive guidance strength $\\alpha$ that fades out once the trace enters a stable reasoning pattern could reduce the overlong, repetitive outputs the paper notes, and might make the gains larger than the fixed $\\alpha=1$ used here."],"forward_implications":["Long reasoning can be added to a frozen large model at inference time for roughly the cost of one extra forward pass of a much smaller model, with no gradient updates to the large model.","Reasoning skills developed cheaply in small models—by SFT on distilled traces or by RL on a handful of questions—can be deployed in a larger model without re-training it.","Preference-aligning the guider to the target recovers a large share of the gains of full fine-tuning: ThinkLogit-DPO reaches pass@8 of 65.6 versus 67.5 for a 32B model fine-tuned on 1K distilled examples.","The method broadens the reasoning boundary, since pass@8 improves more than pass@1, indicating that more of the sampled solution space becomes correct.","Since a length-only control fails, output length alone is not the active ingredient; the content direction of the logit delta matters."],"supporting_citations":[{"why":"Supplies the logits-arithmetic recipe of blending output distributions that ThinkLogit applies to the guider/target setup.","marker":"[Liu et al., 2024]"},{"why":"Supplies DPO, the preference-optimization objective used to align the guider with the target in ThinkLogit-DPO.","marker":"[Rafailov et al., 2023]"},{"why":"Supplies R1-Distill-Qwen-1.5B, the SFT-trained guider used for the main ThinkLogit results.","marker":"[DeepSeek-AI et al., 2025]"},{"why":"Supplies One-Shot-RLVR-1.5B, the RL-trained guider used to show that reinforcement-learned skills transfer.","marker":"[Wang et al., 2025b]"},{"why":"Supplies the s1 baseline and budget forcing, the length-only control that the ablation contrasts against.","marker":"[Muennighoff et al., 2025]"},{"why":"Supplies the MATH training split from which the preference pairs for ThinkLogit-DPO are sampled.","marker":"[Hendrycks et al., 2021]"},{"why":"Supplies the MATH500 benchmark whose hardest problems form the MATH-hard evaluation split, plus the grading script for answer correctness.","marker":"[Lightman et al., 2024]"},{"why":"Supplies GRPO, the reinforcement-learning algorithm behind the RL-trained guider's training.","marker":"[Shao et al., 2024]"}],"fun_headline_variants":["Logit trick transfers long reasoning from 1.5B to frozen 32B","Train-free logit arithmetic elicits long reasoning in 32B","Small model's logit shift gives 32B a 26% math boost","ThinkLogit: No-training method boosts 32B math by 26%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the direction in which a small model's word-scores change when it is turned into a long reasoner is the same direction that makes a much larger model reason at length, so adding that difference to the large model's scores nudges it correctly.","fun_headline_variants_meta":{"raw":{"variants":["Logit trick transfers long reasoning from 1.5B to frozen 32B","Train-free logit arithmetic elicits long reasoning in 32B","Small model's logit shift gives 32B a 26% math boost","ThinkLogit: No-training method boosts 32B math by 26%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000924,"raw_usage":{"total_tokens":4019,"prompt_tokens":1064,"completion_tokens":2955,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":680,"completion_tokens_details":{"reasoning_tokens":2879}},"tokens_in":680,"tokens_out":2955,"duration_ms":22036,"temperature":1.0,"reasoning_tokens":2879,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T16:38:36.135905+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Apply the identical ThinkLogit setup to a 32B target from a different pretrained family whose tokenizer does not overlap with the 1.5B guider's, after aligning the vocabularies by projection; if the 26% relative pass@1 gain disappears or turns negative, the claim that the small-model delta transfers to arbitrary large models is false. A cheaper control: add a sign-reversed delta ($-\\alpha$ instead of $+\\alpha$) or a random delta of the same magnitude; if either also improves accuracy, the specific long-vs-short direction is not what carries the gain.","supporting_citations":[],"review_version":1}