{"id":"62d1ff72-7681-4a5e-bfa2-4953a53bf635","arxiv_id":"2607.19046","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Contrasting a teacher's token scores under light- and heavy-thinking prompts yields a training signal that cuts reasoning length roughly in half without hurting accuracy.","lead":"The paper trains a small vision-language model to reason more tersely by having a frozen teacher score each generated token under “light-thinking” and “heavy-thinking” prompts and using the score gap as a reward. Across nine benchmarks the student keeps or improves accuracy while cutting output length by more than half.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No evidence that the LT/HT contrast A_t correlates with token-level reasoning utility; the method may simply optimize style, leaving the claimed 'efficient reasoning' mechanism unvalidated.","rationale":"The reader's weakest assumption is that A_t is a reliable per-token preference for concise reasoning. I agree with that core concern, and I sharpen it into a concrete falsifiable prediction: the teacher's LT policy should itself be at least as accurate as the HT policy. This is the minimal condition for the contrast to be a proxy for utility. The paper's empirical results are strong and internally consistent, but they rely entirely on this unvalidated link. A simple teacher-rollout comparison would settle whether the direction of the contrast is grounded in task success. If the teacher under LT is less accurate, then the student's observed accuracy improvements would be an artifact of the base model and benchmarks, not of the proposed mechanism. The reader's conditional verdict is appropriate; my concern does not escalate to rejection because the empirical results are still plausible and the test could confirm the mechanism. However, it bolsters the need for the test before full acceptance.","tokens_in":17857,"tokens_out":11735,"duration_ms":144938,"concrete_test":"On a random sample of 500 prompts from the training set, generate greedy responses from the frozen teacher under the LT and HT instructions separately, and measure accuracy and mean length on the same benchmark set. If the teacher's LT responses are not at least as accurate as its HT responses (e.g., LT accuracy is lower than HT accuracy by more than 2 percentage points), then the direction of A_t is not aligned with task success, and the COPD gains cannot be attributed to 'efficient reasoning' rather than to style transfer. Conversely, if LT responses are comparably accurate and substantially shorter, the concern is mitigated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that the per-token advantage A_t = log π_φ(y_t|x_LT,y_<t) − log π_φ(y_t|x_HT,y_<t) (Eq. 3) provides a token-level preference for concise, efficient reasoning. However, A_t is only a log-likelihood ratio between two instruction-conditioned teacher policies. The PPO objective (Eq. 5) contains no task reward; the only link to task performance is the unstated assumption that tokens with higher A_t are more useful for correct, concise answers. The paper provides no token-level or even trajectory-level validation that A_t correlates with answer correctness or with the causal contribution of a token to the final answer. If the teacher's light-thinking distribution is not actually a better task-solving policy (e.g., because the light-thinking instruction degrades reasoning, or because the LT/HT difference mainly reflects stylistic preferences such as skipping verification or using more assertive language), then the student is being optimized toward a concise style, not toward efficient reasoning. The reported gains on nine benchmarks could then be an artifact of the student's base distribution and of these particular benchmarks, rather than a direct consequence of the contrastive mechanism. This is load-bearing because it is the only mechanism connecting the unsupervised objective to the claimed accuracy–efficiency improvements. The paper's ablation in Table 6 shows the contrast is needed for compression, but it does not show that the contrast encodes utility; it only shows that the difference between LT and HT scores produces shorter outputs.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes COPD, a contrastive on-policy distillation method for multimodal LLMs. At each student-generated token, a frozen teacher scores the same token under light-thinking (LT) and heavy-thinking (HT) instruction prefixes; the log-likelihood difference A_t = ℓ_LT − ℓ_HT (Eq. 3) is used as a clipped token-level advantage in a PPO-style objective (Eq. 5). The method is evaluated on nine multimodal benchmarks across four teacher–student configurations, with ablations including a self-distillation variant (COPSD), prompt-template robustness, data/scale generalization, and training efficiency. The reported results show substantial response-length reduction (e.g., 57.0% reduction vs ExOPD in the 8B→2B setting, Table 1) with accuracy maintained or improved, leading to large gains in the Acc@1K metric.","tokens_in":18199,"tokens_out":4348,"duration_ms":49548,"significance":"If validated, COPD would be a significant addition to reasoning-compression research: it derives a dense, token-level training signal without explicit length penalties or answer rewards, works across model scales, and extends to self-distillation. The core idea—contrasting two instruction-conditioned teacher distributions at student-visited states—is simple, elegant, and potentially generalizable beyond LT/HT to other behavioral attributes. The experimental scope is substantial, and the ablations (e.g., Table 6 showing the necessity of the contrast rather than a single LT score) are informative. However, the central assumption that A_t encodes reasoning utility, rather than merely stylistic differences between the two instruction contexts, is not validated. The current evidence is compatible with a style-imitation explanation, so the mechanism claim is not yet established. The paper also lacks code release, single-run results without variance, and full prompt templates.","major_comments":[{"comment":"The advantage A_t is defined purely as a log-likelihood ratio between two instruction-conditioned teacher policies. The PPO objective in Eq. (5) includes no task reward, correctness signal, or length penalty; the entire claimed mechanism—compression while maintaining accuracy—rests on the implicit assumption that tokens with higher A_t are more useful for correct concise reasoning. The paper provides no token-level or trajectory-level validation that A_t correlates with answer correctness or with the causal contribution of each token to the final answer. Table 6 shows that the contrast is necessary for compression, but it does not show that the contrast encodes utility; a stylistic difference (e.g., LT prompts yielding more assertive or less self-verifying language) would produce the same effect. To support the mechanism claim, the authors should provide direct evidence—for example, a co","section":"Sec. 2.3, Eq. (3); Sec. 2.4, Eq. (5)"},{"comment":"All reported results are single-run point estimates without standard deviations, confidence intervals, or significance tests. Some benchmark-level accuracy differences are small relative to likely run-to-run variance (e.g., 52.7 vs 52.3 in Table 1, or 60.4 vs 60.1), while Acc@1K amplifies length effects. The claim of 'consistent' improvements would be considerably stronger with at least three independent seeds per setting and reported variance. This is a load-bearing issue for the central accuracy-maintenance claim, since the method changes both the numerator and denominator of Acc@1K.","section":"Sec. 3, Tables 1, 2, 5"},{"comment":"The exact LT and HT instruction templates are not provided in the paper, despite being the entire source of the contrastive signal. Table 4 names 'Simple,' 'Complex,' and 'default' variants but does not include their text. Without the exact prompts, the method is not reproducible, and it is impossible to assess whether the templates themselves introduce confounds. The full templates must be included in an appendix; ideally, code should also be released.","section":"Sec. 3.3.3, Table 4; Sec. 3.1.2"},{"comment":"The COPSD result—that a frozen student snapshot provides a useful contrastive signal—is interesting and potentially the most generalizable claim. However, the offered explanation (reallocating probability mass to already-existing concise paths) is asserted rather than demonstrated. The same concern as in the first major comment applies: the self-contrast could be selecting style rather than utility. A direct test would be to measure whether the LT/HT contrast of the snapshot correlates with token-level correctness, and whether COPSD-compressed models retain accuracy on held-out tasks requiring genuine multi-step reasoning rather than simple fact recall.","section":"Sec. 3.3.4, Table 5"}],"minor_comments":[{"comment":"The pass-rate filter (retaining only samples with Pass Rate strictly between 0 and 1) is a nontrivial preprocessing choice that could affect both accuracy and length outcomes. Please justify this choice or provide an ablation without the filter.","section":"Sec. 3.1.1, footnote 1"},{"comment":"MMGist is authored by the same research group (Yuan et al., 2026) and included as one of the nine evaluation benchmarks. The paper should explicitly disclose this overlap and, ideally, add an externally maintained benchmark to the evaluation set.","section":"Sec. 3.1.3, Table 1"},{"comment":"Typo: 'two-side-clip' should be 'two-sided clip'.","section":"Sec. 2.3"},{"comment":"The two algebraic forms of Acc@1K should be written more carefully: the first expression as printed is ambiguous about whether the 1/|B| factor applies inside the denominator. Use a single unambiguous definition.","section":"Eq. (6)"},{"comment":"The LT/HT prompt fragments shown in the figures are nearly unreadable and are not defined in the captions. Please include the full instruction text in an appendix and reference it in the captions.","section":"Figures 1 and 2"}],"recommendation":"major_revision","confidential_remarks":"The paper is from the Qwen team and evaluates on MMGist, a benchmark from the same group. For a journal submission, independent evaluation on a non-affiliated benchmark would materially strengthen the generalizability claim. The lack of code release and single-run results also raise reproducibility concerns that I would encourage the editor to weigh in the revision decision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThis paper does one clever thing: for each token the student generates, a frozen teacher scores that token under a light-thinking and a heavy-thinking instruction, and the log-likelihood difference becomes a token-level advantage in a PPO update. That paired contrast is new relative to OPD variants that use a single teacher distribution or visual/logit differences. The empirical story is broad and consistent across nine benchmarks, four teacher–student settings, and a self-distillation variant. I also credit the Table 6 ablation, which shows the contrast itself is doing the work: a single LT likelihood doesn't compress; LT minus HT does.\n\nThe soft spot is the interpretation. The paper never validates that A_t tracks whether a token is useful for correct, concise reasoning. The objective has no task reward, so the only link between the training signal and the reported accuracy–length gains is the assumption that higher-A_t tokens are the useful ones. That assumption could be true, but it could also be false—the LT/HT difference might mostly reflect style, such as skipping verification or adopting a more assertive tone. The benchmark results show accuracy doesn't suffer, and often improves, but they don't establish the mechanism. If the student's base distribution already contains short correct answers, the method may just be reselecting them. The case study is one example, not evidence.\n\nOther issues are smaller. All results are single-run with no error bars, no released code, and the LT/HT prompts are only in the appendix. One of the nine benchmarks, MMGist, is from the same group; that's minor since the other eight are external. There's also no baseline that isolates an explicit length penalty from the contrastive signal, so the 'no length penalty' claim is supported but not directly tested.\n\nOn balance, I'd send this to peer review. It's a simple, plausible method with strong empirical support. A referee should ask for code, multiple seeds, and a direct analysis of A_t—whether token-level A_t predicts answer correctness or correlates with human judgments of useful reasoning. If that analysis weakens the interpretation, the paper still stands as a practical engineering contribution. If it lands, it's a real mechanistic result.","headline":"Paired LT/HT teacher scoring gives a genuinely new distillation signal that compresses outputs across the board, but the claim that A_t encodes token-level reasoning utility is assumed, not demonstrated.","tokens_in":18701,"tokens_out":3339,"would_cite":true,"duration_ms":37169,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that scoring each student-generated token under light- versus heavy-thinking prompts and using the log-likelihood difference as a PPO advantage signal compresses reasoning length while preserving or improving accuracy.","keywords":["on-policy distillation","reasoning compression","token-level advantage","contrastive learning","chain-of-thought efficiency","multimodal language models","PPO","self-distillation"],"falsifier":"A concrete falsifier: run the same training pipeline with the LT and HT instruction templates swapped, or with two arbitrary wordings that are semantically unrelated to reasoning effort; if the response-length reduction persists, the signal is not about reasoning-mode preference. A stronger check is to label each token by whether deleting it preserves the teacher's correct answer and test whether A_t predicts that label.","tokens_in":17750,"feed_emoji":"🧠","tokens_out":5918,"duration_ms":57803,"temperature":0.7,"pith_summary":"This paper is trying to establish that on-policy distillation fails to compress reasoning because matching a single teacher distribution cannot distinguish a token that advances a concise answer from one that merely prolongs the response. COPD replaces that single signal with a paired contrast: a frozen teacher scores the same student-generated token under a light-thinking and a heavy-thinking instruction, and the difference in log-probabilities becomes a token-level advantage for a clipped policy-gradient update. The paper reports that this contrastive signal shortens outputs by roughly half to two-thirds while maintaining or improving accuracy on nine multimodal benchmarks, across several student and teacher sizes, and without any explicit length penalty or answer-level reward. A variant that uses a frozen snapshot of the student itself as the scorer also compresses reasoning, which the authors interpret as re-weighting reasoning modes already present inside the model. A sympathetic reader would care because it proposes a dense, label-free preference signal for making reasoning models cheaper to run.","feed_headline":"Light-vs-heavy teacher contrast halves LLM reasoning length","feed_subtitle":"Token-level difference in teacher log-probabilities doubles accuracy per thousand tokens on nine benchmarks.","key_machinery":"The mechanism is the paired teacher-scoring contrast: the same frozen teacher evaluates each student-generated token under two instruction contexts, light-thinking and heavy-thinking, at the same input and prefix. The difference A_t = ℓ^LT_t − ℓ^HT_t is used as a detached, clipped token-level advantage inside a clipped policy-gradient objective (Eq. 5). Its role is to convert a single teacher distribution into a state-matched comparison between reasoning modes, so the student learns which tokens are preferred under compact reasoning rather than merely imitating token probabilities.","core_discovery":"The central claim is that the teacher's log-probability contrast across reasoning-effort instructions is a valid per-token preference signal for efficient reasoning. For each on-policy token y_t, the paper defines ℓ^LT_t = log π_φ(y_t | x^LT, y_<t) and ℓ^HT_t = log π_φ(y_t | x^HT, y_<t), then sets A_t = ℓ^LT_t − ℓ^HT_t, clipped to a bounded range and detached. A positive A_t means the frozen teacher judges the token more compatible with minimal-effort reasoning; a negative value means it favors exhaustive continuation. The student is trained by maximizing a clipped PPO-style objective weighted by A_t, which raises probability of light-favored tokens and lowers heavy-favored ones. The authors","pith_inferences":["Editorial inference: if the compression signal survives arbitrary instruction-pair swapping, then the mechanism is not about reasoning effort per se but about any stable contrast between two context distributions; that would broaden the method's scope but weaken its semantic interpretation.","Editorial inference: a direct token-utility validation — checking whether suppressed tokens can be deleted from the teacher's own complete answers without changing correctness — would settle whether A_t tracks contribution-to-answer or merely stylistic mode preference.","Editorial inference: the success of self-distillation suggests the approach may be closer to policy selection (re-allocating probability mass to already-supported concise paths) than to knowledge transfer, connecting it to preference-optimization methods that do not need a stronger model.","Editorial inference: the non-monotonic teacher-scaling results imply that the contrast itself, not teacher capability, carries the compression signal; a testable corollary is that even a weaker but similarly-styled teacher should produce comparable compression."],"forward_implications":["If the contrastive advantage is a valid preference signal, reasoning compression can be trained without length penalties, token budgets, accuracy rewards, or curated concise traces.","Because the signal is dense and on-policy, training rollouts themselves get shorter, and the paper reports roughly half the GPU hours of standard OPD baselines.","Accuracy per output token (Acc@1K) can be doubled or better on the tested benchmarks, shifting the accuracy–length frontier rather than trading one for the other.","The self-distillation variant implies that a model can distill its own reasoning toward lightweight modes without any external teacher, as long as its frozen snapshot exhibits the LT/HT contrast.","The framework generalizes in principle to other paired behavioral modes, though the paper only validates reasoning efficiency."],"fun_headline_variants":["Token-level teacher contrast boosts reasoning efficiency","COPD: contrastive on-policy distillation cuts reasoning length","Teacher log-prob contrast halves LLM reasoning length","On-policy contrastive distillation trims LLM outputs","Contrasting teacher instructions yields concise reasoning"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that A_t = ℓ^LT_t − ℓ^HT_t measures a token's genuine contribution to efficient correct reasoning, rather than the teacher's stylistic response to prompt wording.","fun_headline_variants_meta":{"raw":{"variants":["Token-level teacher contrast boosts reasoning efficiency","COPD: contrastive on-policy distillation cuts reasoning length","Teacher log-prob contrast halves LLM reasoning length","On-policy contrastive distillation trims LLM outputs","Contrasting teacher instructions yields concise reasoning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000625,"raw_usage":{"total_tokens":2763,"prompt_tokens":808,"completion_tokens":1955,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":552,"completion_tokens_details":{"reasoning_tokens":1891}},"tokens_in":552,"tokens_out":1955,"duration_ms":14986,"temperature":1.0,"reasoning_tokens":1891,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T13:34:35.574303+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete falsifier: run the same training pipeline with the LT and HT instruction templates swapped, or with two arbitrary wordings that are semantically unrelated to reasoning effort; if the response-length reduction persists, the signal is not about reasoning-mode preference. A stronger check is to label each token by whether deleting it preserves the teacher's correct answer and test whether A_t predicts that label.","supporting_citations":[],"review_version":1}