{"id":"cdbd7925-61d4-4d6a-b456-9d8304fb1670","arxiv_id":"2510.08647","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"UCoT compresses chain-of-thought reasoning by feeding a large language model a compact continuous 'upfront thought' generated by a small model, cutting output tokens roughly in half on math benchmarks while staying close to original accuracy.","lead":"The paper proposes UCoT, a two-model setup in which a small 'compressor' turns a question into a compact set of continuous 'upfront thought' embeddings, and a large 'executor' model is trained to use those embeddings to answer with a much shorter chain of thought. On GSM8K and MATH-500 it reports roughly 50% token reduction with accuracy within a few points of the original model, beating the Tokenskip baseline.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Decoded UTs show structural rather than semantic content, so the 'high-fidelity compression' mechanism is unsupported; a structural-control experiment is needed to see whether the accuracy claim depends on the claimed semantic preservation.","rationale":"The reader's weakest assumption is that L_sem acts as a faithful proxy for preserving reasoning information, and the paper's Appendix C.6 evidence directly challenges that assumption. I agree this is the most load-bearing concern because it targets the paper's stated contribution of high-fidelity CoT compression rather than merely the presence or absence of error bars. However, the central empirical claim—UCoT maintains accuracy while reducing output length relative to the reported baselines—can survive even if the mechanism is structural rather than semantic. The ablations show both UT input and the reward factor matter, so the method is not vacuous. Thus I do not think this concern should change the reader's CONDITIONAL verdict; it should remain conditional pending either the structural-control experiment or a clearer mechanistic account. The other issues (missing error bars, unaccessibile code, absent SoftCoT/CCoT comparisons) are secondary and addressable, but the semantic-vs-structural discrepancy is the one that, if confirmed, would most weaken the paper's central claim of a high-fidelity compression mechanism.","tokens_in":27235,"tokens_out":10102,"duration_ms":95213,"concrete_test":"Train a control executor using the same UTU procedure but replace the projected UT with a discrete structural template derived from the original CoT—for example, the decoded UT with all numbers removed, or an operation skeleton ('divide, then add, then multiply')—while keeping the reward factor R, the Cutoff function, and the length-compression ratio identical. Compare accuracy and output tokens on GSM8K and MATH-500. If the control matches UCoT within about 1 point, the semantic content of the UT is not load-bearing and the 'high-fidelity compression' claim fails. If the control is substantially worse, the UT encodes more than structure and the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's empirical result—UCoT improving over Tokenskip while cutting output tokens—may be real, but the paper's own Appendix C.6 undercuts the mechanism offered for it. The decoded UTs are not faithful reconstructions of the original CoT: they contain arithmetic errors (e.g., '1/100 = 10') and the authors state they provide 'structural information about reasoning rather than semantic information.' This directly contradicts the §3.1 claim that the compressor is trained to 'cover the information of an entire high-quality reasoning path' and calls into question whether L_sem in Eq. (6) actually tracks reasoning content. If UTs contribute only a generic reasoning schema, then the accuracy advantage over Tokenskip may come from the reward factor R in Eq. (7) and the length-cutoff training, not from high-fidelity compression of the CoT. In that case contribution 1 ('High-fidelity CoT Information Compression') is overstated, and UCoT reduces to a soft structural prompt plus reward-based fine-tuning. The central empirical numbers would still stand, but the claimed mechanism and the novelty of the compression approach would not.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes UCoT, a two-stage compressor-executor framework for chain-of-thought (CoT) compression. A small model (compressor) is trained to generate a fixed-length sequence of continuous embeddings (Upfront Thoughts, UT) from a question, and a larger model (executor) is trained to use these UT embeddings to produce a shorter CoT while maintaining accuracy. The first training stage (UTG) optimizes the compressor to reconstruct the original CoT from the UT via a language-modeling loss; the second stage (UTU) trains the executor with a semantic loss that aligns hidden states of the UT-conditioned generation with those of the original CoT, multiplied by a reward factor that penalizes incorrect answers. Experiments on GSM8K, MATH-500, and additional long-CoT tasks (GPQA, HumanEval, AIME, ASDiv) compare UCoT against Prompt, Truncation, CoD, and Tokenskip baselines at compression ratios 0.9, 0.7, and 0.5. The main empirical claim is that UCoT achieves higher accuracy than the SOTA method Tokenskip at comparable or lower token counts, e.g., on GSM8K with Qwen2.5-7B at ratio 0.5, accuracy is 86.55% versus 83.47% for Tokenskip while using fewer tokens (140.36 vs 157.58).","tokens_in":27505,"tokens_out":4490,"duration_ms":40531,"significance":"If the empirical findings hold, UCoT is a practical and relatively simple method for CoT compression that outperforms existing baselines on standard benchmarks while generalizing to long-reasoning models and out-of-domain tasks. The paper's strengths include a clear two-stage training pipeline, extensive comparisons across model families and datasets, ablations showing the importance of the UT and reward factor, and analysis of UT length and information volume. The method is reproducible in principle from the described details. However, the paper's central conceptual contribution—'High-fidelity CoT Information Compression'—is not supported by its own evidence: the decoded UTs in Appendix C.6 are described as providing 'structural information about reasoning rather than semantic information' and contain arithmetic errors. This undermines the claim that Eq. (6)'s semantic loss actually preserves reasoning content, and it calls into question the novelty of the compression mechanism as distinct from a soft structural prompt plus reward-based fine-tuning. The empirical accuracy improvements over Tokenskip are still plausible, but the paper's framing and internal consistency need substan","major_comments":[{"comment":"The paper claims in §3.1 that the compressor is trained to 'cover the information of an entire high-quality reasoning path' and lists 'High-fidelity CoT Information Compression' as Contribution 1. However, Appendix C.6 (Figures 4-5) explicitly states that the decoded UTs provide 'structural information about reasoning rather than semantic information' and shows decoded examples with arithmetic errors such as '1/100 = 10'. This is a direct contradiction. Since Eq. (6) (the L_sem loss) is the only mechanism aligning UT-conditioned hidden states with the original CoT, the paper needs to either provide evidence that hidden-state alignment tracks answer-relevant semantic content (e.g., probing, intervention, or controlled decoding experiments) or revise the contribution/framing to a structural-prompt-plus-reward method. As written, the mechanism for the accuracy improvement is unsupported.","section":"§3.1 vs Appendix C.6"},{"comment":"The abstract and introduction state that UCoT's performance on GSM8K is '3.08% higher than that of the SOTA method.' This phrasing omits two facts that are essential for a fair reading: the comparison is to Tokenskip at compression ratio 0.5, and UCoT is 5.62 points below the uncompressed executor (92.17% to 86.55%). 'Maintains the powerful reasoning ability of executor' is an overstatement given a 5.62-point drop. The relative comparison should be reported alongside the absolute drop from the original model, and the wording should be adjusted to avoid implying that UCoT is 3.08% more accurate than the original.","section":"Abstract and §4.1"},{"comment":"All experimental results are said to be averaged over 5 random seeds, but no standard deviations, confidence intervals, or significance tests are reported. The headline margins are small in several cases (e.g., 0.17 points on HumanEval in Table 3, 0.45 points at ratio 0.9 in Table 1) and may be within seed variance. Please report per-seed variability and, where possible, paired significance tests for the main UCoT-vs-Tokenskip comparisons. This is needed to support the empirical claims quantitatively.","section":"Tables 1–3, Appendix A.3"},{"comment":"Token Gain and Information Gain are computed by decoding the UT with the compressor, but the downstream consumer is the executor, not the compressor. The positive correlation in Table 4 may reflect the compressor's ability to decode its own embeddings rather than the information actually available to the executor. To substantiate the claim that 'information volume in UT matters' for the executor, the authors should provide executor-side diagnostics, e.g., measuring the effect of UT information on executor hidden states or accuracy under controlled compressor checkpoints. The table also has a duplicated 'Latency' column header.","section":"§4.4, Table 4"}],"minor_comments":[{"comment":"Typos and formatting issues: 'intergated' (should be 'integrated'), 'satge' (stage), 'prmopt' (prompt), 'th number' (the number), 'Qwen2.5-7b-Instrust', 'middel' (middle), 'MA TH-500' in Figure 3, and 'Tokens'/'Lantency' in table captions.","section":"Throughout"},{"comment":"In the pseudocode for Upfront Thought Utilization, line 10 says 'Compute loss L_c' but should be 'Compute loss L_e' (the executor loss defined in Eq. (8)).","section":"Algorithm 2, line 10"},{"comment":"The citation '(Kang et al.)' in the Introduction lacks a year; it should be '(Kang et al., 2024)' to match the reference list.","section":"Introduction, Related Work"},{"comment":"The Prompt baseline at ratio 0.9 reports ActRatio 1.02-1.04, indicating it does not actually compress the CoT. The paper acknowledges this in §4.1, but it would be clearer to mark such rows explicitly as failing to meet the target compression ratio.","section":"Table 1"},{"comment":"The figure legend includes a stray '1' and inconsistent labeling; the y-axis and legend should be cleaned up for readability.","section":"Figure 3"}],"recommendation":"major_revision","confidential_remarks":"The central empirical comparison appears credible and the proposed framework is interesting, but the paper's own appendix undermines the central 'high-fidelity compression' claim. I recommend major revision: the authors should either provide direct evidence that the semantic loss preserves answer-relevant reasoning content or substantially reframe the contribution as a structural-prompt-plus-reward method. Adding seed-level variance and significance tests is also necessary to support the small margins in several comparisons. I do not see a fundamental flaw in the experimental setup, so the issues are fixable within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The thing to know up front: the empirical core of this paper is real, and the mechanism story is not. UCoT cuts output tokens roughly in half on GSM8K while staying within a few points of the uncompressed model, and it beats the reported Tokenskip baseline across several ratios and models. That is a useful result for anyone working on inference efficiency. The ablations are honest: removing the upfront thoughts or the reward factor drops accuracy substantially, so the main components matter. The extra generalization tests on long-CoT models like Qwen3 and R1-distill are also a plus, and the limitations section is refreshingly straightforward.\n\nThe genuine novelty is the two-stage design: a small compressor trained to emit fixed-length continuous thoughts, then an executor trained with a truncation-based objective plus a reward-weighted semantic loss. It is a new combination, and the paper shows it works better than the discrete-prompt and dataset-compression baselines. What it does not do is show the compression is high-fidelity. The decoded UT case studies in Appendix C.6 explicitly say the UTs provide structural rather than semantic information, and the examples contain arithmetic errors like “1/100 = 10.” That undercuts the “cover the information of an entire high-quality reasoning path” claim in Section 3.1. The semantic loss may just be aligning hidden states at a coarse structural level, and the real gains could be coming from the reward factor plus the length-cutoff training. This does not kill the paper, but the mechanism section needs to be scaled back or tested with a structural-control condition.\n\nA few smaller soft spots: there are no error bars or significance tests, and the numbers in Table 1 are all point estimates. The abstract’s “3.08% higher” is compared against Tokenskip, which obscures the 5.62-point drop from the original model; that should be stated plainly. The closest continuous-thought baselines, SoftCoT and Compressed CoT, are cited but not empirically compared, which is a gap given the method is directly building on that line. And the promised code and dataset are not available, which makes independent verification harder.\n\nWho is this for? Researchers working on LLM inference efficiency, latent reasoning, or small-model/large-model cooperation. They will get a concrete method and a fair set of experiments. The paper should go to peer review — not desk reject — but it needs revision: the mechanism claim, the missing error bars, and the absent code all need to be addressed before it is publishable.","headline":"Solid empirical contribution to CoT compression; the mechanism story is overstated and the headline numbers hide a real accuracy drop, but the method and ablations deserve review.","tokens_in":28020,"tokens_out":1537,"would_cite":true,"duration_ms":16894,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A small compressor can write the reasoning chain as a few dense 'thought' vectors, and a 7B executor can answer with about half the tokens while staying within a few accuracy points of the uncompressed model.","keywords":["Chain-of-Thought Compression","Continuous Embeddings","Soft Token Prompting","Compressor-Executor Framework","Reasoning Efficiency","Large Language Models","Post-Reasoning Paradigm","Upfront Thoughts"],"falsifier":"Take a trained UCoT system and replace the UT for a question with (a) the UT of a different question or (b) random vectors of the same length, then measure executor accuracy. If accuracy stays near the reported level, the specific semantic content of the UT is not what drives the result, contradicting the high-fidelity compression claim. A direct check: decode the UT to text and verify it matches the original CoT's arithmetic operations and quantities—the appendix examples already show mismatches such as '1/100 = 10'.","tokens_in":27117,"feed_emoji":"🧠","tokens_out":7989,"duration_ms":58023,"temperature":0.7,"pith_summary":"Chain-of-Thought (CoT) gives language models strong reasoning at the cost of long, token-hungry explanations. This paper asks whether the reasoning can be done up front: a small 'compressor' model converts the reasoning chain into a fixed sequence of dense continuous embeddings—called Upfront Thoughts—and the original large 'executor' model is trained to answer correctly from those embeddings plus a short continuation. The central claim is that this preserves most of the executor's reasoning ability while roughly halving the number of tokens it generates. On GSM8K, Qwen2.5-7B-Instruct drops from 298.6 to 140.4 average output tokens (compression ratio 0.5) with accuracy falling only from 92.17% to 86.55%—a result 3.08 points above the Tokenskip baseline. If correct, the paper shows that CoT compression is better framed as a two-model cooperation problem (compress-then-reason) than as a text-summarization or prompt-design problem.","feed_headline":"Front-loaded reasoning embeddings halve LLM output tokens","feed_subtitle":"A small compressor packs the reasoning chain into dense vectors; the 7B executor then answers with about half the tokens.","key_machinery":"Upfront Thought (UT): a fixed-length sequence of continuous embeddings, taken from the last hidden states of placeholder tokens in a small 'compressor' model, that condenses the reasoning chain. The load-bearing mechanism is the pair of training objectives in the Upfront Thought Utilization stage: the semantic loss Lsem forces the executor's last hidden states for the compressed input (UT plus a truncated short CoT) to match its hidden states for the original full CoT via mean absolute error, and the reward factor R multiplies that loss by the squared difference between the log-probability of the correct answer under the compressed input and under the original CoT. Together they train the ex","core_discovery":"The paper's central discovery is that a chain-of-thought can be compressed into a small set of continuous soft-token embeddings before answering, and a large model can be trained to consume those embeddings and still reason almost as well as if it had written the full chain itself. Concretely, a compressor model is trained with prompt-tuning placeholders to emit Upfront Thought (UT) embeddings from which it can regenerate the original CoT text; a projector maps the UT into the executor's embedding space; and the executor is trained with loss Le = Lsem × R, where Lsem is the mean absolute error between the executor's hidden states for the compressed input (UT + truncated CoT) and for the orig","pith_inferences":["The paper's own decoded-UT examples suggest the UT may convey structural reasoning schemas rather than exact arithmetic content (it decodes to a different question's quantities, with errors like '1/100 = 10'). A testable reading: feed the executor UTs decoded from unrelated questions and see whether accuracy holds; if it does, the accuracy gain comes from a consistent reasoning template, not high-","Because the reward factor multiplies the loss by (r_UCoT − r_CoT)^2, training concentrates on examples where the compressed path initially disagrees with the original path's answer likelihood. The method's benefit may therefore be concentrated on a subset of problems; reporting accuracy conditioned on UT-decoding success or problem difficulty would reveal where it actually helps.","The UT itself costs M hidden-state computations on the compressor, and the reported speedups already include that cost. A fair deployment comparison should also account for the fixed UT prefix added to every query, especially at high compression ratios (0.9), where the prefix may negate part of the output-token savings.","The method is trained only on GSM8K and evaluated zero-shot on AIME, GPQA, ASDiv, and HumanEval. A natural stress test is training on a broader corpus or varying UT length per domain; the paper shows a single fixed-length UT generalizes surprisingly well, which is itself a strong claim worth probing."],"forward_implications":["On GSM8K with Qwen2.5-7B-Instruct at compression ratio 0.5, UCoT produces about 53% fewer tokens (140.4 vs 298.6) with accuracy 86.55%, only 5.62 points below the uncompressed model and 3.08 points above Tokenskip.","The same pattern holds on Llama-3.1-8B: at ratio 0.5, tokens drop from 212.1 to 101.8 with a 3.64-point accuracy drop, outperforming Tokenskip by 1.82 points.","UCoT extends to long-CoT reasoning models: on Deepseek-R1-Distill-Qwen-7B (HumanEval) it cuts tokens 50.67% with only a 5.20-point drop, and on Qwen3-8B (HumanEval) 48.05% with a 5.28-point drop, both beating Tokenskip.","Larger executors absorb compression better: Qwen2.5-14B loses only 1.79 points at compression ratio 0.5 on GSM8K, suggesting the method's value grows with model scale.","Ablations show both UT and reward factor are necessary (removing UT drops accuracy about 15 points on GSM8K; removing R drops it about 16 points), and accuracy rises monotonically as UT length grows from 16 to 64 tokens."],"fun_headline_variants":["Pack reasoning chains into embeddings to halve LLM tokens","Front-load reasoning as soft tokens to cut output in half","CoT as embeddings: 50% fewer tokens, higher accuracy","Compressor+executor: reasoning compressed into vectors"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The hidden-state alignment Lsem is assumed to capture exactly the information needed for a correct answer, but the paper's own decoded examples contain arithmetic mistakes and swapped question content, so it is not established that the Upfront Thought actually preserves the reasoning needed to solve the problem.","fun_headline_variants_meta":{"raw":{"variants":["Pack reasoning chains into embeddings to halve LLM tokens","Front-load reasoning as soft tokens to cut output in half","CoT as embeddings: 50% fewer tokens, higher accuracy","Compressor+executor: reasoning compressed into vectors"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000485,"raw_usage":{"total_tokens":2250,"prompt_tokens":783,"completion_tokens":1467,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":527,"completion_tokens_details":{"reasoning_tokens":1408}},"tokens_in":527,"tokens_out":1467,"duration_ms":9134,"temperature":1.0,"reasoning_tokens":1408,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T10:50:49.019205+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a trained UCoT system and replace the UT for a question with (a) the UT of a different question or (b) random vectors of the same length, then measure executor accuracy. If accuracy stays near the reported level, the specific semantic content of the UT is not what drives the result, contradicting the high-fidelity compression claim. A direct check: decode the UT to text and verify it matches the original CoT's arithmetic operations and quantities—the appendix examples already show mismatches such as '1/100 = 10'.","supporting_citations":[],"review_version":1}