{"id":"f965e161-4661-403b-850f-c626f5e85099","arxiv_id":"2507.04742","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A single steering vector extracted from paired verbose and concise rationales compresses chain-of-thought output at inference time without retraining.","lead":"This paper introduces Activation-Steered Compression (ASC), which adds a learned steering vector to a language model's hidden states during decoding so it produces shorter chains of thought without retraining. On math benchmarks, ASC cuts reasoning lengths by up to 67% and speeds up total reasoning time by about 2.7x on an 8B model while keeping accuracy roughly intact.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Calibration set is sampled from the target evaluation set with no stated exclusion; if those 50 examples are not removed, Table 1's in-dataset compression/accuracy numbers are potentially inflated.","rationale":"The reader's stated weakest assumption is that a single linear conciseness direction extracted from complete-CoT final-token activations generalizes when injected at every decoding step on new questions. That is a real concern, but the empirical results in Tables 1 and 2 are the evidence for that generalization. The more immediate threat is that those results may be computed on a set that includes the very 50 calibration examples used to construct the steering vector. If so, the central claim that ASC compresses CoTs without retraining and without accuracy loss is not supported by the reported numbers, regardless of the theoretical KL bound. The KL-bound analysis and the unverified Lipschitz constant are secondary: even if Assumption 1 is hard to verify, the practical claim could still hold empirically. Calibration leakage, by contrast, directly undermines the integrity of every in-dataset number in the main table. The concrete test is cheap and decisive: inspect the code to see whether the calibration split is held out, and recompute the affected rows with those 50 examples excluded. If the exclusion is already present, the concern is resolved and the remaining issues are error bars and hyperparameter tuning. If it is not present, the headline compression and accuracy-retention figures need to be re-estimated before the paper's conclusions can be accepted.","tokens_in":19309,"tokens_out":9785,"duration_ms":115495,"concrete_test":"Inspect the released repository at github.com/ArminAzizi98/ASC: locate how the 50 calibration questions are sampled (seed and split) and whether those indices are filtered out of the MATH500/GSM8K evaluation loops. Then recompute Table 1 rows, especially DeepSeek-R1-Distill-LLaMA-8B on GSM8K and both models on MATH500, with the 50 calibration examples excluded, using the same ASC hyperparameters. If the token reduction or accuracy changes by more than about 1 percentage point (or accuracy by more than 0.5 points), the headline claim is inflated. If the code already excludes the calibration indices, rerun with the exclusion explicitly documented as a sanity check.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"Section 5.1 states that ASC 'randomly sample[s] 50 calibration samples from target dataset' (MATH500 and GSM8K) and then reports accuracy and token counts on those same datasets. The paper never states that the 50 calibration questions are removed from the evaluation set before computing Table 1. If they are not removed, the steering vector v_l is computed from the exact questions whose metrics appear in the main results. With only 50 examples, v_l can overfit to question-specific final-token activation differences, inflating both compression and accuracy retention. The impact is largest on MATH500, where 50 samples are 10% of the evaluation set; for GSM8K the 50 samples are about 3.8% of the test set, still enough to move aggregate numbers. The headline 67.43% GSM8K reduction and all in-dataset rows in Tables 1 and 2 are affected. Cross-dataset rows in Table 2 are less affected because the steering vector comes from a different dataset, but they cannot rescue the main in-dataset claims. This is a correctness risk in the evaluation protocol that is independent of the unverified Lipschitz assumption; it is also directly checkable from the released code.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes Activation-Steered Compression (ASC), a training-free inference-time method that learns a single steering vector from 50 paired verbose and concise chains of thought and injects it into a selected transformer layer at every decoding step to shift generation toward shorter rationales. The authors report up to 67.43% token-count reduction on GSM8K and MATH500 with roughly unchanged accuracy across Qwen-7B, LLaMA-8B, and QwQ-32B, a 2.73x wall-clock speedup on MATH500, and a closed-form KL-divergence-based rule for setting the steering strength. They also report a cosine similarity of 0.92 between steering vectors derived from MATH500 and GSM8K, supporting cross-task generalization. The central claims are that a linear activation injection can compress CoTs without retraining and that the proposed scaling rule selects a gamma that matches the empirical accuracy breakpoint.","tokens_in":19544,"tokens_out":9285,"duration_ms":86174,"significance":"If the empirical results survive scrutiny, ASC would be a practical and unusually lightweight addition to the CoT-compression toolbox, and the cross-task vector-alignment result would be a valuable interpretability finding. The paper is clearly written, the method is simple to implement, and the open-source code release is a strength. The theoretical derivation, though currently incomplete, points in an interesting direction. However, the calibration-set contamination, the absence of repeated-seed evaluation, and the unverified Lipschitz assumption are load-bearing issues that must be resolved before the empirical and theoretical claims can be accepted.","major_comments":[{"comment":"The calibration protocol is not fully specified. In Section 4 the authors state that they \"randomly sample 50 calibration samples from target dataset\" from MATH500 and GSM8K, but the paper never states whether these 50 questions are excluded from the evaluation set when computing the accuracy and token counts in Table 1. MATH500 contains only 500 problems, so the 50 calibration questions constitute 10% of the evaluation set; for GSM8K the overlap is about 3.8%. If the calibration questions are not removed, the steering vector is extracted from the very questions whose metrics are reported, which can inflate both compression and accuracy retention through memorization of question-specific activation differences. This affects the headline 67.43% GSM8K reduction and all in-dataset rows in Tables 1 and 2, and it cannot be fully rescued by the cross-dataset rows in Table 2. The authors must either state the exclusion procedure clearly or re-run the evaluation with a held-out test set.","section":"Section 4, Table 1"},{"comment":"The theoretical guarantee is not established as stated. The KL bound is derived for the next-token distribution conditional on a fixed prefix h, but at inference time the steering vector is injected at every decoding step, so the prefix at step i is itself generated under the steered distribution. The per-step bound does not compose into a bound on the distribution over the entire generated chain, and the paper does not address this accumulation. In addition, Assumption 1 is a global Lipschitz condition on the logit Jacobian along the ray h + tv; estimating L from 50 hidden states (95th percentile of HVP norms) does not verify the required supremum over t in [0, gamma], and the statement \"One checks directly\" accompanying Eq. (13) is not a proof. The empirical gamma selection may still work, but the paper's claim of a 'provable' KL-constrained calibration needs to be revisited or substantially qualified.","section":"Appendix A.1, Eq. (10)"},{"comment":"All reported numbers come from a single run at temperature 0.7. Under stochastic sampling, accuracy differences of 0.2-0.4 percentage points (e.g., the 0.2% GSM8K improvement on LLaMA-8B and the 0.4% MATH500 improvement on QwQ-32B) are within likely sampling noise, and token counts also vary across runs. Without repeated seeds, confidence intervals, or at least a statement of variance, the claim of \"maintaining accuracy\" is not statistically supported. This is a load-bearing part of the paper's central claim and should be addressed with repeated runs or by lowering the significance attached to small accuracy differences.","section":"Table 1, Section 5.1"},{"comment":"The baseline comparison is not fully controlled. The paper does not state whether TCC, DEER, CoD, and SEAL results in Table 1 were re-run with the same decoding settings (temperature 0.7, top_p 0.9, repetition penalty 1.1) or are taken from the original publications. If the numbers are imported from prior work, differences in sampling, prompt format, or length measurement could account for part of the reported gap. The authors should clarify the provenance of every baseline number and, ideally, re-run the baselines in the same harness.","section":"Table 1, Section 5.1"}],"minor_comments":[{"comment":"The text says \"we estimate the two scale parameters alpha and L\" but the notation introduced earlier is a and L; this inconsistency is confusing and should be corrected.","section":"Section 4"},{"comment":"The layer index is \"determined empirically,\" which is not a parameter-free choice; this should be acknowledged in Section 4 where \"no additional hyperparameters are introduced\" is stated.","section":"Appendix C"},{"comment":"The t-SNE visualization does not report the random seed or perplexity value, making the figure hard to reproduce.","section":"Figure 2"},{"comment":"The sentence \"On the DeepSeek-R1-Distill-LLaMA-8B model, ASC reduces CoT length by up to 61.2%\" appears to attribute the 61.2% reduction to the wrong model; Table 1 shows that the 61.2% reduction corresponds to Qwen-7B on MATH500 (3984 to 1543 tokens), while LLaMA-8B achieves 67.43% on GSM8K.","section":"Section 5.2, first paragraph"},{"comment":"The gamma-sweep figure would be more informative if it also showed the epsilon used for the KL rule and the computed gamma_max for each model; currently only the selected gamma is marked.","section":"Figure 5"}],"recommendation":"major_revision","confidential_remarks":"The calibration-overlap concern raised in the stress-test note is well-founded and is the main barrier to acceptance. If the released code shows that the 50 calibration samples were excluded from the evaluation set, the paper may be salvageable with revisions to the theory and the addition of repeated-seed experiments. The provenance of the baseline numbers should also be clarified before any final decision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe one thing to know: ASC is a clean idea—extract a \"conciseness\" direction by averaging final-token residual differences between paired verbose and concise CoTs, then inject it at a mid-layer during decoding. It's genuinely different from SEAL's manually labeled segment damping and from prompt-based methods like CoD. The theory is a straightforward Taylor bound on KL divergence, nothing deep but sensible, and the cross-task cosine similarity of 0.92 is a decent hint that the direction is not just question-specific noise.\n\nThe problem is the evaluation. The paper says it \"randomly sample[s] 50 calibration samples from target dataset\" and then reports accuracy and token counts on the same datasets. It never says those 50 are excluded. If they're not, the steering vector is computed from the exact questions whose metrics appear in Tables 1 and 2. With 50 samples that's a real risk of inflating both compression and accuracy retention—especially on MATH500, where 50 is 10% of the set. The code is available, so this is checkable, but currently the headline 67.43% and all in-dataset rows are suspect. Cross-dataset rows are less affected, but they're on a single model and can't rescue the main claims.\n\nOther soft spots are more minor. No error bars despite temperature 0.7; the layer index and gamma still involve hand-picked components; and the Lipschitz constant L in the KL bound is estimated from 50 hidden states with no validation. The abstract says 100 paired examples while the body says 50—that inconsistency should have been caught. None of those are fatal on their own.\n\nIf the calibration leak is fixed—either by excluding the calibration set or by showing results that don't change—this is a solid, useful paper for the efficient-inference crowd. The idea is certainly worth a serious referee; the current version just needs an honest evaluation protocol. I'd recommend sending it to review, but asking the authors to confirm the exclusion and add repeated-seed numbers.","headline":"A plausible training-free steering method for CoT compression, but the headline numbers rest on an uncleared evaluation-set overlap that needs to be resolved before I'd trust them.","tokens_in":20076,"tokens_out":2313,"would_cite":false,"duration_ms":25156,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A single steering vector injected into a model's hidden states compresses chain-of-thought reasoning by up to 67.43% without retraining or accuracy loss.","keywords":["activation steering","chain-of-thought compression","representation engineering","inference-time intervention","KL-divergence bound","residual stream","reasoning efficiency","large language models"],"falsifier":"Take a held-out set of math problems unlike the calibration distribution in wording and format, apply the same 50-pair vector at the KL-selected gamma, and measure answer accuracy and token counts. If accuracy falls much more than the vanilla-CoT dispersion while tokens keep shrinking, or if the same gamma produces degenerate output on a different model family without recalibration, the single linear direction and the estimated Lipschitz bound are falsified.","tokens_in":19111,"feed_emoji":"🧭","tokens_out":6213,"duration_ms":57927,"temperature":0.7,"pith_summary":"This paper tries to establish that verbosity in chain-of-thought reasoning is not a fixed behavior but a steerable property of a model's internal activations. The claim is that a single vector, computed as the average difference between residual-stream activations of 50 paired verbose and concise chains, can be added at every decoding step to make a model produce much shorter reasoning traces without retraining. On MATH500 and GSM8K, the intervention reduces chain-of-thought length by up to 67.43% across 7B, 8B, and 32B models while keeping accuracy level with the unsteered baseline, and on MATH500 it gives an average 2.73x end-to-end speedup on an 8B model. The paper also claims a closed-form KL-divergence-bounded rule for setting the steering strength, replacing heuristic grid search.","feed_headline":"One steering vector cuts chain-of-thought length by up to 67%","feed_subtitle":"A single linear injection into hidden states yields shorter, faster reasoning with no retraining or accuracy loss.","key_machinery":"The load-bearing object is the steering vector $v_\\ell = \\frac{1}{N}\\sum_{i}\\left(h_\\ell(q_i \\oplus s_i)[-1] - h_\\ell(q_i \\oplus l_i)[-1]\\right)$, the average difference of the final-token residual-stream activation at layer $\\ell$ between concise and verbose chains over $N=50$ calibration pairs. At inference, each decoding step updates the hidden state as $h_\\ell(x_i) \\leftarrow h_\\ell(x_i) + \\gamma v_\\ell$. The scaling $\\gamma$ is set by a KL-constrained rule: under a Lipschitz smoothness assumption on the logit Jacobian, the divergence bound reduces to a cubic equation $x^3 + x^2 - \\beta = 0$ whose real Cardano root gives $\\gamma_{\\text{raw}}$, followed by a curvature safety factor $\\gamma_{\\max} = \\max(0, (1 - L\\gamma_{\\text{raw}}/4a)\\gamma_{\\text{raw}})$, with $a = \\|Wv\\|_2$ estimated from one Jacobian-vector product per calibration point.","core_discovery":"The central discovery is that the mean difference between final-token residual-stream activations of concise and verbose answers to calibration questions forms a valid steering direction for compression. When this vector is injected into a mid-layer hidden state at every decoding step, generation shifts from verbose, natural-language-heavy rationales toward math-centric concise ones. The paper reports that this single linear intervention achieves the largest token reductions among the compared training-free methods, up to 67.43% on GSM8K and around 50.7% on MATH500 for the largest model, without accuracy degradation. It further reports that the gamma chosen by the KL-constrained scaling rule sits exactly at the empirical accuracy-breakdown point in a strength sweep.","pith_inferences":["If the verbosity axis is truly universal, the same calibration procedure could compress reasoning in non-mathematical domains, such as code synthesis or multi-hop QA, and might transfer across model families with one calibration set.","Because the concise chains are generated by a different model than the target, the vector may encode the concise author's style rather than general conciseness; swapping which model writes the verbose and concise members of each pair would separate content from style.","The Lipschitz constant is estimated from only 50 hidden states, so the safety guarantee is only as good as that estimate; a stress test on out-of-distribution prompts or newly released models would show whether the gamma budget holds.","An obvious next step is adaptive steering, where gamma varies per token or per estimated answer confidence instead of being fixed for the whole generation."],"forward_implications":["CoT compression becomes an inference-time knob: a single vector addition, with no retraining, can be composed with prompting, early-exit, or token-pruning methods.","The same steering vector transfers across tasks: MATH500 and GSM8K vectors have cosine similarity 0.92, and cross-dataset steering retains most of the compression and accuracy.","Latency- and cost-sensitive deployments of reasoning models can realize a 2.73x end-to-end speedup on MATH500-scale problems without accuracy loss.","Steering-strength selection no longer requires grid search: the KL-constrained gamma tracks the empirical accuracy breakpoint."],"supporting_citations":[{"why":"Establishes activation steering as a representation-engineering technique that ASC applies to compression.","marker":"[1]"},{"why":"SEAL is the closest prior work, steering CoT compression with manually labeled segment types; ASC compares against it.","marker":"[4]"},{"why":"GSM8K is one of the two benchmarks used for calibration and evaluation.","marker":"[8]"},{"why":"MATH500 is the other benchmark; its calibration pairs provide the steering vector.","marker":"[13]"},{"why":"ActAdd demonstrates that adding an activation direction changes output tone, the basis of vector injection.","marker":"[23]"},{"why":"Chain-of-Draft is a prompt-based baseline for short reasoning that ASC must beat on compression.","marker":"[31]"},{"why":"DEER is an early-exit baseline for shorter reasoning; ASC compares token counts and speed against it.","marker":"[32]"}],"fun_headline_variants":["Steering vector compresses CoTs by 67% with no retraining","Single activation vector shrinks reasoning traces 67%","Activation steering: 67% shorter CoTs, same accuracy","Hidden-state tweak cuts chain-of-thought length 67%","One vector, 67% less CoT verbosity, zero training"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole method rests on the assumption that the average activation difference of 50 paired examples points along a universal \"conciseness direction\" that holds for new questions, datasets, and model sizes; if that direction actually encodes the style of the concise examples rather than conciseness itself, the compression and accuracy-preservation claims collapse.","fun_headline_variants_meta":{"raw":{"variants":["Steering vector compresses CoTs by 67% with no retraining","Single activation vector shrinks reasoning traces 67%","Activation steering: 67% shorter CoTs, same accuracy","Hidden-state tweak cuts chain-of-thought length 67%","One vector, 67% less CoT verbosity, zero training"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000459,"raw_usage":{"total_tokens":2315,"prompt_tokens":972,"completion_tokens":1343,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":588,"completion_tokens_details":{"reasoning_tokens":1252}},"tokens_in":588,"tokens_out":1343,"duration_ms":10725,"temperature":1.0,"reasoning_tokens":1252,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T19:40:31.272711+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a held-out set of math problems unlike the calibration distribution in wording and format, apply the same 50-pair vector at the KL-selected gamma, and measure answer accuracy and token counts. If accuracy falls much more than the vanilla-CoT dispersion while tokens keep shrinking, or if the same gamma produces degenerate output on a different model family without recalibration, the single linear direction and the estimated Lipschitz bound are falsified.","supporting_citations":[{"cited_title":"An introduction to representation engineering: Activation steering","cited_arxiv_id":null,"evidence_quote":"Establishes activation steering as a representation-engineering technique that ASC applies to compression."},{"cited_title":"Measuring mathematical problem solving with the math dataset","cited_arxiv_id":null,"evidence_quote":"MATH500 is the other benchmark; its calibration pairs provide the steering vector."},{"cited_title":"Steering language models with activation engineering","cited_arxiv_id":null,"evidence_quote":"ActAdd demonstrates that adding an activation direction changes output tone, the basis of vector injection."}],"review_version":1}