{"id":"f388f05c-6d6f-4f3e-8fae-0cbd99bd76a4","arxiv_id":"2506.07240","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A fitted progress vector in LLM hidden states can monitor and, when added during decoding, shorten the thinking phase, often improving accuracy and cutting tokens.","lead":"The authors show that a simple linear readout of a reasoning model's internal states can track how far it is through its thinking phase, and that nudging the hidden states along this direction makes the model think less while staying accurate. The work points toward a plug-in way to make white-box reasoning models faster, less prone to overthinking, and easier to monitor.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The overclocking effect is not shown to be caused by a progress-tracking mechanism; without control directions, h + alpha*theta may simply bias the model toward ending its thinking, leaving the causal claim underdetermined.","rationale":"The reader's weakest assumption is precisely the lack of control directions, and I agree this is the most load-bearing concern because it threatens the causal interpretation of both the monitoring and control claims. The paper's own results already show that the abstract's 'improves answer accuracy' is not consistently supported (e.g., Table 1 Math-500 2048: TPV100 gives 284 correct vs Instruct's 316), but this is an evaluative weakness that could be fixed with more careful reporting. The control-direction issue is deeper: without it, the mechanism is unidentified, and the novelty of the paper is severely reduced. The reader's verdict of CONDITIONAL is appropriate; I would not move it to reject because the monitoring result is still empirically interesting and the intervention may work even if the mechanism is misattributed. A conditional acceptance requiring control experiments is the right call. My concrete test directly settles whether the effect is specific, and thus whether the central claim stands.","tokens_in":15154,"tokens_out":3216,"duration_ms":42370,"concrete_test":"Run the same Table 1 protocol (DeepSeek-R1-Distill-Qwen-32B, Math-500, alpha=100) with one control: train theta_shuf on the same hidden states but with progress labels randomly permuted within each trajectory, then intervene with h + alpha*theta_shuf. If the token-count reduction and accuracy changes are comparable to the real theta, the overclocking effect is a generic bias rather than a consequence of a true progress direction. Additionally, compute the cosine similarity between theta and the output embedding of the </think> token; if it is high, this strengthens the concern.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central causal claim—that manipulating an internal progress estimate modulates thinking length—rests entirely on Section 3.2's intervention h_alpha = h + alpha*theta. Theta is fit to predict relative position p = j/N, so it points along whatever direction in hidden state space best separates early and late tokens. That direction could plausibly align with the output direction of the </think> token or with a generic 'conclude now' bias, in which case adding alpha*theta makes the model more likely to terminate its thinking regardless of any progress estimate. The paper provides no control interventions: no random projection directions, no directions trained on shuffled labels, no comparison with a vector that maximizes the probability of the end-of-thinking token. Figure 6 shows higher predicted progress after intervention, but this is partly circular because theta defines the predicted-progress axis. Moreover, the linear probe's low test loss may simply reflect that hidden states encode absolute token position via positional embeddings and token frequencies, not that the model plans or tracks relative position. The monitoring result alone does not establish a causal role, and the intervention result is equally consistent with a generic output bias. This is the fragile link between the two halves of the paper; if the intervention is not specific to the progress direction, the claim of 'first empirical evidence of an internal progress estimate' and the mechanistic interpretation collapse, even though the practical length-reduction effect might still be useful.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies whether reasoning LLMs encode their relative position inside the explicit <think>...</think> thinking phase. The authors train a linear probe (TPV) on last-layer hidden states to predict p = j/N (Equation 1), report low test MSE, and introduce an RNN/GRU-based smoother for visualizing progress. They then intervene by computing h_alpha = h + alpha*theta, reporting that positive alpha shortens thinking, increases the number of correct answers under token budgets, and reduces inference latency on Math500 and GSM8K for two DeepSeek-R1 distilled models. They interpret the results as evidence of an internal progress estimate and present 'overclocking' as a control mechanism that mitigates overthinking.","tokens_in":15428,"tokens_out":5613,"duration_ms":56348,"significance":"If the central causal claim were established, the paper would be a useful contribution: it connects monitoring and control of reasoning length, proposes a lightweight inference-time intervention that does not require fine-tuning, and publicly releases code. The progress-bar visualization is a practical diagnostic for reasoning models. However, the significance is currently limited by the absence of control interventions and by the lack of statistical rigor. The monitoring result is a correlational probe, and the intervention result is equally consistent with a generic 'conclude now' output bias. The paper's own tables also contradict the claim that accuracy improves without increasing errors. These issues must be addressed before the paper can support its strongest conclusions.","major_comments":[{"comment":"The intervention shifts the hidden state along the direction trained to predict p=j/N, so the increase in predicted progress shown in Figure 6 is partly by construction. The observed shortening of thinking could be produced by a generic 'conclude now' bias rather than by an internal progress estimate. I ask for control interventions: random projection directions, directions trained on shuffled labels, and the direction that maximizes the logit of the </think> token. Without such controls, the causal interpretation in Sections 3.2 and 5 is underdetermined.","section":"Section 3.2 (h_alpha = h + alpha*theta)"},{"comment":"The statement that 'these increases in correct answers do not come at the cost of more errors, as the error rate remains unchanged' is contradicted by Table 1. On Math500 at 1024 tokens, #Answered - #Correct is 2 for Base and 12 for TPV alpha=100; on GSM8K at 512, it is 2 vs 11. Thus error counts and error rates increase in several regimes. The claim that overclocking improves accuracy without increasing errors is not supported by the reported data.","section":"Section 4.1, Table 1"},{"comment":"All accuracy and token-length numbers are single-run raw counts with no variance, no repeated seeds, and no significance tests. Because generation uses temperature 0.6, the differences between methods may fall within sampling noise. The paper should report multiple runs or confidence intervals, or at least apply a bootstrap or other statistical test over problems, before claiming improvements of 80% or more.","section":"Section 4, Tables 1-2"},{"comment":"The claim that 'our method outperforms both [baselines] by producing more correct answers' is internally inconsistent with Table 1: at Math500 with a 2048-token budget, Instruct yields 316 correct vs 300 for TPV alpha=100 and 300 for TPV alpha=100 Ins. This is later acknowledged in point (4), but the earlier blanket statement should be revised or qualified to reflect the regimes where prompting is stronger.","section":"Section 4.1, point (3)"}],"minor_comments":[{"comment":"The text says 'under the same 512-token limit on GSM-8K' but then cites completion increases from 28 to 43 and 52, which are the Math500 512-token numbers; the GSM8K 512 completions are 227, 248, and 232. This conflation makes the baseline comparison difficult to follow.","section":"Section 4.1, point (2)"},{"comment":"Several table cells are merged or truncated (e.g., '316321 296' and '40247263 261'), making the data hard to verify. Please reformat the tables so every entry is clearly separated.","section":"Tables 1-2"},{"comment":"The final answer in the overclocked output reads 'The number of ways is \\boxed{36}' but should refer to miles rather than ways.","section":"Example 2 (Appendix D)"},{"comment":"References [10] and [11] appear to cite the same work with different venues and should be deduplicated. Also, Section 3.1 introduces a 'GRU' while Figures 3 and 4 say 'RNN'; please align the terminology.","section":"References and Section 3.1"}],"recommendation":"major_revision","confidential_remarks":"The central idea is timely and the code release is a plus, but the mechanistic claim is the fragile load-bearing point. The lack of control directions and the absence of any repeated-run statistics mean the empirical case is currently correlational. The error-rate contradiction in Table 1 should be fixed in revision, not merely reworded. I would not reject the paper, but it needs a substantive revision with new experiments rather than a light edit."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know before you look at it. The paper's core idea is simple and useful: train a linear probe to estimate a token's normalized position inside the <think> block, then add a scaled version of that probe to the hidden states to make the model think shorter. The token-count reductions in Figure 7 are monotone in the scaling factor and large—up to 30%—which is a real effect. The qualitative examples are striking. But the paper overinterprets this as evidence of an internal progress-tracking mechanism. The intervention is built from the same vector that was trained to predict position, so the shortened thinking could just be a generic output bias toward terminating the chain, not manipulation of a progress estimate. The authors never test random projections or directions that maximize the end-of-think token probability, so the causal claim is underdetermined.\n\nWhat's genuinely good: the monitoring half is solid. The linear probe achieves low MSE, generalizes to zero-shot on GSM8K, and the smoothing/RNN variants are sensible. The progress bar visualization is a nice contribution for transparency. The paper is honest about limitations: math-only domains, white-box access required, and no breadth/uncertainty analysis.\n\nSoft spots, in order. (1) The causal narrative is the main weakness. Without control directions, we can't distinguish progress steering from 'conclude now' biasing. (2) All numbers are single-run raw counts with no variance or significance tests, so claims like 'error rate remains unchanged' are fragile. (3) The accuracy claim is overstated: Table 1 shows the Instruct baseline beats TPV at 2048 tokens on Math500, and the hybrid is often best, which suggests the method is complementary rather than a standalone win. (4) The abstract promises 'improves answer accuracy' but the evidence is mixed outside low token budgets.\n\nThe circularity concern is real but not fatal—the effect on length is not purely circular, because a random probe wouldn't do that. The burden is on the authors to show specificity. This deserves a serious referee. With control interventions and repeated runs, it could be a solid paper. As it stands, I'd want major revision, but I'd engage with it and probably cite it if working on reasoning-length control or mechanistic interpretability of reasoning.","headline":"Useful monitoring/steering idea, but the causal story is under-supported and the accuracy claims are overstated; worth a serious look with added controls.","tokens_in":15935,"tokens_out":3048,"would_cite":true,"duration_ms":33860,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Reasoning LLMs track their own thinking progress, and shifting that internal signal during inference can shorten the thinking phase, cut latency, and improve answer accuracy under tight token budgets.","keywords":["reasoning models","thinking progress vector","overclocking","overthinking","hidden-state intervention","test-time scaling","mechanistic interpretability","structured reasoning"],"falsifier":"Run the same intervention with a random unit vector, or with a vector trained on shuffled position labels, in place of $\\theta$, matching $\\|\\theta\\|$ and $\\alpha$. If random projections shorten the thinking phase as much as the TPV does, length control is not specific to progress encoding; a complementary check is whether overclocking preserves accuracy on non-mathematical open-ended reasoning tasks, where progress is harder to define.","tokens_in":14964,"feed_emoji":"⚡","tokens_out":8762,"duration_ms":80433,"temperature":0.7,"pith_summary":"Reasoning models that mark off a `<think>` phase need to decide when to stop deliberating. This paper claims that such models keep a running estimate of how far they are through the thinking phase, encoded linearly in the last-layer hidden states, and that this estimate is not a passive by-product but a signal the model actually uses. Reading that signal gives an interactive progress bar for the model's reasoning. More strongly, nudging the hidden state along the learned progress direction—the 'overclocking' intervention—shortens the thinking phase, cuts inference cost, and can raise the number of correct answers under tight token budgets. The paper positions this as the first direct evidence of an internal, causally active progress estimate in explicit reasoning.","feed_headline":"One vector shortens LLM thinking and boosts accuracy","feed_subtitle":"Nudging a learned progress direction cuts reasoning tokens and beats prompting baselines on math benchmarks.","key_machinery":"The load-bearing object is the thinking progress vector (TPV), a parameter vector $\\theta \\in \\mathbb{R}^d$ fit by least-squares regression to predict the relative position $p = j/N_k$ of each thinking-phase token from its final-layer hidden state $h_j^{(k)}$. Its role is to turn an opaque hidden state into a scalar progress estimate, and then to act as an intervention direction: adding $\\alpha\\theta$ to $h$ raises $\\theta^\\top h_\\alpha = \\bar{p} + \\alpha\\|\\theta\\|^2$, pushing the model toward the 'later in the thinking phase' region of representation space. Because the edit is applied after the attention layers, each step's perturbation influences only the immediate next-token distribution while the autoregressive history still carries earlier edited tokens. A single-layer GRU over the same hidden states provides a smoother monitoring signal, but the linear TPV is the object used for control.","core_discovery":"The paper's central claim is that a deep reasoning model tracks its own relative position inside the explicit thinking phase, and that this tracking can be both observed and manipulated. The authors train a linear projection (a 'thinking progress vector') on last-layer hidden states to predict the normalized token position $j/N$ within `<think>...</think>`, and show that the prediction transfers across prompts and datasets and improves with smoothing or a sequence model. They then intervene at every decoding step after the attention layers with $h_\\alpha = h + \\alpha\\theta$, which changes the predicted progress but leaves cached states untouched. Positive $\\alpha$ consistently shortens the generated thinking trajectory. Under low token budgets this shortening is not just cheaper: on Math-500 and GSM8K it produces more completed, answered, and correct responses than the base model and than temperature- or instruction-based acceleration baselines, and it combines with those baselines for further gains.","pith_inferences":["If the progress estimate is genuinely causal, the learned vector could become an early-stopping rule: generate until smoothed predicted progress crosses a threshold, then cut the remaining thinking tokens, a control loop the paper does not implement.","The method's open-domain reach is untested; applying the same regression and intervention to open-ended reasoning would clarify whether the progress signal is a general planning mechanism or a math-reasoning artifact.","A stronger causal check than the paper reports would compare the intervention with random projection directions of equal norm; if those shorten thinking too, the effect may be a generic output bias rather than a dedicated progress mechanism.","Because the intervention edits only the final hidden layer, it could be combined with speculative decoding or per-problem token-budget schedulers that decide how aggressively to overclock each question."],"forward_implications":["A reasoning model can be made more concise at inference time with no fine-tuning: shifting hidden states along the progress vector shortens thinking across two model sizes and two math benchmarks.","Accuracy under small token budgets rises with the intervention, so overclocking is a practical counter to overthinking when compute or context is limited.","The progress signal is complementary to prompt-based acceleration; combining TPV intervention with an instruction to be concise gives the largest gains in the paper's tables.","The same hidden-state readout can serve as a real-time progress bar, making the otherwise invisible thinking phase inspectable by users.","Token-level dips in predicted progress line up with self-verification behavior, so the readout exposes when the model re-solves a problem rather than advancing."],"supporting_citations":[{"why":"Supplies the DeepSeek-R1 models, the `<think>`/`</think>` structured-reasoning format, and the prompting recipe all experiments build on.","marker":"[7]"},{"why":"Provides the Math-500 benchmark used for training the progress regressor and evaluating accuracy under token budgets.","marker":"[15]"},{"why":"Provides the GSM8K benchmark used for the zero-shot regression evaluation and the main accuracy and efficiency tables.","marker":"[4]"},{"why":"Defines the overthinking phenomenon that the overclocking intervention is designed to mitigate.","marker":"[25]"},{"why":"Documents the relationship between reasoning length and correctness that motivates controlling thinking length.","marker":"[24]"},{"why":"Supplies the task-vector idea of adding a learned direction to hidden states, which the intervention $h+\\alpha\\theta$ is modeled on.","marker":"[8]"}],"fun_headline_variants":["A progress vector prunes LLM overthinking and lifts answer accuracy","Steer LLM reasoning length with a progress vector to improve accuracy","Read an LLM's own progress signal to trim thinking and answer better","Overclocking LLM reasoning: a progress vector trims thoughts and improves answers","Manipulating an LLM's progress vector cuts overthinking and improves accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that shortened thinking is caused by the model's true sense of progress, not by the nudged vector simply biasing it toward stopping tokens; if a random direction of the same size also shortens reasoning, the progress-specific explanation would collapse.","fun_headline_variants_meta":{"raw":{"variants":["A progress vector prunes LLM overthinking and lifts answer accuracy","Steer LLM reasoning length with a progress vector to improve accuracy","Read an LLM's own progress signal to trim thinking and answer better","Overclocking LLM reasoning: a progress vector trims thoughts and improves answers","Manipulating an LLM's progress vector cuts overthinking and improves accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.002255,"raw_usage":{"total_tokens":8694,"prompt_tokens":908,"completion_tokens":7786,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":524,"completion_tokens_details":{"reasoning_tokens":7689}},"tokens_in":524,"tokens_out":7786,"duration_ms":56059,"temperature":1.0,"reasoning_tokens":7689,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T05:39:39.173410+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same intervention with a random unit vector, or with a vector trained on shuffled position labels, in place of $\\theta$, matching $\\|\\theta\\|$ and $\\alpha$. If random projections shorten the thinking phase as much as the TPV does, length control is not specific to progress encoding; a complementary check is whether overclocking preserves accuracy on non-mathematical open-ended reasoning tasks, where progress is harder to define.","supporting_citations":[{"cited_title":"In-context learning creates task vectors","cited_arxiv_id":null,"evidence_quote":"Supplies the task-vector idea of adding a learned direction to hidden states, which the intervention $h+\\alpha\\theta$ is modeled on."}],"review_version":1}