{"id":"56472b77-ef58-4c0e-8a68-2a40d4bfe276","arxiv_id":"2508.20395","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"In an oracle study on MATH problems, conditional entropy over the answer span decreases across reasoning steps for correct LLM solutions but stays flat or rises for incorrect ones.","lead":"This paper tests whether the uncertainty an LLM shows about its own answer, measured step by step during reasoning, can reveal whether the final answer will be right. On MATH problems, they find that uncertainty that drops over reasoning steps marks correct answers, while flat or rising uncertainty marks mistakes.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Teacher-forced entropy over the model's own answer span (Eq. 1) is not I(Y;Z|X) for incorrect chains; the observed decrease may be confidence in wrong answers, so the central utility claim needs a gold-answer target test.","rationale":"The paper's central empirical claim is that a decreasing conditional-entropy trajectory over the answer span marks correct reasoning. The natural reading is that entropy is being reduced about the correct answer, matching the formal I(Y;Z|X) definition. The measured quantity, however, uses the model's own answer tokens as the target, so for incorrect chains it tracks confidence in a wrong answer. This is not a stylistic mismatch: it changes the target distribution between the correct and incorrect groups being compared, and the paper's own Appendix B shows the inspector can be more confident on incorrect chains. The absence of error bars is secondary; even perfectly measured averages would not resolve the target-sequence confound. This is addressable by re-running the analysis with gold answer spans as the target, which would either validate the metric as reasoning utility or reframe it as an answer-confidence heuristic. Since the reader already judged the paper CONDITIONAL and the proposed test is a reasonable condition, I do not move the verdict.","tokens_in":7988,"tokens_out":7173,"duration_ms":82786,"concrete_test":"Recompute Figure 1 with a single fixed target per problem: the gold answer span from the human solution, used as the teacher-forced target for every chain, both correct and incorrect, for both generators, keeping Eq. 1, cubic-spline alignment, and domain averaging. If the correct-vs-incorrect separation in entropy decrease persists, the proxy concern is refuted; if it disappears or reverses, the signal is self-confidence in wrong answers, not reasoning utility. Also report per-problem effect sizes (e.g., AUC of entropy slope for correctness) with bootstrap confidence intervals instead of only averaged curves.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The formal target is I(Y;Z|X)=H(Y|X)-H(Y|X,Z) with Y the correct answer, but Eq. 1 estimates per-token entropy over the model-generated answer span Y of each chain. For correct chains this target coincides with the gold answer; for incorrect chains it is the wrong answer itself. Thus H(Y|C_k) measures the inspector's confidence in a wrong answer, not uncertainty about the truth. Section 3 even switches to calling Y 'the ground-truth Y' after defining R=Z+Y, an internal inconsistency. A confidently wrong chain will show a steeply decreasing entropy trajectory, exactly the signature attributed to correct reasoning. Appendix B/Figure 3 provides direct evidence of the confound: cross-entropy over the target tokens is often lower (higher confidence) on incorrect chains. Because Figure 1 averages over different answer-span targets for correct vs incorrect groups, the separation could be driven by target-sequence identity and problem difficulty rather than by reasoning utility. The Limitations section does not flag this target mismatch, and the proposed pruning heuristic in Section 5 is not evaluated directly.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents an oracle study of whether the conditional entropy of the answer span, measured under teacher forcing by a separate inspector model, can serve as a training-free signal for reasoning utility in LLMs. Using MATH problems, GPT-4o and Qwen2.5-32B generate reasoning chains, and Qwen3-8B computes token-level entropy over the answer span after each reasoning step. The authors report that averaged entropy trajectories decrease for correct chains and are flat or rising for incorrect chains, that incorrect chains are longer, and that human reasoning shows earlier entropy reduction than LLM reasoning. They propose a pruning heuristic based on entropy slope. Appendices report null results for cosine-similarity and cross-entropy baselines.","tokens_in":8227,"tokens_out":5115,"duration_ms":57157,"significance":"If the central association were established, this would be a useful training-free signal for test-time compute allocation and chain pruning. The oracle design that separates generator and inspector, the parameter-free nature of the entropy measure, and the honest reporting of failed baselines in the appendices are strengths. However, the evidence is currently descriptive: the central association is not quantified, and the estimator's target is the model-generated answer rather than the gold answer, which undermines the information-theoretic interpretation. The proposed pruning heuristic is not evaluated. With additional per-problem analysis and a gold-answer target test, the contribution could be solid and relevant to efficient reasoning pipelines.","major_comments":[{"comment":"The quantity computed in Eq. (1) is not the conditional entropy of the correct answer. The formal target is I(Y;Z|X)=H(Y|X)-H(Y|X,Z) with Y the correct answer, but the estimator teacher-forces over the model-generated answer span Y from R=Z+Y defined in Section 2. For incorrect chains, Y is the wrong answer, so Eq. (1) measures the inspector's confidence in that wrong answer, not uncertainty about the truth. Figure 1 then averages these target-dependent curves separately for correct and incorrect chains, so the separation could be driven by answer-span identity, length, and problem difficulty rather than by reasoning utility. The manuscript itself notes in Appendix B that cross-entropy over the target tokens is often lower on incorrect chains, which is consistent with this confound. Please recompute H(Y_gold|C_k) for all chains, or otherwise show that the result is not an artifact of the","section":"Section 3, Eq. (1)"},{"comment":"The central claim that 'conditional entropy that decreases over steps is strongly associated with correct answers' is never quantified. The evidence consists of averaged trajectories after cubic-spline alignment. No per-problem correlation, ROC/AUC, confidence interval, or statistical test is reported. Since the curves are averaged over many problems, small systematic differences or outliers can produce apparent separation. Please report, for example, the distribution of slopes or entropy reductions for correct versus incorrect chains, the AUC for predicting correctness from trajectory shape, and standard errors over problems. This is necessary to support the word 'strongly' in the abstract and Section 5.","section":"Section 5 and Figure 1"},{"comment":"The paper proposes a concrete selection heuristic: 'prune chains whose entropy does not decrease; if all decrease, rank by the magnitude of the negative slope and keep the top-k.' This is the main practical payoff promised in the introduction, but it is not evaluated. An oracle experiment on the generated chain sets, for example selecting among N sampled chains by entropy slope and comparing accuracy to random selection or self-consistency, would directly test whether the signal is usable. Without such an evaluation, the heuristic remains speculative.","section":"Section 5, pruning heuristic"},{"comment":"The alignment procedure may itself contribute to the reported trend. Each chain is resampled to a common target step count using cubic splines. Incorrect chains are substantially longer (Section 5 and Table 1), so aligning to a fixed number of steps rescales the time axis differently for correct and incorrect chains. If entropy tends to decrease with context and then saturate, a longer chain aligned to the same number of steps will appear to fall more slowly than a shorter chain. The paper does not report whether the separation survives alternative alignments, such as absolute token index, no interpolation, or conditioning on the number of steps. Please add a robustness check.","section":"Section 4, Data Processing"}],"minor_comments":[{"comment":"The symbol Y is used inconsistently. In Section 2, Y is the model-generated answer span in R=Z+Y; in Section 3 it is called 'the correct answer' and then 'ground-truth Y'. Use distinct symbols, e.g., A for the generated answer span and A* for the gold answer, throughout.","section":"Section 3"},{"comment":"The legend for the dashed human curves uses identical dash patterns ('--' and '--'), so the human correct and incorrect trajectories cannot be distinguished. Clarify the line styles or colors.","section":"Figure 1 caption"},{"comment":"The generated solution is attributed to 'Qwen2.5-34B', but the rest of the paper uses Qwen2.5-32B. Fix the inconsistency.","section":"Appendix C"},{"comment":"The columns Kgt and Kllm are used before they are defined in Section 4. Define them in the table caption or introduce them earlier.","section":"Table 1"},{"comment":"The formula E_{v in V}[-p(v|C) log p(v|C)] is a slight abuse of notation: Eq. (1) averages per-token Shannon entropy over the answer span, not an expectation over the vocabulary. Rephrase for consistency with the main text.","section":"Abstract"},{"comment":"No code or data availability statement is provided. Since this is an empirical oracle study, releasing the generation and evaluation scripts would substantially aid reproducibility.","section":"Reproducibility"}],"recommendation":"major_revision","confidential_remarks":"The paper's scope fits cs.CL and the empirical setup is reasonable, but the central claim is currently supported only by averaged curves with a target mismatch in the estimator. The issues are fixable within the manuscript's scope: recompute over gold answer spans, add per-problem quantitative evidence, and evaluate the proposed heuristic. I would not recommend rejection, but the revision is substantive."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is a short oracle study on MATH. It reports that if you take a generated reasoning chain and, with a separate inspector model, compute the per-token entropy of the answer span given the reasoning so far, that entropy tends to decrease as reasoning progresses when the chain ends in the correct answer, and tends to be flat or increasing when it ends in error. That observation is genuinely new to me. The paper also earns credit for checking two alternative metrics—cross-entropy and cosine similarity—and showing they don't separate correct from incorrect chains. The negative results are useful and the limitations section is honest.\n\nThe main soft spot is the mismatch between the formal definition and the estimate. Section 3 defines the utility of reasoning Z as I(Y; Z|X) with Y the gold answer. But Eq. (1) estimates H(Y|C) where Y is the model's own answer span. For incorrect chains, that is the inspector's uncertainty about the wrong answer, not about the truth. The paper even calls Y 'ground-truth' in the estimation paragraph, which is an internal inconsistency. And Appendix B, in showing that cross-entropy is often lower on incorrect chains, gives a concrete hint that confidence in a wrong answer can be high. So the decreasing-entropy signature might in part be a signature of confidently wrong chains. The paper does not address this.\n\nThe second problem is that the central association is never quantified. Every claim rests on averaged curves without error bars, confidence intervals, or a per-problem test. The plots look clean, but 'strongly associated' needs an AUC or a correlation, and it's missing. That, more than the target mismatch, is what makes the paper a good idea rather than a finished result.\n\nMinor issues: the proposed pruning heuristic is never evaluated, and the oracle dependence (knowing correctness to split groups) is acknowledged but not solved.\n\nWho should read it? People working on test-time scaling, early stopping, or self-consistency pruning would want to know about this signal, and the negative results on baselines are worth having. But I wouldn't yet cite it as evidence for any deployed method.\n\nMy recommendation: send it for peer review, expecting major revision. The question is relevant, the observation is new, and the fixes are concrete: quantify the separation per problem, align the estimator with the definition or justify the proxy, and test the heuristic. A serious referee would push exactly those points.","headline":"New empirical observation, but the key claim rests on averaged curves and a target-mismatch the paper never addresses.","tokens_in":8670,"tokens_out":5380,"would_cite":false,"duration_ms":54745,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Reasoning chains whose answer-span entropy falls over steps are systematically more likely to be correct—flat or rising entropy marks errors.","keywords":["conditional entropy","reasoning utility","chain-of-thought","answer-span uncertainty","early stopping","MATH dataset","teacher forcing","entropy trajectory"],"falsifier":"Match correct and incorrect chains for answer-span length and problem difficulty; if the entropy-decrease separation disappears after matching, the signal is an artifact of length or difficulty. Alternatively, an early-stopping rule that halts when entropy rises should preserve or improve accuracy on held-out MATH problems if the trajectory is truly predictive.","tokens_in":7875,"feed_emoji":"📉","tokens_out":6683,"duration_ms":66351,"temperature":0.7,"pith_summary":"This paper tries to establish that the trajectory of a language model's uncertainty about its final answer—computed as conditional entropy over the answer span after each reasoning step—separates correct reasoning chains from incorrect ones. On the MATH dataset, chains that end in correct answers show entropy that decreases over steps, falls faster, and stays below the entropy of incorrect chains, whose entropy is flat or rising. The authors position this as an oracle study: the entropy signal is measured with a separate inspector model under teacher forcing, requires no reward model or training, and could let a system stop early or prune unproductive reasoning. If the pattern holds, it matters because current approaches spend compute on long chains and samples with diminishing returns; a training-free signal during generation would make reasoning pipelines cheaper and less prone to distraction. The paper also reports that incorrect chains are longer on average and that humans narrow the answer space earlier than LLMs do.","feed_headline":"Falling answer entropy across reasoning steps flags correct answers","feed_subtitle":"A training-free signal could prune chains whose uncertainty stays flat or rises—longer wrong chains spend more compute.","key_machinery":"The carrying object is the answer-span conditional-entropy trajectory: for a response split into reasoning steps Z≤k and answer span Y, the metric is the average per-token entropy Ht = −Σ_v p(v | C, y<t) log p(v | C, y<t) over Y, with the context C = [X; Z≤k] expanded step by step. This quantity estimates H(Y | X, Z≤k), the model's residual uncertainty about the answer after k reasoning steps. The trajectory is produced by repeating this computation for k = 0..K and aligning chains of different lengths onto a common step axis with cubic-spline interpolation (linear if steps ≤ 3), so domain-level averages can be compared. Its work in the argument is to translate 'reasoning utility' into a mea","core_discovery":"The central claim is that the conditional entropy of the answer span, H(Y | X, Z≤k), averaged token-wise over the generated answer under teacher forcing, is a proxy for how much reasoning so far has narrowed the space of plausible answers. Computing this quantity at each reasoning step yields an entropy trajectory. Across seven MATH categories and two generator models (GPT-4o and Qwen2.5-32B, inspected by Qwen3-8B), correct chains consistently show decreasing entropy that falls faster and remains below the entropy of incorrect chains; incorrect chains show flat or rising entropy, especially on lower-accuracy categories. The paper further finds that incorrect chains are significantly longer t","pith_inferences":["An intervention the paper does not run: halt generation at the first step where answer-span entropy rises, and measure final accuracy against full-chain accuracy. If the entropy signal is causal rather than merely correlated, accuracy should hold or improve while token cost drops.","On wrong chains the measured entropy is self-confidence about a wrong answer; a stronger test would compute entropy against a paraphrase or candidate answer set, to check whether 'decrease' reflects genuine information gain or growing overconfidence.","The cubic-spline averaging hides per-problem variation; a per-chain early-slope predictor evaluated on held-out MATH problems would tell whether the first few steps alone carry the signal.","The same trajectory analysis could be transferred to non-math reasoning (e.g., commonsense or symbolic tasks) where the answer span is less structured, testing whether entropy reduction is a general signature of useful reasoning."],"forward_implications":["Entropy trajectories provide a training-free, reward-model-free signal for whether a chain is on track, so a generator could prune or stop early during decoding.","A simple selection rule follows from the plots: discard chains whose answer-span entropy does not decrease; if all decrease, keep the ones with the steepest negative slope.","Because incorrect chains are longer on average, early pruning of flat or rising-entropy chains should cut compute as well as reduce answer distractions.","The human-vs-LLM difference suggests LLM decoding could be improved by strategies that narrow the answer space early rather than incrementally.","The metric can be computed from the model's own output logits, so it applies to any autoregressive LLM without extra training."],"supporting_citations":[{"why":"Supplies the MATH dataset, the benchmark on which all correctness comparisons and entropy trajectories are computed.","marker":"Hendrycks et al., 2021"},{"why":"Defines chain-of-thought prompting, the reasoning format whose per-step utility the paper measures.","marker":"Wei et al., 2022"},{"why":"Self-consistency motivates the pruning use-case: sampling many chains has diminishing returns, so early utility estimates are valuable.","marker":"Wang et al., 2023"},{"why":"Best-of-N and step-by-step verification are the compute-intensive baselines the entropy signal could replace or supplement.","marker":"Lightman et al., 2024"},{"why":"Introduces verifier-based reasoning evaluation, the model-based approach the paper contrasts with metric-based entropy.","marker":"Cobbe et al., 2021"},{"why":"Confidence-based reasoning metric whose weak accuracy correlation is tested and contrasted in the paper's appendix.","marker":"Zhou et al., 2025"},{"why":"Another confidence-based method the paper investigates and finds insufficient for correctness prediction.","marker":"Taubenfeld et al., 2025"},{"why":"Representative similarity-based pruning method whose weak link to accuracy the paper corroborates in appendix A.","marker":"Zhu et al., 2025"}],"fun_headline_variants":["Decreasing answer entropy across steps flags correct reasoning","Flat or rising entropy in reasoning predicts wrong LLM answers","Conditional entropy drop marks useful LLM reasoning, study finds","Longer reasoning chains often wrong; entropy signal shows","Entropy trend in reasoning steps reveals answer correctness"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The load-bearing premise is that the entropy the model assigns to the answer span it actually wrote reflects the reasoning's true information gain about the correct answer; on wrong chains this is self-confidence, not gold-answer information.","fun_headline_variants_meta":{"raw":{"variants":["Decreasing answer entropy across steps flags correct reasoning","Flat or rising entropy in reasoning predicts wrong LLM answers","Conditional entropy drop marks useful LLM reasoning, study finds","Longer reasoning chains often wrong; entropy signal shows","Entropy trend in reasoning steps reveals answer correctness"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000218,"raw_usage":{"total_tokens":1284,"prompt_tokens":758,"completion_tokens":526,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":502,"completion_tokens_details":{"reasoning_tokens":449}},"tokens_in":502,"tokens_out":526,"duration_ms":6358,"temperature":1.0,"reasoning_tokens":449,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T15:04:52.341402+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Match correct and incorrect chains for answer-span length and problem difficulty; if the entropy-decrease separation disappears after matching, the signal is an artifact of length or difficulty. Alternatively, an early-stopping rule that halts when entropy rises should preserve or improve accuracy on held-out MATH problems if the trajectory is truly predictive.","supporting_citations":[],"review_version":1}