{"id":"e3c69663-201f-48b3-bd47-03538975eae8","arxiv_id":"2502.03199","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A new decoding-time method, END, uses per-token cross-layer entropy of prediction growth to boost factual tokens, improving truthfulness and informativeness on hallucination benchmarks.","lead":"This paper introduces END, a decoding method that adjusts an LLM's next-token probabilities using per-token entropy of prediction changes across layers, so tokens whose probability rises sharply in later layers are favored. The authors report large gains on hallucination benchmarks such as TruthfulQA while keeping QA accuracy roughly unchanged.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Equation 3 defines cross-layer entropy with the wrong sign: Eq. 4 e^{-λ Entropy} amplifies flat, non-factual tokens and suppresses sharp ones, inverting the stated mechanism unless Eq. 3 is a typo.","rationale":"The reader's conditional verdict is reasonable, and I do not change it, but the strongest unresolved problem is more specific than the layer-window concern. The displayed mathematics appears to invert the method: Eq. 3 defines E as Σ q log q, i.e. -H, so a sharp factual token has E≈0 and a flat token over 15 layers has E≈-2.7. Eq. 4 then multiplies by e^{-λE}, giving flat tokens a large boost and sharp tokens nearly none. If this is a sign typo, the central mechanism survives; if not, the reported TruthfulQA gains cannot come from the described token-wise entropy. The paper does not report a sign-flip ablation or any non-cherry-picked entropy analysis, and the reader's weakest assumption about layer buckets is secondary: even with a perfect layer window, a wrong sign breaks the method. The concrete check is unambiguous because the code is public, so the issue is resolvable and does not by itself demand rejection of the underlying idea.","tokens_in":11579,"tokens_out":6356,"duration_ms":65779,"concrete_test":"Inspect the released END code (github.com/Arcade-Master/END): locate the cross-layer entropy computation. If it returns sum(q * log(q)), run the TruthfulQA open-ended protocol once as-is and once with -sum(q * log(q)); the direction of the effect should reverse or disappear. If the code uses -sum(q * log(q)), treat Eq. 3 as a typo and rerun the main table after stating the sign convention explicitly, including a sign-flip ablation. Additionally, verify on the Figure 2 example that the sharply growing 'Sun' token receives a larger adjusted multiplier than a flat token, as the stated low-entropy rule requires.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Equations (2)-(4) are internally inconsistent with the paper's mechanism. For a fixed candidate token, q_l(vt) = P_l(vt)/Σ_{i∈Layer}P_i(vt) is a probability distribution over the selected layers. Eq. 3 then defines E = Σ_l q_l log q_l, which is the negative Shannon entropy: E∈[-ln|Layer|, 0], with E→0 for a sharply peaked distribution and E→-ln|Layer| for a flat one. The paper says 'low cross-layer entropy value represents a sharp predicting distribution', but under Eq. 4 the multiplier is e^{-λE}. For a flat token e^{-λE}=|Layer|^λ > 1, while for a sharp token the factor is ≈1; after renormalization, flat/non-factual tokens are boosted and sharp/factual tokens are suppressed. To obtain the claimed behavior one needs E = -Σ q log q, or equivalently e^{+λE}. The released code may correct the sign silently, but as printed the headline mechanism is inverted, and no ablation reports which sign is used or how a sign flip changes Table 1.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a training-free decoding intervention, cross-layer Entropy eNhanced Decoding (END), which uses the evolution of a candidate token's probability across upper transformer layers to quantify how much factual knowledge the token requires. The method reweights the final-layer next-token distribution by e^{-λ Entropy(v)}, where Entropy(v) is defined from a normalized cross-layer probability distribution, with the aim of amplifying tokens whose probabilities grow sharply in the higher layers. Experiments on TruthfulQA, FACTOR, TriviaQA, and Natural Questions across LLaMA-2-7B/13B/70B-chat and two other instruction-tuned models report substantial gains in truthfulness and informativeness while preserving QA accuracy. The paper also provides qualitative case studies and a throughput comparison.","tokens_in":11963,"tokens_out":4088,"duration_ms":38573,"significance":"If the stated mechanism were correct and the empirical results were robust, END would offer a simple, training-free method for reducing hallucinations by exploiting internal state dynamics, complementing existing contrastive and activation-based decoding approaches. The paper's central observation—that factual token candidates show distinct layer-wise probability growth—is plausible and consistent with prior work on layer-wise knowledge emergence. The authors also release code and provide qualitative examples, which are useful for reproducibility. However, the significance of the contribution is currently limited by an apparent sign inversion in the core equation, per-benchmark hyperparameter tuning without uncertainty quantification, and an insufficiently justified layer-selection procedure.","major_comments":[{"comment":"Equation (3) defines Entropy(v_t) = Σ_l q_l(v_t) log q_l(v_t), where q_l forms a probability distribution over the selected layers. This quantity is the negative Shannon entropy: it lies in [-ln|L|, 0], equals 0 for a sharply peaked distribution, and equals -ln|L| for a flat distribution. Substituting into Eq. (4), the factor e^{-λ Entropy(v_t)} becomes |L|^λ for a flat distribution and approximately 1 for a sharp distribution. Thus, after normalization, the method as printed suppresses the sharp, 'factual' tokens and amplifies flat, 'non-factual' tokens—the exact opposite of the stated mechanism. The text explicitly claims that 'low cross-layer entropy value represents a sharp predicting distribution' and should be prioritized, so either the sign in Eq. (3), the sign in the exponent of Eq. (4), or the interpretation must be corrected. The paper must state which version was actually implemented, and ideally report an ablation showing the effect of the sign on Table 1.","section":"Section 4.1, Eqs. (3) and (4)"},{"comment":"The hyperparameters λ and α are tuned on validation runs of each benchmark, and the reported results are then obtained on the same benchmarks with the selected values. This makes the headline gains in Tables 1, 2, 4, and 5 selected results rather than predictions of a fixed method. No error bars, multiple seeds, or significance tests are reported, so it is unclear whether the improvements over greedy decoding and DoLa are statistically distinguishable from tuning noise. The paper should provide sensitivity curves over λ and α, standard deviations across repeated runs (including different random seeds where applicable), and, ideally, a hold-out evaluation protocol.","section":"Section 5.1, Implementation Details"},{"comment":"The method's core quantity, the cross-layer distribution in Eq. (2), depends critically on which layers are included in the set 'Layer'. The paper merely states that the same bucket-based strategy as DoLa is used, without specifying the exact layers or providing any justification or sensitivity analysis for END. The only empirical support for the sharp-growth phenomenon is a single LLaMA-2-7B example in Figure 2, which is not sufficient to establish that a fixed layer set works across tokens, contexts, and model families. The authors should report the layer set actually used for each model, show how the results vary with the choice of layer range, and present quantitative evidence of the sharp-versus-flat distinction over a larger sample of tokens.","section":"Section 3 and Section 4.1"},{"comment":"The TruthfulQA open-ended evaluation switched from the Curie-based fine-tuned GPT-3 evaluator to Davinci-002. Although the baselines were re-evaluated with the new evaluator, this change may alter both absolute scores and relative rankings, and the paper provides no analysis of how the evaluator change affects comparisons with previously published numbers (e.g., DoLa's original scores). The authors should either report both evaluator versions on a common subset or discuss the potential impact of the evaluator change on the validity of the TruthfulQA comparisons.","section":"Section 5.1, Footnote 2 and Table 1"}],"minor_comments":[{"comment":"There are several typos and inconsistent spellings that should be corrected, including 'deocding' in Section 5.1, 'Misrtral' in Section 5.4, 'Unlikely' where 'Unlike' is meant in Section 2, 'voabulary-level' in Section 3, and 'DOLA' versus 'DoLa' in Section 6.3 and the references.","section":"Throughout"},{"comment":"The caption and the figure itself are hard to read: the left panel's row and column labels are not clearly explained, and the right panel's numeric values and token names are cramped. Please enlarge the figure and clarify what each panel displays, including the exact definition of the KL-divergence and the layer indices.","section":"Figure 2"},{"comment":"The note that multiple-choice scores are 'obtained from previous authorized work' should be expanded to identify the exact source and version of the baseline numbers, so readers can confirm that the evaluation conditions match those of the current experiments.","section":"Section 5.2, Table 1"},{"comment":"The definition of V_head should state explicitly whether the threshold uses the original final-layer probabilities P_N or the adjusted probabilities P_Final; as written, it could be misread as applying the threshold after reweighting.","section":"Section 4.2, Eq. (5)"}],"recommendation":"major_revision","confidential_remarks":"The sign inversion in Eqs. (3)-(4) is the most important issue and goes to the core of the paper: as printed, the method cannot work as claimed. If the authors can confirm the correct sign and provide code-based evidence that the reported numbers are produced by a consistent implementation, the manuscript could become publishable after substantial revision. I would also ask the editor to encourage the authors to share the exact layer sets and hyperparameter grids, as the current manuscript leaves too much unspecified for a decoding method that is supposed to be generally applicable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: END is a plausible token-wise extension of DoLa, and the open-ended TruthfulQA numbers suggest the idea has legs, but the paper as printed contains a sign inconsistency in Eqs. (3)-(4) that reverses the stated mechanism, and the evaluation protocol has enough loose ends that the central claims need another pass.\n\nWhat's new and what works. The per-token cross-layer entropy statistic - normalizing each candidate's probability across a set of upper layers and then reweighting the final distribution by e^{-lambda Entropy} - is genuinely different from DoLa's single-layer contrast and from Activation Decoding's in-context sharpness. The motivation is sound: a fixed 'caliber layer' can't capture that different factual candidates grow at different layer positions. The empirical pattern is consistent: every LLaMA-2 scale (7B/13B/70B) improves on TruthfulQA open-ended truth*info, Mistral and Qwen improve, and TriviaQA/NQ accuracy doesn't degrade. Throughput is better than DoLa. Code is promised. The limitation section is honest about the absence of a theory and about the method's inability to add knowledge the model lacks.\n\nWhere it gets soft. The sign problem is real. Eq. (3) defines E = sum q log q, which is negative entropy: E is near 0 for a sharp distribution and near -ln|L| for a flat one. Eq. (4) then multiplies by e^{-lambda E}, which boosts flat tokens and leaves sharp ones roughly alone. That is the opposite of what the paper says and opposite of what Figure 3 depicts. Either Eq. (3) should be E = -sum q log q, or Eq. (4) should use e^{+lambda E}. As printed, the headline mechanism is inverted. The released code may silently use the correct sign; the paper needs to state it and, ideally, ablate the sign flip.\n\nThe evaluation also needs care. lambda and alpha are tuned per benchmark via validation runs on the same benchmarks used for the headline numbers; that's selected, not fixed. The TruthfulQA open-ended grader was switched from Curie to Davinci-002, and the paper doesn't report both grades or show the switch doesn't change relative ordering. No error bars or significance tests anywhere. The layer set is taken from DoLa's bucket strategy without sensitivity analysis, and the supporting visualization is a single LLaMA-2-7B example (Figure 2). None of these individually destroys the contribution, but together they make the 21-point truth*info improvement look more fragile than it may be.\n\nBottom line: this deserves a serious referee. The core idea is new enough and the gains are consistent enough to warrant a careful check of the code and a re-run with fixed hyperparameters. If the sign is a typo, the method is a legitimate incremental contribution. As submitted, the sign error and the evaluation protocol prevent me from trusting the headline numbers.","headline":"A plausible token-wise refinement of DoLa with a sign error in Eqs. (3)-(4) that inverts the mechanism as printed, plus an evaluation protocol that needs a tighter pass before the headline gains can be trusted.","tokens_in":12335,"tokens_out":3701,"would_cite":false,"duration_ms":32072,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A token's probability growth across upper layers marks factual knowledge, and a decoding rule built on that signal reduces hallucination without retraining.","keywords":["hallucination mitigation","cross-layer entropy","decoding strategy","factuality","token-level analysis","training-free","truthfulness","language models"],"falsifier":"Rerun TruthfulQA open-ended generation with the selected layer set shifted by two or three layers in either direction; if the truthfulness gain disappears or reverses, the entropy is measuring the chosen window rather than token-level factuality. A second check would be to inspect cases where the sharpest-growing candidate is not the model's final answer and test whether such tokens are usually factually wrong, which would weaken the claim that the proxy tracks answer correctness.","tokens_in":11368,"feed_emoji":"🎯","tokens_out":5786,"duration_ms":51041,"temperature":0.7,"pith_summary":"This paper tries to establish that a large language model's own internal probability trajectories contain a per-token factuality signal: candidate tokens that jump sharply in the upper layers are the ones carrying factual knowledge, while flat tokens are not. On that basis it proposes cross-layer Entropy enhanced Decoding (END), a training-free decoder that reweights the final next-token distribution so sharp-growing tokens are preferred. If the paper is right, hallucination can be reduced without new training, retrieval, or external knowledge, simply by reading the model's existing computation. The paper supports this with experiments on TruthfulQA, FACTOR, TriviaQA, and Natural Questions across several model families and scales, reporting large gains in truthfulness and informativeness while preserving question-answering accuracy.","feed_headline":"Factual tokens jump in late layers; new decoding uses that signal","feed_subtitle":"Training-free END raises TruthfulQA truth-informativeness by up to 21.8 points while keeping QA accuracy.","key_machinery":"The central object is the cross-layer entropy of a candidate token. For each token $v_t$, the paper collects prediction probabilities $P_l(v_t)$ over a chosen set of upper layers, normalizes them as $q_l(v_t) = P_l(v_t)/\\sum_{i \\in \\mathrm{Layer}} P_i(v_t)$, and defines $\\mathrm{Entropy}(v_t) = \\sum_{l \\in \\mathrm{Layer}} q_l(v_t) \\log q_l(v_t)$. The final decoding distribution is $P_{\\mathrm{final}}(v_t) = e^{-\\lambda \\mathrm{Entropy}(v_t)} P_N(v_t)$, where $N$ is the final layer and $\\lambda$ controls the intervention strength. This entropy is what carries the argument: a sharply peaked cross-layer distribution indicates the token's probability is actively growing in the higher layers, which the paper identifies with factual knowledge, and the exponential factor converts that signal into a decoding preference.","core_discovery":"The paper claims that at the individual-candidate-token level, tokens requiring factual knowledge (names, dates, locations) show a sharp growing trend in prediction probability across the upper transformer layers, whereas functional or easy tokens stay nearly flat. To quantify this trend, the paper defines a cross-layer probability distribution for each candidate token from its prediction values over selected higher layers, normalizes those values, and computes a cross-layer entropy. A low entropy means the token's probability is sharply concentrated in the upper layers, which the paper reads as a signal of factual knowledge and answer correctness. END then adjusts the final-layer distribution by multiplying it with $e^{-\\lambda \\mathrm{Entropy}(v_t)}$, suppressing flat tokens and amplifying sharp-growing ones, with a filtering step that restricts the computation to high-probability candidates. The paper reports that this adjustment substantially improves TruthfulQA open-ended truth and informativeness scores, reduces refusal responses, and maintains or slightly improves multiple-choice and QA accuracy.","pith_inferences":["The paper does not pursue the natural extension of using cross-layer entropy as a token-level uncertainty or hallucination detector, for example to trigger abstention or retrieval exactly when the sharp-growth signal is absent.","A variant the paper does not test is making the layer window adaptive per token or per question; if the knowledge-emergence window shifts with token type, a fixed bucket could be replaced by a learned or heuristic selection.","Because END amplifies existing knowledge rather than adding new knowledge, a stress test with deliberately outdated or false memorized facts would clarify how the method behaves when the model is confidently wrong, a boundary the paper itself states in its limitation.","The cross-layer entropy signal could plausibly be combined with contrastive decoding or activation sharpness rather than compared against them, since the mechanisms target different failure modes; this combination is left implicit in the paper."],"forward_implications":["On TruthfulQA open-ended generation, END raises the Truth*Info score by 12.24 to 21.79 percentage points over greedy decoding, DoLa, and Activation Decoding, while cutting the rejection rate by up to 14.81 points.","END preserves or slightly improves question-answering ability, with MC1 up 0.9 points on LLaMA-2-7B and relative Exact Match gains of about 5.6% on TriviaQA and 10.1% on Natural Questions.","The gains hold across model scales from 7B to 70B LLaMA-2 and across different backbones such as Mistral and Qwen, with the paper noting that a very weak base model can show disruptive behavior.","Because the method only reweights among high-probability candidates, it adjusts the model's original prediction without replacing it, which the paper credits for preserving QA accuracy while improving factuality.","The decoding overhead is modest: on a 7B model END achieves about 36.1 tokens per second versus 39.4 for greedy and 35.5 for DoLa, making it practical for open-ended generation."],"supporting_citations":[{"why":"Supplies the observation that higher-layer prediction distributions keep fluctuating for factual tokens and provides the layer-bucket selection strategy that END reuses to build its cross-layer distributions.","marker":"Chuang et al., 2023"},{"why":"Reports the context-activation sharpness signal linking inner representations to answer correctness, and supplies the Activation Decoding baseline and multiple-choice scores END compares against.","marker":"Chen et al., 2024"},{"why":"Motivates candidate filtering; END adopts the head-filtering idea to restrict cross-layer entropy computation to high-probability tokens for efficiency.","marker":"Li et al., 2022"},{"why":"Supplies TruthfulQA, the main benchmark whose open-ended truth and informativeness scores carry the paper's headline improvements.","marker":"Lin et al., 2021"},{"why":"Supplies FACTOR, the long-context reading-comprehension benchmark used to test factuality beyond open-ended generation.","marker":"Muhlgay et al., 2023"},{"why":"Supplies Natural Questions, one of the QA benchmarks used to show that END preserves general question-answering ability.","marker":"Kwiatkowski et al., 2019"},{"why":"Supplies TriviaQA, the second QA benchmark used to demonstrate preserved accuracy alongside factuality gains.","marker":"Joshi et al., 2017"}],"fun_headline_variants":["Token-wise cross-layer entropy decoding curbs LLM hallucinations","Training-free decoding amplifies low-entropy factual tokens","Cross-layer entropy picks factual tokens, cuts hallucination","Sharp token probability jumps signal facts; END amplifies them","Token-wise entropy decoding: curbs hallucination without training"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole signal rests on the assumption that the fixed set of upper layers, chosen by a DoLa-style bucket strategy and applied uniformly to all tokens, is the window where factual knowledge emerges, so that the normalized cross-layer distribution measures what the method claims to measure.","fun_headline_variants_meta":{"raw":{"variants":["Token-wise cross-layer entropy decoding curbs LLM hallucinations","Training-free decoding amplifies low-entropy factual tokens","Cross-layer entropy picks factual tokens, cuts hallucination","Sharp token probability jumps signal facts; END amplifies them","Token-wise entropy decoding: curbs hallucination without training"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000675,"raw_usage":{"total_tokens":3042,"prompt_tokens":888,"completion_tokens":2154,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":504,"completion_tokens_details":{"reasoning_tokens":2075}},"tokens_in":504,"tokens_out":2154,"duration_ms":14037,"temperature":1.0,"reasoning_tokens":2075,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T05:32:22.943361+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Rerun TruthfulQA open-ended generation with the selected layer set shifted by two or three layers in either direction; if the truthfulness gain disappears or reverses, the entropy is measuring the chosen window rather than token-level factuality. A second check would be to inspect cases where the sharpest-growing candidate is not the model's final answer and test whether such tokens are usually factually wrong, which would weaken the claim that the proxy tracks answer correctness.","supporting_citations":[],"review_version":1}