{"id":"3d69097f-dc87-4ec6-a700-2c46220af8db","arxiv_id":"2607.18292","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Hallucination in LLMs is driven by an oracle-invisible “decoding risk” term that grows with scale and causally compounds errors within a response.","lead":"Larger language models get more accurate at the start of their answers but hallucinate more as they continue, and this paper ties the decay to a hidden “decoding risk”—the wobble between the model's token choices and a stronger oracle's. The result says hallucination is not just missing knowledge, and detectors that only read a model's own confidence will miss the dominant failure mode.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Oracle-as-truth is the load-bearing premise: if the larger model's errors correlate with scale, the measured risk mechanism is disagreement with a fallible referent, not hallucination.","rationale":"The reader's weakest_assumption correctly identifies the oracle-as-truth premise as load-bearing. It enters at Equation (1), is required for the risk decomposition's interpretability claim, and is needed for every empirical result from Section 3 onward. The paper does provide substantial internal support: the algebraic identity is exact, the onset half-life asymmetry is reproduced under oracle-size and oracle-family changes, the manipulation check confirms mean preservation, and the verifier-independent marker gives AUROC 0.68–0.71. However, none of these establish that the oracle's errors are uncorrelated with scale or with the measured risk. The manuscript's own Limitations statement concedes verifier error could propagate, and the causal test is described as a ceiling, not a deployable rule. Because the central abstract claims 'dominant, self-perpetuating, causal and invisible' go beyond what the oracle-relative evidence can prove without gold-truth validation, the conditional verdict is appropriate. I would not change the reader's verdict: the core empirical direction is credible, but the strong causal/invisible framing should remain conditional until the oracle-as-truth concern is settled or the claims are softened.","tokens_in":26993,"tokens_out":3951,"duration_ms":42761,"concrete_test":"","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim—that decoding risk Var[δ] is dominant, self-perpetuating, causal, and invisible—rests on treating the oracle p_O as a truth proxy. Equations (1)–(3) define all quantities relative to p_O; the scaling results of Section 3, the onset asymmetry of Section 3.2, and the variance-contraction intervention of Section 4.3 all use a same-family larger model (or DeepSeek-V3 in Appendix D) as the referent. If the oracle is itself confidently wrong, or if oracle error correlates with model scale or topic, then what is called 'knowledge degradation' and 'risk' is disagreement with a fallible model rather than error against truth. The paper's Limitations explicitly concede verifier error ('verifier error could propagate') and that the causal test is a ceiling, but the only guard offered is a white-box marker agreeing with verifier labels at AUROC 0.68–0.71—a weak floor that does not establish oracle calibration. Because every empirical claim, including the 35–74% hallucination reduction, is filtered through this oracle, a systematic oracle bias could reshape the mechanism without any internal inconsistency. The independent support (manipulation check, cross-family checks, verifier label agreement) is real but does not dissolve the assumption: those checks confirm the model disagrees with the oracle, not that the oracle is right.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes that hallucination in long-form generation is driven by a decoding-time risk term that is invisible to the model's own uncertainty signals. The central object is the per-position disagreement δ = log p_M − log p_O between the model and a stronger oracle. Its second moment splits exactly into a bias term KL(p_M∥p_O)² and a decoding-risk term Var[δ] (Eq. 2). The paper argues that as models scale, this risk term grows as a share of the squared error, persists after a fabrication onset while entropy relaxes, forms a 'precarious' regime that bridges consecutive fabrications, is not readable by p_M-only detectors, and causally mediates downstream hallucinations. The claims are supported by scaling analyses on three model families and three benchmarks, onset-aligned trajectory analyses, a three-state Markov-switching regime model, and an in-silico variance-contraction intervention that removes 35–74% of web-verified hallucinations.","tokens_in":27373,"tokens_out":6229,"duration_ms":68932,"significance":"If the central claims hold, the paper would reframe hallucination from a knowledge-gap phenomenon to a self-conditioning decoding-risk phenomenon, with direct implications for detector design and for the interpretation of scaling laws. The paper has notable strengths: the identity in Eq. (2) is exact and correctly stated; bootstrap CIs are used throughout; the onset dynamics are rechecked against a cross-family oracle (Appendix D); the main intervention is explicitly labeled as an oracle-in-the-loop ceiling; and the experimental protocol is unusually transparent about seeds, trajectories, and verifier dependencies. The white-box risk marker (Appendix H) is a useful label-free contribution. However, the causal and interpretive claims depend on assumptions that are only partially defended: the oracle is treated as a truth proxy, and the headline causal estimate is based on a post hoc selected best arm.","major_comments":[{"comment":"The headline causal reduction (35–74%) is reported for the best arm selected post hoc over the causal grid K∈{0,1,2} × ρ∈{0.5,0.75}. The 95% bootstrap CIs for this selected arm are not adjusted for the selection, so they overstate significance. The dose-response paragraph in Appendix F partially mitigates this, but Table F.2 still reports only the best arm per rung. Please report all grid cells (or a pre-specified selection rule) and apply a multiple-comparison correction, or state explicitly that the CIs are conditional on the post hoc selection.","section":"§4.3, Table F.2"},{"comment":"The entire risk decomposition is relative to the oracle p_O, and the paper's 'knowledge degradation', 'risk', and 'fabrication' claims are measured as disagreement with that oracle. The cross-family oracle check in Appendix D shows the asymmetry is not a same-family artifact, but it does not establish that the oracle is calibrated with respect to factual truth. If the oracle's errors correlate with scale or topic, the mechanism becomes disagreement with a fallible referent rather than hallucination. The verifier labels provide some external grounding, but the paper's own Limitations concede verifier error. Please provide a direct calibration check of the oracle against verifier labels, or reframe the claims as risk-against-a-stronger-model and show separately that this risk predicts verifier-labeled hallucinations.","section":"§2, Eqs. (1)–(3); Appendix D"},{"comment":"The intervention fires only at positions where the oracle-informed trigger detects a large divergence (Z=2 above the per-model mean). The manipulation check shows that the contraction changes variance and not the mean gap, but it does not isolate the variance reduction from the trigger mechanism itself. A control that applies the same contraction at non-trigger positions (or at random positions) is needed to rule out the possibility that the effect is driven by the oracle's detection signal rather than by the variance contraction. Without this control, the causal attribution to 'decoding risk' is not fully identified, even as an oracle-in-the-loop ceiling.","section":"§4.3, Fig. 6"}],"minor_comments":[{"comment":"The phrase 'bias² KL(pM ∥ pO)²' is redundant; the square of the KL divergence is already bias². Also, calling H(p_M) 'self-readable uncertainty' is confusing because H is entropy, not variance; the distinction is central to the paper and could be clarified at first use.","section":"Abstract"},{"comment":"The table reports p<10^{-3} (one-sided Mann–Whitney) while the main text states p<10^{-16}. Please reconcile these numbers or state the exact p-values.","section":"Table 2 vs §4.4"},{"comment":"The abbreviation 'MSAR' is used without definition. If it means Markov-switching autoregressive, please spell it out.","section":"Appendix H"},{"comment":"The caption uses 'H(pS)' in what appears to be a typo for H(p_M).","section":"Figure 2"}],"recommendation":"major_revision","confidential_remarks":"The paper is ambitious and the core identity is correct, but the causal claim rests on a post hoc selected arm and an oracle-as-truth assumption that the paper does not fully defend. These are fixable with additional analyses or more careful framing. I would not reject, but the current version is not ready for acceptance without addressing the selection and calibration concerns."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe one thing to know: this paper has a genuine empirical finding — across three model families, larger models show a larger within-response rise in hallucination rate even as initial knowledge improves — and a mechanistic story built on a variance term that grows with scale. The math itself is an identity (E[δ²] = KL² + Var[δ]), so it is not the contribution. The contribution is that this split predicts onset dynamics, persistence, and an intervention effect across families. That is worth a serious referee.\n\nThe strongest part is the scaling and onset analysis. The knowledge-gap-versus-degradation reversal (7× improvement vs 39× degradation) is striking, with bootstrap CIs and a mixed-effects fit that addresses the shrinking-denominator artifact. The persistence asymmetry — H(p_M) relaxing within a token while oracle-referenced risk lingers — is well demonstrated with oracle-size and cross-family checks. The outcome labels come from a web verifier, not from the oracle, so the main scaling results are not purely a disagreement-with-a-fallible-model artifact. Credit where due: this is careful, reproducible-looking work, with manipulation checks, confound tables, and a candid limitations section.\n\nSoft spots, in order. First, the causal 35–74% reduction is a ceiling, as the authors state, but the abstract still sells it as the headline. The best arm is selected post hoc over a grid, the bridge analysis has n ≈ 15–31 per cell, and the trigger uses oracle knowledge. That makes the causal claim suggestive, not established. Second, the oracle-as-truth assumption is real. Every risk measure is disagreement with a larger same-family model (or DeepSeek in the appendix). Cross-family and verifier checks blunt the concern but do not dissolve it: if oracle error correlates with scale, part of the anti-scaling could be disagreement-with-a-fallible-referent rather than error-against-truth. The paper concedes verifier error but does not calibrate the oracle. Third, the 'invisible to the model itself' claim is only proven for pM-only detectors; the paper says this explicitly, but the abstract's language goes further.\n\nOverall: the observational scaling result is credible and deserves publication. The causal mechanism needs either softer language or stronger evidence — pre-registered grid, larger bridge samples, oracle calibration. This paper is for anyone working on hallucination detection or reliability scaling; it will sharpen the debate even if the strong causal form does not survive.\n\nMy recommendation: send it to peer review. I would accept with major revision, and the core empirical direction is worth refereeing.","headline":"Reliability anti-scaling is a real empirical package worth taking seriously, but the causal and 'invisible' claims run ahead of the evidence.","tokens_in":27836,"tokens_out":2516,"would_cite":true,"duration_ms":26271,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that hallucination in larger language models is driven by a decoding-time risk that grows with scale and is invisible to the model itself.","keywords":["hallucination","large language models","inverse scaling","decoding risk","uncertainty","reliability","autoregressive decoding","oracle disagreement"],"falsifier":"Use a benchmark with ground-truth labels (e.g., arithmetic word problems or code execution) and construct two oracles: one known to be highly accurate and one deliberately poisoned or miscalibrated with scale. If the risk term Var[δ] and the intervention's effect are largely unchanged when measured against the poisoned oracle, then the paper's mechanism is a property of divergence from any referent rather than of hallucination; if the effect tracks only the accurate oracle, the truth-referent is essential.","tokens_in":26842,"feed_emoji":"🎲","tokens_out":8083,"duration_ms":68728,"temperature":0.7,"pith_summary":"The paper's central claim is that long-form hallucination in language models is not primarily a knowledge gap but a decoding-time risk: the spread between the model's token probabilities and a stronger oracle's. This risk grows with model scale even as the model's start-of-response knowledge improves, and once a fabrication is committed the risk persists long after the model's felt uncertainty relaxes, making the next fabrication likelier. The paper proves an exact identity splitting model–oracle disagreement into a bias term (which includes the model's own entropy) and a risk term (which does not), and demonstrates by intervention that shrinking the risk at fixed bias removes 35–74% of web-verified hallucinations across three model families. If correct, reliability is a distinct scaling axis that worsens with parameters and is structurally invisible to standard self-monitoring detectors.","feed_headline":"Hallucinations snowball faster as models grow","feed_subtitle":"A decoding risk invisible to the model itself grows with scale and causally drives long-form errors.","key_machinery":"The central object is the disagreement variable δ = log p_M − log p_O between the model and a stronger oracle, together with the exact identity E[δ²] = KL(p_M∥p_O)² + Var[δ], which splits squared error into bias² and decoding risk. The identity does interpretative work before statistical work: the model's self-readable entropy H(p_M) appears only inside the bias term, so the risk term is structurally inaccessible from the model's own probabilities. The causal test is a mean-preserving variance contraction qλ ∝ p_M exp(−λ(δ−μ)² + νδ), which lowers Var[δ] while holding E[δ] fixed (bias drift kept ~11 orders of magnitude below the bias), isolating the decoding-risk channel from the mean-gap cha","core_discovery":"The paper's central discovery is that the per-position disagreement δ = log p_M − log p_O between a model and a stronger oracle has a second moment that splits exactly into bias² = KL(p_M∥p_O)² and decoding risk Var[δ]. The model's own uncertainty H(p_M) enters only the bias term, so the risk term has no model-readable component. Empirically, the risk term takes a growing share of the squared error as models scale (31% to 49% from 1.7B to 14B), persists 23× longer than felt uncertainty after a fabrication onset, and causally drives subsequent fabrications: a mean-preserving contraction of the variance at fixed KL removes 35–74% of web-verified hallucinations across six model×family rungs. Th","pith_inferences":["The decomposition implies a quantitative ceiling on any pM-only hallucination detector: since the risk term has no model-readable component, the best such detector can do is catch the bias side plus whatever transient entropy signal appears at onset. Measuring detector recall on the 'precarious' regime (low entropy, high risk) would test this bound directly.","Because the mechanism is attributed to autoregressive decoding itself, it should transfer to any long-form generator for which a stronger oracle can be defined—code generation, summarization, multimodal captioning—where the same variance contraction may reduce fabrication; the paper's evidence is limited to English parametric text.","The label-free Markov-switching risk marker (AUROC 0.68–0.71 across scale) suggests a deployable online rule: when the model enters the high-risk, low-bias 'precarious' state, it could trigger verification or abstention before the next fabrication is committed, without needing an oracle at inference time beyond the model's own logits."],"forward_implications":["Scaling a model family improves the start-of-response knowledge gap up to 7× but increases within-response knowledge degradation up to 39×, so capability and reliability are distinct scaling axes.","Risk's share of squared model–oracle error grows from 31% to 49% between 1.7B and 14B, and the disagreement tail grows heavier (excess kurtosis 8.9→33.0), meaning larger models agree with the oracle more often but diverge more when they do.","A committed fabrication raises the probability of the next fabrication by a factor that grows with scale (1.08× to 1.71× on a topic-controlled sample), and a confident-but-precarious regime (low entropy, high risk) increasingly occupies the bridge between consecutive fabrications, up to +69% at 14B.","Contracting decoding risk at fixed mean gap reduces rest-of-response web-verified hallucinations by 35–74% across six model×family rungs, with every bootstrap CI excluding zero; confounds like length, claim count, and diversity move insignificantly.","Semantic entropy, a detector that reads only the model's probabilities, fires 28–34% less on the inter-claim bridge than at onset despite the bridge carrying ~4× as many fabrications, consistent with the claim that pM-only detectors are blind to the dominant failure mode."],"fun_headline_variants":["Reliability drops as LLMs scale: hallucinations snowball","Scaling up LLMs worsens hallucination snowballs","Bigger LLMs: hallucination risk grows and self-sustains","Model size amplifies invisible hallucination risk","LLM scaling: hallucinations cascade faster, harder to see"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that a stronger same-family (or cross-family) model serves as a truthful oracle, so every 'knowledge gap,' 'risk,' and 'degradation' is measured as disagreement with that oracle; if the oracle is confidently wrong or miscalibrated in a way that correlates with scale, the mechanism is disagreement with a fallible model rather than error against truth.","fun_headline_variants_meta":{"raw":{"variants":["Reliability drops as LLMs scale: hallucinations snowball","Scaling up LLMs worsens hallucination snowballs","Bigger LLMs: hallucination risk grows and self-sustains","Model size amplifies invisible hallucination risk","LLM scaling: hallucinations cascade faster, harder to see"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000224,"raw_usage":{"total_tokens":1357,"prompt_tokens":866,"completion_tokens":491,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":610,"completion_tokens_details":{"reasoning_tokens":410}},"tokens_in":610,"tokens_out":491,"duration_ms":4803,"temperature":1.0,"reasoning_tokens":410,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T09:13:59.777338+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Use a benchmark with ground-truth labels (e.g., arithmetic word problems or code execution) and construct two oracles: one known to be highly accurate and one deliberately poisoned or miscalibrated with scale. If the risk term Var[δ] and the intervention's effect are largely unchanged when measured against the poisoned oracle, then the paper's mechanism is a property of divergence from any referent rather than of hallucination; if the effect tracks only the accurate oracle, the truth-referent is essential.","supporting_citations":[],"review_version":1}