{"id":"20a89418-ee99-41c9-b830-0b5d39f5da3e","arxiv_id":"2506.08371","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Positional Contrastive Decoding, a training-free method that contrasts standard and over-rotated RoPE logits, improves long-context retrieval and QA by a few points.","lead":"LLMs lose accuracy on long inputs, even though the correct answer often ranks high in their internal scoring. This paper introduces a training-free decoding tweak that contrasts the model's normal attention with an over-rotated 'local' variant to boost long-range signals.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline 8k KV retrieval gain (72.0→79.0) may stem from selecting hyperparameters on InfiniteBench itself; no error bars or code are provided, so the SOTA claim is not yet verified.","rationale":"The reader's weakest assumption is the spectral analysis in Appendix A.1, and that concern is legitimate: Lemma A.4 claims a finite sum of cosines decays to zero, which is mathematically false, and the lower bound in Eq. (20) grows with k, making the proof invalid. However, I argue that the more load-bearing concern for the paper's central claim is the empirical evaluation. The paper's value proposition is that PCD improves long-context performance, and the strongest evidence is the 8k KV retrieval gain. Because Table 1 tunes hyperparameters on InfiniteBench and Table 2 reports InfiniteBench results, the gain may be an artifact of test-set selection. The reported variance (up to 3.1 points) is comparable to the claimed improvement, and no error bars or code are provided. Even if the theory were fixed, an overfit empirical result would not support the SOTA claim. Conversely, a robust held-out evaluation would support the method even with the theory removed. Thus the empirical validation is the linchpin. The reader's conditional verdict already requires held-out hyperparameter selection, error bars, and code, which I support. My stress-test therefore keeps the verdict at CONDITIONAL while shifting the emphasis to the empirical reliability of the headline number.","tokens_in":11128,"tokens_out":12208,"duration_ms":143850,"concrete_test":"Recompute the InfiniteBench KV retrieval result with hyperparameters selected only on a held-out validation split (e.g., the RULER tasks or a 50% holdout of InfiniteBench), then test on the held-in portion with 5+ seeds and report 95% CIs. If the 8k accuracy gain over base falls below 3 points or fails significance, the headline gain is attributable to test-set tuning.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim rests on Table 2's 8k KV retrieval improvement from 72.0 to 79.0 on Llama-3-8B-262k. However, Table 1 reports hyperparameter selection (α=0.2, β=2.5, B'/B=1e-4, γ=30) performed on the same InfiniteBench benchmark used in Table 2, with reported variance as high as 3.1 accuracy points. Selecting parameters on the test set can inflate gains; the 7-point improvement is comparable to the parameter-selection variance, so it may not reflect a real method advantage. No error bars are given for Table 2, and no code or evaluation scripts are released, so the result cannot be independently checked. If the gain shrinks or vanishes under held-out hyperparameter selection, the paper's SOTA claim collapses, regardless of the validity of the spectral analysis. The theoretical flaw in Theorem A.5 is real, but it is secondary: even a correct theory would not rescue an overfit empirical result.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies why long-context LLMs degrade despite large nominal context windows. It defines a salience score S(L), observes that the gold token's rank in the decoding distribution shrinks as context grows, and proposes Positional Contrastive Decoding (PCD), which contrasts standard logits L with logits L* obtained by over-rotating low-frequency RoPE components using a transition function and a smaller base B', then computes L~ = (1+β)L − βL* on a top-γ subset. A spectral analysis in Appendix A.1 is offered as a mechanism, and experiments on RULER, InfiniteBench, and LongBench are used to claim consistent gains, including 8k KV retrieval accuracy on Llama-3-8B-262k rising from 72.0 to 79.0.","tokens_in":11325,"tokens_out":9734,"duration_ms":120597,"significance":"If the empirical gains survive a properly held-out evaluation, PCD is an attractive training-free decoding method with a distinctive positional-encoding interpretation. The paper is commendably concrete about its limitations and reports an ablation over the tested hyperparameter ranges. However, the theoretical derivation that is meant to explain the gain is invalid as written, and the headline result is currently tied to hyperparameters selected on the same InfiniteBench benchmark that is then used for evaluation, with no error bars and no released code. Since both the mechanism and the empirical headline are load-bearing, the contribution is significant but only conditionally supported.","major_comments":[{"comment":"The lower bound |S_CD(k)| ≥ λ Σ_{j≤j0} A_j k δ_j |sin(kθ_j)| − O(k^2) grows with k instead of decaying, so it does not establish the claimed asymptotic improvement in Eq. (17). Moreover, the Taylor expansion in Eq. (19) requires kδ_j to be small, which is not guaranteed for low-frequency components when B'/B = 1e-4 and k is large. The paper's only theoretical justification for PCD therefore needs either a corrected proof or explicit removal of the theorem.","section":"Appendix A.1, Theorem A.5 and Eq. (20)"},{"comment":"The hyperparameters α, β, B'/B, and γ are tuned on InfiniteBench, and the same InfiniteBench KV-retrieval accuracy is reported in Table 2 as a central result. Because Table 1 itself reports variances up to 3.1 accuracy points across three runs, the 7.0-point improvement at 8k is comparable to the tuning noise. Please evaluate with hyperparameters fixed on a held-out validation set, or report robustness over a grid, and provide standard errors for the main tables.","section":"Section 3.3/Table 1 and Table 2"},{"comment":"No error bars, no number of independent runs, and no statistical significance tests are reported for the main comparisons, and the code and evaluation scripts are not released. Without these, an independent check of the claimed state-of-the-art result is impossible, and the small LongBench average gain of 0.89 points in Table 3 cannot be distinguished from noise.","section":"Tables 2 and 3"},{"comment":"The simulations used to support the decay claim are run with settings that do not match the tuned configuration, for example α=0.4 and β=0.6 in the main simulation versus α=0.2 and β=2.5 in Table 1, and a different B' relation (Section 3.4 uses B=10^6 and B'=10^4, while Table 1 recommends B'/B=1e-4). Please clarify whether these curves are illustrative or quantitative evidence, and if the latter, reconcile the parameter settings.","section":"Section 3.4 and Figure 4"}],"minor_comments":[{"comment":"The reference list contains a malformed entry that begins 'TUNING ON SYNTHETIC DATA. From artificial needles to real haystacks: ...' with no authors and no year; it should be completed or removed.","section":"References"},{"comment":"The reported PCD average of 26.87 does not match the arithmetic mean (26.73) of the seven listed task scores; please check the aggregation.","section":"Table 3"},{"comment":"The text states that PCD 'consistently enhances' model performance, but Table 2 shows a decrease on Llama-3-8B-8k at 8k KV retrieval (92.4→91.0) and on the 1048k variant for RULER Variable Tracking at 4k and 8k; please qualify the claim.","section":"Section 3.2 and Table 2"},{"comment":"The notation for B' is inconsistent: Section 2.2 and Table 1 express the perturbation as B'/B with a recommended value of 1e-4, while Section 3.4 states B=10^6 and B'=10^4, a ratio of 1e-2, and Figure 4 uses B'=100000 with B=283461213. Please specify absolute values and ratios consistently.","section":"Section 2.2, Table 1, Section 3.4"},{"comment":"The citation '(DATA; An et al., 2024)' uses an unresolved placeholder DATA; either add the reference or replace the citation.","section":"Appendix A.5"},{"comment":"The word 'substrated' should be 'subtracted'.","section":"Section 3.4"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the scope of the journal, and the proposed method is simple enough to be practically useful if the empirical result is real. I would not accept before the test-set tuning issue and the invalid proof are addressed. I also note that the malformed reference entry and the unresolved citation key suggest the manuscript was not carefully prepared; these should be cleaned before resubmission."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"PCD is a plausible training-free decoding heuristic with some real empirical support, but the paper's theoretical appendix is broken and the headline number is weakened by tuning on the same benchmark. I'd take the empirical claim as promising rather than proven.\n\nWhat's actually new: the PSA metric is a clean way to quantify how gold-token rank degrades with context length, and the observation that gold tokens stay in the top ranks despite being under-selected is interesting. The PCD recipe—over-rotate low-frequency RoPE positions and contrast those logits against the standard ones—is a genuine combination not present in DoLa or MsPoE. The paper also honestly lists limitations and doesn't oversell short-text gains.\n\nThe soft spots are real. The spectral analysis in Appendix A.1 is invalid: Eq. 20 gives a lower bound that grows with k, so it cannot prove slower decay. The simulations in Fig. 4 are illustrative, not a proof. More importantly, the hyperparameters are tuned on InfiniteBench (Table 1) and then InfiniteBench KV retrieval is reported as the headline success (Table 2). With variance up to 3.1 points and no error bars, the 7-point gain at 8k is not clearly outside noise. No code or evaluation scripts are provided. That said, the method also improves RULER variable tracking, which wasn't used for tuning, and the LongBench gains are small but not negative overall. So the heuristic isn't obviously a mirage.\n\nWho should read it: people working on inference-time interventions for long-context LLMs. It deserves a serious referee because the core idea is testable and the empirical pattern is worth checking, but I would not accept it as-is. The authors should fix or drop the proof, report error bars, and evaluate with hyperparameters chosen on a held-out set. I'd recommend peer review with major revision.","headline":"Promising training-free decoding heuristic held back by a broken theoretical proof and benchmark-tuned hyperparameters.","tokens_in":11884,"tokens_out":2232,"would_cite":false,"duration_ms":25280,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Positional contrastive decoding recovers gold tokens that long-context models rank high but fail to select.","keywords":["long-context LLMs","posterior salience attenuation","positional contrastive decoding","contrastive decoding","RoPE","attention decay","key-value retrieval","long-context benchmarks"],"falsifier":"Compute the contrastive attention score $S_{CD}(k)=(1+\\lambda)S(k)-\\lambda S'(k)$ numerically for $k$ from 1 to 16384 with $B=10^6$, $B'=10^4$, and $d=512$, and check whether its log-log decay slope is shallower than the standard score's by roughly $\\left(\\ln B'/\\ln B\\right)^{2/d}$; if the contrastive score decays at the same rate or faster, the theorem's mechanism is falsified. A second check: apply PCD to a long-context model whose positional encoding is RoPE but whose attention head dimension differs, and see whether the predicted exponent shift matches the measured accuracy gain.","tokens_in":10890,"feed_emoji":"🎯","tokens_out":11796,"duration_ms":123867,"temperature":0.7,"pith_summary":"This paper claims that long-context LLMs degrade partly because the salience of the correct next token in the decoding distribution attenuates as the context lengthens, even though that token usually stays near the top of the ranking. To counter this, it proposes Positional Contrastive Decoding (PCD), a training-free decoding step that subtracts logits produced by an over-rotated variant of the rotary position embedding (RoPE) from the standard logits and scales the difference, $\\tilde{L} = (1+\\beta)L - \\beta L^*$. The paper argues that this contrast slows the decay of long-range attention scores, letting the model exploit the short-to-long training it already received. If correct, the method is a cheap inference-time fix for long-context QA and retrieval, with reported gains such as 8k KV retrieval on Llama-3-8B-262k rising from 72.0 to 79.0.","feed_headline":"Decoding trick lifts long-context retrieval from 72 to 79 percent","feed_subtitle":"Training-free PCD contrasts over-rotated RoPE logits with standard ones, slowing attention decay.","key_machinery":"The machinery is the contrastive logit step applied to perturbed rotary position embeddings. PCD constructs a \"local-aware\" attention by interpolating between the standard RoPE base $B$ and a lower base $B'$ through a transition function $T(x)=2-\\exp(\\alpha x)$, so low-frequency blocks get extra rotation while high-frequency blocks stay close to their original angles. It feeds those over-rotated query/key matrices through the same model to obtain perturbed logits $L^*$, then returns $\\tilde{L}=(1+\\beta)L-\\beta L^*$ restricted to the top $\\gamma$ tokens by probability. The load-bearing identity is the spectral representation of the attention score as a sum of cosines, $S(k)=\\sum_j A_j \\cos(k\\theta_j+\\phi_j)$, together with the claim that the contrastive combination saturates the long-distance decay to a slower exponent; this is what is supposed to turn the short-to-long training signal into better long-range attention.","core_discovery":"The central discovery is a decoding-space phenomenon the paper calls Posterior Salience Attenuation (PSA): as input length grows, the gold token's reciprocal rank in the model's output distribution falls on tasks held at constant difficulty, yet the gold token remains inside the top handful of candidates even with a 128k vocabulary. Because the correct answer is still highly ranked, a decoding operation that amplifies salience can recover it. PCD does this by building \"local-aware\" logits from a RoPE variant whose low-frequency rotations are over-rotated (base lowered from $B$ to $B'$ with a smooth transition $T(x)=2-\\exp(\\alpha x)$), then forming $\\tilde{L}=(1+\\beta)L-\\beta L^*$ and applying it to the top $\\gamma$ tokens. The paper's spectral analysis claims the contrastive score's asymptotic decay exponent is at most $\\left(\\ln B'/\\ln B\\right)^{2/d}$ times that of the standard score, so the long-range signal survives better under PCD; experiments on RULER, InfiniteBench, and LongBench report consistent gains.","pith_inferences":["Beyond the paper: if the mechanism is genuinely spectral, a similar contrast should work with other relative position encodings (e.g., ALiBi or Kerple) by perturbing their slope parameters and contrasting the two logits; the paper tests only RoPE.","Beyond the paper: the PSA observation suggests a cheap diagnostic for when decoding-side correction will help: on a fixed task, increase context length while watching the gold token's reciprocal rank; a steep drop with a retained high rank would flag PCD-style methods as candidates.","Beyond the paper: the ablation table shows that some choices of the top-$\\gamma$ window yield accuracy below the no-PCD base (71.5 versus 72.0 in the reported setting), so transferring PCD to new tasks will likely require re-tuning $\\gamma$ rather than treating the default as universal.","Beyond the paper: because PCD reweights existing logits, it should compose with training-based long-context methods rather than replace them; a natural next experiment is PCD on top of fine-tuned long-context models with different RoPE bases."],"forward_implications":["PCD raises InfiniteBench KV retrieval accuracy on Llama-3-8B-262k from 72.0 to 79.0 at 8k and from 52.0 to 55.0 at 16k.","On RULER variable tracking, PCD improves F1 on the 262k model from 74.02 to 81.80 at 4k and from 71.21 to 77.92 at 8k.","The spectral analysis implies the advantage of PCD grows with distance, since the contrastive score decays with a strictly smaller exponent than the standard score.","PCD is training-free and decoding-side, so it can be added to existing RoPE-based long-context models without fine-tuning or prompt rearrangement.","PCD mitigates but does not remove the context limit: it cannot extend the attention window itself."],"supporting_citations":[{"why":"Supplies the RoPE formulation and its long-term decay property, the substrate PCD perturbs.","marker":"Su et al., 2024"},{"why":"Defines RULER and its variable tracking task, one of the two main benchmarks where PCD is evaluated.","marker":"Hsieh et al., 2024a"},{"why":"Provides InfiniteBench, the KV retrieval benchmark, and the rephrasing baseline PCD is compared with.","marker":"Zhang et al., 2024a"},{"why":"Supplies LongBench, used for the real-world QA evaluation of PCD.","marker":"Bai et al., 2024"},{"why":"Defines the Llama-3 models and 128k vocabulary used in the PSA analysis and PCD experiments.","marker":"Dubey et al., 2024"}],"fun_headline_variants":["Training-free PCD slows attention decay in long contexts","Positional contrastive decoding fixes long-context salience loss","No training, just decoding: PCD rescues long-context accuracy","Positional contrastive decoding: a training-free fix for long contexts","Decode smarter: contrastive logits counter long-context drift"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The mathematical argument that PCD keeps long-range attention alive rests on a lower-bound estimate in the spectral derivation that stays positive and grows with distance rather than decaying; if that estimate is invalid, the proposed mechanism is unsupported even if the method still helps empirically.","fun_headline_variants_meta":{"raw":{"variants":["Training-free PCD slows attention decay in long contexts","Positional contrastive decoding fixes long-context salience loss","No training, just decoding: PCD rescues long-context accuracy","Positional contrastive decoding: a training-free fix for long contexts","Decode smarter: contrastive logits counter long-context drift"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000718,"raw_usage":{"total_tokens":3222,"prompt_tokens":940,"completion_tokens":2282,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":556,"completion_tokens_details":{"reasoning_tokens":2194}},"tokens_in":556,"tokens_out":2282,"duration_ms":20668,"temperature":1.0,"reasoning_tokens":2194,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T05:12:29.221920+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the contrastive attention score $S_{CD}(k)=(1+\\lambda)S(k)-\\lambda S'(k)$ numerically for $k$ from 1 to 16384 with $B=10^6$, $B'=10^4$, and $d=512$, and check whether its log-log decay slope is shallower than the standard score's by roughly $\\left(\\ln B'/\\ln B\\right)^{2/d}$; if the contrastive score decays at the same rate or faster, the theorem's mechanism is falsified. A second check: apply PCD to a long-context model whose positional encoding is RoPE but whose attention head dimension differs, and see whether the predicted exponent shift matches the measured accuracy gain.","supporting_citations":[],"review_version":1}