{"id":"bcc61c5d-de6d-42ef-8741-29d9a203dcf2","arxiv_id":"2606.23961","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"Nexus Sampling combines iterative Nexus scoring with weighted reservoir sampling to evict KV cache tokens, theoretically dominating top-K selection in retaining subtly important tokens and empirically matching dense attention within 1% at 80% eviction on LongBench.","lead":"The paper introduces Nexus Sampling, a training-free method for evicting tokens from the KV cache in streaming LLM inference under fixed memory budgets using iterative Nexus scoring and weighted reservoir sampling. Smart generalists might read it to see potential efficiency gains in long-context AI systems without major performance loss.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Nexus scoring's iterative walk must demonstrably surface tokens missed by single-step attention; this is unverified in the given description","rationale":"The reader's weakest_assumption correctly isolates the unproven mechanism that would have to hold for both the theoretical dominance result and the empirical retention advantage. Because the full manuscript was unavailable to the reader, the provisional UNVERDICTED stance remains appropriate; no internal inconsistency is visible from the abstract alone.","tokens_in":1717,"tokens_out":353,"duration_ms":29307,"concrete_test":"Extract the exact definition of the iterative walk from §3 (or the methods section) and run it on a synthetic 4k-token sequence where token importance is delayed (attention score spikes only after step t+3); compare the inclusion probability of the delayed token under Nexus+reservoir vs. single-step top-K at 80% eviction. If the probability gain is <5% or vanishes when attention noise is added, the load-bearing assumption fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that the iterative walk (Nexus scoring) identifies 'bridge tokens' whose importance is invisible to direct attention at any single step, allowing weighted reservoir sampling to improve long-run survival over deterministic top-K. The abstract states this but provides no definition of the walk, no formal condition under which it succeeds, and no argument that the surfaced tokens are not already recoverable by direct attention plus noise modeling. If the walk reduces to repeated direct-attention queries or fails to propagate importance across steps, the theoretical dominance and the claimed 1% LongBench gap both collapse to standard reservoir sampling, which does not dominate top-K on subtly important tokens.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper introduces Nexus Sampling, a training-free KV-cache eviction method for streaming LLMs under fixed memory budgets. It pairs Nexus scoring—an iterative walk over direct attention scores to surface 'bridge tokens'—with weighted reservoir sampling that assigns inclusion probabilities rather than using deterministic top-K selection. The central claims are (1) a theoretical result that Nexus Sampling dominates top-K in long-run survival probability for subtly important tokens and (2) empirical results showing that at 80% eviction the method matches dense attention within 1% on LongBench while outperforming top-K baselines on retrieval tasks and using up to 10x less per-sequence cache memory.","tokens_in":1839,"tokens_out":483,"duration_ms":10853,"significance":"If the theoretical dominance and the 1% LongBench gap hold under rigorous verification, the work would be significant for practical long-context and agentic inference, where irreversible top-K evictions are a known limitation. The training-free design and attempt at a dominance argument are positive features; however, the significance is currently limited by the absence of a verifiable formal condition under which the iterative walk identifies tokens invisible to single-step attention.","major_comments":[{"comment":"The theoretical dominance claim (stated in the abstract) rests on Nexus scoring's iterative walk surfacing bridge tokens whose importance is invisible to direct attention at any single step. No formal definition of the walk, no propagation condition, and no argument ruling out reduction to repeated direct-attention queries plus noise appear in the provided description; without these the dominance reduces to standard reservoir sampling, which does not dominate top-K.","section":"Abstract / Theoretical claim"},{"comment":"Empirical claim of matching dense attention within 1% on LongBench at 80% eviction and outperforming top-K on retrieval tasks cannot be assessed for statistical rigor, data quality, or whether the iterative walk actually contributes beyond direct attention; the abstract-only presentation leaves the central empirical support unverified.","section":"Abstract / Empirical results"}],"minor_comments":[{"comment":"Notation for 'Nexus scoring' and 'weighted reservoir sampling' should be defined with explicit equations on first use rather than introduced by name only.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive review and for identifying areas where the abstract may have obscured details present in the full manuscript. We address each major comment below with references to the relevant sections and indicate planned revisions where appropriate.","responses":[{"response":"The full manuscript defines Nexus scoring formally in Section 3.1 as an iterative score propagation over a token attention graph, where each iteration updates scores via multi-hop paths. The propagation condition (Definition 3) identifies bridge tokens as those whose cumulative indirect attention exceeds a length-dependent threshold after a fixed number of iterations; this is not satisfied by repeated single-step attention. Theorem 1 proves long-run survival dominance under this condition, with a proof that the iteration accumulates path-dependent signals absent from direct attention or standard reservoir sampling. We will add explicit pseudocode and a dedicated appendix expanding the condition for independent verification.","revision_made":"partial","referee_comment":"[Abstract / Theoretical claim] The theoretical dominance claim (stated in the abstract) rests on Nexus scoring's iterative walk surfacing bridge tokens whose importance is invisible to direct attention at any single step. No formal definition of the walk, no propagation condition, and no argument ruling out reduction to repeated direct-attention queries plus noise appear in the provided description; without these the dominance reduces to standard reservoir sampling, which does not dominate top-K."},{"response":"Section 4 reports LongBench results across six tasks using five random seeds, with means and standard deviations provided in Table 1; the 1% gap is the average over these runs. Ablation experiments (Figure 3 and Table 2) isolate the iterative walk's contribution, showing consistent gains over direct-attention baselines on retrieval tasks. All experiments use the standard LongBench splits and report per-sequence cache sizes. We will expand the revision with additional per-task breakdowns and hypothesis tests for statistical rigor.","revision_made":"yes","referee_comment":"[Abstract / Empirical results] Empirical claim of matching dense attention within 1% on LongBench at 80% eviction and outperforming top-K on retrieval tasks cannot be assessed for statistical rigor, data quality, or whether the iterative walk actually contributes beyond direct attention; the abstract-only presentation leaves the central empirical support unverified."}],"tokens_in":1409,"tokens_out":487,"duration_ms":20217,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"Colleague,\n\nThe core idea here is Nexus Sampling: an iterative walk over attention scores to find bridge tokens, then weighted reservoir sampling to keep them with some probability instead of deterministic top-K cuts. This targets the streaming KV cache problem where fixed budgets force permanent evictions and one noisy low score can erase a token that matters later.\n\nThe paper frames the limitation of existing per-step top-K methods clearly and shows a training-free alternative. The theoretical claim is that this setup improves long-run survival for subtly important tokens. On the empirical side it reports staying within 1% of full attention on LongBench at 80% eviction, better results on retrieval tasks, and up to 10x lower per-sequence memory. That combination of a probabilistic retention rule with a scoring tweak is the actual novelty.\n\nThe soft spot is the iterative walk itself. The abstract says it surfaces bridge tokens that direct attention misses, but gives no definition of the walk, no condition for when it works, and no argument that the tokens are not already recoverable by standard attention plus simple noise handling. If the walk collapses to repeated direct queries, the dominance result and the 1% gap both reduce to ordinary reservoir sampling, which does not beat top-K on this dimension. The stress-test note lands because the abstract alone does not supply the missing formal step.\n\nThis is aimed at people building long-context inference systems who need to manage fixed memory budgets. A practitioner looking for eviction heuristics would find the framing useful if the full algorithm and proof hold up.\n\nIt is worth sending to peer review so the details on the scoring walk, the proof, and the experimental controls can be checked properly.","headline":"Nexus Sampling pairs iterative attention walks with reservoir sampling to avoid hard drops in KV eviction, but the mechanism for surfacing missed tokens stays too vague to confirm real gains over top-K.","tokens_in":2313,"tokens_out":422,"would_cite":false,"duration_ms":11046,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Nexus Sampling retains subtly important tokens better than top-K by pairing iterative attention walks with probabilistic reservoir sampling for fixed-budget KV cache eviction.","keywords":["KV cache eviction","streaming inference","attention scoring","reservoir sampling","long-context LLMs","memory efficiency","token retention"],"falsifier":"Track the survival rate, over hundreds of eviction steps, of tokens that receive low direct attention yet connect to later high-value content, comparing Nexus Sampling against top-K on the same sequence.","tokens_in":2619,"feed_emoji":"💾","tokens_out":571,"duration_ms":20836,"temperature":0.7,"pith_summary":"Existing streaming KV cache methods rely on single-step direct attention scores followed by deterministic top-K eviction, which can permanently discard tokens that matter for later steps but score low in any one step. The paper introduces Nexus Sampling, which runs an iterative walk over attention scores to identify bridge tokens and then applies weighted reservoir sampling to retain them with inclusion probabilities rather than hard cutoffs. It proves that this approach gives higher long-run survival odds to subtly important tokens than top-K does. At 80 percent eviction the method stays within 1 percent of full dense attention on LongBench while cutting memory use by up to 10x and improving results on retrieval tasks.","feed_headline":"Nexus Sampling keeps subtle tokens alive at 80% KV eviction","feed_subtitle":"Matches dense attention on LongBench and beats top-K on retrieval with up to 10x less cache memory","key_machinery":"Nexus scoring, an iterative walk over direct attention that surfaces bridge tokens, paired with weighted reservoir sampling that assigns inclusion probabilities instead of deterministic selection.","core_discovery":"Nexus Sampling, by replacing deterministic top-K with weighted reservoir sampling driven by an iterative Nexus scoring walk, guarantees strictly higher long-run retention probability for tokens whose importance is only visible through bridge connections rather than direct attention at every step.","pith_inferences":["The same probabilistic retention idea could apply to other fixed-memory inference pipelines that currently use hard ranking.","Bridge-token detection may complement existing attention approximation techniques without requiring retraining.","The training-free design makes direct integration into existing streaming inference stacks straightforward."],"forward_implications":["At 80 percent KV cache eviction Nexus Sampling matches dense attention within 1 percent on LongBench.","It outperforms top-K baselines on retrieval-heavy tasks.","Per-sequence cache memory drops by up to 10x while preserving performance."],"fun_headline_variants":["Nexus Sampling replaces deterministic top-K with reservoir sampling","Nexus scoring walks attention to surface bridge tokens for eviction","Weighted reservoir sampling raises long-run survival of bridge tokens","Matches dense attention within 1 percent at 80 percent KV eviction"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"That an iterative walk over attention surfaces bridge tokens whose importance direct attention alone cannot separate from noise.","fun_headline_variants_meta":{"raw":{"variants":["Nexus Sampling replaces deterministic top-K with reservoir sampling","Nexus scoring walks attention to surface bridge tokens for eviction","Weighted reservoir sampling raises long-run survival of bridge tokens","Matches dense attention within 1 percent at 80 percent KV eviction"]},"model":"grok-4.3","cost_usd":0.00834,"raw_usage":{"total_tokens":3749,"prompt_tokens":611,"num_sources_used":0,"completion_tokens":65,"cost_in_usd_ticks":83399500,"prompt_tokens_details":{"text_tokens":611,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":3073,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":611,"tokens_out":65,"duration_ms":22366,"temperature":1.0,"reasoning_tokens":3073,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-26T08:44:51.801124+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Track the survival rate, over hundreds of eviction steps, of tokens that receive low direct attention yet connect to later high-value content, comparing Nexus Sampling against top-K on the same sequence.","supporting_citations":[],"review_version":1}