{"id":"f1798905-a25c-4cc6-bfab-7a287c7a3ad2","arxiv_id":"2507.19353","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A chunk-wise inference method called Smooth Reading lets recurrent LLMs match or exceed self-attention LLMs on LongBench and Needle-in-a-Haystack while keeping linear-time efficiency.","lead":"Recurrent language models use a small fixed memory, so they struggle with long documents. This paper trains them to read long texts in chunks, summarize each chunk, and carry the summary in memory, which lets a recurrent model match or beat a standard transformer on long-context benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The evidence for closing the recurrent/attention gap comes from the same two benchmarks used to build the Smooth Reading training set, and A.6 concedes the dataset is not generalizable; the broad gap-closing claim is therefore unsupported outside that distribution.","rationale":"The reader's CONDITIONAL verdict already identifies the fixed-size hidden-state assumption as the weakest point and flags the modified four-needle NIAH, per-task tuning, and missing error bars. I agree with those concerns. My stress-test adds a related but more direct evidentiary issue: the training data for Smooth Reading is derived from the same benchmarks on which the method is evaluated, and the paper's own A.6 limitation explicitly states that the dataset is not generalizable across a wide range of tasks. That does not make the comparisons internally unfair, since all models are trained on the same raw data, but it does mean the headline claim of closing the recurrent/attention gap on long-context tasks is only supported in-distribution. The hidden-state capacity worry is the mechanism by which out-of-distribution failure would occur, but the empirical crux is the train/evaluation overlap. Because this is a scoping problem rather than a demonstrated error, it does not move the verdict; it reinforces the existing CONDITIONAL. If the authors add a held-out benchmark evaluation or explicitly scope the claim, the concern would be substantially resolved.","tokens_in":16088,"tokens_out":10042,"duration_ms":109938,"concrete_test":"","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that Smooth Reading lets recurrent LLMs match or beat self-attention LLMs on long-context tasks rests almost entirely on Table 2 (LongBench) and Table 3 (NIAH). But the supervised fine-tuning dataset used to train all compared models is constructed from those same two benchmarks (Section 3.1.2, Table 5). The paper itself states in A.6 that the collected dataset is not generalizable across a wide range of tasks. This makes the empirical support for the headline in-distribution: the 3.61% LongBench advantage and the 99.8% NIAH result may reflect task-specific teacher supervision (contextual summaries, early-stop decisions, and benchmark-specific clue formats) rather than a general gap-closing inference mechanism. This is not an internal inconsistency, but it is load-bearing: if the claim is scoped to 'these two benchmark families,' the evidence is adequate; if it is stated as a general long-context capability, the current experiments do not establish it. A held-out evaluation is needed before the broad claim is taken at face value.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Smooth Reading, a chunk-wise inference procedure for recurrent LLMs that maintains hidden memory across chunks and generates a structured contextual summary at each step. The authors construct an SFT dataset from LongBench and NIAH raw data using DeepSeek-V3 or rule-based teachers, fine-tune Qwen2.5-derived sliding-window models and RWKV-7 in three formats (One-Step, Unsmooth Reading, Smooth Reading), and report accuracy on LongBench and NIAH plus training/inference time. The headline claims are that SWA-3B-4k-SR beats Qwen-2.5-3B-OS by 3.61% on LongBench, achieves 99.8% average accuracy on a four-needle NIAH task at 256k tokens, and is about 3x faster to train and 2x faster to infer at 64k context than the self-attention baseline. The paper also includes ablations on chunk and window sizes, comparisons with RAG-style baselines, and 7B-model experiments.","tokens_in":16287,"tokens_out":10680,"duration_ms":106152,"significance":"The idea is timely and potentially useful: showing that an inference protocol, rather than an architectural change, can narrow the recurrent/self-attention gap would be a meaningful contribution. The paper's strengths include the fair within-format comparison (all models are fine-tuned on the same raw data in different formats), a clear formulation of Smooth Reading, a useful back-of-envelope efficiency calculation, detailed appendices, a frank limitations section, and a stated plan to release code and data. The NIAH length-extrapolation results, if reproducible under a clearly disclosed protocol, are striking. However, the significance is currently limited by the in-distribution evaluation and by presentation issues around the modified NIAH task and the absence of statistical reliability information.","major_comments":[{"comment":"The supervised fine-tuning dataset is built from the same two benchmark families used for evaluation, and A.6 explicitly concedes that the collected dataset is \"not generalizable across a wide range of tasks.\" All compared models, including the self-attention baselines, are fine-tuned on this in-distribution data, so the reported LongBench gains (e.g., 50.99 vs 47.38 in Table 2) and the near-perfect NIAH results reflect the models' ability to reproduce the benchmark-specific teacher protocol (contextual summaries, <CONTINUE>/<STOP> decisions, and clue formats) rather than a demonstrated general closing of the recurrent/self-attention gap. This is load-bearing because the abstract and Section 4.2 make a general claim about long-context tasks. A held-out evaluation on a benchmark or task family not used in Table 5, or a claim explicitly scoped to the two benchmark families, is required before the headline can be accepted.","section":"Section 3.1.2, Table 5, and A.6"},{"comment":"The near-perfect NIAH results are obtained on a modified four-needle task, disclosed only in A.1.4 (“We use only one needle in our experiments by default, but four needles in Table 3”), while the main text and Table 3 present the results simply as NIAH. Because standard NIAH protocols use a single needle and results are sensitive to needle depth and position, the 256k extrapolation claim is not comparable to published NIAH numbers. Please report one-needle and four-needle results separately with needle placement details, or clearly label Table 3 and all related text as a four-needle variant.","section":"Section 4.3 and Table 3, with A.1.4"},{"comment":"All benchmark tables report single runs without error bars or significance testing, and A.1.4 states that chunk sizes are set differently per model and task (1024 vs 2048 vs 512 vs 256). Given that Section 4.5 and Table 4 show accuracy ranging from 0.0% to 100.0% depending on the chunk/window combination, the reported advantages (e.g., LongBench 50.99 vs 48.37 vs 47.38; SWA-7B-SR within 0.74% of Qwen-7B-OS in Table 6) could be driven by hyperparameter or seed variance rather than by the method itself. Please provide multiple seeds or error bars and a sensitivity analysis for the chunk/window choice on at least one main benchmark, or temper the comparative claims accordingly.","section":"Tables 2, 3, and 6; A.1.4; Section 4.5"}],"minor_comments":[{"comment":"The model names are inconsistent: Table 3 uses \"RWKV-3B-OS\" and \"RWKV-3B-SR\" while the main text and Table 2 use \"RWKV-7-3B-OS\" and \"RWKV-7-3B-SR\"; please use one consistent naming convention throughout.","section":"Table 3 and Section 4.3"},{"comment":"The linearity conclusion depends on the assumption that the per-step generation length g is constant; the paper should state explicitly that the complexity is O(L + n·g), which is linear only if g is O(1), and should discuss tasks (e.g., summarization of very long documents) where g may grow with context length.","section":"Section 3.2"},{"comment":"The efficiency comparison should report the exact experimental setup, including GPU type, batch size, the number of generated tokens per step, and the early-stopping policy, so that the claimed 2x/3x speedups are reproducible.","section":"Section 4.4 and Figure 4"},{"comment":"The RAG/RAG+/CompACT baselines use Qwen-2.5-3B-Instruct and, for RAG, external Wikipedia retrieval, while Smooth Reading uses the fine-tuned SWA-4k model; this mismatch should be stated in the main comparison or the baselines should be re-run on the same base model and input format.","section":"A.5 and Table 8"},{"comment":"The token-count approximation ntoken ≈ Int(1.5 × n_words) should state the tokenizer and language for which this ratio holds, since errors in this estimate can cause chunks to exceed the intended maximum size.","section":"A.1.1"}],"recommendation":"major_revision","confidential_remarks":"To the editor: the stress-test concern lands. The evaluation is in-distribution by construction, and the main text overstates the scope relative to what A.6 concedes. The four-needle NIAH protocol is disclosed only in the appendix, so I do not read it as concealment, but it must be surfaced in the main text and the claims scoped accordingly. The method is plausible and the paper is suitable for major revision rather than rejection; adding a held-out evaluation or explicitly limiting the claims to the two benchmark families would address the central concern."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, here is my read. The new thing is not chunked summarization — that exists in Chain-of-Agents and CompACT — but applying persistent hidden-memory chunkwise inference to recurrent LLMs and supervising fine-tuning on teacher-generated reading traces. That combination is real and worth testing. The LongBench comparison is better controlled than most: all models are fine-tuned on the same raw data in different formats, and the 7B scaling and early-stopping ablation are useful. Credit where due: the efficiency analysis is standard but honest, the Table 4 ablation shows the chunk/window interaction clearly and supports the rule that chunk size should not exceed window size, and A.2 gives a sensible explanation for why SWA extrapolates better than RWKV-7.\n\nThe soft spots, in proportion. The biggest is one the authors half-admit in A.6: the training dataset is built from NIAH and LongBench, and the headline numbers come from those same two benchmarks, with the same task categories used for training. So the 3.61% LongBench advantage and the 99.8% NIAH result are in-distribution evidence. They support “this method works on these benchmark families,” not “recurrent LLMs match self-attention on long-context tasks” generally. That is a load-bearing hole in the abstract’s broad claim. Also: the NIAH headline uses four needles, disclosed only in the appendix; chunk and window sizes are tuned per task; no error bars or significance tests are reported; and the code and data are promised but not yet public. None of these are fatal to the core idea, but they should be addressed before the claim is taken at face value.\n\nThe stress-test note lands. I do not think it makes the paper incoherent; it just means the broad claim is unsupported outside the training distribution. The central mechanism — keeping hidden state while compressing each chunk — is plausible and could well transfer, but the paper does not show that yet. The citation pattern looks fine, with no suspicious self-citation.\n\nWho is this for? People working on recurrent LLM inference, long-context efficiency, or benchmark design. It deserves a serious referee: the method is clearly described, the ablation is informative, and the claim is important if it holds. I would send it to peer review, but with a requirement that the authors add held-out evaluation on tasks not in the SFT set, report error bars, and release code and data. My own verdict would be conditional, not accept.","headline":"Smooth Reading is a plausible, well-controlled chunkwise inference recipe for recurrent LLMs, but the headline gap-closing claim is only shown on the two benchmarks used to build the training data.","tokens_in":16813,"tokens_out":2079,"would_cite":true,"duration_ms":21432,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Smooth Reading closes the long-context gap between recurrent and self-attention LLMs, with a 3B recurrent model beating its 3B attention counterpart by 3.61% on LongBench.","keywords":["Smooth Reading","recurrent LLM","long-context inference","sliding-window attention","linear attention","length extrapolation","contextual summarization","needle-in-a-haystack"],"falsifier":"Run SWA-3B-4k-SR on a 256k-token task whose answer requires every occurrence of a rare pattern, such as all numbers greater than 1000 or every mention of a specific name spread across many chunks; if accuracy falls well below the 99% four-needle retrieval level, the fixed-size summary is discarding the needed detail. The paper's own ablation—accuracy collapses when chunk size exceeds window size—points to exactly where that limit lives.","tokens_in":15895,"feed_emoji":"📖","tokens_out":7254,"duration_ms":61185,"temperature":0.7,"pith_summary":"Recurrent LLMs were thought to lag behind self-attention LLMs on long-context tasks because their memory is small and fixed. This paper argues the real problem is the inference method: feeding the whole context in one forward pass overwhelms that memory. It proposes Smooth Reading, which splits the context into chunks, processes them one by one while keeping the recurrent hidden state alive, and after each chunk writes a short contextual summary (target, clues, reason, continue/stop) that the hidden memory then carries forward. The paper reports that a 3B sliding-window model with a 4k window, trained on a 48,856-sample curated dataset, surpasses a 3B self-attention model on LongBench by 3.61% and reaches 99.6% on a four-needle retrieval test at 256k tokens, while keeping linear-time training and inference. If correct, this gives recurrent architectures a practical path to long contexts without quadratically scaling attention.","feed_headline":"Recurrent LLMs beat self-attention on long context","feed_subtitle":"A 4k-window model hits 99.6% on a 256k needle test while training 3x faster.","key_machinery":"The load-bearing object is Smooth Reading itself, formalized as $I_i, M_i = f_R(C_i, M_{i-1})$: the recurrent model processes chunk $C_i$ with the previous hidden memory $M_{i-1}$, producing an updated hidden memory and a contextual summary. The summary's four fields—Target, Clues, Reason, Continue—tell the model what to attend to, what to remember, why, and whether to keep reading via a special <CONTINUE> or <STOP> token. Because the hidden memory carries the summary forward, the model never re-reads accumulated text, so cost stays linear: the paper derives $T = (1 + g\\beta/c) \\times l \\times p_r$ for total inference time. The ablations show the method's operating constraint: chunk size must be smaller than window size, with a 1:2 chunk-to-window ratio giving the best accuracy-efficiency trade-off.","core_discovery":"The central claim is that a recurrent LLM equipped with Smooth Reading can match or beat a self-attention LLM on long-context benchmarks while retaining linear complexity. The method works chunk-wise: each chunk is read through the recurrent model, the updated hidden memory $M_i$ replaces the need to re-feed any external summary, and the model emits a structured contextual summary with four components—Target, Clues, Reason, and a <CONTINUE>/<STOP> decision—so the model knows what to keep and when to stop. The paper's headline result is SWA-3B-4k-SR improving from 5.68% below to 3.61% above Qwen-2.5-3B-OS on LongBench (50.99 vs 47.38), and reaching 99.93% average on NIAH up to 32k and 99.80% average from 64k to 256k with four needles, despite training on only 32k contexts. It also reports a 3x training speedup and a 2x inference speedup at 64k context compared with the self-attention baseline, and, with early stopping, a 4x inference speedup. The authors state this is the first work to bring recurrent LLMs to parity with self-attention LLMs on long-context tasks.","pith_inferences":["A natural extension the paper does not test: tasks where the answer requires exact aggregation over many chunks (e.g., 'list every transaction above $100' across a 256k document) would stress whether the fixed-size hidden summary retains precise details, not just the presence of one needle.","Since the training data is generated by a self-attention teacher performing Unsmooth Reading, the student's ceiling is the teacher's summary quality; training the recurrent model on its own rollouts or with reinforcement learning on the <CONTINUE>/<STOP> decision could push beyond the teacher and reduce the summary overhead.","The chunk-size < window-size constraint suggests the effective context is governed by the quality of the iterative summary rather than the physical window; this recasts long-context understanding as a recursive compression problem, connecting to state-space and memory-augmented models.","If early stopping is reliable, Smooth Reading could yield adaptive compute that scales with query difficulty, which has implications for serving cost."],"forward_implications":["If the claims hold, any recurrent LLM—sliding-window attention, RWKV, or linear-attention variants—can be turned into a competitive long-context model by changing only the inference loop and fine-tuning on summary-style data, not the architecture.","The 4k-window SWA-3B-4k-SR model's 99.6% accuracy on a four-needle NIAH task at 256k tokens, trained at 32k, implies recurrent models can extrapolate far beyond their training length when information is funneled through summaries.","Because Smooth Reading retains hidden memory, it beats Unsmooth Reading (re-feeding compressed summaries with reset memory) by 2.62% on LongBench and is far more stable on NIAH, suggesting the hidden state is a more faithful carrier than explicit re-input.","The method's linear complexity and early stopping make it attractive for deployment: at 64k context, training is 3x faster and inference 2x (4x with early stopping) than a self-attention baseline.","The method is orthogonal to memory-expanding architectures, so combining Smooth Reading with bigger or more expressive recurrent states should stack further gains."],"supporting_citations":[{"why":"Supplies the Needle-in-a-Haystack benchmark and the four-needle protocol used for the headline long-context retrieval numbers.","marker":"[18]"},{"why":"LongBench provides the benchmark and the raw task datasets used for both evaluation and training-data construction.","marker":"[3]"},{"why":"RWKV-7 is the representative recurrent LLM whose linear-attention state serves as one of the two architectures tested with Smooth Reading.","marker":"[29]"},{"why":"Longformer's sliding-window attention is the basis for the SWA recurrent models that achieve the best Smooth Reading results.","marker":"[4]"},{"why":"Qwen2.5 is the self-attention baseline and the pretrained backbone for the sliding-window models, giving the comparison point the paper must beat.","marker":"[32]"},{"why":"Defines the multi-step chunk-and-summarize inference family that Smooth Reading is contrasted against.","marker":"[30]"},{"why":"Represents the Unsmooth Reading style of iterative context processing with reset memory, which Smooth Reading improves on by keeping hidden memory.","marker":"[44]"},{"why":"CompACT is an explicit compression-based baseline whose information loss Smooth Reading is shown to avoid.","marker":"[42]"},{"why":"DeepSeek-V3 is the teacher model that generates the Smooth Reading training data for complex tasks.","marker":"[12]"},{"why":"Documents the empirical gap between recurrent and self-attention LLMs on long-context tasks, which this paper sets out to close.","marker":"[37]"}],"fun_headline_variants":["Smooth Reading gives recurrent LLMs the long-context edge","Chunk-wise reading lets recurrent LLMs beat attention on long texts","Recurrent LLM with Smooth Reading tops self-attention on LongBench","First recurrent LLM to match self-attention on long-context tasks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a fixed-size hidden state can compress an ever-growing context into a faithful summary without losing the exact details an answer needs; the paper tests this only on NIAH and LongBench, not on tasks where the answer depends on precise details scattered across many chunks.","fun_headline_variants_meta":{"raw":{"variants":["Smooth Reading gives recurrent LLMs the long-context edge","Chunk-wise reading lets recurrent LLMs beat attention on long texts","Recurrent LLM with Smooth Reading tops self-attention on LongBench","First recurrent LLM to match self-attention on long-context tasks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000676,"raw_usage":{"total_tokens":3189,"prompt_tokens":1172,"completion_tokens":2017,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":788,"completion_tokens_details":{"reasoning_tokens":1941}},"tokens_in":788,"tokens_out":2017,"duration_ms":11710,"temperature":1.0,"reasoning_tokens":1941,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T17:54:04.973917+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run SWA-3B-4k-SR on a 256k-token task whose answer requires every occurrence of a rare pattern, such as all numbers greater than 1000 or every mention of a specific name spread across many chunks; if accuracy falls well below the 99% four-needle retrieval level, the fixed-size summary is discarding the needed detail. The paper's own ablation—accuracy collapses when chunk size exceeds window size—points to exactly where that limit lives.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"DeepSeek-V3 is the teacher model that generates the Smooth Reading training data for complex tasks."}],"review_version":2}