{"id":"795b80f6-c08b-4578-ab63-d5d3c23ab281","arxiv_id":"2607.15178","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Routing recurrent state between middle layers of adjacent decoding steps improves Transformer reasoning more than full-layer recurrence, at under 8% inference overhead.","lead":"T2MLR adds a small recurrent connection inside a Transformer: a middle-layer state from the previous token is injected into an earlier layer of the current token, with almost no extra inference cost. Across pretraining, reasoning benchmarks, and a retrofitted 1.7B model, it beats matched Transformers, though training becomes 2–4x slower.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reasoning gains are not tested against a training-compute-matched baseline; Table 12 shows such a baseline beats T2MLR on zero-shot NLP, so the reasoning advantage may be a training-compute artifact rather than middle-layer recurrence.","rationale":"The reader's verdict is CONDITIONAL, and I agree with that level. My concern differs in emphasis from the reader's weakest_assumption: rather than the Jacobi approximation (which the paper partially addresses in B.4 at 1B tokens) or single-run variance, the most direct threat is the missing training-compute control for the reasoning claims. T2MLR's main results are obtained at ~2.24x training cost, and the paper itself shows that a standard Transformer given the same cost wins on zero-shot NLP. Since no equivalent control is reported for the reasoning suite or the 1.7B retrofit, the central 'reasoning better' claim is not yet isolated from additional training compute. This is not an accusation of unfair comparison—the paper is transparent—but it is a load-bearing gap because the abstract and Section 4.3 present the gains as architectural. The proposed test is cheap: reuse the Table 12 checkpoint and finetune it. If the baseline wins, the mechanism claim fails; if T2MLR wins, the concern is retired. I also credit B.4 as decent evidence against the Jacobi concern, and the absence of seeds is real but secondary.","tokens_in":27547,"tokens_out":9545,"duration_ms":85144,"concrete_test":"Take the 2.24-epoch 135M Transformer checkpoint from Table 12 (or retrain it), finetune it with the same recipe used for Figure 6 on GSM-Aug Symbolic/NL, ProsQA-Hard, Variable Assignment depth=5, and HotpotQA-Easy, matching the total finetuning wall-clock budget including T2MLR's Jacobi overhead. Compare against T2MLR(13,18) under identical evaluation. Run 3 seeds. If the longer-trained baseline matches or exceeds T2MLR on these reasoning tasks, the reasoning gains are a training-compute artifact; if T2MLR retains a clear lead, the architectural claim survives.","verdict_should_be":"UNCHANGED","load_bearing_attack":"All main pretraining and finetuning comparisons (Tables 1, 2, 9, 10, Figure 6) match parameter count and per-epoch data, but not training wall-clock. T2MLR's Jacobi refinement makes training roughly 2.24x more expensive for T2MLR(13,18) (Section B.2, Table 12). When the Transformer baseline is given the same 2.24x budget by training for 2.24 epochs, it surpasses T2MLR on the seven zero-shot NLP tasks (45.30 vs 44.14 average). The paper does not run this training-compute-matched baseline on the reasoning suite in Figure 6 or the 1.7B retrofit in Table 3. Therefore the central claim—that middle-layer recurrence makes Transformers reason better—is not yet isolated from the simpler explanation that T2MLR simply spends more training compute. The paper is transparent about this trade-off in Section 6, but the abstract's blanket 'consistently outperforms data- and parameter-matched baselines' remains unsupported for the reasoning tasks without this control. Single-run results (Section 6) compound the uncertainty.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces T2MLR, a Transformer variant that maintains a fixed-size recurrent cache computed from the representation after a middle layer ℓ_end at token t and fuses it into the representation before an earlier middle layer ℓ_start at token t+1 via a gated fusion module. Training uses a temporal-parallel Jacobi approximation (Algorithm 1, d_forward=16, d_backward=4) to avoid sequential recurrence, with inference remaining a standard single forward pass plus a small fusion overhead. The authors evaluate T2MLR on a purpose-built S5-Retrieval task, on 135M–1B parameter pretraining with up to 50B FineWeb-Edu tokens, on finetuned reasoning tasks (Variable Assignment, ProsQA-Hard, HotpotQA-Simple, GSM-Aug), and on retrofitting a SmolLM2-1.7B-Instruct model with continued finetuning. Headline claims are that T2MLR consistently outperforms data- and parameter-matched Transformers, that middle-layer recurrence often outperforms full-layer recurrence, that the Jacobi approximation closely matches exact recurrent training, and that the architecture adds at most ~8% per-token inference overhead.","tokens_in":27873,"tokens_out":6165,"duration_ms":56133,"significance":"If the central claim holds, the paper makes a meaningful contribution: it identifies middle-layer recurrence as a high-leverage design choice, shows that approximate temporal-parallel training can scale latent-reasoning architectures under dense teacher forcing, and demonstrates a practical retrofit path for existing pretrained models. Strengths of the manuscript include the parameter/data-matched comparisons in Tables 1, 2, 9, and 10; the direct comparison of Jacobi-trained versus exact-recurrent-trained models at three scales in Appendix B.4; the gradient-similarity analysis in B.3; the explicit inference-overhead measurements in B.5; and the unusually transparent statement of the training-compute tradeoff in Section 6. The main weakness is that the reasoning gains are not yet separated from the substantially larger training budget of T2MLR, and all headline results are single runs without variance estimates. These issues are fixable and do not invalidate the architectural idea, but they must be addressed before the central reasoning claim can be considered established.","major_comments":[{"comment":"The central claim that T2MLR improves reasoning is not yet isolated from training-compute differences. All reasoning-task comparisons (Variable Assignment, ProsQA-Hard, HotpotQA-Simple, GSM-Aug, and the 1.7B retrofit in Table 3) use a baseline trained for one epoch, while T2MLR's Jacobi refinement makes training roughly 2.24× more expensive for the (13,18) configuration. The only training-compute-matched control, Table 12, is limited to zero-shot NLP and shows that a Transformer trained for 2.24 epochs surpasses T2MLR (45.30 vs 44.14 average). No equivalent control is provided for the reasoning suite or for Figure 6. Similarly, the comparison across recurrence boundaries is not compute-matched: full recurrence D=30 costs more per step than D=6, so the finding that middle-layer recurrence outperforms full-layer recurrence could reflect an optimization-budget effect rather than recurrence","section":"Section 4.3 / Figure 6; Section C.2 / Table 12"},{"comment":"All headline results are single runs, with no error bars, confidence intervals, or seed variance. Many zero-shot differences are small in absolute terms (e.g., 42.83 vs 44.14 average in Table 1, individual task gaps of 1–3 points), and the reasoning-task gaps in Figure 6 are plotted without quantifying run-to-run variability. Section 6 defers multi-seed variance to future work, but given the small effect sizes, this is load-bearing for the 'consistently outperforms' claim. Reporting at least 3–5 seeds for the principal pretraining and reasoning comparisons, with mean and standard deviation, would materially strengthen the paper.","section":"Section 6; Tables 1, 2, 9, 10; Figure 6; Table 3"},{"comment":"The entire training pipeline rests on the assumption that d_forward=16 Jacobi iterations produce a cache close enough to exact sequential recurrence for all tested tasks and recurrence boundaries. Appendix B.4 validates this on validation loss/perplexity at three scales for ℓ_start=8 after 1B tokens, and B.3 reports gradient similarity for one checkpoint. However, no convergence guarantee is given, and the validation does not cover the downstream reasoning tasks or the full range of recurrence placements (D=2 through D=30) used in the main experiments. If Jacobi error interacts with task difficulty or recurrence location, the reported gains could depend on the approximation rather than on middle-layer recurrence per se. Please include an ablation of d_forward on at least one reasoning finetuning task and an exact-recurrent rollout evaluation for the main model variants, or otherwise boun","section":"Section 2.4; Appendix B.4; Appendix B.3"}],"minor_comments":[{"comment":"The abstract's 'consistently outperforms data- and parameter-matched Transformer baselines' is technically accurate but should be qualified as 'under fixed parameter/data/inference-compute budgets,' given the training-compute-matched result in Table 12. The paper already does this in Section 6, but the qualification should appear where the headline claim is made.","section":"Abstract / Section 6"},{"comment":"Equation (2.3) states that the scalar gates are initialized to zero and 'the gates randomly'; if γ_cur and γ_rec are initialized to zero, the tanh factors are zero and the gated terms are identically zero, so the phrase 'gates randomly' is misleading. Please clarify the exact initialization and what is random.","section":"Section 2.3"},{"comment":"The sentence 'T2MLR cannot directly adopt the standard token-parallel training procedure used by vanilla Transformers' is followed by a description that relies on 'ShiftRight' and cache indexing (R^⟨1⟩, R^⟨2⟩). The index alignment in Algorithm 1 between the initial cache seed and the first recurrent refinement is easy to misread; a short worked example or a note that R^⟨1⟩ corresponds to the cache for the first token position would help.","section":"Section 2.4"},{"comment":"The HotpotQA result is restricted to the easy subset and uses GPT-4o-mini-generated CoT data; the paper is transparent about this, but the main-text phrasing 'HotpotQA' could oversell the finding. Please state 'easy subset' in the main text whenever the result is discussed, and consider reporting a filtered accuracy on medium/hard for completeness.","section":"Section 4.3 / Appendix F.2"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The novel bit is real. I don't know of prior work that propagates a middle-layer hidden state across adjacent decoding steps into an earlier middle layer; looped transformers recur over depth within a single token, and Coconut/CODI/HRPO feed back through the embedding. The mechanism here is clean and the paper does most of the right controlled work: parameter- and data-matched pretraining and finetuning, scaling to 361M and 1B, a retrofit on a pretrained 1.7B model, and a Jacobi-style training approximation validated against exact sequential recurrence at three scales in B.4. The inference overhead claim is also measured, not asserted — under 8% per token, which matters for practical adoption.\n\nThe main soft spot is the training-compute confound. T2MLR's Jacobi refinement makes training roughly 2.24x more expensive, and when the Transformer baseline is given the same budget (2.24 epochs), it surpasses T2MLR on zero-shot NLP (Table 12). The paper is transparent about this and explicitly frames the contribution as architectural and inference-side, which is honest. But the same compute-matched control was never run on the reasoning suite in Figure 6 or on the 1.7B retrofit, so we don't actually know whether the reasoning gains come from middle-layer recurrence or just from more training compute. That caveat belongs in the abstract or at minimum a prominent place in the conclusions, not buried in Section 6.\n\nOther issues are minor and mostly addressable: all headline results are single-run, HotpotQA is restricted to the easy subset, and the paper defers multi-seed variance to future work. I would not fault the S5-Retrieval comparison for unequal training steps — T2MLR got 150k steps vs 400k for baselines, so the inequality runs against T2MLR. The recurrence-location ablation (Table 11) is a nice touch and supports the middle-layer claim.\n\nWho is this for? Anyone working on latent reasoning, recurrent Transformers, or inference-efficient reasoning architectures. The central mechanism deserves a serious referee; the paper is not there yet as a definitive claim because of the missing compute-matched reasoning control and the seed variance, but those are fixable in revision. I'd send it to a strong venue and ask for the reasoning-suite compute-matched baseline plus at least 2-3 seeds before accepting.","headline":"A genuinely new recurrent pathway for Transformers — fusing a cached middle-layer state into an earlier layer of the next token — with honest controlled experiments that are not yet fully training-compute-matched.","tokens_in":28331,"tokens_out":2083,"would_cite":true,"duration_ms":19462,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a Transformer with a gated recurrent pathway running through its middle layers outperforms matched Transformers on reasoning tasks, adding only a few percent of inference overhead.","keywords":["T2MLR","middle-layer recurrence","latent reasoning","Transformers","autoregressive decoding","state tracking","Jacobi approximation","temporal recurrence"],"falsifier":"Train a small T2MLR with exact sequential recurrent states (no Jacobi approximation) on S5-Retrieval and on one math task, and compare to the Jacobi-trained model and to the baseline Transformer. If exact-recurrence training does not reproduce or improve upon the reported gains over the baseline, the benefit depends on the approximation rather than on the temporal recurrent pathway. Alternatively, zero out the trained fusion gates (set γ_rec=0) at inference and measure accuracy: if accuracy does not fall toward the baseline, the recurrent pathway is not the cause of the improvement.","tokens_in":27429,"feed_emoji":"🧠","tokens_out":7459,"duration_ms":57062,"temperature":0.7,"pith_summary":"This paper argues that the token-level information bottleneck of autoregressive Transformers can be relaxed by carrying a cached representation from the middle layers of the previous token into an earlier middle layer of the current token. The proposed architecture, T2MLR, fuses this cached state through a lightweight gated module, allowing intermediate reasoning states to persist and evolve across decoding steps without looping over the whole network. In data- and parameter-matched comparisons, T2MLR outperforms plain Transformers on a synthetic state-tracking-plus-retrieval task, on pretraining perplexity and downstream NLP benchmarks, and on finetuned multi-hop and math reasoning tasks. The authors also show that applying recurrence to only a localized middle block (as little as 20% of the layers) often beats full-layer recurrence, and that the pathway can be grafted onto an existing pretrained 1.7B model, raising grade-school math accuracy from 35.8% to 39.9% and a harder math benchmark from 12.8% to 18.0% after brief finetuning. The practical stake is that effective latent reasoning may not require expensive full-depth loops; targeted middle-layer recurrence can deliver similar or better gains at near-standard decoding cost.","feed_headline":"Fusing cached middle layers yields stronger Transformer reasoning","feed_subtitle":"A gated recurrent pathway between tokens adds under 8% decode cost and fits existing pretrained models.","key_machinery":"The load-bearing mechanism is the gated fusion module Φ combined with the recurrent cache update. Φ takes the current token's pre-layer-ℓ_start representation h_t and the previous recurrent cache R_{t-1}, and computes a residual-style update with two input-dependent sigmoid gates and two scalar tanh gates (γ_cur, γ_rec), so the recurrent stream is added only when it contributes beyond the current-token stream. The cache is updated after layer ℓ_end as R_t = RMSNorm(h_t^{ℓ_end} + R_{t-1}), making the recurrence a temporal residual. A second, equally essential mechanism is the Jacobi fixed-point training scheme: for scalable teacher-forced training, the cache for all token positions is approxi","core_discovery":"The central discovery is that temporal recurrence placed specifically in the middle layers of a Transformer yields reasoning gains that full-layer recurrence or no recurrence do not. Concretely, the paper defines T2MLR by choosing two layer indices, ℓ_start and ℓ_end, maintaining a recurrent cache R_t that stores a normalized sum of the current token's layer-ℓ_end representation and the previous cache, and fusing R_{t-1} into the current token's representation just before layer ℓ_start via a gated sum. This establishes a deep-to-shallow shortcut: information computed late for token t−1 becomes an explicit input early for token t, without changing the autoregressive interface. On the S5-Retri","pith_inferences":["Because the recurrent cache is a fixed-size vector, it functions as a learned, task-dependent memory of the recent reasoning state; one could test whether combining T2MLR with long-context segment-level memory compounds the benefit.","The Jacobi approximation's success suggests that the exact recurrence may not be needed during training; a cheaper alternative like a learned predictor of the cache or a lower refinement depth could be explored, especially for RL settings where exact states are naturally produced during rollout.","The finding that middle-layer recurrence beats full-layer recurrence hints that placing recurrence at the depth where abstract computation is most active is a general design principle, which might transfer to other architectures such as mixture-of-experts or state-space models.","One could test the robustness of the claim by varying the task family: if middle-layer recurrence mainly helps tasks that require maintaining a continuous latent state (state tracking, multi-hop composition), it may not help purely lexical associative tasks; the paper's task set samples this space but not exhaustively."],"forward_implications":["If middle-layer recurrence is the right inductive bias, latent-reasoning architectures do not need to loop over all layers; routing recurrence through a localized block can give better or equal gains with far less compute.","Because the pathway is a simple gated residual added to an existing Transformer, it can be retrofitted into already-pretrained models with brief finetuning, so the benefit may be achievable without a costly full pretraining run.","At inference, T2MLR adds only a constant gated-fusion cost per token (measured at most ~8% overhead, decreasing with model size and generation length), making it attractive for deployment where looped or pause-token methods multiply per-token compute.","The gains persist and grow when scaling from 135M to 1B parameters and from 10B to 50B tokens, and a fixed-width ablation shows that middle-layer placement, not just recurrence depth, is what drives the improvement."],"fun_headline_variants":["Middle-layer recurrence sharpens Transformer reasoning","Targeted middle-layer recurrence beats full-layer for reasoning","Retrofit middle-layer recurrence into pretrained Transformer","Cached middle-layer state boosts reasoning with tiny overhead","Piggyback middle-layer recurrence onto pretrained models for math gains"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that 16 Jacobi fixed-point iterations during training produce a recurrent cache close enough to the exact sequential recurrence for every task, scale, and difficulty level tested; if that approximation degrades with task difficulty or interacts with model scale, the reported gains could be an artifact of the approximation rather than of middle-layer recurrence.","fun_headline_variants_meta":{"raw":{"variants":["Middle-layer recurrence sharpens Transformer reasoning","Targeted middle-layer recurrence beats full-layer for reasoning","Retrofit middle-layer recurrence into pretrained Transformer","Cached middle-layer state boosts reasoning with tiny overhead","Piggyback middle-layer recurrence onto pretrained models for math gains"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000416,"raw_usage":{"total_tokens":1981,"prompt_tokens":740,"completion_tokens":1241,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":484,"completion_tokens_details":{"reasoning_tokens":1178}},"tokens_in":484,"tokens_out":1241,"duration_ms":10498,"temperature":1.0,"reasoning_tokens":1178,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T23:54:45.447415+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a small T2MLR with exact sequential recurrent states (no Jacobi approximation) on S5-Retrieval and on one math task, and compare to the Jacobi-trained model and to the baseline Transformer. If exact-recurrence training does not reproduce or improve upon the reported gains over the baseline, the benefit depends on the approximation rather than on the temporal recurrent pathway. Alternatively, zero out the trained fusion gates (set γ_rec=0) at inference and measure accuracy: if accuracy does not fall toward the baseline, the recurrent pathway is not the cause of the improvement.","supporting_citations":[],"review_version":1}