{"id":"25b20ecc-2783-47cc-afa8-3b61cc68b069","arxiv_id":"2411.15785","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":2.0,"correctness_risk":"high","formal_verification":"none","parameter_count":1,"one_line_summary":"The Bounded-Cache Transformer uses a fixed-size key-value memory updated by outer-product writes, claiming constant memory and preserved quality without providing experimental data.","lead":"This paper proposes the Bounded-Cache Transformer, an LLM attention layer that writes incoming tokens into a fixed-size key-value memory instead of an ever-growing cache. The claimed payoff is constant memory use and steady throughput on long contexts, but the paper presents no numerical evidence.","discovery_kind":"incremental","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central failure is architectural: Section 3.2 and Algorithm 1 define only a write/update path for the cache MV and never define the readout that would produce token logits, so the claimed BCT language model cannot be reproduced.","rationale":"I agree with the reader's weakest-assumption analysis. The paper never defines how the updated key-value vector sequence MV is converted into an output token distribution. The entire architecture, as written, is write-only: Algorithm 1 computes wq_t, wv_t, scores against MK, and updates MV_t, but no operation reads MV_t to produce the layer output. Without this readout there is no language model, no training loss, and no generation process, so the experimental claims in Section 4.3 (constant memory, comparable BLEU) are not backed by a reproducible system. The reader's verdict of REJECT is well-supported, and my stress test does not change it. I note that the missing readout is the most load-bearing concern because it precedes all others: even if the experimental section were full of numbers, they would be meaningless unless the forward pass is defined. I also note secondary issues such as the absence of any quantitative data and the vague references to 'adaptive learning rate' and 'self-supervised learning during inference,' but these are less fundamental. The reader and I are fully aligned on the primary objection, so the verdict remains unchanged.","tokens_in":4170,"tokens_out":2608,"duration_ms":26853,"concrete_test":"Check the paper's supplementary code or project page for the full forward pass of a BCT layer. If code is unavailable, implement a minimal BCT layer by completing Algorithm 1 with an explicit readout (e.g., o_t = wq_t^T MV_t or o_t = softmax(wq_t^T MK^T) MV_t), then train it on a small language modeling corpus and measure BLEU/perplexity against an ordinary transformer with a 2048-token cache. If no readout can be specified that is differentiable and yields token logits, the central claim fails; if the authors supply a readout, verify it is actually described in the paper and reproduces the claimed memory/quality tradeoff.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's core proposal, BCT, replaces the standard attention layer with a bounded cache that is updated by MV_t = MV_{t-1} + ww_t^T wv_t (Section 3.2, Algorithm 1). The pseudocode returns MV_N, and the text describes how wq, wv, and ww are computed, but nowhere does it specify how the updated cache is read to produce the layer's output, let alone the next-token distribution. Standard attention is a read operation: output = softmax(QK^T)V, followed by an output projection. BCT's description stops after the write; no equivalent read is given. If no readout exists, the architecture cannot generate text, making the experimental claims in Section 4.3 ('BLEU scores are comparable... differences within 3 points') vacuous. This is not a minor notational omission, because the paper is proposing a full Transformer layer and its central claim is that bounded memory preserves inference quality. The missing readout is load-bearing: every subsequent claim assumes a complete, differentiable forward pass that maps tokens to logits, and the paper never supplies one. Even if a readout could be invented (e.g., output = MV^T wq or attention from a query to MV), that invention is not in the manuscript, so the method as presented is incomplete. An additional but secondary weakness is that Section 4 contains no actual numeric results, baselines, or model/training details, but the readout omission is the prior flaw.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes the Bounded-Cache Transformer (BCT), a Transformer attention layer with a fixed-size key-value (KV) cache. Input tokens are projected to write-query and write-value vectors, attention weights are computed over a stored key-vector sequence MK, and the M cache entries are updated via MV_t = MV_{t-1} + ww_t^T wv_t. The authors claim that this design keeps memory bounded, preserves inference speed, and maintains BLEU within 3 points of traditional methods for a 1B-parameter model on Common Crawl, RefinedWeb, and The Pile. The manuscript contains pseudocode for the update path and a schematic figure, but it does not specify how the cache is read to produce token logits, and its experimental section contains no quantitative results.","tokens_in":4605,"tokens_out":3420,"duration_ms":31783,"significance":"Bounded-memory attention is a practically important direction, and the NTM-style writing mechanism is a plausible starting point for compressing KV caches. However, as written, the paper cannot be evaluated: the architecture is incompletely specified, the experimental section contains no numbers, and the headline memory claim is definitional rather than empirical. If the readout path were supplied and the quality claims were backed by real comparisons with StreamingLLM, LESS, and standard attention, the approach could be a useful contribution. In its current form it does not meet the standard of a citable research paper.","major_comments":[{"comment":"The manuscript specifies only the write/update path of the KV cache. Algorithm 1 returns MV_N, but the paper never defines how the layer maps the updated MV (or MK and MV) to an output vector or to next-token logits. Standard attention is a read operation followed by an output projection; no analogous readout is given for BCT. This is load-bearing because the central claim is that bounded-memory inference preserves quality, and without a readout the architecture cannot be instantiated to generate text at all.","section":"Section 3.2, Algorithm 1"},{"comment":"The Results Analysis section asserts that memory usage remains constant after the inference length exceeds 2048, that inference speed is stable, and that BLEU scores are within 3 points of traditional methods. However, there are no tables, figures, numeric values, task descriptions, baseline names beyond 'traditional methods', error bars, or training/evaluation protocol details. These assertions cannot be checked, so the paper's central empirical claims are unsupported.","section":"Section 4.3"},{"comment":"The 'Adaptive Update Rule' bullet states that BCT 'refines the hidden state through a self-supervised learning approach during inference,' and the 'Learning Rate Adaptation' bullet mentions an adaptive learning rate mechanism. Neither is formalized: no objective function, training algorithm, or inference-time adaptation rule is defined anywhere in the paper. Since the quality claim depends on this adaptation, the mechanism is load-bearing and its omission leaves the method incomplete.","section":"Section 3.5"},{"comment":"The constant-memory claim is true by construction rather than by experimental measurement. Algorithm 1 and Section 3.1 fix M and use only fixed-size data structures, so memory remaining constant after the context length exceeds M is a definitional consequence, not an empirical finding. The paper should not present this as evidence for BCT's value; the substantive empirical questions are whether inference quality and throughput are maintained, and those are exactly the claims left unmeasured.","section":"Sections 3.1, 4.3"}],"minor_comments":[{"comment":"The update rule is written both as `MV = MV' + ww^T · wv` and as `MV_t = MV_{t-1} + ww_t^T · wv_t`, but the shapes of ww and wv and the meaning of the transposition are never defined, making the dimensionality of the update unclear.","section":"Section 3.2, 3.4"},{"comment":"The key-vector sequence MK is described only as 'composed of M key vectors'; the paper does not state whether MK is initialized, learned, fixed, or updated over time, even though attention weights in Step 2 depend on MK.","section":"Section 3.1, 3.2"},{"comment":"The 'Optimized for Parallelism' bullet claims that updates at each time step are independent, but Algorithm 1 defines a sequential recurrence in which MV_t depends on MV_{t-1}; the parallelism claim needs clarification.","section":"Section 3.5"},{"comment":"The table header appears garbled ('State Mem Comp' spans three columns), and the column labels are not clearly separated from the table body.","section":"Table 1"},{"comment":"Figure 1 is a schematic block diagram with no quantitative content; it does not provide experimental evidence, so its placement in the Experimental Results section is misleading.","section":"Figure 1"},{"comment":"Several references have inconsistent formatting, and reference [7] lists 'Borg et al.' for Common Crawl, which appears inaccurate; this should be checked.","section":"References"}],"recommendation":"reject","confidential_remarks":"The manuscript is far below the acceptance bar for a research paper: the architecture is incomplete, the experiments contain no data, and the main claimed advantage is definitional. I recommend rejection. If the authors supply a complete readout path, formalize the adaptation mechanism, and report actual comparisons with named baselines, a substantially revised submission might be worth reconsidering."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick take: this is a sketch, not a paper. The bounded-KV-cache idea is genuinely relevant, but the manuscript doesn't define a complete architecture, and the experiments contain no actual data. I agree with the reader's rejection, and the stress-test note lands: the readout is missing.\n\nThe paper does one good thing: it names a real problem—KV cache growth in long-context decoding—and proposes a fixed-capacity cache with a dynamic update. The bounded-memory claim is true by construction, and the authors cite Streaming LLM and LESS, so they're not ignoring prior work.\n\nThe problems are serious, and one is load-bearing. Section 3.2 and Algorithm 1 define only the write path: compute wq and wv, form attention weights over MK, update MV via MV = MV + ww^T wv. They never specify how MV is read to produce the layer output or the next-token distribution. Standard attention is a read operation (output = softmax(QK^T)V plus an output projection). BCT stops after the write. Without a readout, the model cannot generate tokens, so the claimed BLEU results in Section 4.3 are vacuous. This isn't a minor omission; it's the core of the layer.\n\nThe other issues are proportionate: Section 4 has no tables, no baselines, no error bars, no training details. The phrase \"differences within 3\" is not a result. And the update rule itself is the classic fast-weight/linear-attention outer-product update (Schmidhuber 1992; Katharopoulos et al. 2020), so the novelty is thin. The fixed-key memory also resembles the NTM write operation the authors cite.\n\nTo be fair, a reader could supply a readout (e.g., treat MV columns as value slots and attend from queries), but that invention is not in the manuscript. As specified, the method is incomplete.\n\nWho is this for? Possibly someone exploring bounded-memory architectures might glance at it, but they'd get more from the references. It doesn't deserve a serious referee in its current state. Desk reject is appropriate. If the authors add the readout and run real experiments with baselines and reported numbers, a workshop paper could work.\n\nRecommendation: reject, no peer review. Not because the idea is worthless, but because the presentation is incomplete and the evidence is absent.","headline":"A sketch with no readout path and no numbers; the bounded-cache idea is real but the paper cannot be evaluated as a method.","tokens_in":24,"tokens_out":2534,"would_cite":false,"duration_ms":53616,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims a Transformer with a fixed KV cache of M vectors keeps memory flat after capacity is reached while holding BLEU within 3 points of standard attention.","keywords":["bounded KV cache","key-value cache","Transformer inference","memory efficiency","long-context language models","dynamic cache update","fixed-capacity cache","large language models"],"falsifier":"Implement Algorithm 1 exactly as written with cache capacity $M=2048$, run a 4096-token prompt, and attempt to generate the next token; since the paper specifies only how $M_V$ is written and never how it is read to produce output, generation is undefined and the claimed BLEU scores from Section 4.3 cannot be reproduced without adding a readout equation.","tokens_in":3995,"feed_emoji":"🧠","tokens_out":6262,"duration_ms":52967,"temperature":0.7,"pith_summary":"The paper proposes the Bounded-Cache Transformer (BCT), an attention-layer design for decoder-only Transformers that replaces the ever-growing key-value cache with a fixed set of M key-value vectors. It claims this keeps inference memory and speed constant once the context exceeds the cache capacity, while preserving output quality. The stated experiments on a 1B-parameter model with cache capacity 2048 report BLEU within 3 points of a conventional KV-cache Transformer. If true, this would make long-context inference predictable in memory cost and useful for memory-constrained deployment.","feed_headline":"Fixed-size KV cache keeps LLM memory flat after 2048 tokens","feed_subtitle":"A Transformer variant updates a bounded key-value cache dynamically, holding BLEU within 3 points of standard attention.","key_machinery":"The load-bearing object is the bounded key-value cache $(M_K, M_V)$ and its additive update rule. $M_K$ is a fixed matrix of M key vectors and $M_V$ is a fixed-size sequence of M value vectors. Each input token writes into $M_V$ by an outer product of softmax attention weights over $M_K$ with a projected write vector $wv$, so memory stays $O(M)$ and each token's contribution is accumulated in place. This is what lets the model keep a fixed cache while still updating the stored information.","core_discovery":"The central claim is that a Transformer can maintain inference quality while bounding the KV cache to a fixed M slots, by treating the cache as a writable memory updated per token. The update is: for each input $x_t$, compute a write query $wq_t = W_{wq} x_t$ and a write key-value vector $wv_t = W_{wv} x_t$; score $wq_t$ against the fixed key matrix $M_K$; take softmax weights $ww_t$; then add into the cache $MV_t = MV_{t-1} + ww_t^T wv_t$. With this rule, the cache size never depends on sequence length, and the paper's experiments on a 1B-parameter model with $M=2048$ report that after 2048 tokens memory and speed flatten while BLEU stays within 3 points of the unbounded-cache baseline.","pith_inferences":["The bounded-cache update resembles a linear recurrent state; choosing a linear readout of $M_V$ would connect BCT to linear-attention and state-space models and invite stability analysis of the additive update.","A natural extension is to vary the cache capacity $M$; if quality degrades smoothly as $M$ shrinks, capacity can be chosen purely by memory budget, but a sharp threshold would make capacity tuning critical.","The memory claim and the quality claim are separable: the $O(M)$ bound follows from the write path alone, whereas BLEU parity depends on the readout, so future work should report them independently."],"forward_implications":["Long-context inference memory becomes $O(M)$ per layer instead of $O(\\text{context length})$, enabling deployment on devices with fixed memory budgets.","Inference speed stops degrading once the context exceeds the cache capacity, so throughput stays stable for arbitrarily long prompts.","BLEU within 3 points implies bounded-cache models could serve as drop-in replacements for standard attention in generation tasks.","The additive update can be trained in parallel across tokens because the update at each time step is independent, as the paper argues in its implementation notes."],"supporting_citations":[{"why":"Defines the standard Transformer attention and KV cache that BCT modifies.","marker":"[3]"},{"why":"Prior streaming LLM approach with attention sinks that BCT positions against.","marker":"[4]"},{"why":"Prior KV cache compression method LESS that BCT compares itself to.","marker":"[5]"},{"why":"Neural Turing Machines provide the external-memory update inspiration for dynamic KV updates.","marker":"[6]"},{"why":"Common Crawl is one of the datasets used in the experiments.","marker":"[7]"},{"why":"RefinedWeb is one of the datasets used in the experiments.","marker":"[8]"},{"why":"The Pile is one of the datasets used in the experiments.","marker":"[9]"}],"fun_headline_variants":["Transformer cache fixed to 2048 slots holds LLM quality","Bounded KV cache keeps LLM memory flat, BLEU near baseline","LLM inference with fixed-size KV cache: memory capped, quality kept","BCT predefines cache capacity, memory flat after 2048 tokens","Fixed KV cache for LLMs: quality within 3 BLEU, memory constant"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes the updated key-value sequence in the cache is enough to generate the next word, but the paper never specifies the readout computation that turns the cache into a token prediction.","fun_headline_variants_meta":{"raw":{"variants":["Transformer cache fixed to 2048 slots holds LLM quality","Bounded KV cache keeps LLM memory flat, BLEU near baseline","LLM inference with fixed-size KV cache: memory capped, quality kept","BCT predefines cache capacity, memory flat after 2048 tokens","Fixed KV cache for LLMs: quality within 3 BLEU, memory constant"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000675,"raw_usage":{"total_tokens":3020,"prompt_tokens":841,"completion_tokens":2179,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":457,"completion_tokens_details":{"reasoning_tokens":2080}},"tokens_in":457,"tokens_out":2179,"duration_ms":13104,"temperature":1.0,"reasoning_tokens":2080,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T13:53:22.453606+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Implement Algorithm 1 exactly as written with cache capacity $M=2048$, run a 4096-token prompt, and attempt to generate the next token; since the paper specifies only how $M_V$ is written and never how it is read to produce output, generation is undefined and the claimed BLEU scores from Section 4.3 cannot be reproduced without adding a readout equation.","supporting_citations":[{"cited_title":"Attention is all you need","cited_arxiv_id":null,"evidence_quote":"Defines the standard Transformer attention and KV cache that BCT modifies."},{"cited_title":"Efficient streaming language models with attention sinks","cited_arxiv_id":null,"evidence_quote":"Prior streaming LLM approach with attention sinks that BCT positions against."},{"cited_title":"Get more with less: Synthesizing recurrence with kv cache compression for efficient llm inference","cited_arxiv_id":null,"evidence_quote":"Prior KV cache compression method LESS that BCT compares itself to."},{"cited_title":"Common crawl: A corpus for web-scale information extraction","cited_arxiv_id":null,"evidence_quote":"Common Crawl is one of the datasets used in the experiments."},{"cited_title":"Threshold solutions for the intercritical inhomogeneous NLS","cited_arxiv_id":"2205.09714","evidence_quote":"RefinedWeb is one of the datasets used in the experiments."}],"review_version":1}