{"id":"3d89ef1f-8d94-470d-8dd6-3d2af0451406","arxiv_id":"2608.02560","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"For SSMs, retrieved documents can be pre-encoded into hidden states and injected at query time, making context ingestion O(1); PRECOG implements this, but real retrieval is not evaluated and the 4500x latency claim applies only to context ingestion.","lead":"PRECOG pre-encodes text chunks into the fixed-size hidden state of an SSM language model, then injects the best-matching stored state at query time instead of re-reading the retrieved text, cutting context-ingestion cost from seconds to milliseconds. The same trick is stretched into a hierarchical \"SMC\" persistent memory for edge devices, but the headline 4500x latency gain is overstated because query tokens still take about half a second on the target hardware.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 1 is proven for the abstract SSM recurrence, but TENNs-LLM's causal-conv front-end has a buffer omitted from the stored 192 KB state; without it, injection is not the same computation as in-context ingestion.","rationale":"The reader's weakest_assumption is exactly the issue I consider load-bearing: the saved state is not a complete snapshot of TENNs-LLM's memory because the causal-conv front-end has a buffer omitted from the 192 KB figure. This is central because the paper's core contribution is the claim that PRECOG is 'the same computation, algebraically refactored' (Section 4.3), used to justify both the quality match and the O(1) prefill. If the conv buffer is required for equality, the theorem as instantiated is false: the state trajectory after injecting only SSM states differs from the in-context trajectory. The empirical Table 3 could still show small differences because the effect is limited to the first few query tokens, but the paper explicitly says deviations larger than FP16 quantization 'indicate implementation issues, not method failure'—this is an implementation issue that is not disclosed. The paper's limitation section (Section 7) acknowledges top-k inexactness and memory horizon but does not mention auxiliary state, despite the architecture including a causal convolution. The proposed test directly checks whether including the buffer restores equality. If it does, the fix is straightforward and the conditional verdict stands; if it does not, the central 'same computation' claim needs to be weakened to an approximation. I agree with the reader's conditional verdict; the concern is already reflected there.","tokens_in":18519,"tokens_out":7124,"duration_ms":73446,"concrete_test":"Instrument the TENNs-LLM forward pass and compare three rollouts on the SQuAD subset used in Table 3: (A) in-context RAG S(h0, c⊕q); (B) PRECOG as described, injecting only the 24-layer SSM states; (C) PRECOG with per-layer causal-conv buffers (last 3 input vectors) saved and injected alongside the SSM states. If the output logits of (C) match (A) to within the FP16 bound while (B) diverges for early query tokens, then the 192 KB state is incomplete and Theorem 1's application to TENNs-LLM fails as stated. Also report the magnitude and decay of the divergence over the query length: if it vanishes within one or two tokens the practical impact may be small, but the exactness claim still needs revision.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that PRECOG is 'the same computation, algebraically refactored' (Section 4.3), because the SSM update map Φ(h,x)=h⊙α(x)+β(x) is time-translation invariant. However, Theorem 1 is stated and proved only for this abstract recurrence (Appendix A). The actual TENNs-LLM block (Table 1; Appendix E) includes a causal convolution front-end with kernel 4 before the SSM. A causal conv is itself stateful: at the first query token its buffer must contain the last (kernel−1) context tokens of each layer's input stream. The paper's 192 KB state calculation (Appendix D.1) is 24×4096×2 bytes—only the recurrent SSM state. The conv buffer is not stored; at 2048 dims × 3 slots × 2 bytes × 24 layers it is ≈288 KB, larger than the saved state. Consequently, rolling the recurrence over context and then query, versus injecting only the SSM states and processing the query, are not equal for the full model: the first few query tokens see a different conv-filtered input, and the difference propagates through the recurrence. Theorem 1's equality for TENNs-LLM is therefore unproven unless the conv buffer is explicitly saved and injected. The paper's limitations section (Section 7) acknowledges top-k inexactness and memory horizon but does not mention this auxiliary state, so the 'complete summary' claim is too strong.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes PRECOG, a retrieval mechanism for SSM-based language models that pre-encodes corpus chunks offline into the model's recurrent hidden state and, at query time, injects the retrieved state as the initial condition instead of re-ingesting context tokens. The central formal claim is Theorem 1: for a time-translation-invariant SSM update Φ(h,x), rolling the recurrence over context then query equals rolling over the concatenated sequence. The authors instantiate this on TENNs-LLM, a 1.2B gated SSM with a 192 KB nominal hidden state, and report that top-1 PRECOG matches in-context RAG on SQuAD v1.1 while cutting the reported context-ingestion latency from ~27 s to <6 ms. A second mechanism, SMC, organizes accumulated interaction states into cognitive-domain clusters and uses the same injection primitive for persistent memory.","tokens_in":18964,"tokens_out":10314,"duration_ms":115930,"significance":"If the state-capture issue is resolved, the core insight is simple and appealing: for an SSM whose full recurrent state is saved and injectable, context ingestion at prefill becomes independent of retrieved-context length. The algebraic identity is proved correctly in the abstract setting, and the paper contains useful, reproducible storage/bandwidth calculations (Appendix D) plus a clean implementation-consistency experiment. However, the headline claims currently overreach: Theorem 1 is not connected to the actual stateful components of TENNs-LLM, the latency numbers are internally inconsistent, the main RAG experiment uses the gold paragraph in both arms, and one ablation directly contradicts the theorem. These issues are fixable, but they are load-bearing for the paper's claims.","major_comments":[{"comment":"Theorem 1 is proved only for the abstract recurrence Φ(h,x). The actual TENNs-LLM block includes a causal convolution front-end (Table 1, kernel 4; Appendix E) whose buffer of previous input tokens is stateful. The artifact counted in Appendix D.1 (24×4096×2 = 192 KB) contains only the per-layer SSM recurrent state; it does not include the conv buffers (roughly (kernel−1)×d_inner×2 bytes per layer, i.e., ~576 KB if operating on the inner dimension). With an empty conv buffer after injection, the first query tokens in PRECOG see a different conv-filtered input than in in-context RAG, so the equality S(S(h0,c),q)=S(h0,c⊕q) is not established for the full model. The state definition must be expanded to include all stateful components, or the 'complete summary' claim must be withdrawn.","section":"§3/Table 1, §4.2, §4.3, Appendix D.1"},{"comment":"The abstract and Table 2 state that TTFT/prefill is <6 ms, but Appendix D.6 decomposes PRECOG TTFT as 585 ms (5 ms retrieval, ~1 ms load/inject, 0.5 ms tokenization, 526 ms query ingestion at 19 tok/s, ~53 ms first-token compute). The <6 ms figure is only the overhead beyond normal query processing. Since the user-visible time to first token includes query processing, the headline '~27 s to <6 ms' is an order-of-magnitude overstatement. Please correct all such statements and consistently distinguish 'context-ingestion eliminated' from total TTFT. Figure 3 already shows 585 ms; the main text should match.","section":"Abstract, Table 2, §4.2, Appendix D.6, Figure 3"},{"comment":"The main SQuAD evaluation does not evaluate retrieval. Both arms use the gold paragraph: in-context RAG prepends the gold paragraph, and PRECOG injects the hidden state of that same paragraph. The reported 0.2 EM/F1 gap therefore tests only the state-injection implementation under Theorem 1; it says nothing about retrieval quality, false negatives, or the FAISS/embedding pipeline. A RAG paper needs an open-domain or distractor setting where the index is actually searched and retrieval recall/hits are reported. The HotpotQA ablation moves in this direction but is in an appendix and is not the headline result.","section":"§6, Appendix F"},{"comment":"Table 8 reports PRECOG losing up to 15 F1 to in-context RAG on chunks longer than ~600 tokens, with the text saying in-context RAG 'retains positional access to all tokens.' But both configurations use the same TENNs-LLM SSM backbone; under Theorem 1, in-context RAG after reading the same long paragraph also has only the fixed-size recurrent state and should have the same memory horizon. If the observation is real, then either the two arms differ in some unstated way (e.g., a different backbone or a non-recurrent path), or Theorem 1 does not hold for the full model. This appendix directly undermines the paper's central equivalence and must be reconciled.","section":"Appendix G.3, Table 8"},{"comment":"SMC's end-to-end claims—consolidation into semantic states, O(1) session initialization, and joint episodic/corpus retrieval—are supported only by cluster-separation ratios on Harry Potter film transcripts and a t-SNE visualization. Appendix I.2 explicitly states that full validation on naturalistic data with the deployed pipeline is left to future work. As written, SMC is an architecture proposal without task-level evidence. Include at least one quantitative downstream evaluation of memory-augmented QA or dialogue to substantiate the persistent-memory contribution.","section":"§5, Appendix I.2"}],"minor_comments":[{"comment":"The softmax weights in Eq. (8) are written as softmax of similarities but no temperature or normalization detail is given. Clarify the exact composition rule used in the top-k experiments.","section":"§4.4, Eq. (8)"},{"comment":"The generation protocol uses top-p sampling (p=0.9), so the 0.2 EM/F1 difference between top-1 PRECOG and in-context RAG may be sampling noise. Report multiple seeds or confidence intervals before calling it a quantization-bound match.","section":"Table 3"},{"comment":"Figure 1 says 'first generated token at ~6 ms after retrieval,' which conflicts with the 585 ms TTFT in Appendix D.6. Make the figure and caption consistent with the corrected latency accounting.","section":"§4.2, Figure 1"},{"comment":"The phrase 'complete summary of everything the model has read' is too strong; Proposition 1 gives a more precise statement (sufficient statistic for the continuation under model dynamics). Use the qualified phrasing throughout.","section":"Abstract, §4.1"},{"comment":"The appendix acknowledges mixing binary and decimal units; it would be cleaner to use one convention consistently in tables.","section":"Appendix D.5"},{"comment":"Some references carry 2026 arXiv identifiers (e.g., [9], [36] and the paper itself). Please verify all identifiers and dates are correct at publication time.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper is tied to a pending patent and a custom unreleased backbone, which limits reproducibility; this is not disqualifying if the technical claims are corrected. The most serious concern for the editor is that the paper's central 'exact equivalence' is not established for the actual model because auxiliary state is omitted, and Table 8 appears to contradict the equivalence even setting that aside. These need to be resolved before a positive decision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is a good example of a useful engineering idea buried under an overclaimed formal guarantee. The core observation—that for a time-invariant recurrent model, pre-computing the hidden state for a document chunk and injecting it is the same as ingesting the chunk—is correct, but it's also a restatement of the definition of a deterministic state transition. The proof of Theorem 1 is just associativity of rolling the recurrence. So the 'mathematical guarantee' is a tautology, not a discovery. That doesn't make the system useless; it makes the claim oversized.\n\nWhat's genuinely useful is the system design: pre-encoding chunks offline, retrieving by embedding, injecting a 192KB state, and the storage/latency analysis against KV-cache RAG. The appendix roofline numbers are transparent and check out. The ablations, though limited, give some practical guidance on layer depth and top-k.\n\nThe soft spots are real. First, the exactness theorem is stated for the abstract SSM recurrence, but TENNs-LLM has a causal-conv front-end whose buffer is not part of the saved state. The stress-test note gets this right: without storing and injecting those conv registers, rolling context then query is not the same as injecting the SSM state and starting the query. The first few query tokens see different inputs, and the difference propagates. So the paper's central claim that PRECOG 'is the same computation, algebraically refactored' is unproven for the actual model. That's a load-bearing flaw, not a nitpick.\n\nSecond, the evaluation doesn't actually test retrieval. Both arms use the gold SQuAD paragraph, so the experiment is a consistency check of the implementation, not evidence that the retrieval system works end-to-end. The paper admits this implicitly, but the narrative still leans on it.\n\nThird, the latency numbers are confusing. The abstract and Figure 1 imply the first token arrives in <6ms; Appendix D.6 says the TTFT is 585ms, with ~526ms spent ingesting the query at 19 tok/s. The <6ms is only the injection overhead, not the prefill or TTFT. The paper should say that plainly.\n\nThese issues are fixable. Save the conv buffer, test with real retrieval over a corpus, and report TTFT honestly. If the authors do that, the contribution is credible: O(1) context ingestion for SSM RAG on edge is a practical improvement, even if not a new mathematical principle.\n\nI'd send it to peer review, but with the expectation that the reviewers press on the exactness claim and the evaluation. It's worth a serious look, not a desk reject, and not a pass as-is.","headline":"PRECOG's exactness guarantee is a tautology, and the actual model's conv buffer breaks it; still, the state-injection idea is worth taking seriously.","tokens_in":19455,"tokens_out":2961,"would_cite":false,"duration_ms":30043,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"For state-space models, retrieved context can be injected as a precomputed hidden state in O(1), exactly matching in-context reading.","keywords":["state-space models","retrieval-augmented generation","hidden state injection","O(1) prefill","persistent memory","edge inference","time-translation invariance","state-based RAG"],"falsifier":"Run the same chunk through the model twice: once by prepending it to a query (in-context), once by saving the post-chunk state and injecting it. Compare the log-probabilities of the first generated token. Any deviation larger than the FP16 rounding bound (~2^-10 per element, accumulated over query length) disproves the exactness claim as implemented. The same test across chunk lengths from 100 to 3,000 tokens would separate a genuine memory-horizon effect from an unsaved auxiliary state: Theorem 1 predicts exact equality at every length, so a systematic divergence beyond roughly 600 tokens wou","tokens_in":18444,"feed_emoji":"⚡","tokens_out":8873,"duration_ms":86918,"temperature":0.7,"pith_summary":"PRECOG is the claim that for any state-space language model with a time-translation-invariant recurrence, the retrieved-context phase of RAG can be moved offline: encode each chunk once into its fixed-size hidden state, then inject the best-matching state as the initial condition at query time. The paper proves this is not an approximation by showing the recurrence satisfies S(h0, c ⊕ q) = S(S(h0, c), q), so the injected-state trajectory is bit-identical to re-reading the chunk. On a 1.2-billion-parameter gated SSM with a 192 KB state, this cuts prefill from about 27 seconds to under 6 milliseconds on edge hardware while matching answer quality on extractive QA. The same primitive is extended into SMC, a hierarchical persistent memory with O(1) session initialization. If correct, recurrent models can do retrieval and long-term memory without paying any context-token ingestion cost, a structural advantage Transformers cannot replicate.","feed_headline":"Retrieval prefill drops from 27 s to 6 ms with exact state injection","feed_subtitle":"A 1.2B model matches in-context retrieval exactly while cutting prefill by ~4,500x.","key_machinery":"The load-bearing object is Theorem 1, the PRECOG–RAG equivalence: for a time-translation-invariant SSM update Φ(h, x), S(h0, c ⊕ q) = S(S(h0, c), q). The property that makes it true is that the per-token update depends only on the current hidden state and token, so the fixed-size recurrent state is a position-agnostic sufficient statistic of everything read. This identity converts retrieval from a token-ingestion operation into a state-copy operation: store a 192 KB hidden state per chunk, retrieve by embedding similarity, and initialize the model's recurrent buffers with it. The paper argues the same identity fails for Transformer KV-caches because positional encodings make cache contents p","core_discovery":"The central discovery is an algebraic identity with a deployment consequence: because the SSM update Φ(h, x) depends only on the current hidden state and token, not on absolute position, rolling the recurrence over a context and then a query equals rolling it over the concatenated sequence. Therefore a hidden state precomputed offline from a retrieved chunk is a sufficient statistic for that chunk; injecting it as the initial state reproduces, exactly, what the model would compute if it had read the chunk at the start of the query. The paper instantiates this on a 1.2B gated SSM, reports that top-1 injection matches in-context RAG within FP16 rounding on a 1,000-question extractive QA sample","pith_inferences":["If the completeness condition is satisfied in deployed code, this design point extends beyond edge devices: any recurrent model serving hot chunks could trade roughly 200x more storage per chunk for a four-orders-of-magnitude reduction in per-query ingestion latency.","A straightforward engineering check follows from the theorem: compare first-token logits under injection versus in-context reading; any divergence above the FP16 rounding bound identifies an auxiliary state, such as a convolution buffer or normalization statistic, that was not saved.","The same algebraic identity suggests a route to exact multi-document retrieval: instead of averaging states in hidden space, one could search for a composition rule that respects the recurrence, or fine-tune retrieval to select a single state that already contains fused context.","The memory-horizon ablation implies a measurable deployment rule: chunk at the model's empirical forgetting length, and reserve in-context reading for the long tail, unless the backbone's effective memory is extended."],"forward_implications":["For any recurrent backbone whose update depends only on the current state and token, PRECOG turns retrieved-context ingestion into a single state copy; the answer distribution is identical to in-context RAG by construction.","On the paper's 1.2B gated SSM with a 192 KB state, prefill drops from about 27 seconds to under 6 ms on edge hardware, roughly a 4,500x speedup at matched answer quality on a 1,000-question extractive QA sample.","Per-chunk storage is constant in context length: 192 KB versus a KV cache that grows at 32 KB per token, crossing at 6 tokens and reaching an 85x gap at 512-token chunks, with the gap widening further at longer contexts.","The same injection substrate gives persistent device memory: consolidated semantic states are written into the recurrent state at session start, so initialization stays O(1) no matter how much history has accumulated.","The backbone's effective memory length bounds the useful chunk size: PRECOG and in-context RAG stay statistically indistinguishable below roughly 600 tokens and diverge beyond that because PRECOG inherits the model's forgetting profile exactly."],"fun_headline_variants":["Prefill drops from 27s to 6ms via SSM state injection","State injection: 27s to 6ms retrieval prefill","O(1) prefill: SSM hidden state replaces in-context RAG","1.2B SSM matches RAG with 4500x faster prefill","SSM state injection: prefill 27s to 6ms, matches RAG"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The saved state must be a complete snapshot of every memory-carrying buffer in the model—not just the recurrent state but also the causal convolution front-end, normalization statistics, and any gating or sampling registers—because the exactness identity holds only if the injected state is the state the model would have reached by reading the chunk.","fun_headline_variants_meta":{"raw":{"variants":["Prefill drops from 27s to 6ms via SSM state injection","State injection: 27s to 6ms retrieval prefill","O(1) prefill: SSM hidden state replaces in-context RAG","1.2B SSM matches RAG with 4500x faster prefill","SSM state injection: prefill 27s to 6ms, matches RAG"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001028,"raw_usage":{"total_tokens":4230,"prompt_tokens":867,"completion_tokens":3363,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":611,"completion_tokens_details":{"reasoning_tokens":3256}},"tokens_in":611,"tokens_out":3363,"duration_ms":23059,"temperature":1.0,"reasoning_tokens":3256,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T04:42:30.946990+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same chunk through the model twice: once by prepending it to a query (in-context), once by saving the post-chunk state and injecting it. Compare the log-probabilities of the first generated token. Any deviation larger than the FP16 rounding bound (~2^-10 per element, accumulated over query length) disproves the exactness claim as implemented. The same test across chunk lengths from 100 to 3,000 tokens would separate a genuine memory-horizon effect from an unsaved auxiliary state: Theorem 1 predicts exact equality at every length, so a systematic divergence beyond roughly 600 tokens wou","supporting_citations":[],"review_version":1}