{"id":"e9690141-7357-4cc7-ac7f-04f7c443d538","arxiv_id":"2505.14398","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"By storing KV caches of prior reasoning traces and retrieving them during generation, LAG improves LLM agent accuracy and efficiency over standard agentic systems and reflection methods.","lead":"This paper introduces log-augmented generation (LAG), which lets an LLM reuse the internal key-value representations of past reasoning traces when tackling new questions. On multi-hop QA and science and math benchmarks, the method reports higher accuracy and fewer reasoning steps than standard agentic systems, reflection-based memory, and prior KV-cache reuse methods.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed KV-reuse mechanism is never isolated: without a control using irrelevant or random KV caches, the reported gains could come from any injected hidden states or from RoPE re-rotation artifacts rather than from reusing the retrieved reasoning content.","rationale":"The reader's weakest_assumption identifies the same load-bearing issue: KV transfer across contexts and RoPE re-application are asserted rather than demonstrated. I agree that this is the most fundamental gap. The seen/unseen leakage and test-set tuning concerns are real but less central: the unseen columns in Tables 1 and 2 show comparable gains to the seen columns, so the headline does not rest primarily on self-retrieval of the exact query. The missing content-control is more important because it targets the mechanism itself: if random or irrelevant KV caches produce similar gains, the paper's central explanation of reusable reasoning is unsupported, even though the empirical numbers might be reproducible. The proposed test would settle this directly. I therefore keep the reader's CONDITIONAL verdict unchanged, with acceptance contingent on the content-control and a genuinely held-out validation split for configuration choices such as top-k and storage-token selection.","tokens_in":13856,"tokens_out":8461,"duration_ms":92374,"concrete_test":"Run LAGKV unchanged on the four unseen test splits, but in the injection step replace each retrieved log's KV vectors with KV vectors of the same shape extracted from a matched but semantically unrelated last response (e.g., a random log from the store, or scrambled token positions within the same log). Keep retrieval ranking, top-k, prompt, and RoPE re-rotation identical. If the random-KV condition retains most or all of the LAGKV gain over standard agentic, the claimed content-based reuse is not supported. A complementary check: where log text is available, recompute the stored last-response KV in-place at the new context positions instead of re-rotating, and compare to LAGKV; a large gap would quantify RoPE re-rotation corruption. Report per-dataset EM/F1 with the same significance test used in Appendix E.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 2.2's load-bearing premise is that 'the KV representation of a single token encapsulates more than just the token's meaning' and that, after undoing and re-applying RoPE (Eqs. 2-3), a stored last-response KV cache can carry a prior reasoning trace into a new context. For the central claim in Tables 1-2 to hold, the accuracy gain must be caused by the semantic content of the retrieved reasoning. The experiments never establish this. LAGKV is compared to no-log, reflection, KV-cache, and text-log baselines, but each differs from LAGKV in more than one way; none controls for the possibility that injecting any KV vectors, even from an unrelated or random response, shifts the model's predictions in the observed direction. The RoPE re-rotation step is itself approximate: RoPE is applied inside every layer's attention, so hidden states entering later layers are already position-dependent; inverting only the final key rotation and re-applying a new angle cannot restore a fully position-free representation. If this approximation injects a systematic bias, the gains on GPQA unseen (18.5 to 30.4 EM) and Musique unseen (27.0 to 32.2 EM) may reflect that bias rather than reusable reasoning. The missing experiment is a content-control: keep retrieval, storage, and injection identical, but replace the retrieved log KV vectors with KV vectors from irrelevant or random logs. Without such a control, the central mechanism is under-determined.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes log-augmented generation (LAG), a framework in which reasoning traces from previous tasks are stored as KV caches of tokens from the last model response, retrieved by semantic similarity to the current subtask, and injected into the model's generation after re-rotating positional embeddings. The implementation, LAGKV, is evaluated on Musique, 2WikiMultiHop, GPQA, and MMLU-Pro against a ReAct agent, a reflection baseline, a KV-cache baseline, and text-log variants. The authors report consistent accuracy and efficiency gains, especially on unseen questions, and include ablations on stored token subset and number of retrieved logs.","tokens_in":14203,"tokens_out":7285,"duration_ms":68773,"significance":"If the KV-reuse mechanism is real, LAG offers a genuinely new way to reuse computation across tasks, with practical efficiency gains and accuracy improvements, and the paper's ablations on token subset and number of logs are useful. The work is empirical; the central result is measured on held-out questions and is not circular in the sense of fitting constants to the data. The static log-store setting and the qualitative examples are strengths. However, the reported gains cannot yet be attributed to the semantic content of retrieved reasoning, because no control with irrelevant or random KV vectors is provided. The significance therefore hinges on a single missing experiment.","major_comments":[{"comment":"The central claim that reusing the retrieved reasoning content causes the accuracy gains is underdetermined because LAGKV is never compared with a content-control condition. All existing baselines differ from LAGKV in more than one way: the KV-cache baseline encodes only the last response without the full reasoning context, LAGtext uses text, and reflection uses extracted insights. None controls for the possibility that injecting any KV vectors—even from an irrelevant log or random tokens—after the approximate RoPE inversion/re-rotation in Eqs. (2)-(3) shifts the model's predictions. A proper control would keep the storage, retrieval, and injection pipeline unchanged but replace the retrieved log KV values with KV values from unrelated logs (or random token sequences) of matched length. Without such a control, the large gains on GPQA unseen (30.4 vs. 18.5 EM) and Musique unseen (32.2 vs. 27.0 EM) cannot be uniquely attributed to the semantic content of the retrieved reasoning.","section":"Section 2.2 (Eqs. 2-3), Tables 1-2"},{"comment":"The 'seen' evaluation is contaminated. Since the log store is built from the same 70% of questions that are later evaluated as 'seen,' the retriever can retrieve a log of the exact same question, whose last model response contains the answer, making the seen numbers a measure of memorization or in-distribution retrieval rather than generalization. The abstract's claim of 'new, unseen challenges' is supported by the unseen columns, but the seen columns should either be removed or explicitly reframed as an in-distribution memory setting, and the conclusion should not rely on them.","section":"Section 3.1, Tables 1-2"},{"comment":"The conceptual premise that 'a token's KV value is a weighted aggregation of embeddings from the entire surrounding context' is technically inaccurate for the layer at which the KV is stored. In self-attention, the key and value vectors for a token are produced by linear projections of that token's hidden state; it is the attention output, not the stored KV, that is a weighted sum of value vectors. The token's hidden state is context-dependent through earlier layers, so the claim can be repaired, but as written it misdescribes the mechanism and should be corrected with a precise statement about which quantities are context-dependent.","section":"Section 2.2"},{"comment":"The statistical significance analysis is insufficient. A paired t-test over per-question exact-match outcomes from a single run does not account for run-to-run variance due to sampling in LLM generation and retrieval, and it is unclear whether the t-test was computed on all questions or only those with observed variation. The paper should report means and standard deviations across multiple seeds (or at least bootstrap confidence intervals) for the main comparisons in Tables 1 and 2.","section":"Appendix E and Section 3.1"}],"minor_comments":[{"comment":"There are spacing and formatting errors in the table: '57.565.6' should be '57.5 65.6', and '65.02.29' should be '65.0 2.29'. Similar spacing issues appear in several entries.","section":"Table 1"},{"comment":"The terminology is inconsistent: 'LAG KV', 'LAGKV', 'LAGtext', and 'LAG text' are used interchangeably. Pick one notation per method and use it consistently.","section":"Throughout"},{"comment":"The x-axis label is rendered as 'Number/uni00A0of/uni00A0max/uni00A0iterations', apparently due to non-breaking spaces; it should read 'Number of max iterations'.","section":"Figure 3"},{"comment":"The column 'KV size (GB)' is ambiguous: the values (e.g., 6.1 for 'last action') are difficult to interpret without knowing whether they are per-log, per-question averages, or total store sizes for the 70% split. Please clarify the unit and scope.","section":"Table 4"},{"comment":"The definition of 'unsolvable' for Table 3 should be stated more precisely; the text says questions remain unanswered at the performance plateau (iteration 20/25), but it is not specified whether this is a hard cutoff in the experiment or an operational criterion applied post hoc.","section":"Table 3 and Section 4.1"}],"recommendation":"major_revision","confidential_remarks":"The core idea is interesting and the empirical direction is publishable, but the missing content-control for the KV injection mechanism is the key weakness. The 'seen' evaluation also needs to be reframed or removed. With an added content-control experiment (or a clear argument why it is unnecessary), the paper could be suitable for a top conference. I recommend major revision rather than reject because the missing experiment is feasible and the central claim is not inherently circular."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Peter,\n\nThis paper proposes a genuinely new mechanism: storing KV values from the last response of a reasoning trace (after encoding the full trace) and injecting them into new contexts via RoPE re-rotation. That is a real departure from both reflection-based memory and KV cache efficiency methods, and the experiments are broader than most—four datasets, comparisons against reflection and KV-cache baselines, plus textual-log ablations. The unseen-question gains on Musique, 2WikiMultiHop, and GPQA are consistent, and the method cuts reasoning iterations substantially. That is worth a serious look.\n\nThe soft spots are real but fixable. First and most load-bearing: the central mechanism is never isolated. LAGKV is always compared to something that differs in more than one way—text logs, no logs, or a KV cache baseline that encodes only the last response. There is no control that injects KV vectors from irrelevant or random logs. If any hidden-state injection shifts the model's predictions, the accuracy gains on GPQA unseen (18.5→30.4) and Musique unseen (27.0→32.2) could be partly an artifact of the injection rather than of reusing the retrieved reasoning. This is a missing control, not a demonstrated flaw, but it matters for the title claim.\n\nSecond, the storage costs are enormous: Table 4 lists 38–46 GB for the last-response strategy on a single dataset. The paper calls the system 'efficient and scalable,' but that only holds if you measure reasoning steps, not memory. The 'last action' variant cuts storage to ~6 GB, but then the gains and efficiency advantages shrink. The efficiency claim needs reframing.\n\nThird, the 'seen' results are contaminated by self-retrieval—the log store contains logs of the very same questions being evaluated. The unseen numbers are the ones that matter, and they are the relevant results in Tables 1 and 2.\n\nFourth, the t-test appendix is a good addition, but it reports only P-values, not error bars or multiple seeds, and MMLU-Pro unseen is not significant. That is minor; the headline claims rest on the other three datasets.\n\nWho is this for? Anyone working on test-time learning, memory, or KV-cache reuse for LLM agents. It deserves a serious referee. The idea is novel and the empirical picture is promising, but the missing content-control needs to be added and the storage/efficiency claims recalibrated. I would send it to review, not desk-reject, with a request for major revisions.","headline":"New mechanism worth a referee's time, but the missing random-KV control means the reuse claim is under-determined.","tokens_in":14697,"tokens_out":3257,"would_cite":true,"duration_ms":32277,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that storing and re-injecting the key-value states of past reasoning traces lets LLM agents solve new tasks more accurately and in fewer steps.","keywords":["log-augmented generation","KV cache reuse","test-time reasoning","agentic retrieval-augmented generation","multi-hop question answering","reasoning trace memory","retrieval augmentation","transformer attention states"],"falsifier":"Run LAGKV again but replace the retrieved KV logs with KV caches taken from unrelated questions (or with random vectors of the same shape); if exact match stays near the reported LAGKV level, the gain is not coming from reused reasoning, whereas a drop back toward the no-log baseline would confirm the reuse mechanism is doing the work.","tokens_in":13701,"feed_emoji":"🧠","tokens_out":10527,"duration_ms":98654,"temperature":0.7,"pith_summary":"The paper sets out to show that an LLM agent can learn from past experience by reusing the computation itself, not a summary of it. Its log-augmented generation (LAG) framework stores the key-value (KV) attention states of previous reasoning traces, retrieves the most relevant ones for a new question, and injects them into the model's context. In the LAGKV implementation the model encodes all earlier model responses in a task but stores only the KV values of the last response, relying on the attention mechanism's context-aggregation property to preserve the whole reasoning trace compactly. The paper reports that this approach outperforms standard agentic systems, reflection-based memory, and existing KV-cache methods on knowledge-intensive multi-hop QA and on reasoning-intensive math and science benchmarks, while also reducing the number of reasoning steps. If the mechanism holds, agents could accumulate reusable reasoning across tasks without extra distillation, fine-tuning, or retraining.","feed_headline":"KV-cached reasoning lifts LLM agent accuracy and cuts steps","feed_subtitle":"Stored key-value states of past reasoning are injected into new prompts, beating reflection and cache baselines on four benchmarks.","key_machinery":"The load-bearing object is the KV-cache log representation used by LAGKV: the key-value attention states of the tokens in the last model response, computed while the model attends to the entire multi-turn reasoning trace. This is what lets the method store a small token subset yet claim to preserve full context. The second piece of machinery is rotary-position re-embedding: stored KV caches are stripped of their original RoPE rotation and re-rotated with the positional IDs of the new context before being concatenated with the model's working context.","core_discovery":"The central claim, stated on the paper's own terms, is that directly reusing prior reasoning via KV caches is a better form of test-time memory than re-deriving, reflecting on, or distilling that reasoning into text. Concretely, LAGKV encodes the full reasoning context of a previous task into KV values, stores only the KV values of the tokens from the last model response, and, when a new task arrives, retrieves the most relevant of these stored caches by semantic similarity and feeds them into the model's context. Because each token's KV value is a weighted aggregation of all preceding context, the stored last-response caches are claimed to carry the essential reasoning of the whole trace. Across Musique, 2WikiMultiHop, GPQA, and MMLU-Pro, the paper reports that LAGKV beats a standard agentic system, a reflection-based memory method, and a KV-cache baseline on exact match while using fewer iterations, with the gains attributed to both knowledge reuse and insight reuse.","pith_inferences":["Editorial inference: the same stored log could be retrieved at finer granularity than a whole trace, letting a new task reuse only the specific intermediate reasoning step that matches its current sub-question.","Editorial inference: the paper's storage analysis implies an adaptive compression policy—store only final-action tokens for easy tasks and more tokens for hard ones—could keep most of the accuracy gain at a fraction of the reported 6.35 GB average store size.","Editorial inference: because KV caches are tied to a specific model's hidden-state geometry, a log store built with one open model cannot be consumed by another unless the models share architecture or a projection is learned between KV spaces; the paper does not address cross-model reuse."],"forward_implications":["Agent systems can drop explicit reflection or distillation steps, because directly injecting stored KV states from prior tasks yields better accuracy than first converting logs into abstract insights.","Since KV cache size depends only on the number of stored tokens and not on the amount of encoded context, a full reasoning trace can be encoded at no extra storage cost; the paper attributes LAGKV's edge over KV-cache baselines to this encode-versus-store distinction.","Retrieving more relevant logs (up to k=3 in the paper) generally improves exact match on the tested datasets, making log-store quality and retrieval depth direct levers on accuracy.","LAGKV reaches a given accuracy in fewer reasoning steps than a standard agent, so the same compute budget can answer more questions or leave room for additional reasoning on harder ones.","Because the store-retrieve-augment loop is described as compatible with any sequential generation workflow, the framework is claimed to extend beyond ReAct-style agents to other LLM generation paradigms."],"supporting_citations":[{"why":"Defines the standard agentic problem-solving framework that LAG builds on and uses as the primary no-log baseline.","marker":"[14]"},{"why":"Implements the reflection-based test-time learning baseline that distills logs into insights and is compared against LAGKV.","marker":"[3]"},{"why":"Is the KV-cache baseline and supplies the block-attention technique for reusing cached KV values with re-rotated positional embeddings.","marker":"[9]"},{"why":"Provides the precomputed-KV-cache and position-rewriting approach for chunked text that the paper adapts from document caching to reasoning traces.","marker":"[7]"},{"why":"Supplies the Musique multi-hop QA benchmark used to evaluate knowledge-intensive reuse.","marker":"[10]"},{"why":"Supplies the 2WikiMultiHop multi-hop QA benchmark used as the second knowledge-intensive evaluation.","marker":"[11]"},{"why":"Supplies the GPQA science benchmark used to evaluate reasoning-intensive reuse.","marker":"[12]"},{"why":"Supplies the MMLU-Pro benchmark used as the second reasoning-intensive evaluation.","marker":"[13]"},{"why":"Is the text-embedding model the log retriever uses to rank stored logs by cosine similarity.","marker":"[15]"}],"fun_headline_variants":["Reuse KV caches from past tasks to boost LLM reasoning","KV-cache reuse beats reflection for agent reasoning accuracy","Log-augmented generation: smarter test-time reasoning by caching","Store past reasoning caches to boost agent accuracy with fewer steps"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a token's stored key-value state from an earlier reasoning trace keeps its meaning when moved into a new question's context; if attention states do not transfer that way, the reported accuracy gains disappear.","fun_headline_variants_meta":{"raw":{"variants":["Reuse KV caches from past tasks to boost LLM reasoning","KV-cache reuse beats reflection for agent reasoning accuracy","Log-augmented generation: smarter test-time reasoning by caching","Store past reasoning caches to boost agent accuracy with fewer steps"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000821,"raw_usage":{"total_tokens":3602,"prompt_tokens":961,"completion_tokens":2641,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":577,"completion_tokens_details":{"reasoning_tokens":2571}},"tokens_in":577,"tokens_out":2641,"duration_ms":60605,"temperature":1.0,"reasoning_tokens":2571,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T15:34:45.660584+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run LAGKV again but replace the retrieved KV logs with KV caches taken from unrelated questions (or with random vectors of the same shape); if exact match stays near the reported LAGKV level, the gain is not coming from reused reasoning, whereas a drop back toward the no-log baseline would confirm the reuse mechanism is doing the work.","supporting_citations":[],"review_version":1}