{"id":"2f36a77a-0925-423d-9e32-8f1be27ba84e","arxiv_id":"2608.07855","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":10,"one_line_summary":"AI agents that use tools remember each past step as cached tokens; CommitKV deletes a chunk only when its influence drops from high before a tool call to low after the observation returns.","lead":"The paper presents CommitKV, a way to compress the memory that AI agents keep about past tool use: it measures how much each chunk of history matters right before a tool call and right after the tool's answer arrives, and removes only chunks whose importance clearly drops. If the reported results hold, long tool-using AI agents could run faster and in less memory without losing accuracy.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The W=8 post-commit window can label a page as 'completed' when it is merely not used in the first eight next-turn tokens; the measurement layer is also unspecified.","rationale":"The paper's core mechanism is the commit transition, and the strongest claim is that this transition identifies pages whose role is complete. The ablations support that the lifecycle mechanism helps average accuracy, but they do not test the semantic content of the high-to-low label. The W=8 post-commit window is the only place where 'completion' is observed, so if a page is needed later but not attended in those eight tokens, it will be retired. The joint validation in Eq. (11) cannot catch this because it only measures the immediate deletion effect of the accepted set. This is a correctness risk in the central argument, not a disagreement with prior work or a stylistic complaint. The unspecified attention layer adds a second uncontrolled degree of freedom: the deletion effect R(E;Q) is the maximum over heads at one layer, and the choice of layer can change which pages are labeled high or low. The proposed test directly checks whether retired pages are needed later, which is the operational meaning of the central claim. The paper has real strengths: the paired pre/post measurement is a novel idea, the ablation table shows each component contributes, and the memory and latency gains are plausible. Those strengths do not resolve the window problem, because high accuracy on averages can coexist with occasional eviction of pages that the final answer later needs. I did not find an internal mathematical error in the deletion-effect formula; the issue is whether the measured signal means what the paper says it means. Since the reader already identified the W=8 window and the layer choice as the weakest assumption, I agree with that assessment and would keep the CONDITIONAL verdict: the paper needs the replay test, a specified layer, and ideally a sensitivity analysis of W before the central claim can be accepted.","tokens_in":16176,"tokens_out":7280,"duration_ms":82526,"concrete_test":"Replay CommitKV's labeling on FullKV trajectories for the three main backbones at the 4096-token budget, recording attention outputs for all layers and all query positions. For every page classified as a completion candidate and retired, compute its deletion effect over all subsequent query positions in the same trajectory, not just Q_c^+. If more than 5% of retired pages have a later deletion effect above tau_use = 0.05, the W=8 window is evicting genuinely dormant pages and the central claim is falsified. As part of the same replay, rerun the labeling with each attention layer as the measurement layer and report whether the retirement set changes materially; if it does, the unspecified layer choice is a confound.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim depends on the commit transition of Eqs. (5)-(10), but Q_c^+ contains only the first W=8 decoding queries after the observation is appended (Section 3.1). A page whose role resumes later in the same turn, such as when the agent formulates the final answer or a subsequent tool call, has a low post-commit deletion effect over Q_c^+ and, if its pre-commit effect was high, is classified as a completion candidate and retired after joint validation. The joint test in Eq. (11) only bounds the immediate post-commit effect of the combined retirement set; it does not check later queries or later turns. Thus the measured high-to-low transition does not establish that the page has 'completed its observed role'; it only establishes that the page is not attended in the first eight tokens after the observation. Additionally, Section 3.2 says the deletion effect is computed from one attention layer but does not specify which layer; attention distributions differ sharply across layers, so the high/low labels may be an artifact of a particular layer. Both issues make the abstract's claim that CommitKV can distinguish dormant from completed information unsupported without further evidence.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces CommitKV, a lifecycle-aware KV cache compression method for multi-turn ReAct agents. It partitions completed tool calls and returned observations into event pages, measures each page's deletion effect in a pre-commit query window (the last W decoding queries of the tool call) and a post-commit window (the first W queries of the next turn), and uses the paired high-to-low transition as evidence that the page's role is complete. Completion candidates are then accepted into a retirement set via a greedy joint-validation test that bounds the combined post-commit deletion effect; pages awaiting post-commit measurement are temporarily protected within a reserved budget. At compression checkpoints, retired pages are excluded, protected pages are retained, and the remaining resident tokens are selected under a token budget, with the same index set applied to keys, values, and absolute positions. The method is evaluated on three main LLM backbones (with three additional backbones reported in the appendix) across eight benchmarks and cache budgets of 2048, 4096, and 8192 tokens, reporting accuracy, peak KV memory, and end-to-end latency, along with ablations against five reduced variants.","tokens_in":16379,"tokens_out":3362,"duration_ms":40066,"significance":"If the claims hold, CommitKV addresses a real limitation of snapshot-based KV eviction for agentic workloads: low current attention does not distinguish temporarily dormant information from information whose role has finished. The page-level paired deletion-effect measurement and the joint validation step are a sensible and non-obvious design, and the paper gives credit to a concrete mechanism rather than an abstract intuition. The ablations in Table 3 provide initial evidence that each component contributes to the reported accuracy. The main-table results show large gains over strong baselines under aggressive budgets, and the memory/latency figures indicate practical benefit. However, the paper's central scientific claim—that CommitKV can distinguish dormant from completed information—is stronger than what the current measurements establish, and the missing specification of the fixed-budget selection rule makes the method incompletely defined. The lack of statistical uncertainty, code release, validation-set discipline for the many thresholds, and sensitivity analysis further limits the strength of the empirical evidence.","major_comments":[{"comment":"The selection of the retained-token index set I_j is never specified. Eq. (13) only states the constraints N_j ⊆ I_j and |I_j| ≤ B, and Algorithm 1 line 22 says to 'select I_j' without defining the ordering, scoring, or tie-breaking rule for the resident tokens that are neither retired nor protected. This is load-bearing because the actual cache contents, and therefore the accuracy and memory results, depend entirely on how the remaining budget is filled. Without this rule, the method is not reproducible and cannot be compared fairly with the baselines, which do specify their selection criteria. Please provide the exact selection algorithm, and clarify whether that selection is part of CommitKV or an inherited baseline mechanism.","section":"§3.3, Eq. (13) and Algorithm 1, line 22"},{"comment":"The deletion effect R(E; Q) is computed from 'one attention layer', but the paper never identifies which layer is used. Attention distributions differ sharply across layers, so the high/low labels in Eq. (9) and the resulting retirement decisions may be artifacts of an arbitrary layer choice. Please state the layer selection rule and justify it, ideally with an ablation across layers or with an aggregation over layers that is invariant to the choice.","section":"§3.2, Eqs. (6)–(7)"},{"comment":"The post-commit window Q_c^+ contains only the first W=8 decoding queries after the returned observation is appended. The joint validation in Eq. (11) bounds only the combined post-commit deletion effect over that same window. A page whose role resumes after the first eight tokens of the next turn, or in a later turn, will have a low measured R_c^+(E) and, if its pre-commit effect was high, will be classified as a completion candidate and retired. Thus the measured high-to-low transition does not establish that the page has 'completed its observed role'; it establishes only that the page is not attended in the first eight post-commit queries. The abstract's central claim is therefore stronger than what the current measurement supports. Please either temper the claim or add direct evidence that pages retired by this rule are not needed later, for example by measuring the deletion effect over longer post-commit windows or by tracking whether retired pages would have been attended in subsequent turns.","section":"§3.1, Eq. (5) and §3.3, Eq. (11)"},{"comment":"The method introduces at least six thresholds (τ_use, τ_dead, τ_joint, ρ_use, ρ_dead, η) plus W, G, the scan cap, and the pending cap, and the paper fixes them without any validation split or sensitivity analysis. The reported accuracy gains could in part encode tuning to the test benchmarks. Please report how these values were chosen, add a sensitivity study over the most influential thresholds, and, if possible, evaluate on a held-out validation set or report cross-benchmark stability.","section":"§4.1 Implementation Details"},{"comment":"All accuracy, memory, and latency numbers are reported as single point estimates without error bars, repeated seeds, or significance tests. Given the small per-dataset sample sizes and the fact that several reported differences are only a few percentage points, this makes it hard to judge whether CommitKV's advantage over the strongest baselines is reliable. Please provide variance estimates or confidence intervals, and release the evaluation code and prompts to make the results reproducible.","section":"§4.1–§4.4, Tables 1–3 and Figure 3"}],"minor_comments":[{"comment":"Several entries in the Qwen3.6-27B block are malformed, e.g., '74.1122.0029.29' in the R-KV row and 'R-KV 10.924.8562.69' in the DeepSeek row; these need to be split into separate percentages.","section":"Table 1"},{"comment":"The line says to compute R_ct-1^+(E) and P_ct-1^±(E), but P^- was already computed at the commit; please clarify which quantities are newly computed at this step and which are reused.","section":"Algorithm 1, line 5"},{"comment":"The text repeatedly refers to an 'Appendix' for complete protocol and additional backbones, but the submission as provided does not include that appendix; please ensure the appendix is present in the final version.","section":"§4.1"},{"comment":"The bar chart reports single-run wall-clock times; adding error bars or a note on measurement variance would make the speedup claims more credible.","section":"Figure 3"},{"comment":"The phrase 'ensure that CommitKV can distinguish dormant information from information that has completed its observed role' is too strong given the W=8 window and the missing fixed-budget selection rule; a more cautious formulation would better match the evidence.","section":"Abstract and §5"}],"recommendation":"major_revision","confidential_remarks":"The paper presents an interesting and potentially useful idea, and the ablation study is a strength. However, the missing fixed-budget selection rule is a genuine load-bearing gap: without it the method is not fully specified, and the empirical comparison may partly reflect an unspecified selection mechanism rather than the lifecycle signal. The W=8 issue also means the headline claim should be softened unless additional evidence is supplied. These are fixable within the manuscript's scope, so I support a major revision rather than rejection. I also note that the related-work section cites several agent-specific KV management papers (e.g., IntentKV, AgentKV, MemDecay) but does not compare against them experimentally; adding at least one such baseline would strengthen the positioning."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The commit-transition idea is genuinely new. Instead of scoring tokens by snapshot attention, CommitKV measures the deletion effect of each event page right before a tool-call commit and right after the returned observation is incorporated. A page that goes high-to-low across that boundary becomes a completion candidate; low-to-low pages are kept as dormant. That paired measurement is not in SnapKV, R-KV, or TriAttention, and it is a sensible way to separate 'done' from 'dormant.' The paper also has a nice safety mechanism: the greedy joint validation prevents retiring a set whose combined deletion would exceed a threshold, and pending pages are protected until their post-commit measurement exists. The ablations show each piece helps.\n\nThe main gap is fixed-budget selection. Equation (13) and Algorithm 1 say 'select I_j' with the pending indices retained and |I_j| <= B, but never specify how the remaining tokens are chosen. That is load-bearing: the method ends at a step that is undefined. A reader cannot reproduce or evaluate the approach without knowing whether the rest is filled by attention scores, random sampling, or something else.\n\nThe stress-test point about the W=8 window is fair. The post-commit effect is measured only over the first eight next-turn queries. A page that matters when the agent composes the final answer later in the same turn would look low in that window and could be misclassified as completed. The joint test only bounds the immediate post-commit effect of the retirement set; it does not check later queries or later turns. So the abstract's 'distinguish dormant from completed' claim is stronger than the evidence. This is a limitation, not a disproof, but the authors should either broaden the window, run a sensitivity study on W, or soften the claim.\n\nTwo smaller issues: the paper does not say which attention layer the deletion effect uses, and there are no error bars or significance tests. Some gains over baselines are small (about 1.5 points on the DeepSeek model), and without variance estimates I would not trust those to generalize. The paper also cites agent-specific methods (AgentKV, IntentKV, MemDecay, SideQuest) but never compares against them; that is a major omission given the paper's stated niche.\n\nNet: the idea deserves a serious referee. Send it out, but expect heavy revision. The missing selection rule and the missing baselines need to be fixed before I would cite it.","headline":"The commit-transition idea is new and worth engaging, but the paper is not reproducible yet because the fixed-budget selection step is undefined and the empirical claims lack error bars.","tokens_in":16946,"tokens_out":3511,"would_cite":false,"duration_ms":37969,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A cache-compression method that retires tool-call pages only after their observed role ends keeps agent accuracy near full-cache levels.","keywords":["KV cache compression","multi-turn agents","ReAct","lifecycle-aware eviction","commit transitions","deletion effect","tool calling","LLM inference"],"falsifier":"Run a trajectory where a tool-call page has low deletion effect over the first 8 post-commit queries but is needed by query 9 or in a later turn, for example the model re-reads the tool arguments after the observation; if CommitKV retires that page and the final answer's accuracy drops below FullKV while a variant that protects it does not, the lifecycle signal is not sufficient.","tokens_in":15936,"feed_emoji":"⚙️","tokens_out":6060,"duration_ms":58021,"temperature":0.7,"pith_summary":"CommitKV is a method for compressing the key-value cache of multi-turn ReAct agents that decides what to evict from a page's lifecycle, not from its current attention score. It measures how much the attention output would change if a page of tool-call or observation tokens were deleted, once just before the tool-call commit and once right after the returned observation is processed. A page whose deletion effect drops from high to low across that boundary is treated as having completed its role and becomes a candidate for retirement, while a low-to-low page is treated as dormant and kept. Retirement candidates are only removed after a joint validation bounds the cumulative effect of deleting them together. The paper reports that this keeps agent accuracy close to the uncompressed cache while cutting peak KV memory by up to 5.00x and speeding up end-to-end inference by up to 5.62x.","feed_headline":"CommitKV cuts agent KV memory 5x by retiring finished pages","feed_subtitle":"Comparing page influence before and after tool-call commits keeps dormant context while trimming memory up to 5.00x.","key_machinery":"The commit transition is the central mechanism: for each tool call, the deletion effect of the same event page is measured in the pre-commit window (last W decode queries of the tool call) and the post-commit window (first W next-turn queries after the observation is appended). Deletion effect R(E;Q) is the maximum, over query positions and attention heads, of the relative change in an attention head's output when the page's tokens are deleted and remaining attention weights renormalized. Comparing the paired measurements through absolute thresholds and percentile ranks yields the lifecycle state that separates dormant from completed pages.","core_discovery":"The central claim is that snapshot-based importance scores conflate two situations that lifecycle signals separate: an event page may be temporarily dormant but needed later, or it may have finished its observed role after the tool's observation arrives. CommitKV establishes this separation by computing a deletion effect R(E;Q) for each eligible page over paired query windows on either side of a tool-call commit, combining absolute effect with percentile rank so both large-but-common and small-but-rare effects are labeled consistently. The lifecycle state lambda_c(E) classifies the page as completion candidate (high before, low after), dormant (low before, low after), newly active, still active, or uncertain, and only completion candidates proceed to a greedy joint test. A page is retired only if its own post-commit effect and the cumulative effect of already accepted pages stay below a threshold, and pages awaiting post-commit measurement are protected within a reserved budget. The authors argue this makes eviction decisions depend on whether information has completed its observed role rather than on whether it looks important right now.","pith_inferences":["A possible extension would make the post-commit window adaptive, re-measuring pages that are still low after W=8 queries at later checkpoints, so pages that become important later are not retired prematurely.","The deletion effect is computed on one attention layer with unspecified layer choice; averaging the high-to-low signal across layers or selecting the layer that best predicts held-out accuracy would likely make lifecycle labels more robust.","The commit-transition idea could generalize beyond tool calls to any agent or program state with an explicit commit boundary, such as code execution or document editing, where earlier context has served its purpose.","A testable prediction is that the accuracy benefit of CommitKV over snapshot baselines grows with the number of tool calls per trajectory and shrinks on single-turn reasoning tasks, consistent with the paper's tool/evidence versus reasoning benchmark split."],"forward_implications":["With lifecycle-aware retirement, agent accuracy stays close to FullKV while using a fixed token budget, because dormant pages are kept and only role-completed pages are removed.","Snapshot-based baselines lose accuracy on tool-heavy benchmarks because they cannot distinguish dormant from completed pages; CommitKV's gap over the strongest baseline is up to 22.24 percentage points.","Retiring pages only after joint validation and protecting pending pages keeps the cache within budget without sacrificing the information needed to make future lifecycle decisions.","The method is robust across cache budgets of 2048, 4096, and 8192 tokens and across three tested backbones, with the largest gains at the most aggressive budget."],"supporting_citations":[{"why":"Defines the ReAct paradigm of alternating reasoning, tool calls, and observations that CommitKV operates on.","marker":"[Yao et al., 2023]"},{"why":"SnapKV supplies the window-based importance estimation that inspired CommitKV's paired query windows.","marker":"[Li et al., 2024]"},{"why":"R-KV is a redundancy-aware compression baseline that CommitKV must beat.","marker":"[Cai et al., 2025a]"},{"why":"TriAttention is a geometry-based compression baseline that CommitKV must beat.","marker":"[Mao et al., 2026]"},{"why":"RoPE explains why absolute positions must be preserved when compressing the KV cache.","marker":"[Su et al., 2024]"},{"why":"Attention mechanism provides the output formula used to define the deletion effect.","marker":"[Vaswani et al., 2017]"}],"fun_headline_variants":["CommitKV tells dormant KV pages from done ones, cutting memory 5x","Retire only done KV pages: CommitKV uses commit transitions for 5x memory cut","CommitKV: 5x less KV memory by knowing which agent turns are truly done","Distinguish dormant KV from retired: CommitKV compresses agent memory 5x","CommitKV retires finished KV pages, keeps dormant ones, cuts memory 5x"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that a page whose deletion effect stays low during the first W=8 next-turn queries after the returned observation has finished its role, so a page that only becomes important later would be wrongly retired.","fun_headline_variants_meta":{"raw":{"variants":["CommitKV tells dormant KV pages from done ones, cutting memory 5x","Retire only done KV pages: CommitKV uses commit transitions for 5x memory cut","CommitKV: 5x less KV memory by knowing which agent turns are truly done","Distinguish dormant KV from retired: CommitKV compresses agent memory 5x","CommitKV retires finished KV pages, keeps dormant ones, cuts memory 5x"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000828,"raw_usage":{"total_tokens":3670,"prompt_tokens":1046,"completion_tokens":2624,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":662,"completion_tokens_details":{"reasoning_tokens":2515}},"tokens_in":662,"tokens_out":2624,"duration_ms":17850,"temperature":1.0,"reasoning_tokens":2515,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T00:46:01.092370+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a trajectory where a tool-call page has low deletion effect over the first 8 post-commit queries but is needed by query 9 or in a later turn, for example the model re-reads the tool arguments after the observation; if CommitKV retires that page and the final answer's accuracy drops below FullKV while a variant that protects it does not, the lifecycle signal is not sufficient.","supporting_citations":[],"review_version":1}