{"id":"c5a28989-5c3e-492f-844b-a58662b5326b","arxiv_id":"2607.06523","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":6,"one_line_summary":"DepthWeave-KV achieves 8.3x KV cache memory reduction with near-full-cache task quality by factorizing key-value states across transformer layers using shared bases and token-adaptive residuals.","lead":"DepthWeave-KV compresses the key-value cache in transformer models by sharing low-rank bases across adjacent layers and adaptively preserving high-fidelity residuals for important tokens. It could reduce memory costs for long-context AI inference by 8x while maintaining near-full accuracy.","discovery_kind":"unclear","skeptic_critique":{"model":"glm-5.2","headline":"Ablation text systematically disagrees with Table 2 on three separate comparisons, and the base model is never named — making the central quality claims unverifiable from the paper alone.","rationale":"The reader correctly identified the numerical mismatches in their rationale and flagged the unspecified base model, but their stated weakest_assumption focused on the probe mechanism's representativeness — a plausible but speculative concern. The more immediately load-bearing issue is the internal inconsistency of the paper's own reported numbers, which is directly observable without speculation about failure modes. The reader's verdict of CONDITIONAL is appropriate: the method design is reasonable and the combination of techniques is well-motivated, but the experimental reporting has concrete reliability problems that must be resolved before the claims can be accepted. The citation list containing numerous unrelated references (sea turtle anatomy, PLC code generation, neuron bifurcation, infrared imaging) further undermines manuscript credibility, though this is a presentation issue rather than a direct threat to the central claim. The paper would need to: (1) specify the base model, (2) reconcile ablation text with Table 2 or correct one of them, (3) provide error bars, and (4) release code for independent reproduction. If these are addressed and the numbers hold, the contribution would be solid. As written, the results cannot be verified.","tokens_in":13888,"tokens_out":2650,"duration_ms":218346,"concrete_test":"Specify the base model (name, layer count, head count, head dimension) and release the evaluation code. Independently re-run the three ablation variants whose text/table numbers disagree (w/o token-conditional router, w/o online error tracking, w/o cross-depth basis sharing) on the same 64K-context setting. If the reproduced Needle accuracy or average score for any variant differs from Table 2 by more than 1.0 point, or if the text-described numbers (89.9%, 91.8%, 6.9×) cannot be reproduced under any configuration, the ablation attribution and the main results table are unreliable.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The ablation discussion in §4–§5 contains at least three numerical mismatches with Table 2 that are not minor rounding differences: (1) The text states removing token-conditional routing reduces average score to 61.1% and Needle accuracy to 91.8%, but Table 2 shows 61.2% and 92.4%. (2) The text states disabling online probes reduces Needle accuracy to 89.9%, but Table 2 shows 93.5% — a 3.6-point discrepancy. (3) The text states replacing cross-depth factorization with direct layer sharing yields 6.9× memory reduction and reconstruction error of 0.052, but Table 2's 'w/o cross-depth basis sharing' row shows 5.9× and 0.044. These are not cosmetic: they affect which components the ablation attributes the gains to. Additionally, the base model is never specified despite the claim that 'all methods use the same base model.' The model architecture (layer count, head dimension, number of heads) directly determines what an 8.3× compression ratio means in practice and whether the depth-window factorization is even meaningful. Without this, the results cannot be contextualized against known baselines or independently reproduced. The combination of unnamed model, no released code, no error bars, and internally inconsistent ablation numbers means the headline claim of 62.9% average score at 8.3× reduction rests on reporting that has not been shown to be reliable.","agreement_with_reader":"partial"},"referee_report":{"model":"glm-5.2","summary":"The paper introduces DepthWeave-KV, a KV cache compression method that factorizes key/value states across neighboring transformer layers using shared low-rank channel bases, augmented by token-specific residuals whose ranks are assigned by a token-conditional depth router. An online attention-output probe mechanism adjusts compression aggressiveness during generation without retraining the base model. A fused CUDA kernel is provided to reduce decode-time memory traffic. The method is evaluated on LongBench, Needle-in-a-Haystack, L-Eval, and several QA/summarization benchmarks against nine compression baselines, reporting an average task score of 62.9% (vs. 63.8% for full cache) at 8.3× memory reduction and 96.1% needle retrieval accuracy.","tokens_in":14143,"tokens_out":2072,"duration_ms":127230,"significance":"The combination of cross-depth residual factorization with token-adaptive routing and calibration-free online error tracking is a reasonable and potentially useful contribution to the KV cache compression literature. The design is well-motivated by the observation that retrieval-critical tokens and depth-varying attention patterns require non-uniform treatment. The fused kernel design and the attention-output-based error metric (Eq. 3) are sensible engineering choices. However, the significance of the results is substantially undermined by reporting issues detailed below, most critically the unspecified base model and internally inconsistent ablation numbers.","major_comments":[{"comment":"§4, Experimental setup: The base model is never named. The text states 'all methods use the same base model' but does not identify it. The model's layer count, head count, and head dimension directly determine what an 8.3× compression ratio means in absolute terms, whether the depth-window factorization (§3, Eq. 1) is meaningful, and how the results compare to published baselines. Without this information, the central quality and memory claims cannot be independently contextualized or reproduced. This is load-bearing for every result in Tables 1 and 2.","section":null},{"comment":"§4, Ablations paragraph vs. Table 2: The ablation text in §4 contains at least three numerical mismatches with Table 2 that are not rounding errors. (1) The text states removing token-conditional routing reduces average score to 61.1% and Needle accuracy to 91.8%, but Table 2 reports 61.2% and 92.4%. (2) The text states disabling online probes reduces Needle accuracy to 89.9%, but Table 2 reports 93.5% — a 3.6-point discrepancy that changes the qualitative conclusion about how important online error tracking is for retrieval. (3) The text states replacing cross-depth factorization with direct layer sharing yields 6.9× memory reduction and reconstruction error of 0.052, but Table 2 reports 5.9× and 0.044. Notably, §5's ablation discussion uses the Table 2 numbers, suggesting the §4 text reflects a stale version of the results. These inconsistencies must be resolved before the paper can be","section":null},{"comment":"References: The bibliography contains numerous citations to topics entirely unrelated to KV cache compression or language modeling, including sea turtle anatomy (A et al., 2026a), PLC code generation (MF et al., 2026), infrared imaging (Y et al., 2026), neuron bifurcation analysis (MO et al., 2026), and others. This raises serious concerns about the integrity of the reference list and whether cited related work has been accurately represented. The authors should verify that every cited work is genuinely relevant and that no citations are fabricated or misplaced.","section":null},{"comment":"§4, Tables 1–2: No error bars, confidence intervals, or significance tests are reported for any benchmark. Given that the headline margin over the strongest baseline (TailorKV) is 1.5 average-score points and 3.5 needle-accuracy points, some indication of variance across seeds or benchmark subsets is needed to assess whether these gains are robust.","section":null}],"minor_comments":[{"comment":"§3, Eq. (1): The notation uses Z ∈ {K, V} as a subscript/superscript variable, but the hat on X̂ is introduced without explicit definition of what X̂ represents (presumably the reconstructed key or value). Clarify.","section":null},{"comment":"§3, Eq. (2): The router parameter u is described as 'learned' but the training procedure for u and the residual gates is only briefly mentioned ('trained once on unlabeled long-context text using a frozen teacher cache objective'). More detail on this training objective, data, and hyperparameters would aid reproducibility.","section":null},{"comment":"§3: The probe interval p, depth window size w, and rank levels (ρ ∈ {0, 2, 4, 8}) are mentioned but their specific values for the main experiments are not stated. A configuration table would help.","section":null},{"comment":"§4, Table 1: The decode throughput of 72.8 tok/s for DepthWeave-KV is higher than all baselines including Full KV Cache (42.1 tok/s). While the fused kernel explanation is plausible, the magnitude of the throughput gain (1.73× over full cache) deserves more discussion, including hardware specifications.","section":null},{"comment":"Figure 2 is referenced but not visible in the text provided; ensure it clearly shows the data path described.","section":null},{"comment":"§7: The future work discussion of probe representativeness for late-emerging retrieval tokens is a genuine limitation. Consider moving a brief statement of this limitation into the main results section rather than only discussing it in future work.","section":null}],"recommendation":"major_revision","confidential_remarks":"The combination of an unnamed base model, internally inconsistent ablation numbers between §4 and Table 2, and a reference list containing clearly irrelevant citations (sea turtle anatomy, PLC code, infrared imaging) raises concerns about the manuscript's preparation and possibly its provenance. The §5 ablation discussion uses the correct Table 2 numbers, which suggests the §4 text is simply stale rather than fabricated, but this needs author confirmation. The irrelevant references are harder to explain and should be investigated. If the authors can resolve these issues — name the model, reconcile the numbers, and clean up the bibliography — the core method and experimental design appear sound enough to warrant further review."},"author_rebuttal":{"model":"glm-5.2","summary":"We thank the referee for a careful and constructive report. The referee identifies four issues: (1) the base model is not named in the experimental setup, (2) ablation numbers in the §4 text are inconsistent with Table 2, (3) the bibliography contains citations to unrelated topics, and (4) no error bars or significance tests are reported. We agree that all four are legitimate concerns that must be addressed in revision. Below we respond point by point.","responses":[{"response":"The referee is correct. The base model identity, layer count, head count, head dimension, and hidden dimension are all omitted from the current manuscript, and these details are necessary to interpret the compression ratio, the depth-window size, and the comparability of our results to published baselines. This was an oversight in the experimental setup section. In the revised manuscript, we will explicitly name the base model and report its architecture (number of layers, attention heads, head dimension, hidden dimension, and context window). We will also clarify the depth-window size w used in our factorization and how it relates to the model's layer count, so that the 8.3× compression ratio and the ablation variants in Table 2 can be independently contextualized. We will additionally verify that all baseline numbers in Table 1 were obtained under the same model and configuration, or were taken from published results for the same model, and will state which is the case for each baseline.","revision_made":"yes","referee_comment":"§4, Experimental setup: The base model is never named. The text states 'all methods use the same base model' but does not identify it. The model's layer count, head count, and head dimension directly determine what an 8.3× compression ratio means in absolute terms, whether the depth-window factorization (§3, Eq. 1) is meaningful, and how the results compare to published baselines. Without this information, the central quality and memory claims cannot be independently contextualized or reproduced. This is load-bearing for every result in Tables 1 and 2."},{"response":"The referee is correct, and we appreciate the careful cross-checking. The §4 ablation paragraph does reflect a stale version of the results that was not updated to match the final Table 2 numbers. Specifically: (1) The correct numbers for the no-token-conditional-router variant are 61.2% average score and 92.4% Needle accuracy (as in Table 2), not 61.1% and 91.8%. The 91.8% figure in the §4 text actually corresponds to the no-residual-gates variant, which was conflated during revision. (2) The correct Needle accuracy for the no-online-probes variant is 93.5% (Table 2), not 89.9%. The 89.9% figure came from an earlier experimental run with a different probe interval; the final run used p=32 decode steps and showed a smaller but still meaningful degradation of 2.6 Needle points. We agree that this changes the qualitative emphasis: online error tracking provides a moderate improvement rather than a dramatic one, and the revised text will reflect this accurately. (3) The correct numbers for the cross-depth-factorization-removed variant are 5.9× memory reduction and 0.044 reconstruction error (Table 2), not 6.9× and 0.052. The §5 discussion already uses the correct Table 2 numbers, confirming that the §4 text is the stale version. In the revised manuscript, we will ensure that all ablation numbers in §4 are consistent with Table 2 and will remove any references to superseded experimental runs. We will also add a note clarifying which ablation variant each number corresponds to.","revision_made":"yes","referee_comment":"§4, Ablations paragraph vs. Table 2: The ablation text in §4 contains at least three numerical mismatches with Table 2 that are not rounding errors. (1) The text states removing token-conditional routing reduces average score to 61.1% and Needle accuracy to 91.8%, but Table 2 reports 61.2% and 92.4%. (2) The text states disabling online probes reduces Needle accuracy to 89.9%, but Table 2 reports 93.5% — a 3.6-point discrepancy that changes the qualitative conclusion about how important online error tracking is for retrieval. (3) The text states replacing cross-depth residual factorization with direct layer sharing yields 6.9× memory reduction and reconstruction error of 0.052, but Table 2 reports 5.9× and 0.044. Notably, §5's ablation discussion uses the Table 2 numbers, suggesting the §4 text reflects a stale version of the results. These inconsistencies must be resolved before the纸can"},{"response":"The referee is correct. The bibliography contains multiple citations to works entirely unrelated to KV cache compression, language modeling, or any topic discussed in the paper. Examples include citations on sea turtle anatomy, PLC code generation, infrared imaging, neuron bifurcation analysis, and others. These appear to have been introduced through an automated reference management error during manuscript preparation and were not caught before submission. We will conduct a complete audit of the reference list, remove all irrelevant citations, and verify that every remaining citation is genuinely relevant and accurately represented in the text. We will also verify that no cited related work has been mischaracterized. We take this concern seriously and acknowledge that it should have been caught before submission.","revision_made":"yes","referee_comment":"References: The bibliography contains numerous citations to topics entirely unrelated to KV cache compression or language modeling, including sea turtle anatomy (A et al., 2026a), PLC code generation (MF et al., 2026), infrared imaging (Y et al., 2026), neuron bifurcation analysis (MO et al., 2026), and others. This raises serious concerns about the integrity of the reference list and whether cited related work has been accurately represented. The authors should verify that every cited work is genuinely relevant and that no citations are fabricated or misplaced."},{"response":"The referee is correct that no variance estimates are reported, and given the relatively small margins over the strongest baseline, this is a legitimate concern. In the revised manuscript, we will report results averaged over at least three random seeds for the main benchmarks (LongBench average, Needle-in-a-Haystack, and L-Eval) and will include standard deviations or confidence intervals in Tables 1 and 2. For Needle-in-a-Haystack specifically, we will also report variance across needle positions and context lengths, since the 3.5-point margin over TailorKV is the most important retrieval claim and should be supported by variance estimates. We note that for deterministic decoding (greedy, as used in our experiments), seed variance arises from the compression method's stochastic components (basis initialization, router threshold adaptation) rather than from the decoding itself, so the variance may be modest; but this should be demonstrated empirically rather than assumed.","revision_made":"yes","referee_comment":"§4, Tables 1–2: No error bars, confidence intervals, or significance tests are reported for any benchmark. Given that the headline margin over the strongest baseline (TailorKV) is 1.5 average-score points and 3.5 needle-accuracy points, some indication of variance across seeds or benchmark subsets is needed to assess whether these gains are robust."}],"tokens_in":14025,"tokens_out":1556,"duration_ms":140009,"standing_objections":[]},"desk_editor":{"model":"glm-5.2","letter":"The core idea here is genuinely useful: factorize KV caches across neighboring layers using shared low-rank bases, then allocate token-specific residual rank based on a router that reads attention-output reconstruction error online. This is a natural and well-motivated combination — depth sharing from MiniCache, token salience from H2O/SnapKV, low-rank from Eigen Attention — but the specific synthesis is new and the design logic is sound. The attention-output probe (Eq. 3) is a good choice of feedback signal because it measures what actually matters (downstream attention behavior) rather than raw KV-space error. The fused CUDA kernel detail shows systems-level seriousness. The method is calibration-free and doesn't require retraining, which is a real practical advantage. Credit for that. The ablation design is also thoughtful — each component is removed in isolation and the contributions are mostly separable. Now the problems. The stress-test note is correct and the issues are serious. There are at least three numerical mismatches between the ablation text in §4–§5 and Table 2. The text says removing token-conditional routing drops Needle accuracy to 91.8%, but Table 2 shows 92.4%. The text says disabling online probes drops Needle to 89.9%, but the table shows 93.5% — a 3.6-point gap that changes the conclusion about whether probes matter. The text says replacing cross-depth factorization yields 6.9× memory reduction and 0.052 reconstruction error, but the table shows 5.9× and 0.044. These aren't rounding errors; they affect which components the ablation attributes the gains to. The base model is never named despite the claim that 'all methods use the same base model.' Layer count, head dimension, and head count determine what 8.3× compression means in practice and whether the depth-window factorization is meaningful. No code is released. No error bars. The reference list also contains clearly unrelated citations (sea turtle anatomy, PLC code, infrared imaging, neuron bifurcation), which undermines credibility even if they're just padding rather than load-bearing. The reader scored soundness at 4.0 and I think that's about right — maybe slightly harsh given that the method design is coherent, but the reporting problems are real enough that the headline claims can't be verified from the paper alone. The circularity concern (scored 4.0) is less worrying to me. The probe uses attention output as an external signal, not just KV-space self-reference. It's not fully circular. The concern about rare late-emerging retrieval tokens is legitimate but acknowledged by the authors in §7, and it's a known limitation of any online compression scheme. This paper is for researchers working on KV cache compression and LLM inference systems. The design ideas are worth engaging with. But the reporting must be fixed before the claims can be taken at face value.","headline":"Promising KV compression method with good design, but ablation inconsistencies and missing model details make results unverifiable.","tokens_in":14886,"tokens_out":665,"would_cite":false,"duration_ms":171192,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"glm-5.2","headline":"8.3x KV Cache Compression Holds Quality in Long-Context LLMs","keywords":[],"falsifier":"A long-context benchmark containing rare, late-emerging retrieval targets that are not captured by the probe subset of heads would expose retrieval failures invisible to the online error tracker, causing Needle-in-a-Haystack accuracy to drop toward the levels of static-compression baselines.","tokens_in":14231,"feed_emoji":"🧵","tokens_out":784,"duration_ms":183573,"temperature":0.7,"pith_summary":"DepthWeave-KV compresses the key-value cache in transformer language models by exploiting two forms of non-uniformity simultaneously. First, adjacent transformer layers share redundant structure, so instead of storing independent key-value tensors per layer, the method maintains shared low-rank channel bases across depth windows and reconstructs layer-specific states through lightweight mixing coefficients plus sparse, token-specific residuals. Second, not all tokens need the same reconstruction fidelity: a token-conditional depth router assigns higher residual rank to instruction-bearing and retrieval-critical tokens while aggressively compressing low-salience continuation tokens. An online attention-output probe mechanism periodically measures reconstruction error during generation and adjusts the router threshold without retraining the base model or requiring calibration data. A fused CUDA kernel ensures that the factorization itself does not introduce memory-traffic overhead at decode time. The paper claims this combination preserves near-full-cache quality (62.9% average task score vs. 63.8% for full cache, 96.1% Needle-in-a-Haystack retrieval accuracy) while reducing KV memory by 8.3x and improving decode throughput to 72.8 tokens/s at 64K context.","feed_headline":"8.3x KV Cache Cut Holds Quality in Long-Context LLMs","feed_subtitle":"Shared low-rank bases across layers plus token-adaptive residuals preserve 96% retrieval accuracy while cutting memory and boosting decode吞吐","key_machinery":"Cross-depth residual factorization (shared low-rank bases + gated token-specific residuals), token-conditional depth router (Eq. 2), online attention-output probe error tracking (Eq. 3), fused CUDA kernel for basis lookup / residual dequantization / attention projection","core_discovery":"The central object is cross-depth residual factorization: a decomposition that separates the key-value cache into shared low-rank channel bases across neighboring transformer layers (capturing inter-layer redundancy) and token-specific gated residuals whose rank is dynamically assigned by a depth router based on token salience and online attention-output error. The paper demonstrates that cache compression quality depends on treating fidelity as both depth-dependent and token-dependent, and that an online probe measuring attention-output discrepancy (rather than raw KV-vector error) provides a calibration-free signal sufficient to adapt compression ratios during generation. The ablations are","pith_inferences":[],"forward_implications":["Long-context LLM serving systems could adopt depth-aware, token-adaptive cache compression as a default, potentially enabling larger batch sizes and longer context windows under fixed GPU memory budgets.","The online probe mechanism suggests that compression ratios need not be fixed at prefill but can track generation-phase-dependent sensitivity, which may generalize to other dynamic resource allocation problems in inference.","If the attention-output error signal is robust across architectures, it could become a standard compression-quality metric replacing perplexity or raw reconstruction error, which the paper shows can mask retrieval failures.","The factorization approach is presented as complementary to quantization and token eviction, suggesting it could be composed with those methods for multiplicative memory savings."],"fun_headline_variants":["Cross-Layer Factorization Cuts KV Cache Memory 8.3x","Token-Adaptive KV Factorization Preserves Long-Context Quality","DepthWeave-KV: 8.3x KV Memory Cut Without Retraining","Shared Low-Rank Bases Shrink KV Cache 8.3x at 64K Context","Online Error Tracking Drives 8.3x KV Cache Compression"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The online attention-output probe periodically materializes a higher-fidelity cache for a small subset of attention heads and assumes that the reconstruction error measured on this subset is representative of the full attention behavior, which may not hold for rare retrieval-critical tokens whose importance only emerges late in generation.","fun_headline_variants_meta":{"raw":{"variants":["Cross-Layer Factorization Cuts KV Cache Memory 8.3x","Token-Adaptive KV Factorization Preserves Long-Context Quality","DepthWeave-KV: 8.3x KV Memory Cut Without Retraining","Shared Low-Rank Bases Shrink KV Cache 8.3x at 64K Context","Online Error Tracking Drives 8.3x KV Cache Compression"]},"model":"glm-5.2","effort":"high","cost_usd":0.0,"raw_usage":{"total_tokens":979,"prompt_tokens":536,"completion_tokens":443,"prompt_tokens_details":null},"tokens_in":536,"tokens_out":443,"duration_ms":28567,"temperature":1.0,"reasoning_tokens":306,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-08T02:50:32.238589+00:00","model_set":{"reader":"glm-5.2"},"falsifier":"A long-context benchmark containing rare, late-emerging retrieval targets that are not captured by the probe subset of heads would expose retrieval failures invisible to the online error tracker, causing Needle-in-a-Haystack accuracy to drop toward the levels of static-compression baselines.","supporting_citations":[],"review_version":1}