{"id":"a2b7e88d-9682-4f5c-b25b-5f86e5915c09","arxiv_id":"2608.07585","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":9,"one_line_summary":"LAVE lets video agents reuse pre-verbal visual hidden states from past tool calls via timestamp-aligned residual injection, improving Video-MME by 3.76 points over the strongest baseline without extra training or frames.","lead":"Video tool-use agents usually communicate with their planner through text, so visual details left out of a tool's summary are lost. This paper stores the tool's internal visual states, timestamps them, and re-injects the useful ones into the planner's hidden states, gaining 3.76 points on Video-MME without extra frames or training.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 3.76-point headline may be a test-set selection artifact: the communication block, evidence budget, residual gain, and temporal coverage are chosen on the full Video-MME test set from single deterministic runs, with no held-out validation or released code.","rationale":"I read the paper in good faith. LAVE is carefully specified: the read-then-write ordering in Appendix A prevents circular dependency, the matched no-latent reference controls for the agent loop, and the Gaussian noise control in Table 2 is strong evidence that the gains are not caused by arbitrary hidden-state perturbation. The ablation structure is sensible, and the paired case study is a useful process-level illustration. However, the single most load-bearing condition for the central claim is that the reported 3.76-point improvement is a valid estimate of what the method achieves, not the best outcome of a small hyperparameter search on the same test set. The paper does not describe a validation split, reports one seed, and provides no code, so the numerical claim cannot currently be independently reproduced or separated from selection effects. The reader's weakest assumption, semantic alignment of same-block hidden states, is a genuine theoretical gap, but it is not the most immediate threat: if the empirical gain is robust on held-out data, LAVE would still be a useful contribution even if the retrieval similarity is only imperfectly aligned, and if the empirical gain is not robust, the method fails regardless of how well the states align. For this reason I focus on the validation issue. I do not consider the paper deceptive; the concern is about the evidential strength of the reported number. The reader's conditional verdict already reflects this uncertainty, so I recommend keeping the verdict unchanged rather than moving it. My agreement is partial because the reader identified a different weakest assumption in the structured verdict, although the rationale does mention the single-run and test-tuning issues.","tokens_in":23110,"tokens_out":8081,"duration_ms":87871,"concrete_test":"Partition Video-MME into a tuning split and a held-out evaluation split, stratified by duration and content type. Run the complete ablation grid from Tables 2 and 4 on the tuning split only, select the best configuration, freeze it, and evaluate it once on the held-out split alongside the no-latent VideoSeek baseline under the same frame budget. If the held-out LAVE-minus-baseline gain is materially smaller than the reported 3.76 points, or if its bootstrap confidence interval includes zero, the headline gain is likely a test-set selection artifact. As a secondary check, also report the frozen configuration on LongVideoBench and CG-Bench without any further tuning; because the current gains on those benchmarks were obtained after Video-MME-based hyperparameter selection, they are not independent evidence unless the configuration is frozen before any benchmark is consulted.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is empirical: under a comparable frame budget, LAVE improves Video-MME by 3.76 points over VideoSeek. For this claim to be trustworthy, the reported configuration must not be the best cell of a grid searched on the same benchmark that later supplies the headline number. But that is exactly what the paper does. Table 2 selects the communication block (15, 19, 23, 27) by Video-MME test accuracy; Table 4 sweeps evidence tokens B in {4, 8, 16}, residual gain g in {0.5, 1.0, 2.0}, and temporal coverage (fixed 2, 4, 8 groups versus entropy-adaptive), again reporting Video-MME test accuracy. The main result, 62.77%, uses the best settings from these sweeps (Block 19, B=8, g=1.0, entropy-adaptive), plus the additional fixed hyperparameters in Table 8. With roughly 900 Video-MME questions, taking the maximum over about fifteen settings can inflate accuracy by several points by chance. All runs are deterministic, with one seed and no error bars, and the code is not released. The semantic-alignment concern raised by the reader is real but secondary: even a perfectly aligned retrieval mechanism would not establish the headline if the number is a selection artifact. Conversely, the paper's Gaussian noise control and matched no-latent reference are good evidence that the method is not just random perturbation, so the concern is specifically that the magnitude of the gain is unvalidated, not that the mechanism is fake.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper identifies what it calls the Tool observation bottleneck in video tool-use agents: after a visual Tool call, only a filtered textual observation reaches the Planner, so unverbalized visual evidence is discarded. It proposes LAVE, a training-free framework that captures hidden-state block updates ('pre-verbal visual evidence') during Tool prefills, stores them as (key, value, source-address) triples with timestamps and visual locations, and injects bounded residuals into the Planner's hidden states at timestamp-matched receiver anchors. Retrieval is driven by cosine relevance between the Planner query and Tool keys, a text-redundancy penalty, and an entropy-constrained temporal routing that selects source-time groups. Experiments on Video-MME, LongVideoBench, and CG-Bench with two backbones report gains over a matched no-latent VideoSeek baseline, with the headline result a 3.76-point Video-MME improvement (62.77 vs 59.01) at nearly identical frame counts.","tokens_in":23396,"tokens_out":5432,"duration_ms":48851,"significance":"If the headline gain is robust, LAVE would be a valuable training-free contribution: it reuses visual computation from completed Tool calls without frame replay or additional training, and it comes with an unusually detailed appendix containing exact equations, a matched no-latent reference, a Gaussian-noise control, and a paired zero-residual case study. These are genuine strengths: the controlled comparison differs only in whether latent evidence is routed, and the Gaussian control shows that structured value directions matter relative to norm-matched random perturbations. However, the quantitative claim is not yet trustworthy because key hyperparameters are selected directly on the benchmark that supplies the headline number, and the abstract's 'consistently improves across backbones' claim is contradicted by one reported cell. The core idea is promising and within scope, but the evaluation needs tightening before the central empirical claim can be accepted.","major_comments":[{"comment":"The headline 3.76-point gain is a selection artifact risk because the hyperparameters are chosen by accuracy on the full Video-MME test set. Table 2 selects the communication block ℓ from {15, 19, 23, 27}; Table 4 selects the evidence token budget B from {4, 8, 16}, residual gain g from {0.5, 1.0, 2.0}, and temporal coverage (fixed 2/4/8 groups vs. entropy-adaptive), all by Video-MME overall accuracy. All runs are deterministic with one seed and no held-out split. With roughly 900 questions, taking the maximum over about fifteen configurations can inflate the reported margin by several points by chance. The paper must either select hyperparameters on a validation set and then report test performance once, or provide multiple seeds with standard deviations and show that the reported settings are not the best cell of a grid searched on the test set. Without this, the abstract's central claim 'LAVE improves the Video-MME overall score by 3.76 points' is not an evidence-based estimate.","section":"Experiments, Tables 2 and 4; Eq. (23)"},{"comment":"The mechanism's grounding is weakened by the paper's own admission that same-block states are not guaranteed to be semantically aligned. The retrieval, utility scoring, and entropy routing all rely on cosine similarity between Tool-side block-19 keys and Planner-side block-19 queries (Eqs. 16, 19, 20, 23). The Gaussian-noise control in Appendix B shows that structured values outperform norm-matched random perturbations, but it does not test whether retrieval actually selects evidence by semantic content. Direct evidence is needed, for example comparing retrieval against an oracle based on frame captions or a small labeled relevance set, or probing whether same-block states form semantic clusters. Without such a test, the interpretation 'latent visual evidence reuse' remains an assumption, and the observed gain could be a side effect of a generic hidden-state intervention rather than content-based retrieval.","section":"Appendix A, after Eq. (15)"},{"comment":"The abstract claims that 'LAVE consistently improves video tool-use agents across backbones' through experiments on Video-MME, LongVideoBench, and CG-Bench. This literal claim is contradicted by Table 1: for GLM-4.6V-Flash-9B, LAVE scores 72.02 on CG-Bench whereas the VideoSeek baseline scores 72.36, a 0.34-point regression. The consistency claim should be qualified to the benchmarks where gains are actually observed, and the CG-Bench regression should be discussed in the main text.","section":"Abstract and Table 1"}],"minor_comments":[{"comment":"The symbol p_tj is used for two different objects: the group probability in Eq. (21) and the receiver anchor in the main text and Figure 3. Appendix A uses π_tj for the anchor, which resolves the ambiguity only within the appendix. Please use a distinct symbol for the receiver anchor throughout the main text.","section":"Method, Eq. (7) and Appendix A"},{"comment":"The caption says 'Baseline (dashed) L15 L19 L23 L27' but the figure itself does not label which curve is the baseline. Please add a legend or explicit labels to each curve.","section":"Figure 4"},{"comment":"There is a formatting artifact in the GLM-4.6V-Flash-9B + VideoSeek row: '55.6968.22' should be two separate numbers. Please check the table for similar column misalignments.","section":"Table 1"},{"comment":"The limitations section mentions 'unquantified memory and latency overhead.' Since the paper's selling point is comparable frame budget, please add a brief quantification of wall-clock time or memory overhead in the appendix to give readers a practical sense of the cost.","section":"Appendix B, Limitations"}],"recommendation":"major_revision","confidential_remarks":"The decisive issue is test-set hyperparameter selection. If the authors can supply a validation-based protocol or repeated-seed error bars, the central empirical claim becomes credible. The semantic-alignment concern is real but addressable with an additional analysis section and does not require retraining. The paper fits the journal's scope and has unusually good process documentation; my recommendation to revise rather than reject is based on the fixability of the evaluation issue."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know. First, the core idea is real: storing pre-verbal visual block updates from completed tool calls and injecting them into the planner's hidden state at timestamped anchors is a sensible, training-free way around the lossy text bottleneck, and the paper builds it carefully. Second, the headline number—62.77 vs 59.01 on Video-MME—is probably inflated by test-set selection. Tables 2 and 4 pick the communication block, evidence budget, residual gain, and temporal coverage by looking at Video-MME accuracy on the same split that produces the headline. That doesn't make the mechanism fake; it makes the magnitude unvalidated.\n\nWhat's genuinely good: the controlled comparison is unusually disciplined. The no-latent condition shares prompts, tools, frame budgets, decoding seed, and everything else; only latent routing differs. The Gaussian noise control shows structured value directions matter, not just perturbation. The appendix gives exact equations, including a closed-form solution for the entropy coverage constraint, and it clearly flags that same-block cosine similarity doesn't guarantee semantic alignment. The paired case study is process-level evidence, not proof, but it's honest about being only that.\n\nSoft spots, in order of weight. (1) Selection on the test set. Block 19, B=8, g=1.0, entropy-adaptive are all chosen by Video-MME overall score, and all runs are single-seed, deterministic, with no error bars and no code. With roughly 900 questions and around fifteen settings, the max can move several points by chance. The paper states final answering is text-only, but that doesn't address selection. (2) The retrieval premise. Appendix A says explicitly that same-block states don't guarantee semantic alignment; cosine between planner query and stored keys may be retrieving something useful, but the grounding is indirect. (3) The relation to \"Latent Visual Cache for Video Reasoning\" is cited as a baseline but never discussed; given the closeness, that's a gap. Overhead is acknowledged as unquantified, which is fair, though I'd like the latency and memory numbers.\n\nMy take: the framework is worth engaging and deserves a serious referee, but the empirical claim needs re-evaluation on held-out validation, with multiple seeds and released code, before I'd trust the 3.76 points. I wouldn't cite the number yet. It's a conditional accept with the conditions stated, not a rejection.","headline":"The latent-channel idea is real and carefully built, but the headline gain is likely inflated by hyperparameter selection on the same test set that produces it.","tokens_in":24036,"tokens_out":2879,"would_cite":false,"duration_ms":27755,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that video tool-use agents can reuse pre-verbal visual evidence from completed tool calls as timestamp-addressed latent residuals in the planner's hidden state, improving accuracy without training or frame replay.","keywords":["video tool-use agents","latent visual evidence","tool observation bottleneck","hidden-state reuse","training-free","entropy-constrained routing","long-video understanding","planner hidden-state update"],"falsifier":"A concrete falsifier is a video question where the Tool's textual observation deliberately omits the deciding visual detail: run LAVE with the same keys and timestamps but replace every stored value vector with norm-matched random noise; if the planner still picks the same Tool trajectory and the accuracy holds, the gains are perturbation artifacts, while a sharp accuracy drop would confirm that the specific visual content is being carried.","tokens_in":22821,"feed_emoji":"🎬","tokens_out":8117,"duration_ms":68483,"temperature":0.7,"pith_summary":"Video tool-use agents select frames with visual Tools and pass observations to a Planner as text, so any visual evidence the Tool does not verbalize is discarded. This paper identifies that loss as the Tool observation bottleneck and proposes LAVE, a training-free framework that stores pre-verbal hidden-state updates from completed Tool calls, tagged with source timestamps and spatial locations, and reuses the relevant ones as bounded residual updates injected into the Planner's hidden state. The paper reports that, under a comparable frame budget, LAVE improves Video-MME by 3.76 points over the strongest baseline and by 14.27 points over the base VLM, with consistent gains on LongVideoBench and CG-Bench. If the claim holds, long-video agents can carry omitted visual evidence forward in latent space instead of re-watching frames.","feed_headline":"Reusing hidden tool states lifts video agents 3.76 points","feed_subtitle":"Training-free latent channel reuses unspoken visual evidence from finished tool calls on the same frame budget.","key_machinery":"The central mechanism is the dual-channel observation interface with timestamp-addressed latent evidence. Each stored entry keeps a matching key (block-output hidden state), a transferable value (block-local residual update), and a source address consisting of frame time and visual location; the Planner query and Tool keys come from the same decoder block, allowing train-free cosine comparison. Entropy-constrained frame-time routing turns the source-time relevance distribution into an adaptive number of covered moments, and a utility score $u_{tin}=r_{tin}-\\lambda[\\max_{o\\in O_i^{(\\ell)}}\\cos(k_{in},o)]_+$ discounts evidence already verbalized. Bounded residual injection at the line-final token of the corresponding timestamped observation lets later blocks integrate visible description and aligned latent evidence, which is what lets omitted visual content steer the next Tool call.","core_discovery":"At its core, LAVE treats the block-level hidden states of a Tool's visual tokens as reusable evidence. During Tool prefill it captures, for each visual token, a key $k_{in}=H^{(\\ell)}_{i,out}[n]$ and a value $v_{in}=H^{(\\ell)}_{i,out}[n]-H^{(\\ell)}_{i,in}[n]$ at decoder block $\\ell$, along with the source-frame timestamp and patch location; during Planner prefill it uses the final-token state $q_t=H^{(\\ell)}_{t,out}[L_t]$ at the same block to retrieve entries by cosine similarity, subtracts content already covered by text, groups entries by source time, and adaptively selects how many time groups to cover based on the entropy of the source-time relevance distribution. The selected values are read out into a bounded residual, scale-matched to the query RMS, and written at the line-final token of the matching timestamped observation, leaving the visible trajectory and tokenization unchanged. The latent channel is active only during planning, so any effect on the final answer must come through improved Tool selection and evidence acquisition rather than direct answer injection.","pith_inferences":["Beyond the paper: the same timestamp-addressed latent-carry mechanism could apply to any tool-using agent whose tool output is a lossy text summary, as long as the underlying model exposes intermediate activations.","Beyond the paper: the entropy-constrained coverage rule suggests a general design principle, retrieve narrowly when the planner's relevant-moment distribution is peaked and broadly when it is flat, which could be tested as an explicit calibration between planning confidence and evidence breadth.","Beyond the paper: a strong probe of the core assumption would be to train a linear classifier on the stored block-level keys against the visual content of their source frames; if same-block cosine similarity does not track content relevance, part of the observed gain would likely be hidden-state steering rather than evidence retrieval."],"forward_implications":["Existing video tool-use agents can be upgraded by a training-free plug-in that stores and routes hidden states, improving accuracy without additional frames, parameter updates, or modifications to the agent orchestration.","Long-video questions benefit most, with a 4.77-point gain on Video-MME long videos, so latent evidence reuse is most valuable when information spans multiple Tool interactions.","Tool-exploration trajectories become more precise: on CG-Bench, exact clue overlap improves by 7.15 points with nearly unchanged tool-call counts, indicating better localization rather than more search.","Because the latent channel is disabled at final answer generation, the reported accuracy improvement is attributable to planning and evidence acquisition, not to direct answer leakage.","The gain appears across two backbones and three benchmarks, so the mechanism is not tied to a single model or dataset."],"supporting_citations":[{"why":"Supplies the VideoSeek agent loop and its Overview–Skim–Focus tools, the primary baseline LAVE augments and outperforms.","marker":"(Lin et al. 2026)"},{"why":"FrameMind, a hierarchical tool-use agent whose text-only Tool–Planner interface exemplifies the observation bottleneck LAVE targets.","marker":"(Ge et al. 2025)"},{"why":"Establishes the activation-steering premise that additive hidden-state interventions can shift subsequent model behavior.","marker":"(Turner et al. 2023)"},{"why":"Supplies the state-delta encoding idea of transferring token-aligned state changes, which LAVE adapts to block-level visual updates.","marker":"(Tang et al. 2025)"},{"why":"Shows visual semantics can be preserved without full verbalization, supporting the feasibility of pre-verbal latent evidence.","marker":"(Li et al. 2026)"},{"why":"Provides the Video-MME benchmark and its short/medium/long splits used for the main evaluation.","marker":"(Fu et al. 2025)"},{"why":"Provides CG-Bench with answer-bearing clue times, enabling the planner–tool temporal exploration analysis.","marker":"(Chen et al. 2025)"}],"fun_headline_variants":["Reuse hidden tool states for 3.76-point video gain","Video agents gain 3.76 from reusing latent tool states","Training-free latent reuse lifts video agents by 3.76","No training, just reuse: video agents gain 3.76 points","Latent visual evidence reuse: +3.76 points for video agents"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that tool-side and planner-side hidden states at the same middle decoder layer share a common semantic coordinate space, so the cosine similarity between the stored visual keys and the planner query finds evidence genuinely relevant to the next decision.","fun_headline_variants_meta":{"raw":{"variants":["Reuse hidden tool states for 3.76-point video gain","Video agents gain 3.76 from reusing latent tool states","Training-free latent reuse lifts video agents by 3.76","No training, just reuse: video agents gain 3.76 points","Latent visual evidence reuse: +3.76 points for video agents"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001106,"raw_usage":{"total_tokens":4662,"prompt_tokens":1050,"completion_tokens":3612,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":666,"completion_tokens_details":{"reasoning_tokens":3520}},"tokens_in":666,"tokens_out":3612,"duration_ms":24241,"temperature":1.0,"reasoning_tokens":3520,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T00:32:02.569678+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete falsifier is a video question where the Tool's textual observation deliberately omits the deciding visual detail: run LAVE with the same keys and timestamps but replace every stored value vector with norm-matched random noise; if the planner still picks the same Tool trajectory and the accuracy holds, the gains are perturbation artifacts, while a sharp accuracy drop would confirm that the specific visual content is being carried.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the VideoSeek agent loop and its Overview–Skim–Focus tools, the primary baseline LAVE augments and outperforms."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the state-delta encoding idea of transferring token-aligned state changes, which LAVE adapts to block-level visual updates."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides CG-Bench with answer-bearing clue times, enabling the planner–tool temporal exploration analysis."}],"review_version":1}