{"id":"cf33f05c-d884-464c-9226-a5282b63692b","arxiv_id":"2607.05511","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.5,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Dual global+latent states with hierarchical episodic merging enable reflexive, low-latency long-video agents that beat iterative reasoning baselines on accuracy and efficiency.","lead":"Light-Omni replaces multi-step “detective” reasoning in long-video agents with dual contextual states that produce actions and retrieval embeddings in one forward pass. It reports higher accuracy than M3-Agent while cutting latency by ~12× and GPU memory by ~2.6×, and can be plugged into existing MLLMs as a memory module.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"Synthetic training distribution may not transfer to real query–memory gaps that the dual-state design claims to close without iterative reasoning.","rationale":"The reader correctly isolates the synthetic interactive dataset + multi-LoRA premise as the weakest load-bearing assumption for both accuracy and the reflex-over-reasoning slogan. The paper’s own evidence (robustness under artificial C4/audio noise, hierarchical-merge ablations, near-constant latency) is internally consistent and the efficiency numbers are carefully measured, so the work remains a solid systems contribution. No stronger internal inconsistency appears: the dual-state equations, hierarchical merge (k=8), and joint contrastive objective are well-specified. The residual risk is purely distributional transfer from Gemini-generated supervision to real benchmarks; that risk is already reflected in the reader’s CONDITIONAL verdict and does not warrant a harsher rejection. A targeted human-supervised retrain or transfer test would settle the concern cleanly.","tokens_in":22821,"tokens_out":631,"duration_ms":75150,"concrete_test":"Hold out a non-overlapping subset of real long-video queries (or re-annotate 200–300 LVBench / VideoMME-long items with human-written intermediate topics and retrieval positives that never saw Gemini). Retrain only the reaction LoRA + projection head on this human-supervised set (or evaluate zero-shot transfer of the existing Sl). If accuracy drops >3–4 points relative to the Gemini-trained numbers in Table 1 / Fig. 4, or if the SNR advantage in Fig. 6 collapses, the synthetic-data premise fails and the dual-state claim weakens.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that dual states (Sg hierarchical merge + Sl latent embeddings) close the query–memory semantic gap so that a single forward pass yields accurate retrieval and action control, eliminating the need for detective-style iteration (abstract; §1; §3.2–3.3). That claim rests on the training pipeline in §3.3 / App. B: public videos are concatenated, synthetic user queries are TTS-injected, and Gemini-3-Flash-Preview supplies both the ground-truth responses and the intermediate supervisory signals (topics, actions, retrieval positives) used for multi-LoRA + contrastive loss L2 (Eq. 4). Because the same commercial model that generates the “correct” memory topics and positive keys also defines the target distribution that Sl is trained to match, the learned alignment may be an artifact of the synthetic teacher rather than a general solution to the noisy, coreferential, context-dependent queries that appear on VideoMME-long / LVBench / HippoVlog. If the synthetic distribution is easier or differently biased than real interactive streams, both the reported accuracy gains over M3-Agent and the “reflex over reasoning” narrative become overstated; the ablations in Tables 3–4 and Figs. 5–6 only probe within the same synthetic-trained regime.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"Light-Omni is a multimodal agent framework for long-horizon video understanding that replaces multi-step “detective-style” iterative reasoning with dual contextual states computed in a single forward pass. A non-parametric global state Sg is a finite multimodal script obtained by resolution-decaying hierarchical merging (capacity factor k) of episodic memory; a parametric latent state Sl, produced from soft prompts conditioned on Sg, simultaneously triggers actions (search/speech) via Bernoulli heads and supplies a rectified retrieval embedding that is added to the query embedding. The system is built on a three-component long-term memory (user profile, semantic, episodic), trained with multi-LoRA adapters and a hybrid NTP + contrastive objective on a 43–46 k synthetic interactive dataset, and is evaluated for accuracy, latency and memory footprint on VideoMME-long, LVBench, HippoVlog and OVO-Bench, with additional plug-in experiments on other MLLMs.","tokens_in":23256,"tokens_out":1303,"duration_ms":15869,"significance":"If the dual-state design truly closes the query–memory semantic gap without iterative reasoning, the work offers a practical route to near-constant-latency agentic video understanding and a reusable memory module that improves both accuracy and efficiency of existing MLLMs. The reported gains (approximately +2.4 % accuracy / 12.1\times speedup / 2.6\times memory reduction versus M3-Agent, and larger gains versus the 7 B backbone) together with noise-robustness matrices, latency-versus-duration curves and hierarchical-merge scalability plots constitute concrete, falsifiable evidence that would be of clear interest to the long-video and agent communities. The explicit multi-LoRA skill split, hierarchical consolidation algorithm and open project page further strengthen reproducibility.","major_comments":[{"comment":"The central “reflex-over-reasoning” claim (Abstract, §1, §3.2–3.3) rests on the premise that latent embeddings trained with contrastive loss L2 (Eq. 4) on Gemini-generated topics, actions and positive keys (§3.3, Appendix B) close the real query–memory gap. Because the same commercial model supplies both the supervisory signals and the target distribution that Sl is optimized to match, the observed alignment (Figs. 5–6, SNR improvement) may be an artifact of the synthetic teacher rather than a general solution to noisy, coreferential queries on VideoMME-long / LVBench / HippoVlog. A load-bearing experiment is missing: either a human-annotated interactive subset, a cross-teacher transfer test, or an explicit comparison of retrieval precision when positives are taken from a different model family. Without such evidence the accuracy and latency claims versus iterative agents remain only par","section":"§3.3, Eq. (4), Appendix B"},{"comment":"Table 1 and Table 2 compare Light-Omni (Qwen2.5-Omni-7B) with M3-Agent (Qwen3-32B) and WorldMM-8B (which uses GPT-5-mini for memory construction). While the efficiency numbers are unambiguous, the 2.4 % average accuracy edge is confounded by backbone scale and external API quality. A controlled ablation that freezes the same 7 B backbone for both the dual-state agent and a multi-turn reasoning baseline (or reports M3-Agent numbers with a 7 B backbone) is required before the accuracy superiority can be attributed solely to the dual-state design.","section":"Table 1, Table 2, Fig. 4"},{"comment":"The hierarchical merge (Eq. 2, k = 8) and default retrieval counts (Ns = 12, Ne = 4) are free parameters whose sensitivity is only partially explored (Fig. 7). Fig. 8 shows that the topic count stays bounded, yet no ablation quantifies how accuracy or retrieval SNR degrades when k is varied or when the merge is replaced by a simple FIFO buffer of equal token budget. Because Sg is claimed to be the essential global context that enables single-pass retrieval, a short sensitivity table for k and for the merge operator itself is needed to confirm that the reported gains are not an artifact of a carefully tuned capacity factor.","section":"Eq. (2), Fig. 7–8, §4.2"}],"minor_comments":[{"comment":"Fig. 1 caption and the right-hand plot claim “near-constant latency regardless of video duration,” yet Fig. 9 still shows a mild upward trend; a quantitative slope or O-notation statement would avoid over-claiming.","section":"Fig. 1, Fig. 9"},{"comment":"The multi-LoRA design is described only at a high level; the exact adapter-switching logic at inference time (which adapter is active for topic generation versus response versus reaction) should be stated in Algorithm 1 or a short paragraph in §3.3.","section":"§3.3, Algorithm 1"},{"comment":"Several citations appear as arXiv preprints without venue or year (e.g., [29], [52]); standardizing them would improve readability.","section":"References"},{"comment":"In Table 3 the “Vanilla” baseline already reaches 63.11 %; clarifying whether this baseline still uses the same synthetic-trained retrieval head would make the +2.99 % gain of Sg easier to interpret.","section":"Table 3"}],"recommendation":"major_revision","confidential_remarks":"The synthetic-data circularity risk is the single most important issue for the central claim; if the authors can supply even a modest human-annotated transfer experiment or a same-backbone multi-turn baseline, the paper becomes a solid accept. The future-dated arXiv identifier (2607.05511) is odd but does not affect technical assessment."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The useful takeaway is simple: Light-Omni replaces multi-turn “detective” agent loops with a dual contextual state—resolution-decaying hierarchical merge of episodic scripts (Sg, k=8) plus a parametric latent state (Sl) that jointly fires Bernoulli action heads and rectifies retrieval embeddings in one forward pass. On VideoMME-long / LVBench / HippoVlog it beats M3-Agent by ~2.4% average accuracy while cutting interactive latency ~12× and GPU memory ~2.6× (larger gains vs the Qwen2.5-Omni-7B baseline). The same memory stack also lifts other MLLMs when plugged in. That combination of accuracy + near-constant latency is the real contribution.\n\nWhat works: the architecture is cleanly specified (Eqs. 1–4, Fig. 2), the multi-LoRA split (memorization / generation / reaction) is pragmatic, and the evidence is multi-benchmark with ablations of Sg/Sl, memory types, noise robustness (Figs. 5–6, SNR), latency-vs-duration curves, and scalability of the merge. Efficiency numbers are reported against clear baselines; the hierarchical merge keeps the global state bounded even at multi-month horizons. This is honest systems work, not pure marketing.\n\nSoft spots, in proportion. The load-bearing training set is synthetic: public videos + TTS queries + Gemini-3-Flash responses and intermediate signals (topics, positives for the contrastive term). That creates a mild teacher-distribution risk for the “we closed the semantic gap without iteration” claim. The stress-test note is right to flag it, but it does not sink the paper—the evaluation is still on public benchmarks, noise injection still helps, and the plug-in results on other backbones are independent of the synthetic teacher. Comparison asymmetries (backbone size, offline construction cost) exist but are disclosed. Free parameters (k, λ, Ns/Ne, r_comp) are standard and ablated enough. No public code/data yet is a practical annoyance, not a scientific flaw.\n\nWho should read it: anyone building streaming video agents or memory modules for MLLMs who cares about wall-clock latency. It is not a foundational theory paper; it is a well-executed engineering result that moves the practical frontier. I would bring it to reading group, cite the dual-state + efficiency numbers if I am working in the area, and send it to peer review. The synthetic-data caveat belongs in the discussion, not a desk reject.","headline":"Solid systems paper: dual-state design (hierarchical global script + single-pass latent controller) delivers real latency wins and solid accuracy on long-video agents; synthetic Gemini supervision is the main caveat, not a collapse of the claim.","tokens_in":23796,"tokens_out":624,"would_cite":true,"duration_ms":10553,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Dual contextual states let a video agent answer from long-term memory in one forward pass instead of iterative detective reasoning.","keywords":["agentic video understanding","long-term multimodal memory","hierarchical memory consolidation","latent retrieval embeddings","reflexive action control","semantic alignment","video MLLMs"],"falsifier":"On the same long-video benchmarks, replace the latent-state retrieval embedding with ordinary text-query embeddings (or disable hierarchical merging of Sg) while keeping every other component fixed; if accuracy and noise robustness fall back to ordinary RAG levels and the claimed speed/accuracy gains disappear, the dual-state claim fails.","tokens_in":23763,"feed_emoji":"🎬","tokens_out":567,"duration_ms":20261,"temperature":0.7,"pith_summary":"Agentic video systems that keep long-term memory usually still need multi-step “detective” reasoning to decide when to search and what to retrieve, because their memory is fragmented and query and memory sit in mismatched distributions. Light-Omni claims this heavy reasoning is mostly compensating for the missing global context. It replaces the loop with two coupled states that are produced together: a compact hierarchical global script that keeps recent detail and summarizes older events, and a parametric latent state that both triggers actions and emits a retrieval embedding already aligned to the memory space. The result is reflexive answers and near-constant latency on hour-scale video while still beating strong iterative agents on accuracy and resource use, and the same memory layer can be dropped onto other multimodal models.","feed_headline":"Video agent answers from memory in one pass","feed_subtitle":"Hierarchical global script plus latent retrieval embeddings beat iterative agents at lower latency","key_machinery":"Dual contextual states: a resolution-decaying hierarchical global script Sg (merge factor k=8) plus a soft-prompt latent state Sl that is decoded into action probabilities and a retrieval embedding added to the query representation.","core_discovery":"The paper establishes that a hierarchical non-parametric global state (Sg) consolidated from episodic memory, together with a parametric latent state (Sl) that jointly decides actions and produces a rectified retrieval embedding, is enough to close the query–memory semantic gap and drive accurate, low-latency video understanding without multi-turn reasoning.","pith_inferences":[],"forward_implications":[],"fun_headline_variants":["One-pass video agent uses dual states for reflexive answers","Hierarchical global script plus latent state skips multi-turn reasoning","Light-Omni closes query-memory gap without iterative search","Parametric latent embeddings drive low-latency video actions","Global memory state enables single-forward video understanding"],"cache_read_input_tokens":5760,"weakest_assumption_plain":"That a synthetic interactive training set (responses and intermediate signals generated by a commercial model on public videos) plus light multi-LoRA fine-tuning is enough for the learned latent embeddings and hierarchical merge to close the real query–memory gap without iterative reasoning.","fun_headline_variants_meta":{"raw":{"variants":["One-pass video agent uses dual states for reflexive answers","Hierarchical global script plus latent state skips multi-turn reasoning","Light-Omni closes query-memory gap without iterative search","Parametric latent embeddings drive low-latency video actions","Global memory state enables single-forward video understanding"]},"model":"grok-4.5","effort":"low","cost_usd":0.00469,"raw_usage":{"total_tokens":1397,"prompt_tokens":827,"num_sources_used":0,"completion_tokens":62,"cost_in_usd_ticks":46900000,"prompt_tokens_details":{"text_tokens":827,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":508,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":827,"tokens_out":62,"duration_ms":3876,"temperature":1.0,"reasoning_tokens":508,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-11T06:33:13.353520+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"On the same long-video benchmarks, replace the latent-state retrieval embedding with ordinary text-query embeddings (or disable hierarchical merging of Sg) while keeping every other component fixed; if accuracy and noise robustness fall back to ordinary RAG levels and the claimed speed/accuracy gains disappear, the dual-state claim fails.","supporting_citations":[],"review_version":1}