Pith. sign in

REVIEW 4 major objections 4 minor 2 cited by

This paper argues that interleaving explicit chain-of-thought reasoning with incoming video clips — 'thinking while watching' — gives streaming video-language models both higher accuracy and far lower query-answer latency than post-query re

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 18:18 UTC pith:N644DWVL

load-bearing objection A genuinely useful recipe for streaming video reasoning, with broad and promising numbers, but the real-time amortization claim is asserted, not measured, and a few reporting inconsistencies need fixing. the 4 major comments →

arxiv 2603.12262 v2 pith:N644DWVL submitted 2026-03-12 cs.CV

Video Streaming Thinking: VideoLLMs Can Watch and Think Simultaneously

Yiran Guan , Liang Yin , Dingkang Liang , Jianzhong Ju , Zhenbo Luo , Jian Luan , Yuliang Liu , Xiang Bai This is my paper
classification cs.CV
keywords streaming video understandingchain-of-thoughttest-time scalingvideo large language modelsreinforcement learningknowledge graphlatency amortizationonline video QA
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper closes a gap between real-time streaming video understanding and explicit reasoning. It claims that the missing ingredient is a synchronized stream of logical thoughts generated while the video plays, before any question is asked. This 'streaming thinking' is formalized as a product of per-clip thought factors and a final answer factor, and it is trained by a two-stage recipe (SFT then RL) on data synthesized from video knowledge graphs. If the claim holds, video assistants can reason about what they see without forcing the user to wait for a long post-query thinking phase — and the same mechanism also lifts accuracy on offline long-video benchmarks.

Core claim

The central claim is that by producing textual thoughts between incoming video clips and accumulating them in a memory, a VideoLLM can front-load its reasoning so that when the query arrives it answers directly from the accumulated thoughts plus the current visual context. The paper shows that this protocol — realized with supervised fine-tuning under a streaming attention mask, followed by reinforcement learning on verifiable rewards — outperforms prior streaming-perception-only methods on online benchmarks and stays competitive on offline long-form reasoning, while cutting query-answer latency by over an order of magnitude relative to post-query chain-of-thought baselines.

What carries the argument

The load-bearing object is the streaming-thought loop defined by p(y|q,cK,mK) ∏ p(zk|ck,m_{k-1}): at each clip interval the model samples a thought summary from the current clip and the accumulated memory, and the memory is updated by a simple FIFO eviction. This factorization turns the model into a causal, memory-driven process that hides reasoning computation inside the natural inter-clip gaps, so the final answer is a single forward call on the query. Supporting this are a dual-memory design (short-term visual buffer plus long-term textual semantic memory), a streaming attention mask that enforces temporal causality within a fixed visual budget, and a knowledge-graph-based data synthesis

Load-bearing premise

The load-bearing premise is that the model's streaming thoughts can always be generated within the time between successive video clips, so the reasoning cost is hidden and adds nothing to real-world response time.

What would settle it

Measure the wall-clock completion time of the streaming-think phase on a live video with a range of clip inter-arrival intervals (e.g., 0.5 s, 1 s, 2 s). If thought generation finishes after the next clip arrives in a substantial fraction of cases, the latency-amortization claim fails and the QA-latency advantage would not translate to real-time responsiveness.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If VST is right, streaming video assistants can answer questions in under a second even when the backend is a reasoning-tuned LLM, because the thinking happens before the question arrives.
  • The same amortized test-time scaling idea could be carried to other streaming modalities — audio, sensor feeds, embodied perception — wherever there are natural inter-arrival intervals to hide computation.
  • Streaming thinking builds a more durable memory than token-compression streaming methods, as shown by strong backward-tracing and forward-prediction results on online benchmarks.
  • An offline VideoLLM can be converted to streaming use without changing its architecture — only its inference schedule and training data.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A straightforward test of the mechanism: ablate the thought tokens (feed only the memory summaries) and measure whether backward tracing degrades more than forward prediction; if not, the gains may come from the SFT/RL recipe rather than the streaming-thought loop itself.
  • The latency-amortization premise is falsifiable by measuring thought-completion time against clip inter-arrival intervals; if thinking frequently overruns the gap, the claimed real-time responsiveness collapses.
  • The knowledge-graph-driven synthesis could be reused for other long-horizon embodied tasks — e.g., navigation or monitoring — where incremental reasoning over a stream of observations is needed.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. This paper proposes Video Streaming Thinking (VST), a paradigm for online VideoLLMs in which the model generates explicit textual 'streaming thoughts' for each incoming video clip before any user query arrives. The model maintains a short-term visual buffer and a long-term FIFO textual memory; at query time it produces the final answer directly from the accumulated memory and current clip. Training consists of VST-SFT, which uses a streaming attention mask over interleaved clip-thought sequences, followed by VST-RL (GRPO/DAPO) with a verifiable final-answer reward. A data-synthesis pipeline builds knowledge graphs with Gemini and samples evidence chains to produce 100K streaming QA/CoT examples. Evaluations on StreamingBench, OVO-Bench, VideoMME, LongVideoBench, and VideoHolmes report strong online performance (79.5% and 59.3% on the two online benchmarks) and competitive offline results, with 0.56s QA latency and a 15.7x QA-latency reduction relative to Video-R1. Equation (1) is a chain-rule factorization of the generative process, and the RL reward is final-answer correctness, so I do not see a circularity issue.

Significance. The VST idea directly targets a real and important trade-off between test-time scaling and streaming latency. If the latency-amortization claim can be substantiated, the paper would be a valuable contribution to online video understanding. The manuscript is commendably concrete about the architecture and training recipe: Eqs. (2)-(4) define the sequence and temporal segmentation, Eqs. (5)-(6) give the RL objective, and Sec. 2.3 provides a skeletal but reproducible data-synthesis pipeline. The ablations support the value of the two-stage recipe, and the 3B/7B/32B scaling study is a useful generalization check. I found no circularity in the objective. The main gaps are experimental: the real-time amortization claim is asserted without wall-clock evidence, and reproducibility safeguards (code/data release, decontamination, error bars) are missing.

major comments (4)
  1. [Sec. 3.6, Table 6, Fig. 7] The central latency claim is not measured. The paper asserts that streaming-think computation 'finishes within the clip inter-arrival interval' and is amortized, so it 'does not increase the real-world end-to-end inference time.' Table 6, however, reports only query-to-answer latency, which excludes streaming-thought generation time. No per-clip wall-clock thinking time, clip inter-arrival distribution, throughput, or hardware settings are given; the Limitation section itself concedes that LLM token consumption is non-negligible. If per-clip thinking exceeds the clip cadence, the system must either truncate/drop thoughts (changing the reported accuracy) or accumulate a pre-query backlog that surfaces after the query. The advertised 15.7x speedup compares only post-query latency and does not establish end-to-end responsiveness. Please measure per-clip streaming-think generation time again
  2. [Tables 4 and 5] The same base model is evaluated inconsistently. Table 4 reports Qwen2.5-VL-7B (Base model) at 50.5 on OVO-Bench Overall and 62.9 on VideoMME w/o sub. Overall; Table 5 lists the same 7B base as 55.0 and 62.3. Both are presented as the same Qwen2.5-VL-7B base with the evaluation setup described in Sec. 3.1. The discrepancy changes the measured benefit of the full pipeline on OVO-Bench from +8.8 to +4.3 points. Please reconcile the numbers or explicitly state which evaluation configuration differs for each table. Relatedly, no seeds or error bars are reported; several margins are small (e.g., LongVideoBench 58.0 vs 57.5, VideoMME-long 55.3 vs 55.2), so single-run comparisons are not robust.
  3. [Sec. 2.3 / Sec. 3.2] No training/evaluation decontamination check is described. The VST SFT/RL training data are collected from LLaVA-Vid and Video-Marathon (Sec. 2.3), while evaluation uses StreamingBench, OVO-Bench, VideoMME, LongVideoBench, and VideoHolmes (Sec. 3.2), all of which draw on public video collections. Please report exact overlap between the training videos/questions and each benchmark, and describe how near-duplicates are removed. Without this, the reported SOTA gaps (e.g., +2.2 over StreamForest on StreamingBench) could be inflated by leakage.
  4. [Sec. 2.3] The data-synthesis pipeline is a core contribution, but no validation statistics are given for the Gemini-generated knowledge graphs, evidence chains, or QA/CoT samples. The paper states only that a 'strict post-generation filtering rubric' is applied. Please report filter pass rates, a human agreement/quality study on a random sample, or some failure analysis. The claim that 100K samples are 'high-quality' currently rests on the rubric description alone, which is insufficient to audit or reproduce the pipeline.
minor comments (4)
  1. [Sec. 2.1 / Sec. 3.1] The visual buffer size L, knowledge-graph window W, and the token-to-frame mapping are never given numeric values. These parameters determine the clip inter-arrival interval that is central to the latency analysis. Please report L, W, and the resulting average clip duration.
  2. [Fig. 5] The x-axis ('Max Streaming Thinking Times') has no numeric tick labels and no error bars, making it difficult to read the saturation/plateau behavior described in the text. Please add actual step values and variance estimates.
  3. [Table 6 caption] The column label 'QA Latency' should be defined explicitly (e.g., time from query arrival to complete/first-token answer) and the hardware/software configuration used for measurement should be listed. As written, the label invites confusion with end-to-end latency.
  4. [Abstract / Sec. 5] The abstract and limitation section state that code, data, and models 'will be released,' but none is available at review time. Please provide release details or include representative generated samples and model outputs in the supplement.

Circularity Check

0 steps flagged

No significant circularity; the core factorization and training pipeline are self-contained and benchmark results are external.

full rationale

Equation (1) is a purely formal chain-rule factorization of the joint generation process p(y|q,V) = p(y|q,c_K,m_K) * product_k p(z_k|c_k,m_{k-1}); it introduces no fitted quantity and no benchmark value as an input. The training pipeline (VST-SFT and VST-RL) optimizes against synthetic demonstrations and verifiable final-answer rewards, not against the reported StreamingBench, OVO-Bench, VideoMME, LongVideoBench, or VideoHolmes scores. The data synthesis pipeline uses video knowledge graphs and Gemini to generate QA pairs; this is distillation/synthesis rather than circular reuse of the evaluation targets. The latency comparison is an explicit definition of QA latency as query-to-response time, and the paper separately acknowledges that streaming-thought token consumption is non-negligible; whether the amortization assumption holds empirically is a measurement/robustness concern, not a circularity. Self-citations to the authors' prior work (Thinkomni, Shuffle-R1, embodied agents, etc.) appear only in related work or as general references and are not load-bearing for the central claim. No uniqueness theorem is imported, no fitted parameter is renamed as a prediction, and no known result is merely re-labeled in a way that reduces the contribution to its own inputs. The claims are independently assessable against external benchmarks, so the circularity score is 0.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 0 invented entities

No physical or theoretical entities are postulated. 'Streaming thought' is a generated token sequence and 'dual memory' is stored text plus a visual buffer; these are algorithmic constructs, not new explanatory entities. The main load-bearing assumptions are causal factorization, finite-memory sufficiency, RL reward transfer to thoughts, and trust in the Gemini-generated supervision.

free parameters (5)
  • max_streaming_think_times = 4
    Inference cap on the number of streaming-thought generations per video (Sec. 3.1, Fig. 5). Hand-chosen as a latency/accuracy trade-off; ablations show gains continue to 16 steps on OVO-Bench Backward, so the chosen cap affects reported results.
  • visual_buffer_size_L = 8192 video tokens
    Capacity L in the attention mask Eq. (3) and the inference video-token cap (Sec. 3.1). Hand-set; controls how much current visual context the model can attend to, directly affecting which evidence is available at answer time.
  • sampling_fps = 2 fps
    Input frame rate used for all experiments (Sec. 3.1); determines clip composition and the visual information available to streaming thoughts.
  • knowledge_graph_window_W = sliding window size W, exact value not stated
    Data-synthesis memory window for entity extraction (Sec. 2.3); chosen by hand, affects evidence-chain quality but not runtime inference.
  • entity_overlap_threshold = 10%
    Diversity constraint between sampled evidence chains during data synthesis (Sec. 2.3); hand-set data-generation hyperparameter.
axioms (4)
  • standard math The joint probability of thoughts and answer factorizes causally as p(y|q,cK,mK) ∏ p(zk|ck,m_{k-1}) (Eq. 1), i.e., thoughts depend only on the current clip and previous memory.
    True by the chain rule under the stated causal generation order; standard, but it assumes thoughts cannot see future clips.
  • domain assumption Textual FIFO memory captures the semantically important history; evicting old memory entries does not lose information needed for final answers.
    The method relies on finite memory and FIFO eviction (Sec. 2.1). No evidence is given that critical long-range facts survive eviction; this is a design assumption.
  • domain assumption A final-answer-only verifiable reward, with advantage backpropagated to all thought tokens, improves streaming-thought quality.
    VST-RL (Sec. 2.2, Eq. 5) assigns group advantage to every generated token including thoughts, without any direct reward for thought quality; the paper relies on this standard RL hypothesis.
  • ad hoc to paper Knowledge-graph triples extracted by Gemini and the synthesized QA/CoT data are faithful to the video content and free of leakage.
    The SFT supervision (Sec. 2.3) is generated by an offline proprietary VideoLLM over an entity bank; correctness of these labels is assumed, and no human validation rate is reported.

pith-pipeline@v1.3.0-alltime-deepseek · 17624 in / 14084 out tokens · 128883 ms · 2026-08-02T18:18:27.051350+00:00 · methodology

0 comments
read the original abstract

Online Video Large Language Models (VideoLLMs) play a critical role in supporting responsive, real-time interaction. Existing methods focus on streaming perception, lacking a synchronized logical reasoning stream. However, directly applying test-time scaling methods incurs unacceptable response latency. To address this trade-off, we propose Video Streaming Thinking (VST), a novel paradigm for streaming video understanding. It supports a thinking while watching mechanism, which activates reasoning over incoming video clips during streaming. This design improves timely comprehension and coherent cognition while preserving real-time responsiveness by amortizing LLM reasoning latency over video playback. Furthermore, we introduce a comprehensive post-training pipeline that integrates VST-SFT, which structurally adapts the offline VideoLLM to causal streaming reasoning, and VST-RL, which provides end-to-end improvement through self-exploration in a multi-turn video interaction environment. Additionally, we devise an automated training-data synthesis pipeline that uses video knowledge graphs to generate high-quality streaming QA pairs, with an entity-relation grounded streaming Chain-of-Thought to enforce multi-evidence reasoning and sustained attention to the video stream. Extensive evaluations show that VST-7B performs strongly on online benchmarks, e.g. 79.5% on StreamingBench and 59.3% on OVO-Bench. Meanwhile, VST remains competitive on offline long-form or reasoning benchmarks. Compared with Video-R1, VST responds 15.7 times faster and achieves +5.4% improvement on VideoHolmes, demonstrating higher efficiency and strong generalization across diverse video understanding tasks. Code, data, and models will be released at https://github.com/1ranGuan/VST.

Figures

Figures reproduced from arXiv: 2603.12262 by Dingkang Liang, Jian Luan, Jianzhong Ju, Liang Yin, Xiang Bai, Yiran Guan, Yuliang Liu, Zhenbo Luo.

Figure 1
Figure 1. Figure 1: Benchmark results and paradigm comparison. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Illustration of the Video Stream Thinking pipeline. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Overview of the training pipeline. (a) VST-SFT applies a streaming attention mask to enforce temporal causality, restricting attention to the current visual buffer and history textual context. (b) VST-RL performs on-policy optimization via an agentic loop, improving the quality of streaming thoughts through verifiable rewards computed solely from the final answer. This formulation yields two distinct advan… view at source ↗
Figure 4
Figure 4. Figure 4: Stream-Thought QA data curation pipeline. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Ablation study on max thinking times [PITH_FULL_IMAGE:figures/full_fig_p011_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Case Study from VideoHolmes. We compare VST-7B with Video-R1-7B. VST-7B processes the video stream and performs streaming thinking before the query, then answers directly once the query arrives. In contrast, Video-R1-7B generates CoT after the query, resulting in higher QA latency. VST-7B achieves better performance with lower QA latency in this example. Case Study [PITH_FULL_IMAGE:figures/full_fig_p013_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: The streaming inference pipeline of VST. [PITH_FULL_IMAGE:figures/full_fig_p020_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. FOLIO: Focused Semantic Memory for Streaming Video Understanding

    cs.CV 2026-07 conditional novelty 6.0

    Entity-centered focus-guided streaming memory lifts Qwen3-VL-8B to 82.0/69.1 Perception/Backward on OVO-Bench and 74.5 on StreamingBench while cutting writer tokens by ~32%.

  2. LLaVA-OneVision-2: Towards Next-Generation Perceptual Intelligence

    cs.CV 2026-05 unverdicted novelty 4.0

    LLaVA-OV-2 uses codec-stream tokenization and a shared 3D RoPE to improve video, spatial, and tracking performance over Qwen3-VL-8B, while introducing the JumpScore benchmark for fine-grained motion localization.

Reference graph

Works this paper leans on

58 extracted references · 15 linked inside Pith · cited by 2 Pith papers

  1. [1]

    5-vl technical report

    Bai, S., Chen, K., Liu, X., Wang, J., Ge, W., Song, S., Dang, K., Wang, P., Wang, S., Tang, J., et al.: Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923 (2025) 2, 8, 9

    Pith/arXiv arXiv 2025

  2. [2]

    In: Proc

    Chen, J., Lv, Z., Wu, S., Lin, K.Q., Song, C., Gao, D., Liu, J.W., Gao, Z., Mao, D., Shou, M.Z.: Videollm-online: Online video large language model for streaming video. In: Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition. pp. 18407–18418 (2024) 9, 10, 14

    2024

  3. [3]

    In: Proc

    Chen, J., Zeng, Z., Lin, Y., Li, W., Ma, Z., Shou, M.Z.: Livecc: Learning video llm with streaming speech transcription at scale. In: Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition. pp. 29083–29095 (2025) 1

    2025

  4. [4]

    In: Proc

    Chen, Y., Huang, W., Shi, B., Hu, Q., Ye, H., Zhu, L., Liu, Z., Molchanov, P., Kautz, J., Qi, X., et al.: Scaling rl to long videos. In: Proc. of Advances in Neural Information Processing Systems (2025) 2, 8, 10, 14

    2025

  5. [5]

    Cheng, J., Ge, Y., Wang, T., Ge, Y., Liao, J., Shan, Y.: Video-holmes: Can mllm think like holmes for complex video reasoning? arXiv preprint arXiv:2505.21374 (2025) 9

    Pith/arXiv arXiv 2025

  6. [6]

    In: Proc

    Di, S., Yu, Z., Zhang, G., Li, H., Cheng, H., Li, B., He, W., Shu, F., Jiang, H., et al.: Streaming video question-answering with in-context video kv-cache retrieval. In: Proc. of Intl. Conf. on Learning Representations (2025) 2, 14

    2025

  7. [7]

    In: Proc

    Driess, D., Xia, F., Sajjadi, M.S., Lynch, C., Chowdhery, A., Ichter, B., Wahid, A., Tompson, J., Vuong, Q., Yu, T., et al.: Palm-e: an embodied multimodal language model. In: Proc. of Intl. Conf. on Machine Learning. pp. 8469–8488 (2023) 1

    2023

  8. [8]

    In: Proc

    Feng, K., Gong, K., Li, B., Guo, Z., Wang, Y., Peng, T., Wu, J., Zhang, X., Wang, B., Yue, X.: Video-r1: Reinforcing video reasoning in mllms. In: Proc. of Advances in Neural Information Processing Systems (2025) 2, 10, 14

    2025

  9. [9]

    In: Proc

    Feng, K., Zhang, M., Li, H., Fan, K., Chen, S., Jiang, Y., Zheng, D., Sun, P., Zhang, Y., Sun, H., et al.: Onethinker: All-in-one reasoning model for image and video. In: Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition (2026) 8

    2026

  10. [10]

    In: Proc

    Fu, C., Dai, Y., Luo, Y., Li, L., Ren, S., Zhang, R., Wang, Z., Zhou, C., Shen, Y., Zhang, M., et al.: Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis. In: Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition. pp. 24108–24118 (2025) 9

    2025

  11. [11]

    In: Proc

    Guan, Y., Tu, S., Liang, D., Zhu, L., Ju, J., Luo, Z., Luan, J., Liu, Y., Bai, X.: Thinkomni: Lifting textual reasoning to omni-modal scenarios via guidance decoding. In: Proc. of Intl. Conf. on Learning Representations (2026) 2

    2026

  12. [12]

    Nature645(8081), 633–638 (2025) 2, 6, 14

    Guo, D., Yang, D., Zhang, H., Song, J., Wang, P., Zhu, Q., Xu, R., Zhang, R., Ma, S., Bi, X., et al.: Deepseek-r1 incentivizes reasoning in llms through reinforcement learning. Nature645(8081), 633–638 (2025) 2, 6, 14

    2025

  13. [13]

    Hagberg, A., Swart, P.J., Schult, D.A.: Exploring network structure, dynamics, and function using networkx. Tech. rep., Los Alamos National Laboratory (LANL) (2007) 7

    2007

  14. [14]

    In: Proc

    Han, S., Huang, W., Shi, H., Zhuo, L., Su, X., Zhang, S., Zhou, X., Qi, X., Liao, Y., Liu, S.: Videoespresso: A large-scale chain-of-thought dataset for fine-grained video reasoning via core frame selection. In: Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition. pp. 26181–26191 (2025) 14

    2025

  15. [15]

    In: Proc

    Hannan, T., Islam, M.M., Gu, J., Seidl, T., Bertasius, G.: Revisionllm: Recursive vision-language model for temporal grounding in hour-long videos. In: Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition. pp. 19012–19022 (2025) 14 16VideoStreamingThinking

    2025

  16. [16]

    Science303(5664), 1634–1640 (2004) 2

    Hasson,U.,Nir,Y.,Levy,I.,Fuhrmann,G.,Malach,R.:Intersubjectsynchronization of cortical activity during natural vision. Science303(5664), 1634–1640 (2004) 2

    2004

  17. [17]

    arXiv preprint arXiv:2204.01018 (2022) 8

    Hu, H., Dong, S., Zhao, Y., Lian, D., Li, Z., Gao, S.: Transrac: Encoding multi-scale temporal correlation with transformers for repetitive action counting. arXiv preprint arXiv:2204.01018 (2022) 8

    Pith/arXiv arXiv 2022

  18. [18]

    In: Conference on Language Modeling (2024) 9

    Hu, S., Tu, Y., Han, X., Cui, G., He, C., Zhao, W., Long, X., Zheng, Z., Fang, Y., Huang, Y., Zhang, X., Thai, Z.L., Wang, C., Yao, Y., Zhao, C., Zhou, J., Cai, J., Zhai, Z., Ding, N., Jia, C., Zeng, G., dahai li, Liu, Z., Sun, M.: MiniCPM: Unveiling the potential of small language models with scalable training strategies. In: Conference on Language Model...

    2024

  19. [19]

    In: Proc

    Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C.H., Gonzalez, J.E., Zhang, H., Stoica, I.: Efficient memory management for large language model serving with pagedattention. In: Proc. of the ACM SIGOPS 29th Symposium on Operating Systems Principles (2023) 8

    2023

  20. [20]

    Transactions on Machine Learning Research (2025) 9, 10

    Li, B., Zhang, Y., Guo, D., Zhang, R., Li, F., Zhang, H., Zhang, K., Zhang, P., Li, Y., Liu, Z., Li, C.: LLaVA-onevision: Easy visual task transfer. Transactions on Machine Learning Research (2025) 9, 10

    2025

  21. [21]

    In: Proc

    Li, J., Yin, H., Tan, W., Chen, J., Xu, B., Qu, Y., Chen, Y., Ju, J., Luo, Z., Luan, J.: Revisor: Beyond textual reflection, towards multimodal introspective reasoning in long-form video understanding. In: Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition (2026) 10, 14

    2026

  22. [22]

    In: Proc

    Li, Z., Yang, B., Liu, Q., Ma, Z., Zhang, S., Yang, J., Sun, Y., Liu, Y., Bai, X.: Monkey: Image resolution and text label are important things for large multi-modal models. In: Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition. pp. 26763–26773 (2024) 2

    2024

  23. [23]

    In: Proc

    Liang, D., Zhang, C., Xu, X., Ju, J., Luo, Z., Bai, X.: Cook and clean together: Teaching embodied agents for parallel task execution. In: Proc. of the AAAI Conf. on Artificial Intelligence (2025) 2

    2025

  24. [24]

    In: Proc

    Lin, J., Yin, H., Ping, W., Molchanov, P., Shoeybi, M., Han, S.: Vila: On pre- training for visual language models. In: Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition. pp. 26689–26699 (2024) 9

    2024

  25. [25]

    arXiv preprint arXiv:2506.05332 (2025) 8

    Lin, J., Wu, J., Sun, X., Wang, Z., Liu, J., Su, Y., Yu, X., Chen, H., Luo, J., Liu, Z., et al.: Unleashing hour-scale video training for long video-language understanding. arXiv preprint arXiv:2506.05332 (2025) 8

    arXiv 2025

  26. [26]

    Lin,J.,Fang,Z.,Chen,C.,Wan,Z.,Luo,F.,Li,P.,Liu,Y.,Sun,M.:Streamingbench: Assessingthegapformllmstoachievestreamingvideounderstanding.arXivpreprint arXiv:2411.03628 (2024) 8

    Pith/arXiv arXiv 2024

  27. [27]

    arXiv preprint arXiv:2503.20783 (2025) 6

    Liu, Z., Chen, C., Li, W., Qi, P., Pang, T., Du, C., Lee, W.S., Lin, M.: Understanding r1-zero-like training: A critical perspective. arXiv preprint arXiv:2503.20783 (2025) 6

    Pith/arXiv arXiv 2025

  28. [28]

    arXiv preprint arXiv:2505.15269 (2025) 2

    Ning, Z., Liu, G., Jin, Q., Ding, W., Guo, M., Zhao, J.: Livevlm: Efficient online video understanding via streaming-oriented kv cache and retrieval. arXiv preprint arXiv:2505.15269 (2025) 2

    Pith/arXiv arXiv 2025

  29. [29]

    of IEEE Intl

    Niu, J., Li, Y., Miao, Z., Ge, C., Zhou, Y., He, Q., Dong, X., Duan, H., Ding, S., Qian, R., et al.: Ovo-bench: How far is your video-llms from real-world online video understanding? In: Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition. pp. 18902–18913 (2025) 8

    2025

  30. [30]

    OpenAI: Gpt-4o system card (2024) 9, 10

    2024

  31. [31]

    In: Proc

    Qian, R., Ding, S., Dong, X., Zhang, P., Zang, Y., Cao, Y., Lin, D., Wang, J.: Dispider: Enabling video llms with active real-time interaction via disentangled VideoStreamingThinking17 perception, decision, and reaction. In: Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition. pp. 24045–24055 (2025) 9, 10, 14

    2025

  32. [32]

    In: Proc

    Qian, R., Dong, X., Zhang, P., Zang, Y., Ding, S., Lin, D., Wang, J.: Streaming long video understanding with large language models. In: Proc. of Advances in Neural Information Processing Systems. vol. 37, pp. 119336–119360 (2024) 14

    2024

  33. [33]

    In: Proc

    Shen, X., Xiong, Y., Zhao, C., Wu, L., Chen, J., Zhu, C., Liu, Z., Xiao, F., Varadarajan, B., Bordes, F., et al.: Longvu: Spatiotemporal adaptive compression for long video-language understanding. In: Proc. of Intl. Conf. on Machine Learning (2025) 10

    2025

  34. [34]

    arXiv preprint arXiv: 2409.19256 (2024) 8

    Sheng, G., Zhang, C., Ye, Z., Wu, X., Zhang, W., Zhang, R., Peng, Y., Lin, H., Wu, C.: Hybridflow: A flexible and efficient rlhf framework. arXiv preprint arXiv: 2409.19256 (2024) 8

    Pith/arXiv arXiv 2024

  35. [35]

    In: Proc

    Song, E., Chai, W., Wang, G., Zhang, Y., Zhou, H., Wu, F., Chi, H., Guo, X., Ye, T., Zhang, Y., et al.: Moviechat: From dense token to sparse memory for long video understanding. In: Proc. of IEEE Intl. Conf. on Computer Vision and Pattern Recognition. pp. 18221–18232 (2024) 2, 14

    2024

  36. [36]

    Stephens, G.J., Silbert, L.J., Hasson, U.: Speaker–listener neural coupling underlies successful communication. Proc. of the National Academy of Sciences107(32), 14425–14430 (2010) 2

    2010

  37. [37]

    arXiv preprint arXiv:2403.05530 (2024) 9, 10

    Team, G., Georgiev, P., Lei, V.I., Burnell, R., Bai, L., Gulati, A., Tanzer, G., Vincent, D., Pan, Z., Wang, S., et al.: Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530 (2024) 9, 10

    Pith/arXiv arXiv 2024

  38. [38]

    arXiv preprint arXiv:2510.17238 (2025) 14

    Tong, J., Fan, Y., Zhao, A., Ma, Y., Shen, X.: Streamingthinker: Large language models can think while reading. arXiv preprint arXiv:2510.17238 (2025) 14

    arXiv 2025

  39. [39]

    arXiv preprint arXiv:2409.12191 (2024) 10

    Wang, P., Bai, S., Tan, S., Wang, S., Fan, Z., Bai, J., Chen, K., Liu, X., Wang, J., Ge, W., et al.: Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution. arXiv preprint arXiv:2409.12191 (2024) 10

    Pith/arXiv arXiv 2024

  40. [40]

    In: Proc

    Wang, Q., Yu, Y., Yuan, Y., Mao, R., Zhou, T.: VideoRFT: Incentivizing video reasoning capability in MLLMs via reinforced fine-tuning. In: Proc. of Advances in Neural Information Processing Systems (2025) 2, 14

    2025

  41. [41]

    In: Proc

    Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q.V., Zhou, D., et al.: Chain-of-thought prompting elicits reasoning in large language models. In: Proc. of Advances in Neural Information Processing Systems. vol. 35, pp. 24824–24837 (2022) 14

    2022

  42. [42]

    In: Proc

    Wu, H., Li, D., Chen, B., Li, J.: Longvideobench: A benchmark for long-context in- terleaved video-language understanding. In: Proc. of Advances in Neural Information Processing Systems. vol. 37, pp. 28828–28857 (2024) 9

    2024

  43. [43]

    arXiv preprint arXiv:2512.21334 (2025) 9, 10

    Xia, J., Chen, P., Zhang, M., Sun, X., Zhou, K.: Streaming video instruction tuning. arXiv preprint arXiv:2512.21334 (2025) 9, 10

    Pith/arXiv arXiv 2025

  44. [44]

    Xiaomi, L.C.T.: Mimo-vl technical report (2025),https://arxiv.org/abs/2506. 035692

    2025

  45. [45]

    In: Proc

    Xu, R., Xiao, G., Chen, Y., He, L., Peng, K., Lu, Y., Han, S.: StreamingVLM: Real-time understanding for infinite video streams. In: Proc. of Intl. Conf. on Learning Representations (2026) 14

    2026

  46. [46]

    In: Proc

    Yan, Z., He, Y., Li, X., Yue, Z., Zeng, X., Wang, Y., Qiao, Y., Wang, L., Wang, Y.: Videochat-r1.5: Visual test-time scaling to reinforce multimodal reasoning by iterative perception. In: Proc. of Advances in Neural Information Processing Systems (2025) 14

    2025

  47. [47]

    arXiv preprint arXiv:2508.15717 (2025) 2, 14 18VideoStreamingThinking

    Yang, Y., Zhao, Z., Shukla, S.N., Singh, A., Mishra, S.K., Zhang, L., Ren, M.: Streammem: Query-agnostic kv cache memory for streaming video understanding. arXiv preprint arXiv:2508.15717 (2025) 2, 14 18VideoStreamingThinking

    Pith/arXiv arXiv 2025

  48. [48]

    In: Proc

    Yao, L., Li, Y., Wei, Y., Li, L., Ren, S., Liu, Y., Ouyang, K., Wang, L., Li, S., Li, S., et al.: Timechat-online: 80% visual tokens are naturally redundant in streaming videos. In: Proc. of ACM Intl. Conf. on Multimedia. pp. 10807–10816 (2025) 2, 9, 10, 14

    2025

  49. [49]

    arXiv preprint arXiv:2507.02259 (2025) 6

    Yu, H., Chen, T., Feng, J., Chen, J., Dai, W., Yu, Q., Zhang, Y.Q., Ma, W.Y., Liu, J., Wang, M., et al.: Memagent: Reshaping long-context llm with multi-conv rl-based memory agent. arXiv preprint arXiv:2507.02259 (2025) 6

    Pith/arXiv arXiv 2025

  50. [50]

    In: Proc

    Yu, Q., Zhang, Z., Zhu, R., Yuan, Y., Zuo, X., Yue, Y., Dai, W., Fan, T., Liu, G., Liu, L., et al.: Dapo: An open-source llm reinforcement learning system at scale. In: Proc. of Advances in Neural Information Processing Systems (2025) 6, 7

    2025

  51. [51]

    In: Proc

    Zeng, X., Qiu, K., Zhang, Q., Li, X., Wang, J., Li, J., Yan, Z., Tian, K., Tian, M., Zhao, X., et al.: Streamforest: Efficient online video understanding with persistent event memory. In: Proc. of Advances in Neural Information Processing Systems (2025) 2, 8, 9, 10, 14

    2025

  52. [52]

    arXiv preprint arXiv:2601.23224 (2026) 2

    Zeng, X., Zhang, Z., Zhu, Y., Li, X., Wang, Z., Ma, C., Zhang, Q., Huang, Z., Ouyang, K., Jiang, T., et al.: Video-o3: Native interleaved clue seeking for long video multi-hop reasoning. arXiv preprint arXiv:2601.23224 (2026) 2

    Pith/arXiv arXiv 2026

  53. [53]

    In: Proc

    Zhang, H., Wang, Y., Tang, Y., Liu, Y., Feng, J., Jin, X.: Flash-vstream: Efficient real-time understanding for long video streams. In: Proc. of IEEE Intl. Conf. on Computer Vision. pp. 21059–21069 (2025) 9, 14

    2025

  54. [54]

    Zhang, K., Li, B., Zhang, P., Pu, F., Cahyono, J.A., Hu, K., Liu, S., Zhang, Y., Yang, J., Li, C., Liu, Z.: Lmms-eval: Reality check on the evaluation of large multimodal models (2024),https://arxiv.org/abs/2407.127728

    Pith/arXiv arXiv 2024

  55. [55]

    Transactions on Machine Learning Research (2025) 9, 10

    Zhang, P., Zhang, K., Li, B., Zeng, G., Yang, J., Zhang, Y., Wang, Z., Tan, H., Li, C., Liu, Z.: Long context transfer from language to vision. Transactions on Machine Learning Research (2025) 9, 10

    2025

  56. [56]

    Transactions on Machine Learning Research (2025) 8, 10

    Zhang, Y., Wu, J., Li, W., Li, B., MA, Z., Liu, Z., Li, C.: LLaVA-video: Video instruction tuning with synthetic data. Transactions on Machine Learning Research (2025) 8, 10

    2025

  57. [57]

    arXiv preprint arXiv:2304.11277 (2023) 8

    Zhao, Y., Gu, A., Varma, R., Luo, L., Huang, C.C., Xu, M., Wright, L., Shojanazeri, H., Ott, M., Shleifer, S., et al.: Pytorch fsdp: experiences on scaling fully sharded data parallel. arXiv preprint arXiv:2304.11277 (2023) 8

    Pith/arXiv arXiv 2023

  58. [58]

    HUB-AND-SPOKE

    Zhu, L., Guan, Y., Liang, D., Ju, J., Luo, Z., Qin, B., Luan, J., Liu, Y., Bai, X.: Shuffle-r1: Efficient rl framework for multimodal large language models via data-centric dynamic shuffle. In: Proc. of Intl. Conf. on Learning Representations (2026) 2 VideoStreamingThinking19 Appendix A Details of VST Inference A.1 Inference Prompt We detail the inference...

    2026