Pith. sign in

REVIEW 3 major objections 4 minor 67 references

A training-free memory that organizes streaming video around latent object tracks improves question-answering by up to 10 points on a real-time perception benchmark while cutting peak GPU memory and time-to-first-token by roughly half.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 03:15 UTC pith:BBYKBJMU

load-bearing objection ObjectStream is a well-specified training-free streaming-memory system that works, but its central 'latent object' claim is undercut by a near-tie with a non-object token-selection baseline and a lack of objectness validation. the 3 major comments →

arxiv 2607.28312 v2 pith:BBYKBJMU submitted 2026-07-30 cs.CV cs.AI

ObjectStream: Latent Objects as Memory Anchors for Streaming Video Understanding

Mingkang Dong , Muxin Pu , Jie Li , Bohan Guo , Songruo Chen , Bin Ren , Xu Zheng , Chen Zhao
show 3 more authors
Tianwen Qian Mohamed Elhoseiny Yuqian Fu
This is my paper
classification cs.CV cs.AI
keywords streaming video understandingvideo language modelsobject-centric memorylatent objectsmemory compressiontraining-freevideo question answeringtemporal reasoning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper proposes that a streaming video model's memory should be organized around latent objects — clusters of visual tokens that are visually distinctive and spatially connected, discovered directly in the frozen model's representations — rather than around token importance or coarse segments. It argues that maintaining persistent object tracks, supplemented by short residual bursts when an object changes abruptly and a small window of the latest frames, lets an unchanged video language model answer both real-time and historical questions while using far fewer tokens. The method reports a 10-point gain on a real-time perception benchmark and a 2.9-point gain on another streaming benchmark, while cutting peak GPU memory and time-to-first-token by roughly half. If true, this establishes object-level structure as a practical and training-free organizing principle for compact streaming video memory.

Core claim

The central claim is that a training-free memory composed of latent-object anchors, object-conditioned temporal residuals, and a recent visual grounding window improves the streaming and offline video understanding of a frozen video LLM, and does so while cutting peak GPU memory and time-to-first-token by about half. Offline, the method surpasses the full-token baseline while discarding 82.5% of visual tokens. Latent objects are discovered by computing token saliency as cosine distance from the frame-level mean vector, keeping a per-frame quantile of the most salient tokens, and grouping them into connected components on the 4-neighbor patch grid. These components are matched across frames b

What carries the argument

The key mechanism is the latent-object anchor and its three-part memory. For each frame, normalized patch tokens are averaged into a frame-level context vector; token saliency is the cosine dissimilarity from that context. A quantile threshold selects active tokens, and 4-connected components on the patch grid define spatially coherent object candidates. Candidate features (saliency-weighted token means) are matched to existing tracks by cosine similarity; a match updates the track feature by exponential moving average, while a bounded object set is maintained by evicting the lowest-ranked tracks by recency, saliency, and temporal consistency. The long-term memory keeps the top-K saliency to

Load-bearing premise

The whole approach rests on the premise that tokens that stand out from the frame average and are spatially connected on the patch grid correspond to stable, meaningful object identities over time; if they are just salient patches, the memory is still useful but not truly object-centric.

What would settle it

Measure identity consistency of the latent tracks against ground-truth instance segmentation on a video benchmark: if tracks frequently split, merge, or swap identities across frames, the object-centric explanation is unsupported.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Streaming video models can maintain object identities and state changes across long streams without fine-tuning or external detectors, because objects emerge from the model's own frozen representations.
  • A small token budget suffices: discarding 82.5% of visual tokens offline still beats the full-token baseline, meaning memory cost and latency can be roughly halved while improving accuracy.
  • The same memory organization works in offline long-video QA, not just online streaming, suggesting object-structured memory is a general compression strategy.
  • Because it is plug-and-play, existing video LLMs can be upgraded to streaming without retraining, lowering the barrier for real-time video assistants.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The saliency-based object discovery may be biased toward visually distinctive regions, so objects that blend with the background could be under-represented; testing on a benchmark with low-contrast objects would reveal whether the object assumption truly holds.
  • If the latent tracks are not identity-consistent (e.g., they split, merge, or swap labels), the gains might come simply from selecting recent salient patches, not from object-centric organization. A direct track-consistency evaluation against ground-truth instance segmentation would separate these explanations.
  • The object memory could be combined with query-aware selection at answer time, since the anchors provide a compact index over the stream; a testable extension is retrieving object tracks conditioned on the question before generating.
  • The same anchor-discovery mechanism might transfer to other modalities (e.g., audio or tactile streams) where 'objects' are not visual, as long as token saliency and spatial/temporal locality are defined.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper presents ObjectStream, a training-free visual-memory framework for streaming video understanding. It proposes to discover 'latent objects' directly from frozen Video-LLM visual tokens by computing query-agnostic saliency as cosine dissimilarity from the frame-level mean (Eq. 3), thresholding per-frame quantiles (Eq. 4), and extracting 4-connected components of active tokens (Eq. 5). These components are represented by a saliency-weighted normalized feature (Eq. 6), associated across frames by cosine matching with an EMA update (Eqs. 7-8), and maintained under a bounded object budget with lexicographic eviction (Eq. 9). Per-object representative tokens (Eq. 10), object-conditioned temporal residuals for abrupt changes (Eqs. 11-12), and a recent-frame window (Eq. 13) constitute the retained memory (Eq. 1). Experiments on OVO-Bench, StreamingBench, VideoMME-Long, and EgoSchema report accuracy gains over Qwen2.5-VL baselines and roughly 40-50% reductions in peak memory and TTFT.

Significance. The framework is clearly specified, training-free, and evaluated on multiple benchmarks with accuracy and efficiency metrics. The code release and the explicit equations for each memory stage are strengths. If latent objects genuinely serve as persistent identity anchors, the work would be a useful step toward object-centric streaming memory. However, the evidence for the central mechanism is incomplete: no quantitative objectness or identity-stability evaluation is provided, and the strongest ablation comparison (Patch Top-K) nearly matches ObjectStream's accuracy, so the object-centric explanation of the gains is not yet established.

major comments (3)
  1. [§4, Table 4] The paper's central claim is that latent objects, not mere token selection, drive performance. But Patch Top-K achieves 72.8 on OVO-Bench versus ObjectStream's 73.3 (Δ=0.5), and the text asserts 'unstructured token selection is insufficient.' A 0.5-point difference without error bars does not support that assertion. ObjectStream is substantially faster (TTFT 6.697 vs 16.223 s), which suggests object grouping is a latency/compression win rather than an accuracy win. Please provide (i) quantitative evaluation of objectness and identity-consistency (e.g., mask/box IoU with annotated objects and ID-switch rates) on a subset with ground truth, and (ii) a matched-token/latency comparison between Patch Top-K and ObjectStream, or explicitly reframe the contribution as an efficiency-driven memory organization.
  2. [§4, Table 2] The component ablation is internally inconsistent with the text. Row 3 (RVGW+LOAM) reports OVO 60.6, which is lower than RVGW-only (62.3) and LOAM-only (62.0), and likely lower than the no-memory Qwen2.5-VL-7B baseline (63.3 from Table 1). The text says 'Combining the two consistently improves performance across both streaming and offline settings.' This needs reconciliation or correction; the table should also include the no-memory baseline and the backbone/token-budget setting.
  3. [§4, Tables 1-4] No error bars, seeds, or significance tests are reported anywhere. Several headline gains are small (StreamingBench 76.8 vs 73.9 = 2.9; EgoSchema 60.8 vs 58.5 = 2.3; Table 4 0.5), so variance could alter conclusions. Please report at least 3 seeds or bootstrap confidence intervals for the main results and ablations, especially for comparisons to Patch Top-K and FluxMem.
minor comments (4)
  1. [Eq. (7)] The time superscript is inconsistent: i* is chosen by argmax over cos(f(C_{t,m}), μ_i), but the match condition then uses μ_{t-1}^{i*}. Make the time indices explicit and consistent.
  2. [Eqs. (6), (8), (11)] Notation is unclear in places: μ_i^t is defined implicitly; TopK vs Top-K is inconsistent; the temporal index l and timestamp t_{i,l} in Eq. (11) should be defined before use.
  3. [Figures 1-2] Figure 1 contains the typo 'Backgournd'; Figure 2 has very small text and the flow is hard to follow. Table 1 also omits the Frames value for OASIS.
  4. [Table 2 caption] The caption should specify the backbone (3B or 7B), the sampling rate for online/offline, and the token budget used in the ablation, since Tables 1 and 3 use different memory parameters.

Circularity Check

0 steps flagged

No significant circularity: ObjectStream's memory construction is a deterministic heuristic evaluated against external benchmarks; no equation reduces to a fitted value or to the reported result by construction.

full rationale

The paper's derivation chain is self-contained and empirically grounded rather than circular. Saliency (Eq. 3), spatial clustering (Eq. 5), cross-frame association (Eqs. 7-8), eviction (Eq. 9), and token retention (Eq. 10) are all deterministic transformations of the frozen Video-LLM's visual features. No parameter is fitted to the benchmark answers, and no prediction is defined in terms of the target metric. The evaluation uses external benchmarks (OVO-Bench, StreamingBench, EgoSchema, VideoMME-Long), and the reported gains are comparisons against those external tasks. The use of the same frozen model's representations to discover latent objects and to answer questions is a design choice, not a logical reduction: the object clusters are not derived from or defined by the question-answer pairs. Self-citations in the related work are contextual and not load-bearing for any central claim; no uniqueness theorem or prior result by the authors is invoked to justify the method. The near-tie between ObjectStream and Patch Top-K in Table 4 weakens the paper's interpretive claim that object-centric memory is the key driver, but that is a validity/interpretability concern, not circularity, because the ObjectStream result is still produced by its stated algorithm and measured on external benchmarks. Overall, no load-bearing step reduces by construction to its own inputs.

Axiom & Free-Parameter Ledger

9 free parameters · 4 axioms · 1 invented entities

The central claim rests on several hand-set hyperparameters and on the domain assumption that saliency-based connected components in frozen VLM token space are persistent, semantically meaningful objects. The method is empirical and does not derive from first principles; its contribution is the specific memory architecture plus benchmark evaluations.

free parameters (9)
  • active-token quantile q = not reported
    Eq. 4 selects active tokens above the q-th saliency quantile; q controls object-candidate recall but no value or sensitivity analysis is given.
  • association threshold theta = not reported
    Eq. 7 requires cosine similarity >= theta for cross-frame matching; object identity persistence depends critically on theta.
  • EMA coefficient beta = not reported
    Eq. 8 updates object features; beta controls temporal smoothing versus responsiveness.
  • object budget M = 64 online, 512 offline
    Cap on maintained object tracks; set per benchmark in Implementation Details.
  • tokens per object k_i = 8 online, 80 offline
    Per-object token retention budget; larger offline budgets allow longer-range context.
  • recent-window length L = 8 online, 80 offline
    Number of raw frames retained in the Recent Visual Grounding Window; directly controls recency fidelity.
  • temporal residual radius r = 1
    Local temporal window around detected object changes.
  • tokens per residual frame k_b = 2
    Top-k raw tokens kept per selected frame in the temporal residual module.
  • max temporal events = 32
    Cap on residual events retained; not justified in the text.
axioms (4)
  • domain assumption Saliency measured as cosine dissimilarity to the frame-mean context is a valid proxy for object importance.
    Eq. 3; if background dominates or a close-up fills the frame, object tokens may not be distinctive and objects could be missed.
  • domain assumption 4-connected components of active tokens in the patch grid correspond to coherent objects.
    Eq. 5; no objectness verification is provided, only the qualitative Fig. 4.
  • domain assumption Frozen Video-LLM token features are stable enough across frames for EMA-based track matching.
    Eqs. 7-8; cosine matching is assumed to handle pose, scale, and viewpoint changes, but no tracking metrics are reported.
  • ad hoc to paper The three memory types jointly cover the information needed for streaming QA.
    Design choice; the component ablation in Tab. 2 supports it internally but no external principle guarantees completeness.
invented entities (1)
  • latent object anchors no independent evidence
    purpose: Persistent memory units that organize retained visual tokens by object identity.
    The paper introduces this abstraction and validates it only through downstream QA benchmarks; it gives no external handle to verify that the units are actually objects.

pith-pipeline@v1.3.0-alltime-deepseek · 14394 in / 11514 out tokens · 372393 ms · 2026-08-04T03:15:09.952264+00:00 · methodology

0 comments
read the original abstract

Streaming video understanding requires models to continuously retain useful visual evidence before future questions are known. Existing approaches primarily manage the growing visual context according to token importance, temporal redundancy, or segment-level relevance, but rarely organize evidence around objects that persist and evolve over time. Thus, in this paper, we introduce ObjectStream, a training-free framework that treats latent objects as memory anchors for streaming video understanding. ObjectStream induces spatially coherent latent objects directly from frozen Video-LLM representations, links them across frames into persistent anchors, and maintains their histories under a bounded memory budget, without requiring external object detectors or segmentation models. Built on these anchors, ObjectStream preserves three complementary forms of evidence: persistent object histories, transient object changes, and recent visual context. This design enables existing Video Large Language Models (Video-LLMs) to reason over object identities, interactions, and state changes while leaving the underlying model unchanged. Extensive experiments on online streaming and offline long-video benchmarks demonstrate both effectiveness and efficiency. In online streaming evaluation, ObjectStream improves Qwen2.5-VL-7B by 10.0 points on OVO-Bench Real-Time Visual Perception, while reducing peak GPU mem-ory and TTFT by approximately 50%. On offline long-video benchmarks, it surpasses the full-token baseline while discarding 82.5% of visual tokens. These results highlight latent objects as a practical and effective organizing principle for compact streaming video memory.

Figures

Figures reproduced from arXiv: 2607.28312 by Bin Ren, Bohan Guo, Chen Zhao, Jie Li, Mingkang Dong, Mohamed Elhoseiny, Muxin Pu, Songruo Chen, Tianwen Qian, Xu Zheng, Yuqian Fu.

Figure 1
Figure 1. Figure 1: Comparison between existing streaming memory [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of ObjectStream. It comprises three memory modules: [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Illustration of latent object discovery. ObjectStream [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative analysis of latent object discovery. ObjectStream maintains task-relevant object states across consecutive [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

67 extracted references · 39 linked inside Pith

  1. [1]

    Bai, S.; Chen, K.; Liu, X.; Wang, J.; Ge, W.; Song, S.; Dang,K.;Wang,P.;Wang,S.;Tang,J.;Zhong,H.;Zhu, Y.; Yang, M.; Li, Z.; Wan, J.; Wang, P.; Ding, W.; Fu, Z.;Xu,Y.;Ye,J.;Zhang,X.;Xie,T.;Cheng,Z.;Zhang, H.; Yang, Z.; Xu, H.; and Lin, J. 2025. Qwen2.5-VL Technical Report. arXiv:2502.13923

    Pith/arXiv arXiv 2025

  2. [2]

    Brödermann, T.; Sakaridis, C.; Fu, Y.; and Van Gool, L. 2024. Cafuser: Condition-aware multimodal fusion forrobustsemanticperceptionofdrivingscenes.arXiv preprint arXiv:2410.10791

    Pith/arXiv arXiv 2024

  3. [3]

    Q.; Song, C.; Gao, D.; Liu, J.-W.; Gao, Z.; Mao, D.; and Shou, M

    Chen, J.; Lv, Z.; Wu, S.; Lin, K. Q.; Song, C.; Gao, D.; Liu, J.-W.; Gao, Z.; Mao, D.; and Shou, M. Z

  4. [4]

    Chen, X.; Tao, K.; Shao, K.; and Wang, H. 2026. StreamingTOM:StreamingTokenCompressionforEf- ficient Video Understanding. arXiv:2510.18269

    arXiv 2026

  5. [5]

    Chen, Y.; Bai, X.; Wang, Z.; Bai, C.; Dai, Y.; Lu, M.; and Zhang, S. 2025. StreamKV: Streaming Video Question-Answering with Segment-based KV Cache Retrieval and Compression. arXiv:2511.07278

    arXiv 2025

  6. [6]

    Chen, Z.; Wang, W.; Cao, Y.; Liu, Y.; Gao, Z.; Cui, E.; Zhu, J.; Ye, S.; Tian, H.; Liu, Z.; Gu, L.; Wang, X.; Li, Q.; Ren, Y.; Chen, Z.; Luo, J.; Wang, J.; Jiang, T.; Wang, B.; He, C.; Shi, B.; Zhang, X.; Lv, H.; Wang, Y.; Shao, W.; Chu, P.; Tu, Z.; He, T.; Wu, Z.; Deng, H.; Ge, J.; Chen, K.; Zhang, K.; Wang, L.; Dou, M.; Lu, L.; Zhu, X.; Lu, T.; Lin, D.;...

    Pith/arXiv arXiv 2025

  7. [7]

    Di,S.;Yu,Z.;Zhang,G.;Li,H.;Zhong,T.;Cheng,H.; Li, B.; He, W.; Shu, F.; and Jiang, H. 2025. Streaming VideoQuestion-AnsweringwithIn-contextVideoKV- Cache Retrieval. arXiv:2503.00540

    Pith/arXiv arXiv 2025

  8. [8]

    Du,Y.;Lin,T.;Zhong,Z.;Li,R.;Chen,X.;Liu,J.;Liu, C.; Chen, Y.-C.; Fu, Y.; and Zhao, B. 2026. Focusable Monocular Depth Estimation. arXiv:2605.11756

    Pith/arXiv arXiv 2026

  9. [9]

    Dumitru, E.; and Spînu, S. 2026. A Multi-Task Deep Learning Framework for Real-Time Intelligent Video Surveillance with Temporal Event Validation. arXiv:2607.03131

    Pith/arXiv arXiv 2026

  10. [10]

    Fu, C.; Dai, Y.; Luo, Y.; Li, L.; Ren, S.; Zhang, R.; Wang, Z.; Zhou, C.; Shen, Y.; Zhang, M.; Chen, P.; Li, Y.; Lin, S.; Zhao, S.; Li, K.; Xu, T.; Zheng, X.; Chen, E.; Shan, C.; He, R.; and Sun, X. 2025. Video-MME: The First-Ever Comprehensive Evalua- tion Benchmark of Multi-modal LLMs in Video Anal- ysis. arXiv:2405.21075

    Pith/arXiv arXiv 2025

  11. [11]

    arXiv:2503.12769

    Fu, S.; Yang, Q.; Li, Y.-M.; Peng, Y.-X.; Lin, K.-Y.; Wei,X.;Hu,J.-F.;Xie,X.;andZheng,W.-S.2025.ViS- peak:VisualInstructionFeedbackinStreamingVideos. arXiv:2503.12769

    Pith/arXiv arXiv 2025

  12. [12]

    Fu, Y.; Wang, C.; Fu, Y.; Wang, Y.-X.; Bai, C.; Xue, X.; and Jiang, Y.-G. 2019. Embodied one-shot video recognition:Learningfromactionsofavirtualembod- iedagent.InProceedingsofthe27thACMinternational conference on multimedia

    2019

  13. [13]

    P.; Huang, X.; and Van Gool, L

    Fu, Y.; Wang, R.; Ren, B.; Sun, G.; Gong, B.; Fu, Y.; Paudel, D. P.; Huang, X.; and Van Gool, L. 2025. Ob- jectrelator: Enabling cross-view object relation under- standing across ego-centric and exo-centric perspec- tives. InProceedings of the IEEE/CVF International Conference on Computer Vision

    2025

  14. [14]

    Ge, H.; Wang, Y.; Wu, H.; and Cai, Y. 2026. What Should a Streaming Video Model Remember? arXiv:2606.16353

    arXiv 2026

  15. [15]

    Guan, Y.; Yin, L.; Liang, D.; Ju, J.; Luo, Z.; Luan, J.; Liu, Y.; and Bai, X. 2026. Video Streaming Think- ing:VideoLLMsCanWatchandThinkSimultaneously. arXiv:2603.12262

    Pith/arXiv arXiv 2026

  16. [16]

    Li, F.; Zhang, C.; and Chen, G. 2026. Sparse- Aware Vector Quantization for Bandwidth-Efficient Collaborative 3D Semantic Occupancy Prediction. arXiv:2607.01928

    Pith/arXiv arXiv 2026

  17. [17]

    P.; Van Gool, L.; and Wang, X

    Li, Y.; Fu, Y.; Qian, T.; Xu, Q.; Dai, S.; Paudel, D. P.; Van Gool, L.; and Wang, X. 2026. Egocross: Bench- marking multimodal large language models for cross- domain egocentric video question answering. InPro- ceedings of the AAAI Conference on Artificial Intelli- gence

    2026

  18. [18]

    Li, Y.; Niu, J.; Miao, Z.; Ge, C.; Zhou, Y.; He, Q.; Dong, X.; Duan, H.; Ding, S.; Qian, R.; Zhang, P.; Zang, Y.; Cao, Y.; He, C.; and Wang, J. 2025. OVO- Bench:HowFarisYourVideo-LLMsfromReal-World Online Video Understanding? arXiv:2501.05510

    Pith/arXiv arXiv 2025

  19. [19]

    Liang,Z.;Li,J.;Chen,W.;Zhang,Y.;Lu,H.;andLi,G

  20. [20]

    Lin, J.; Fang, Z.; Chen, C.; Wan, Z.; Luo, F.; Li, P.; Liu, Y.; and Sun, M. 2024. StreamingBench: Assess- ing the Gap for MLLMs to Achieve Streaming Video Understanding. arXiv:2411.03628

    Pith/arXiv arXiv 2024

  21. [21]

    Lin, T.; Du, Y.; Liu, J.; Zhu, N.; Li, Y.; Fu, Y.; Chen, Y.; Cai, H.; Ye, Z.; Cheng, B.; et al. 2026. Evo-depth: A lightweight depth-enhanced vision-language-action model.arXiv preprint arXiv:2605.14950

    Pith/arXiv arXiv 2026

  22. [22]

    Lin, T.; Du, Y.; Mao, Y.; Ye, Z.; Zhong, Y.; Cheng, B.; Wang, Y.; Liu, J.; Tian, Y.; Yan, J.; et al

  23. [23]

    Lin, T.; Zhong, Y.; Du, Y.; Zhang, J.; Liu, J.; Chen, Y.; Gu, E.; Liu, Z.; Cai, H.; Zou, Y.; et al

  24. [24]

    Liu, J.; Yu, Z.; Lan, S.; Wang, S.; Fang, R.; Kautz, J.; Li, H.; and Alvare, J. M. 2025. StreamChat: Chatting with Streaming Video. arXiv:2412.08646

    Pith/arXiv arXiv 2025

  25. [25]

    LA4VLA: Learning to Act without See- ing via Language-Action Pretraining.arXiv preprint arXiv:2606.27295

    Pith/arXiv arXiv

  26. [26]

    Lu, X.; Bo, Y.; Chen, J.; Li, S.; Guo, X.; Guan, H.; Liu, F.; Xu, D.; Sun, P.; Sun, H.; Liu, R.; and Li, H

  27. [27]

    Maaz, M.; Rasheed, H.; Khan, S.; and Khan, F. S

  28. [28]

    Mangalam, K.; Akshulakov, R.; and Malik, J

  29. [29]

    Liu, Z.; Guo, L.; Li, H.; Zhen, R.; He, X.; Ji, R.; Ren, X.; Zhang, Y.; Lu, H.; and Liu, J. 2026. Thinking in Streaming Video. arXiv:2603.12938

    arXiv 2026

  30. [30]

    P.; and Fu, Y

    Pan, J.; Wang, R.; Qian, T.; Mahdi, M.; Fu, Y.; Xue, X.; Huang, X.; Van Gool, L.; Paudel, D. P.; and Fu, Y

  31. [31]

    AURA: Always-On Understanding and Real- TimeAssistanceviaVideoStreams.arXiv:2604.04184

    Pith/arXiv arXiv

  32. [32]

    Qian, R.; Ding, S.; Dong, X.; Zhang, P.; Zang, Y.; Cao, Y.; Lin, D.; and Wang, J. 2025. Dispider: En- abling Video LLMs with Active Real-Time Interaction via Disentangled Perception, Decision, and Reaction. arXiv:2501.03218

    Pith/arXiv arXiv 2025

  33. [33]

    arXiv:2306.05424

    Video-ChatGPT: Towards Detailed Video Un- derstanding via Large Vision and Language Models. arXiv:2306.05424

    Pith/arXiv arXiv

  34. [34]

    Shen, Y.; Tian, S.; Yang, J.; and Liu, Z. 2026. A Simple Baseline for Streaming Video Understanding. arXiv:2604.02317

    arXiv 2026

  35. [35]

    Team, G.; and Vinyals, P. G. O. 2024. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv:2403.05530

    Pith/arXiv arXiv 2024

  36. [36]

    arXiv:2410.21276

    OpenAI:;Dai,Y.;andMalkov,Y.2024.GPT-4oSystem Card. arXiv:2410.21276

    Pith/arXiv arXiv 2024

  37. [37]

    Wang, K.; Fan, K.; Qiu, C.; Shangguan, Z.; Fu, Y.; Fu, Y.; Seita, D.; and Xue, X. 2026. Oflow: Injecting object-awaretemporalflowmatchingforrobustrobotic manipulation.arXiv preprint arXiv:2604.17876

    Pith/arXiv arXiv 2026

  38. [38]

    V2-SAM: Marrying SAM2 with Multi-Prompt ExpertsforCross-ViewObjectCorrespondence.arXiv preprint arXiv:2511.20886

    Pith/arXiv arXiv

  39. [39]

    Plizzari, C.; Goletto, G.; Furnari, A.; Bansal, S.; Ra- gusa,F.;Farinella,G.M.;Damen,D.;andTommasi,T

  40. [40]

    arXiv:2308.07123

    AnOutlookintotheFutureofEgocentricVision. arXiv:2308.07123

    Pith/arXiv arXiv

  41. [41]

    Wang,Y.;Li,Z.;Qian,T.;Zheng,H.;Wang,Z.;Fu,Y.; and Wang, X. 2025. StreamEQA: Towards Streaming Video Understanding for Embodied Scenarios.arXiv preprint arXiv:2512.04451

    arXiv 2025

  42. [42]

    J.; Soran, B.; Krishnamoorthi, R.; Elhoseiny, M.; and Chandra, V

    Shen, X.; Xiong, Y.; Zhao, C.; Wu, L.; Chen, J.; Zhu, C.; Liu, Z.; Xiao, F.; Varadarajan, B.; Bordes, F.; Liu, Z.; Xu, H.; Kim, H. J.; Soran, B.; Krishnamoorthi, R.; Elhoseiny, M.; and Chandra, V. 2024. LongVU: SpatiotemporalAdaptiveCompressionforLongVideo- Language Understanding. arXiv:2410.17434

    Pith/arXiv arXiv 2024

  43. [43]

    FluxMem:AdaptiveHierarchicalMemoryfor Streaming Video Understanding

    Xie, Y.; He, B.; Wang, J.; Zheng, X.; Ye, Z.; and Wu, Z.2026. FluxMem:AdaptiveHierarchicalMemoryfor Streaming Video Understanding. arXiv:2603.02096

    arXiv 2026

  44. [44]

    Xu, Q.; Qian, T.; Fu, Y.; Li, K.; Jiao, Y.; Zhang, J.; Wang, X.; and He, L. 2025. ToG-Bench: Task- Oriented Spatio-Temporal Grounding in Egocentric Videos.arXiv preprint arXiv:2512.03666

    Pith/arXiv arXiv 2025

  45. [45]

    Wang,K.;Fan,K.;Fu,Y.;Lin,S.;Luo,H.;Seita,D.;Fu, Y.;Jiang,Y.-G.;andXue,X.2026.Ocra:Object-centric learning with 3d and tactile priors for human-to-robot action transfer.arXiv preprint arXiv:2603.14401

    arXiv 2026

  46. [46]

    TimeChat-Online:80 arXiv:2504.17343

    Yao, L.; Li, Y.; Wei, Y.; Li, L.; Ren, S.; Liu, Y.; Ouyang, K.; Wang, L.; Li, S.; Li, S.; Kong, L.; Liu, Q.;Zhang,Y.;andSun,X.2025. TimeChat-Online:80 arXiv:2504.17343

    Pith/arXiv arXiv 2025

  47. [47]

    Wang, P.; Bai, S.; Tan, S.; Wang, S.; Fan, Z.; Bai, J.; Chen,K.;Liu,X.;Wang,J.;Ge,W.;Fan,Y.;Dang,K.; Du, M.; Ren, X.; Men, R.; Liu, D.; Zhou, C.; Zhou, J.; and Lin, J. 2024. Qwen2-VL: Enhancing Vision- LanguageModel’sPerceptionoftheWorldatAnyRes- olution. arXiv:2409.12191

    Pith/arXiv arXiv 2024

  48. [48]

    Wang, R.; Fu, Y.; Li, Y.; Lin, T.; Qian, T.; Elhoseiny, M.; Zhao, B.; Fu, Y.; Jiang, Y.-G.; and Xue, X. 2026. Afford-VLA:Action-AlignedVisualPlanningviaInter- nalized Affordance.arXiv preprint arXiv:2605.24203

    Pith/arXiv arXiv 2026

  49. [49]

    HoloAssist:an Egocentric Human Interaction Dataset for Interactive AI Assistants in the Real World

    Wang,X.;Kwon,T.;Rad,M.;Pan,B.;Chakraborty,I.; Andrist, S.; Bohus, D.; Feniello, A.; Tekin, B.; Frujeri, F.V.;Joshi,N.;andPollefeys,M.2023. HoloAssist:an Egocentric Human Interaction Dataset for Interactive AI Assistants in the Real World. arXiv:2309.17024

    Pith/arXiv arXiv 2023

  50. [50]

    Zhang, H.; Wang, Y.; Tang, Y.; Liu, Y.; Feng, J.; Dai, J.; and Jin, X. 2024. Flash-VStream: Memory-Based Real-Time Understanding for Long Video Streams. arXiv:2406.08085

    Pith/arXiv arXiv 2024

  51. [51]

    M.; Cai, Y.; Yang, M.-H.; and Wang, Y

    Wu, H.; Mathews, S. M.; Cai, Y.; Yang, M.-H.; and Wang, Y. 2026. Semantic-Aware Adaptive Vi- sual Memory for Streaming Video Understanding. arXiv:2605.07897

    Pith/arXiv arXiv 2026

  52. [52]

    Zhang, K.; Yang, Z.; Wang, B.; Qian, S.; and Xu, C

  53. [53]

    Zhang, Y.; Wu, J.; Li, W.; Li, B.; Ma, Z.; Liu, Z.; and Li, C. 2025. LLaVA-Video: Video Instruction Tuning With Synthetic Data. arXiv:2410.02713

    Pith/arXiv arXiv 2025

  54. [54]

    Xu, R.; Xiao, G.; Chen, Y.; He, L.; Lu, Y.; and Han, S

  55. [55]

    arXiv:2510.09608

    StreamingVLM: Real-Time Understanding for Infinite Video Streams. arXiv:2510.09608

    Pith/arXiv arXiv

  56. [56]

    Zou, Z.; Sun, G.; Wei, Z.; Pan, J.; Li, Y.; Peng, M.; and Xu, W. 2026. Self in Space: Benchmarking Self- Awareness and Spatial Cognition in UAV Embodied Intelligence. arXiv:2607.12477

    Pith/arXiv arXiv 2026

  57. [57]

    Zeng, X.; Qiu, K.; Zhang, Q.; Li, X.; Wang, J.; Li, J.; Yan, Z.; Tian, K.; Tian, M.; Zhao, X.; Wang, Y.; and Wang, L. 2025. StreamForest: Efficient Online Video Understanding with Persistent Event Memory. arXiv:2509.24871

    arXiv 2025

  58. [58]

    Egonight:Towardsegocentricvisionunder- standing at night with a challenging benchmark.arXiv preprint arXiv:2510.06218

    Zhang,D.;Fu,Y.;Yang,R.;Miao,Y.;Qian,T.;Zheng, X.; Sun, G.; Chhatkuli, A.; Huang, X.; Jiang, Y.-G.; etal.2025. Egonight:Towardsegocentricvisionunder- standing at night with a challenging benchmark.arXiv preprint arXiv:2510.06218

    arXiv 2025

  59. [59]

    P.; Gool, L

    Zhang, G.; Zhang, D.; Duan, L.; Han, G.; Fu, Y.; Paudel, D. P.; Gool, L. V.; and Vollenweider, E. 2026. EgoSpot:Egocentric Multimodal Control for Hands- Free Mobile Manipulation. arXiv:2306.02393

    arXiv 2026

  60. [61]

    Zhang, H.; Yang, S.; Fu, J.; Ng, S.-K.; and Qiu, X. 2026. HERMES: KV Cache as Hierarchical Memory for Efficient Streaming Video Understanding. arXiv:2601.14724

    Pith/arXiv arXiv 2026

  61. [63]

    InTheFourteenthInternationalConference on Learning Representations

    QueryStream: Advancing Streaming Video Un- derstanding with Query-Aware Pruning and Proactive Response. InTheFourteenthInternationalConference on Learning Representations

  62. [65]

    Zhao, J.; Wu, Y.; Deng, R.; Xu, S.; Gao, J.; and Burke, A. 2025. A survey of autonomous driving from a deep learning perspective.ACM Computing Surveys

    2025

  63. [66]

    P.; Xue, X.; and Fu, Y

    Zhu, B.; Fu, Y.; Dong, Q.; Sun, G.; Qian, T.; Wu, Y.; Paudel, D. P.; Xue, X.; and Fu, Y. 2026. Egosound:Benchmarkingsoundunderstandinginego- centric videos.arXiv preprint arXiv:2602.14122

    Pith/arXiv arXiv 2026

  64. [2023]

    arXiv:2308.09126

    EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understanding. arXiv:2308.09126

    Pith/arXiv arXiv

  65. [2024]

    arXiv:2406.11816

    VideoLLM-online: Online Video Large Lan- guage Model for Streaming Video. arXiv:2406.11816

    Pith/arXiv arXiv

  66. [2025]

    Evo-1: Lightweight vision-language-action model with preserved semantic alignment.arXiv preprint arXiv:2511.04555

    arXiv

  67. [2026]

    OASIS: On-Demand Hierarchical Event Mem- oryforStreamingVideoReasoning.arXiv:2604.17052

    Pith/arXiv arXiv