REVIEW 6 cited by
Video-XL-Pro: Reconstructive Token Compression for Extremely Long Video Understanding
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Despite advanced token compression techniques, existing multimodal large language models (MLLMs) still struggle with hour-long video understanding. In this work, we propose Video-XL-Pro, an efficient method for extremely long video understanding, built upon Reconstructive Compression of Tokens (ReCoT), a learnable module that leverages self-supervised learning to generate comprehensive and compact video tokens. ReCoT introduces two key components: (i) Dynamic Token Synthesizer (DTS): DTS generates pseudo-video tokens from static image tokens by learning intra-token relationships, which are then used in masked video modeling. (ii) Semantic-Guided Masking (SGM): SGM adaptively masks redundant visual tokens to facilitate more effective reconstructive learning. To improve training efficiency in MLLMs fine-tuning, we introduce a video-specific dataset pruning strategy and design a simple yet Query-aware Selector that enables the model to precisely locate query-relevant video tokens. With only 3B parameters, Video-XL-Pro outperforms most 7B models trained on larger datasets across multiple long video understanding benchmarks. Moreover, it can process over 8K frames on a single A100 GPU while maintaining high-quality performance.
Forward citations
Cited by 6 Pith papers
-
TrajTok: Learning Trajectory Tokens enables better Video Understanding
TrajTok learns to tokenize video into object-trajectory tokens end-to-end, improving video CLIP, probing, and VLM performance over patch and token-merging baselines.
-
ViSAGE: Constructing Self-Correcting Memories for Long-Form Video Understanding
ViSAGE builds entity-centered, self-correcting memories for long-form video understanding and reports state-of-the-art accuracy on M3-Bench and Video-MME-long.
-
StreamMem: Query-Agnostic KV Cache Memory for Streaming Video Understanding
A training-free, query-agnostic KV cache compression method for streaming video MLLMs, using chat-template attention as a saliency proxy, matches or beats prior streaming methods at a fixed 6K memory budget.
-
Task-Aware KV Compression For Cost-Effective Long Video Understanding
Video-X2L uses bi-level KV compression with task-aware selective reloading to improve long-video QA accuracy and reduce decode-time memory versus uniform KV compression.
-
Video-XL-2: Towards Very Long-Video Understanding Through Task-Aware KV Sparsification
Video-XL-2 cuts long-video inference cost with chunked pre-filling and query-gated dense-or-sparse KV reloading, reporting half the FLOPs and a third less decoding memory at roughly equal benchmark scores.
-
Infinite Video Understanding
The paper argues that video understanding research should aim at processing streams of arbitrary, unbounded duration and outlines the challenges, directions, and metrics needed.
Discussion (0). Sign in to comment.