Pith. sign in

REVIEW 10 cited by

MovieChat+: Question-aware Sparse Memory for Long Video Question Answering

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2404.17176 v1 pith:KHUIB7ST submitted 2024-04-26 cs.CV

Enxin Song , Wenhao Chai , Tian Ye , Jenq-Neng Hwang , Xi Li , Gaoang Wang This is my paper
classification cs.CV
keywords videolongmemorymodelsmoviechattemporalunderstandingadditional
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Recently, integrating video foundation models and large language models to build a video understanding system can overcome the limitations of specific pre-defined vision tasks. Yet, existing methods either employ complex spatial-temporal modules or rely heavily on additional perception models to extract temporal features for video understanding, and they only perform well on short videos. For long videos, the computational complexity and memory costs associated with long-term temporal connections are significantly increased, posing additional challenges.Taking advantage of the Atkinson-Shiffrin memory model, with tokens in Transformers being employed as the carriers of memory in combination with our specially designed memory mechanism, we propose MovieChat to overcome these challenges. We lift pre-trained multi-modal large language models for understanding long videos without incorporating additional trainable temporal modules, employing a zero-shot approach. MovieChat achieves state-of-the-art performance in long video understanding, along with the released MovieChat-1K benchmark with 1K long video, 2K temporal grounding labels, and 14K manual annotations for validation of the effectiveness of our method. The code along with the dataset can be accessed via the following https://github.com/rese1f/MovieChat.

Discussion (0). Sign in to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. HumanVBench: Probing Human-Centric Video Understanding in MLLMs with Automatically Synthesized Benchmarks

    cs.CV 2024-12 unverdicted novelty 7.0 of 10

    HumanVBench provides a 16-task benchmark for human-centric video understanding in MLLMs, created through automated annotation and distractor synthesis pipelines, and shows top models lag human performance on emotion p...

  2. Don't Pause: Streaming Video-Language Synchrony for Online Video Understanding

    cs.CV 2026-06 unverdicted novelty 6.0 of 10

    LyraV uses FDTC and SToP for per-frame incremental decoding to reach 98.29% video synchrony at 3.89 FPS while preserving general understanding.

  3. AdsQA: Towards Advertisement Video Understanding

    cs.CV 2025-09 conditional novelty 6.0 of 10

    AdsQA adds an ad-video question-answering benchmark and ReAd-R, a GRPO-trained model that beats 7B baselines but not larger closed models.

  4. InterAct-Video: Reasoning-Rich Video QA for Urban Traffic

    cs.CV 2025-07 conditional novelty 6.0 of 10

    A new traffic-intersection VideoQA benchmark containing roughly 28,800 human-verified GPT-seeded QA pairs, with evaluations showing fine-tuning improves three video-language models.

  5. AuroraLong: Bringing RNNs Back to Efficient Open-Ended Video Understanding

    cs.CV 2025-07 conditional novelty 6.0 of 10

    A 2B-parameter video-language model using an RWKV linear-RNN backbone and sorted token merging achieves competitive long-video QA accuracy with far lower memory cost than transformer-based models.

  6. LiveVLM: Efficient Online Video Understanding via Streaming-Oriented KV Cache and Retrieval

    cs.CV 2025-05 unverdicted novelty 6.0 of 10

    LiveVLM introduces VSB and PaR to compress and retrieve KV cache in streaming video LLMs, enabling LLaVA-OneVision to reach SOTA accuracy among training-free query-agnostic and training-based online models.

  7. LiveStarPro: Proactive Streaming Video Understanding with Hierarchical Memory for Long-Horizon Streams

    cs.CV 2026-06 unverdicted novelty 5.0 of 10

    LiveStarPro uses SVeD for response timing via perplexity, SCAM for incremental alignment, and TSHM for event-chain memory to achieve 28.9% better semantic correctness and 1.58x speedup on long video streams.

  8. StreamMem: Query-Agnostic KV Cache Memory for Streaming Video Understanding

    cs.CV 2025-08 conditional novelty 5.0 of 10

    A training-free, query-agnostic KV cache compression method for streaming video MLLMs, using chat-template attention as a saliency proxy, matches or beats prior streaming methods at a fixed 6K memory budget.

  9. InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Output

    cs.CV 2024-07 conditional novelty 5.0 of 10

    InternLM-XComposer-2.5 is a 7B vision-language model supporting up to 96K context that reaches GPT-4V-level performance on image, video, and multi-turn tasks and adds LoRA-driven text-image composition capabilities.

  10. VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understanding

    cs.CV 2025-01 unverdicted novelty 4.0 of 10

    VideoLLaMA3 uses a vision-centric training paradigm and token-reduction design to reach competitive results on image and video benchmarks.

Pith tools