Pith. sign in

REVIEW 4 cited by

MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understanding

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2404.05726 v2 pith:MMGBJHTE submitted 2024-04-08 cs.CV

Bo He , Hengduo Li , Young Kyun Jang , Menglin Jia , Xuefei Cao , Ashish Shah , Abhinav Shrivastava , Ser-Nam Lim This is my paper
classification cs.CV
keywords videomodelunderstandingllmslargelong-termmemorymodels
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

With the success of large language models (LLMs), integrating the vision model into LLMs to build vision-language foundation models has gained much more interest recently. However, existing LLM-based large multimodal models (e.g., Video-LLaMA, VideoChat) can only take in a limited number of frames for short video understanding. In this study, we mainly focus on designing an efficient and effective model for long-term video understanding. Instead of trying to process more frames simultaneously like most existing work, we propose to process videos in an online manner and store past video information in a memory bank. This allows our model to reference historical video content for long-term analysis without exceeding LLMs' context length constraints or GPU memory limits. Our memory bank can be seamlessly integrated into current multimodal LLMs in an off-the-shelf manner. We conduct extensive experiments on various video understanding tasks, such as long-video understanding, video question answering, and video captioning, and our model can achieve state-of-the-art performances across multiple datasets. Code available at https://boheumd.github.io/MA-LMM/.

Discussion (0). Sign in to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Task-Aware KV Compression For Cost-Effective Long Video Understanding

    cs.CV 2025-06 conditional novelty 5.0 of 10

    Video-X2L uses bi-level KV compression with task-aware selective reloading to improve long-video QA accuracy and reduce decode-time memory versus uniform KV compression.

  2. Video-XL-2: Towards Very Long-Video Understanding Through Task-Aware KV Sparsification

    cs.CV 2025-06 conditional novelty 5.0 of 10

    Video-XL-2 cuts long-video inference cost with chunked pre-filling and query-gated dense-or-sparse KV reloading, reporting half the FLOPs and a third less decoding memory at roughly equal benchmark scores.

  3. Towards General Continuous Memory for Vision-Language Models

    cs.LG 2025-05 conditional novelty 5.0 of 10

    A vision-language model can act as its own continuous memory encoder, compressing external multimodal knowledge into eight embeddings that improve reasoning when prepended to the frozen model.

  4. ViFusion: In-Network Tensor Fusion for Scalable Video Feature Indexing

    cs.MM 2025-06 reject novelty 4.0 of 10

    ViFusion combines dynamic tensor fusion with hierarchical AllReduce to speed up distributed video feature indexing, but the 8-22x throughput claim is an overstatement of bandwidth gains over a self-defined baseline.

Pith tools