Pith. sign in

REVIEW 6 cited by

VideoAgent: A Memory-augmented Multimodal Agent for Video Understanding

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2403.11481 v2 pith:T3TF6VUG submitted 2024-03-18 cs.CV

Yue Fan , Xiaojian Ma , Rujie Wu , Yuntao Du , Jiaqi Li , Zhi Gao , Qing Li This is my paper · ORCID
classification cs.CV
keywords modelsvideomemoryunderstandingvideoagentagentfoundationincluding
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

We explore how reconciling several foundation models (large language models and vision-language models) with a novel unified memory mechanism could tackle the challenging video understanding problem, especially capturing the long-term temporal relations in lengthy videos. In particular, the proposed multimodal agent VideoAgent: 1) constructs a structured memory to store both the generic temporal event descriptions and object-centric tracking states of the video; 2) given an input task query, it employs tools including video segment localization and object memory querying along with other visual foundation models to interactively solve the task, utilizing the zero-shot tool-use ability of LLMs. VideoAgent demonstrates impressive performances on several long-horizon video understanding benchmarks, an average increase of 6.6% on NExT-QA and 26.0% on EgoSchema over baselines, closing the gap between open-sourced models and private counterparts including Gemini 1.5 Pro.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. MAGNET: A Multi-agent Framework for Finding Audio-Visual Needles by Reasoning over Multi-Video Haystacks

    cs.CV 2025-06 conditional novelty 6.0 of 10

    AVHaystacks is a new 3100-question benchmark for audio-visual QA across 500 videos, and the MAGNET multi-agent pipeline beats current baselines on it.

  2. LongViTU: Instruction Tuning for Long-Form Video Understanding

    cs.CV 2025-01 conditional novelty 6.0 of 10

    LongViTU, an automatically generated 121k-pair video QA dataset with 4.6-minute average certificate length, provides modest SFT gains on long-video benchmarks.

  3. Less Data, Faster Convergence: Goal-Driven Data Optimization for Multimodal Instruction Tuning

    cs.CV 2026-03 conditional novelty 5.5 of 10

    Goal-driven selection of 1× multimodal instruction subsets reaches a 512k Uni-10x baseline after ~27–35k samples and improves accuracy by up to +3.08 pp under a fixed Qwen3-VL recipe.

  4. Towards Sparse Video Understanding and Reasoning

    cs.CV 2026-02 conditional novelty 5.0 of 10

    A video-QA agent that carries only a structured text summary between rounds beats dense-frame baselines on accuracy while using a handful of frames per video.

  5. Augmented Vision-Language Models: A Systematic Review

    cs.CL 2025-07 conditional novelty 5.0 of 10

    A structured taxonomy of inference-time augmentation techniques that connect vision-language models to external symbolic systems, tools, and knowledge sources.

  6. DIVE: Deep-search Iterative Video Exploration A Technical Report for the CVRR Challenge at CVPR 2025

    cs.CV 2025-06 conditional novelty 5.0 of 10

    DIVE, an iterative question-decomposition system with intent estimation and object-centric video summarization, achieves 81.44% on CVRR-ES.

Pith tools