pith. sign in

arxiv: 2406.00258 · v1 · pith:QVHAUVK2new · submitted 2024-06-01 · 💻 cs.CV · cs.AI

Artemis: Towards Referential Understanding in Complex Videos

Jihao Qiu , Yuan Zhang , Xi Tang , Lingxi Xie , Tianren Ma , Pengyu Yan , David Doermann , Qixiang Ye
show 1 more author
Yunjie Tian
This is my paper
classification 💻 cs.CV cs.AI
keywords videoartemisreferentialunderstandingcomplexfeaturesscenariosvideo-based
0
0 comments X
read the original abstract

Videos carry rich visual information including object description, action, interaction, etc., but the existing multimodal large language models (MLLMs) fell short in referential understanding scenarios such as video-based referring. In this paper, we present Artemis, an MLLM that pushes video-based referential understanding to a finer level. Given a video, Artemis receives a natural-language question with a bounding box in any video frame and describes the referred target in the entire video. The key to achieving this goal lies in extracting compact, target-specific video features, where we set a solid baseline by tracking and selecting spatiotemporal features from the video. We train Artemis on the newly established VideoRef45K dataset with 45K video-QA pairs and design a computationally efficient, three-stage training procedure. Results are promising both quantitatively and qualitatively. Additionally, we show that \model can be integrated with video grounding and text summarization tools to understand more complex scenarios. Code and data are available at https://github.com/qiujihao19/Artemis.

This paper has not been read by Pith yet.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. TRACE: Evidence Grounding-Guided Multi-Video Event Understanding and Claim Generation

    cs.CV 2026-05 unverdicted novelty 6.0

    TRACE builds structured text timelines from videos via OCR and detection, then applies text-only LLM evidence localization before LVLM claim generation, raising MiRAGE F1 from 0.705 to 0.811 on MAGMaR.

  2. TRACE: Evidence Grounding-Guided Multi-Video Event Understanding and Claim Generation

    cs.CV 2026-05 unverdicted novelty 6.0

    TRACE improves multi-video event understanding by grounding evidence in structured timelines before visual reasoning, raising MiRAGE F1 from 0.705 to 0.811 on MAGMaR 2026.

  3. Intend, Reflect, Refine: An Adaptive Multimodal Reflection Framework for Autonomous Driving

    cs.CV 2026-06 unverdicted novelty 4.0

    IRR-Drive adds an adaptive multimodal reflection step (text intention plus predicted future BEV) that lets a VLA model self-correct its trajectory plan according to scene complexity and reports SOTA on NAVSIM.

  4. LMMs Meet Object-Centric Vision: Understanding, Segmentation, Editing and Generation

    cs.CV 2026-04 unverdicted novelty 3.0

    This review organizes literature on large multimodal models and object-centric vision into four themes—understanding, referring segmentation, editing, and generation—while summarizing paradigms, strategies, and challe...