Pith. sign in

REVIEW 6 cited by

xGen-MM-Vid (BLIP-3-Video): You Only Need 32 Tokens to Represent a Video Even in VLMs

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2410.16267 v2 pith:JCMH23DT submitted 2024-10-21 cs.CV cs.AIcs.CLcs.LG

Michael S. Ryoo , Honglu Zhou , Shrikant Kendre , Can Qin , Le Xue , Manli Shu , Jongwoo Park , Kanchana Ranasinghe
show 4 more authors
Silvio Savarese Ran Xu Caiming Xiong Juan Carlos Niebles
This is my paper
classification cs.CVcs.AIcs.CLcs.LG
keywords tokensblip-3-videovisualmodelsmuchtemporalxgen-mm-vidfewer
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

We present xGen-MM-Vid (BLIP-3-Video): a multimodal language model for videos, particularly designed to efficiently capture temporal information over multiple frames. BLIP-3-Video takes advantage of the 'temporal encoder' in addition to the conventional visual tokenizer, which maps a sequence of tokens over multiple frames into a compact set of visual tokens. This enables BLIP3-Video to use much fewer visual tokens than its competing models (e.g., 32 vs. 4608 tokens). We explore different types of temporal encoders, including learnable spatio-temporal pooling as well as sequential models like Token Turing Machines. We experimentally confirm that BLIP-3-Video obtains video question-answering accuracies comparable to much larger state-of-the-art models (e.g., 34B), while being much smaller (i.e., 4B) and more efficient by using fewer visual tokens. The project website is at https://www.salesforceairesearch.com/opensource/xGen-MM-Vid/index.html

Discussion (0). Sign in to comment.

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. DisTime: Distribution-based Time Representation for Video Large Language Models

    cs.CV 2025-05 conditional novelty 7.0 of 10

    A single learnable time token, decoded into a probability distribution over time bins, improves temporal grounding in Video-LLMs and is trained partly on a new 1.25M-event pseudo-labeled dataset.

  2. Efficient Tracking and Understanding Object Transformations

    cs.CV 2026-07 conditional novelty 6.0 of 10

    FluxGraph detects object transformations reactively via SAM2's multi-mask disagreement, cutting TubeletGraph's inference cost by 3.3–10.7x with comparable tracking and state-graph quality.

  3. Searching Videos as Trees: Self-Correcting Agents for Grounded Long Video QA

    cs.CV 2026-07 conditional novelty 6.0 of 10

    An agent that searches a long video by navigating an adaptive temporal tree with zoom-in/zoom-out/shift actions improves grounded long-video QA, but the headline CG-Bench result is measured on a heavily filtered subset.

  4. Strefer: Empowering Video LLMs with Space-Time Referring and Reasoning via Synthetic Instruction Data

    cs.CV 2025-09 conditional novelty 6.0 of 10

    Adding Strefer's synthetic space-time reference questions to video instruction tuning improves mask-referred description/QA, timestamp QA, and temporal reasoning over a video-LLM baseline.

  5. Multi-Granular Spatio-Temporal Token Merging for Training-Free Acceleration of Video LLMs

    cs.CV 2025-07 conditional novelty 6.0 of 10

    STTM is a training-free token merging method that reduces video LLM token counts by 50-70% while keeping accuracy nearly unchanged, yielding 2-3x speedups on six video QA benchmarks.

  6. DynTok: Dynamic Compression of Visual Tokens for Efficient and Effective Video Understanding

    cs.CL 2025-06 conditional novelty 4.0 of 10

    DynTok dynamically merges similar adjacent visual tokens into groups, reducing video token counts to 44.4% with comparable or better video understanding accuracy.

Pith tools