MiniGPT4-Video: Advancing Multimodal LLMs for Video Understanding with Interleaved Visual-Textual Tokens

cs.CV 2025-01 conditional novelty 7.0 of 10

Consistent object IDs across video frames and a reconstructed bird's-eye view let large VLMs, and a fine-tuned 7B model, reach state-of-the-art results on ScanNet 3D question answering, captioning, and grounding.

cs.CV 2026-08 conditional novelty 6.0 of 10

SignLlama uses filtered pseudo-gloss pretraining and visual-prioritized distillation to make Llama-based sign language translation competitive on four benchmarks.

cs.CV 2026-08 conditional novelty 6.0 of 10

Coordinates global and local evidence views from a temporal hierarchy, with verification-guided routing, to improve long-video multiple-choice QA.

cs.CV 2026-08 conditional novelty 6.0 of 10

GSTEP prunes visual tokens in VideoLLMs using a global spatio-temporal density score with farthest point sampling, retaining close to full benchmark accuracy at 75 to 90 percent token pruning.

cs.CV 2026-07 conditional novelty 6.0 of 10

Training-free latent-object memory anchors let frozen Video-LLMs retain object histories under a tight token budget and improve streaming and long-video QA.

cs.AI 2026-07 conditional novelty 6.0 of 10

Sign Language QA benchmarks are introduced from PHOENIX14T and CSL-Daily via template-generated questions, and a question-conditioned baseline outperforms video-language and cascaded baselines.

cs.CV 2026-07 accept novelty 6.0 of 10

TimeThink adds step-wise temporal process rewards (max IoU of referenced intervals) to GRPO for Video-LLMs, improving grounding and reasoning over outcome-only RL baselines.

cs.CV 2025-05 conditional novelty 6.0 of 10

VUDG is a domain-generalization benchmark for video understanding with 11 domains and 36,388 QA pairs, and it shows that current large video-language models lose accuracy across visual domains.

cs.CV 2025-05 conditional novelty 6.0 of 10

EmoSign is a 200-clip American Sign Language video dataset with native-signer sentiment and emotion labels plus baseline multimodal LLM results showing poor visual-only emotion recognition.

cs.CV 2025-05 conditional novelty 6.0 of 10

A three-stage curriculum (concept alignment, instruction tuning, downstream fine-tuning) adapted LLaVA-NeXT-Video to soccer, raising action classification accuracy from 11.8% to 63.5% and the VQA relative score from a...

cs.CV 2025-04 conditional novelty 6.0 of 10

A streaming video assistant drops temporally redundant tokens (over 80%) with minimal accuracy loss and uses the drop-ratio curve to trigger proactive responses at scene changes.

cs.CV 2025-04 conditional novelty 6.0 of 10

EventVAD improves training-free video anomaly detection by detecting event boundaries from CLIP and RAFT features and feeding coherent event segments to a 7B multimodal LLM, achieving 82.03 AUC on UCF-Crime and 64.04 ...

cs.CV 2024-11 conditional novelty 6.0 of 10

A graph-guided self-training method lets video-language models generate their own reasoning training data from raw videos, improving multi-step compositional reasoning accuracy.

cs.CV 2024-11 conditional novelty 6.0 of 10

Encoding video as a small set of object tokens, produced by off-the-shelf detection, segmentation, and tracking models, improves video QA accuracy and enables video-based referring in a 7B video-LLM.

cs.CV 2026-04 unverdicted novelty 5.0 of 10

The paper offers the first focused review of MLLM-based video translation organized by a three-role taxonomy of Semantic Reasoner, Expressive Performer, and Visual Synthesizer, plus open challenges.

cs.MM 2025-09 conditional novelty 5.0 of 10

LGSRR uses LLM-generated semantic descriptions and rankings to improve multimodal intent recognition, reporting SOTA results on MIntRec2.0 and IEMOCAP-DA with gains around 0.5-1.3%.

cs.IR 2025-08 conditional novelty 5.0 of 10

Replacing raw video and audio features with MLLM-generated natural-language captions improves hit rate and nDCG for two-tower and SASRec recommenders on MicroLens-100K.

q-bio.NC 2025-05 conditional novelty 5.0 of 10

On movie-watching fMRI data, multi-modal vision-audio transformers predict brain activity better than unimodal video or speech models, with video dominating cross-modal alignment and video plus audio jointly contribut...

cs.MM 2025-05 conditional novelty 5.0 of 10

A chain-of-thought fine-tuned vision-language model can infer audio descriptions from silent videos, and using those descriptions as prompts improves video-to-audio generation.

cs.CV 2025-01 conditional novelty 5.0 of 10

Adding a trainable convolutional temporal encoder to LLaVA-style video models improves benchmark scores and allows heavy frame compression, but the controlled evidence for the causal claim is weak.

cs.CV 2025-01 conditional novelty 5.0 of 10

A small video model with a group resampler reduces video input to a few hundred tokens and beats several 7B models on common video benchmarks.

cs.CV 2024-12 conditional novelty 5.0 of 10

A single 3B-parameter end-to-end model with object-aware video perceiving and multi-granularity text fusion reports SOTA results across four instructed visual segmentation tasks.

cs.CV 2024-12 conditional novelty 5.0 of 10

A plug-and-play linear video tokenizer converts existing image-based LLMs into video-understanding LLMs by condensing frames into weighted-average tokens while preserving image capabilities.

cs.CV 2024-11 reject novelty 5.0 of 10

DynFocus dynamically allocates a few tokens to selected frames and two tokens to the rest, reporting competitive video QA accuracy with lower token budgets.

cs.CL 2024-11 conditional novelty 5.0 of 10

Training a multimodal LLM on mostly text-only instructions with a small vision-language tail matches or beats vision-heavy instruction tuning on held-out text and vision tasks at about half the token cost.

cs.CV 2025-07 reject novelty 4.0 of 10

MCAM is a video captioning model combining 3DResNet and VidSwin features with a graph-inspired fusion module, reporting mixed gains on BDD-X and CoVLA but failing to implement the promised causal reasoning.

cs.MM 2025-06 reject novelty 4.0 of 10

ViFusion combines dynamic tensor fusion with hierarchical AllReduce to speed up distributed video feature indexing, but the 8-22x throughput claim is an overstatement of bandwidth gains over a self-defined baseline.

cs.CV 2025-06 conditional novelty 4.0 of 10

LeanPO improves Video-LLM alignment by using a reference-free average-likelihood reward, self-generated winning/losing pairs, and dynamic label smoothing, yielding gains on six video benchmarks.

cs.RO 2025-05 reject novelty 4.0 of 10

A new 124K-clip dataset with hierarchical text annotations, plus a pipeline that couples an LLM planner, a text-to-pose VAE, diffusion in-betweening, and physics control to generate long-horizon human behaviors.

cs.CV 2025-06 conditional novelty 3.0 of 10

A structured review of vision generalist models, classifying them into encoding-based and sequence-to-sequence frameworks and summarizing datasets, benchmarks, techniques, and open problems.

cs.CL 2025-04 conditional novelty 3.0 of 10

A survey that organizes PEFT methods into additive, selective, reparameterized, hybrid, and unified families, but with no new method or verified experiments.

cs.CV 2025-01 conditional novelty 3.0 of 10

This paper reviews and taxonomizes VLLM applications into vision-to-text, vision-to-action, and text-to-vision, adding ethics and future-work discussion.

cs.CV 2024-12 conditional novelty 3.0 of 10

A survey arguing that video-LLMs rely on pretrained encoders and short-biased datasets, leaving them weak at long-term temporal reasoning such as causality and event progression.

cs.CV 2024-12 conditional novelty 2.0 of 10

Fine-tuning LLaVA on prior TRECVid video-text pairs improves scores on the VTT24 captioning test set for BLEU, METEOR, CIDEr, and CIDEr-D, while SPICE and STS results are mixed.