Pith. sign in

REVIEW 21 cited by

TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2312.02051 v2 pith:5GGSPYKQ submitted 2023-12-04 cs.CV cs.AIcs.CL

Shuhuai Ren , Linli Yao , Shicheng Li , Xu Sun , Lu Hou This is my paper
classification cs.CVcs.AIcs.CL
keywords videotimechatlanguagelargemodeltasksunderstandingframe
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

This work proposes TimeChat, a time-sensitive multimodal large language model specifically designed for long video understanding. Our model incorporates two key architectural contributions: (1) a timestamp-aware frame encoder that binds visual content with the timestamp of each frame, and (2) a sliding video Q-Former that produces a video token sequence of varying lengths to accommodate videos of various durations. Additionally, we construct an instruction-tuning dataset, encompassing 6 tasks and a total of 125K instances, to further enhance TimeChat's instruction-following performance. Experiment results across various video understanding tasks, such as dense captioning, temporal grounding, and highlight detection, demonstrate TimeChat's strong zero-shot temporal localization and reasoning capabilities. For example, it achieves +9.2 F1 score and +2.8 CIDEr on YouCook2, +5.8 HIT@1 on QVHighlights, and +27.5 R@1 (IoU=0.5) on Charades-STA, compared to state-of-the-art video large language models, holding the potential to serve as a versatile video assistant for long-form video comprehension tasks and satisfy realistic user requirements.

Discussion (0). Sign in to comment.

Forward citations

Cited by 21 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. AdaThinkV: Adaptive Thinking for Token-Efficient Video Reasoning

    cs.CV 2026-08 conditional novelty 7.0 of 10

    A video reasoning model learns per question whether to reason aloud or answer directly, improving accuracy by about 3 points over the best adaptive baseline while using about 23% fewer output tokens.

  2. TimeLens2: Generalist Video Temporal Grounding with Multimodal LLMs

    cs.CV 2026-07 conditional novelty 7.0 of 10

    TimeLens2 shows that a compact video MLLM can localize multiple evidence intervals in long videos by training on verified interval labels and a Wasserstein-based time-distance reward.

  3. MarkIt: Training-Free Visual Markers for Precise Video Temporal Grounding

    cs.MM 2026-04 unverdicted novelty 7.0 of 10

    MarkIt uses a query-to-mask bridge with open-vocabulary segmentation to add visual markers and frame indices to videos, enabling Vid-LLMs to achieve state-of-the-art temporal grounding on moment retrieval and highligh...

  4. A Paradigm Shift: Fully End-to-End Training for Temporal Sentence Grounding in Videos

    cs.CV 2026-04 unverdicted novelty 7.0 of 10

    Fully end-to-end training with a sentence-conditioned adapter outperforms frozen-backbone baselines for localizing video segments that match sentence queries.

  5. LVBench: An Extreme Long Video Understanding Benchmark

    cs.CV 2024-06 accept novelty 7.0 of 10

    LVBench is a new benchmark for extreme long video understanding that evaluates multimodal large language models on hour-scale videos using tasks designed to probe extended memory and comprehension.

  6. MLVU: Benchmarking Multi-task Long Video Understanding

    cs.CV 2024-06 conditional novelty 7.0 of 10

    MLVU is a new benchmark for long video understanding that uses extended videos across diverse genres and multi-task evaluations, revealing that current MLLMs struggle significantly and degrade sharply with longer durations.

  7. TimePLE: Rethinking Temporal Representation for Video Temporal Grounding

    cs.CV 2026-07 conditional novelty 6.0 of 10

    TimePLE predicts a whole video interval as a joint distribution over a position-duration square, rather than predicting start and end separately, and reports higher mIoU across four VTG benchmarks.

  8. Mixture of Probes: Learning from Privileged Modalities in Multimodal LLMs Through Probing

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Structured intermediate probing that separates modality-specific from modality-general signals lets privileged training modalities improve single-modality MLLM inference by large margins over naive multimodal training.

  9. EFlow: Learning Evidence Flow for Long-Video Reasoning with Adaptive Reflection

    cs.CV 2026-07 conditional novelty 6.0 of 10

    EFlow improves long-video QA by separating clip-finding from answering and adding a confidence trigger that re-reads the full video when unsure.

  10. DELTAVID: Enhancing Fine-Grained Spatiotemporal Perception with Cross-Video Differences

    cs.CV 2026-06 conditional novelty 6.0 of 10

    Rule-reward training on controllable cross-video differences (Grounding + MCQ) improves Video MLLM local spatiotemporal evidence localization and transfers to general video QA benchmarks.

  11. HPP: Hierarchical Programmatic Probing for Long Video Understanding by Decoupling Perception and Reasoning

    cs.CV 2026-06 unverdicted novelty 6.0 of 10

    HPP decouples perception from reasoning in long-video VLMs by having an LLM run iterative programmatic probes on hierarchically segmented video, reporting gains on LongVideoBench, EgoSchema, VideoMME, and MLVU.

  12. VTI-CoT: Visual-Textual Interleaved Chain of Thought for Video Reasoning

    cs.CV 2026-06 unverdicted novelty 6.0 of 10

    VTI-CoT proposes a visual-textual interleaved chain-of-thought method for video reasoning, built via automated annotation and OCR compression, claiming SOTA performance and better training efficiency on same-scale models.

  13. MarkIt: Training-Free Visual Markers for Precise Video Temporal Grounding

    cs.MM 2026-04 unverdicted novelty 6.0 of 10

    MarkIt converts videos into query-conditioned marked versions via a linguistic-parsing and open-vocabulary segmentation bridge that embeds instance masks, semantic markers, and frame indices to improve Vid-LLM tempora...

  14. TempCompass: Do Video LLMs Really Understand Videos?

    cs.CV 2024-03 unverdicted novelty 6.0 of 10

    TempCompass benchmark reveals that state-of-the-art Video LLMs have poor ability to perceive temporal aspects such as speed, direction, and ordering in videos.

  15. EFlow: Learning Evidence Flow for Long-Video Reasoning with Adaptive Reflection

    cs.CV 2026-07 conditional novelty 5.0 of 10

    Separating temporal grounding from answer reasoning, plus low-confidence full-video re-reading, modestly improves long-video QA on Qwen3-VL across five benchmarks.

  16. EFlow: Learning Evidence Flow for Long-Video Reasoning with Adaptive Reflection

    cs.CV 2026-07 unverdicted novelty 5.0 of 10

    EFlow separates temporal grounding from logical reasoning via two CoT stages and adds confidence-aware reflection, trained via SFT and RL on custom trajectory data, yielding gains on five video benchmarks.

  17. InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modeling

    cs.CV 2025-01 unverdicted novelty 5.0 of 10

    InternVideo2.5 improves video MLLMs by incorporating dense vision task annotations via direct preference optimization and compact spatiotemporal representations via adaptive hierarchical token compression, yielding be...

  18. CogVLM2: Visual Language Models for Image and Video Understanding

    cs.CV 2024-08 conditional novelty 5.0 of 10

    CogVLM2 family achieves state-of-the-art results on image and video understanding benchmarks through improved visual expert architecture, higher resolution inputs, and automated temporal grounding for videos.

  19. Watch, Remember, Reason: Human-View Video Understanding with MLLMs

    cs.CV 2026-06 unverdicted novelty 4.0 of 10

    This is a survey that frames video MLLM research via a human-view formulation of perceptual representations, memory states, reasoning traces, and predictions, then reviews methods, datasets, benchmarks, and open problems.

  20. LLaVA-OneVision-2: Towards Next-Generation Perceptual Intelligence

    cs.CV 2026-05 unverdicted novelty 4.0 of 10

    LLaVA-OV-2 uses codec-stream tokenization and a shared 3D RoPE to improve video, spatial, and tracking performance over Qwen3-VL-8B, while introducing the JumpScore benchmark for fine-grained motion localization.

  21. VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs

    cs.CV 2024-06 unverdicted novelty 4.0 of 10

    VideoLLaMA 2 improves video LLMs via a new STC connector for spatial-temporal dynamics and joint audio training, reaching competitive results on video QA and captioning benchmarks.

Pith tools