Pith. sign in

REVIEW 5 cited by

Fostering Video Reasoning via Next-Event Prediction

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2505.22457 v1 pith:ZRT7EDYY submitted 2025-05-28 cs.CV cs.AIcs.CL

Haonan Wang , Hongfu Liu , Xiangyan Liu , Chao Du , Kenji Kawaguchi , Ye Wang , Tianyu Pang This is my paper
classification cs.CVcs.AIcs.CL
keywords videoreasoningtasktemporalfutureframeslearningmllms
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Next-token prediction serves as the foundational learning task enabling reasoning in LLMs. But what should the learning task be when aiming to equip MLLMs with temporal reasoning capabilities over video inputs? Existing tasks such as video question answering often rely on annotations from humans or much stronger MLLMs, while video captioning tends to entangle temporal reasoning with spatial information. To address this gap, we propose next-event prediction (NEP), a learning task that harnesses future video segments as a rich, self-supervised signal to foster temporal reasoning. We segment each video into past and future frames: the MLLM takes the past frames as input and predicts a summary of events derived from the future frames, thereby encouraging the model to reason temporally in order to complete the task. To support this task, we curate V1-33K, a dataset comprising 33,000 automatically extracted video segments spanning diverse real-world scenarios. We further explore a range of video instruction-tuning strategies to study their effects on temporal reasoning. To evaluate progress, we introduce FutureBench to assess coherence in predicting unseen future events. Experiments validate that NEP offers a scalable and effective training paradigm for fostering temporal reasoning in MLLMs.

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. FeVOS: Foresight Expression Video Object Segmentation

    cs.CV 2026-06 unverdicted novelty 7.0 of 10

    Introduces the FeVOS task, a 968-clip dataset with foresight expressions, and an MLLM model FeVOS-R1 trained via SFT then RL that reports SOTA on the new task plus generalization to prior RVOS benchmarks.

  2. Imagine Before You Predict: Interleaved Latent Visual Reasoning for Video Event Prediction

    cs.CV 2026-06 unverdicted novelty 7.0 of 10

    Future-L1 interleaves latent visual spans with text in MLLM decoding, trained on a custom Future-L1-50K dataset via LA-DAPO RL, and reports SOTA gains on FutureBench (61.0 to 85.4) and TwiFF-Bench (2.44 to 3.04).

  3. EmoTrans: A Benchmark for Understanding, Reasoning, and Predicting Emotion Transitions in Multimodal LLMs

    cs.CV 2026-04 unverdicted novelty 7.0 of 10

    EmoTrans is a new video benchmark with four progressive tasks that measures how well current multimodal LLMs handle dynamic emotion transitions rather than static recognition.

  4. FutureOmni: Evaluating Future Forecasting from Omni-Modal Context for Multimodal LLMs

    cs.CL 2026-01 conditional novelty 6.0 of 10

    FutureOmni, a 919-video, 1,034-question audio-visual future-forecasting benchmark, shows top MLLMs reach only 64.8% accuracy, and OFF tuning improves open models.

  5. EgoSelf: From Memory to Personalized Egocentric Assistant

    cs.CV 2026-04 unverdicted novelty 5.0 of 10

    EgoSelf uses graph-based memory of user interactions to derive personalized profiles and predict future behaviors for egocentric assistants.

Pith tools