pith:T7EF4SUO
MLVU: Benchmarking Multi-task Long Video Understanding
MLVU benchmark shows current multimodal models struggle with most long video tasks and degrade sharply on longer clips.
arxiv:2406.04264 v3 · 2024-06-06 · cs.CV · cs.AI · cs.CL
Add to your LaTeX paper
\usepackage{pith}
\pithnumber{T7EF4SUOFTLLSQDQNLC2BWIURX}
Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge
Record completeness
Claims
The empirical study with 23 latest MLLMs reveals significant room for improvement in today's technique, as all existing methods struggle with most of the evaluation tasks and exhibit severe performance degradation when handling longer videos.
The chosen video lengths, genres, and tasks in MLVU sufficiently represent the core challenges of real-world long video understanding and that performance on these tasks generalizes beyond the benchmark.
MLVU is a new benchmark for long video understanding that uses extended videos across diverse genres and multi-task evaluations, revealing that current MLLMs struggle significantly and degrade sharply with longer durations.
References
Formal links
Cited by
Receipt and verification
| First computed | 2026-05-17T23:39:21.958186Z |
|---|---|
| Builder | pith-number-builder-2026-05-17-v1 |
| Signature | Pith Ed25519
(pith-v1-2026-05) · public key |
| Schema | pith-number/v1.0 |
Canonical hash
9fc85e4a8e2cd6b940706ac5a0d9148de724540951e474f6b45f20ab24059c3a
Aliases
· · · · ·Agent API
Verify this Pith Number yourself
curl -sH 'Accept: application/ld+json' https://pith.science/pith/T7EF4SUOFTLLSQDQNLC2BWIURX \
| jq -c '.canonical_record' \
| python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: 9fc85e4a8e2cd6b940706ac5a0d9148de724540951e474f6b45f20ab24059c3a
Canonical record JSON
{
"metadata": {
"abstract_canon_sha256": "41148e2d78f4d31b4e397c1a844d4b6f5ce24671e5f8064a2655bd442342726a",
"cross_cats_sorted": [
"cs.AI",
"cs.CL"
],
"license": "http://creativecommons.org/licenses/by-nc-sa/4.0/",
"primary_cat": "cs.CV",
"submitted_at": "2024-06-06T17:09:32Z",
"title_canon_sha256": "18a5bf14850be2ac26f321769b1dc07c5948b5228a893e1a555f8f75b88e6ccd"
},
"schema_version": "1.0",
"source": {
"id": "2406.04264",
"kind": "arxiv",
"version": 3
}
}