Pith Number

pith:IL2GFSKZ

pith:2024:IL2GFSKZ2657G56SVJAYSIRIN6

not attested not anchored not stored refs resolved

TempCompass: Do Video LLMs Really Understand Videos?

Lei Li, Lu Hou, Shicheng Li, Shuhuai Ren, Sishuo Chen, Xu Sun, Yi Liu, Yuanxin Liu, Yuxiang Wang

Video LLMs exhibit notably poor temporal perception ability across aspects like speed and direction.

arxiv:2403.00476 v3 · 2024-03-01 · cs.CV

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{IL2GFSKZ2657G56SVJAYSIRIN6}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

Based on TempCompass, these models exhibit notably poor temporal perception ability.

C2weakest assumption

That the constructed conflicting videos successfully isolate specific temporal aspects without introducing unintended biases or allowing models to exploit other cues, and that the LLM-based automatic evaluation accurately reflects model performance.

C3one line summary

TempCompass benchmark reveals that state-of-the-art Video LLMs have poor ability to perceive temporal aspects such as speed, direction, and ordering in videos.

References

135 extracted · 135 resolved · 27 Pith anchors

[1] LLaMA: Open and Efficient Foundation Language Models , author=. ArXiv , year=

[2] Llama 2: Open Foundation and Fine-Tuned Chat Models , author=. ArXiv , year=

[3] and Stoica, Ion and Xing, Eric P

[4] Hashimoto , year =

[5] Tom B. Brown and Benjamin Mann and Nick Ryder and Melanie Subbiah and Jared Kaplan and Prafulla Dhariwal and Arvind Neelakantan and Pranav Shyam and Girish Sastry and Amanda Askell and Sandhini Agarwa

Formal links

2 machine-checked theorem links

Cited by

30 papers in Pith

The TIME Machine: On The Power of Motion for Efficient Perception

MotionBench: Benchmarking and Improving Fine-grained Video Motion Understanding for Vision Language Models

LongVT: Incentivizing "Thinking with Long Videos" via Native Tool Calling

EgoCoT-Bench: Benchmarking Grounded and Verifiable Operation-Centric Chain of Thought Reasoning for MLLMs

One Trajectory, One Token: Grounded Video Tokenization via Panoptic Sub-object Trajectory

Receipt and verification

First computed	2026-05-17T23:38:15.358530Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

42f462c959d7bbf377d2aa418922286f88f1771da0803e0dec95e98189ceb55c

Aliases

arxiv: 2403.00476 · arxiv_version: 2403.00476v3 · doi: 10.48550/arxiv.2403.00476 · pith_short_12: IL2GFSKZ2657 · pith_short_16: IL2GFSKZ2657G56S · pith_short_8: IL2GFSKZ

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/IL2GFSKZ2657G56SVJAYSIRIN6 \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: 42f462c959d7bbf377d2aa418922286f88f1771da0803e0dec95e98189ceb55c

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "8ea710614dbdefbfee1caca2d0d10902ea897a7deb524d9d333cb07dccfc7a7e",
    "cross_cats_sorted": [],
    "license": "http://creativecommons.org/licenses/by/4.0/",
    "primary_cat": "cs.CV",
    "submitted_at": "2024-03-01T12:02:19Z",
    "title_canon_sha256": "5c3fcd68707f7e1fe9fdbfdc44467dac5589f4b0191d98f5a0d2c1e1882aaf32"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2403.00476",
    "kind": "arxiv",
    "version": 3
  }
}