Pith. sign in

REVIEW 3 major objections 6 minor 213 references

PhyCheck: Fine-Grained Evidence-Grounded Dataset for Physical Law Understanding in Video-LLMs

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper aims to establish that PhyCheck, a fine-grained, evidence-grounded video QA dataset, both diagnoses and improves Video-LLMs' physical law understanding, with hierarchical training lifting Qwen2.5-VL from 50.93% to 81.00%…

desk verdict A solid, reusable benchmark whose headline 81% fine-tuning gain is plausible but under-supported; it deserves peer review, with protocol details and a second base model requested. read the letter →

arxiv 2608.02150 v3 pith:POUTR42Y submitted 2026-08-03 cs.CV cs.AI

Zhongjie Ba , Shengwang Xu , Peng Cheng , Jinyang Zou , Ting Yu , Zhibo Wang , Zhan Qin This is my paper · ORCID
classification cs.CVcs.AI
keywords physicallawunderstandingvideoquestionansweringlargelanguagemodelsevidence-groundedannotationphysicsconsistencybenchmarkhierarchicalsupervisionsyntheticvideoscontext-sensitivereasoning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

PhyCheck is a video question-answering dataset built on 6,399 synthetic videos from VideoPhy2, re-annotated with about 69,825 binary yes/no questions at two granularities: coarse-grained questions ask whether an event obeys or violates a physical law, and fine-grained questions ask whether specific visual evidence of a violation is present. The paper's central claim is that current video-LLMs only superficially understand physical laws: they score reasonably on conforming examples but their recall collapses on violating examples. Using the dataset as structured supervision, with fine-grained questions trained before coarse-grained ones, fine-tuned Qwen2.5-VL reaches 81.00% accuracy on the PhyCheck test set, up from 50.93% for the base model. The authors also report a 50-pair pilot in which a model trained on PhyCheck becomes more cautious on ambiguous clips yet jumps to 0.98 once external causal context is supplied, suggesting the training fosters context-sensitive recalibration.

What carries the argument

The load-bearing mechanism is the two-tier, evidence-grounded VQA annotation: each video carries coarse-grained yes/no judgments of law compliance plus fine-grained questions naming a specific visual detail, such as 'a hammer moving without external force.' Template-based generation deterministically assigns answer polarity from VideoPhy2's followed and violated labels, and LLM-assisted drafting (Gemini-3 generates negative prompts; GPT-5 converts them into questions) is filtered by human verification with 99.74% agreement. The training-order finding—fine-grained first, coarse-grained second—is presented as the key that makes the hierarchy effective.

What would settle it

Run an annotation study where a fresh set of annotators, blind to VideoPhy2 labels and to the LLM-generated questions, independently answer the fine-grained PhyCheck questions from the video alone; if their answers diverge from the dataset's labels beyond a few percent, the evidence-grounded training signal is not what the paper claims.

Watch

Extended reading notes

Core claim

The discovery is that a benchmark which decomposes physical-law judgments into explicit visual-evidence questions—with a hierarchy where fine-grained questions are learned before coarse-grained consistency questions—serves both as a reliable diagnostic and as training data that materially improves Video-LLMs. Across a spectrum of open-source and proprietary models, most achieve around 60% accuracy on PhyCheck, with high recall on 'conform' examples but low recall on 'violate' examples, confirming that apparent competence hides a bias toward accepting physical statements. Sequential fine-tuning of Qwen2.5-VL on fine-grained then coarse-grained VQA yields 81.00% overall accuracy, and per-category analysis shows the largest gains on law categories requiring implicit reasoning, such as conservation and Newtonian mechanics. The paper presents the context-sensitive pilot as initial evidence that alignment on PhyCheck teaches models to integrate hidden causal factors rather than fixate on superficial visual anomalies.

Load-bearing premise

The benchmark's ground truth is inherited from VideoPhy2's followed/violated labels, and the fine-grained questions are LLM-drafted and only filtered by human agreement; if those labels or questions are noisy, every accuracy number inherits that noise.

Editorial extensions

If this is right

  • If correct, physical-law understanding in video models is measurable by fine-grained evidence questions, not just coarse verdicts.
  • A training recipe of evidence questions before consistency questions can lift a Video-LLM's accuracy from about 51% to 81% on this benchmark.
  • Existing models' high conform-versus-violate recall imbalance indicates a systematic bias, and benchmark builders should include violation-rich splits.
  • The context-sensitive pilot suggests that recalibrating physical judgments in response to hidden causal factors is trainable.
  • Category-wise failures, especially in conservation and Newtonian mechanics, point to specific supervision targets for future model training.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If PhyCheck's supervision signal were shown to generalize, a natural extension is to use its question decomposition as a prompt template at inference time for models not fine-tuned on it, testing whether the evidence chain itself causes the gain.
  • The reported 81% is on a test set drawn from the same VideoPhy2 distribution as training; nothing in the paper shows transfer to real-world videos or to generators outside the training set, so a cross-distribution benchmark is the obvious next test.
  • The context-sensitive subset's dramatic jump (0.18 to 0.98) could indicate genuine recalibration or could reflect the model learning to flip its answer whenever supplementary text is present; a control with arbitrary irrelevant context would distinguish these.
  • The 99.74% agreement only measures agreement between annotators who already saw the LLM-generated questions, not whether fresh viewers can answer them from the video alone.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. PhyCheck is a video question-answering dataset constructed from 6,399 videos and 69,825 binary QA pairs, organized into coarse-grained physical-law compliance questions and fine-grained evidence questions, with a 50-pair context-sensitive pilot subset. The coarse-grained labels are deterministically derived from VideoPhy2's followed/violated annotations, while the fine-grained questions are generated by Gemini-3 and GPT-5 and filtered by human annotators. The paper benchmarks several Video-LLMs on PhyCheck, reporting accuracies around 60% and a sharp recall drop on violation examples. It then fine-tunes Qwen2.5-VL on different subsets and training orders and reports that fine-grained-then-coarse-grained supervision raises accuracy from 50.93% to 81.00%. A context pilot suggests the fine-tuned model updates its judgments when external causal context is provided (0.18 to 0.98).

Significance. If the headline result is robust, PhyCheck would be a valuable resource: it is large-scale, hierarchically structured, and offers both evaluation and training data, with strong inter-annotator agreement on the generated fine-grained questions. The paper's public code/data URL and the systematic decomposition of physical laws into visual evidence are concrete strengths. However, the central empirical claim currently lacks the protocol details needed to distinguish genuine physical understanding from in-distribution label learning: no held-out generalization test, no validation-based model selection, no statistical significance testing, and no independent verification of the inherited VideoPhy2 labels. The context result is intriguing but based on 50 pairs. For these reasons the contribution is promising but not yet established.

major comments (3)
  1. [Section 4.3, Table 3] The reported 81.00% accuracy is selected as the best among four supervision configurations, yet the paper does not state whether the test set was used for this selection, whether checkpoints were chosen on a held-out validation split, or how many random seeds were used. Section 4.1 states only that Qwen2.5-VL was 'fully fine-tuned ... for 12 epochs' on the PhyCheck training set, without giving the number of videos or questions in the train and test splits or the split criterion (e.g., video-level vs. per-generation-model). Because the same test set may have been used for model selection, the gap between 50.93% and 81.00% could be inflated by chance. Please report a fixed train/test split, validation-based checkpoint selection, results over multiple seeds with variance, and a held-out generalization test (e.g., on generation models not seen in training or on new physical-law categories).
  2. [Section 3.2] The coarse-grained answers are deterministically derived from VideoPhy2's followed/violated labels, and the fine-grained questions are generated by Gemini-3/GPT-5 and filtered by human binary agreement. This means the training signal and the evaluation ground truth come from the same annotation distribution, and no independent verification of the VideoPhy2 labels (e.g., by fresh human annotators under a protocol that allows them to disagree) is reported. The 99.74% raw inter-annotator agreement measures agreement on the LLM-generated questions, not the correctness of the underlying labels. The 81.00% result is therefore compatible with in-distribution label-cue learning. Please provide an independent human-verified test subset with freshly collected labels, or evaluate on videos and physical laws not used in training, to support the claim of improved physical understanding.
  3. [Section 4.5, Appendix C3] The context-sensitive result is computed on the same 50 hand-crafted pairs that define the pilot; the text gives 0.18 video-only and 0.98 context-assisted accuracy for the fine-tuned model but does not report the context-assisted accuracy of the baseline models (only states that 'many struggle'), nor any confidence interval or statistical test. The interpretation that the low video-only score reflects 'epistemic caution' is not uniquely supported; it could be a calibration artifact from fine-tuning on a balanced binary dataset. Please report per-condition numbers for all models on the context subset and validate the finding on a larger or held-out set.
minor comments (6)
  1. [Abstract and Section 3.1] The dataset size is reported as 'approximately 50k human-verified VQA pairs' in the abstract and introduction, but Section 3.1 says PhyCheck contains 69,825 QA pairs; please reconcile these numbers.
  2. [Section 4.3, Table 3] The checkmark notation for the sequential training rows (e.g., '✓ 1⁄bigcircle✓2⁄bigcircle') is unclear; replace it with explicit labels such as 'fine-grained then coarse-grained'.
  3. [Section 4.1] Provide the full training hyperparameters (optimizer, learning rate, batch size, number of frames per video, prompt template) and the exact split statistics; without these, the reproducibility of the 81.00% result is limited.
  4. [Section 4.5] The appendix is referenced as 'Appendix C3' but no appendix text is included; either include the appendix or report the numbers inline.
  5. [Section 3.3] The construction of the 50 context pairs is underspecified; specify how many source videos were used, what crops were made, and how the context is presented to models (text-only, image, video, or a combination).
  6. [Table 4] The row labeled 'Qwen2.5-Finetune' is not identified in Table 2 or Table 3; add a footnote identifying this as the fine-tuned Qwen2.5-VL model and the training configuration.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the claimed improvement is a standard held-out benchmark evaluation, and no equation-level reduction to the model's own outputs or to a fitted parameter is present.

full rationale

PhyCheck's coarse-grained answers are deterministically derived from VideoPhy2's external human annotations via template questions, and its fine-grained questions are LLM-generated with human verification; neither step defines the target result in terms of the fine-tuned model's own outputs. The central claim, that fine-tuning Qwen2.5-VL raises accuracy from 50.93% to 81.00% on the PhyCheck test set, is a conventional supervised-learning evaluation on the benchmark's held-out split, not a fitted parameter renamed as a prediction. The training uses the training split and Table 3 reports test-set numbers across supervision settings; while the best training order may have been selected with knowledge of test-set results and the benchmark inherits VideoPhy2's label distribution, those are generalization and validation concerns rather than definitional circularity. There are no load-bearing self-citations, no imported uniqueness theorems, and no ansatz smuggled in via citation. The paper explicitly cautions that its 50-pair context pilot is preliminary and limited in scale. No specific reduction of the form 'Eq. X = Eq. Y by construction' or 'fitted parameter called prediction' can be exhibited, so the appropriate finding is no significant circularity.

Assumptions & free parameters 2 free parameters · 3 assumptions · 1 invented entities

The central claim rests on three external inputs: VideoPhy2's labels, the reliability of LLM-generated fine-grained questions after human filtering, and the transferability of synthetic-video findings. The paper also selects the best training configuration from a small set without significance testing. No new physical entities or forces are introduced.

free parameters (2)
  • VideoPhy2 followed/violated labels
    The entire benchmark depends on VideoPhy2's labels as ground truth; the paper discards 'undetermined' samples and treats 'followed' and 'violated' as binary, so any error in the source label propagates to PhyCheck.
  • Seq training order (fine-grained before coarse-grained)
    The best configuration (81.00%) is selected from a small grid of two training orders; the paper does not present significance testing or a held-out model selection protocol beyond the chosen test set.
assumptions (3)
  • domain assumption VideoPhy2's physical rule annotations are correct for the 6,399 videos.
    Section 3.2: the pipeline 'leverages the physical rule annotations provided by VideoPhy2' and deterministically derives answer polarity from them; no independent re-verification of the source labels is reported.
  • domain assumption The fine-grained questions generated by Gemini-3 and GPT-5 are answerable from the video content alone.
    Section 3.2: questions are filtered by human binary agreement, but the paper does not report a protocol for checking visual answerability or the fraction of generated questions rejected.
  • domain assumption Synthetic AI-generated videos are a valid proxy for testing physical law understanding that transfers to real-world videos.
    The dataset uses only VideoPhy2 synthetic videos; the context pilot uses 50 real-world clips but the main benchmark and fine-tuning rely entirely on generated footage.
invented entities (1)
  • PhyCheck dataset structure (coarse VQA, fine-grained VQA, context subset)
    purpose: Benchmark and supervision signal for Video-LLM physical understanding.
    The dataset is the paper's own artifact; its value as a benchmark is exactly what the paper is testing, so it does not provide independent falsifiable evidence outside the paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PhyCheck: Fine-Grained Evidence-Grounded Dataset for Physical Law Understanding in Video-LLMs." pith.science (2026). https://pith.science/paper/POUTR42Y

@misc{pith2026260802150,
  author       = {Pith},
  title        = {Pith review of: PhyCheck: Fine-Grained Evidence-Grounded Dataset for Physical Law Understanding in Video-LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/POUTR42Y}},
  note         = {Machine review of arXiv:2608.02150}
}
read the original abstract

Embodied intelligence and world models require video understanding systems to go beyond recognizing objects and actions and develop an understanding of physical regularities. However, despite their strong performance on general video understanding tasks, current video-language models still struggle to reliably determine whether an observed event conforms to specific physical laws. Existing benchmarks primarily assess the physical quality of generated videos, providing limited support for systematically evaluating and improving the physical-law understanding of Video Large Language Models (VideoLLMs). To address this gap, we introduce PhyCheck, a video question answering dataset organized at two complementary levels of granularity. The coarse-grained subset asks models to determine whether the phenomenon shown in a video conforms to or violates physical laws, while the fine-grained subset further examines whether models can capture physical details responsible for the violation or compliance. We use these subsets as structured supervision to improve physical understanding. In addition, the dataset contains a diagnostic subset with external causal context that reveal hidden factors affecting physical plausibility, assessing whether models can recalibrate their judgments accordingly. Experiments with Fine-tune Qwen2.5-VL show that training with the proposed data substantially improves the understanding of physical-consistency, while evaluations in the diagnostic subset reveal that current models still have difficulty incorporating additional causal conditions into their decisions. These findings highlight the gap between recognizing surface-level inconsistencies and understanding underlying physical mechanisms, and provide a foundation for evaluating and improving physical understanding in Video-LLMs.

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

213 extracted references · 3 canonical work pages

  1. [1]

    FirstName LastName , title =

  2. [2]

    FirstName Alpher , title =

  3. [3]

    Journal of Foo , volume = 13, number = 1, pages =

    FirstName Alpher and FirstName Fotheringham-Smythe , title =. Journal of Foo , volume = 13, number = 1, pages =

  4. [4]

    Journal of Foo , volume = 14, number = 1, pages =

    FirstName Alpher and FirstName Fotheringham-Smythe and FirstName Gamow , title =. Journal of Foo , volume = 14, number = 1, pages =

  5. [5]

    FirstName Alpher and FirstName Gamow , title =

  6. [6]

    Advances in Neural Information Processing Systems , volume=

    Representing long-range context for graph neural networks with global attention , author=. Advances in Neural Information Processing Systems , volume=

  7. [7]

    arXiv preprint arXiv:2310.11829 , year=

    Towards graph foundation models: A survey and beyond , author=. arXiv preprint arXiv:2310.11829 , year=

    arXiv

  8. [8]

    Nature , volume=

    Foundation models for generalist medical artificial intelligence , author=. Nature , volume=. 2023 , publisher=

    2023

Show all 213 references
  1. [9]

    arXiv preprint arXiv:2407.01290 , year=

    Hypformer: Exploring Efficient Hyperbolic Transformer Fully in Hyperbolic Space , author=. arXiv preprint arXiv:2407.01290 , year=

    arXiv

  2. [10]

    arXiv preprint arXiv:2401.12874 , year=

    From understanding to utilization: A survey on explainability for large language models , author=. arXiv preprint arXiv:2401.12874 , year=

    arXiv

  3. [11]

    Locret: Enhancing Eviction in Long-Context

    Yuxiang Huang and Binhang Yuan and Xu Han and Chaojun Xiao and Zhiyuan Liu , journal=. Locret: Enhancing Eviction in Long-Context. 2025 , note=

    2025

  4. [12]

    Cheng-Ping Hsieh and Simeng Sun and Samuel Kriman and Shantanu Acharya and Dima Rekesh and Fei Jia and Boris Ginsburg , booktitle=

  5. [13]

    arXiv preprint arXiv:2412.15204 , year=

    LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks , author=. arXiv preprint arXiv:2412.15204 , year=

    arXiv

  6. [14]

    arXiv preprint arXiv:2311.06786 , year=

    Explainability of Vision Transformers: A Comprehensive Review and New Perspectives , author=. arXiv preprint arXiv:2311.06786 , year=

    arXiv

  7. [15]

    SIAM Review , volume=

    Variance and covariance of distributions on graphs , author=. SIAM Review , volume=. 2022 , publisher=

    2022

  8. [16]

    International conference on machine learning , pages=

    Concept bottleneck models , author=. International conference on machine learning , pages=. 2020 , organization=

    2020

  9. [17]

    arXiv preprint arXiv:2412.10319 , year=

    Scbench: A kv cache-centric analysis of long-context methods , author=. arXiv preprint arXiv:2412.10319 , year=

    arXiv

  10. [18]

    arXiv preprint arXiv:2504.13173 , year=

    It's All Connected: A Journey Through Test-Time Memorization, Attentional Bias, Retention, and Online Optimization , author=. arXiv preprint arXiv:2504.13173 , year=

    arXiv

  11. [19]

    arXiv preprint arXiv:2309.17453 , year=

    Efficient streaming language models with attention sinks , author=. arXiv preprint arXiv:2309.17453 , year=

    arXiv

  12. [20]

    arXiv preprint arXiv:2308.16137 , year=

    Lm-infinite: Zero-shot extreme length generalization for large language models , author=. arXiv preprint arXiv:2308.16137 , year=

    arXiv

  13. [21]

    Advances in Neural Information Processing Systems , volume=

    H2o: Heavy-hitter oracle for efficient generative inference of large language models , author=. Advances in Neural Information Processing Systems , volume=

  14. [22]

    Advances in Neural Information Processing Systems , volume=

    Snapkv: Llm knows what you are looking for before generation , author=. Advances in Neural Information Processing Systems , volume=

  15. [23]

    Forty-first International Conference on Machine Learning , year=

    Cam: Cache merging for memory-efficient llms inference , author=. Forty-first International Conference on Machine Learning , year=

  16. [24]

    arXiv preprint arXiv:2406.13035 , year=

    D2o: Dynamic discriminative operations for efficient generative inference of large language models , author=. arXiv preprint arXiv:2406.13035 , year=

    arXiv

  17. [25]

    arXiv preprint arXiv:2407.08454 , year=

    Model tells you where to merge: Adaptive kv cache merging for llms on long-context tasks , author=. arXiv preprint arXiv:2407.08454 , year=

    arXiv

  18. [26]

    arXiv preprint arXiv:2406.06484 , year=

    Parallelizing linear transformers with the delta rule over sequence length , author=. arXiv preprint arXiv:2406.06484 , year=

    arXiv

  19. [27]

    arXiv preprint arXiv:2407.04620 , year=

    Learning to (learn at test time): Rnns with expressive hidden states , author=. arXiv preprint arXiv:2407.04620 , year=

    arXiv

  20. [28]

    arXiv preprint arXiv:2501.00663 , year=

    Titans: Learning to memorize at test time , author=. arXiv preprint arXiv:2501.00663 , year=

    arXiv

  21. [29]

    arXiv preprint arXiv:2504.05646 , year=

    Lattice: Learning to efficiently compress the memory , author=. arXiv preprint arXiv:2504.05646 , year=

  22. [30]

    arXiv preprint arXiv:2410.13276 , year=

    Seerattention: Learning intrinsic sparse attention in your llms , author=. arXiv preprint arXiv:2410.13276 , year=

    arXiv

  23. [31]

    arXiv preprint arXiv:2503.06692 , year=

    Inftythink: Breaking the length limits of long-context reasoning in large language models , author=. arXiv preprint arXiv:2503.06692 , year=

  24. [32]

    arXiv preprint arXiv:2505.24133 , year=

    R-KV: Redundancy-aware KV Cache Compression for Training-Free Reasoning Models Acceleration , author=. arXiv preprint arXiv:2505.24133 , year=

  25. [33]

    arXiv preprint arXiv:2503.00979 , year=

    Dialogue without limits: Constant-sized kv caches for extended responses in llms , author=. arXiv preprint arXiv:2503.00979 , year=

    arXiv

  26. [34]

    arXiv preprint arXiv:2403.09636 , year=

    Dynamic memory compression: Retrofitting llms for accelerated inference , author=. arXiv preprint arXiv:2403.09636 , year=

    arXiv

  27. [35]

    arXiv preprint arXiv:2404.15574 , year=

    Retrieval head mechanistically explains long-context factuality , author=. arXiv preprint arXiv:2404.15574 , year=

    arXiv

  28. [36]

    Advances in Neural Information Processing Systems , volume=

    Minference 1.0: Accelerating pre-filling for long-context llms via dynamic sparse attention , author=. Advances in Neural Information Processing Systems , volume=

  29. [37]

    arXiv preprint arXiv:1905.09418 , year=

    Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned , author=. arXiv preprint arXiv:1905.09418 , year=

    1905 arXiv

  30. [38]

    arXiv preprint arXiv:2307.08691 , year=

    Flashattention-2: Faster attention with better parallelism and work partitioning , author=. arXiv preprint arXiv:2307.08691 , year=

    arXiv

  31. [39]

    arXiv preprint arXiv:2412.05496 , year=

    Flex attention: A programming model for generating optimized attention kernels , author=. arXiv preprint arXiv:2412.05496 , year=

    arXiv

  32. [40]

    Annals of neurosciences , volume=

    [image] Memory: A Contribution to Experimental Psychology , author=. Annals of neurosciences , volume=

  33. [41]

    Let's (not) just put things in Context: Test-time Training for Long-context

    Rachit Bansal and Aston Zhang and Rishabh Tiwari and Lovish Madaan and Sai Surya Duvvuri and Fnu Devvrit and David Brandfonbrener and David Alvarez-Melis and Prajjwal Bhargava and Mihir Kale and Samy Jelassi , booktitle=. Let's (not) just put things in Context: Test-time Train...

  34. [42]

    Acta neurobiologiae experimentalis , volume=

    Two components of long-term memory , author=. Acta neurobiologiae experimentalis , volume=

  35. [43]

    arXiv preprint arXiv:2506.08889 , year=

    SeerAttention-R: Sparse Attention Adaptation for Long Reasoning , author=. arXiv preprint arXiv:2506.08889 , year=

    arXiv

  36. [44]

    arXiv preprint arXiv:2103.03874 , year=

    Measuring mathematical problem solving with the math dataset , author=. arXiv preprint arXiv:2103.03874 , year=

    arXiv

  37. [45]

    Open r1: A Fully Open Reproduction of DeepSeek-R1 , date =

  38. [46]

    arXiv preprint arXiv:2110.14168 , year=

    Training verifiers to solve math word problems , author=. arXiv preprint arXiv:2110.14168 , year=

    arXiv

  39. [47]

    arXiv preprint arXiv:2501.05414 , year=

    Longproc: Benchmarking long-context language models on long procedural generation , author=. arXiv preprint arXiv:2501.05414 , year=

  40. [48]

    arXiv preprint arXiv:2501.12948 , year=

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning , author=. arXiv preprint arXiv:2501.12948 , year=

    arXiv

  41. [49]

    arXiv preprint arXiv:2505.09388 , year=

    Qwen3 technical report , author=. arXiv preprint arXiv:2505.09388 , year=

    arXiv

  42. [50]

    The annals of mathematical statistics , volume=

    On information and sufficiency , author=. The annals of mathematical statistics , volume=. 1951 , publisher=

    1951

  43. [51]

    arXiv preprint arXiv:2503.08879 , year=

    LLMs Know What to Drop: Self-Attention Guided KV Cache Eviction for Efficient Long-Context Inference , author=. arXiv preprint arXiv:2503.08879 , year=

    arXiv

  44. [52]

    arXiv preprint arXiv:2410.02660 , year=

    How to train long-context language models (effectively) , author=. arXiv preprint arXiv:2410.02660 , year=

  45. [53]

    arXiv preprint arXiv:2412.19442 , year=

    A survey on large language model acceleration based on kv cache management , author=. arXiv preprint arXiv:2412.19442 , year=

    arXiv

  46. [54]

    arXiv preprint arXiv:2406.10774 , year=

    Quest: Query-aware sparsity for efficient long-context llm inference , author=. arXiv preprint arXiv:2406.10774 , year=

    arXiv

  47. [55]

    Advances in Neural Information Processing Systems , volume=

    Kvquant: Towards 10 million context length llm inference with kv cache quantization , author=. Advances in Neural Information Processing Systems , volume=

  48. [56]

    arXiv preprint arXiv:2504.15364 , year=

    KeyDiff: Key Similarity-Based KV Cache Eviction for Long-Context LLM Inference in Resource-Constrained Environments , author=. arXiv preprint arXiv:2504.15364 , year=

  49. [57]

    arXiv preprint arXiv:2406.02069 , year=

    Pyramidkv: Dynamic kv cache compression based on pyramidal information funneling , author=. arXiv preprint arXiv:2406.02069 , year=

    arXiv

  50. [58]

    arXiv preprint arXiv:2501.07278 , year=

    Lifelong learning of large language model based agents: A roadmap , author=. arXiv preprint arXiv:2501.07278 , year=

  51. [59]

    arXiv preprint arXiv:2503.09567 , year=

    Towards reasoning era: A survey of long chain-of-thought for reasoning large language models , author=. arXiv preprint arXiv:2503.09567 , year=

    arXiv

  52. [60]

    Advances in neural information processing systems , volume=

    Long-short transformer: Efficient transformers for language and vision , author=. Advances in neural information processing systems , volume=

  53. [61]

    arXiv preprint arXiv:2410.12876 , year=

    In-context kv-cache eviction for llms via attention-gate , author=. arXiv preprint arXiv:2410.12876 , year=

    arXiv

  54. [62]

    Advances in Neural Information Processing Systems , volume=

    Learning to compress prompts with gist tokens , author=. Advances in Neural Information Processing Systems , volume=

  55. [63]

    arXiv preprint arXiv:2502.11089 , year=

    Native sparse attention: Hardware-aligned and natively trainable sparse attention , author=. arXiv preprint arXiv:2502.11089 , year=

    arXiv

  56. [65]

    2025 , urldate =

    Qwen/Qwen3-4B-Instruct-2507 , date =. 2025 , urldate =

    2025

  57. [66]

    2025 , howpublished =

    Lazarevich, Ivan and Bick, David and Gupta, Harsh and Mukherjee, Srinjoy and Neema, Nishit and Ramakrishnan, Gokul and Venkatesh, Ganesh , title =. 2025 , howpublished =

    2025

  58. [67]

    2021 , eprint=

    BookSum: A Collection of Datasets for Long-form Narrative Summarization , author=. 2021 , eprint=

    2021

  59. [68]

    2024 , month =

    Chaitanya Singhal , title =. 2024 , month =

    2024

  60. [69]

    arXiv preprint arXiv:2407.20485 , year=

    A2sf: Accumulative attention scoring with forgetting factor for token pruning in transformer decoder , author=. arXiv preprint arXiv:2407.20485 , year=

    arXiv

  61. [70]

    arXiv preprint arXiv:2408.03675 , year=

    Nacl: A general and effective kv cache eviction framework for llms at inference time , author=. arXiv preprint arXiv:2408.03675 , year=

    arXiv

  62. [71]

    arXiv preprint arXiv:2509.09754 , year=

    LAVa: Layer-wise KV Cache Eviction with Dynamic Budget Allocation , author=. arXiv preprint arXiv:2509.09754 , year=

    arXiv

  63. [72]

    Advances in Neural Information Processing Systems , volume=

    Scissorhands: Exploiting the persistence of importance hypothesis for llm kv cache compression at test time , author=. Advances in Neural Information Processing Systems , volume=

  64. [73]

    arXiv preprint arXiv:2402.02750 , year=

    Kivi: A tuning-free asymmetric 2bit quantization for kv cache , author=. arXiv preprint arXiv:2402.02750 , year=

    arXiv

  65. [74]

    arXiv preprint arXiv:2402.12065 , year=

    Wkvquant: Quantizing weight and key/value cache for large language models gains more , author=. arXiv preprint arXiv:2402.12065 , year=

    arXiv

  66. [75]

    arXiv preprint arXiv:2410.21465 , year=

    Shadowkv: Kv cache in shadows for high-throughput long-context llm inference , author=. arXiv preprint arXiv:2410.21465 , year=

    arXiv

  67. [76]

    arXiv preprint arXiv:2409.10516 , year=

    Retrievalattention: Accelerating long-context llm inference via vector retrieval , author=. arXiv preprint arXiv:2409.10516 , year=

    arXiv

  68. [77]

    arXiv preprint arXiv:2405.17424 , year=

    LARM: Large Auto-Regressive Model for Long-Horizon Embodied Intelligence , author=. arXiv preprint arXiv:2405.17424 , year=

    arXiv

  69. [78]

    arXiv preprint arXiv:2503.12491 , year=

    Cake: Cascading and adaptive kv cache eviction with layer preferences , author=. arXiv preprint arXiv:2503.12491 , year=

  70. [79]

    arXiv preprint arXiv:2502.01941 , year=

    Can LLMs Maintain Fundamental Abilities under KV Cache Compression? , author=. arXiv preprint arXiv:2502.01941 , year=

    arXiv

  71. [80]

    International conference on machine learning , pages=

    Transformers are rnns: Fast autoregressive transformers with linear attention , author=. International conference on machine learning , pages=. 2020 , organization=

    2020

  72. [81]

    arXiv preprint arXiv:2006.04768 , year=

    Linformer: Self-attention with linear complexity , author=. arXiv preprint arXiv:2006.04768 , year=

    2006 arXiv

  73. [82]

    arXiv preprint arXiv:2307.08621 , year=

    Retentive network: A successor to transformer for large language models , author=. arXiv preprint arXiv:2307.08621 , year=

    arXiv

  74. [83]

    arXiv preprint arXiv:2312.06635 , year=

    Gated linear attention transformers with hardware-efficient training , author=. arXiv preprint arXiv:2312.06635 , year=

    arXiv

  75. [84]

    TRELLIS: Learning to Compress Key-Value Memory in Attention Models , author=

  76. [85]

    arXiv preprint arXiv:2408.05646 , year=

    Eigen attention: Attention in low-rank space for kv cache compression , author=. arXiv preprint arXiv:2408.05646 , year=

    arXiv

  77. [86]

    International Conference on Machine Learning , pages=

    Large language models can be easily distracted by irrelevant context , author=. International Conference on Machine Learning , pages=. 2023 , organization=

    2023

  78. [87]

    International conference on algorithmic learning theory , pages=

    On the computational complexity of self-attention , author=. International conference on algorithmic learning theory , pages=. 2023 , organization=

    2023

  79. [88]

    arXiv preprint arXiv:2512.03324 , year=

    Cache what lasts: Token retention for memory-bounded kv cache in llms , author=. arXiv preprint arXiv:2512.03324 , year=

  80. [89]

    arXiv preprint arXiv:2410.23317 , year=

    VL-cache: Sparsity and modality-aware KV cache compression for vision-language model inference acceleration , author=. arXiv preprint arXiv:2410.23317 , year=

    arXiv

  81. [90]

    arXiv preprint arXiv:2506.15724 , year=

    MadaKV: Adaptive Modality-Perception KV Cache Eviction for Efficient Multimodal Long-Context Inference , author=. arXiv preprint arXiv:2506.15724 , year=

    arXiv

  82. [91]

    arXiv preprint arXiv:2503.06749 , year=

    Vision-r1: Incentivizing reasoning capability in multimodal large language models , author=. arXiv preprint arXiv:2503.06749 , year=

    arXiv

  83. [92]

    arXiv preprint arXiv:2505.04769 , year=

    Vision-language-action models: Concepts, progress, applications and challenges , author=. arXiv preprint arXiv:2505.04769 , year=

  84. [93]

    Advances in Neural Information Processing Systems , volume=

    Streaming long video understanding with large language models , author=. Advances in Neural Information Processing Systems , volume=

  85. [94]

    Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

    Towards long-horizon vision-language navigation: Platform, benchmark and method , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

  86. [95]

    MMMU -Pro: A More Robust Multi-discipline Multimodal Understanding Benchmark

    Yue, Xiang and Zheng, Tianyu and Ni, Yuansheng and Wang, Yubo and Zhang, Kai and Tong, Shengbang and Sun, Yuxuan and Yu, Botao and Zhang, Ge and Sun, Huan and Su, Yu and Chen, Wenhu and Neubig, Graham. MMMU -Pro: A More Robust Multi-discipline Multimodal Understanding Benchmar...

    2025 doi

  87. [96]

    arXiv preprint arXiv:2506.05349 , year=

    Videomathqa: Benchmarking mathematical reasoning via multimodal understanding in videos , author=. arXiv preprint arXiv:2506.05349 , year=

    arXiv

  88. [97]

    arXiv preprint arXiv:2412.16117 , year=

    Prunevid: Visual token pruning for efficient video large language models , author=. arXiv preprint arXiv:2412.16117 , year=

    arXiv

  89. [98]

    arXiv preprint arXiv:2406.18139 , year=

    Look-m: Look-once optimization in kv cache for efficient multimodal long-context inference , author=. arXiv preprint arXiv:2406.18139 , year=

    arXiv

  90. [99]

    arXiv preprint arXiv:2506.10967 , year=

    Beyond Attention or Similarity: Maximizing Conditional Diversity for Token Pruning in MLLMs , author=. arXiv preprint arXiv:2506.10967 , year=

    arXiv

  91. [100]

    Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

    Divprune: Diversity-based visual token pruning for large multimodal models , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

  92. [101]

    European Conference on Computer Vision , pages=

    An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models , author=. European Conference on Computer Vision , pages=. 2024 , organization=

    2024

  93. [102]

    arXiv preprint arXiv:2407.11550 , year=

    Ada-kv: Optimizing kv cache eviction by adaptive budget allocation for efficient llm inference , author=. arXiv preprint arXiv:2407.11550 , year=

    arXiv

  94. [103]

    arXiv preprint arXiv:2503.17407 , year=

    A comprehensive survey on long context language modeling , author=. arXiv preprint arXiv:2503.17407 , year=

  95. [104]

    Neurocomputing , volume=

    Roformer: Enhanced transformer with rotary position embedding , author=. Neurocomputing , volume=. 2024 , publisher=

    2024

  96. [105]

    2024 , eprint=

    LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Models , author=. 2024 , eprint=

    2024

  97. [106]

    LMMs-Eval: Accelerating the Development of Large Multimoal Models , url=

    Bo Li, Peiyuan Zhang, Kaichen Zhang, Fanyi Pu, Xinrun Du, Yuhao Dong, Haotian Liu, Yuanhan Zhang, Ge Zhang, Chunyuan Li and Ziwei Liu , publisher =. LMMs-Eval: Accelerating the Development of Large Multimoal Models , url=

  98. [107]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Towards vqa models that can read , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  99. [108]

    Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

    Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

  100. [109]

    Advances in Neural Information Processing Systems , volume=

    Measuring multimodal mathematical reasoning with math-vision dataset , author=. Advances in Neural Information Processing Systems , volume=

  101. [110]

    The Thirty-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track , year=

    MME: A comprehensive evaluation benchmark for multimodal large language models , author=. The Thirty-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track , year=

  102. [111]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Gqa: A new dataset for real-world visual reasoning and compositional question answering , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  103. [112]

    Advances in Neural Information Processing Systems , volume=

    Are we on the right way for evaluating large vision-language models? , author=. Advances in Neural Information Processing Systems , volume=

  104. [113]

    Ziyu Liu and Tao Chu and Yuhang Zang and Xilin Wei and Xiaoyi Dong and Pan Zhang and Zijian Liang and Yuanjun Xiong and Yu Qiao and Dahua Lin and Jiaqi Wang , booktitle=

  105. [114]

    arXiv preprint arXiv:2501.13826 , year=

    Video-mmmu: Evaluating knowledge acquisition from multi-discipline professional videos , author=. arXiv preprint arXiv:2501.13826 , year=

    arXiv

  106. [115]

    2025 , eprint=

    Qwen3 Technical Report , author=. 2025 , eprint=

    2025

  107. [116]

    arXiv preprint arXiv:2502.13923 , year=

    Qwen2.5-VL Technical Report , author=. arXiv preprint arXiv:2502.13923 , year=

    arXiv

  108. [117]

    arXiv preprint arXiv:2409.12191 , year=

    Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution , author=. arXiv preprint arXiv:2409.12191 , year=

    arXiv

  109. [118]

    arXiv preprint arXiv:2308.12966 , year=

    Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond , author=. arXiv preprint arXiv:2308.12966 , year=

    arXiv

  110. [119]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Improved baselines with visual instruction tuning , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  111. [120]

    IEEE Transactions on Mobile Computing , volume=

    Adapyramid: Adaptive pyramid for accelerating high-resolution object detection on edge devices , author=. IEEE Transactions on Mobile Computing , volume=. 2023 , publisher=

    2023

  112. [121]

    arXiv preprint arXiv:2503.10615 , year=

    R1-onevision: Advancing generalized multimodal reasoning through cross-modal formalization , author=. arXiv preprint arXiv:2503.10615 , year=

    arXiv

  113. [122]

    Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

    Visionzip: Longer is better but not necessary in vision language models , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

  114. [123]

    arXiv preprint arXiv:2502.11494 , year=

    Stop looking for important tokens in multimodal language models: Duplication matters more , author=. arXiv preprint arXiv:2502.11494 , year=

    arXiv

  115. [124]

    Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

    Llava-prumerge: Adaptive token reduction for efficient large multimodal models , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

  116. [125]

    arXiv preprint arXiv:2407.07895 , year=

    Llava-next-interleave: Tackling multi-image, video, and 3d in large multimodal models , author=. arXiv preprint arXiv:2407.07895 , year=

    arXiv

  117. [126]

    arXiv preprint arXiv:2410.04417 , year=

    Sparsevlm: Visual token sparsification for efficient vision-language model inference , author=. arXiv preprint arXiv:2410.04417 , year=

    arXiv

  118. [127]

    arXiv preprint arXiv:2502.17599 , year=

    Meda: Dynamic kv cache allocation for efficient multimodal long-context inference , author=. arXiv preprint arXiv:2502.17599 , year=

    arXiv

  119. [128]

    Proceedings of the 31st International Conference on Computational Linguistics , year=

    Zigzagkv: Dynamic kv cache compression for long-context modeling based on layer uncertainty , author=. Proceedings of the 31st International Conference on Computational Linguistics , year=

  120. [129]

    Proceedings of the 29th symposium on operating systems principles , pages=

    Efficient memory management for large language model serving with pagedattention , author=. Proceedings of the 29th symposium on operating systems principles , pages=

  121. [130]

    arXiv preprint arXiv:2312.11805 , year=

    Gemini: a family of highly capable multimodal models , author=. arXiv preprint arXiv:2312.11805 , year=

    arXiv

  122. [131]

    arXiv preprint arXiv:2507.06261 , year=

    Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities , author=. arXiv preprint arXiv:2507.06261 , year=

    arXiv

  123. [132]

    Advances in neural information processing systems , volume=

    Visual instruction tuning , author=. Advances in neural information processing systems , volume=

  124. [133]

    LLaVA-NeXT: Improved reasoning, OCR, and world knowledge , url=

    Liu, Haotian and Li, Chunyuan and Li, Yuheng and Li, Bo and Zhang, Yuanhan and Shen, Sheng and Lee, Yong Jae , month=. LLaVA-NeXT: Improved reasoning, OCR, and world knowledge , url=

  125. [134]

    2024 , eprint=

    Video Instruction Tuning With Synthetic Data , author=. 2024 , eprint=

    2024

  126. [135]

    Advances in neural information processing systems , volume=

    Pay attention to mlps , author=. Advances in neural information processing systems , volume=

  127. [136]

    arXiv preprint arXiv:2108.12409 , year=

    Train short, test long: Attention with linear biases enables input length extrapolation , author=. arXiv preprint arXiv:2108.12409 , year=

    arXiv

  128. [137]

    Advances in Neural Information Processing Systems , volume=

    Found in the middle: How language models use long contexts better via plug-and-play positional encoding , author=. Advances in Neural Information Processing Systems , volume=

  129. [138]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    How is llm reasoning distracted by irrelevant context? an analysis using a controlled benchmark , author=. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    2025

  130. [139]

    Transactions of the association for computational linguistics , volume=

    Lost in the middle: How language models use long contexts , author=. Transactions of the association for computational linguistics , volume=

  131. [140]

    How sparse attention approximates exact attention? your attention is naturally

    Deng, Yichuan and Song, Zhao and Xiong, Jing and Yang, Chiwun , journal=. How sparse attention approximates exact attention? your attention is naturally

  132. [141]

    Advances in Neural Information Processing Systems , volume=

    Scatterbrain: Unifying sparse and low-rank attention , author=. Advances in Neural Information Processing Systems , volume=

  133. [142]

    arXiv preprint arXiv:2009.14794 , year=

    Rethinking attention with performers , author=. arXiv preprint arXiv:2009.14794 , year=

    2009 arXiv

  134. [143]

    Xiong, Yunyang and Zeng, Zhanpeng and Chakraborty, Rudrasis and Tan, Mingxing and Fung, Glenn and Li, Yin and Singh, Vikas , booktitle=. Nystr

  135. [144]

    Advances in neural information processing systems , volume=

    Attention is all you need , author=. Advances in neural information processing systems , volume=

  136. [145]

    arXiv preprint arXiv:2512.06866 , year=

    Less Is More, but Where? Dynamic Token Compression via LLM-Guided Keyframe Prior , author=. arXiv preprint arXiv:2512.06866 , year=

  137. [146]

    and Roth, Dan , journal=

    Jiang, Bowen and Hao, Zhuoqun and Cho, Young-Min and Li, Bryan and Yuan, Yuan and Chen, Sihao and Ungar, Lyle and Taylor, Camillo J. and Roth, Dan , journal=. Know Me, Respond to Me: Benchmarking

  138. [147]

    Jiang, Bowen and Yuan, Yuan and Shen, Maohao and Hao, Zhuoqun and Xu, Zhangchen and Chen, Zichen and Liu, Ziyi and Vijjini, Anvesh Rao and He, Jiashu and Yu, Hanchao and Poovendran, Radha and Wornell, Gregory and Ungar, Lyle and Roth, Dan and Chen, Sihao and Taylor, Camillo Jo...

  139. [148]

    Li, Shuyue Stella and Paranjape, Bhargavi and Oktar, Kerem and Ma, Zhongyao and Zhou, Gelin and Guan, Lin and Zhang, Na and Park, Sem and Chen, Lin and Yang, Diyi and Tsvetkov, Yulia and Celikyilmaz, Asli , journal=

  140. [149]

    Hu, Chuanrui and Gao, Xingze and Zhou, Zuyi and Xu, Dannong and Bai, Yi and Li, Xintong and Zhang, Hui and Li, Tong and Zhang, Chong and Bing, Lidong and Deng, Yafeng , journal=

  141. [150]

    arXiv preprint arXiv:2605.05583 , year=

    Belief Memory: Agent Memory Under Partial Observability , author=. arXiv preprint arXiv:2605.05583 , year=

    arXiv

  142. [151]

    Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Hypermem: Hypergraph memory for long-term conversations , author=. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  143. [152]

    Transactions of the Association for Computational Linguistics , volume=

    Lost in the Middle: How Language Models Use Long Contexts , author=. Transactions of the Association for Computational Linguistics , volume=

  144. [153]

    Yang, An and others , journal=

  145. [154]

    Chen, Jianlv and Xiao, Shitao and Zhang, Peitian and Luo, Kun and Lian, Defu and Liu, Zheng , journal=

  146. [155]

    and Zhang, Hao and Stoica, Ion , booktitle=

    Kwon, Woosuk and Li, Zhuohan and Zhuang, Siyuan and Sheng, Ying and Zheng, Lianmin and Yu, Cody Hao and Gonzalez, Joseph E. and Zhang, Hao and Stoica, Ion , booktitle=. Efficient Memory Management for Large Language Model Serving with

  147. [156]

    Brown, Dorian , year=

  148. [157]

    Evaluating Very Long-Term Conversational Memory of

    Maharana, Adyasha and Lee, Dong-Ho and Tulyakov, Sergey and Bansal, Mohit and Barbieri, Francesco and Fang, Yuwei , booktitle=. Evaluating Very Long-Term Conversational Memory of

  149. [158]

    Dynamic Epistemic Logic

    Pacuit, Eric , journal=. Dynamic Epistemic Logic. 2013 , publisher=

    2013

  150. [159]

    2007 , publisher=

    Dynamic Epistemic Logic , author=. 2007 , publisher=

    2007

  151. [160]

    International Conference on Machine Learning (ICML) , pages=

    Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , author=. International Conference on Machine Learning (ICML) , pages=

  152. [161]

    Advances in Neural Information Processing Systems , year =

    Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks , author =. Advances in Neural Information Processing Systems , year =. 2005.11401 , archivePrefix =

    2005 arXiv

  153. [162]

    arXiv preprint arXiv:2304.03442 , year =

    Generative Agents: Interactive Simulacra of Human Behavior , author =. arXiv preprint arXiv:2304.03442 , year =

    arXiv

  154. [163]

    arXiv preprint arXiv:2305.10250 , year =

    MemoryBank: Enhancing Large Language Models with Long-Term Memory , author =. arXiv preprint arXiv:2305.10250 , year =

    arXiv

  155. [164]

    arXiv preprint arXiv:2310.08560 , year =

    MemGPT: Towards LLMs as Operating Systems , author =. arXiv preprint arXiv:2310.08560 , year =

    arXiv

  156. [165]

    arXiv preprint arXiv:2504.19413 , year =

    Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory , author =. arXiv preprint arXiv:2504.19413 , year =

    arXiv

  157. [166]

    arXiv preprint arXiv:2505.22101 , year =

    MemOS: An Operating System for Memory-Augmented Generation (MAG) in Large Language Models , author =. arXiv preprint arXiv:2505.22101 , year =

    arXiv

  158. [167]

    International Conference on Learning Representations , year =

    LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory , author =. International Conference on Learning Representations , year =. 2410.10813 , archivePrefix =

    arXiv

  159. [168]

    Proceedings of the ACM Web Conference , year =

    ES-MemEval: Benchmarking Conversational Agents on Personalized Long-Term Emotional Support , author =. Proceedings of the ACM Web Conference , year =. 2602.01885 , archivePrefix =

  160. [169]

    Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics , year =

    Neural Belief Tracker: Data-Driven Dialogue State Tracking , author =. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics , year =. 1606.03777 , archivePrefix =

    arXiv

  161. [170]

    Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , year =

    Dialogue State Tracking with a Language Model using Schema-Driven Prompting , author =. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , year =. 2109.07506 , archivePrefix =

    2021 arXiv

  162. [171]

    arXiv preprint arXiv:2303.11366 , year =

    Reflexion: Language Agents with Verbal Reinforcement Learning , author =. arXiv preprint arXiv:2303.11366 , year =

    arXiv

  163. [172]

    arXiv preprint arXiv:2205.14334 , year =

    Teaching Models to Express Their Uncertainty in Words , author =. arXiv preprint arXiv:2205.14334 , year =

    arXiv

  164. [173]

    International Conference on Learning Representations , year =

    Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation , author =. International Conference on Learning Representations , year =. 2302.09664 , archivePrefix =

    arXiv

  165. [174]

    International Conference on Machine Learning, Position Track , year =

    Hallucinations Undermine Trust; Metacognition is a Way Forward , author =. International Conference on Machine Learning, Position Track , year =. 2605.01428 , archivePrefix =

    arXiv

  166. [175]

    arXiv preprint arXiv:2511.03506 , year=

    Halumem: Evaluating hallucinations in memory systems of agents , author=. arXiv preprint arXiv:2511.03506 , year=

  167. [176]

    arXiv preprint arXiv:2605.20926 , year=

    MemConflict: Evaluating Long-Term Memory Systems Under Memory Conflicts , author=. arXiv preprint arXiv:2605.20926 , year=

    arXiv

  168. [177]

    arXiv preprint arXiv:2605.06527 , year=

    STALE: Can LLM Agents Know When Their Memories Are No Longer Valid? , author=. arXiv preprint arXiv:2605.06527 , year=

    arXiv

  169. [178]

    arXiv preprint arXiv:2509.04664 , year=

    Why language models hallucinate , author=. arXiv preprint arXiv:2509.04664 , year=

    arXiv

  170. [179]

    arXiv preprint arXiv:2207.05221 , year=

    Language models (mostly) know what they know , author=. arXiv preprint arXiv:2207.05221 , year=

    arXiv

  171. [180]

    Findings of the Association for Computational Linguistics: EMNLP 2024 , pages=

    Diahalu: A dialogue-level hallucination evaluation benchmark for large language models , author=. Findings of the Association for Computational Linguistics: EMNLP 2024 , pages=

    2024

  172. [181]

    Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    Multichallenge: A realistic multi-turn conversation evaluation benchmark challenging to frontier llms , author=. Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    2025

  173. [182]

    Computational Linguistics , volume=

    Siren’s Song in the AI Ocean: A Survey on Hallucination in Large Language Models , author=. Computational Linguistics , volume=. 2025 , publisher=

    2025

  174. [183]

    Physbench: Benchmarking and enhancing vision-language models for physical world understanding

    Chow, Wei and Mao, Jiageng and Li, Boyi and Seita, Daniel and Campagnolo Guizilini, Vitor and Wang, Yue. Physbench: Benchmarking and enhancing vision-language models for physical world understanding. The Thirteenth International Conference on Learning Representations

  175. [184]

    2024 , eprint=

    PhysGame: Uncovering Physical Commonsense Violations in Gameplay Videos , author=. 2024 , eprint=

    2024

  176. [185]

    2026 , eprint=

    UniVideo: Unified Understanding, Generation, and Editing for Videos , author=. 2026 , eprint=

    2026

  177. [186]

    2026 , eprint=

    LLaVA-OneVision-2: Towards Next-Generation Perceptual Intelligence , author=. 2026 , eprint=

    2026

  178. [187]

    Do generative video models understand physical principles?

    Motamed, Saman and Culp, Laura and Swersky, Kevin and Jaini, Priyank and Geirhos, Robert. Do generative video models understand physical principles?. 2026 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)

    2026

  179. [188]

    2025 , eprint=

    A Shortcut-aware Video-QA Benchmark for Physical Understanding via Minimal Video Pairs , author=. 2025 , eprint=

    2025

  180. [189]

    UNIVID: Unified Vision-Language Model for Video Moderation

    Yang, Kejuan and Zhang, Yizhuo and Du, Mingyuan and Zhang, Yue and Zheng, Dixin and Zhao, Kaili and Xiao, Yang and Liang, Hanzhong and Xiao, Kenan. UNIVID: Unified Vision-Language Model for Video Moderation. Proceedings of the 64th Annual Meeting of the Association for Computa...

    2026

  181. [190]

    Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi

    Yue, Xiang and Ni, Yuansheng and Zhang, Kai and Zheng, Tianyu and Liu, Ruoqi and Zhang, Ge and Stevens, Samuel and Jiang, Dongfu and Ren, Weiming and Sun, Yuxuan and others. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. Proce...

  182. [191]

    Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts

    Lu, Pan and Bansal, Hritik and Xia, Tony and Liu, Jiacheng and Li, Chunyuan and Hajishirzi, Hannaneh and Cheng, Hao and Chang, Kai-Wei and Galley, Michel and Gao, Jianfeng. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. International Conf...

  183. [192]

    NEWTON: Are large language models capable of physical reasoning?

    Wang, Yi and Duan, Jiafei and Fox, Dieter and Srinivasa, Siddhartha. NEWTON: Are large language models capable of physical reasoning?. Findings of the Association for Computational Linguistics: EMNLP 2023

    2023

  184. [193]

    Cogvideox: Text-to-video diffusion models with an expert transformer

    Yang, Zhuoyi and Teng, Jiayan and Zheng, Wendi and Ding, Ming and Huang, Shiyu and Xu, Jiazheng and Yang, Yuanming and Hong, Wenyi and Zhang, Xiaohan and Feng, Guanyu and others. Cogvideox: Text-to-video diffusion models with an expert transformer. International Conference on ...

  185. [194]

    Videocrafter2: Overcoming data limitations for high-quality video diffusion models

    Chen, Haoxin and Zhang, Yong and Cun, Xiaodong and Xia, Menghan and Wang, Xintao and Weng, Chao and Shan, Ying. Videocrafter2: Overcoming data limitations for high-quality video diffusion models. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

  186. [195]

    2024 , eprint=

    MOKA: Open-World Robotic Manipulation through Mark-Based Visual Prompting , author=. 2024 , eprint=

    2024

  187. [196]

    2024 , eprint=

    PhyGrasp: Generalizing Robotic Grasping with Physics-informed Large Multimodal Models , author=. 2024 , eprint=

    2024

  188. [197]

    2024 , eprint=

    OpenVLA: An Open-Source Vision-Language-Action Model , author=. 2024 , eprint=

    2024

  189. [198]

    2024 , eprint=

    LLARVA: Vision-Action Instruction Tuning Enhances Robot Learning , author=. 2024 , eprint=

    2024

  190. [199]

    2024 , eprint=

    3D-VLA: A 3D Vision-Language-Action Generative World Model , author=. 2024 , eprint=

    2024

  191. [200]

    2025 , eprint=

    VideoPhy-2: A Challenging Action-Centric Physical Commonsense Evaluation in Video Generation , author=. 2025 , eprint=

    2025

  192. [201]

    2024 , eprint=

    VideoPhy: Evaluating Physical Commonsense for Video Generation , author=. 2024 , eprint=

    2024

  193. [202]

    2025 , eprint=

    VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulness , author=. 2025 , eprint=

    2025

  194. [203]

    2024 , eprint=

    Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generation , author=. 2024 , eprint=

    2024

  195. [204]

    2026 , eprint=

    Skyra: AI-Generated Video Detection via Grounded Artifact Reasoning , author=. 2026 , eprint=

    2026

  196. [205]

    2022 , note =

    A Path Towards Autonomous Machine Intelligence , author =. 2022 , note =

    2022

  197. [206]

    2025 , note =

    Luma Ray2 , author =. 2025 , note =

    2025

  198. [207]

    2024 , note=

    Video generation models as world simulators , author=. 2024 , note=

    2024

  199. [208]

    2025 , eprint=

    Qwen2.5-VL Technical Report , author=. 2025 , eprint=

    2025

  200. [209]

    2025 , eprint=

    Emerging Properties in Unified Multimodal Pretraining , author=. 2025 , eprint=

    2025

  201. [210]

    2025 , eprint=

    HunyuanVideo: A Systematic Framework For Large Video Generative Models , author=. 2025 , eprint=

    2025

  202. [211]

    2025 , eprint=

    Cosmos World Foundation Model Platform for Physical AI , author=. 2025 , eprint=

    2025

  203. [212]

    2025 , eprint=

    Wan: Open and Advanced Large-Scale Video Generative Models , author=. 2025 , eprint=

    2025

  204. [213]

    ACM Computing Surveys , volume=

    Understanding World or Predicting Future? A Comprehensive Survey of World Models , author=. ACM Computing Surveys , volume=. 2025 , publisher=

    2025

  205. [214]

    2026 , eprint=

    MVPBench: A Multi-Video Perception Evaluation Benchmark for Multi-Modal Video Understanding , author=. 2026 , eprint=

    2026

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.