Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Large multimodal models fail at inductive physical reasoning: a new benchmark shows they cannot infer physics that contradicts what they learned in training, and what little reasoning they do is driven by language, not vision.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 17:41 UTC pith:OSBUDV5U

load-bearing objection A solid new benchmark with a robust language-bias finding, but the 'weak inductive reasoning' claim needs calibration against human/oracle baselines. the 4 major comments →

arxiv 2509.12263 v3 pith:OSBUDV5U submitted 2025-09-12 cs.AI cs.LG

InPhyRe Discovers: Large Multimodal Models Struggle in Inductive Physical Reasoning

Gautam Sreekumar , Vishnu Naresh Boddeti This is my paper
classification cs.AI cs.LG
keywords inductive physical reasoninglarge multimodal modelsvisual question answeringbenchmarkphysical reasoninglanguage biasmomentum conservationin-context learning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces InPhyRe, a visual-question-answering benchmark that tests whether large multimodal models (LMMs) can infer physical laws from a few demonstration videos, including collisions that violate universal laws such as momentum conservation. Across 13 open and proprietary models, it finds that LMMs use demonstrations only when those demonstrations confirm what the model already knows; when the demonstrated physics contradicts their parametric knowledge, few-shot accuracy drops sharply. The paper also shows that the limited inductive physical reasoning that does occur is largely language-driven—when exemplars contain only videos, accuracy falls to near chance for several models. The authors conclude that LMMs treat physical laws as fixed rules rather than transferable mathematical models, casting doubt on their trustworthiness in safety-critical applications.

Core claim

InPhyRe is the first visual question-answering benchmark purpose-built to measure inductive physical reasoning in LMMs by confronting them with collision videos that violate real-world physical laws, generated by intervening in a physics simulator. The paper reports three findings from 13 models: (1) LMMs can recite momentum and energy conservation but apply these laws inconsistently even in regular scenarios; (2) when exemplar videos follow laws unseen in training, almost all models show a substantial accuracy drop relative to regular scenarios, indicating weak inductive physical reasoning; and (3) removing question-answer pairs from the exemplars (video-only) cuts accuracy dramatically, sh

What carries the argument

The central object is InPhyRe, a synthetic benchmark of collision videos whose trajectories are generated by manually overriding a PyBullet simulation at the moment of collision so that they violate laws such as momentum conservation. Scenarios are grouped into irregular (law-violating) and regular (law-abiding) counterparts, and the key metric is the difference between few-shot accuracy in the irregular scenario and the model's best regular-scenario accuracy; a negative value is interpreted as weak inductive physical reasoning. A second experimental manipulation—exemplars with both video and question-answer pairs versus exemplars with video only—isolates the contribution of language and exp

Load-bearing premise

The paper assumes that the accuracy gap between irregular and regular scenarios measures inductive physical reasoning—that is, that the regular and irregular versions are matched in all respects except the violated law and that the intended law is uniquely inferable from three exemplars.

What would settle it

If human participants, given the same images and three exemplars, showed a similar or larger accuracy drop on the irregular scenarios, the gap would reflect task ambiguity or difficulty rather than a model-specific deficit in inductive physical reasoning.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • In safety-critical settings where novel physics can occur, an LMM cannot be assumed to adapt from demonstrations; its predictions will default to parametric knowledge.
  • Exemplars help LMMs only when they align with the physical laws already encoded in the model's parameters; conflicting demonstrations are not incorporated.
  • The language-bias result implies that standard visual-question-answering accuracy can overstate a model's visual understanding; multimodal evaluation should separate textual and visual contributions.
  • Instruction tuning as currently practiced does not address this gap; the authors suggest simulation-based feedback signals, similar to reinforcement learning from human feedback, as a direction.
  • The same benchmark methodology—impossible or law-violating scenarios—can be applied to other branches of physics beyond mechanics.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The authors do not draw this conclusion, but the irregular-versus-regular gap likely conflates inductive reasoning with task difficulty: if the irregular versions are harder for reasons other than the violated law, part of the drop would appear even in a perfectly inductive agent.
  • A testable extension the paper leaves implicit: adding human participants to the same scenarios would calibrate the gap; if humans show a comparable drop, the metric is not measuring model-specific inductive ability.
  • One could also vary exemplar count beyond three and provide explicit textual statements of the law to separate failure of visual perception, rule induction, and rule application.
  • The observation that larger models show larger language bias suggests that scaling up models may worsen, not fix, the reliance on text, which runs counter to the usual assumption that larger models are more robust reasoners.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces InPhyRe, a synthetic video question-answering benchmark designed to measure inductive physical reasoning in large multimodal models (LMMs). Scenarios depict collision events that either follow or violate universal physical laws such as momentum conservation. Models are evaluated zero-shot and few-shot, with exemplars containing either videos plus question-answer pairs or videos only. The main findings are: (1) LMMs have limited and poorly applied parametric knowledge of physical laws; (2) LMMs show weak inductive physical reasoning when exemplars violate the laws encoded in their parameters; and (3) the observed inductive behavior is driven primarily by language, with visual inputs playing little role. The headline metric is the accuracy gap between irregular and regular scenarios, with negative gaps interpreted as poor inductive physical reasoning.

Significance. If properly validated, InPhyRe would be a valuable benchmark for a question that is both scientifically interesting and practically important: whether LMMs can adapt their physical reasoning to novel or counter-physical environments from a few demonstrations. The paper evaluates a diverse cohort of 13 LMMs, uses a reproducible synthetic generation pipeline, and reports clear, structured accuracy tables. The finding that models often fail when exemplars contradict parametric knowledge, and that performance collapses in video-only settings, is suggestive and worth community attention. However, the strength of the conclusions depends on the validity of the regular–irregular gap as a measure of inductive ability, and on the absence of human/oracle calibration. Those issues are currently unresolved.

major comments (4)
  1. [§4.4, Fig. 4] The central metric compares every irregular scenario against the best regular scenario accuracy, but for AMC, Red-LMC, Red-Pass, and CC, the 'corresponding regular' is LMC(Reg), despite these scenarios differing in question type (rotation, color-conditional motion, passing-through, shape/object permanence) and visual properties. A negative gap therefore conflates induction failure with task-specific difficulty. To support the claim that 'a negative value indicates poor inductive physical reasoning,' the paper needs per-scenario matched regular controls or some other calibration that controls for task difficulty.
  2. [§4.4, Table 3] There is no human baseline or oracle calibration for irregular scenarios. The paper asserts in §1 that humans would 'easily adapt' from demonstrations, but this is not tested. Meanwhile, several irregular accuracies are near ceiling (e.g., InternVL3-8B at 94–100% on LMC, Wall, AMC), so the aggregate negative average is not a uniform failure signature. Without a competent-reasoner reference, the absolute irregular accuracies cannot be interpreted as showing weak inductive physical reasoning.
  3. [§4.5, Fig. 5] In the video-only condition, exemplars include videos plus randomly chosen option letters. In the video-text condition, exemplars include correct question-answer pairs. The comparison therefore varies not only the presence of textual information but also whether the labels are informative. A model may perform worse in the video-only condition because the random labels provide no usable signal, not necessarily because it is visually incapable. A control that keeps labels informative but removes the question text, or another design that separates label informativeness from modality, is needed to support Finding 3.
  4. [§4.4, Fig. 4] The reported accuracy differences lack confidence intervals or repeated-seed variability. Many entries are small (e.g., -0.20, +0.35, +0.30), and without uncertainty quantification it is hard to distinguish genuine effects from noise. Since each scenario contains around 2000 samples and 13 models are evaluated, paired bootstrap or stratified sampling would be straightforward and should be reported.
minor comments (5)
  1. [§B.2] The 'zero-shot' setting includes three assistant messages with random option labels. This is a reasonable formatting control, but calling it 'zero-shot' is potentially confusing. Clarify in the main text that random options are used only to enforce a constrained answer format and do not provide task information.
  2. [§4.4 conclusion] The sentence 'Almost all LMMs showed significant deterioration in performance' uses 'significant' in a non-statistical sense. Recommend replacing with 'substantial' or adding a statistical test.
  3. [§3] The claim that InPhyRe is 'the first visual question answering benchmark to measure inductive physical reasoning in LMMs' should be softened to 'to our knowledge,' as the paper does not exhaustively survey all recent benchmarks.
  4. [Fig. 4 and Fig. 5] The heatmap labels and the additional 'Average over LMMs' and 'Average over scenarios' rows are visually dense and difficult to read. Consider a cleaner formatting or a separate table for the averages.
  5. [§E.3] The heading 'AMC (regular)' in E.3 appears to be a misnomer, since AMC is an irregular scenario; the subsection describes open-ended outputs for AMC. Rename for clarity.

Circularity Check

1 steps flagged

Headline Finding 2 is the regular–irregular accuracy gap relabeled as 'weak inductive physical reasoning'; the conclusion is stipulated by the metric rather than independently established.

specific steps
  1. self definitional [§4.1 Evaluation tools; §4.4 How strong is inductive physical reasoning in LMMs?; §4.4 Conclusion]
    "Inductive physical reasoning in LMMs is evaluated by comparing their few-shot performances in regular and irregular scenarios. ... A negative value indicates poor inductive physical reasoning in that scenario. ... Conclusion: LMMs demonstrate only weak inductive physical reasoning when exemplars violate parametric knowledge."

    The central construct is not independently measured: 'weak inductive physical reasoning' is operationalized as a negative gap between irregular and regular 3-shot accuracy, and the headline discovery restates that same gap. Thus 'LMMs struggle in inductive physical reasoning' follows by definition from the metric, not from an external standard. No human or oracle baseline calibrates the gap, and for AMC, Red-LMC, Red-Pass, and CC the comparison uses LMC(Reg) even though the questions concern rotation, color-conditional motion, pass-through, and shape change rather than the same velocity-change task. A negative gap can therefore reflect task difficulty or question mismatch rather than an induction deficit, but the paper's conclusion adopts the metric's interpretation directly.

full rationale

The paper is mostly transparent: all absolute accuracies are reported in Tables 1, 3, and 4, and the irregular/regular comparison is explicitly stated. However, the headline Finding 2 is an interpretive label attached to the gap metric rather than an independent discovery. The metric defines 'weak inductive physical reasoning' as a negative gap, so the conclusion is baked into the operationalization. This is a partial circularity: the benchmark would produce the same 'finding' for any model that performed worse on irregular than regular scenarios, regardless of why. The language-bias finding in §4.5 is a separate controlled comparison and is less circular, and the qualitative AMC analysis in §E.3 is an independent check for one scenario. The paper would be strengthened by an oracle/human baseline and by matching question types across regular/irregular comparisons. Given that the raw accuracy tables remain informative, the circularity is partial, not total.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

This is an empirical benchmark paper, not a derivation, so the ledger records the evaluation choices and unvalidated assumptions that the headline conclusions rest on. Key choices are 3 exemplars, 8 sampled frames, CLIP-L retrieval, and random options in the video-only condition. The axioms concern training-data coverage, matched difficulty between regular and irregular scenarios, uniqueness of the inferred law, and standard physics for regular scenes.

free parameters (4)
  • Exemplar count = 3
    All headline results use 3-shot exemplars; Sec. 5 varies 1 to 3, but the main figures and conclusions depend on the 3-shot choice.
  • Frames sampled per video = 8 of 240
    Sec. A.1 samples 8 frames per video; this determines whether the motion and the physical anomaly are visually discernible, directly affecting the video-only condition.
  • NNER retrieval feature space = CLIP-L cosine distance, top-k unspecified
    Sec. 5 uses CLIP-L features for nearest-neighbor exemplar retrieval; the negative result in Fig. 7 is tied to this retriever choice.
  • Random options in video-only exemplars = Random letters A-D
    Sec. B.2 inserts random answer letters into video-only exemplars to force option-only outputs; this may add noise and affects the language-bias comparison.
axioms (4)
  • domain assumption Physically impossible scenarios are less likely to have been seen during training, so performance on them isolates inductive ability rather than memorized priors.
    Stated in Sec. 3: the benchmark uses 'physically impossible scenarios that are less likely to have been observed by LMMs during training.' The models' training data are closed, so this premise is unverifiable.
  • domain assumption The accuracy gap between irregular and regular scenarios is a valid measure of inductive physical reasoning.
    Sec. 4.4 assumes 'a negative value indicates poor inductive physical reasoning.' This requires that regular and irregular versions are matched in difficulty except for the violated law, which is not calibrated.
  • domain assumption Three exemplar videos (or their text labels) uniquely determine the intended violated law.
    The ground-truth answer in irregular scenarios is defined by the intended law, but any finite set of demonstrations is consistent with many alternative laws; no simplicity prior or human baseline is supplied.
  • standard math Standard conservation laws correctly describe the regular scenarios.
    Regular scenario ground truth relies on standard elastic collision momentum and energy conservation, which is appropriate background physics.

pith-pipeline@v1.3.0-alltime-deepseek · 33875 in / 14395 out tokens · 158597 ms · 2026-08-04T17:41:52.966797+00:00 · methodology

0 comments
read the original abstract

Large multimodal models (LMMs) encode physical laws observed during training, such as momentum conservation, as parametric knowledge. It allows LMMs to answer physical reasoning queries, such as the outcome of a potential collision event from visual input. However, since parametric knowledge includes only the physical laws seen during training, it is insufficient for reasoning in inference scenarios that follow physical laws unseen during training. In such novel physical environments, humans could adapt their physical reasoning based on provided demonstrations. This inductive physical reasoning ability is indispensable for LMMs if they are to replace human agents in safety-critical applications. Despite its importance, existing visual benchmarks do not evaluate inductive physical reasoning and only consider the parametric knowledge in LMMs. To this end, we propose InPhyRe, the first visual question answering benchmark to measure inductive physical reasoning in LMMs. InPhyRe evaluates LMMs' ability to predict the outcome of collision events in algorithmically generated synthetic videos. By inspecting over 13 open-source and proprietary LMMs, InPhyRe informs us that (1) LMMs struggle to apply their limited parametric knowledge about universal physical laws to reasoning, (2) inductive physical reasoning in LMMs is weak when the physical laws underlying inference scenarios were unseen during training, and (3) inductive physical reasoning in LMMs suffers from language bias and may ignore the visual inputs, questioning the trustworthiness of LMMs regarding visual inputs.

Figures

Figures reproduced from arXiv: 2509.12263 by Gautam Sreekumar, Vishnu Naresh Boddeti.

Figure 1
Figure 1. Figure 1: (Left) A large multimodal model (LMM) is asked to predict the change in vertical velocity of an object colliding with a vertical wall. The model will output “possibility 2” if it uses its parametric knowledge that encodes the universal physical laws (in this case, the momentum conservation principle). However, parametric knowledge would be insufficient if the collision event violated the physical laws enco… view at source ↗
Figure 2
Figure 2. Figure 2: INPHYRE comprises videos (“visual inputs”) of collision events that violate a real-world physical law (“violation”). LMMs must predict state changes in objects due to the collisions, while accounting for the violated physical law (“task”). The videos are grouped into “scenarios”, which are further grouped into three categories based on the nature of physical law they violate. Arrows indicate object motion … view at source ↗
Figure 3
Figure 3. Figure 3: To generate a video, we pass the initial object state to PyBullet [ [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Difference in 3-shot accuracy of LMMs be [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Difference in 3-shot accuracy in irregular sce [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Effect of number of exemplars on the accuracy. [PITH_FULL_IMAGE:figures/full_fig_p009_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Change in accuracy when NNER is used to find exemplars instead of random sampling. Effect of exemplar retrieval method: The choice of re￾trieved samples has been shown to affect the performance of LLMs [43, 31, 18, 42, 54, 63]. In this section, we verify if this proposition holds for visual inductive physical rea￾soning. By adapting the textual exemplar retriever from [43] for vision, we design a “nearest-… view at source ↗
Figure 8
Figure 8. Figure 8: Regular scenario where linear momentum conservation is followed – [PITH_FULL_IMAGE:figures/full_fig_p015_8.png] view at source ↗
Figure 11
Figure 11. Figure 11: Irregular scenario where linear momentum conservation is violated – [PITH_FULL_IMAGE:figures/full_fig_p015_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Irregular scenario where angular momentum conservation is violated – [PITH_FULL_IMAGE:figures/full_fig_p016_12.png] view at source ↗
Figure 14
Figure 14. Figure 14: Irregular scenario where only red-colored objects violate linear momentum conservation – [PITH_FULL_IMAGE:figures/full_fig_p016_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Irregular scenario where red-colored objects can pass through other objects – [PITH_FULL_IMAGE:figures/full_fig_p016_15.png] view at source ↗
Figure 17
Figure 17. Figure 17: Irregular scenario where the colliding object assumes the hue and the shape of the other [PITH_FULL_IMAGE:figures/full_fig_p016_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: Effect of weight quantization on inductive physical reasoning billidlih iifhf [PITH_FULL_IMAGE:figures/full_fig_p021_18.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. PhysMRV: Physical Memory Retrieval and Verification for Physics Plausibility Reasoning

    cs.LG 2026-07 conditional novelty 6.0

    Structured physical memories (captions, event graphs, rule cues) retrieved and verified at inference time consistently improve frozen VLMs on physical-plausibility benchmarks without fine-tuning.

Reference graph

Works this paper leans on

113 extracted references · 22 linked inside Pith · cited by 1 Pith paper

  1. [1]

    PROST: Phys- ical Reasoning about Objects through Space and Time

    Stéphane Aroca-Ouellette, Cory Paik, Alessandro Roncone, and Katharina Kann. PROST: Phys- ical Reasoning about Objects through Space and Time. InAnnual Meeting of the Association for Computational Linguistics, 2021

    2021

  2. [2]

    CRAFT: A Benchmark for Causal Reasoning About Forces and inTeractions

    Tayfun Ates, M Ate¸ so˘glu, Ça˘gatay Yi˘git, Ilker Kesen, Mert Kobas, Erkut Erdem, Aykut Erdem, Tilbe Goksun, and Deniz Yuret. CRAFT: A Benchmark for Causal Reasoning About Forces and inTeractions. InAnnual Meeting of the Association for Computational Linguistics, 2022

    2022

  3. [3]

    Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond.arXiv preprint arXiv:2308.12966, 2023

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond.arXiv preprint arXiv:2308.12966, 2023

    Pith/arXiv arXiv 2023

  4. [4]

    Qwen2.5-VL Technical Report.arXiv preprint arXiv:2502.13923, 2025

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2.5-VL Technical Report.arXiv preprint arXiv:2502.13923, 2025

    Pith/arXiv arXiv 2025

  5. [5]

    Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selection

    Yu Bai, Fan Chen, Huan Wang, Caiming Xiong, and Song Mei. Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selection. InAdvances in Neural Information Processing Systems, 2023

    2023

  6. [6]

    PHYRE: A New Benchmark for Physical Reasoning

    Anton Bakhtin, Laurens van der Maaten, Justin Johnson, Laura Gustafson, and Ross Girshick. PHYRE: A New Benchmark for Physical Reasoning. InAdvances in Neural Information Processing Systems, 2019

    2019

  7. [7]

    What Makes Multimodal In-Context Learning Work? InIEEE/CVF Conference on Computer Vision and Pattern Recognition Workshop, 2024

    Folco Bertini Baldassini, Mustafa Shukor, Matthieu Cord, Laure Soulier, and Benjamin Pi- wowarski. What Makes Multimodal In-Context Learning Work? InIEEE/CVF Conference on Computer Vision and Pattern Recognition Workshop, 2024

    2024

  8. [8]

    CoPhy: Counterfactual Learning of Physical Dynamics

    Fabien Baradel, Natalia Neverova, Julien Mille, Greg Mori, and Christian Wolf. CoPhy: Counterfactual Learning of Physical Dynamics. InInternational Conference on Learning Representations, 2020

    2020

  9. [9]

    Physion: Evaluating physical prediction from vision in humans and machines

    Daniel Bear, Elias Wang, Damian Mrowca, Felix Jedidja Binder, Hsiao-Yu Tung, RT Pramod, Cameron Holdaway, Sirui Tao, Kevin A Smith, Fan-Yun Sun, et al. Physion: Evaluating physical prediction from vision in humans and machines. InAdvances in Neural Information Processing Systems, 2021

    2021

  10. [10]

    PIQA: Reasoning about Physical Commonsense in Natural Language

    Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, et al. PIQA: Reasoning about Physical Commonsense in Natural Language. InAAAI Conference on Artificial Intelligence, 2020

    2020

  11. [11]

    A Comprehensive Evaluation of Inductive Reasoning Capabilities and Problem Solving in Large Language Models

    Chen Bowen, Rune Sætre, and Yusuke Miyao. A Comprehensive Evaluation of Inductive Reasoning Capabilities and Problem Solving in Large Language Models. InConference of the European Chapter of the Association for Computational Linguistics, 2024

    2024

  12. [12]

    PhysGame: Uncovering Physical Commonsense Violations in Gameplay Videos.arXiv preprint arXiv:2412.01800, 2024

    Meng Cao, Haoran Tang, Haoze Zhao, Hangyu Guo, Jiaheng Liu, Ge Zhang, Ruyang Liu, Qiang Sun, Ian Reid, and Xiaodan Liang. PhysGame: Uncovering Physical Commonsense Violations in Gameplay Videos.arXiv preprint arXiv:2412.01800, 2024

    Pith/arXiv arXiv 2024

  13. [13]

    ShareGPT4Video: Improving Video Understanding and Generation with Better Captions

    Lin Chen, Xilin Wei, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Zhenyu Tang, Li Yuan, et al. ShareGPT4Video: Improving Video Understanding and Generation with Better Captions. InAdvances in Neural Information Processing Systems, 2024. 10

    2024

  14. [14]

    Can Multimodal Large Language Models Truly Perform Multimodal In-Context Learning? InWinter Conference on Applications of Computer Vision, 2025

    Shuo Chen, Zhen Han, Bailan He, Jianzhe Liu, Mark Buckley, Yao Qin, Philip Torr, V olker Tresp, and Jindong Gu. Can Multimodal Large Language Models Truly Perform Multimodal In-Context Learning? InWinter Conference on Applications of Computer Vision, 2025

    2025

  15. [15]

    InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024

    2024

  16. [16]

    Compositional Physical Reasoning of Objects and Events from Videos.arXiv preprint arXiv:2408.02687, 2024

    Zhenfang Chen, Shilong Dong, Kexin Yi, Yunzhu Li, Mingyu Ding, Antonio Torralba, Joshua B Tenenbaum, and Chuang Gan. Compositional Physical Reasoning of Objects and Events from Videos.arXiv preprint arXiv:2408.02687, 2024

    Pith/arXiv arXiv 2024

  17. [17]

    ComPhy: Compositional Physical Reasoning of Objects and Events from Videos

    Zhenfang Chen, Kexin Yi, Yunzhu Li, Mingyu Ding, Antonio Torralba, Joshua B Tenenbaum, and Chuang Gan. ComPhy: Compositional Physical Reasoning of Objects and Events from Videos. InInternational Conference on Learning Representations, 2022

    2022

  18. [18]

    UPRISE: Universal Prompt Retrieval for Improving Zero-Shot Evaluation

    Daixuan Cheng, Shaohan Huang, Junyu Bi, Yuefeng Zhan, Jianfeng Liu, Yujing Wang, Hao Sun, Furu Wei, Weiwei Deng, and Qi Zhang. UPRISE: Universal Prompt Retrieval for Improving Zero-Shot Evaluation. InConference on Empirical Methods in Natural Language Processing, 2023

    2023

  19. [19]

    Inductive or deductive? Rethinking the fundamental reasoning abilities of LLMs

    Kewei Cheng, Jingfeng Yang, Haoming Jiang, Zhengyang Wang, Binxuan Huang, Ruirui Li, Shiyang Li, Zheng Li, Yifan Gao, Xian Li, et al. Inductive or deductive? Rethinking the fundamental reasoning abilities of LLMs. InACL Workshop on Natural Language Reasoning and Structure Explanations, 2024

    2024

  20. [20]

    LLMPhy: Complex Physical Reasoning Using Large Language Models and World Models.arXiv preprint arXiv:2411.08027, 2024

    Anoop Cherian, Radu Corcodel, Siddarth Jain, and Diego Romeres. LLMPhy: Complex Physical Reasoning Using Large Language Models and World Models.arXiv preprint arXiv:2411.08027, 2024

    Pith/arXiv arXiv 2024

  21. [21]

    Sora as an AGI World Model? A Complete Survey on Text-to-Video Generation.arXiv preprint arXiv:2403.05131, 2024

    Joseph Cho, Fachrina Dewi Puspitasari, Sheng Zheng, Jingyao Zheng, Lik-Hang Lee, Tae-Ho Kim, Choong Seon Hong, and Chaoning Zhang. Sora as an AGI World Model? A Complete Survey on Text-to-Video Generation.arXiv preprint arXiv:2403.05131, 2024

    arXiv 2024

  22. [22]

    On the Measure of Intelligence.arXiv preprint arXiv:1911.01547, 2019

    François Chollet. On the Measure of Intelligence.arXiv preprint arXiv:1911.01547, 2019

    Pith/arXiv arXiv 1911

  23. [23]

    Blender 4.4

    The Blender Community. Blender 4.4. https://docs.blender.org/manual/en/4.4/index. html, 2025

    2025

  24. [24]

    PyBullet, a Python module for physics simulation for games, robotics and machine learning.http://pybullet.org, 2016–2021

    Erwin Coumans and Yunfei Bai. PyBullet, a Python module for physics simulation for games, robotics and machine learning.http://pybullet.org, 2016–2021

    2016

  25. [25]

    Insight-V: Exploring Long-Chain Visual Reasoning with Multimodal Large Language Models.arXiv preprint arXiv:2411.14432, 2024

    Yuhao Dong, Zuyan Liu, Hai-Long Sun, Jingkang Yang, Winston Hu, Yongming Rao, and Ziwei Liu. Insight-V: Exploring Long-Chain Visual Reasoning with Multimodal Large Language Models.arXiv preprint arXiv:2411.14432, 2024

    Pith/arXiv arXiv 2024

  26. [26]

    What Can Transformers Learn In-Context? A Case Study of Simple Function Classes

    Shivam Garg, Dimitris Tsipras, Percy S Liang, and Gregory Valiant. What Can Transformers Learn In-Context? A Case Study of Simple Function Classes. InAdvances in Neural Information Processing Systems, 2022

    2022

  27. [27]

    Large Language Models Are Not Strong Abstract Reasoners

    Gaël Gendron, Qiming Bao, Michael Witbrock, and Gillian Dobbie. Large Language Models Are Not Strong Abstract Reasoners. InInternational Joint Conference on Artificial Intelligence, 2024

    2024

  28. [28]

    Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering

    Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh. Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2017

    2017

  29. [29]

    The Development of Inductive Reasoning.Inductive Reasoning Experimental, Developmental, and Computational Approaches, pages 25–54, 2007

    Brett K Hayes. The Development of Inductive Reasoning.Inductive Reasoning Experimental, Developmental, and Computational Approaches, pages 25–54, 2007

    2007

  30. [30]

    IDEA: Enhancing the Rule Learning Ability of Large Language Model Agent through Induction, Deduction, and Abduction.arXiv preprint arXiv:2408.10455, 2024

    Kaiyu He, Mian Zhang, Shuo Yan, Peilin Wu, and Zhiyu Zoey Chen. IDEA: Enhancing the Rule Learning Ability of Large Language Model Agent through Induction, Deduction, and Abduction.arXiv preprint arXiv:2408.10455, 2024. 11

    arXiv 2024

  31. [31]

    In-Context Demonstration Selection with Cross Entropy Difference

    Dan Iter, Reid Pryzant, Ruochen Xu, Shuohang Wang, Yang Liu, Yichong Xu, and Chenguang Zhu. In-Context Demonstration Selection with Cross Entropy Difference. InConference on Empirical Methods in Natural Language Processing, 2023

    2023

  32. [32]

    Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference

    Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, and Dmitry Kalenichenko. Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018

    2018

  33. [33]

    CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning

    Justin Johnson, Bharath Hariharan, Laurens Van Der Maaten, Li Fei-Fei, C Lawrence Zitnick, and Ross Girshick. CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, 2017

    2017

  34. [34]

    Gemma 3 Technical Report

    Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, et al. Gemma 3 Technical Report. arXiv preprint arXiv:2503.19786, 2025

    Pith/arXiv arXiv 2025

  35. [35]

    VideoPoet: A Large Language Model for Zero-Shot Video Generation

    Dan Kondratyuk, Lijun Yu, Xiuye Gu, Jose Lezama, Jonathan Huang, Grant Schindler, Rachel Hornung, Vighnesh Birodkar, Jimmy Yan, Ming-Chang Chiu, et al. VideoPoet: A Large Language Model for Zero-Shot Video Generation. InInternational Conference on Machine Learning, 2024

    2024

  36. [36]

    Learning Physical Intuition of Block Towers by Example

    Adam Lerer, Sam Gross, and Rob Fergus. Learning Physical Intuition of Block Towers by Example. InInternational Conference on Machine Learning, 2016

    2016

  37. [37]

    LLaV A-OneVision: Easy Visual Task Transfer.arXiv preprint arXiv:2408.03326, 2024

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. LLaV A-OneVision: Easy Visual Task Transfer.arXiv preprint arXiv:2408.03326, 2024

    Pith/arXiv arXiv 2024

  38. [38]

    ARIA: An Open Multimodal Native Mixture-of- Experts Model.arXiv preprint arXiv:2410.05993, 2024

    Dongxu Li, Yudong Liu, Haoning Wu, Yue Wang, Zhiqi Shen, Bowen Qu, Xinyao Niu, Fan Zhou, Chengen Huang, Yanpeng Li, et al. ARIA: An Open Multimodal Native Mixture-of- Experts Model.arXiv preprint arXiv:2410.05993, 2024

    Pith/arXiv arXiv 2024

  39. [39]

    LLaV A-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Models.arXiv preprint arXiv:2407.07895, 2024

    Feng Li, Renrui Zhang, Hao Zhang, Yuanhan Zhang, Bo Li, Wei Li, Zejun Ma, and Chunyuan Li. LLaV A-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Models.arXiv preprint arXiv:2407.07895, 2024

    Pith/arXiv arXiv 2024

  40. [40]

    MIRAGE: Evaluating and Explaining Inductive Reasoning Process in Language Models

    Jiachun Li, Pengfei Cao, Zhuoran Jin, Yubo Chen, Kang Liu, and Jun Zhao. MIRAGE: Evaluating and Explaining Inductive Reasoning Process in Language Models. InInternational Conference on Learning Representations, 2025

    2025

  41. [41]

    What Do Language Models Learn in Context? The Structured Task Hypothesis

    Jiaoda Li, Yifan Hou, Mrinmaya Sachan, and Ryan Cotterell. What Do Language Models Learn in Context? The Structured Task Hypothesis. InAnnual Meeting of the Association for Computational Linguistics, 2024

    2024

  42. [42]

    Unified Demonstration Retriever for In-Context Learning

    Xiaonan Li, Kai Lv, Hang Yan, Tianyang Lin, Wei Zhu, Yuan Ni, Guotong Xie, Xiaoling Wang, and Xipeng Qiu. Unified Demonstration Retriever for In-Context Learning. InAnnual Meeting of the Association for Computational Linguistics, 2023

    2023

  43. [43]

    What Makes Good In-Context Examples for GPT-3? InDeep Learning Inside Out (DeeLIO): The 3rd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures, 2022

    Jiachang Liu, Dinghan Shen, Yizhe Zhang, William B Dolan, Lawrence Carin, and Weizhu Chen. What Makes Good In-Context Examples for GPT-3? InDeep Learning Inside Out (DeeLIO): The 3rd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures, 2022

    2022

  44. [44]

    The Violation-of-Expectation Paradigm: A Conceptual Overview.Psychological Review, 131(3):716, 2024

    Francesco Margoni, Luca Surian, and Renée Baillargeon. The Violation-of-Expectation Paradigm: A Conceptual Overview.Psychological Review, 131(3):716, 2024

    2024

  45. [45]

    Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? InConference on Empirical Methods in Natural Language Processing, 2022

    Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? InConference on Empirical Methods in Natural Language Processing, 2022. 12

    2022

  46. [46]

    Large Language Models as General Pattern Machines

    Suvir Mirchandani, Fei Xia, Pete Florence, Brian Ichter, Danny Driess, Montserrat Gonzalez Arenas, Kanishka Rao, Dorsa Sadigh, and Andy Zeng. Large Language Models as General Pattern Machines. InConference on Robot Learning, 2023

    2023

  47. [47]

    Do generative video models understand physical principles?arXiv preprint arXiv:2501.09038, 2025

    Saman Motamed, Laura Culp, Kevin Swersky, Priyank Jaini, and Robert Geirhos. Do generative video models understand physical principles?arXiv preprint arXiv:2501.09038, 2025

    Pith/arXiv arXiv 2025

  48. [48]

    Newtonian Image Understanding: Unfolding the Dynamics of Objects in Static Images

    Roozbeh Mottaghi, Hessam Bagherinezhad, Mohammad Rastegari, and Ali Farhadi. Newtonian Image Understanding: Unfolding the Dynamics of Objects in Static Images. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, 2016

    2016

  49. [49]

    FEABench: Evaluating Language Models on Multiphysics Reasoning Ability

    Nayantara Mudur, Hao Cui, Subhashini Venugopalan, Paul Raccuglia, Michael P Brenner, and Peter Norgaard. FEABench: Evaluating Language Models on Multiphysics Reasoning Ability. arXiv preprint arXiv:2504.06260, 2025

    Pith/arXiv arXiv 2025

  50. [50]

    Learning vs Retrieval: The Role of In-Context Examples in Regression with LLMs

    Aliakbar Nafar, Kristen Brent Venable, and Parisa Kordjamshidi. Learning vs Retrieval: The Role of In-Context Examples in Regression with LLMs. InAnnual Conference of the North American Chapter of the Association for Computational Linguistics, 2025

    2025

  51. [51]

    Sora – Creating video from text.https://openai.com/index/sora/, 2024

    OpenAI. Sora – Creating video from text.https://openai.com/index/sora/, 2024

    2024

  52. [52]

    Training language models to follow instructions with human feedback

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. InAdvances in Neural Information Processing Systems, 2022

    2022

  53. [53]

    Physics Reasoner: Knowledge-Augmented Reasoning for Solving Physics Problems with Large Language Models

    Xinyu Pang, Ruixin Hong, Zhanke Zhou, Fangrui Lv, Xinwei Yang, Zhilong Liang, Bo Han, and Changshui Zhang. Physics Reasoner: Knowledge-Augmented Reasoning for Solving Physics Problems with Large Language Models. InInternational Conference on Computational Linguistics, 2025

    2025

  54. [54]

    Revisiting Demonstration Selection Strategies in In-Context Learning

    Keqin Peng, Liang Ding, Yancheng Yuan, Xuebo Liu, Min Zhang, Yuanxin Ouyang, and Dacheng Tao. Revisiting Demonstration Selection Strategies in In-Context Learning. InAnnual Meeting of the Association for Computational Linguistics, 2024

    2024

  55. [55]

    Learning Transferable Visual Models From Natural Language Supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning Transferable Visual Models From Natural Language Supervision. InInternational Conference on Machine Learning, 2021

    2021

  56. [56]

    The Development of Reasoning.Handbook of Child Psychology and Develop- mental Science, pages 1–52, 2015

    Robert B Ricco. The Development of Reasoning.Handbook of Child Psychology and Develop- mental Science, pages 1–52, 2015

    2015

  57. [57]

    IntPhys 2019: A Benchmark for Visual Intuitive Physics Understanding.IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(9):5016– 5025, 2021

    Ronan Riochet, Mario Ynocente Castro, Mathieu Bernard, Adam Lerer, Rob Fergus, Véronique Izard, and Emmanuel Dupoux. IntPhys 2019: A Benchmark for Visual Intuitive Physics Understanding.IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(9):5016– 5025, 2021

    2019

  58. [58]

    Glitchbench: Can large multimodal models detect video game glitches? InIEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024

    Mohammad Reza Taesiri, Tianjun Feng, Cor-Paul Bezemer, and Anh Nguyen. Glitchbench: Can large multimodal models detect video game glitches? InIEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024

    2024

  59. [59]

    CLIP meets GamePhysics: Towards bug identification in gameplay videos using zero-shot transfer learning

    Mohammad Reza Taesiri, Finlay Macklon, and Cor-Paul Bezemer. CLIP meets GamePhysics: Towards bug identification in gameplay videos using zero-shot transfer learning. InInternational Conference on Mining Software Repositories, 2022

    2022

  60. [60]

    Large Language Models are Pretty Good Zero-Shot Video Game Bug Detectors.arXiv preprint arXiv:2210.02506, 2022

    Mohammad Reza Taesiri, Finlay Macklon, Yihe Wang, Hengshuo Shen, and Cor-Paul Bezemer. Large Language Models are Pretty Good Zero-Shot Video Game Bug Detectors.arXiv preprint arXiv:2210.02506, 2022

    Pith/arXiv arXiv 2022

  61. [61]

    Physion++: Evaluating Physical Scene Understanding that Requires Online Inference of Different Physical Properties

    Hsiao-Yu Tung, Mingyu Ding, Zhenfang Chen, Daniel Bear, Chuang Gan, Josh Tenenbaum, Dan Yamins, Judith Fan, and Kevin Smith. Physion++: Evaluating Physical Scene Understanding that Requires Online Inference of Different Physical Properties. InAdvances in Neural Information Processing Systems, 2023. 13

    2023

  62. [62]

    From Words to Numbers: Your Large Language Model Is Secretly A Capable Regressor When Given In-Context Examples

    Robert Vacareanu, Vlad-Andrei Negru, Vasile Suciu, and Mihai Surdeanu. From Words to Numbers: Your Large Language Model Is Secretly A Capable Regressor When Given In-Context Examples. InConference on Language Modeling, 2024

    2024

  63. [63]

    Learning to retrieve in-context examples for large language models

    Liang Wang, Nan Yang, and Furu Wei. Learning to retrieve in-context examples for large language models. InConference of the European Chapter of the Association for Computational Linguistics, 2024

    2024

  64. [64]

    Qwen2-VL: Enhancing Vision- Language Model’s Perception of the World at Any Resolution.arXiv preprint arXiv:2409.12191, 2024

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. Qwen2-VL: Enhancing Vision- Language Model’s Perception of the World at Any Resolution.arXiv preprint arXiv:2409.12191, 2024

    Pith/arXiv arXiv 2024

  65. [65]

    Can In-context Learning Really Generalize to Out-of-distribution Tasks?arXiv preprint arXiv:2410.09695, 2024

    Qixun Wang, Yifei Wang, Yisen Wang, and Xianghua Ying. Can In-context Learning Really Generalize to Out-of-distribution Tasks?arXiv preprint arXiv:2410.09695, 2024

    Pith/arXiv arXiv 2024

  66. [66]

    Hypothesis Search: Inductive Reasoning with Language Models

    Ruocheng Wang, Eric Zelikman, Gabriel Poesia, Yewen Pu, Nick Haber, and Noah Goodman. Hypothesis Search: Inductive Reasoning with Language Models. InInternational Conference on Learning Representations, 2024

    2024

  67. [67]

    NEWTON: Are Large Language Models Capable of Physical Reasoning? InConference on Empirical Methods in Natural Language Processing, 2023

    Yi Ru Wang, Jiafei Duan, Dieter Fox, and Siddhartha S Srinivasa. NEWTON: Are Large Language Models Capable of Physical Reasoning? InConference on Empirical Methods in Natural Language Processing, 2023

    2023

  68. [68]

    Qwen2.5-Omni Technical Report.arXiv preprint arXiv:2503.20215, 2025

    Jin Xu, Zhifang Guo, Jinzheng He, Hangrui Hu, Ting He, Shuai Bai, Keqin Chen, Jialin Wang, Yang Fan, Kai Dang, Bin Zhang, Xiong Wang, Yunfei Chu, and Junyang Lin. Qwen2.5-Omni Technical Report.arXiv preprint arXiv:2503.20215, 2025

    Pith/arXiv arXiv 2025

  69. [69]

    MIR-Bench: Benchmarking LLM’s Long-Context Intelligence via Many-Shot In-Context Inductive Reasoning.arXiv preprint arXiv:2502.09933, 2025

    Kai Yan, Zhan Ling, Kang Liu, Yifan Yang, Ting-Han Fan, Lingfeng Shen, Zhengyin Du, and Jiecao Chen. MIR-Bench: Benchmarking LLM’s Long-Context Intelligence via Many-Shot In-Context Inductive Reasoning.arXiv preprint arXiv:2502.09933, 2025

    arXiv 2025

  70. [70]

    Qwen2 Technical Report.arXiv preprint arXiv:2407.10671, 2024

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. Qwen2 Technical Report.arXiv preprint arXiv:2407.10671, 2024

    Pith/arXiv arXiv 2024

  71. [71]

    Qwen2.5 Technical Report.arXiv preprint arXiv:2412.15115, 2024

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2.5 Technical Report.arXiv preprint arXiv:2412.15115, 2024

    Pith/arXiv arXiv 2024

  72. [72]

    CLEVRER: CoLlision Events for Video REpresentation and Reasoning

    Kexin Yi, Chuang Gan, Yunzhu Li, Pushmeet Kohli, Jiajun Wu, Antonio Torralba, and Joshua B Tenenbaum. CLEVRER: CoLlision Events for Video REpresentation and Reasoning. In International Conference on Learning Representations, 2020

    2020

  73. [73]

    Sigmoid Loss for Language Image Pre-Training

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid Loss for Language Image Pre-Training. InIEEE/CVF International Conference on Computer Vision, 2023

    2023

  74. [74]

    VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understanding.arXiv preprint arXiv:2501.13106, 2025

    Boqiang Zhang, Kehan Li, Zesen Cheng, Zhiqiang Hu, Yuqian Yuan, Guanzheng Chen, Sicong Leng, Yuming Jiang, Hang Zhang, Xin Li, Peng Jin, Wenqi Zhang, Fan Wang, Lidong Bing, and Deli Zhao. VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understanding.arXiv preprint arXiv:2501.13106, 2025

    Pith/arXiv arXiv 2025

  75. [75]

    How Vision Language Models Will Shape The Future Of Self-Driving Cars, March 2025

    Xingjian Zhang. How Vision Language Models Will Shape The Future Of Self-Driving Cars, March 2025

    2025

  76. [76]

    LLaV A-NeXT: A Strong Zero-shot Video Understanding Model, April 2024

    Yuanhan Zhang, Bo Li, haotian Liu, Yong jae Lee, Liangke Gui, Di Fu, Jiashi Feng, Ziwei Liu, and Chunyuan Li. LLaV A-NeXT: A Strong Zero-shot Video Understanding Model, April 2024

    2024

  77. [77]

    P Xing, Hao Zhang, Joseph E

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric. P Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging LLM-as-a-judge with MT-Bench and Chatbot Arena.arXiv preprint arXiv:2306.05685, 2023. 14

    Pith/arXiv arXiv 2023

  78. [78]

    Vision Language Models in Autonomous Driving: A Survey and Outlook.IEEE Transactions on Intelligent Vehicles, 2024

    Xingcheng Zhou, Mingyu Liu, Ekim Yurtsever, Bare Luka Zagar, Walter Zimmer, Hu Cao, and Alois C Knoll. Vision Language Models in Autonomous Driving: A Survey and Outlook.IEEE Transactions on Intelligent Vehicles, 2024

    2024

  79. [79]

    role": "system

    Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Yuchen Duan, Hao Tian, Weijie Su, Jie Shao, et al. InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models.arXiv preprint arXiv:2504.10479, 2025. A Data-generation Details A.1 Rendering the Videos We use Blender’s Python wrapper3 (v4.4.0) to ren...

    Pith/arXiv arXiv 2025

  80. [80]

    E.g.,<|im_start|>,<fim_suffix>

    Placeholders for image and video tokens. E.g.,<|im_start|>,<fim_suffix>

Showing first 80 references.