Pith. sign in

REVIEW 5 major objections 5 minor 85 references

Unveiling the Lack of LVLM Robustness to Fundamental Visual Variations: Why and Path Forward

T0 review · 5 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Modern vision-language models, including GPT-4o, can drop to near-chance accuracy on simple object recognition when an object's position, scale, orientation, or context changes, and the paper argues this is a structural architectural…

desk verdict A useful benchmark with a likely-true fragility finding, but the quantitative evidence needs a serious cleanup before the architectural conclusions can be taken seriously. read the letter →

arxiv 2504.16727 v3 pith:VVLZBRQV submitted 2025-04-23 cs.CV cs.AI

Zhiyuan Fan , Yumeng Wang , Sandeep Polisetty , Yi R. Fung This is my paper · ORCID
classification cs.CVcs.AI
keywords visualvariationrobustnesslargevision-languagemodelsV2R-Benchmultimodalalignmentpositionbiasacuitythresholdarchitecturaldeficiencyobjectrecognitionbenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces V2R-Bench, a benchmark of 428K images that systematically applies position, scale, orientation, and context variations to visual tasks and evaluates 21 large vision-language models. It claims that even models that excel at complex vision-language tasks fail at basic object recognition and direction detection once these everyday visual variations are introduced. The paper argues the failures are not a data shortage but an architectural deficiency: errors accumulate across the vision-encoder-to-projector-to-language-model pipeline, and the multimodal alignment stage loses visual information. A sympathetic reader would care because real-world deployment constantly exposes models to exactly these variations, so the finding shifts attention from task-level accuracy to structural robustness.

What carries the argument

The central object is V2R-Bench, an automated evaluation framework that generates variants $\mathcal{D} = \{T(I,v) \mid v \in P \times S \times R \times C\}$ across position, scale, rotation, and context, and scores robustness through a consistency metric $C_m(I)$, semantic and token-level stability measures $S_s(I)$ and $S_t(I)$, and LLM-as-a-judge ratings. The diagnostic machinery is component-level analysis: linear probing of pre- and post-projection features to measure visual information loss, and a novel visualization that decodes an aligned visual feature into language tokens via $t = \mathrm{topk}(\mathrm{softmax}(h E^\top))$, revealing what semantics the language model receives. This machinery carries the argument by turning vague robustness concerns into measurable, localizable failures.

What would settle it

Re-score the V2R-Bench object and direction outputs of GPT-4o and several open models using human annotation or a forced-choice format; if accuracy jumps from roughly 30% to above 90%, the claimed visual-perception vulnerability is mostly an answer-scoring artifact. A second check: run the same position-bias protocol with human participants; if humans show central superiority, the claimed human-like acuity threshold and edge-favoring bias need reinterpretation.

Watch

Extended reading notes

Core claim

The paper claims that modern LVLMs lack robustness to fundamental visual variations, with object recognition accuracy collapsing under position shifts (e.g., GPT-4o at 31.5% on the object task) despite strong performance on mainstream benchmarks. It reports a counter-intuitive position bias in which peripheral image regions are perceived better than the center, contradicting effective receptive field theory; a scale threshold near 1/100 of image area below which accuracy flattens at minimum; orientation-dependent failures; and a tendency to ground answers in contextual inference rather than direct visual evidence. Component analysis attributes the vulnerability primarily to inadequate multimodal alignment: linear probing accuracy drops sharply across the projector, aligned visual features sit far from language embeddings, and their decoded semantics shift under variations. The paper further shows that training on more varied data improves task performance but not variation robustness, which it reads as evidence that the limitations are fundamentally architectural rather than data-driven.

Load-bearing premise

The load-bearing premise is that the task-accuracy scores behind every headline table number are measured by a fair, valid procedure; if free-form model outputs are being scored by exact string match rather than by semantic or multiple-choice evaluation, the near-chance accuracies could indicate format mismatch rather than perception failure, and no human baseline is given to anchor the numbers.

Editorial extensions

If this is right

  • If the claims are right, deployed LVLMs should be stress-tested with V2R-Bench-style variations before use in settings where object position, scale, or orientation is uncontrolled.
  • The scale threshold implies that fine-grained visual tasks such as small-object detection in medical imaging or remote sensing will be unreliable for current models below roughly 1/100 of the object's image area.
  • The position-bias finding means current vision encoders and LVLMs process image centers worse than edges, which is the opposite of classic convolutional receptive-field expectations and argues for rethinking patch tokenization and positional encoding.
  • The paper's synthetic-training result implies that simply adding more varied training data will not fix these failures, so progress depends on architectural changes such as unified multimodal models or stronger alignment mechanisms.
  • The component analysis suggests that the language model's own spatial reasoning, tested through text-matrix inputs, is also limited, meaning downstream fixes alone cannot fully solve the problem.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The accuracy-scoring procedure is unspecified, so some of the near-chance numbers could reflect exact-string mismatch on free-form outputs rather than perception failure; a multiple-choice or human-judged re-scoring of the same outputs would separate these explanations, and the position-bias pattern may survive even if absolute accuracy rises.
  • A direct human psychophysics comparison on the same position, scale, and orientation grids would test whether the 'human-like acuity threshold' is truly human-like or merely a model-specific artifact.
  • The ethics statement implies a no-training attack vector: natural-looking object placement or rotation could manipulate model outputs in the wild, a claim that can be tested adversarially by optimizing object positions to flip predictions.
  • The soft-prompt interpretation of aligned visual features suggests a testable extension: anchoring visual features to discrete semantic tokens, rather than continuous soft prompts, should improve robustness if the paper's diagnosis is correct.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper introduces V2R-Bench, an automated framework for evaluating LVLM robustness to four visual variations (position, scale, orientation, and context), and applies it to 21 LVLMs on basic object and direction tasks plus extended benchmarks. The authors report that even strong LVLMs perform poorly under these variations, exhibit a peripheral position bias that contradicts effective-receptive-field intuition, show a scale threshold reminiscent of human acuity, and trace the failures to information loss in the multimodal projector and to the cascading pipeline architecture, concluding that the limitations are fundamentally architectural. The paper also proposes a token-decoding visualization of aligned visual features and supports the architectural claim with text-based LLM experiments and a fine-tuning experiment on a partial benchmark subset.

Significance. If the main claims hold, V2R-Bench fills a real gap in multimodal evaluation and the findings would be practically important for deployment of LVLMs in robotics, medical imaging, and other domains where object position, scale, and orientation vary naturally. The paper's strengths include broad model coverage, a public code/data release, the use of controlled synthetic tasks to isolate basic visual capabilities, and a component-level diagnostic strategy that combines linear probing, t-SNE, and text-based LLM analogs. These are valuable contributions. However, the quantitative backbone currently has unresolved issues—an unspecified accuracy scoring protocol, suspicious duplicated numbers in Table 1, and an 'upper bound' that is not a true upper bound—so the central quantitative claims are not yet established to the standard required for acceptance.

major comments (5)
  1. [Section 3.3, Eq. (2); Table 1; Appendix B] The task-specific accuracy M(I_v) used in Eq. (2) and reported in Table 1 is never defined. The paper does not state whether responses are scored by exact string match, normalized containment, LLM-as-a-judge, or another rule. This matters because the object prompt in Appendix B is free-form ('Identify the object in the image'), so exact-match scoring would mark semantically correct answers such as 'It is a cat' as incorrect. The low object accuracies in Table 1, several near or below chance on a 10-class task, are consistent with a format-mismatch artifact rather than a visual perception failure. Since the same accuracy numbers feed the position, scale, and orientation analyses, the scoring protocol is load-bearing for the paper's central claim that advanced LVLMs significantly underperform on simple object recognition. The authors must specify the scoring rule precisely and release the model outputs and scoring code for verification.
  2. [Table 1] Table 1 contains duplicated robustness values across different models. For example, Molmo-7B-D and Qwen2-VL-7B share identical robustness values for orientation (95.5/66.4), scale (86.2/92.3), and context (93.7/89.0), and the direction robustness components are also identical. This pattern is unlikely to arise from genuine model behavior and suggests a copying or processing error. Because these numbers are central to the paper's comparative robustness claims, all entries in Table 1 need to be audited, regenerated, or individually verified, and any duplication must be explained.
  3. [Section 6, Figure 6] The 'upper bound' in Figure 6 is defined as the test result of an LVLM fine-tuned on a partial benchmark dataset. This is not an upper bound on achievable performance; it is one particular model trained with one particular data recipe. It cannot support the conclusion that the observed vulnerabilities are 'fundamentally architectural deficiencies' rather than training-data limitations, because another training strategy, more data, or a different architecture could exceed this value. To support the architectural claim, the paper needs either a principled upper bound (for example, a ceiling implied by information preservation through the components) or a substantially weakened conclusion that explicitly acknowledges the bound is a performance reference, not a theoretical ceiling.
  4. [Table 2; Section 5.1, Figure 2(c)] The position-bias claim relies on Table 2, but no error bars, confidence intervals, or statistical tests are provided, and some differences are very small (e.g., LLaVA1.6 object accuracy differs by +0.01 between Middle and Surrounding). The statement that models 'consistently demonstrate better perception of surrounding regions' is not supported without uncertainty quantification. Similarly, the abstract and Section 5.1 describe a 'human-like visual acuity threshold' on the basis of Figure 2(c), but no human baseline is measured or cited. To substantiate the human-like claim, the authors should include human evaluations on the same variation grid or remove the 'human-like' language.
  5. [Section 3.3, paragraph 4] The overall robustness score is said to be a 'weighted aggregation' of consistency, semantic stability, and token stability, but the weights are never specified. Without the weights, the aggregated robustness numbers in Table 1 are not reproducible or interpretable. The authors should state the exact aggregation formula and weights, or report the three component metrics separately for every model.
minor comments (5)
  1. [Section 3.1] The text contains a duplicated word: 'automatically automatically generating diverse visual variations'; this should be corrected.
  2. [Abstract] The phrase 'scoring the need for architectural innovations' appears to be a typo; 'scoring' should likely be 'signaling' or 'underscoring'.
  3. [Section 3.2 and Appendix C.3] The object dataset is described as a '90-class object classification task' in Section 3.2, while Appendix C.3 says the dataset 'comprises ten carefully selected object categories' and that the 90-class setting is used only for linear probing. This inconsistency should be clarified.
  4. [Section 5.1, second paragraph] The text mentions that 'models like Fuyu and BLIP' show a left-orientation bias, but Fuyu and BLIP do not appear in Table 1 or in the enumerated evaluated-model list in Appendix D.1; please specify where those results are reported or add the missing model entries.
  5. [Section 4.2, Eq. (8)] The value of top-k in the token decoding step is not specified. Since k controls how many decoded tokens are examined, reporting the value or a sensitivity analysis would make the visualization more reproducible.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the benchmark measurements, component-level probes, and synthetic-data experiments are external to the claims, so the derivation chain is not self-referential.

full rationale

I find no step in V2R-Bench's derivation chain that reduces to its own inputs by the paper's equations or by load-bearing self-citation. The headline results in Table 1 are empirical task accuracies and robustness aggregates; even if the object/direction accuracy scoring rule is under-specified, that is a reproducibility and validity risk (exact-match on free-form outputs could turn semantically correct answers into failures), not a circular reduction, because the paper never defines accuracy in terms of the robustness metrics it uses to support the vulnerability claim. The component-level diagnosis rests on independent evidence: linear probing before and after the multimodal projector, t-SNE feature geometry, and token reconstruction from aligned visual features. These probes are external to the benchmark scores, so the conclusions about inadequate multimodal alignment and error accumulation are not equivalent to the evaluation metrics by construction. The Figure 6 'upper bound' is explicitly the testing result of an LVLM fine-tuned on a partial benchmark dataset; calling it an upper bound is an over-interpretation of a single fine-tuned run, but the argument that path tracing still underperforms is inductive rather than definitional, and the conclusion does not follow tautologically from the label. Self-citations such as He et al. 2025, Qin et al. 2025, Zhou et al. 2024, and Wang et al. 2025 appear only in related work or future-direction contexts and are not load-bearing. The Limitations section honestly states that architectural modifications, pre-training strategies, test-time correction, and novel alignment methods were not explored, which is consistent with the paper's stated scope and does not signal a hidden dependence on prior author work. Overall, the derivation chain is self-contained against external measurements and contains no exhibits of self-definition, fitted-input-as-prediction, or author-imported uniqueness.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

No new physical or conceptual entities are postulated; the benchmark and visualization are methods, not entities. The free parameters are methodological choices that the authors did not disclose, which affects reproducibility.

free parameters (2)
  • Robustness aggregation weights = Not reported
    Section 3.3 states overall robustness is a weighted aggregation of C_m, S_s, S_t, and LLM-as-a-judge, but the weights are not given, making the reported robustness numbers in Table 1 unreproducible.
  • top-k for feature decoding = Not reported
    Equation (8) uses k to select decoded tokens, but the value of k is not specified in the main text, though it affects the word-cloud and token-alignment visualizations in Section 5.2.
assumptions (5)
  • domain assumption The extended benchmark variations preserve ground-truth validity.
    Section 3.2 states this requirement but does not verify for every generated image, e.g., scaling could crop an object or context inpainting could alter the answer.
  • domain assumption The text-matrix representation simulates ideal visual encoding without loss.
    Section 4.1 uses matrix text to emulate perfectly extracted visual features; this assumes discrete token grids are equivalent to continuous visual features.
  • domain assumption Effective receptive field theory for CNNs applies to ViT-based LVLMs such that central perception should be better.
    Section 5.1 interprets the position bias as contradicting ERF theory (Luo et al. 2017; Raghu et al. 2022), but those analyses are for CNN and standard ViT classifiers, not for generative LVLMs with learned position embeddings.
  • domain assumption LLM-as-a-judge emulates human assessment without bias.
    Section 3.3 introduces LLM-as-a-judge as the qualitative dimension, but no validation against human ratings is reported.
  • ad hoc to paper The 'upper bound' in Figure 6 is an upper bound on achievable performance.
    Section 6 defines the upper bound as the testing result of the LVLM fine-tuned on a partial benchmark dataset; this is a specific trained model's result, not a mathematical bound, which can mislead the architectural-deficiency conclusion.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Unveiling the Lack of LVLM Robustness to Fundamental Visual Variations: Why and Path Forward." pith.science (2026). https://pith.science/paper/VVLZBRQV

@misc{pith2026250416727,
  author       = {Pith},
  title        = {Pith review of: Unveiling the Lack of LVLM Robustness to Fundamental Visual Variations: Why and Path Forward},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VVLZBRQV}},
  note         = {Machine review of arXiv:2504.16727}
}
abstract

Large Vision Language Models (LVLMs) excel in various vision-language tasks. Yet, their robustness to visual variations in position, scale, orientation, and context that objects in natural scenes inevitably exhibit due to changes in viewpoint and environment remains largely underexplored. To bridge this gap, we introduce V$^2$R-Bench, a comprehensive benchmark framework for evaluating Visual Variation Robustness of LVLMs, which encompasses automated evaluation dataset generation and principled metrics for thorough robustness assessment. Through extensive evaluation on 21 LVLMs, we reveal a surprising vulnerability to visual variations, in which even advanced models that excel at complex vision-language tasks significantly underperform on simple tasks such as object recognition. Interestingly, these models exhibit a distinct visual position bias that contradicts theories of effective receptive fields, and demonstrate a human-like visual acuity threshold. To identify the source of these vulnerabilities, we present a systematic framework for component-level analysis, featuring a novel visualization approach for aligned visual features. Results show that these vulnerabilities stem from error accumulation in the pipeline architecture and inadequate multimodal alignment. Complementary experiments with synthetic data further demonstrate that these limitations are fundamentally architectural deficiencies, scoring the need for architectural innovations in future LVLM designs.

Figures

Figures reproduced from arXiv: 2504.16727 by the authors.

Figure 1
Figure 1. Our V2R-Bench benchmark contains: 1) An automated data generation pipeline that adds visual variations to existing benchmarks; (2) Synthetic data generation for the fundamental visual tasks proposed (e.g., object, direction, coordinate, path and OCR tasks). The subfigure on the right shows metrics for robustness testing, including performance consistency, semantic and token-level stability, and LLM-as-a-judge. which… view at source ↗
Figure 2
Figure 2. (a) Demonstration of position bias effects. (b) Accuracy heatmaps for object recognition and direction recognition, across object scales and position variations. (c) Model accuracy as a function of relative object scale. Despite their impressive performance demonstrated on complex visual tasks, these models exhibit sur￾prising vulnerability to simple visual variations, resulting in significantly degraded performance… view at source ↗
Figure 4
Figure 4. Changes in vision encoder classification prob [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figures from the paper (5 more)
Figure 3
Figure 3. Figure 3: Performance evaluation on a 6-point path trac [PITH_FULL_IMAGE:figures/full_fig_p007_3.png]
Figure 5
Figure 5. Figure 5: Comparison of different feature representation visualizations. (a) Visualization of visual features, aligned [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: The comparison of benchmark evaluation results with upper bound on coordinate and path tasks, where the upper bound is the testing result of the LVLM fine-tuned on partial benchmark dataset. robustness against position and orientation varia￾tions. As [PITH_FULL_IMAGE:…
Figure 7
Figure 7. Figure 7: Linear probing results for Vision Encoder show position bias patterns similar to those observed in LVLMs. [PITH_FULL_IMAGE:figures/full_fig_p018_7.png]
Figure 8
Figure 8. Figure 8: t-SNE visualization of aligned features under [PITH_FULL_IMAGE:figures/full_fig_p019_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

85 extracted references · 6 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, Alon Benhaim, Misha Bilenko, Johan Bjorck, Sébastien Bubeck, Martin Cai, Qin Cai, Vishrav Chaudhary, Dong Chen, Dongdong Chen, Weizhu Chen, Yen-Chun Chen, Yi-Ling Chen, Hao Cheng, Parul Chopra, Xiyang Dai, Matt...

    arXiv 2024

  4. [4]

    Guillaume Alain and Yoshua Bengio. 2018. https://arxiv.org/abs/1610.01644 Understanding intermediate layers using linear classifier probes . Preprint, arXiv:1610.01644

    arXiv 2018

  5. [5]

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katie Millican, Malcolm Reynolds, Roman Ring, Eliza Rutherford, Serkan Cabi, Tengda Han, Zhitao Gong, Sina Samangooei, Marianne Monteiro, Jacob Menick, Sebastian Borgeaud, Andrew Brock, Aida Nematzadeh, Sahand Sharifzadeh, Mikolaj Binkowski,...

    arXiv 2022

  6. [6]

    Amith Ananthram, Elias Stengel-Eskin, Carl Vondrick, Mohit Bansal, and Kathleen McKeown. 2024. https://arxiv.org/abs/2406.11665 See it from my perspective: Diagnosing the western cultural bias of large vision-language models in image understanding . Preprint, arXiv:2406.11665

    arXiv 2024

  7. [7]

    Federico Arenas. 2020. https://github.com/federicoarenasl/blender-sdg Blender SDG Icons Generator . https://github.com/federicoarenasl/blender-sdg. Blender add-on for generating 3D Sustainable Development Goals (SDG) icons

    2020

  8. [8]

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, Peng Wang, Shijie Wang, Wei Wang, Shengguang Wu, Benfeng X...

    arXiv 2023

Show all 85 references
  1. [9]

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023 b . Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond. arXiv preprint arXiv:2308.12966

    2023 arXiv

  2. [10]

    Luke Bailey, Gustaf Ahdritz, Anat Kleiman, Siddharth Swaroop, Finale Doshi-Velez, and Weiwei Pan. 2024. Soft prompting might be a bug, not a feature

    2024

  3. [11]

    Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Conghui He, Jiaqi Wang, Feng Zhao, and Dahua Lin. 2023 a . https://arxiv.org/abs/2311.12793 Sharegpt4v: Improving large multi-modal models with better captions . Preprint, arXiv:2311.12793

    2023 arXiv

  4. [12]

    Pengcheng Chen, Jin Ye, Guoan Wang, Yanjun Li, Zhongying Deng, Wei Li, Tianbin Li, Haodong Duan, Ziyan Huang, Yanzhou Su, Benyou Wang, Shaoting Zhang, Bin Fu, Jianfei Cai, Bohan Zhuang, Eric J Seibel, Junjun He, and Yu Qiao. 2024 a . https://arxiv.org/abs/2408.03361 Gmai-mmben...

    2024 arXiv

  5. [13]

    Shuo Chen, Jindong Gu, Zhen Han, Yunpu Ma, Philip Torr, and Volker Tresp. 2023 b . https://openreview.net/forum?id=4d8dO5sAeM Benchmarking robustness of adaptation methods on pre-trained vision-language models . In Thirty-seventh Conference on Neural Information Processing Sys...

    2023

  6. [14]

    Xiaokang Chen, Zhiyu Wu, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, and Chong Ruan. 2025 a . https://arxiv.org/abs/2501.17811 Janus-pro: Unified multimodal understanding and generation with data and model scaling . Preprint, arXiv:2501.17811

    2025 arXiv

  7. [15]

    Zeren Chen, Zhelun Shi, Xiaoya Lu, Lehan He, Sucheng Qian, Hao Shu Fang, Zhenfei Yin, Wanli Ouyang, Jing Shao, Yu Qiao, Cewu Lu, and Lu Sheng. 2024 b . https://arxiv.org/abs/2403.19622 Rh20t-p: A primitive-level robotic dataset towards composable generalization agents . Prepri...

    2024 arXiv

  8. [16]

    Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, Lixin Gu, Xuehui Wang, Qingyun Li, Yimin Ren, Zixuan Chen, Jiapeng Luo, Jiahao Wang, Tan Jiang, Bo Wang, Conghui He, Botian Shi, Xingcheng Zhang, Han Lv, Yi...

    2025 arXiv

  9. [17]

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. 2024 c . Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In Proceedings of the IEEE/CVF Conference on Com...

    2024

  10. [18]

    Ciprian Corneanu, Raghudeep Gadde, and Aleix M Martinez. 2024. https://doi.org/10.1109/WACV57701.2024.00428 Latentpaint: Image inpainting in latent space with diffusion models . In 2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 4322--4331

    2024

  11. [19]

    Smith, Hannaneh Hajishirzi, Ross Girshick, Ali Farhadi, and Aniruddha Kembhavi

    Matt Deitke, Christopher Clark, Sangho Lee, Rohun Tripathi, Yue Yang, Jae Sung Park, Mohammadreza Salehi, Niklas Muennighoff, Kyle Lo, Luca Soldaini, Jiasen Lu, Taira Anderson, Erin Bransom, Kiana Ehsani, Huong Ngo, YenSung Chen, Ajay Patel, Mark Yatskar, Chris Callison-Burch,...

    2024 arXiv

  12. [20]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. https://doi.org/10.18653/v1/N19-1423 BERT : Pre-training of deep bidirectional transformers for language understanding . In Proceedings of the 2019 Conference of the North A merican Chapter of the Associat...

    2019 doi

  13. [21]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zha...

    2024 arXiv

  14. [22]

    Hao Fei, Yuan Yao, Zhuosheng Zhang, Fuxiao Liu, Ao Zhang, and Tat-Seng Chua. 2024. https://aclanthology.org/2024.lrec-tutorials.1/ From multimodal LLM to human-level AI : Modality, instruction, reasoning, efficiency and beyond . In Proceedings of the 2024 Joint International C...

    2024

  15. [23]

    Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, Yunsheng Wu, and Rongrong Ji. 2024 a . https://arxiv.org/abs/2306.13394 Mme: A comprehensive evaluation benchmark for multimodal large language models . Preprin...

    2024 arXiv

  16. [24]

    Smith, Wei-Chiu Ma, and Ranjay Krishna

    Xingyu Fu, Yushi Hu, Bangzheng Li, Yu Feng, Haoyu Wang, Xudong Lin, Dan Roth, Noah A. Smith, Wei-Chiu Ma, and Ranjay Krishna. 2024 b . https://arxiv.org/abs/2404.12390 Blink: Multimodal large language models can see but not perceive . Preprint, arXiv:2404.12390

    2024 arXiv

  17. [25]

    Shaikat Galib, Shanshan Wang, Guanshuo Xu, Pascal Pfeiffer, Ryan Chesler, Mark Landry, and Sri Satish Ambati. 2024. https://arxiv.org/abs/2410.13611 H2ovl-mississippi vision language models technical report . Preprint, arXiv:2410.13611

    2024 arXiv

  18. [26]

    Team GLM, Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Diego Rojas, Guanyu Feng, Hanlin Zhao, Hanyu Lai, Hao Yu, Hongning Wang, Jiadai Sun, Jiajie Zhang, Jiale Cheng, Jiayi Gui, Jie Tang, Jing Zhang, Juanzi Li, Lei Zhao, Lindong Wu, Lucen Zhong, Mingdao Liu, Minlie H...

    2024 arXiv

  19. [27]

    Yuxian Gu, Xu Han, Zhiyuan Liu, and Minlie Huang. 2022. https://doi.org/10.18653/v1/2022.acl-long.576 PPT : Pre-trained prompt tuning for few-shot learning . In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pag...

    2022 doi

  20. [28]

    Jiayi He, Hehai Lin, Qingyun Wang, Yi Fung, and Heng Ji. 2024. https://arxiv.org/abs/2410.04055 Self-correction is more than refinement: A learning framework for visual and language reasoning tasks . Preprint, arXiv:2410.04055

    2024 arXiv

  21. [29]

    Zhitao He, Sandeep Polisetty, Zhiyuan Fan, Yuchen Huang, Shujin Wu, and Yi R. Fung. 2025. https://arxiv.org/abs/2505.23224 Mmboundary: Advancing mllm knowledge boundary awareness through reasoning step confidence calibration . Preprint, arXiv:2505.23224

    2025 arXiv

  22. [30]

    Fraser, and Svetlana Kiritchenko

    Phillip Howard, Anahita Bhiwandiwalla, Kathleen C. Fraser, and Svetlana Kiritchenko. 2024. https://arxiv.org/abs/2404.00166 Uncovering bias in large vision-language models with counterfactuals . Preprint, arXiv:2404.00166

    2024 arXiv

  23. [31]

    Yutao Hu, Tianbin Li, Quanfeng Lu, Wenqi Shao, Junjun He, Yu Qiao, and Ping Luo. 2024. https://arxiv.org/abs/2402.09181 Omnimedvqa: A new large-scale comprehensive evaluation benchmark for medical lvlm . Preprint, arXiv:2402.09181

    2024 arXiv

  24. [32]

    Kung-Hsiang Huang, Hou Pong Chan, May Fung, Haoyi Qiu, Mingyang Zhou, Shafiq Joty, Shih-Fu Chang, and Heng Ji. 2025. https://doi.org/10.1109/TKDE.2024.3513320 From pixels to insights: A survey on automatic chart understanding in the era of large foundation models . IEEE Transa...

    2025

  25. [33]

    Kung-Hsiang Huang, Mingyang Zhou, Hou Pong Chan, Yi Fung, Zhenhailong Wang, Lingyu Zhang, Shih-Fu Chang, and Heng Ji. 2024. https://doi.org/10.18653/v1/2024.findings-acl.41 Do LVLM s understand charts? analyzing and correcting factual errors in chart captioning . In Findings o...

    2024 doi

  26. [34]

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas ...

    2023 arXiv

  27. [35]

    Taehee Jung, Dongyeop Kang, Lucas Mentch, and Eduard Hovy. 2019. https://doi.org/10.18653/v1/D19-1327 Earlier isn`t always better: Sub-aspect analysis on corpus and system biases in summarization . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language ...

    2019 doi

  28. [36]

    Ryan Koo, Minhwa Lee, Vipul Raheja, Jong Inn Park, Zae Myung Kim, and Dongyeop Kang. 2024. https://arxiv.org/abs/2309.17012 Benchmarking cognitive biases in large language models as evaluators . Preprint, arXiv:2309.17012

    2024 arXiv

  29. [37]

    Tony Lee, Haoqin Tu, Chi Heem Wong, Wenhao Zheng, Yiyang Zhou, Yifan Mai, Josselin Somerville Roberts, Michihiro Yasunaga, Huaxiu Yao, Cihang Xie, and Percy Liang. 2024. https://arxiv.org/abs/2410.07112 Vhelm: A holistic evaluation of vision language models . Preprint, arXiv:2...

    2024 arXiv

  30. [38]

    Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. https://arxiv.org/abs/2104.08691 The power of scale for parameter-efficient prompt tuning . Preprint, arXiv:2104.08691

    2021 arXiv

  31. [39]

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. 2024. https://arxiv.org/abs/2408.03326 Llava-onevision: Easy visual task transfer . Preprint, arXiv:2408.03326

    2024 arXiv

  32. [40]

    Bohao Li, Yuying Ge, Yixiao Ge, Guangzhi Wang, Rui Wang, Ruimao Zhang, and Ying Shan. 2023 a . https://arxiv.org/abs/2311.17092 Seed-bench-2: Benchmarking multimodal large language models . Preprint, arXiv:2311.17092

    2023 arXiv

  33. [41]

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023 b . https://arxiv.org/abs/2301.12597 Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models . Preprint, arXiv:2301.12597

    2023 arXiv

  34. [42]

    Lei Li, Yuwei Yin, Shicheng Li, Liang Chen, Peiyi Wang, Shuhuai Ren, Mukai Li, Yazheng Yang, Jingjing Xu, Xu Sun, Lingpeng Kong, and Qi Liu. 2023 c . https://arxiv.org/abs/2306.04387 M ^3 it: A large-scale dataset towards multi-modal multilingual instruction tuning . Preprint,...

    2023 arXiv

  35. [43]

    Paul Pu Liang, Akshay Goindani, Talha Chafekar, Leena Mathur, Haofei Yu, Ruslan Salakhutdinov, and Louis-Philippe Morency. 2024. https://arxiv.org/abs/2407.03418 Hemm: Holistic evaluation of multimodal foundation models . Preprint, arXiv:2407.03418

    2024 arXiv

  36. [44]

    Daizong Liu, Xiaowen Cai, Pan Zhou, Xiaoye Qu, Xiang Fang, Lichao Sun, and Wei Hu. 2024 a . https://openreview.net/forum?id=q8XGHj7yrC Are large vision-language models robust to adversarial visual transformations?

    2024

  37. [45]

    Daizong Liu, Mingyu Yang, Xiaoye Qu, Pan Zhou, Xiang Fang, Keke Tang, Yao Wan, and Lichao Sun. 2024 b . https://openreview.net/forum?id=gDpWYpocE1 Pandora's box: Towards building universal attackers against real-world large vision-language models . In The Thirty-eighth Annual ...

    2024

  38. [46]

    Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. 2024 c . https://llava-vl.github.io/blog/2024-01-30-llava-next/ Llava-next: Improved reasoning, ocr, and world knowledge

    2024

  39. [47]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. https://arxiv.org/abs/2304.08485 Visual instruction tuning . Preprint, arXiv:2304.08485

    2023 arXiv

  40. [48]

    Xiao Liu, Kaixuan Ji, Yicheng Fu, Weng Tam, Zhengxiao Du, Zhilin Yang, and Jie Tang. 2022. https://doi.org/10.18653/v1/2022.acl-short.8 P -tuning: Prompt tuning can be comparable to fine-tuning across scales and tasks . In Proceedings of the 60th Annual Meeting of the Associat...

    2022 doi

  41. [49]

    Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, Kai Chen, and Dahua Lin. 2024 d . https://arxiv.org/abs/2307.06281 Mmbench: Is your multi-modal model an all-around player? Preprint, arXiv:2307.06281

    2024 arXiv

  42. [50]

    Haoyu Lu, Wen Liu, Bo Zhang, Bingxuan Wang, Kai Dong, Bo Liu, Jingxiang Sun, Tongzheng Ren, Zhuoshu Li, Hao Yang, Yaofeng Sun, Chengqi Deng, Hanwei Xu, Zhenda Xie, and Chong Ruan. 2024 a . https://arxiv.org/abs/2403.05525 Deepseek-vl: Towards real-world vision-language underst...

    2024 arXiv

  43. [51]

    Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. 2024 b . https://arxiv.org/abs/2310.02255 Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts . Preprint...

    2024 arXiv

  44. [52]

    Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. 2022. https://arxiv.org/abs/2201.09865 Repaint: Inpainting using denoising diffusion probabilistic models . Preprint, arXiv:2201.09865

    2022 arXiv

  45. [53]

    Gen Luo, Xue Yang, Wenhan Dou, Zhaokai Wang, Jiawen Liu, Jifeng Dai, Yu Qiao, and Xizhou Zhu. 2024 a . https://arxiv.org/abs/2410.08202 Mono-internvl: Pushing the boundaries of monolithic multimodal large language models with endogenous visual pre-training . Preprint, arXiv:2410.08202

    2024 arXiv

  46. [54]

    Weidi Luo, Siyuan Ma, Xiaogeng Liu, Xiaoyu Guo, and Chaowei Xiao. 2024 b . https://arxiv.org/abs/2404.03027 Jailbreakv: A benchmark for assessing the robustness of multimodal large language models against jailbreak attacks . Preprint, arXiv:2404.03027

    2024 arXiv

  47. [55]

    Wenjie Luo, Yujia Li, Raquel Urtasun, and Richard Zemel. 2017. https://arxiv.org/abs/1701.04128 Understanding the effective receptive field in deep convolutional neural networks . Preprint, arXiv:1701.04128

    2017 arXiv

  48. [56]

    OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff ...

    2024 arXiv

  49. [57]

    Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Mahmoud Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Michael Rabba...

    2024 arXiv

  50. [58]

    Fung, Weizhu Chen, Minhao Cheng, and Furu Wei

    Zeyu Qin, Qingxiu Dong, Xingxing Zhang, Li Dong, Xiaolong Huang, Ziyi Yang, Mahmoud Khademi, Dongdong Zhang, Hany Hassan Awadalla, Yi R. Fung, Weizhu Chen, Minhao Cheng, and Furu Wei. 2025. https://arxiv.org/abs/2503.19551 Scaling laws of synthetic data for language models . P...

    2025

  51. [59]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. https://arxiv.org/abs/2103.00020 Learning transferable visual models from natural lan...

    2021 arXiv

  52. [60]

    Maithra Raghu, Thomas Unterthiner, Simon Kornblith, Chiyuan Zhang, and Alexey Dosovitskiy. 2022. https://arxiv.org/abs/2108.08810 Do vision transformers see like convolutional neural networks? Preprint, arXiv:2108.08810

    2022 arXiv

  53. [61]

    Ryan Steed and Aylin Caliskan. 2021. https://doi.org/10.1145/3442188.3445932 Image representations learned with unsupervised pre-training contain human-like biases . In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT '21, page 701–71...

    2021

  54. [62]

    Qwen Team. 2024. https://qwenlm.github.io/blog/qvq-72b-preview/ Qvq: To see the world with wisdom

    2024

  55. [63]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. https://arxiv.org/abs/2302.13971 Llama:...

    2023 arXiv

  56. [64]

    Jiaan Wang, Yunlong Liang, Fandong Meng, Zengkui Sun, Haoxiang Shi, Zhixu Li, Jinan Xu, Jianfeng Qu, and Jie Zhou. 2023 a . https://arxiv.org/abs/2303.04048 Is chatgpt a good nlg evaluator? a preliminary study . Preprint, arXiv:2303.04048

    2023 arXiv

  57. [65]

    Junke Wang, Lingchen Meng, Zejia Weng, Bo He, Zuxuan Wu, and Yu-Gang Jiang. 2023 b . https://arxiv.org/abs/2311.07574 To see is to believe: Prompting gpt-4v for better visual instruction tuning . Preprint, arXiv:2311.07574

    2023 arXiv

  58. [66]

    Junyang Wang, Haiyang Xu, Jiabo Ye, Ming Yan, Weizhou Shen, Ji Zhang, Fei Huang, and Jitao Sang. 2024 a . https://arxiv.org/abs/2401.16158 Mobile-agent: Autonomous multi-modal mobile device agent with visual perception . Preprint, arXiv:2401.16158

    2024 arXiv

  59. [67]

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. 2024 b . https://arxiv.org/abs/2409.12191 Qwen...

    2024 arXiv

  60. [68]

    Sibo Wang, Xiangkui Cao, Jie Zhang, Zheng Yuan, Shiguang Shan, Xilin Chen, and Wen Gao. 2024 c . https://arxiv.org/abs/2406.14194 Vlbiasbench: A comprehensive benchmark for evaluating bias in large vision-language model . Preprint, arXiv:2406.14194

    2024 arXiv

  61. [69]

    Tai Wang, Xiaohan Mao, Chenming Zhu, Runsen Xu, Ruiyuan Lyu, Peisen Li, Xiao Chen, Wenwei Zhang, Kai Chen, Tianfan Xue, Xihui Liu, Cewu Lu, Dahua Lin, and Jiangmiao Pang. 2023 c . https://arxiv.org/abs/2312.16170 Embodiedscan: A holistic multi-modal 3d perception suite towards...

    2023 arXiv

  62. [70]

    Yumeng Wang, Zhiyuan Fan, Qingyun Wang, May Fung, and Heng Ji. 2025. https://arxiv.org/abs/2501.18457 Calm: Unleashing the cross-lingual self-aligning ability of language model question answering . Preprint, arXiv:2501.18457

    2025 arXiv

  63. [71]

    Penghao Wu and Saining Xie. 2023. https://arxiv.org/abs/2312.14135 V*: Guided visual search as a core mechanism in multimodal llms . Preprint, arXiv:2312.14135

    2023 arXiv

  64. [72]

    Shujin Wu, Yi Fung, Sha Li, Yixin Wan, Kai-Wei Chang, and Heng Ji. 2024 a . Macaroon: Training vision-language models to be your engaged partners. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 7715--7731

    2024

  65. [73]

    Sadler, Dinesh Manocha, and Amrit Bedi

    Xiyang Wu, Ruiqi Xian, Tianrui Guan, Jing Liang, Souradip Chakraborty, Fuxiao Liu, Brian M. Sadler, Dinesh Manocha, and Amrit Bedi. 2024 b . https://openreview.net/forum?id=4FpuOMoxsX On the safety concerns of deploying LLM s/ VLM s in robotics: Highlighting the risks and vuln...

    2024

  66. [74]

    Peng Xia, Ze Chen, Juanxi Tian, Yangrui Gong, Ruibo Hou, Yue Xu, Zhenbang Wu, Zhiyuan Fan, Yiyang Zhou, Kangyu Zhu, Wenhao Zheng, Zhaoyang Wang, Xiao Wang, Xuchao Zhang, Chetan Bansal, Marc Niethammer, Junzhou Huang, Hongtu Zhu, Yun Li, Jimeng Sun, Zongyuan Ge, Gang Li, James ...

    2024 arXiv

  67. [75]

    Rui Yang, Lin Song, Yanwei Li, Sijie Zhao, Yixiao Ge, Xiu Li, and Ying Shan. 2023. https://arxiv.org/abs/2305.18752 Gpt4tools: Teaching large language model to use tools via self-instruction . Preprint, arXiv:2305.18752

    2023 arXiv

  68. [76]

    Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, Qianyu Chen, Huarong Zhou, Zhensheng Zou, Haoye Zhang, Shengding Hu, Zhi Zheng, Jie Zhou, Jie Cai, Xu Han, Guoyang Zeng, Dahai Li, Zhiyuan Liu, and Maosong Sun. 2...

    2024 arXiv

  69. [77]

    Weihao Yu, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang, and Lijuan Wang. 2024. https://arxiv.org/abs/2308.02490 Mm-vet: Evaluating large multimodal models for integrated capabilities . Preprint, arXiv:2308.02490

    2024 arXiv

  70. [78]

    Chi Zhang, Zhao Yang, Jiaxuan Liu, Yucheng Han, Xin Chen, Zebiao Huang, Bin Fu, and Gang Yu. 2023. https://arxiv.org/abs/2312.13771 Appagent: Multimodal agents as smartphone users . Preprint, arXiv:2312.13771

    2023 arXiv

  71. [79]

    Hao Zhang, Wenqi Shao, Hong Liu, Yongqiang Ma, Ping Luo, Yu Qiao, Nanning Zheng, and Kaipeng Zhang. 2024 a . https://arxiv.org/abs/2403.09346 B-avibench: Towards evaluating the robustness of large vision-language model on black-box adversarial visual-instructions . Preprint, a...

    2024 arXiv

  72. [80]

    Jianshu Zhang, Dongyu Yao, Renjie Pi, Paul Pu Liang, and Yi R. Fung. 2025. https://arxiv.org/abs/2502.12084 Vlm2-bench: A closer look at how well vlms implicitly link explicit matching visual cues . Preprint, arXiv:2502.12084

    2025 arXiv

  73. [81]

    Yi-Fan Zhang, Huanyu Zhang, Haochen Tian, Chaoyou Fu, Shuangqing Zhang, Junfei Wu, Feng Li, Kun Wang, Qingsong Wen, Zhang Zhang, Liang Wang, Rong Jin, and Tieniu Tan. 2024 b . https://arxiv.org/abs/2408.13257 Mme-realworld: Could your multimodal llm challenge high-resolution r...

    2024 arXiv

  74. [82]

    Bo Zhao, Boya Wu, Muyang He, and Tiejun Huang. 2023. https://arxiv.org/abs/2307.04087 Svit: Scaling up visual instruction tuning . Preprint, arXiv:2307.04087

    2023 arXiv

  75. [83]

    Xing, Hao Zhang, Joseph E

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. https://arxiv.org/abs/2306.05685 Judging llm-as-a-judge with mt-bench and chatbot arena . P...

    2023 arXiv

  76. [84]

    Yiyang Zhou, Zhiyuan Fan, Dongjie Cheng, Sihan Yang, Zhaorun Chen, Chenhang Cui, Xiyao Wang, Yun Li, Linjun Zhang, and Huaxiu Yao. 2024. https://arxiv.org/abs/2405.14622 Calibrated self-rewarding vision language models . Preprint, arXiv:2405.14622

    2024 arXiv

  77. [85]

    Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. 2023. https://arxiv.org/abs/2304.10592 Minigpt-4: Enhancing vision-language understanding with advanced large language models . Preprint, arXiv:2304.10592

    2023 arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.