Pith. sign in

REVIEW 4 major objections 7 minor 4 cited by

Weaving Context Across Images: Improving Vision-Language Models through Focus-Centric Visual Chains

T0 review · 4 major / 7 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Teaching vision-language models to reason stepwise across images improves multi-image benchmark accuracy by 3.16 percentage points in LLaVA-OneVision and 2.24 in Qwen2-VL.

desk verdict A genuinely useful dataset and recipe for multi-image VLM training, but the headline claim that the chain format causes the gains is not isolated by the experiments. read the letter →

arxiv 2504.20199 v1 pith:S4NMTYBN submitted 2025-04-28 cs.CV cs.AIcs.CL

Juntian Zhang , Chuanqi cheng , Yuhan Liu , Wei Liu , Jian Luan , Rui Yan This is my paper · ORCID
classification cs.CVcs.AIcs.CL
keywords multi-imageunderstandingvision-languagemodelsreasoningchaindatasynthesisfine-tuningVISC-150KFocus-CentricVisualLoRA
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that vision-language models can be made substantially better at tasks involving several images at once by teaching them to reason in a stepwise 'focus-centric' manner: at each step the model names a sub-question, points to the specific image or images it is looking at, answers, and then moves on. To make this trainable, the authors build a 150K-sample dataset, VISC-150K, whose entries consist of such visual reasoning chains, synthesized bottom-up with open-source models. Fine-tuning two strong base VLMs on this dataset improves their average accuracy on seven multi-image benchmarks by 3.16 and 2.24 percentage points respectively, and reaches a new best score on four of them, while general single-image capabilities stay roughly flat. The paper's core assertion is that the chain structure, not just additional multi-image data, is what drives the gains.

What carries the argument

The central object is the Focus-Centric Visual Chain, an ordered reasoning trace $R = [(q_i, G_i, a_i, z_i)]_{i=1}^N$ in which each step $i$ generates a sub-question $q_i$, selects a visual focus $G_i$ (a subset of the input image collection), answers $a_i$, and emits a stopping signal $z_i$. The chain is what the synthesized dataset encodes and what the fine-tuned model learns to reproduce. Its load-bearing property is that every step is anchored to a minimal image subset, so the model practises selective attention across images rather than attending to all inputs at once. The data-side machinery is the four-stage bottom-up synthesis loop (feature extraction, pair connection, relevance annotation, question generation) that turns raw image collections into chained question–answer paths without relying on expensive closed-source generation.

What would settle it

Train the same two base models with LoRA on a matched 150K-sample dataset that contains the same questions, images, and final answers but replaces the multi-step focus-chain traces with a single flat answer (or with an unstructured chain-of-thought that does not explicitly select image subsets). If MMIU and MuirBench gains shrink to near zero, the focus-centric format is the active ingredient; if they persist, the paper's attribution to the chain approach is unsupported.

Watch

Extended reading notes

Core claim

The discovery is a training approach and a data-generation pipeline. In the Focus-Centric Visual Chain approach, a VLM confronted with a set of images and a question iteratively produces a sub-question, selects the minimal subset of images needed to answer it, derives an answer, and accumulates these until it can synthesize a final answer. The accompanying Focus-Centric Data Synthesis framework builds training examples from the bottom up: an extractor writes detailed textual profiles of each image, a connector links related image pairs, an annotator labels each link as temporal, spatial, or semantic, and a questioner produces chained sub-questions plus a composite question. All components use open-source models, so the pipeline is reproducible and cheap. Fine-tuning LLaVA-OneVision-7B and Qwen2-VL-7B with LoRA on the resulting VISC-150K yields average gains of 3.16% and 2.24% across seven multi-image benchmarks, and the authors report that the model learns to apply the chained reasoning format at test time, including on video frames and GUI navigation screenshots.

Load-bearing premise

The paper assumes that the performance gains come from the focus-centric chain format itself, but it never compares against fine-tuning on an equally large set of multi-image instruction data without that chain structure, so extra data alone could explain part or all of the improvement.

Editorial extensions

If this is right

  • Fine-tuning on VISC-150K improves both LLaVA-OneVision-7B and Qwen2-VL-7B on every one of the seven benchmarks except one (Mantis-Eval for Qwen2-VL, which drops 0.48 points), with average gains of 3.16 and 2.24 percentage points.
  • The approach transfers to video understanding: MVBench improves by 1.53 and 1.01 points, consistent with treating video as a temporally ordered multi-image input.
  • General capabilities are not sacrificed: on HallusionBench, MMStar, MMMU, and MathVista, the fine-tuned Qwen2-VL stays within ±0.3 points on three benchmarks and improves by 1.5 on HallusionBench.
  • Performance scales with data size: increasing the fine-tuning subset from 25K to 125K yields rapid gains, with diminishing but continued improvements toward 150K, suggesting the synthetic pipeline can be scaled further.
  • The model generalizes to sub-tasks and image types not present in VISC-150K, e.g., geographic understanding in MuirBench, indicating the chains teach a transferable skill rather than memorized question formats.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The most direct test the paper does not run is a matched control: fine-tuning on the same 150K question–answer pairs with the chain structure flattened or removed. If the gains persist, the improvement may come from more multi-image instruction data rather than from the focus-centric format itself; if they vanish, the format is the active ingredient.
  • Because the synthesis pipeline uses only open-source models and structured prompts, it can be pointed at new image domains (diagrams, charts, code screenshots) that the authors note are currently untested, effectively turning the framework into a general multi-image reasoning data generator.
  • The dynamic focus selection mechanism resembles an attention-routing procedure; a natural extension is to make the focus subset explicit at test time (e.g., allowing the model to crop or zoom into chosen images), which could further reduce interference from irrelevant images in large sets.
  • The reported 97.5% human-validated accuracy of the synthesized chains, with inter-annotator agreement kappa = 0.637, suggests the data pipeline is reliable enough to serve as a cheaper substitute for closed-source distillation in other multimodal reasoning settings.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper introduces Focus-Centric Visual Chain (FCVC), a multi-step reasoning format for multi-image vision-language tasks, and Focus-Centric Data Synthesis (FCDS), a bottom-up pipeline that builds the VISC-150K dataset of 150K reasoning instances using open-source models. The authors fine-tune LLaVA-OneVision-7B and Qwen2-VL-7B via LoRA on VISC-150K and report average accuracy gains of 3.16% and 2.24% on seven multi-image benchmarks, along with analyses of data scale, sub-task breakdown, input-image count, general capability retention, and a human quality audit. The paper claims the gains validate the FCVC paradigm and that the method achieves state-of-the-art on four of the seven benchmarks.

Significance. The VISC-150K dataset and the FCDS pipeline are potentially useful community resources: they are built entirely with open-source models, are accompanied by a human-evaluation audit (97.5% validity on 200 samples), and the paper demonstrates consistent benchmark improvements across two different model architectures. If the causal attribution to the focus-centric format were established, the work would offer a scalable recipe for improving multi-image reasoning in VLMs. However, the current experimental design does not isolate the proposed paradigm from the generic effect of adding multi-image instruction data, and the state-of-the-art claim is not supported as written when closed-source models are included.

major comments (4)
  1. [§4.4, §4.5 (RQ1–RQ5)] The central claim that the performance gains are due to the Focus-Centric Visual Chain format is not supported by the experimental design. There is no control condition in which the same 150K instances are fine-tuned without the focus-centric sub-question decomposition (e.g., as flat QA or with a generic chain-of-thought prompt), nor a comparison against an equally sized alternative multi-image instruction dataset. RQ1–RQ5 vary data scale, sub-tasks, image counts, general ability, and data quality, but none of them manipulates the reasoning format while holding the underlying images, questions, and answers fixed. Without such an ablation, the observed improvements could be a generic effect of additional multi-image instruction data rather than a benefit of the proposed paradigm.
  2. [§4.4, Table 1, Abstract] The statement that the method 'achieves new state-of-the-art on four out of the seven' benchmarks is only true if the comparison is restricted to open-source models. Table 1 lists GPT-4V/GPT-4o scores of 55.70 on MMIU and 68.00 on MuirBench, both higher than the best VISC-fine-tuned model's 52.76 and 60.16, respectively. The text should explicitly qualify that the state-of-the-art claim is among open-source models, or otherwise revise the claim to reflect the full baseline set.
  3. [Abstract, §4.4, Table 1] The paper states that VISC-150K 'consistently brings performance improvements across all challenging benchmarks,' but Table 1 reports a 0.48-point drop for Qwen2-VL-VISC on Mantis-Eval relative to the Qwen2-VL baseline. The reported average gain of 2.24% includes this negative value, so the wording 'consistently' and 'across all' is inaccurate. This should be reworded to describe the actual pattern (e.g., improvements on six of seven benchmarks).
  4. [§3.2, §4.1] The FCDS pipeline uses LLaVA-OneVision-7B as both the Extractor and the Annotator, and LLaVA-OneVision-7B is also one of the two models fine-tuned on the resulting dataset. This creates a potential self-distillation effect that could inflate the reported gains on the LLaVA-OneVision architecture, because the training data is generated by the same model family that is later evaluated. The paper does not discuss or control for this distributional overlap. The gains on Qwen2-VL suggest this is not the sole explanation, but the missing control condition in the first major comment also prevents this confound from being resolved.
minor comments (7)
  1. [Abstract] The abstract ends with ': /gtbVISC', which appears to be a leftover placeholder or malformed link and should be removed or replaced with the actual dataset URL.
  2. [§4.3] The text says 'The statics of each benchmark' and should read 'statistics'.
  3. [§4.2] The phrase 'close-source VLMs' should be 'closed-source VLMs'.
  4. [§3.1] In the formalization, 'the model focuses on a visual evidence subset G⊆G' uses the same symbol for the subset and the full collection; this should be written as G_i ⊆ G (or an equivalent distinct notation) to avoid ambiguity.
  5. [Appendix F] The appendix ends with '...Fleiss’κ = 0.637). s', which contains a stray 's' at the end; this is likely a typographical artifact.
  6. [References] The MVBench reference (Li et al., 2024b) contains a long sequence of identical-looking author names that appears to be a placeholder or formatting error; the authors should verify the citation.
  7. [Figure 7 caption] The caption contains the typo 'Foucs' and should read 'Focus'.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the central claim is tested on external benchmarks, not derived from the dataset-generation equations or from a self-citation chain.

full rationale

I walked the claimed derivation chain: Focus-Centric Data Synthesis builds image profiles, pairwise connections, relevance annotations, and chained sub-questions (Section 3.2), then VISC-150K is used for LoRA fine-tuning of two base VLMs (Section 4.1), and the resulting models are evaluated on seven independent multi-image benchmarks (Table 1). None of these benchmarks is constructed from the paper's own equations, and the benchmark scores are not algebraically or statistically forced by the synthesis procedure. Equations (1)-(11) define the reasoning format and data-generation modules but do not assert that benchmark accuracy reduces to those definitions. RQ1-RQ5 examine data scale, sub-tasks, image counts, general abilities, and a human quality audit of 200 samples; the human audit provides external validation of data quality rather than a circular justification of downstream gains. The only self-citations (e.g., Cheng et al. 2024 in related work) are not load-bearing for the core benchmark claims. The closest concern is experimental, not circular: Section 4.1 fine-tunes on VISC-150K and Section 4.4 attributes the gains to the Focus-Centric Visual Chain paradigm, but no equivalently sized control dataset without the focus-centric formatting is included, so the specific cause of the improvements is not isolated. That is an attribution/confound limitation, not a case where the prediction is equivalent to its inputs by construction. Accordingly, no circular step meeting the required quote-and-reduction standard can be exhibited; score 1 reflects only the self-distillation flavor of training on data generated in part by the same model family, which is not itself circular.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The central claim relies on the quality of a model-generated dataset and on the transferability of fine-tuning on that dataset. No numeric free parameters are fit to the benchmark results, and no new physical or conceptual entities are introduced.

assumptions (3)
  • domain assumption The automated synthesis pipeline produces correct profiles, pair connections, relationship annotations, and reasoning chains.
    The entire training signal is built by the FCDS pipeline described in Section 3.2. Only a stratified sample of 200 instances is human-validated in Appendix F.
  • domain assumption Fine-tuning on VISC-150K transfers to the evaluated benchmarks rather than memorizing the synthetic distribution.
    The method is evaluated on seven external benchmarks, but no analysis separates generalization from distribution overlap in Sections 4.2 to 4.5.
  • domain assumption LLaVA-OneVision and Qwen2-VL are representative VLM architectures for drawing general conclusions.
    Only two base models are tested, both in the 7B size class, so claims of universality rest on this pair.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Weaving Context Across Images: Improving Vision-Language Models through Focus-Centric Visual Chains." pith.science (2026). https://pith.science/paper/S4NMTYBN

@misc{pith2026250420199,
  author       = {Pith},
  title        = {Pith review of: Weaving Context Across Images: Improving Vision-Language Models through Focus-Centric Visual Chains},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/S4NMTYBN}},
  note         = {Machine review of arXiv:2504.20199}
}
read the original abstract

Vision-language models (VLMs) achieve remarkable success in single-image tasks. However, real-world scenarios often involve intricate multi-image inputs, leading to a notable performance decline as models struggle to disentangle critical information scattered across complex visual features. In this work, we propose Focus-Centric Visual Chain, a novel paradigm that enhances VLMs'perception, comprehension, and reasoning abilities in multi-image scenarios. To facilitate this paradigm, we propose Focus-Centric Data Synthesis, a scalable bottom-up approach for synthesizing high-quality data with elaborate reasoning paths. Through this approach, We construct VISC-150K, a large-scale dataset with reasoning data in the form of Focus-Centric Visual Chain, specifically designed for multi-image tasks. Experimental results on seven multi-image benchmarks demonstrate that our method achieves average performance gains of 3.16% and 2.24% across two distinct model architectures, without compromising the general vision-language capabilities. our study represents a significant step toward more robust and capable vision-language systems that can handle complex visual scenarios.

Figures

Figures reproduced from arXiv: 2504.20199 by the authors.

Figure 1
Figure 1. A multi-image QA example: Using Focus￾Centric Visual Chain, our model LLaVA-OneVision￾VISC successfully answers a question that both GPT-4o and LLaVA-OneVision fail to solve correctly. sual discontinuity: The fragmentation of infor￾mation between images makes it challenging to accurately grasp cross-image relationships. To address these challenges, we propose Focus￾Centric Visual Chain, a multi-image reasoning parad… view at source ↗
Figure 2
Figure 2. The Focus-Centric Data Synthesis framework consists of four stages: Feature Extraction processes raw visual input, extracting object-level attributes and interactions into structured image profiles. Pair Connection links related image nodes based on their profiles. Relevance Annotation identifies and annotates relationships between nodes from temporal, spatial, and semantic perspectives. Question Generation utilizes… view at source ↗
Figure 3
Figure 3. The impact of dataset scale on LLaVA￾OneVision’s performance across MMIU and MuirBench benchmarks. As the data scale increases, the model’s accuracy progressively improves. maintains efficacy. This result substantiates that our Focus-Centric Visual Chain reasoning paradigm operates in a domain-agnostic manner, exhibiting robust applicability as well as transferability. 4.5 Discussions In this section, we propose fiv… view at source ↗
Figures from the paper (4 more)
Figure 5
Figure 5. Figure 5: The distribution of task accuracy for LLaVA [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: The image count distribution in VISC-150K [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: In the cross-image visual reasoning case, Qwen2-VL-VISC trained on the VISC-150K dataset with the [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: In a cross-image GUI navigation case study, LLaVA-OneVision-VISC, guided by the [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Mimic Human Cognition, Master Multi-Image Reasoning: A Meta-Action Framework for Enhanced Visual Understanding

    cs.CV 2026-01 conditional novelty 6.0 of 10

    A structured five-step reasoning template plus diverse-trajectory cold start and diversity-preserving two-stage RL lifts a 7B multimodal model to state-of-the-art multi-image reasoning on several benchmarks.

  2. MiCo: Multi-image Contrast for Reinforcement Visual Reasoning

    cs.CV 2025-06 conditional novelty 6.0 of 10

    A self-supervised contrastive triplets plus weak-to-strong augmented GRPO training method improves multi-image reasoning in Qwen2.5-VL-7B.

  3. PeRL: Permutation-Enhanced Reinforcement Learning for Interleaved Vision-Language Reasoning

    cs.CV 2025-06 conditional novelty 6.0 of 10

    PeRL applies reinforcement learning to a vision-language model with image-order permutation and difficulty-based data filtering, improving multi-image reasoning while keeping single-image performance.

  4. The Truth Becomes Clearer Through Debate! Multi-Agent Systems with Large Language Models Unmask Fake News

    cs.SI 2025-05 conditional novelty 5.0 of 10

    TED uses structured pro/con debates between LLM agents plus a graph-based analysis model to detect fake news more accurately than prior methods on two benchmark datasets.

Reference graph

Works this paper leans on

53 extracted references · 12 canonical work pages · cited by 4 Pith papers

  1. [1]

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katie Millican, Malcolm Reynolds, Roman Ring, Eliza Rutherford, Serkan Cabi, Tengda Han, Zhitao Gong, Sina Samangooei, Marianne Monteiro, Jacob Menick, Sebastian Borgeaud, Andrew Brock, Aida Nematzadeh, Sahand Sharifzadeh, Mikolaj Binkowski,...

    arXiv 2022

  2. [2]

    Anas Awadalla, Irena Gao, Josh Gardner, Jack Hessel, Yusuf Hanafy, Wanrong Zhu, Kalyani Marathe, Yonatan Bitton, Samir Gadre, Shiori Sagawa, Jenia Jitsev, Simon Kornblith, Pang Wei Koh, Gabriel Ilharco, Mitchell Wortsman, and Ludwig Schmidt. 2023. Openflamingo: An open-source framework for training large autoregressive vision-language models. arXiv prepri...

    arXiv 2023

  3. [3]

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023. https://arxiv.org/abs/2308.12966 Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond . arXiv preprint arXiv:2308.12966

    arXiv 2023

  4. [4]

    Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Michal Podstawski, Hubert Niewiadomski, Piotr Nyczyk, and Torsten Hoefler. 2023. https://arxiv.org/abs/2308.09687 Graph of thoughts: Solving elaborate problems with large language models . arXiv preprint arXiv:2308.09687

    arXiv 2023

  5. [5]

    Nicolò De Sabbata, Kia Ghods, Amogh Joshi, Alexander Ku, Steven M

    Declan Iain Campbell, Sunayana Rane, Tyler Giallanza, C. Nicolò De Sabbata, Kia Ghods, Amogh Joshi, Alexander Ku, Steven M. Frankland, Thomas L. Griffiths, Jonathan D. Cohen, and Taylor Whittington Webb. 2024. https://arxiv.org/abs/2411.00238 Understanding the limits of vision language models through the lens of the binding problem . arXiv preprint arXiv:...

    arXiv 2024

  6. [6]

    Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, et al. 2024 a . Are we on the right way for evaluating large vision-language models? arXiv preprint arXiv:2403.20330

    arXiv 2024

  7. [7]

    Zhe Chen, Weiyun Wang, Hao Tian, Shenglong Ye, Zhangwei Gao, Erfei Cui, Wenwen Tong, Kongzhi Hu, Jiapeng Luo, Zheng Ma, et al. 2024 b . https://arxiv.org/abs/2412.05271 Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling . arXiv preprint arXiv:2412.05271

    arXiv 2024

  8. [8]

    Zhe Chen, Weiyun Wang, Hao Tian, Shenglong Ye, Zhangwei Gao, Erfei Cui, Wenwen Tong, Kongzhi Hu, Jiapeng Luo, Zheng Ma, et al. 2024 c . https://arxiv.org/abs/2404.16821 How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites . arXiv preprint arXiv:2404.16821

    arXiv 2024

Show all 53 references
  1. [9]

    Chuanqi Cheng, Jian Guan, Wei Wu, and Rui Yan. 2024. From the least to the most: Building a plug-and-play visual reasoner via data synthesis. arXiv preprint arXiv:2406.19934

    2024 arXiv

  2. [10]

    Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. 2023. https://arxiv.org/abs/2305.06500 InstructBLIP : Towards general-purpose vision-language models with instruction tuning . arXiv preprint arXiv:...

    2023 arXiv

  3. [11]

    Maryam Daniali and Edward Kim. 2023. Perception over time: Temporal dynamics for robust image understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5656--5665

    2023

  4. [12]

    Joseph L. Fleiss. 1971. https://doi.org/10.1037/h0031619 Measuring nominal scale agreement among many raters . Psychological Bulletin, 76(5):378--382

    1971 doi

  5. [13]

    Xingyu Fu, Yushi Hu, Bangzheng Li, Yu Feng, Haoyu Wang, Xudong Lin, Dan Roth, Noah A Smith, Wei-Chiu Ma, and Ranjay Krishna. 2024. Blink: Multimodal large language models can see but not perceive. In European Conference on Computer Vision, pages 148--166. Springer

    2024

  6. [14]

    Jiahui Gao, Renjie Pi, Jipeng Zhang, Jiacheng Ye, Wanjun Zhong, Yufei Wang, Lanqing Hong, Jianhua Han, Hang Xu, Zhenguo Li, and Lingpeng Kong. 2023. https://arxiv.org/abs/2312.11370 G-LLaVA: Solving Geometric Problem with Multi-Modal Large Language Model . arXiv preprint arXiv...

    2023 arXiv

  7. [15]

    Google Gemini Team. 2024. https://arxiv.org/abs/2403.05530 Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context . arXiv preprint arXiv:2403.05530

    2024 arXiv

  8. [16]

    Tianrui Guan, Fuxiao Liu, Xiyang Wu, Ruiqi Xian, Zongxia Li, Xiaoyu Liu, Xijun Wang, Lichang Chen, Furong Huang, Yaser Yacoob, et al. 2023. Hallusionbench: An advanced diagnostic suite for entangled language hallucination and visual illusion in large vision-language models. ar...

    2023 arXiv

  9. [17]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. https://openreview.net/forum?id=nZeVKeeFYf9 LoRA : Low-rank adaptation of large language models . In International Conference on Learning Representations (ICLR)

    2022

  10. [18]

    Shaohan Huang, Li Dong, Wenhui Wang, Yaru Hao, Saksham Singhal, Shuming Ma, Tengchao Lv, Lei Cui, Owais Khan Mohammed, Barun Patra, Qiang Liu, Kriti Aggarwal, Zewen Chi, Johan Bjorck, Vishrav Chaudhary, Subhojit Som, Xia Song, and Furu Wei. 2023. https://arxiv.org/abs/2302.140...

    2023 arXiv

  11. [19]

    Dongfu Jiang, Xuan He, Huaye Zeng, Cong Wei, Max Ku, Qian Liu, and Wenhu Chen. 2024. https://arxiv.org/abs/2405.01483 MANTIS : Interleaved multi-image instruction tuning . arXiv preprint arXiv:2405.01483

    2024 arXiv

  12. [20]

    Hugo Laurençon, Léo Tronchon, Matthieu Cord, and Victor Sanh. 2024. https://arxiv.org/abs/2405.02246 What matters when building vision-language models? arXiv preprint arXiv:2405.02246

    2024 arXiv

  13. [21]

    Junlin Lee, Yequan Wang, Jing Li, and Min Zhang. 2024. Multimodal reasoning with multimodal knowledge graph. arXiv preprint arXiv:2406.02030

    2024 arXiv

  14. [22]

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. 2024 a . https://arxiv.org/abs/2408.03326 LLaVA-OneVision : Easy visual task transfer . arXiv preprint arXiv:2408.03326

    2024 arXiv

  15. [23]

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. BLIP -2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In Proceedings of the 40th International Conference on Machine Learning, pages 19730--19742. PMLR

    2023

  16. [24]

    Yining Li, Yizhuo Zhang, Yifan Zhang, Zihan Zhang, Yuchen Zhang, Yixin Wang, Yixin Zhang, Yizhuo Wang, Yining Li, Yizhuo Zhang, et al. 2024 b . https://arxiv.org/abs/2406.12345 MV-Bench: A Comprehensive Benchmark for Multimodal Video Understanding . arXiv preprint arXiv:2406.12345

    2024 arXiv

  17. [25]

    Ji Lin, Hongxu Yin, Chunyuan Zhang, Xiyang Chen, Lu Yuan, and Song Han. 2023. https://arxiv.org/abs/2312.07533 VILA: On Pre-training for Visual Language Models . arXiv preprint arXiv:2312.07533

    2023 arXiv

  18. [26]

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2023 a . https://arxiv.org/abs/2310.03744 Improved baselines with visual instruction tuning . arXiv preprint arXiv:2310.03744

    2023 arXiv

  19. [27]

    Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. 2024 a . https://llava-vl.github.io/blog/2024-01-30-llava-next/ LLaVA-NeXT: Improved reasoning, OCR, and world knowledge

    2024

  20. [28]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023 b . https://arxiv.org/abs/2304.08485 Visual instruction tuning . arXiv preprint arXiv:2304.08485

    2023 arXiv

  21. [29]

    Yuhan Liu, Xiuying Chen, Xiaoqing Zhang, Xing Gao, Ji Zhang, and Rui Yan. 2024 b . From skepticism to acceptance: simulating the attitude dynamics toward fake news. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, pages 7886--7894

    2024

  22. [30]

    Yuhan Liu, Zirui Song, Xiaoqing Zhang, Xiuying Chen, and Rui Yan. 2024 c . From a tiny slip to a giant leap: An llm-based simulation for fake news evolution. arXiv preprint arXiv:2410.19064

    2024 arXiv

  23. [31]

    Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. 2023. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. arXiv preprint arXiv:2310.02255

    2023 arXiv

  24. [32]

    Fanqing Meng, Jin Wang, Chuanhao Li, Quanfeng Lu, Hao Tian, Jiaqi Liao, Xizhou Zhu, Jifeng Dai, Yu Qiao, Ping Luo, Kaipeng Zhang, and Wenqi Shao. 2024. https://arxiv.org/abs/2408.02718 MMIU : Multimodal multi-image understanding for evaluating large vision-language models . ar...

    2024 arXiv

  25. [33]

    OpenAI. 2024 a . https://arxiv.org/abs/2410.21276 GPT-4o system card . arXiv preprint arXiv:2410.21276

    2024 arXiv

  26. [34]

    OpenAI. 2024 b . https://openai.com/index/openai-o1-system-card/ OpenAI o1 System Card

    2024

  27. [35]

    Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, and Furu Wei. 2023. https://arxiv.org/abs/2306.14824 Kosmos-2: Grounding multimodal large language models to the world . Preprint, arXiv:2306.14824

    2023 arXiv

  28. [36]

    Yichong Shi, Yucheng Li, Yifan Zhang, Yizhuo Zhang, Yuchen Zhang, Yixin Wang, Yining Li, Zihan Zhang, Yizhuo Wang, Yixin Zhang, et al. 2024. https://aclanthology.org/2024.findings-emnlp.268 Math-LLaVA: Bootstrapping Mathematical Reasoning for Multimodal Large Language Models ....

    2024

  29. [37]

    Alane Suhr, Stephanie Zhou, Iris Zhang, Huajun Zhang, Rui Bai, Yichen Xiong, and Yoav Artzi. 2019. https://aclanthology.org/P19-1644/ A Corpus for Reasoning about Natural Language Grounded in Photographs . In Proceedings of the 57th Annual Meeting of the Association for Comput...

    2019

  30. [38]

    Fei Wang, Xingyu Fu, James Y Huang, Zekun Li, Qin Liu, Xiaogeng Liu, Mingyu Derek Ma, Nan Xu, Wenxuan Zhou, Kai Zhang, et al. 2024 a . https://arxiv.org/abs/2406.09411 Muirbench: A comprehensive benchmark for robust multi-image understanding . arXiv preprint arXiv:2406.09411

    2024 arXiv

  31. [39]

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. 2024 b . Qwen2-vl: Enhancing vision-language m...

    2024 arXiv

  32. [40]

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. https://openreview.net/forum?id=1PL1NIMMrw Self-consistency improves chain of thought reasoning in language models . In International Conference on Learning Repr...

    2023

  33. [41]

    Yiqi Wang, Wentao Chen, Xiaotian Han, Xudong Lin, Haiteng Zhao, Yongfei Liu, Bohan Zhai, Jianbo Yuan, Quanzeng You, and Hongxia Yang. 2024 c . https://arxiv.org/abs/2401.06805 Exploring the reasoning abilities of multimodal large language models (mllms): A comprehensive survey...

    2024 arXiv

  34. [42]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. 2022. https://proceedings.neurips.cc/paper_files/paper/2022/file/9d5609613524ecf4f15af0f7b31abca4-Paper-Conference.pdf Chain-of-thought prompting elicits reasoning i...

    2022

  35. [43]

    Guowei Xu, Peng Jin, Hao Li, Yibing Song, Lichao Sun, and Li Yuan. 2024. https://arxiv.org/abs/2411.10440 Llava-cot: Let vision language models reason step-by-step . arXiv preprint arXiv:2411.10440

    2024 arXiv

  36. [44]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Me...

    2024 arXiv

  37. [45]

    Huanjin Yao, Jiaxing Huang, Wenhao Wu, Jingyi Zhang, Yibo Wang, Shunyu Liu, Yingjie Wang, Yuxin Song, Haocheng Feng, Li Shen, et al. 2024. Mulberry: Empowering mllm with o1-like reasoning and reflection via collective monte carlo tree search. arXiv preprint arXiv:2412.18319

    2024 arXiv

  38. [46]

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. https://arxiv.org/abs/2305.10601 Tree of thoughts: Deliberate problem solving with large language models . arXiv preprint arXiv:2305.10601

    2023 arXiv

  39. [47]

    Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al. 2024. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. In Proceedings of the IEEE/CVF Conferenc...

    2024

  40. [48]

    Hanbo Zhang, Xin Li, Yiduo Zhang, Xiaojun Zhang, and Lei Li. 2023. https://arxiv.org/abs/2302.00923 Multimodal chain-of-thought reasoning in language models . arXiv preprint arXiv:2302.00923

    2023 arXiv

  41. [49]

    Xiangyu Zhang, Jiajun Li, Zihan Wang, Xi Chen, Wen Zhang, and Huajun Chen. 2024 a . https://www.techrxiv.org/doi/full/10.36227/techrxiv.171259566.60211714/v1 Multi-modal knowledge graph completion in the wild . arXiv preprint arXiv:2405.12345

    2024

  42. [50]

    Yizhuo Zhang, Yifan Zhang, Yuchen Zhang, Yixin Wang, Yining Li, Zihan Zhang, Yizhuo Wang, Yixin Zhang, Yucheng Li, Yichong Shi, et al. 2024 b . https://arxiv.org/abs/2407.08739 MAVIS: Mathematical Visual Instruction Tuning . arXiv preprint arXiv:2407.08739

    2024 arXiv

  43. [51]

    Bingchen Zhao, Yongshuo Zong, Letian Zhang, and Timothy Hospedales. 2024. https://arxiv.org/abs/2406.12742 Benchmarking multi-image understanding in vision and language models: Perception, knowledge, reasoning, and multi-hop reasoning . arXiv preprint arXiv:2406.12742

    2024 arXiv

  44. [52]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  45. [53]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.