Pith. sign in

REVIEW 4 major objections 5 minor 40 references

This paper claims that a self-training loop called See-Think-Learn (STL) improves vision-language reasoning by making a model generate structured captions and both positive and negative rationales, and it reports consistent gains over answe

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 18:58 UTC pith:3CASNF7H

load-bearing objection A plausible self-training recipe for VLMs that is worth referee time, but the paper must state the train/eval split before the central accuracy claim can be trusted. the 4 major comments →

arxiv 2512.02456 v2 pith:3CASNF7H submitted 2025-12-02 cs.CV cs.CL

See, Think, Learn: A Self-Taught Multimodal Reasoner

Sourabh Sharma , Sonam Gupta , Sadbhawna This is my paper
classification cs.CV cs.CL
keywords multimodal reasoningself-trainingvision-language modelschain-of-thoughtrationale generationnegative examplesvisual perceptionmultiple-choice visual question answering
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

STL is a self-training framework for vision-language models that replaces human- or teacher-written reasoning chains with rationales the model generates itself. The key idea is to force the model to 'see before thinking': first write a detailed image caption, then reason from those visual details, then answer. Only self-generated rationales that lead to correct answers are kept, and for each such sample the model also writes negative rationales explaining why each wrong choice is incorrect. The model is then fine-tuned on this union of positive and negative rationales, iteratively improving both perception and reasoning. The paper reports that STL outperforms STaR and direct answer fine-tuning on M3CoT domains, and that its rationales are preferred by human annotators over STaR's.

Core claim

STL introduces a structured 'see-before-thinking' rationale template: given an image and a multiple-choice question, the model first produces a Caption describing the visual elements relevant to the question, then a Reasoning step grounded in that caption, and finally a Conclusion selecting an answer. Only samples the model answers correctly are used as positive rationales; for each of those, the model also generates a negative rationale for every wrong option, explaining why that option is incorrect. Fine-tuning on the combined set of self-generated positive and negative rationales iteratively strengthens both visual perception and discriminative reasoning. On M3CoT, STL achieves 54.34% ave

What carries the argument

The central mechanism is the structured positive rationale (Caption → Reasoning → Conclusion) combined with negative rationalization. The Caption forces explicit perceptual grounding before reasoning, counteracting the shortcut-seeking behavior that chain-of-thought prompting often exposes in open VLMs. Negative rationales—explanations of why each wrong choice fails—teach the model to reject distractors and suppress spurious visual details. These self-generated rationales form the training set for LoRA fine-tuning, and the loop repeats until convergence, letting the model progressively refine both its perception and its reasoning.

Load-bearing premise

The reported improvements rest on the assumption that the samples used to generate rationales and fine-tune the model are disjoint from the samples used for evaluation; the paper never states this split, so if self-training reuses evaluation items the gains would be inflated by memorization.

What would settle it

Take the released code and data, split M3CoT into disjoint train and evaluation folds by domain, run the STL self-training loop using only the train fold to generate rationales and fine-tune, and measure accuracy on the untouched evaluation fold. If the gap over STaR and Direct SFT vanishes or reverses on a truly held-out split, the central claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • VLMs can improve multimodal reasoning without human-written chain-of-thought annotations or expensive proprietary teacher models.
  • Explicitly separating perception (caption) from inference (reasoning) reduces shortcut answers and yields rationales that human annotators prefer 35% more often than STaR's.
  • Adding negative rationales improves accuracy and rationale fidelity—e.g., correctly identifying a buffet breakfast rather than hallucinating a fork and knife in the image.
  • The gains transfer to a stronger model (Qwen2.5-VL-7B) and the framework runs on a single 48GB GPU with LoRA and one epoch of training.
  • STL's self-generated rationales are competitive with human-annotated rationales on two M3CoT domains, pointing to a scalable alternative to annotation-heavy pipelines.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The 'see-before-thinking' template is a generic inductive bias that could apply beyond multiple-choice VQA—to diagram understanding, medical imaging, or embodied agents that must first describe a scene before deciding an action.
  • Negative rationales could serve as a transferable source of contrastive supervision for open-ended visual question answering, potentially reducing hallucination without pairwise human feedback.
  • The validity of the reported gains depends on the self-training data being disjoint from the evaluation set; the paper does not explicitly state this split, so an independent check on a held-out split is the first thing to confirm before relying on the numbers.
  • If the gains hold, self-training with structured rationales may become a default recipe for upgrading open-source VLMs between architecture releases, removing the need to curate reasoning chains externally.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes See-Think-Learn (STL), a self-training framework for vision-language models (VLMs). STL prompts the model to produce structured rationales in a Caption–Reasoning–Conclusion format ('see before thinking'), keeps only rationales from correctly answered samples, and adds negative rationales that explain why incorrect choices are wrong. The model is then iteratively fine-tuned on this self-generated set. Experiments on LLaVA-1.5-7B and Qwen2.5-VL-7B over M3CoT domains report average accuracy gains over STaR and R3V, and an ablation attributes the gains to the structured prompt and negative rationalization. The central claim is that STL consistently outperforms baselines trained on answers alone or on self-generated reasoning.

Significance. If the empirical results are valid, STL is a useful contribution to multimodal reasoning: it is self-supervised in the sense of not requiring human- or proprietary-model rationales, it explicitly couples perception with reasoning, and the negative-rationale idea is a plausible mechanism for discriminative learning. The paper also includes an ablation study and a human preference comparison, which strengthen the qualitative claims. However, the empirical support is currently incomplete: the train/eval split is not specified, all results are single runs without variance estimates, and one domain contradicts the 'consistently outperforms' wording. The conceptual framework is potentially publishable, but the evidence needs substantial strengthening.

major comments (4)
  1. [§4.1, Algorithm 1, Tables 1–2] The manuscript never states whether the self-training dataset D is disjoint from the evaluation items. Algorithm 1 performs all self-training (inference, positive/negative rationale construction, fine-tuning) on D, while Table titles read 'M3CoT Evaluation Splits' and §4.1 says samples are 'drawn from the M3CoT dataset.' If the same image–question pairs are used for both self-training and the reported accuracies, the gains (e.g., 54.34 vs. 51.21 in Table 1; 85.36 vs. 81.06 in Table 2) could be inflated by memorization. The supplement mentions 'test set responses' (Fig. 7), but no formal split definition, sample sizes, or disjointness statement is provided. Please specify exactly how D is defined and confirm that Tables 1–2 report held-out items.
  2. [§5, Table 1, Abstract] The abstract claims STL 'consistently outperforms' baselines, but Table 1 shows STaR beating STL on Natural-Science (53.90 vs. 50.45). The text acknowledges this exception but does not qualify the abstract. Additionally, every result is a single run with no error bars, confidence intervals, or significance tests; several margins are small (e.g., Commonsense 67.19 vs. 64.98). The paper should report multiple seeds or confidence intervals and should either remove 'consistently' or explain the domain-specific failure.
  3. [§3.3, Algorithm 1, Fig. 3] Negative rationales are generated with the gold answer in the prompt ('The correct choice is {correct_choice}'), and Algorithm 1 line 13 passes a_i to the negative-prompt inference. The text says the gold answer is withheld during training, but the generated rationale itself contains the correct answer (e.g., Fig. 8: 'The correct answer is (a) because...'). This makes the claim that the model is 'reason[ing] independently about distractors' questionable. Please clarify whether the training input actually omits not only a_i but also any textual occurrence of the correct choice in the target, or provide an analysis showing that test-time generations do not exploit answer tokens.
  4. [§5, Table 4] The text states that STL-generated rationales 'achieve competitive or better performance' than human-annotated M3CoT rationales, but for LLaVA the average is 61.56 vs. 63.63 (worse), with only Commonsense better. The claim that 'STL can approximate human-level reasoning' is not supported by the averaged numbers. Please restrict the claim to the specific domain(s) where it holds, or run significance tests.
minor comments (5)
  1. [Table 1] R3V is reported on only Commonsense and Language-Science, so the comparison claimed in the text ('surpassing STaR by 3% and R3V by 7%') is not supported by a complete table. Report the missing domains or qualify the statement.
  2. [§6, Subjective Analysis] Inter-annotator agreement is not reported, and the selection procedure (sampling from questions both methods answered correctly) introduces a conditional comparison; please report agreement and describe the sampling more precisely.
  3. [§4.3 and Algorithm 1] Algorithm 1 loops 'until convergence,' but §4.3 only says one epoch of LoRA training. The number of self-training iterations and the exact data sizes per iteration are missing. Include these for reproducibility.
  4. [Introduction / Related Work] The manuscript uses both 'Vision-Language Models' and 'Visual Language Models' / 'MLLMs' inconsistently; unify the terminology.
  5. [Figure 6] The caption says 'across all domains' but displays counts out of 150; the total number of domains in the figure and the per-domain totals should be stated clearly.

Circularity Check

0 steps flagged

No significant circularity: STL's reported gains are external benchmark accuracies, not quantities derived from its own definitions; the self-training loop is the method itself, not a circular validation.

full rationale

The central claim is an empirical accuracy comparison on M3CoT evaluation splits (Tables 1-2), produced by fine-tuned VLMs and measured against baselines; it is not a quantity that follows from the framework's equations by construction. Algorithm 1 is a self-training procedure in which the model generates rationales, filters them using the gold label, and fine-tunes on its own outputs. That is self-referential by design but not circular in the sense of assuming the conclusion: no parameter is fitted to the reported evaluation numbers, no uniqueness theorem is imported, and no ansatz is smuggled in via self-citation. Two reporting ambiguities are worth flagging as validity risks, not circularity: Section 4.1 says only that samples are 'drawn from the M3CoT dataset' without explicitly stating that the self-training set D in Algorithm 1 is disjoint from the 'M3CoT Evaluation Splits' named in Tables 1-2; and Figure 3's negative-rationale prompt includes 'The correct choice is {correct_choice}' during generation, while Section 3.3 says the gold answer is later withheld during training. If D overlaps the evaluation split, the reported accuracies would be training-set performance, but the paper does not exhibit that reduction, and the supplementary's reference to 'test set responses' suggests a held-out split may exist. These are experimental-reporting concerns, not demonstrated definitional equivalences, so they do not make the derivation circular.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

The framework relies on standard supervised VQA data and training hyperparameters chosen by hand. The only genuinely novel conceptual load is the assumption that a structured caption-first prompt plus negative rationales improves self-training; this is empirically asserted rather than independently established. The unstated train/eval split is the most consequential implicit assumption.

free parameters (4)
  • LoRA rank r = 128
    Chosen without reported tuning; assumed shared across baselines, but baseline hyperparameters are not detailed.
  • LoRA scaling factor alpha = 256
    Chosen by hand; affects the expressiveness of the adapter and is not justified.
  • Number of self-training iterations = not reported ('until convergence')
    Algorithm 1 loops until convergence; the paper does not state how many iterations were run or what criterion was used.
  • Learning rate (initial value) = not reported
    A cosine schedule is mentioned, but the peak learning rate and warmup are absent; this affects training dynamics.
axioms (4)
  • domain assumption The multiple-choice labels a_i in M3CoT are ground truth.
    All filtering of positive samples and generation of negative rationales depends on these labels being correct.
  • domain assumption A correct prediction implies the model's understanding of the image and question is good enough for its rationales to be used as supervision.
    Stated in Section 3.3: 'We assume that a correct prediction means the model has a good understanding of the image and the question.' This is unproven; correct answers can arise from shortcut learning.
  • ad hoc to paper The 'see-before-thinking' structure (caption, reasoning, conclusion) improves perception and reasoning.
    This is the paper's core hypothesis, motivated by intuition; it is not derived or externally supported beyond the presented experiments.
  • domain assumption The self-training dataset D is disjoint from the evaluation split.
    Required for valid held-out evaluation but never explicitly stated; if false, all reported accuracies are invalid.

pith-pipeline@v1.3.0-alltime-deepseek · 13763 in / 14127 out tokens · 128922 ms · 2026-08-03T18:58:12.407630+00:00 · methodology

0 comments
read the original abstract

Vision-Language Models (VLMs) have achieved remarkable progress in integrating visual perception with language understanding. However, effective multimodal reasoning requires both accurate perception and robust reasoning, and weakness in either limits the performance of VLMs. Prior efforts to enhance reasoning often depend on high-quality chain-of-thought (CoT) data, obtained via labor-intensive human annotations, costly proprietary models, or self-training methods that overlook perception. To address these limitations, we propose a simple yet effective self-training framework called See-Think-Learn (STL). At its core, STL introduces a structured reasoning template that encourages the model to see before thinking, first extracting visual attributes in textual form, then using them to guide reasoning. The framework jointly improves perception and reasoning by having the model generate and learn from its own structured rationales in a self-training loop. Furthermore, we augment the training data with negative rationales, i.e. explanations that justify why certain answer choices are incorrect, to enhance the model's ability to distinguish between correct and misleading responses. This fosters more discriminative and robust learning. Experiments across diverse domains show that STL consistently outperforms baselines trained directly only on answers or self-generated reasoning, while qualitative analysis confirms the high quality of its rationales. STL thus provides a cost-effective solution to enhance multimodal reasoning ability of VLMs.

Figures

Figures reproduced from arXiv: 2512.02456 by Sadbhawna, Sonam Gupta, Sourabh Sharma.

Figure 1
Figure 1. Figure 1: Comparison of reasoning generated by our “See￾Think-Learn” (STL) framework with STaR [36] and R3V [10]. STL produces more detailed and perceptually grounded rationales, whereas STaR and R3V tend to overlook contextual cues and pro￾vide shorter, less comprehensive explanations. tasks [3, 7, 21, 37]. Motivated by this success, recent works [20, 28, 29] have attempted to extend CoT prompting to multimodal ext… view at source ↗
Figure 2
Figure 2. Figure 2: An overview of our detailed “See-Think-Learn” (STL) framework. In this framework, each image-question pair with multiple choices, together with a positive rationale prompt, is fed into the VLM to generate a caption, reasoning, and conclusion. If the model predicts the correct answer, the tuple [Question, (Caption, Reasoning, Answer)] is stored as a Positive Rationale in the Ratio￾nale Trainset. The remaini… view at source ↗
Figure 3
Figure 3. Figure 3: Prompt templates used for positive and negative rationalization in the STL framework. This dataset is used for iterative fine-tuning, encouraging the model to improve both perception and reasoning. Discriminative negative rationales: Rationales from correct samples capture only one side of reasoning spectrum. To emulate the human strategy of reflective learning, where incorrect options are critically analy… view at source ↗
Figure 4
Figure 4. Figure 4: Comparison of our “See-Think-Learn” (STL) framework with CoT Prompting. The example is taken from the Commonsense Split of M3CoT Dataset [6]. Unlike CoT prompting (a), our STL framework ((b) and (c)) effectively generates a detailed description and accurate reasoning for the image by leveraging the proposed Positive and Negative Rationale Prompts. In (a), the answer is incorrect, and the image description … view at source ↗
Figure 5
Figure 5. Figure 5: Qualitative Comparison on Natural Science Domain. Qualitative analysis shows that STL (ours) produces more coherent and logically consistent explanations than STaR, indicating deeper understanding and more faithful reasoning. condition represents our STL framework implemented with￾out the proposed structured rationale prompt and negative rationalization. i.e. this setup utilizes a prompt without cap￾tion a… view at source ↗
Figure 6
Figure 6. Figure 6: Comparison of Preferred Rationale Counts across Domains. Each subplot displays the number of times the rationale generated by each method (STaR and STL) was preferred (out of 150) within a domain. Across all domains, the reasoning generated by STL is preferred for more samples than that of STaR, highlight￾ing its superior quality. 5.2. Qualitative Results Qualitative Example on CommonSense [PITH_FULL_IMAG… view at source ↗
Figure 7
Figure 7. Figure 7: Response for Commonsense, Natural Science, Language Science, Social Science Domains. (top to bottom). The response of our STL framework is a detailed caption and reasoning along with the answer [PITH_FULL_IMAGE:figures/full_fig_p011_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Visualization of Positive and Negative Rationales from the proposed STL. The examples illustrate correct identification and reasoning for the chosen answer and rejection of an incorrect alternative [PITH_FULL_IMAGE:figures/full_fig_p012_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Screenshot of our subjective annotation GUI. Please note that the name of the rationale generator model has been kept anonymous to ensure a fair comparison. (a) Training Loss (b) Learning Rate (c) Global Step (d) Epochs [PITH_FULL_IMAGE:figures/full_fig_p013_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Tracking Data from WandB (Training). (a) Depicts the decay of the training loss over steps, indicating effective learning and convergence of the model. (b) Illustrates a learning rate schedule where the rate initially increases slightly before gradually decreasing, following a cosine strategy. (c) Shows a linear increase in the global training steps. (d) Displays the linear progression of training epochs … view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

40 extracted references · 16 linked inside Pith

  1. [1]

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023. 1

    Pith/arXiv arXiv 2023

  2. [2]

    Qwen-vl: A frontier large vision-language model with versatile abilities

    J Bai, S Bai, S Yang, S Wang, S Tan, P Wang, J Lin, C Zhou, and J Zhou. Qwen-vl: A frontier large vision-language model with versatile abilities. arxiv 2023.arXiv preprint arXiv:2308.12966, 2023. 1, 2

    Pith/arXiv arXiv 2023

  3. [3]

    Thinking machines: A survey of llm based reasoning strategies.arXiv preprint arXiv:2503.10814, 2025

    Dibyanayan Bandyopadhyay, Soham Bhattacharjee, and Asif Ekbal. Thinking machines: A survey of llm based reasoning strategies.arXiv preprint arXiv:2503.10814, 2025. 1

    Pith/arXiv arXiv 2025

  4. [4]

    Experiment tracking with weights and biases,

    Lukas Biewald. Experiment tracking with weights and biases,

  5. [5]

    Multimodal structured generation: Cvpr’s 2nd mmfm challenge technical report.arXiv preprint arXiv:2406.11403, 2024

    Franz Louis Cesista. Multimodal structured generation: Cvpr’s 2nd mmfm challenge technical report.arXiv preprint arXiv:2406.11403, 2024. 2

    Pith/arXiv arXiv 2024

  6. [6]

    M3CoT: A novel benchmark for multi- domain multi-step multi-modal chain-of-thought

    Qiguang Chen, Libo Qin, Jin Zhang, Zhi Chen, Xiao Xu, and Wanxiang Che. M3CoT: A novel benchmark for multi- domain multi-step multi-modal chain-of-thought. InPro- ceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8199–8221, Bangkok, Thailand, 2024. Association for Com- putational Linguistic...

    2024

  7. [7]

    Towards reasoning era: A survey of long chain-of-thought for reasoning large language models.arXiv preprint arXiv:2503.09567, 2025

    Qiguang Chen, Libo Qin, Jinhao Liu, Dengyun Peng, Jiannan Guan, Peng Wang, Mengkang Hu, Yuhang Zhou, Te Gao, and Wanxiang Che. Towards reasoning era: A survey of long chain-of-thought for reasoning large language models.arXiv preprint arXiv:2503.09567, 2025. 1

    Pith/arXiv arXiv 2025

  8. [8]

    Self-play fine-tuning converts weak lan- guage models to strong language models.arXiv preprint arXiv:2401.01335, 2024

    Zixiang Chen, Yihe Deng, Huizhuo Yuan, Kaixuan Ji, and Quanquan Gu. Self-play fine-tuning converts weak lan- guage models to strong language models.arXiv preprint arXiv:2401.01335, 2024. 2

    Pith/arXiv arXiv 2024

  9. [9]

    How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites

    Zhe Chen, Weiyun Wang, Hao Tian, Shenglong Ye, Zhangwei Gao, Erfei Cui, Wenwen Tong, Kongzhi Hu, Jiapeng Luo, Zheng Ma, et al. How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites. Science China Information Sciences, 67(12):220101, 2024. 2

    2024

  10. [10]

    Vision-language models can self- improve reasoning via reflection.The North American Chap- ter of the Association for Computational Linguistics, 2025

    Kanzhi Cheng, Yantao Li, Fangzhi Xu, Jianbing Zhang, Hao Zhou, and Yang Liu. Vision-language models can self- improve reasoning via reflection.The North American Chap- ter of the Association for Computational Linguistics, 2025. 1, 2, 5

    2025

  11. [11]

    Internlm-xcomposer2: Mastering free-form text-image composition and compre- hension in vision-language large model.arXiv preprint arXiv:2401.16420, 2024

    Xiaoyi Dong, Pan Zhang, Yuhang Zang, Yuhang Cao, Bin Wang, Linke Ouyang, Xilin Wei, Songyang Zhang, Haodong Duan, Maosong Cao, et al. Internlm-xcomposer2: Mastering free-form text-image composition and compre- hension in vision-language large model.arXiv preprint arXiv:2401.16420, 2024. 1

    Pith/arXiv arXiv 2024

  12. [12]

    Cantor: Inspiring multimodal chain-of-thought of mllm

    Timin Gao, Peixian Chen, Mengdan Zhang, Chaoyou Fu, Yunhang Shen, Yan Zhang, Shengchuan Zhang, Xiawu Zheng, Xing Sun, Liujuan Cao, et al. Cantor: Inspiring multimodal chain-of-thought of mllm. InProceedings of the 32nd ACM International Conference on Multimedia, pages 9096–9105,

  13. [13]

    V-star: Training verifiers for self-taught reasoners.arXiv preprint arXiv:2402.06457, 2024

    Arian Hosseini, Xingdi Yuan, Nikolay Malkin, Aaron Courville, Alessandro Sordoni, and Rishabh Agarwal. V-star: Training verifiers for self-taught reasoners.arXiv preprint arXiv:2402.06457, 2024. 2

    Pith/arXiv arXiv 2024

  14. [14]

    Large language models are zero-shot reasoners

    Takeshi Kojima, Shixiang (Shane) Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. InAdvances in Neural Information Pro- cessing Systems, pages 22199–22213. Curran Associates, Inc.,

  15. [15]

    Improved baselines with visual instruction tuning, 2023

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning, 2023. 1, 2

    2023

  16. [16]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. InNeurIPS, 2023. 1, 2, 4, 5, 10

    2023

  17. [17]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26296–26306, 2024. 2

    2024

  18. [18]

    Learn to explain: Multimodal reasoning via thought chains for science question answering.Advances in Neural Information Processing Systems, 35:2507–2521, 2022

    Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. Learn to explain: Multimodal reasoning via thought chains for science question answering.Advances in Neural Information Processing Systems, 35:2507–2521, 2022. 1, 2

    2022

  19. [19]

    Faithful chain-of-thought reasoning

    Qing Lyu, Shreya Havaldar, Adam Stein, Li Zhang, Delip Rao, Eric Wong, Marianna Apidianaki, and Chris Callison-Burch. Faithful chain-of-thought reasoning. InThe 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (IJCNLP-AACL 2023), 2023. 1

    2023

  20. [20]

    Compositional chain-of-thought prompting for large multimodal models

    Chancharik Mitra, Brandon Huang, Trevor Darrell, and Roei Herzig. Compositional chain-of-thought prompting for large multimodal models. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 14420–14431, 2024. 1, 2

    2024

  21. [21]

    Reasoning with large language models, a survey.arXiv preprint arXiv:2407.11511,

    Aske Plaat, Annie Wong, Suzan Verberne, Joost Broekens, Niki van Stein, and Thomas Back. Reasoning with large language models, a survey.arXiv preprint arXiv:2407.11511,

    arXiv

  22. [22]

    A-okvqa: A bench- mark for visual question answering using world knowledge

    Dustin Schwenk, Apoorv Khandelwal, Christopher Clark, Kenneth Marino, and Roozbeh Mottaghi. A-okvqa: A bench- mark for visual question answering using world knowledge. InEuropean conference on computer vision, pages 146–162. Springer, 2022. 1

    2022

  23. [23]

    Visual cot: Advancing multi-modal language models with a comprehen- sive dataset and benchmark for chain-of-thought reasoning

    Hao Shao, Shengju Qian, Han Xiao, Guanglu Song, Zhuofan Zong, Letian Wang, Yu Liu, and Hongsheng Li. Visual cot: Advancing multi-modal language models with a comprehen- sive dataset and benchmark for chain-of-thought reasoning. Advances in Neural Information Processing Systems, 37:8612– 8642, 2024. 2

    2024

  24. [24]

    Gemini: a family of highly capable multimodal models.arXiv preprint arXiv:2312.11805, 2023

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean- Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models.arXiv preprint arXiv:2312.11805, 2023. 1

    Pith/arXiv arXiv 2023

  25. [25]

    Qwen2.5-vl, 2025

    Qwen Team. Qwen2.5-vl, 2025. 4, 5

    2025

  26. [26]

    Llamav- o1: Rethinking step-by-step visual reasoning in llms.arXiv preprint arXiv:2501.06186, 2025

    Omkar Thawakar, Dinura Dissanayake, Ketan More, Ritesh Thawkar, Ahmed Heakl, Noor Ahsan, Yuhao Li, Mohammed Zumri, Jean Lahoud, Rao Muhammad Anwer, et al. Llamav- o1: Rethinking step-by-step visual reasoning in llms.arXiv preprint arXiv:2501.06186, 2025. 2

    Pith/arXiv arXiv 2025

  27. [27]

    Chain-of- thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of- thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824– 24837, 2022. 1

    2022

  28. [28]

    Boosting multimodal reasoning with mcts-automated structured thinking.arXiv preprint arXiv:2502.02339, 2025

    Jinyang Wu, Mingkuan Feng, Shuai Zhang, Ruihan Jin, Feihu Che, Zengqi Wen, and Jianhua Tao. Boosting multimodal reasoning with mcts-automated structured thinking.arXiv preprint arXiv:2502.02339, 2025. 1

    arXiv 2025

  29. [29]

    Llava-cot: Let vision language models reason step-by-step.URL https://arxiv

    Guowei Xu, Peng Jin, Li Hao, Yibing Song, Lichao Sun, and Li Yuan. Llava-cot: Let vision language models reason step-by-step.URL https://arxiv. org/abs/2411.10440, 2024. 1, 2

    Pith/arXiv arXiv 2024

  30. [30]

    Llava-cot: Let vision language models reason step-by-step, 2024

    Guowei Xu, Peng Jin, Hao Li, Yibing Song, Lichao Sun, and Li Yuan. Llava-cot: Let vision language models reason step-by-step, 2024. 2

    2024

  31. [31]

    Mulberry: Empowering mllm with o1- like reasoning and reflection via collective monte carlo tree search.arXiv preprint arXiv:2412.18319, 2024

    Huanjin Yao, Jiaxing Huang, Wenhao Wu, Jingyi Zhang, Yibo Wang, Shunyu Liu, Yingjie Wang, Yuxin Song, Haocheng Feng, Li Shen, et al. Mulberry: Empowering mllm with o1- like reasoning and reflection via collective monte carlo tree search.arXiv preprint arXiv:2412.18319, 2024. 2

    Pith/arXiv arXiv 2024

  32. [32]

    Tree of thoughts: Deliberate problem solving with large language models.Ad- vances in neural information processing systems, 36:11809– 11822, 2023

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Grif- fiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models.Ad- vances in neural information processing systems, 36:11809– 11822, 2023. 1

    2023

  33. [33]

    Self- rewarding language models, 2025

    Weizhe Yuan, Richard Yuanzhe Pang, Kyunghyun Cho, Xian Li, Sainbayar Sukhbaatar, Jing Xu, and Jason Weston. Self- rewarding language models, 2025. 2

    2025

  34. [34]

    Scaling relationship on learning mathematical reasoning with large language models.arXiv preprint arXiv:2308.01825,

    Zheng Yuan, Hongyi Yuan, Chengpeng Li, Guanting Dong, Keming Lu, Chuanqi Tan, Chang Zhou, and Jingren Zhou. Scaling relationship on learning mathematical reasoning with large language models.arXiv preprint arXiv:2308.01825,

    Pith/arXiv arXiv

  35. [35]

    Mmmu: A massive multi-discipline multi- modal understanding and reasoning benchmark for expert agi

    Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al. Mmmu: A massive multi-discipline multi- modal understanding and reasoning benchmark for expert agi. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9556–9567, 2024. 1

    2024

  36. [36]

    STar: Bootstrapping reasoning with reasoning

    Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah Goodman. STar: Bootstrapping reasoning with reasoning. InAdvances in Neural Information Processing Systems, 2022. 1, 2, 3, 5

    2022

  37. [37]

    Llm as a mastermind: A survey of strate- gic reasoning with large language models.arXiv preprint arXiv:2404.01230, 2024

    Yadong Zhang, Shaoguang Mao, Tao Ge, Xun Wang, Adrian de Wynter, Yan Xia, Wenshan Wu, Ting Song, Man Lan, and Furu Wei. Llm as a mastermind: A survey of strate- gic reasoning with large language models.arXiv preprint arXiv:2404.01230, 2024. 1

    Pith/arXiv arXiv 2024

  38. [38]

    Multimodal chain-of-thought rea- soning in language models.arXiv preprint arXiv:2302.00923,

    Zhuosheng Zhang, Aston Zhang, Mu Li, Hai Zhao, George Karypis, and Alex Smola. Multimodal chain-of-thought rea- soning in language models.arXiv preprint arXiv:2302.00923,

    Pith/arXiv arXiv

  39. [39]

    Let’s think step by step

    Orr Zohar, Xiaohan Wang, Yonatan Bitton, Idan Szpektor, and Serena Yeung-Levy. Video-star: Self-training enables video instruction tuning with any supervision.arXiv preprint arXiv:2407.06189, 2024. 2 See, Think, Learn: A Self-Taught Multimodal Reasoner Supplementary Materials A. Visualization Figure 7 illustrate the test set responses of domain-specific m...

    Pith/arXiv arXiv 2024

  40. [2020]

    Software available from wandb.com. 10