Critique Before Thinking: Mitigating Hallucination through Rationale-Augmented Instruction Tuning

Dayan Wu; Dian Li; Gang Liu; Weiping Wang; Zexian Yang

REVIEW 5 major objections 7 minor 34 references

Critique Before Thinking: Mitigating Hallucination through Rationale-Augmented Instruction Tuning

T0 review · 5 major / 7 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Re-Critic inserts reasoning rationales into visual questions before the answer, reducing hallucination while improving general multimodal reasoning.

desk verdict Re-Critic's rationale-in-the-question twist and self-critic DPO show consistent gains, but the missing generic-insertion control and lack of seeds keep this at promising-preliminary rather than established. read the letter →

arxiv 2505.07172 v1 pith:CWNVAYXO submitted 2025-05-12 cs.CV

Zexian Yang , Dian Li , Dayan Wu , Gang Liu , Weiping Wang This is my paper · ORCID

classification cs.CV

keywords hallucinationlargevision-languagemodelsinstructiontuningrationaleaugmentationchain-of-thoughtself-criticdirectpreferenceoptimizationmultimodalreasoning

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that a major cause of hallucination in large vision-language models is missing reasoning context: the model sees a question and an answer but never learns the judgment basis that connects them to the image. To fix that, Re-Critic inserts a machine-generated rationale—a short statement of the relevant principles plus a chain of ideas—into the question before answering, then fine-tunes the model on these augmented instructions. A second stage uses the model's own critique of two of its candidate responses to build preference pairs for DPO, avoiding any external reward model. The experiments report that this recipe improves hallucination benchmarks by 6.2% on average for LLaVA-v1.5 and 2.7% for InternVL2, while also lifting scores on general multimodal reasoning benchmarks. If correct, this makes rationale augmentation a cheap, scalable alternative to curating larger datasets.

What carries the argument

The central mechanism is the visual chain insertion technique (VCIT), which rewrites each training question by inserting a generated rationale—a statement of the underlying judgment basis and a brief chain of ideas—before the answer is expected. This turns the rationale into part of the input context rather than part of the output, so the model is trained to reason from the rationale first. The second mechanism is the self-critic preference-learning loop, where the currently optimized LVLM evaluates two of its own responses under explicit criteria (image-content understanding, contextual reasoning) and the preferred pair is used for Direct Preference Optimization (DPO). Together they let the model learn methodology before answering.

What would settle it

Retrain Re-Critic with rationale texts that are deliberately scrambled or describe a different image while preserving the same format and training pipeline, then compare hallucination and reasoning benchmarks: if performance stays about the same, the gains are not caused by the rationale content itself.

Watch

Extended reading notes

Core claim

Re-Critic claims that hallucinations in LVLMs stem from insufficient context reasoning, and that inserting rationale guidance between question and answer during instruction tuning teaches the model to reason before responding. Concretely, the visual chain insertion technique (VCIT) prompts a proprietary LVLM to generate a "basic judgment basis" and a brief chain of ideas from each raw question-answer pair, and places that rationale inside the question input. Then an in-context self-critic mechanism makes the tuned model rank two of its own responses using criteria of image-content understanding and comprehensive contextual reasoning, turning the winner-loser pair into DPO training data. The paper reports consistent gains: average hallucination-benchmark improvements of 6.2% for LLaVA-v1.5 and 2.7% for InternVL2, plus gains on MME, MathVista, LLaVA-Bench, and OCRBench, and larger relative gains when training on the smaller LLaVA-Instruct-80K set.

Load-bearing premise

The approach assumes the GPT-4o-generated rationales are factually accurate and grounded in the image, yet the paper does not audit, filter, or verify them.

Editorial extensions

If this is right

Mixing rationale-augmented instructions into standard SFT data improves hallucination benchmarks without replacing the original dataset.
Self-critic preference learning produces DPO pairs from the model's own judgments, avoiding distribution shift from third-party feedback.
Data efficiency: with only 10k augmented instructions in LLaVA-Instruct-80K, Re-Critic yields large gains on MME and POPE.
Gains transfer to general reasoning: improvements on MathVista, MME, LLaVA-Bench, and OCRBench across two backbones.
The recipe is architecture-agnostic, showing gains on LLaVA-v1.5, InternVL2, and MiniGPT-4.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

We infer that the method's ceiling is set by rationale quality: because nothing checks the teacher's reasoning, any systematic factual error in the generated rationales would be imprinted into the fine-tuned model.
We infer that the self-critic step inherits the base model's blind spots; a model that cannot see its own mistakes may rank a plausible hallucination over a correct answer, so gains should depend on base capability.
We infer the recipe is portable: inserting rationales into prompts is not tied to vision data and could be applied to any instruction-tuning corpus where a teacher can generate a judgment basis.
A direct test of causality is to corrupt a random fraction of rationales and measure the drop on POPE and MMHalBench; a monotone drop would confirm the rationale content drives the effect.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Re-Critic is a cheap, scalable recipe: take a small slice of your instruction data, have GPT-4o write a short 'judgment basis' rationale grounded in the image and gold answer, insert that rationale into the question rather than the answer, and fine-tune. Then use the fine-tuned model's own self-critic to pick preferred vs dispreferred responses for DPO. The result is consistent, modest gains on hallucination benchmarks and some general reasoning benchmarks over LLaVA-v1.5-7B and InternVL2-2B.

The genuinely new things are the insertion into the question (VCIT) rather than appending to the answer, and the self-critic pair selection with no external reward model. That combination is worth taking seriously. The ablations in Table 4 show both components contribute, and the MiniGPT-4 result gives some generality. The gains on MathVista and LLaVA-Bench suggest the effect is not just on hallucination metrics.

The main soft spot is the missing control for rationale content. VCIT bundles two changes at once: adding extra instructive text to the prompt, and grounding that text in the image. Without a control that inserts a generic, image-independent rationale ('read the image carefully, reason step by step') into the same subset, you cannot attribute the gains to the image-grounded content. The paper's central claim of 'methodological learning before reasoning' is under-supported without that control. This is a real weakness, though not a fatal one; the evidence is at least consistent with the rationale content mattering.

Other issues: no error bars or multiple seeds, and several baseline numbers are taken from prior papers without re-running, so the deltas are not apples-to-apples. The 'average improvement' mixes higher-better and lower-better metrics, which is sloppy. The self-critic DPO is self-referential—the same model labels its own pairs—which could reinforce biases, but it's a known bootstrap risk, not a demonstrated failure. And there's no audit of the GPT-4o rationales; if they contain errors, training on them could reinforce flawed reasoning.

Who is this for? Practitioners working on hallucination mitigation and instruction data augmentation for LVLMs. It's a legitimate empirical contribution, but it needs revision before it can be trusted as a strong claim. I'd send it to peer review rather than desk reject, but I'd make the revision conditional on adding a generic-insertion control, reporting seeds/error bars, and releasing the data and code. If those hold, the method is a useful addition to the toolbox.

Referee Report

5 major / 7 minor

Summary. The paper proposes Re-Critic, a two-stage framework for reducing hallucination in large vision-language models (LVLMs). The first stage, VCIT, augments raw instruction-tuning samples with GPT-4o-generated rationales that are inserted into the question text; the second stage builds preference pairs by having the fine-tuned model critique its own two responses and applies DPO. Experiments on LLaVA-v1.5-7B, InternVL2-2B, and MiniGPT-4 report gains on hallucination benchmarks (POPE, MMHalBench, HallusionBench, Object HalBench) and on several general multimodal benchmarks (MME, MathVista, LLaVA-Bench, MMBench, RWQA, GQA, OCRBench). Ablations in Table 4 attribute improvements to both VCIT and the self-critic DPO.

Significance. If the claims hold, Re-Critic is a simple and scalable data-augmentation recipe that improves both hallucination metrics and general reasoning without external reward models or human preference annotation. The paper includes ablations, multiple backbone models, and a small-data regime analysis, and it provides a concrete recipe that other groups could reproduce. However, the significance is currently limited by the absence of a control that isolates the content of the inserted rationale from the mere fact of insertion, by the lack of any statistical uncertainty quantification, and by the self-referential nature of the preference-pair construction.

major comments (5)

[§3.1 / Table 4] The central claim that rationale-augmented instruction tuning drives the gains is not supported because the VCIT condition confounds the content of the inserted rationale with the presence of extra instructive text. Table 4 compares VCIT on/off, but an arm that inserts a generic, image-independent instruction (e.g., "Read the image carefully and reason step by step before answering") or rationales generated without access to the image is missing. Without such a control, the improvements could be caused by longer prompts or pedagogical framing alone, not by the image-grounded rationale content that the paper emphasizes.
[§4.2, Tables 2 and 3] The reported "average improvement" is not defined and mixes higher-better and lower-better metrics. For example, in Table 2, HallusionBench drops from 59.4 to 56.0 for LLaVA-v1.5 and from 56.0 to 55.0 for InternVL2, and in Table 3 MME drops by 3.4 points and InternVL2 MMBench drops by 0.1 points, yet the text states consistent average improvements of 6.2%, 2.7%, 2.8%, and 3.7%. The paper must state the direction of each metric, how percentage changes are computed (e.g., relative to the base, with lower-better metrics inverted), and which benchmarks are included in each average; otherwise the headline claim is misleading.
[§4.2 / Tables 2 and 3] No error bars, confidence intervals, or multiple seeds are provided. Many reported deltas are small (e.g., POPE +0.6, MMHal-B +0.14 for LLaVA-v1.5, MMBench -0.1 for InternVL2). Without estimates of variance, the paper cannot establish that these differences are statistically meaningful, especially on binary or score-based benchmarks where a single-seed run can differ by this amount.
[§3.2 / Table 4] The self-critic mechanism is self-referential: the same model that is being optimized generates the two response candidates and also evaluates them using its own judgment. The paper argues that this avoids distribution shift, but it also introduces a risk that the model simply prefers outputs that match its own style, independent of factual correctness. No validation of the preference pairs is provided (e.g., human agreement, comparison with an external critic, or a random-pair baseline). Table 4 shows that self-critic alone improves LLaVA-Bench by 6.7 points, which is larger than the VCIT-only gain and suggests that generic DPO effects may be at play.
[§4.1 / Baselines] The comparison with prior methods (LLaVA-RLHF, HA-DPO, POVID) relies on numbers taken from previous papers without re-evaluation under a single protocol. Since Object HalBench evaluation typically depends on the prompt set and decoding settings, and MMHal-Bench relies on a GPT-4 judge, the baseline values may not be comparable to the runs of Re-Critic. The paper should either re-run all baselines under the same protocol or clearly state the source and the exact evaluation settings for each reported number.

minor comments (7)

[§3.2] The section title contains a typo: "Prefernece" should be "Preference".
[Table 2] The caption reads "on on four hallucination benchmarks"; remove the duplicated "on".
[Figure 4] The label "T extQA" should be "TextQA".
[§4.4 / Table 5] The "Base Model" row in Table 5 reports OCRBench 802, whereas Table 3 reports 784 for the same InternVL2 base model. This inconsistency should be resolved.
[§4.2 / Table 3] The text claims an average improvement of 2.8% on general benchmarks for LLaVA-v1.5, but Table 3 shows a decrease on MME (-3.4). The definition of the average and the direction of each metric must be clarified before the claim can be assessed.
[§4.4 / Table 5] The hard-sample sampling strategy (BERT-based clustering, top-k selection, difficulty sorting) is described only briefly. The paper should provide details (number of clusters, k, how difficulty accuracy was measured, and whether the selection was done on the training set or a held-out set) so that the experiment is reproducible.
[§4.3 / Figure 3] The small-data experiment uses LLaVA-Instruct-80K, but it is unclear whether the base model is LLaVA-v1 or LLaVA-v1.5. State the exact base model and training recipe for this ablation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central claims are evaluated on external benchmarks, and the self-critic mechanism is a training-data construction rather than a self-derived prediction.

full rationale

The paper's derivation chain is not circular. The central empirical claim—that Re-Critic improves hallucination and general multimodal benchmarks—is tested against external benchmarks (POPE, MMHalBench, HallusionBench, Object HalBench, MME, MathVista, LLaVA-Bench, MMBench, RWQA, GQA, OCRBench) with numbers reported in Tables 2 and 3. The rationale-augmented training data is produced by GPT-4o from questions, images, and gold answers, and the DPO preference pairs are generated and labeled by the current model itself; however, neither of these steps defines the evaluation metric in terms of the training signal. The self-critic pipeline is a training-data construction method, not a prediction derived from itself; any bias it introduces would be an experimental weakness, not a circular reduction. There is no self-citation load-bearing argument, no imported uniqueness theorem, and no renaming of known results. The absence of a generic-rationale control is a legitimate experimental-design criticism, but it does not make the reported gains equivalent to the inputs by construction. Therefore no circularity is identified.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central empirical claims rest on unverified teacher-generated rationales, self-generated preference labels, and an unstated inference-time protocol; the reported gains are not backed by released artifacts.

free parameters (4)

Augmentation ratio = 10% (e.g., 10k of 665k; 10% of InternVL data)
Chosen by hand; no sensitivity analysis across ratios except the 80K data experiment.
DPO coefficient beta
Appears in Eq. (3) but its value and any tuning are not reported.
Hard-sample top-k and difficulty sorting
Section 4.4 describes cluster-based selection but does not specify cluster count, k, or difficulty metric.
Self-critic generation parameters
Temperature and number of samples for generating the two response candidates are not reported.

assumptions (3)

domain assumption GPT-4o-generated rationales are factually grounded in the image and pedagogically useful
Used to create all augmented training samples in Section 3.1; no human or automated verification of rationale quality.
domain assumption The model's self-critique reliably identifies the better response
Section 3.2 builds DPO pairs from the current LVLM's own judgments; no external validation or agreement analysis.
domain assumption Training with rationale-inserted inputs transfers to test-time inputs without rationales
Benchmarks use standard prompts; the paper does not state whether rationales are inserted at inference, implying a distribution shift that is assumed benign.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Critique Before Thinking: Mitigating Hallucination through Rationale-Augmented Instruction Tuning." pith.science (2026). https://pith.science/paper/CWNVAYXO

@misc{pith2026250507172,
  author       = {Pith},
  title        = {Pith review of: Critique Before Thinking: Mitigating Hallucination through Rationale-Augmented Instruction Tuning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CWNVAYXO}},
  note         = {Machine review of arXiv:2505.07172}
}

read the original abstract

Despite significant advancements in multimodal reasoning tasks, existing Large Vision-Language Models (LVLMs) are prone to producing visually ungrounded responses when interpreting associated images. In contrast, when humans embark on learning new knowledge, they often rely on a set of fundamental pre-study principles: reviewing outlines to grasp core concepts, summarizing key points to guide their focus and enhance understanding. However, such preparatory actions are notably absent in the current instruction tuning processes. This paper presents Re-Critic, an easily scalable rationale-augmented framework designed to incorporate fundamental rules and chain-of-thought (CoT) as a bridge to enhance reasoning abilities. Specifically, Re-Critic develops a visual rationale synthesizer that scalably augments raw instructions with rationale explanation. To probe more contextually grounded responses, Re-Critic employs an in-context self-critic mechanism to select response pairs for preference tuning. Experiments demonstrate that models fine-tuned with our rationale-augmented dataset yield gains that extend beyond hallucination-specific tasks to broader multimodal reasoning tasks.

Figures

Figures reproduced from arXiv: 2505.07172 by the authors.

**Figure 2.** Overview of Re-Critic. The overall process is divided into three parts: 1) augmented standard QA with synthesized visual rationales, then 2) fine-tuning LVLM with rewritten QA, and finally 3) perform DPO through the self-critic preference learning. primarily applied at test time and rely heavily on the inherent capabilities of the LLMs. This dependency may limit the broader applicability and scalability of these ap… view at source ↗

**Figure 3.** Ablation study on the effectiveness of using the alternative base model MiniGPT4 (left) and smaller scale of training data (right). [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Performance comparison of different models on six gen [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

34 extracted references · 9 canonical work pages

[1]

Gpt-4 technical report

[Achiam et al., 2023] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Alt- man, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774,

arXiv 2023
[3]

An aug- mented benchmark dataset for geometric question answer- ing through dual parallel text encoding

[Cao and Xiao, 2022] Jie Cao and Jing Xiao. An aug- mented benchmark dataset for geometric question answer- ing through dual parallel text encoding. In Proceedings of the 29th International Conference on Computational Lin- guistics, pages 1511–1520,

work page 2022
[6]

Mme: A comprehensive evaluation benchmark for multimodal large language mod- els

[Fu et al., 2023] Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, et al. Mme: A comprehensive evaluation benchmark for multimodal large language mod- els. arXiv preprint arXiv:2306.13394,

arXiv 2023
[7]

Hallusion- bench: an advanced diagnostic suite for entangled lan- guage hallucination and visual illusion in large vision- language models

[Guan et al., 2024] Tianrui Guan, Fuxiao Liu, Xiyang Wu, Ruiqi Xian, Zongxia Li, Xiaoyu Liu, Xijun Wang, Lichang Chen, Furong Huang, Yaser Yacoob, et al. Hallusion- bench: an advanced diagnostic suite for entangled lan- guage hallucination and visual illusion in large vision- language models. In Proceedings of the IEEE/CVF Con- ference on Computer Vision ...

work page 2024
[8]

Detecting and preventing hallucinations in large vi- sion language models

[Gunjal et al., 2024] Anisha Gunjal, Jihan Yin, and Erhan Bas. Detecting and preventing hallucinations in large vi- sion language models. In Proceedings of the AAAI Confer- ence on Artificial Intelligence, pages 18135–18143,

work page 2024
[9]

Mini- monkey: Multi-scale adaptive cropping for multimodal large language models

[Huang et al., 2024] Mingxin Huang, Yuliang Liu, Dingkang Liang, Lianwen Jin, and Xiang Bai. Mini- monkey: Multi-scale adaptive cropping for multimodal large language models. CoRR,

work page 2024
[10]

Gqa: A new dataset for real-world visual reasoning and compositional question answering

[Hudson and Manning, 2019] Drew A Hudson and Christo- pher D Manning. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6700–6709,

2019
[12]

Faithscore: Evaluating hallucinations in large vision-language models

[Jing et al., 2023] Liqiang Jing, Ruosen Li, Yunmo Chen, Mengzhao Jia, and Xinya Du. Faithscore: Evaluating hallucinations in large vision-language models. arXiv preprint arXiv:2311.01477,

arXiv 2023

Show all 34 references

[13]

Dvqa: Understanding data visu- alizations via question answering

[Kafle et al., 2018] Kushal Kafle, Brian Price, Scott Cohen, and Christopher Kanan. Dvqa: Understanding data visu- alizations via question answering. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 5648–5656,

2018
[16]

On the hidden mystery of ocr in large multi- modal models

[Liu et al., 2023b] Yuliang Liu, Zhang Li, Biao Yang, Chun- yuan Li, Xucheng Yin, Cheng-lin Liu, Lianwen Jin, and Xiang Bai. On the hidden mystery of ocr in large multi- modal models. arXiv preprint arXiv:2305.07895,

arXiv
[17]

A survey on hallucina- tion in large vision-language models

[Liu et al., 2024a] Hanchao Liu, Wenyuan Xue, Yifei Chen, Dapeng Chen, Xiutian Zhao, Ke Wang, Liping Hou, Rongjun Li, and Wei Peng. A survey on hallucina- tion in large vision-language models. arXiv preprint arXiv:2402.00253,

arXiv
[18]

Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts.arXiv preprint arXiv:2310.02255,

[Lu et al., 2023] Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai- Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts.arXiv preprint arXiv:2310.02255,

2023 arXiv
[19]

Deepseek-vl: towards real-world vision-language understanding

[Lu et al., 2024] Haoyu Lu, Wen Liu, Bo Zhang, Bingxuan Wang, Kai Dong, Bo Liu, Jingxiang Sun, Tongzheng Ren, Zhuoshu Li, Hao Yang, et al. Deepseek-vl: towards real-world vision-language understanding. arXiv preprint arXiv:2403.05525,

2024 arXiv
[20]

Chartqa: A bench- mark for question answering about charts with visual and logical reasoning

[Masry et al., 2022] Ahmed Masry, Do Xuan Long, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. Chartqa: A bench- mark for question answering about charts with visual and logical reasoning. arXiv preprint arXiv:2203.10244,

2022 arXiv
[23]

Direct preference optimization: Your lan- guage model is secretly a reward model.Advances in Neu- ral Information Processing Systems,

[Rafailov et al., 2024] Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your lan- guage model is secretly a reward model.Advances in Neu- ral Information Processing Systems,

2024
[24]

Object hallucination in image captioning

[Rohrbach et al., 2018] Anna Rohrbach, Lisa Anne Hen- dricks, Kaylee Burns, Trevor Darrell, and Kate Saenko. Object hallucination in image captioning. arXiv preprint arXiv:1809.02156,

2018 arXiv
[25]

Aligning large multimodal models with factually aug- mented rlhf

[Sun et al., 2023] Zhiqing Sun, Sheng Shen, Shengcao Cao, Haotian Liu, Chunyuan Li, Yikang Shen, Chuang Gan, Liang-Yan Gui, Yu-Xiong Wang, Yiming Yang, et al. Aligning large multimodal models with factually aug- mented rlhf. arXiv preprint arXiv:2309.14525,

2023 arXiv
[26]

Gemini 1.5: Unlocking multimodal understand- ing across millions of tokens of context

[Team et al., 2024] Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. Gemini 1.5: Unlocking multimodal understand- ing across millions of tokens of context. arXiv preprint arXiv:2403.05530,

2024 arXiv
[27]

To see is to believe: Prompting gpt-4v for better visual instruction tuning

[Wang et al., 2023] Junke Wang, Lingchen Meng, Zejia Weng, Bo He, Zuxuan Wu, and Yu-Gang Jiang. To see is to believe: Prompting gpt-4v for better visual instruction tuning. arXiv preprint arXiv:2311.07574,

2023 arXiv
[28]

Chain-of-thought prompting elicits rea- soning in large language models

[Wei et al., 2022] Jason Wei, Xuezhi Wang, Dale Schuur- mans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits rea- soning in large language models. Advances in neural in- formation processing systems, pages 24824–24837,

2022
[29]

Mmbench: Bench- marking end-to-end multi-modal dnns and understand- ing their hardware-software implications

[Xu et al., 2023] Cheng Xu, Xiaofeng Hou, Jiacheng Liu, Chao Li, Tianhao Huang, Xiaozhi Zhu, Mo Niu, Lingyu Sun, Peng Tang, Tongqiao Xu, et al. Mmbench: Bench- marking end-to-end multi-modal dnns and understand- ing their hardware-software implications. In 2023 IEEE Internatio...

2023
[30]

Minicpm-v: A gpt-4v level mllm on your phone

[Yao et al., 2024] Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, et al. Minicpm-v: A gpt-4v level mllm on your phone. arXiv preprint arXiv:2408.01800 ,

2024 arXiv
[31]

Rlaif-v: Aligning mllms through open-source ai feedback for super gpt-4v trustworthiness

[Yu et al., 2024b] Tianyu Yu, Haoye Zhang, Yuan Yao, Yunkai Dang, Da Chen, Xiaoman Lu, Ganqu Cui, Taiwen He, Zhiyuan Liu, Tat-Seng Chua, et al. Rlaif-v: Aligning mllms through open-source ai feedback for super gpt-4v trustworthiness. arXiv preprint arXiv:2405.17220,
[32]

Beyond hallucinations: Enhancing lvlms through hallucination- aware direct preference optimization

[Zhao et al., 2023] Zhiyuan Zhao, Bin Wang, Linke Ouyang, Xiaoyi Dong, Jiaqi Wang, and Conghui He. Beyond hallucinations: Enhancing lvlms through hallucination- aware direct preference optimization. arXiv preprint arXiv:2311.16839,

2023 arXiv
[33]

Aligning modal- ities in vision large language models via preference fine- tuning

[Zhou et al., 2024] Yiyang Zhou, Chenhang Cui, Rafael Rafailov, Chelsea Finn, and Huaxiu Yao. Aligning modal- ities in vision large language models via preference fine- tuning. arXiv preprint arXiv:2402.11411,

2024 arXiv
[34]

Minigpt-4: Enhancing vision-language understanding with advanced large lan- guage models

[Zhu et al., 2023] Deyao Zhu, Jun Chen, Xiaoqian Shen, Xi- ang Li, and Mohamed Elhoseiny. Minigpt-4: Enhancing vision-language understanding with advanced large lan- guage models. arXiv preprint arXiv:2304.10592, 2023

2023 arXiv
[2016]

Evaluating ob- ject hallucination in large vision-language models

[Li et al., 2023] Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. Evaluating ob- ject hallucination in large vision-language models. arXiv preprint arXiv:2305.10355,

2023 arXiv
[2018]

A diagram is worth a dozen images

[Kembhavi et al., 2016] Aniruddha Kembhavi, Mike Sal- vato, Eric Kolve, Minjoon Seo, Hannaneh Hajishirzi, and Ali Farhadi. A diagram is worth a dozen images. In Computer Vision–ECCV 2016: 14th European Confer- ence, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings,...

2016
[2019]

Forward-backward reasoning in large language models for mathematical verification

[Jiang et al., 2024] Weisen Jiang, Han Shi, Longhui Yu, Zhengying Liu, Yu Zhang, Zhenguo Li, and James Kwok. Forward-backward reasoning in large language models for mathematical verification. In Findings of the Association for Computational Linguistics ACL 2024 , pages 6647– 6661,

2024
[2021]

Reasoning with language model prompting: A survey

[Qiao et al., 2022] Shuofei Qiao, Yixin Ou, Ningyu Zhang, Xiang Chen, Yunzhi Yao, Shumin Deng, Chuanqi Tan, Fei Huang, and Huajun Chen. Reasoning with language model prompting: A survey. arXiv preprint arXiv:2212.09597 ,

2022 arXiv
[2022]

Docvqa: A dataset for vqa on document images

[Mathew et al., 2021] Minesh Mathew, Dimosthenis Karatzas, and CV Jawahar. Docvqa: A dataset for vqa on document images. In Proceedings of the IEEE/CVF winter conference on applications of computer vision , pages 2200–2209,

2021
[2023]

Qwen-vl: A frontier large vision- language model with versatile abilities

[Bai et al., 2023] Jinze Bai, Shuai Bai, Shusheng Yang, Shi- jie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A frontier large vision- language model with versatile abilities. arXiv preprint arXiv:2308.12966,

2023 arXiv
[2024]

Sharegpt4v: Improving large multi-modal models with better captions

[Chen et al., 2025] Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Conghui He, Jiaqi Wang, Feng Zhao, and Dahua Lin. Sharegpt4v: Improving large multi-modal models with better captions. In European Conference on Com- puter Vision, pages 370–387. Springer,

2025
[2025]

Active prompting with chain-of-thought for large language mod- els

[Diao et al., 2023] Shizhe Diao, Pengcheng Wang, Yong Lin, Rui Pan, Xiang Liu, and Tong Zhang. Active prompting with chain-of-thought for large language mod- els. arXiv preprint arXiv:2302.12246,

2023 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

Gpt-4 technical report

[Achiam et al., 2023] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Alt- man, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774,

arXiv 2023

[2] [3]

An aug- mented benchmark dataset for geometric question answer- ing through dual parallel text encoding

[Cao and Xiao, 2022] Jie Cao and Jing Xiao. An aug- mented benchmark dataset for geometric question answer- ing through dual parallel text encoding. In Proceedings of the 29th International Conference on Computational Lin- guistics, pages 1511–1520,

work page 2022

[3] [6]

Mme: A comprehensive evaluation benchmark for multimodal large language mod- els

[Fu et al., 2023] Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, et al. Mme: A comprehensive evaluation benchmark for multimodal large language mod- els. arXiv preprint arXiv:2306.13394,

arXiv 2023

[4] [7]

Hallusion- bench: an advanced diagnostic suite for entangled lan- guage hallucination and visual illusion in large vision- language models

[Guan et al., 2024] Tianrui Guan, Fuxiao Liu, Xiyang Wu, Ruiqi Xian, Zongxia Li, Xiaoyu Liu, Xijun Wang, Lichang Chen, Furong Huang, Yaser Yacoob, et al. Hallusion- bench: an advanced diagnostic suite for entangled lan- guage hallucination and visual illusion in large vision- language models. In Proceedings of the IEEE/CVF Con- ference on Computer Vision ...

work page 2024

[5] [8]

Detecting and preventing hallucinations in large vi- sion language models

[Gunjal et al., 2024] Anisha Gunjal, Jihan Yin, and Erhan Bas. Detecting and preventing hallucinations in large vi- sion language models. In Proceedings of the AAAI Confer- ence on Artificial Intelligence, pages 18135–18143,

work page 2024

[6] [9]

Mini- monkey: Multi-scale adaptive cropping for multimodal large language models

[Huang et al., 2024] Mingxin Huang, Yuliang Liu, Dingkang Liang, Lianwen Jin, and Xiang Bai. Mini- monkey: Multi-scale adaptive cropping for multimodal large language models. CoRR,

work page 2024

[7] [10]

Gqa: A new dataset for real-world visual reasoning and compositional question answering

[Hudson and Manning, 2019] Drew A Hudson and Christo- pher D Manning. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6700–6709,

2019

[8] [12]

Faithscore: Evaluating hallucinations in large vision-language models

[Jing et al., 2023] Liqiang Jing, Ruosen Li, Yunmo Chen, Mengzhao Jia, and Xinya Du. Faithscore: Evaluating hallucinations in large vision-language models. arXiv preprint arXiv:2311.01477,

arXiv 2023