Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

MIRAGE: Assessing Hallucination in Multimodal Reasoning Chains of MLLM

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MIRAGE isolates reasoning hallucinations in vision-language models by keeping only questions where the image is perceived correctly yet reasoning still fails.

desk verdict A useful benchmark for reasoning-chain hallucinations, but the central isolation claim rests on a perception filter that does not actually cover the evaluated models. read the letter →

arxiv 2505.24238 v2 pith:IMI3GSBI submitted 2025-05-30 cs.CV cs.LG

Bowen Dong , Minheng Ni , Zitong Huang , Guanglei Yang , Wangmeng Zuo , Lei Zhang This is my paper
classification cs.CVcs.LG
keywords multimodalhallucinationreasoningbenchmarklargelanguagemodelschain-of-thoughtcurriculumreinforcementlearningevaluation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Multimodal models often give wrong answers for two distinct reasons: they misperceive the image, or they reason badly about what they saw. Existing benchmarks mix these sources. MIRAGE is a benchmark built from 1,329 questions deliberately filtered so that several open multimodal models describe the image correctly yet still fail to reason correctly, thus isolating reasoning-induced hallucinations. The paper also introduces three evaluation levels—final-answer accuracy, factuality of intermediate steps and claims, and a whole-chain hallucination score from LLM judges—and uses MIRAGE to show that scaling models and data reduces logical, fabrication, and factual errors but hardly touches spatial-reasoning errors. A proposed method, Logos, applies curriculum reinforcement fine-tuning with collaborative hint inference to reduce logical hallucinations and improve accuracy on MIRAGE and MathVista.

What carries the argument

The load-bearing object is MIRAGE's difficulty-curation filter. For each candidate question, three open-source MLLMs generate image descriptions; a secondary LLM verifies the descriptions as accurate; only questions where those models still commit frequent reasoning errors are retained. This filter operationalizes 'correctly perceived but wrongly reasoned' and is what separates reasoning-induced from perception-induced hallucination. The measurement machinery then has three tiers: answer accuracy, F1 factuality scores over extracted intermediate steps and claims, and the LLMs Hallucination Score (LHS), an ensemble over multiple judge LLMs and reference-chain rewrites. Logos is the training-side counterpart: it applies curriculum reinforcement fine-tuning (CRFT) that filters training questions by sampled reward so that advantages stay non-zero, plus online reward filtration, and at inference adds collaborative hint inference (CHI) in which an auxiliary LLM supplies topic- and question-specific hints.

What would settle it

If a direct perceptual probe of the models evaluated on MIRAGE (for instance, asking them to localize or redraw the key visual elements of the retained questions) reveals frequent perception errors on those same questions, the benchmark's claim to isolate reasoning hallucinations would be falsified.

Watch

Extended reading notes

Core claim

MIRAGE claims to be the first benchmark devoted to reasoning hallucinations in multimodal reasoning chains. Its construction pipeline asks three open-source MLLMs to describe each candidate image, keeps a secondary LLM's verification that the descriptions are accurate, and then keeps only questions where those models still reach wrong answers via faulty reasoning. The resulting dataset carries three-tier annotations: final answers, intermediate steps and claims, and full ground-truth reasoning chains, plus auxiliary image descriptions and hints. Evaluation uses accuracy, step/claim factuality F1 scores, and an LLMs Hallucination Score averaged over multiple judge LLMs and reference rewrites. The benchmark's empirical findings are that scale and training quality reduce logical, factual, and fabrication hallucinations, spatial hallucinations (misread spatial relations) barely improve with scale, and question types correlate with specific hallucination patterns. The paper further proposes Logos, which combines curriculum reinforcement fine-tuning with online reward filtration and collaborative hint inference, and reports that it lowers logical hallucinations and raises accuracy relative to the base model on both MIRAGE and MathVista.

Load-bearing premise

The benchmark assumes that if three open-source MLLMs describe an image consistently and a secondary LLM agrees, then the evaluated models truly perceive the image correctly, so any wrong answer must be a reasoning error.

Editorial extensions

If this is right

  • MIRAGE provides a way to measure reasoning-induced hallucination separately from perception-induced hallucination in multimodal models.
  • Scaling model size, data quality, and training stages reduces logical, fabrication, and factual hallucinations, but not spatial hallucinations caused by misinterpreted spatial relations.
  • Question types drive distinct hallucination patterns—logical and spatial errors cluster in logical-reasoning questions, while statistical and scientific questions show more factuality errors.
  • Logos, combining curriculum reinforcement fine-tuning with collaborative hint inference, reduces logical hallucinations and improves accuracy on MIRAGE and MathVista compared with the base model.
  • Manual correction of reasoning chains in a 10% sample raises answer accuracy to roughly 70%, showing that reasoning hallucination directly degrades final answers.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the isolation claim holds, MIRAGE-type filtering could be reused to audit other failure modes, such as knowledge-based vs. instruction-following errors, by holding perception constant.
  • The near-zero improvement in spatial hallucinations suggests that spatial reasoning may need architectural or training changes rather than more data; one could test this by adding spatial-targeted curricula and measuring MIRAGE spatial accuracy.
  • Because the curation filter relies on a small set of open-source models to certify perception, extending it to a broader, more diverse set of perceptually strong models—or to direct perception probes—would make the benchmark's separation claim more robust.
  • The strong correlation between logical hallucination and other hallucination types suggests that improving logical consistency may be a lever that also reduces fabrication and context errors, which Logos partially confirms.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces MIRAGE, a benchmark of 1,329 multimodal questions intended to isolate reasoning-induced hallucinations from perception-induced ones. Each question comes with a final answer, intermediate reasoning steps and claims, a human-verified reference reasoning chain, and auxiliary image descriptions and hints. Three evaluation metrics are proposed: answer accuracy, step/claim factuality (F_step, F_claim), and an LLM-based hallucination score (LHS). The authors report that scaling model size and pretraining data reduces logical, factual, and fabrication hallucinations but not spatial hallucinations, and that question type correlates with hallucination type. They also propose Logos, a baseline combining curriculum reinforcement fine-tuning (CRFT) with online reward filtration (ORF) and collaborative hint inference (CHI), and report improvements on MIRAGE and MathVista.

Significance. If the construction is valid, MIRAGE addresses a real gap: existing benchmarks such as POPE, MMVP, HallusionBench, and MathVista do not separate perception errors from reasoning errors in chain-of-thought evaluation. The multi-level annotations and the explicit five-type hallucination taxonomy are valuable diagnostic resources, and the paper ships prompts, cost estimates, and planned public release, which support reproducibility. The Logos baseline is a reasonable first attempt at reasoning-hallucination mitigation and its ablations are informative. However, the central claim that MIRAGE isolates reasoning hallucinations rests on an indirect perception filter and on LLM-based judging; the load-bearing validation of those components is currently incomplete, so the significance of the headline findings is conditional on the requested revisions.

major comments (4)
  1. [§3.1 (Data Curation)] The curation filter establishes correct perception only for the three open-source MLLMs used to generate descriptions (Qwen2.5-VL, InternVL-2.5, Llama-3.2-Vision), verified by DeepSeek-V3, and even then only that a coarse description is judged accurate. It does not establish correct perception for the black-box and 72B models in Table 2, nor does a generally accurate description guarantee that the specific visual quantity needed for the question was perceived correctly; the Fig. 18 statistical example illustrates the gap, since the auxiliary description reports Blue≈80 while the ground-truth reasoning uses Blue≈60. Because the benchmark's central claim is that MIRAGE isolates reasoning hallucinations from perception-induced ones, I ask for a direct perception probe administered to the evaluated models (e.g., extracting all task-relevant visual values and spatial relations and comparing with ground truth), and for reasoning metrics to be reported separately for questions where the probe confirms correct perception. Without this, the headline spatial-hallucination finding in Sec. 6.1 may confound perception failures with reasoning failures. The limitation statement in App. G does not acknowledge this gap.
  2. [§3.2 and Table 18] The automated annotation pipeline reaches 73.7% accuracy after DeepSeek-R1 refinement, and while a human-AI collaborative verification step is described, the paper does not report the final post-verification accuracy, the human correction rate, or inter-annotator agreement. Since F_step, F_claim, the LHS, and the five-type hallucination labels are all computed against these reference chains, any residual error in the chains directly bounds the reliability of the fine-grained metrics. Please report human verification statistics and a robustness check, for example recomputing the metrics after excluding questions whose reference chains required substantial human correction.
  3. [§4.2–§4.3 and Fig. 10–11] The hallucination-type detector and the LHS use LLM judges comparing predictions to LLM-generated reference chains; the only human validation reported is a 100-response LHS agreement rate with an average difference of 7.5% (Sec. 4.3). This does not validate the five-way hallucination-type classification or the step/claim factuality matching, and it leaves open the possibility that measured hallucination rates reflect failure modes shared by the judges and the evaluated models rather than genuine reasoning errors. Please provide per-type human agreement, a confusion analysis of the LLM judge against human labels, and a small human-verified subset on which all three metrics are reported.
  4. [§4.1 and §5.2] Several load-bearing thresholds are unspecified or unexamined: the relative-error threshold for approximate free-form answers (Sec. 4.1), the agreement threshold for deciding that image descriptions are 'consistently accurate' in the curation filter (Sec. 3.1), and the CRFT difficulty thresholds (mean accuracy >0 in the first stage and <0.5 in later stages, Sec. 5.2). Without reporting these values and a sensitivity analysis, the benchmark scores and the Logos gains in Table 8 cannot be fully reproduced or assessed for arbitrariness. Table 15 also shows that k=1 is chosen even though k=2 gives the same MathVista score and +0.1 on MIRAGE, so the curriculum design choice needs explicit justification.
minor comments (6)
  1. [§6.1 (model-size paragraph)] The paragraph beginning 'Relation between model size and hallucination types' repeats the phrase 'pretraining data' and should refer to model size; the same typo appears in the heading of the model-size analysis.
  2. [Tables 6–7] 'Pertaining data' should be 'pretraining data', and 'halucination' in Sec. 6.2 should be 'hallucination'.
  3. [Fig. 18] The label 'Answer: I' does not match the A–E choices shown, and the auxiliary description (Blue≈80) conflicts with the reasoning chain (Blue≈60); please correct or clarify this example.
  4. [§4.2] In Eq. (1)–(2), the sets underlying Precision_s, Recall_s, Precision_c, and Recall_c are only described informally; please define M_s,pred, M_s,gt, M_c,pred, and M_c,gt explicitly.
  5. [§B.1 and §5.2] The statement 'The number of CRFT stages is set to 1' is confusing given the k-round description in Sec. 5.2 and the exploration of k in Table 15; please clarify the relationship between curriculum stages and rounds.
  6. [Table 8] The claim that applying full CHI to the base model 'does not lead to performance improvement' is not supported by the numbers, which show 28.8→29.0 on MIRAGE and 68.2→68.3 on MathVista; either report significance or soften the claim.

Circularity Check

2 steps flagged · score 4.0 of 10

The benchmark's isolation claim rests on a definitional proxy: 'correctly perceived' is operationalized as LLM-verified descriptions, and the curation models are later reported as empirical findings.

  1. fitted input called prediction [Sec 3.1 Data Curation; Table 2]
    "For difficulty curation, we use three open-source MLLMs [2, 5, 18] to generate image descriptions, retaining only questions where these descriptions are consistently accurate (verified by a secondary LLM [36]) but lead to frequent reasoning errors, aligning with our benchmark focus."

    The same model families used for curation (Qwen2.5-VL, InternVL-2.5, Llama-3.2-Vision) are then evaluated in Table 2 and reported as having low accuracy and high hallucination rates. Because the dataset was explicitly filtered to questions where these models make frequent reasoning errors, their poor performance on MIRAGE is partly guaranteed by construction rather than discovered. Reporting these rates as evidence of reasoning hallucination, without an independent perception check for the evaluated models, reduces the benchmark's headline finding to the curation criterion.

  2. self definitional [Abstract and Sec 3.1]
    "we propose the MIRAGE benchmark, which isolates reasoning hallucinations by constructing questions where input images are correctly perceived by MLLMs yet reasoning errors persist. ... retaining only questions where these descriptions are consistently accurate (verified by a secondary LLM [36]) but lead to frequent reasoning errors."

    The claim that MIRAGE isolates reasoning hallucinations depends on the assertion that inputs are 'correctly perceived.' In the construction, 'correctly perceived' is defined operationally as generating image descriptions that a secondary LLM judges to be consistently accurate. This is a proxy, not a direct perception measurement, so the isolation property is true by the paper's own definition rather than by independent evidence. The leap from 'accurate LLM-generated description' to 'correct visual perception by the evaluated MLLM' is not demonstrated, making the benchmark's central premise definitionally self-supporting.

full rationale

The paper's central benchmark property—that MIRAGE isolates reasoning hallucinations because inputs are correctly perceived—is not independently established. The curation in Sec 3.1 verifies perception only through LLM-generated descriptions judged by another LLM, and only for three open-source MLLMs. For black-box models such as GPT-4o and Gemini, and even for the curation models themselves, no direct perception probe is provided. Consequently, the reported reasoning-hallucination rates may include perception-induced errors, and the benchmark's isolation claim reduces to the curation definition. There is no problematic self-citation chain: references to the authors' prior work ([9], [10], [32]) are peripheral implementation/background citations, not load-bearing. The Logos method is also validated on an external benchmark (MathVista), which provides independent support for its training components. However, because the headline isolation result depends on a definitional proxy and the curation models are reused as evaluated models, a moderate circularity score of 4 is appropriate rather than a higher score.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

The central benchmark rests on assumptions about LLM perception proxies and LLM-based evaluation. The free parameters are mainly curation thresholds and training hyperparameters selected by the authors. No new physical or conceptual entity is postulated.

free parameters (6)
  • Perception-filter agreement threshold = Not specified
    Data Curation (Sec 3.1) keeps questions where three MLLMs' descriptions are 'consistently accurate' and lead to 'frequent reasoning errors', but no numeric threshold is given for consistency or error frequency.
  • Balanced sampling imbalance rate = Small, unspecified
    Data Curation (Sec 3.1) samples to ensure balanced distribution 'maintaining a small imbalanced rate'; the target rate is not quantified.
  • CRFT difficulty thresholds = Stage 1: avg reward > 0; later stages: avg reward < 0.5
    Sec 5.2: thresholds are hand-chosen to filter training questions; no sensitivity analysis is provided except for the stage count.
  • Number of curriculum stages k = 1
    Sec D.8 and Table 15: k=1 is selected because further stages give marginal improvement; the choice is tuned on MIRAGE and MathVista.
  • Free-form answer relative-error accuracy threshold = Not specified
    Sec 4.1: statistical questions use relative error with a threshold, but the exact threshold value is never stated.
  • Manual-fix sample selection = 10% of commonly misanswered questions
    Sec 6.1: the manual correction study selects 10% of commonly misanswered questions; this heuristic may inflate the observed accuracy improvement.
assumptions (6)
  • domain assumption Accurate LLM-generated image descriptions imply correct visual perception by the evaluated MLLMs.
    Data curation (Sec 3.1) filters questions based on descriptions from three open-source MLLMs verified by a secondary LLM; no ground-truth perception label is used.
  • domain assumption A secondary LLM can reliably verify the correctness of image descriptions.
    Sec 3.1: descriptions 'verified by a secondary LLM' are treated as ground truth.
  • domain assumption LLM judges can reliably detect reasoning hallucinations from step and claim comparisons.
    Sec 4.2 and 4.3: hallucination type detection and LHS use LLM-as-a-judge with in-context learning; only 100 sampled responses are human-checked for LHS (7.5% difference).
  • domain assumption Automatically extracted steps and claims faithfully represent the reasoning process.
    Sec 3.2: steps and claims are extracted by a state-of-the-art LLM with few-shot prompts and parsed with regex; no human verification of extraction quality is reported.
  • domain assumption Removing the KL-divergence term from GRPO is safe for reasoning models.
    Sec 5.1 and D.6: KL is removed because of distribution gap; the ablation shows larger KL hurts performance, but removing KL entirely can cause reward hacking or instability in general.
  • domain assumption Answer-accuracy reward is a sufficient training signal to reduce reasoning hallucinations.
    Sec 5: Logos optimizes only format and answer-accuracy rewards without a process-level reward, yet the paper claims logical hallucination reduction.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MIRAGE: Assessing Hallucination in Multimodal Reasoning Chains of MLLM." pith.science (2026). https://pith.science/paper/IMI3GSBI

@misc{pith2026250524238,
  author       = {Pith},
  title        = {Pith review of: MIRAGE: Assessing Hallucination in Multimodal Reasoning Chains of MLLM},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IMI3GSBI}},
  note         = {Machine review of arXiv:2505.24238}
}
read the original abstract

Multimodal hallucination in multimodal large language models (MLLMs) restricts the correctness of MLLMs. However, multimodal hallucinations are multi-sourced and arise from diverse causes. Existing benchmarks fail to adequately distinguish between perception-induced hallucinations and reasoning-induced hallucinations. This failure constitutes a significant issue and hinders the diagnosis of multimodal reasoning failures within MLLMs. To address this, we propose the {\dataset} benchmark, which isolates reasoning hallucinations by constructing questions where input images are correctly perceived by MLLMs yet reasoning errors persist. {\dataset} introduces multi-granular evaluation metrics: accuracy, factuality, and LLMs hallucination score for hallucination quantification. Our analysis reveals that (1) the model scale, data scale, and training stages significantly affect the degree of logical, fabrication, and factual hallucinations; (2) current MLLMs show no effective improvement on spatial hallucinations caused by misinterpreted spatial relationships, indicating their limited visual reasoning capabilities; and (3) question types correlate with distinct hallucination patterns, highlighting targeted challenges and potential mitigation strategies. To address these challenges, we propose {\method}, a method that combines curriculum reinforcement fine-tuning to encourage models to generate logic-consistent reasoning chains by stepwise reducing learning difficulty, and collaborative hint inference to reduce reasoning complexity. {\method} establishes a baseline on {\dataset}, and reduces the logical hallucinations in original base models.

Figures

Figures reproduced from arXiv: 2505.24238 by the authors.

Figure 1
Figure 1. Data distribution and structure of MIRAGE benchmark. (a) shows the classes and amounts [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The construction and evaluation of MIRAGE. (a) shows the construction of MIRAGE.And [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Overview of our baseline Logos. During training, Logos adopts curriculum reinforcement [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (17 more)
Figure 5
Figure 5. Figure 5: Pearson correlation among reasoning hallucination types [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Qualitative results of manually fixing the reasoning hallucination in the reasoning chain and [PITH_FULL_IMAGE:figures/full_fig_p020_6.png]
Figure 7
Figure 7. Figure 7: Pearson correlation regarding hallucination types from six most representative MLLMs. [PITH_FULL_IMAGE:figures/full_fig_p021_7.png]
Figure 8
Figure 8. Figure 8: The evaluation prompt used to extract intermediate results. [PITH_FULL_IMAGE:figures/full_fig_p025_8.png]
Figure 9
Figure 9. Figure 9: The evaluation prompt used for factuality assessment ( [PITH_FULL_IMAGE:figures/full_fig_p026_9.png]
Figure 10
Figure 10. Figure 10: The evaluation prompt used to detect hallucination types in reasoning chains. [PITH_FULL_IMAGE:figures/full_fig_p027_10.png]
Figure 11
Figure 11. Figure 11: The evaluation prompt used for LLMs hallucination score extraction. [PITH_FULL_IMAGE:figures/full_fig_p028_11.png]
Figure 12
Figure 12. Figure 12: The example of geometry question in MIRAGE. [PITH_FULL_IMAGE:figures/full_fig_p029_12.png]
Figure 13
Figure 13. Figure 13: The example of algebraic question in MIRAGE. [PITH_FULL_IMAGE:figures/full_fig_p030_13.png]
Figure 14
Figure 14. Figure 14: The example of arithmetic question in MIRAGE. [PITH_FULL_IMAGE:figures/full_fig_p031_14.png]
Figure 15
Figure 15. Figure 15: The example of scientific question in MIRAGE. [PITH_FULL_IMAGE:figures/full_fig_p032_15.png]
Figure 16
Figure 16. Figure 16: The example of spatial question in MIRAGE. [PITH_FULL_IMAGE:figures/full_fig_p033_16.png]
Figure 17
Figure 17. Figure 17: The example of logical question in MIRAGE. [PITH_FULL_IMAGE:figures/full_fig_p034_17.png]
Figure 18
Figure 18. Figure 18: The example of statistical question in MIRAGE. [PITH_FULL_IMAGE:figures/full_fig_p035_18.png]
Figure 19
Figure 19. Figure 19: The topic-specific hints used in MIRAGE. [PITH_FULL_IMAGE:figures/full_fig_p036_19.png]
Figure 20
Figure 20. Figure 20: Response example from Qwen2.5-VL-7B-Instruct. Red font means reasoning hallucination [PITH_FULL_IMAGE:figures/full_fig_p037_20.png]
Figure 21
Figure 21. Figure 21: Response examples from Logos-7B and Gemini-2-flash-thinking. [PITH_FULL_IMAGE:figures/full_fig_p038_21.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Models

    cs.CL 2025-09 conditional novelty 4.0 of 10

    A structured literature survey concluding that reasoning capabilities do not automatically make LLMs more trustworthy and can introduce new vulnerabilities in safety, robustness, and privacy.

Reference graph

Works this paper leans on

87 extracted references · 20 canonical work pages · cited by 1 Pith paper

  1. [1]

    Phi-3 technical report: A highly capable language model locally on your phone.arXiv preprint arXiv:2404.14219, 2024

    Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, et al. Phi-3 technical report: A highly capable language model locally on your phone.arXiv preprint arXiv:2404.14219, 2024

    arXiv 2024

  2. [2]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report.arXiv preprint arXiv:2502.13923, 2025

    arXiv 2025

  3. [3]

    Mocha: Multi-objective reinforcement mitigating caption hallucinations.arXiv preprint arXiv:2312.03631, 2, 2023

    Assaf Ben-Kish, Moran Yanuka, Morris Alper, Raja Giryes, and Hadar Averbuch-Elor. Mocha: Multi-objective reinforcement mitigating caption hallucinations.arXiv preprint arXiv:2312.03631, 2, 2023

    arXiv 2023

  4. [4]

    Mm-iq: Benchmarking human-like abstraction and reasoning in multimodal models.arXiv preprint arXiv:2502.00698, 2025

    Huanqia Cai, Yijun Yang, and Winston Hu. Mm-iq: Benchmarking human-like abstraction and reasoning in multimodal models.arXiv preprint arXiv:2502.00698, 2025

    arXiv 2025

  5. [5]

    Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 24185–24198, 2024

    2024

  6. [6]

    Mitigating hallucination in visual language models with visual supervision.arXiv preprint arXiv:2311.16479, 2023

    Zhiyang Chen, Yousong Zhu, Yufei Zhan, Zhaowen Li, Chaoyang Zhao, Jinqiao Wang, and Ming Tang. Mitigating hallucination in visual language models with visual supervision.arXiv preprint arXiv:2311.16479, 2023

    arXiv 2023

  7. [7]

    Puzzlevqa: Diagnosing multimodal reasoning challenges of language models with abstract visual patterns

    Yew Ken Chia, Vernon Toh, Deepanway Ghosal, Lidong Bing, and Soujanya Poria. Puzzlevqa: Diagnosing multimodal reasoning challenges of language models with abstract visual patterns. InFindings of the Association for Computational Linguistics: ACL 2024, pages 16259–16273, 2024. 11

    2024

  8. [8]

    Zero-shot generalizable incremental learning for vision-language object detection

    Jieren Deng, Haojian Zhang, Kun Ding, Jianhua Hu, Xingxuan Zhang, and Yunkuan Wang. Zero-shot generalizable incremental learning for vision-language object detection. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems

Show all 87 references
  1. [9]

    Mr-gdino: efficient open-world continual object detection.arXiv preprint arXiv:2412.15979, 2024

    Bowen Dong, Zitong Huang, Guanglei Yang, Lei Zhang, and Wangmeng Zuo. Mr-gdino: efficient open-world continual object detection.arXiv preprint arXiv:2412.15979, 2024

    2024 arXiv

  2. [10]

    Lpt: Long-tailed prompt tuning for image classification

    Bowen Dong, Pan Zhou, Y AN Shuicheng, and Wangmeng Zuo. Lpt: Long-tailed prompt tuning for image classification. InThe Eleventh International Conference on Learning Representations, 2023

    2023

  3. [11]

    A survey on in-context learning.arXiv preprint arXiv:2301.00234, 2022

    Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Tianyu Liu, et al. A survey on in-context learning.arXiv preprint arXiv:2301.00234, 2022

    2022 arXiv

  4. [12]

    Virgo: A preliminary exploration on reproducing o1-like mllm.arXiv preprint arXiv:2501.01904, 2025

    Yifan Du, Zikang Liu, Yifan Li, Wayne Xin Zhao, Yuqi Huo, Bingning Wang, Weipeng Chen, Zheng Liu, Zhongyuan Wang, and Ji-Rong Wen. Virgo: A preliminary exploration on reproducing o1-like mllm.arXiv preprint arXiv:2501.01904, 2025

    2025 arXiv

  5. [13]

    Detecting hallucinations in large language models using semantic entropy.Nature, 630(8017):625–630, 2024

    Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn, and Yarin Gal. Detecting hallucinations in large language models using semantic entropy.Nature, 630(8017):625–630, 2024

    2024

  6. [14]

    Mme: A comprehensive evaluation benchmark for multimodal large language models.arXiv preprint arXiv:2306.13394, 2023

    Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, et al. Mme: A comprehensive evaluation benchmark for multimodal large language models.arXiv preprint arXiv:2306.13394, 2023

    2023 arXiv

  7. [15]

    GPTScore: Evaluate as you desire

    Jinlan Fu, See-Kiong Ng, Zhengbao Jiang, and Pengfei Liu. GPTScore: Evaluate as you desire. In Kevin Duh, Helena Gomez, and Steven Bethard, editors,Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language...

    2024

  8. [16]

    The capacity for moral self-correction in large language models.Parameters, 109(1010):1011

    Deep Ganguli, Amanda Askell, Nicholas Schiefer, Thomas I Liao, and Kamile Lukošiute. The capacity for moral self-correction in large language models.Parameters, 109(1010):1011

  9. [17]

    Are language models puzzle prodigies? algorithmic puzzles unveil serious challenges in multimodal reasoning

    Deepanway Ghosal, Vernon Toh Yan Han, Yew Ken Chia, , and Soujanya Poria. Are language models puzzle prodigies? algorithmic puzzles unveil serious challenges in multimodal reasoning. arXiv preprint arXiv:2403.03864, 2024

    2024 arXiv

  10. [18]

    The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  11. [19]

    A survey on llm-as-a-judge.arXiv preprint arXiv:2411.15594, 2024

    Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, et al. A survey on llm-as-a-judge.arXiv preprint arXiv:2411.15594, 2024

    2024 arXiv

  12. [20]

    Hallusionbench: an advanced diagnostic suite for entangled language hallucination and visual illusion in large vision-language models

    Tianrui Guan, Fuxiao Liu, Xiyang Wu, Ruiqi Xian, Zongxia Li, Xiaoyu Liu, Xijun Wang, Lichang Chen, Furong Huang, Yaser Yacoob, et al. Hallusionbench: an advanced diagnostic suite for entangled language hallucination and visual illusion in large vision-language models. InProcee...

    2024

  13. [21]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    2025 arXiv

  14. [22]

    When continue learning meets multimodal large language model: A survey.arXiv preprint arXiv:2503.01887, 2025

    Yukang Huo and Hao Tang. When continue learning meets multimodal large language model: A survey.arXiv preprint arXiv:2503.01887, 2025

    2025 arXiv

  15. [23]

    Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024. 12

    2024 arXiv

  16. [24]

    Openai o1 system card.arXiv preprint arXiv:2412.16720, 2024

    Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai o1 system card.arXiv preprint arXiv:2412.16720, 2024

    2024 arXiv

  17. [25]

    Towards mitigating LLM hallucination via self reflection

    Ziwei Ji, Tiezheng Yu, Yan Xu, Nayeon Lee, Etsuko Ishii, and Pascale Fung. Towards mitigating LLM hallucination via self reflection. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Findings of the Association for Computational Linguistics: EMNLP 2023, pages 1827–1843, S...

    2023

  18. [26]

    Hallucination augmented contrastive learning for multimodal large language model

    Chaoya Jiang, Haiyang Xu, Mengfan Dong, Jiaxing Chen, Wei Ye, Ming Yan, Qinghao Ye, Ji Zhang, Fei Huang, and Shikun Zhang. Hallucination augmented contrastive learning for multimodal large language model. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern ...

    2024

  19. [27]

    Mme-cot: Benchmarking chain-of-thought in large multimodal models for reasoning quality, robustness, and efficiency.arXiv preprint arXiv:2502.09621, 2025

    Dongzhi Jiang, Renrui Zhang, Ziyu Guo, Yanwei Li, Yu Qi, Xinyan Chen, Liuhui Wang, Jianhan Jin, Claire Guo, Shen Yan, et al. Mme-cot: Benchmarking chain-of-thought in large multimodal models for reasoning quality, robustness, and efficiency.arXiv preprint arXiv:2502.09621, 2025

    2025 arXiv

  20. [28]

    Decoupling representation and classifier for long-tailed recognition

    Bingyi Kang, Saining Xie, Marcus Rohrbach, Zhicheng Yan, Albert Gordo, Jiashi Feng, and Yannis Kalantidis. Decoupling representation and classifier for long-tailed recognition. In International Conference on Learning Representations

  21. [29]

    Gonzalez, Hao Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large lan- guage model serving with pagedattention. InProceedings of the ACM SIGOPS 29th Symposium on Operating System...

    2023

  22. [30]

    Seed- bench: Benchmarking multimodal llms with generative comprehension.arXiv preprint arXiv:2307.16125, 2023

    Bohao Li, Rui Wang, Guangzhi Wang, Yuying Ge, Yixiao Ge, and Ying Shan. Seed- bench: Benchmarking multimodal llms with generative comprehension.arXiv preprint arXiv:2307.16125, 2023

    2023 arXiv

  23. [31]

    Llms-as-judges: a comprehensive survey on llm-based evaluation methods.arXiv preprint arXiv:2412.05579, 2024

    Haitao Li, Qian Dong, Junjie Chen, Huixue Su, Yujia Zhou, Qingyao Ai, Ziyi Ye, and Yiqun Liu. Llms-as-judges: a comprehensive survey on llm-based evaluation methods.arXiv preprint arXiv:2412.05579, 2024

    2024 arXiv

  24. [32]

    Salad-bench: A hierarchical and comprehensive safety benchmark for large language models

    Lijun Li, Bowen Dong, Ruohui Wang, Xuhao Hu, Wangmeng Zuo, Dahua Lin, Yu Qiao, and Jing Shao. Salad-bench: A hierarchical and comprehensive safety benchmark for large language models. InFindings of the Association for Computational Linguistics: ACL 2024, pages 3923–3954, 2024

    2024

  25. [33]

    Long-tailed visual recognition via gaussian clouded logit adjustment

    Mengke Li, Yiu-ming Cheung, and Yang Lu. Long-tailed visual recognition via gaussian clouded logit adjustment. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6929–6938, 2022

    2022

  26. [34]

    Evaluating object hallucination in large vision-language models

    Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Xin Zhao, and Ji-Rong Wen. Evaluating object hallucination in large vision-language models. InThe 2023 Conference on Empirical Methods in Natural Language Processing

    2023

  27. [35]

    Omnibench: Towards the future of universal omni-language models.arXiv preprint arXiv:2409.15272, 2024

    Yizhi Li, Ge Zhang, Yinghao Ma, Ruibin Yuan, Kang Zhu, Hangyu Guo, Yiming Liang, Jiaheng Liu, Zekun Wang, Jian Yang, et al. Omnibench: Towards the future of universal omni-language models.arXiv preprint arXiv:2409.15272, 2024

    2024

  28. [36]

    Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437, 2024

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437, 2024

    2024 arXiv

  29. [37]

    Mitigat- ing hallucination in large multi-modal models via robust instruction tuning

    Fuxiao Liu, Kevin Lin, Linjie Li, Jianfeng Wang, Yaser Yacoob, and Lijuan Wang. Mitigat- ing hallucination in large multi-modal models via robust instruction tuning. InThe Twelfth International Conference on Learning Representations

  30. [38]

    Visual instruction tuning, 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, 2023. 13

    2023

  31. [39]

    Visual-rft: Visual reinforcement fine-tuning.arXiv preprint arXiv:2503.01785, 2025

    Ziyu Liu, Zeyi Sun, Yuhang Zang, Xiaoyi Dong, Yuhang Cao, Haodong Duan, Dahua Lin, and Jiaqi Wang. Visual-rft: Visual reinforcement fine-tuning.arXiv preprint arXiv:2503.01785, 2025

    2025 arXiv

  32. [40]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. InInternational Conference on Learning Representations, 2019

    2019

  33. [41]

    Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts

    Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. InInternational Conference on Learning Representati...

    2024

  34. [42]

    Learn to explain: Multimodal reasoning via thought chains for science question answering

    Pan Lu, Swaroop Mishra, Tony Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. Learn to explain: Multimodal reasoning via thought chains for science question answering. InThe 36th Conference on Neural Information Process- ing Systems...

    2022

  35. [43]

    Ursa: Understanding and verifying chain-of-thought reasoning in multimodal mathematics.arXiv preprint arXiv:2501.04686, 2025

    Ruilin Luo, Zhuofan Zheng, Yifan Wang, Yiyao Yu, Xinzhe Ni, Zicheng Lin, Jin Zeng, and Yujiu Yang. Ursa: Understanding and verifying chain-of-thought reasoning in multimodal mathematics.arXiv preprint arXiv:2501.04686, 2025

    2025

  36. [44]

    Ok-vqa: A visual question answering benchmark requiring external knowledge

    Kenneth Marino, Mohammad Rastegari, Ali Farhadi, and Roozbeh Mottaghi. Ok-vqa: A visual question answering benchmark requiring external knowledge. InProceedings of the IEEE/cvf conference on computer vision and pattern recognition, pages 3195–3204, 2019

    2019

  37. [45]

    Chartqa: A benchmark for question answering about charts with visual and logical reasoning

    Ahmed Masry, Xuan Long Do, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. Chartqa: A benchmark for question answering about charts with visual and logical reasoning. InFindings of the Association for Computational Linguistics: ACL 2022, pages 2263–2279, 2022

    2022

  38. [46]

    Mm-eureka: Exploring visual aha moment with rule-based large-scale reinforcement learning.arXiv preprint arXiv:2503.07365, 2025

    Fanqing Meng, Lingxiao Du, Zongkai Liu, Zhixiang Zhou, Quanfeng Lu, Daocheng Fu, Botian Shi, Wenhai Wang, Junjun He, Kaipeng Zhang, et al. Mm-eureka: Exploring visual aha moment with rule-based large-scale reinforcement learning.arXiv preprint arXiv:2503.07365, 2025

    2025 arXiv

  39. [47]

    Compositional chain-of- thought prompting for large multimodal models

    Chancharik Mitra, Brandon Huang, Trevor Darrell, and Roei Herzig. Compositional chain-of- thought prompting for large multimodal models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14420–14431, 2024

    2024

  40. [48]

    Pytorch: An imperative style, high-performance deep learning library

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu ...

    2019

  41. [49]

    Lmm-r1: Empowering 3b lmms with strong reasoning abilities through two-stage rule-based rl.arXiv preprint arXiv:2503.07536, 2025

    Yingzhe Peng, Gongrui Zhang, Miaosen Zhang, Zhiyuan You, Jie Liu, Qipeng Zhu, Kai Yang, Xingzhong Xu, Xin Geng, and Xu Yang. Lmm-r1: Empowering 3b lmms with strong reasoning abilities through two-stage rule-based rl.arXiv preprint arXiv:2503.07536, 2025

    2025 arXiv

  42. [50]

    Receval: Evaluating reasoning chains via correctness and informativeness.arXiv preprint arXiv:2304.10703, 2023

    Archiki Prasad, Swarnadeep Saha, Xiang Zhou, and Mohit Bansal. Receval: Evaluating reasoning chains via correctness and informativeness.arXiv preprint arXiv:2304.10703, 2023

    2023 arXiv

  43. [51]

    Mitigating object hallucination in mllms via data-augmented phrase-level alignment

    Pritam Sarkar, Sayna Ebrahimi, Ali Etemad, Ahmad Beirami, Sercan O Arik, and Tomas Pfister. Mitigating object hallucination in mllms via data-augmented phrase-level alignment. InThe Thirteenth International Conference on Learning Representations

  44. [52]

    Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347, 2017

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  45. [53]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024. 14

    2024 arXiv

  46. [54]

    Monte carlo tree search: A review of recent modifications and applications.Artificial Intelligence Review, 56(3):2497–2562, 2023

    Maciej ´Swiechowski, Konrad Godlewski, Bartosz Sawicki, and Jacek Ma´ndziuk. Monte carlo tree search: A review of recent modifications and applications.Artificial Intelligence Review, 56(3):2497–2562, 2023

    2023

  47. [55]

    Gemini: a family of highly capable multimodal models.arXiv preprint arXiv:2312.11805, 2023

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models.arXiv preprint arXiv:2312.11805, 2023

    2023 arXiv

  48. [56]

    QVQ: To See the World with Wisdom

    Qwen Team. QVQ: To See the World with Wisdom. https://qwenlm.github.io/blog/ qvq-72b-preview/, 2024

    2024

  49. [57]

    Uncertainty-based abstention in llms improves safety and reduces hallucinations.arXiv preprint arXiv:2404.10960, 2024

    Christian Tomani, Kamalika Chaudhuri, Ivan Evtimov, Daniel Cremers, and Mark Ibrahim. Uncertainty-based abstention in llms improves safety and reduces hallucinations.arXiv preprint arXiv:2404.10960, 2024

    2024 arXiv

  50. [58]

    Eyes wide shut? exploring the visual shortcomings of multimodal llms

    Shengbang Tong, Zhuang Liu, Yuexiang Zhai, Yi Ma, Yann LeCun, and Saining Xie. Eyes wide shut? exploring the visual shortcomings of multimodal llms. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9568–9578, 2024

    2024

  51. [59]

    Measuring multimodal mathematical reasoning with math-vision dataset

    Ke Wang, Junting Pan, Weikang Shi, Zimu Lu, Houxing Ren, Aojun Zhou, Mingjie Zhan, and Hongsheng Li. Measuring multimodal mathematical reasoning with math-vision dataset. InThe Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024

    2024

  52. [60]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191, 2024

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191, 2024

    2024 arXiv

  53. [61]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022

    2022

  54. [62]

    Ritual: Ran- dom image transformations as a universal anti-hallucination lever in lvlms.arXiv preprint arXiv:2405.17821, 2024

    Sangmin Woo, Jaehyuk Jang, Donguk Kim, Yubin Choi, and Changick Kim. Ritual: Ran- dom image transformations as a universal anti-hallucination lever in lvlms.arXiv preprint arXiv:2405.17821, 2024

    2024 arXiv

  55. [63]

    Autohallusion: Automatic gen- eration of hallucination benchmarks for vision-language models

    Xiyang Wu, Tianrui Guan, Dianqi Li, Shuaiyi Huang, Xiaoyu Liu, Xijun Wang, Ruiqi Xian, Abhinav Shrivastava, Furong Huang, Jordan Boyd-Graber, et al. Autohallusion: Automatic gen- eration of hallucination benchmarks for vision-language models. InFindings of the Association for ...

    2024

  56. [64]

    Grok 3 Beta — The Age of Reasoning Agents.https://x.ai/grok, 2025

    xAI. Grok 3 Beta — The Age of Reasoning Agents.https://x.ai/grok, 2025

    2025

  57. [65]

    Mitigating object hallucination via concentric causal attention.Advances in Neural Information Processing Systems, 37:92012–92035, 2024

    Yun Xing, Yiheng Li, Ivan Laptev, and Shijian Lu. Mitigating object hallucination via concentric causal attention.Advances in Neural Information Processing Systems, 37:92012–92035, 2024

    2024

  58. [66]

    Llava-cot: Let vision language models reason step-by-step, 2024

    Guowei Xu, Peng Jin, Hao Li, Yibing Song, Lichao Sun, and Li Yuan. Llava-cot: Let vision language models reason step-by-step, 2024

    2024

  59. [67]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report.arXiv preprint arXiv:2412.15115, 2024

    2024 arXiv

  60. [68]

    Soft-prompting with graph-of-thought for multi-modal representation learning

    Jun Cheng Yang, Zuchao Li, Shuai Xie, Wei Yu, Shijun Li, and Bo Du. Soft-prompting with graph-of-thought for multi-modal representation learning. InProceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLI...

    2024

  61. [69]

    Mitigating hallucination in large vision- language models via modular attribution and intervention

    Tianyun Yang, Ziniu Li, Juan Cao, and Chang Xu. Mitigating hallucination in large vision- language models via modular attribution and intervention. InThe Thirteenth International Conference on Learning Representations, 2025. 15

    2025

  62. [70]

    R1-onevision: Advancing generalized multimodal reasoning through cross-modal formalization.arXiv preprint arXiv:2503.10615, 2025

    Yi Yang, Xiaoxuan He, Hongkun Pan, Xiyan Jiang, Yan Deng, Xingtao Yang, Haoyu Lu, Dacheng Yin, Fengyun Rao, Minfeng Zhu, et al. R1-onevision: Advancing generalized multimodal reasoning through cross-modal formalization.arXiv preprint arXiv:2503.10615, 2025

    2025 arXiv

  63. [71]

    Mulberry: Empowering mllm with o1-like reasoning and reflection via collective monte carlo tree search.arXiv preprint arXiv:2412.18319, 2024

    Huanjin Yao, Jiaxing Huang, Wenhao Wu, Jingyi Zhang, Yibo Wang, Shunyu Liu, Yingjie Wang, Yuxin Song, Haocheng Feng, Li Shen, et al. Mulberry: Empowering mllm with o1-like reasoning and reflection via collective monte carlo tree search.arXiv preprint arXiv:2412.18319, 2024

    2024 arXiv

  64. [72]

    React: Synergizing reasoning and acting in language models

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. InInternational Conference on Learning Representations (ICLR), 2023

    2023

  65. [73]

    Limo: Less is more for reasoning.arXiv preprint arXiv:2502.03387, 2025

    Yixin Ye, Zhen Huang, Yang Xiao, Ethan Chern, Shijie Xia, and Pengfei Liu. Limo: Less is more for reasoning.arXiv preprint arXiv:2502.03387, 2025

    2025 arXiv

  66. [74]

    Hallucidoctor: Mitigating hallucinatory toxicity in visual instruction data

    Qifan Yu, Juncheng Li, Longhui Wei, Liang Pang, Wentao Ye, Bosheng Qin, Siliang Tang, Qi Tian, and Yueting Zhuang. Hallucidoctor: Mitigating hallucinatory toxicity in visual instruction data. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, ...

    2024

  67. [75]

    Dapo: An open-source llm reinforcement learning system at scale.arXiv preprint arXiv:2503.14476, 2025

    Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, et al. Dapo: An open-source llm reinforcement learning system at scale.arXiv preprint arXiv:2503.14476, 2025

    2025 arXiv

  68. [76]

    Rlhf-v: Towards trustworthy mllms via behavior alignment from fine-grained correctional human feedback

    Tianyu Yu, Yuan Yao, Haoye Zhang, Taiwen He, Yifeng Han, Ganqu Cui, Jinyi Hu, Zhiyuan Liu, Hai-Tao Zheng, Maosong Sun, et al. Rlhf-v: Towards trustworthy mllms via behavior alignment from fine-grained correctional human feedback. InProceedings of the IEEE/CVF Conference on Com...

    2024

  69. [77]

    Llama-berry: Pairwise optimization for o1-like olympiad-level mathematical reasoning.arXiv preprint arXiv:2410.02884, 2024

    Di Zhang, Jianbo Wu, Jingdi Lei, Tong Che, Jiatong Li, Tong Xie, Xiaoshui Huang, Shufei Zhang, Marco Pavone, Yuqiang Li, et al. Llama-berry: Pairwise optimization for o1-like olympiad-level mathematical reasoning.arXiv preprint arXiv:2410.02884, 2024

    arXiv

  70. [78]

    Reflective instruction tuning: Mitigating hallucinations in large vision-language models

    Jinrui Zhang, Teng Wang, Haigang Zhang, Ping Lu, and Feng Zheng. Reflective instruction tuning: Mitigating hallucinations in large vision-language models. InEuropean Conference on Computer Vision, pages 196–213. Springer, 2024

    2024

  71. [79]

    Mathverse: Does your multi-modal llm truly see the diagrams in visual math problems? InEuropean Conference on Computer Vision, pages 169–186

    Renrui Zhang, Dongzhi Jiang, Yichi Zhang, Haokun Lin, Ziyu Guo, Pengshuo Qiu, Aojun Zhou, Pan Lu, Kai-Wei Chang, Yu Qiao, et al. Mathverse: Does your multi-modal llm truly see the diagrams in visual math problems? InEuropean Conference on Computer Vision, pages 169–186. Springer, 2024

    2024

  72. [80]

    Vl-uncertainty: Detecting hallucination in large vision-language model via uncertainty estimation.arXiv preprint arXiv:2411.11919, 2024

    Ruiyang Zhang, Hu Zhang, and Zhedong Zheng. Vl-uncertainty: Detecting hallucination in large vision-language model via uncertainty estimation.arXiv preprint arXiv:2411.11919, 2024

    2024 arXiv

  73. [81]

    The lessons of developing process reward models in mathematical reasoning.arXiv preprint arXiv:2501.07301, 2025

    Zhenru Zhang, Chujie Zheng, Yangzhen Wu, Beichen Zhang, Runji Lin, Bowen Yu, Dayiheng Liu, Jingren Zhou, and Junyang Lin. The lessons of developing process reward models in mathematical reasoning.arXiv preprint arXiv:2501.07301, 2025

    2025 arXiv

  74. [82]

    Multimodal chain-of-thought reasoning in language models.Transactions on Machine Learning Research

    Zhuosheng Zhang, Aston Zhang, Mu Li, George Karypis, Alex Smola, et al. Multimodal chain-of-thought reasoning in language models.Transactions on Machine Learning Research

  75. [83]

    Beyond hallucinations: Enhancing lvlms through hallucination-aware direct preference optimization

    Zhiyuan Zhao, Bin Wang, Linke Ouyang, Xiaoyi Dong, Jiaqi Wang, and Conghui He. Beyond hallucinations: Enhancing lvlms through hallucination-aware direct preference optimization. arXiv preprint arXiv:2311.16839, 2023

    2023 arXiv

  76. [84]

    A picture is worth a graph: A blueprint debate paradigm for multimodal reasoning

    Changmeng Zheng, Dayong Liang, Wengyu Zhang, Xiao-Yong Wei, Tat-Seng Chua, and Qing Li. A picture is worth a graph: A blueprint debate paradigm for multimodal reasoning. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 419–428, 2024. 16

    2024

  77. [85]

    Thinking before looking: Improving multimodal llm reasoning via mitigating visual hallucination.arXiv preprint arXiv:2411.12591, 2024

    Haojie Zheng, Tianyang Xu, Hanchi Sun, Shu Pu, Ruoxi Chen, and Lichao Sun. Thinking before looking: Improving multimodal llm reasoning via mitigating visual hallucination.arXiv preprint arXiv:2411.12591, 2024

    2024 arXiv

  78. [86]

    Relying on the unreliable: The impact of language models’ reluctance to express uncertainty

    Kaitlyn Zhou, Jena Hwang, Xiang Ren, and Maarten Sap. Relying on the unreliable: The impact of language models’ reluctance to express uncertainty. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3623–3643, 2024

    2024

  79. [87]

    the answer is [answer in the input]

    Yiyang Zhou, Chenhang Cui, Rafael Rafailov, Chelsea Finn, and Huaxiu Yao. Aligning modalities in vision large language models via preference fine-tuning. InICLR 2024 Workshop on Reliable and Responsible Foundation Models. 17 Table 10: Categories of multimodal reasoning halluci...

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.