Drawing the Line: Enhancing Trustworthiness of MLLMs Through the Power of Refusal

Heyang Liu; Hongcheng Liu; Yanfeng Wang; Yuhao Wang; Yusheng Liao; Yu Wang; Zhiyuan Zhu

REVIEW 5 major objections 5 minor 16 references

Drawing the Line: Enhancing Trustworthiness of MLLMs Through the Power of Refusal

T0 review · 5 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that teaching MLLMs to refuse questions outside their information boundaries sharply improves trustworthiness without sacrificing helpfulness.

desk verdict Table 2 contradicts Eq. (5), so the headline trustworthiness gain is unverified; the framework and OOD results are promising enough to send to review with major-revision demands. read the letter →

arxiv 2412.11196 v1 pith:FH2KGVYP submitted 2024-12-15 cs.CL cs.CV

Yuhao Wang , Zhiyuan Zhu , Heyang Liu , Yusheng Liao , Hongcheng Liu , Yanfeng Wang , Yu Wang This is my paper · ORCID

classification cs.CLcs.CV

keywords multimodallargelanguagemodelstrustworthinessrefusalinformationboundarieshallucinationmitigationdirectpreferenceoptimizationconfidenceestimationvisualquestionanswering

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that a multimodal large language model (MLLM) becomes more trustworthy when it is trained to refuse answering questions that lie outside its information boundaries. It introduces the Information Boundary-aware Learning Framework (InBoL), which defines extrinsic boundaries (what the image actually contains) and intrinsic boundaries (what the model itself can know and perceive), then trains the model with instruction tuning and confidence-aware direct preference optimization. The paper's key reported result is a large jump in a user-centric trustworthiness score: LLaVA1.5-7B's in-domain score rises from -6.50 to 28.50 with the CA-DPO variant, while accuracy is preserved. This matters because it offers a concrete path to reducing hallucinations in vision-language assistants without forcing them to be silent or unhelpful.

What carries the argument

The central object is a pair of information boundaries: the extrinsic boundary, which asks whether the needed information is present in the visual input, and the intrinsic boundary, which asks whether this particular model's knowledge and perceptual capacity can supply the answer. Confidence estimation locates the intrinsic boundary by sampling multiple responses from the base model and measuring the accuracy rate with a hybrid evaluator (string matching plus an LLM semantic check), then splitting questions into Known, Mixed, and Unknown using thresholds $\delta_k$ and $\delta_{uk}$. The training machinery is Confidence-aware DPO, a loss that weights two preference pairs - 'correct over incorrect' and 'refusal over incorrect' - by the estimated confidence $conf_x$, so high-confidence questions push toward answering and low-confidence ones push toward refusal. The evaluation machinery is a user-centric trustworthiness score $s_{trust} = 2\cdot Acc + RefR - 1$, which rewards correct answers, rewards refusals over wrong ones, and lets different models be compared without knowing each model's boundary.

What would settle it

Give the pipeline a model whose confidence is demonstrably miscalibrated on a controlled topic (for example, images modified so that answerability changes while the question text stays identical), and check where IDK-IT and CA-DPO draw the refusal boundary; if refusals track the biased confidence rather than true answerability, the learned boundary is an artifact of the estimate. A cheaper check on the quantitative claim: recompute the in-domain trustworthiness scores from the reported Acc and RefR values with the paper's own formula $s_{trust} = 2\cdot Acc + RefR - 1$, since the published in-domain numbers do not reproduce and the magnitude of the gain should be re-verified with the released evaluation code or per-sample scores.

Watch

Extended reading notes

Core claim

The paper's central claim is that an MLLM should refuse exactly when a question crosses either of two information boundaries, and that this refusal behavior is learnable from the model's own signals. The extrinsic boundary separates questions answerable from the image from those that are not, such as questions about absent objects or false premises; the intrinsic boundary separates what the model's own knowledge and perception can supply from what they cannot. The paper claims that both boundaries can be taught with data generated from any VQA set: confidence estimation for the intrinsic side and GPT-4o-generated unanswerable questions for the extrinsic side. The reported evidence is that IDK-IT and especially the confidence-aware DPO variant (CA-DPO) raise the trustworthiness score above baselines on every benchmark tested, with CA-DPO preserving accuracy while boosting refusal, for example lifting LLaVA1.5-7B's in-domain trustworthiness from -6.50 to 28.50 at a 49.1% refusal rate.

Load-bearing premise

The framework rests on the assumption that the accuracy of the base model's sampled outputs, judged by a hybrid string-matching-plus-LLM checker, truthfully measures whether the model can answer the question; if that estimate is biased, the learned refusal boundary is trained at the wrong place.

Editorial extensions

If this is right

Training with IDK-IT and CA-DPO raises the trustworthiness score on all out-of-domain benchmarks tested (general VQA, knowledge-intensive VQA, and unanswerable VQA) for both LLaVA1.5-7B and 13B, so the effect is not confined to the training distribution.
Trained models refuse more on low-confidence questions and answer more on high-confidence ones, and their accuracy on the questions they do answer exceeds the original model's, indicating that boundary awareness improves both caution and usefulness.
On unanswerable out-of-domain sets (VizWiz-unanswerable, filtered VQAv2-IDK, BeyondVisQA), refusal rates jump from single digits to roughly 70-81%, showing the model can recognize the extrinsic boundary.
CA-DPO outperforms plain DPO trained on the same preference data, implying that the confidence-weighted balancing is what drives the improvement rather than the preference data alone.
The data-generation pipeline works with an open-source generator (Qwen2-VL-72B) as well as GPT-4o, so the framework does not depend on a closed API.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper does not spell out that the refusal boundary is per-model: applying InBoL to a different base model requires regenerating the IDK data with that model's own confidence estimates, so the framework transfers as a procedure, not as a fixed refusal line.
A natural extension of the linear-probing result is to condition generated refusals on whether the missing information is visual or knowledge-based, and to score the resulting explanations with the same user-centric metric; the paper lists such explanatory refusals as future work.
Because the CA-DPO loss weights the answer-versus-refusal trade-off through a single confidence factor, varying that factor should produce a predictable monotone sweep of helpfulness against caution, which would let deployers set the refusal threshold to the cost of error in their application.
The framework's ceiling is set by the calibration of the sampling-based confidence estimate; replacing it with a calibrated uncertainty measure (ensembles, logit-based tests) would be a direct upgrade path that the paper does not explore.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

First thing to know: Table 2, the in-domain results that drive the paper's headline claim, does not match the paper's own scoring equation. Eq. (5) gives strust = 2*Acc + RefR - 1, and this works for Table 1 (e.g., AOKVQA: 2*0.7856 + 0 - 1 = 0.5712). But Table 2's in-domain numbers cannot be reproduced: for LLaVA1.5-7B, Acc=12.00 and RefR=46.10 give -29.9, not -6.50; for IDK-IT, 2*0.92 + 0.386 - 1 = 1.226, not 16.00. Table 4 even lists the same Strust=28.50 for CA-DPO 7B with different Acc and RefR. So the claim that CA-DPO raises the score from -6.50 to 28.50 is currently unsupported.

What is genuinely new here: the intrinsic/extrinsic boundary split is a sensible way to organize refusal training for MLLMs, and the data generation pipeline is a real contribution—it produces unanswerable questions from any VQA set, and the authors show it works with Qwen2-VL-72B as well as GPT-4o. The CA-DPO loss, weighing preference pairs by the model's own confidence, is a reasonable idea. The hybrid evaluator (string match plus Llama2-13B) is a practical detail, and the reported Kappa of 0.885 against humans is good evidence it works.

The soft spots beyond Table 2: the confidence thresholds are inconsistent (main text says 0.8/0.2; Appendix F.1 varies them over 2–8). Section 5.5 has a missing Table?? placeholder for the linear-probing accuracy. OOD evaluation uses string matching only, while in-domain uses the hybrid evaluator—so OOD accuracy may be undercounted. The confidence estimator (sample counts, temperature, variance) is not documented. The 'maintaining helpfulness' claim has no defined helpfulness metric. And the trustworthiness score is used both to select hyperparameters (Appendices F.1/F.2) and to report final results, so the evaluation is partly circular.

None of these individually kill the framework, but Table 2 is load-bearing: the abstract's central improvement is unverifiable until the authors explain the calculation. If they fix that, release the numbers, and add code/data, this becomes a solid paper. As it stands, I would send it to review with a request for major revision—not desk-reject it, because the OOD results are consistent and the framework is useful. I would not cite it in its current form.

Who it is for: people working on hallucination mitigation and refusal behaviors in MLLMs. A serious referee should spend time on it.

Recommendation: conditional—engage, but require the Table 2 fix and a documented confidence estimator.

Referee Report

5 major / 5 minor

Summary. The paper proposes InBoL, an information-boundary-aware learning framework for multimodal large language models (MLLMs) that trains models to refuse answering when information is insufficient. The framework introduces a data construction pipeline that labels training questions as known, mixed, or unknown based on a confidence estimate derived from the base model's sampled-response accuracy, and then applies IDK instruction tuning (IDK-IT) and confidence-aware direct preference optimization (CA-DPO). The paper also proposes a user-centric trustworthiness score, strust = 2*Acc + RefR - 1, and evaluates it on in-domain and out-of-domain datasets with LLaVA1.5-7B/13B, claiming significant trustworthiness improvements while preserving helpfulness.

Significance. If the reported gains are reliable, the paper addresses an important problem in MLLM trustworthiness by systematically combining refusal behavior with confidence-aware training. The proposed hybrid evaluator (string matching plus LLM judgment) is validated against human annotations with Cohen's Kappa 0.885, which is a useful practical contribution. The data-generation pipeline is shown to work with an open-source model (Qwen2-VL) as well as GPT-4o, enhancing reproducibility. However, the central experimental claim is undermined by an internal inconsistency in the main in-domain results table, which prevents the reader from verifying the reported trustworthiness scores.

major comments (5)

[Table 2 and Eq. (5)]
[Table 4 vs Table 2]
[Section 3.2 and Figure 4]
[Section 5.5]
[Appendix E and Section 4.2]

minor comments (5)

[Section 3.1]
[Throughout]
[Section 3.2]
[Tables 5 and 6]
[Section 2.2]

Circularity Check

2 steps flagged · score 4.0 of 10

Evaluation-loop reductions: the self-awareness curve reuses the confidence signal that built the training labels, and the headline configuration was selected by maximizing the reported trustworthiness metric; the central OOD results remain independent.

fitted input called prediction [Section 3.2 (Estimating the Model's Confidence), Eq. (7) (CA-DPO loss), Section 5.3 (Awareness of the Intrinsic Information Boundary), Figure 4(a)]
"we sample multiple responses from the original model and calculate the accuracy rate to estimate its confidence. ... we analyze changes in the refusal rates of both IDK-IT and CA-DPO models relative to the confidence levels of LLaVA1.5-7B ... The results indicate that the model exhibits an awareness of its own confidence, effectively refusing to answer when appropriate."

Section 3.2 uses the base model's sampled-response accuracy as the confidence signal, then thresholds δk/δuk convert that signal into Known/Mixed/Unknown training labels; Eq. (7) then trains with weight conf on the 'answer' preference and (1−conf) on the 'refuse' preference. Section 5.3 bins the same base-model confidence scores and reports that trained models refuse more at low confidence, calling this 'awareness of its own confidence.' The correlation is the training objective itself, not an independent observation: the model was explicitly trained to map low-confidence inputs to refusal. The analysis therefore does not provide separate evidence for intrinsic self-awareness; it restates the construction of the data and loss.
fitted input called prediction [Appendix F.1 (Confidence threshold), Appendix F.2 (Data Composition), Tables 1–2]
"We conducted experiments to analyze the impact of the confidence thresholds δk and δuk. Both thresholds were varied within the range δk, δuk ∈ {2, 3, 4, 5, 6, 7, 8}, ensuring that δk > δuk. ... the combination of δk = 8 and δuk = 2 yields the best performance. ... As shown in Figure 11(b), the optimal trustworthiness score is achieved when the data ratio is 1:1:2."

Appendices F.1 and F.2 select δk=8, δuk=2 and a 1:1:2 data ratio by maximizing 'average trustworthiness scores across in-domain and OOD datasets' on the same evaluation sets reported in Tables 1–2. The headline scores are thus the best point of a hyperparameter sweep over the reported metric, not independent predictions. The improvement of CA-DPO over baselines is partly forced by choosing the configuration that maximizes that exact strust measure on those exact datasets. Without a held-out model-selection set, the main quantitative claim over-reports the method's expected benefit; the numbers are optimized, not predicted.

full rationale

The paper's core training pipeline is not circular in the self-citation sense: it does not lean on prior work by the same authors to justify refusal boundaries, and the OOD evaluations on AOKVQA, GQA, MMMU, MMBench, BeyondVisQA, VizWiz, and VQAv2-IDK provide external measurements of refusal behavior that are independent of the training labels. Two loops, however, compromise the strength of the reported evidence. First, the Section 5.3 self-awareness analysis reuses the exact confidence estimator that generated the training labels and the CA-DPO loss, so the observed refusal-vs-confidence curve is the trained objective rather than an emergent discovery. Second, the main configuration (δk=8, δuk=2, ratio 1:1:2) was selected by maximizing the same trustworthiness metric on the same in-domain and OOD sets that later appear as the headline tables; the reported improvement is therefore a selected maximum rather than an unbiased estimate. Additionally, Table 2's Strust values are not reproducible from Eq. (5) (e.g., LLaVA1.5-7B: 2·12.00+46.10−100=−29.9, not −6.50), an internal inconsistency that further weakens the central quantitative claim. These are not self-citation or definitional circularities in the sense of importing an author's own theorem, but they are input-to-conclusion reductions in the evaluation loop.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on model-derived confidence labels, the proposed user-centric metric, and the correctness of GPT-4o-assisted data generation and filtering. No new physical entities are introduced. The 'information boundary' is a definition rather than an independently testable entity.

free parameters (4)

Confidence thresholds δk and δuk = Reported as δk=0.8, δuk=0.2 in Section 4.1 but as δk=8, δuk=2 in Appendix F.1; Figure 10 uses a 3-8 grid
Chosen by sweeping thresholds in Appendix F.1 to maximize average trustworthiness score; the main results use the best combination. The 0.8/0.2 values in Section 4.1 are inconsistent with the 0-10 confidence scale used elsewhere in the paper.
IDK-IT known-data ratio = 0.75 (about 25% IDK responses)
Appendix F.2 varies the proportion of known data in the 11K instruction set and selects 0.75 because it yields the highest trustworthiness score on in-domain and OOD averages.
CA-DPO data composition ratio (unknown:mixed:known) = 1:1:2
Appendix F.2 sweeps the known-data proportion for CA-DPO and selects 1:1:2 as the optimal balance between accuracy and refusal rate under the proposed trustworthiness metric.
Confidence estimation sampling configuration = Unspecified
Section 3.2 says the model samples multiple responses to estimate confidence, but the paper does not state how many samples are drawn or at what temperature. This directly determines the known/unknown labels used in training.

assumptions (5)

domain assumption A model's confidence can be estimated by sampling multiple responses and computing accuracy against a hybrid evaluator.
Section 3.2: 'we sample multiple responses from the original model and calculate the accuracy rate to estimate its confidence.' All known/unknown training labels depend on this proxy, so a biased estimate would train refusal at the wrong boundary.
domain assumption All questions in the base VQA datasets are answerable from the provided visual information.
Section 3.2 states this assumption to place the data within the extrinsic information boundary. It may not hold for knowledge-intensive items in Oven or ScienceQA, which can require external knowledge.
domain assumption User preferences are captured by v(correct)=1, v(refusal)=0, v(incorrect)=-1, equivalently maximizing 2*Acc+RefR-1.
Section 2.1 introduces this as a user-centric alignment goal without a user study. The entire evaluation and the trustworthiness score rest on this particular weighting of correctness, refusal, and error.
domain assumption GPT-4o-generated unanswerable questions and the three-criteria filter correctly identify true extrinsic-boundary violations.
Section 3.2 and Appendix C.2 keep only questions GPT-4o marks as unanswerable and exclude questions the base model already refuses. Errors in this filtering would mislabel training examples.
domain assumption The hybrid evaluator (string matching plus Llama2-13B judgments) is a reliable proxy for human correctness judgment.
Appendix C.1 reports Cohen's Kappa of 0.885 on 200 samples and uses this evaluator for in-domain scoring, while OOD scoring uses string matching alone, which is weaker.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Drawing the Line: Enhancing Trustworthiness of MLLMs Through the Power of Refusal." pith.science (2026). https://pith.science/paper/FH2KGVYP

@misc{pith2026241211196,
  author       = {Pith},
  title        = {Pith review of: Drawing the Line: Enhancing Trustworthiness of MLLMs Through the Power of Refusal},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FH2KGVYP}},
  note         = {Machine review of arXiv:2412.11196}
}

read the original abstract

Multimodal large language models (MLLMs) excel at multimodal perception and understanding, yet their tendency to generate hallucinated or inaccurate responses undermines their trustworthiness. Existing methods have largely overlooked the importance of refusal responses as a means of enhancing MLLMs reliability. To bridge this gap, we present the Information Boundary-aware Learning Framework (InBoL), a novel approach that empowers MLLMs to refuse to answer user queries when encountering insufficient information. To the best of our knowledge, InBoL is the first framework that systematically defines the conditions under which refusal is appropriate for MLLMs using the concept of information boundaries proposed in our paper. This framework introduces a comprehensive data generation pipeline and tailored training strategies to improve the model's ability to deliver appropriate refusal responses. To evaluate the trustworthiness of MLLMs, we further propose a user-centric alignment goal along with corresponding metrics. Experimental results demonstrate a significant improvement in refusal accuracy without noticeably compromising the model's helpfulness, establishing InBoL as a pivotal advancement in building more trustworthy MLLMs.

Figures

Figures reproduced from arXiv: 2412.11196 by the authors.

**Figure 2.** The Pipeline of Data Construction: Given a VQA dataset, we design a pipeline to collect different types [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** Construction of ‘IDK’ instruction and preference data: The restructured data is categorized into ‘Known,’ [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Figures from the paper (9 more)

**Figure 4.** Figure 4: Refusal rate and accuracy of models across different confidence levels. (a) Refusal Rate by Confidence: [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]

**Figure 5.** Figure 5: Predefined refusal template of unanswerable questions. Liu et al. (2023a) proposed three types of negative instructions involving misleading or false premises in images, which models must learn to refuse. Cha et al. (2024) introduce the VQAv2-IDK dataset, which also…

**Figure 6.** Figure 6: LLM prompt for our hybrid evaluation, We use Llama2-13B for LLM evaluation. [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]

**Figure 7.** Figure 7: Example 1 and Example 2 demonstrate the effectiveness of our hybrid evaluator. “England” and “the [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]

**Figure 8.** Figure 8: Prompt for GPT-4o to Generate Unanswerable Questions [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]

**Figure 9.** Figure 9: Examples of unanswerable questions. Method Model for AOKVQA GQA MMMU MMBench(en-dev) data generation Acc RefR Strust Acc RefR Strust Acc RefR Strust Acc RefR Strust IDK-IT GPT-4o 55.50 36.24 47.24 50.46 23.88 24.81 15.22 69.67 0.11 46.39 39.09 31.87 IDK-IT Qwen2-VL-72B…

**Figure 10.** Figure 10: Impact of confidence thresholds on perfor [PITH_FULL_IMAGE:figures/full_fig_p018_10.png]

**Figure 11.** Figure 11: Data Composition Analysis. (a) For IDK-IT, varying the ratio of "known" data shows that a proportion [PITH_FULL_IMAGE:figures/full_fig_p019_11.png]

**Figure 12.** Figure 12: Examples illustrating the comparison between LLaVA1.5-7B and models trained with our framework [PITH_FULL_IMAGE:figures/full_fig_p020_12.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

16 extracted references · 12 canonical work pages

[1]

Does this question inquire about subjects that are not depicted in the image?

work page
[2]

Advances in Neu- ral Information Processing Systems, 36

Direct preference optimization: Your language model is secretly a reward model. Advances in Neu- ral Information Processing Systems, 36. Dustin Schwenk, Apoorv Khandelwal, Christopher Clark, Kenneth Marino, and Roozbeh Mottaghi. 2022. A-okvqa: A benchmark for visual question answer- ing using world knowledge. In European conference on computer vision, pag...

arXiv 2022
[3]

arXiv preprint arXiv:2408.04840

mplug-owl3: Towards long image-sequence understanding in multi-modal large language models. arXiv preprint arXiv:2408.04840. Zhangyue Yin, Qiushi Sun, Qipeng Guo, Jiawen Wu, Xipeng Qiu, and Xuanjing Huang. 2023. Do large language models know what they don’t know? In Findings of the Association for Computational Lin- guistics: ACL 2023 , pages 8653–8665, T...

arXiv 2023
[4]

arXiv preprint arXiv:2401.10020

Self-rewarding language models. arXiv preprint arXiv:2401.10020. 12 Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, Cong Wei, Botao Yu, Ruibin Yuan, Renliang Sun, Ming Yin, Boyuan Zheng, Zhenzhu Yang, Yibo Liu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. 2024. Mmmu: A massive...

arXiv 2024
[5]

is a dataset designed for real-world visual reasoning and compositional question answering

GQA (Hudson and Manning, 2019) is a dataset for real-world visual reasoning and compositional question answering. is a dataset designed for real-world visual reasoning and compositional question answering

work page 2019
[6]

MMMU (Yue et al., 2024) is a benchmark developed to assess multimodal models across a variety of complex, multidisciplinary tasks that require college-level subject knowledge and advanced reasoning

work page 2024
[7]

Does this question include an incorrect or mis- leading premise?

work page
[8]

Questions that receive a ‘no’ for all three criteria are filtered out

Does this question ask for information that is not available in the image? Given the generated question and its corresponding image, we prompt GPT-4 to verify whether the question meets the specified criteria. Questions that receive a ‘no’ for all three criteria are filtered out. Additionally, we prompt the original model to generate a response to the una...

work page

Show all 16 references

[9]

VQAv2 (Antol et al., 2015) is a widely-used dataset containing open-ended questions re- lated to images, aimed at evaluating visual question answering

2015
[10]

OVEN (Hu et al., 2023) contains open- domain visual entity questions based on Wikipedia entries, requiring the model to pos- sess extensive visual knowledge to provide accurate answers

2023
[11]

ScienceQA (Lu et al., 2022) comprises mul- timodal, multiple-choice questions across a diverse array of scientific topics

2022
[12]

AOKVQA (Schwenk et al., 2022) is a crowd- sourced dataset featuring a wide range of ques- tions that demand a broad understanding of commonsense and world knowledge

2022
[15]

MMBench (Liu et al., 2023e)is a comprehen- sive benchmark for evaluating the multimodal capabilities of MLLMs, featuring questions that challenge both reasoning and perception
[16]

known unknowns

BeyondVisQA (Wang et al., 2024b)is specif- ically designed to evaluate the self-awareness of MLLMs, particularly their ability to rec- ognize “known unknowns." The questions in this dataset require information beyond the information provided by the input images. To construct t...
[142]

I don’t know,

Springer. Weihong Zhong, Xiaocheng Feng, Liang Zhao, Qim- ing Li, Lei Huang, Yuxuan Gu, Weitao Ma, Yuan Xu, and Bing Qin. 2024. Investigating and miti- gating the multimodal hallucination snowballing in large vision-language models. In Proceedings of the 62nd Annual Meeting of...

2024
[2024]

arXiv preprint arXiv:2404.18930

Hallucination of multimodal large language models: A survey. arXiv preprint arXiv:2404.18930. Sungguk Cha, Jusung Lee, Younghyun Lee, and Che- oljong Yang. 2024. Visually dehallucinative instruc- tion generation. In ICASSP 2024-2024 IEEE Interna- tional Conference on Acoustics...

2024 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

Does this question inquire about subjects that are not depicted in the image?

work page

[2] [2]

Advances in Neu- ral Information Processing Systems, 36

Direct preference optimization: Your language model is secretly a reward model. Advances in Neu- ral Information Processing Systems, 36. Dustin Schwenk, Apoorv Khandelwal, Christopher Clark, Kenneth Marino, and Roozbeh Mottaghi. 2022. A-okvqa: A benchmark for visual question answer- ing using world knowledge. In European conference on computer vision, pag...

arXiv 2022

[3] [3]

arXiv preprint arXiv:2408.04840

mplug-owl3: Towards long image-sequence understanding in multi-modal large language models. arXiv preprint arXiv:2408.04840. Zhangyue Yin, Qiushi Sun, Qipeng Guo, Jiawen Wu, Xipeng Qiu, and Xuanjing Huang. 2023. Do large language models know what they don’t know? In Findings of the Association for Computational Lin- guistics: ACL 2023 , pages 8653–8665, T...

arXiv 2023

[4] [4]

arXiv preprint arXiv:2401.10020

Self-rewarding language models. arXiv preprint arXiv:2401.10020. 12 Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, Cong Wei, Botao Yu, Ruibin Yuan, Renliang Sun, Ming Yin, Boyuan Zheng, Zhenzhu Yang, Yibo Liu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. 2024. Mmmu: A massive...

arXiv 2024

[5] [5]

is a dataset designed for real-world visual reasoning and compositional question answering

GQA (Hudson and Manning, 2019) is a dataset for real-world visual reasoning and compositional question answering. is a dataset designed for real-world visual reasoning and compositional question answering

work page 2019

[6] [6]

MMMU (Yue et al., 2024) is a benchmark developed to assess multimodal models across a variety of complex, multidisciplinary tasks that require college-level subject knowledge and advanced reasoning

work page 2024

[7] [7]

Does this question include an incorrect or mis- leading premise?

work page

[8] [8]

Questions that receive a ‘no’ for all three criteria are filtered out

Does this question ask for information that is not available in the image? Given the generated question and its corresponding image, we prompt GPT-4 to verify whether the question meets the specified criteria. Questions that receive a ‘no’ for all three criteria are filtered out. Additionally, we prompt the original model to generate a response to the una...

work page