MLLMs show late-layer textual override of correct visual predictions, with a directional signature enabling a simple inference-time recovery method that improves conflict benchmarks by up to 9.4%.
How easy is it to fool your multimodal llms? an empirical analysis on deceptive prompts.arXiv preprint arXiv:2402.13220, 2 (7)
2 Pith papers cite this work. Polarity classification is still indexing.
2
Pith papers citing it
fields
cs.CV 2representative citing papers
A new 1,248-instance paired benchmark shows 18 vision-language models drop substantially in accuracy when images contain misleading visual cues.
citing papers explorer
-
MLLMs Get It Right, Then Get It Wrong: Tracing and Correcting Late-Layer Textual Bias
MLLMs show late-layer textual override of correct visual predictions, with a directional signature enabling a simple inference-time recovery method that improves conflict benchmarks by up to 9.4%.
-
MVI-Bench: A Comprehensive Benchmark for Evaluating Robustness to Misleading Visual Inputs in LVLMs
A new 1,248-instance paired benchmark shows 18 vision-language models drop substantially in accuracy when images contain misleading visual cues.