Pith. sign in

REVIEW 4 major objections 5 minor 32 references

Hydra: An Agentic Reasoning Approach for Enhancing Adversarial Robustness and Mitigating Hallucinations in Vision-Language Models

T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Agentic verification curbs VLM hallucinations and survives adversarial image attacks.

desk verdict Hydra's clean dehallucination results are strong and the agentic combination is new, but the adversarial-robustness headline depends on auxiliary models that are never put under attack. read the letter →

arxiv 2504.14395 v1 pith:XRVS4RSK submitted 2025-04-19 cs.CV cs.AIcs.MA

Chung-En (Johnny) Yu , Hsuan-Chih (Neil) Chen , Brian Jalaian , Nathaniel D. Bastian This is my paper · ORCID
classification cs.CVcs.AIcs.MA
keywords agenticreasoningvision-languagemodelshallucinationmitigationadversarialrobustnessobject-leveljointembeddingspaceattackschain-of-thoughtin-contextlearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes Hydra, a training-free wrapper that places a large language model agent around any plug-in vision-language model (VLM). The agent repeatedly asks the VLM and a small suite of auxiliary vision models to describe or verify image content, critiques the answers, and refines its final response. Hydra targets object-level hallucinations—false claims that an object is present—and aims to keep outputs correct when images are adversarially perturbed in the joint embedding space. On POPE, MME-Existence, and AMBER-Generative, across four LVLMs, the paper reports that Hydra outperforms both the bare models and two post-hoc dehallucination methods, on clean and attacked inputs, with no adversarial training.

What carries the argument

The load-bearing mechanism is the Action-Critique Loop, an iterative cycle in which a single LLM agent moves between an Action Space (issuing queries to a vision-language suite) and a Critique Space (assessing consistency and factual correctness), with a shared memory of retrieved information. Five subtasks—initial perceptual querying, adaptive model critique, critique-driven decision-making, attribute-based inquiry formulation, and cross-model object discovery—turn the loop into a complete verification pipeline. The auxiliary suite's diversity is what the paper credits for robustness: because the agent fuses textual evidence from models with different visual encoders, an attack that fools one model need not fool the others.

What would settle it

Run Adversarial Illusions and AttackVLM directly against the auxiliary suite (DETR, Paligemma, BLIPvqa, and the captioning model) and show that their object-presence answers shift on attacked images; if those shifts propagate into the agent's final answers, Hydra's robustness would not exceed the weakest auxiliary model. A cleaner test: adversarially attack the auxiliary models alone and measure whether Hydra's final accuracy degrades proportionally.

Watch

Extended reading notes

Core claim

Hydra's central claim is that a single agentic loop can unify two problems usually treated separately: resisting adversarial perturbations and suppressing intrinsic hallucinations. The agent has no direct access to the image; it reasons only over textual outputs from the plug-in VLM and from auxiliary models. When initial object-presence judgments conflict, the agent generates attribute-based questions and queries multiple models again, iterating until the critiques agree. The paper reports gains such as raising one model's POPE-Random accuracy from 54% to 94.7%, sustaining above 90% accuracy on two other models, and retaining the highest accuracy after two adversarial attacks, without any explicit defense.

Load-bearing premise

Hydra's reliability rests on the auxiliary vision models remaining trustworthy under attack, since the agent never sees the image and can only read their textual reports.

Editorial extensions

If this is right

  • If Hydra's results hold, a training-free wrapper can improve an LVLM's factual consistency on object-presence questions without modifying the LVLM's weights.
  • Adversarial robustness and hallucination mitigation can be addressed by the same mechanism: cross-model textual verification makes the system harder to fool in both settings.
  • Small auxiliary models under 3B parameters can provide large robustness gains, which matters for deployment under computational constraints.
  • Applying standard preprocessing defenses like JPEG compression and feature squeezing changes Hydra's trade-off: hallucination drops further but object coverage falls, so the choice of defense must consider the task.
  • Hydra's interpretable critiques give users a human-readable rationale for each answer, a direct corollary of the agent's structured decision trail.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not report attack success on the auxiliary suite; a natural extension would be adaptive attacks specifically targeting DETR, Paligemma, and BLIPvqa, since Hydra's claimed robustness would collapse if those models are also fooled.
  • Because POPE, MME, and AMBER all draw from COCO images and classes, DETR's strong performance may partly reflect closed-set familiarity with those categories; testing on out-of-distribution object sets would clarify whether the mechanism generalizes.
  • The agent's reliance on attribute-based questions suggests the loop could extend beyond object-level verification to attribute- and relationship-level hallucinations if the auxiliary models can answer those question types reliably.
  • Hydra's inference cost grows with each iteration, so a test-time-compute versus accuracy trade-off curve, rather than a single accuracy number, would be the more informative deployment comparison.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Hydra, a training-free agentic framework that wraps plug-in vision-language models (VLMs) and refines their outputs through an iterative Action-Critique Loop. Hydra queries a suite of additional vision models (DETR, Paligemma, BLIPvqa, and Llama-3.2-Vision-Instruct) using only their textual outputs, then critiques and reconciles evidence from multiple sources to reduce object-level hallucinations and resist adversarial perturbations. The authors evaluate Hydra on four VLMs (mPLUG-Owl, LLaVA-1.5, MiniGPT-4, Qwen-VL-Chat), three hallucination benchmarks (POPE, MME-Existence, AMBER-Generative), two adversarial attacks (Adversarial Illusions, AttackVLM), and two preprocessing defenses (JPEG compression, feature squeezing). The main claimed results are consistent improvements in accuracy and F1 on POPE subsets across all four base VLMs, and a claim that Hydra maintains higher accuracy than plug-in VLMs and existing dehallucination methods even when the input images are adversarially perturbed.

Significance. If the adversarial-robustness claim were fully supported, the paper would make a meaningful contribution: it would show that a model-agnostic, training-free wrapper can simultaneously mitigate object-level hallucinations and resist joint-embedding adversarial attacks. The hallucination-mitigation results on POPE are consistent and reasonably convincing, and the integration of agentic reasoning with multi-model verification is a sensible research direction. The paper also usefully evaluates the interaction of preprocessing defenses with a hallucination-correction pipeline. However, the adversarial-robustness claim currently rests on an unexamined assumption that the auxiliary vision models remain reliable under attack, which is load-bearing because the agent never sees the image. That gap, plus the selective reporting of results, prevents the paper from fully substantiating its headline claim.

major comments (4)
  1. [3.1, 4.1.3] The adversarial-robustness claim is not adequately supported because the paper never measures attack success on the auxiliary vision models. Hydra's agent relies entirely on textual outputs from DETR, Paligemma, BLIPvqa, and the Llama-3.2 captioner (Section 4.1.3), and Section 3.1 asserts that a diverse suite 'lowers the risk of compromising to the adversarial attacks' without providing any per-model attack success rates, oracle controls, or analysis of how often the agent is misled by compromised auxiliary outputs. Since the injected attacks target the joint embedding space and are known to transfer across encoders, the observed robustness could be an artifact of the auxiliary models being unintentionally robust rather than a property of Hydra's reasoning. Please report attack success on each auxiliary model, or provide a control condition in which the auxiliary outputs are intentionally corrupted, to demonstrate that Hydra's cross-model verification is actually the source of robustness.
  2. [Figure 3, Tables 3–5] The evidence for adversarial robustness is presented on a selective subset of the claimed evaluation grid. Figure 3 shows only LLaVA-1.5 on POPE-Popular under Adversarial Illusions and Qwen-VL-Chat on POPE-Adversarial under AttackVLM; Tables 3–5 omit LLaVA-1.5 entirely and cover only two of the four base VLMs on MME and AMBER. The abstract claims evaluation across four VLMs, three benchmarks, and two attacks, but the reader cannot verify the full cross product. Please provide complete results for all model–benchmark–attack combinations, or clearly state in the text which combinations were omitted and why.
  3. [4.1.3] The design conflates the agent backbone with one of the evidence sources: Llama-3.2-Vision-Instruct serves both as Hydra's reasoning agent and as the captioning model in the visual-language suite. This creates a possible failure mode in which the model used for reasoning and the model used for verification share the same vulnerabilities, weakening the claim that cross-model verification provides diverse independent evidence. The paper should either use a different captioning model or explicitly discuss how this shared-model dependency affects the robustness and diversity of the evidence suite.
  4. [4.2.4, Tables 4 and 5] The paper reports that performance sometimes improves after attack, which it labels an 'open question.' This may be an artifact of small sample sizes (e.g., 50 AMBER images and 300 POPE questions per subset) combined with the absence of statistical significance tests. Given that the main conclusions rely on differences of a few percentage points, please report confidence intervals or significance tests, and explain the post-attack improvement with a concrete analysis rather than leaving it as an unexplained observation.
minor comments (5)
  1. [References, [19]] The citation for the MME benchmark is incorrect: [19] is a survey of multimodal large language models, not the MME evaluation paper by Fu et al. Please replace it with the appropriate reference.
  2. [3.1] The Action-Critique Loop is described only at a high level. Please provide the concrete prompt templates, the iteration-limit value, the consistency threshold used in the critiques, and the exact decision rule for when the agent finalizes an answer or enters the loop. This is needed for reproducibility.
  3. [4.1.3] The paper does not mention whether code or model configurations will be released. Given the large number of components (four base VLMs, three auxiliary models, two attacks, two defenses), a reproducibility appendix with the exact inference scripts and random seeds would strengthen the paper.
  4. [1] There is a typo in the introduction: 'approches' should be 'approaches.' Additionally, several reference entries in the bibliography appear to lack venue information or have inconsistent formatting (e.g., [1], [19]); a thorough proofread of the reference list is recommended.
  5. [5] The Limitations section mentions increased inference time but provides no quantitative estimate. Since Hydra involves iterative reasoning and multiple model queries, a brief measurement of wall-clock time or number of iterations per query would help readers gauge practical applicability.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: Hydra is an empirical agentic pipeline; the agent-backbone/suite-captioner overlap and unverified auxiliary-model integrity are robustness concerns, not reductions by construction.

full rationale

Hydra is an empirical inference-time framework rather than a formal derivation, and its reported gains are benchmark measurements, so the circularity burden is low. The pipeline contains no fitted parameters and no equation linking a fitted quantity to a predicted quantity; the agent's answers are produced by prompting an LLM with textual outputs from DETR, Paligemma, BLIPvqa, and captioners (Sections 3.1 and 4.1.3). The closest thing to a self-reference is that Llama-3.2-Vision-Instruct is both the agent backbone and the suite's captioning model, which weakens the 'cross-model' independence claim but does not make the output equivalent to the input by construction; likewise, the unverified assumption that auxiliary models stay intact under transferable attacks (the Section 3.1 diversity assertion, with no per-model attack success reported) is an empirical robustness concern, not a circularity. The dehallucination baselines are re-implemented with the same correction model for comparability, which is a control rather than a circular justification. No quoted equation, fitted parameter, or self-citation chain forces the reported conclusion, so under the hard-evidence rule the score is 0; the auxiliary-model integrity gap and the agent-backbone overlap should be addressed as correctness and limitation issues, not circularity.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

Hydra introduces no new mathematical entities. The ledger contains hyperparameters (iteration limit, consistency threshold) and domain assumptions about the auxiliary models, the object detector's training overlap, the agent's reasoning reliability, and benchmark validity. The COCO overlap between DETR and the benchmarks is the most consequential hidden dependency.

free parameters (2)
  • Action-Critique Loop iteration limit
    The loop stops after a 'predefined iteration limit' (Section 3.1). The value is not reported and likely trades accuracy against inference cost; the central performance numbers depend on when the loop terminates.
  • Critique consistency threshold
    The agent decides whether two critiques are 'consistent' enough to finalize (Section 3.1, VQA workflow step 3). No formal rule or threshold is given, leaving a hand-chosen judgment call inside the agent.
assumptions (4)
  • domain assumption Auxiliary vision models (DETR, Paligemma, BLIPvqa, Llama-3.2 captioner) are not all fooled by the adversarial perturbation.
    Section 3.1 states the diverse suite 'lowers the risk' of compromise, but the paper never measures attack success on these models. If the attack transfers to all of them, Hydra's cross-model verification fails.
  • domain assumption The closed-set object detector DETR reliably detects the objects queried in POPE, MME, and AMBER.
    POPE/MME/AMBER use COCO images and object classes, which is DETR's training distribution. The paper does not test out-of-distribution objects or acknowledge this favorable overlap (Section 4.1.3).
  • domain assumption The LLM agent (Llama-3.2-Vision-Instruct) correctly aggregates and critiques textual evidence without seeing the image.
    The agent's reasoning accuracy is not evaluated in isolation; the whole framework assumes the agent's binary existence decisions and follow-up questions are sound (Section 3.1).
  • domain assumption The benchmarks (POPE, MME, AMBER) are valid measures of object-level hallucination.
    The paper adopts these benchmarks without questioning their annotation reliability or coverage (Section 4.1.2). This is standard practice, but it is an assumption about measurement validity.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Hydra: An Agentic Reasoning Approach for Enhancing Adversarial Robustness and Mitigating Hallucinations in Vision-Language Models." pith.science (2026). https://pith.science/paper/XRVS4RSK

@misc{pith2026250414395,
  author       = {Pith},
  title        = {Pith review of: Hydra: An Agentic Reasoning Approach for Enhancing Adversarial Robustness and Mitigating Hallucinations in Vision-Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XRVS4RSK}},
  note         = {Machine review of arXiv:2504.14395}
}
read the original abstract

To develop trustworthy Vision-Language Models (VLMs), it is essential to address adversarial robustness and hallucination mitigation, both of which impact factual accuracy in high-stakes applications such as defense and healthcare. Existing methods primarily focus on either adversarial defense or hallucination post-hoc correction, leaving a gap in unified robustness strategies. We introduce \textbf{Hydra}, an adaptive agentic framework that enhances plug-in VLMs through iterative reasoning, structured critiques, and cross-model verification, improving both resilience to adversarial perturbations and intrinsic model errors. Hydra employs an Action-Critique Loop, where it retrieves and critiques visual information, leveraging Chain-of-Thought (CoT) and In-Context Learning (ICL) techniques to refine outputs dynamically. Unlike static post-hoc correction methods, Hydra adapts to both adversarial manipulations and intrinsic model errors, making it robust to malicious perturbations and hallucination-related inaccuracies. We evaluate Hydra on four VLMs, three hallucination benchmarks, two adversarial attack strategies, and two adversarial defense methods, assessing performance on both clean and adversarial inputs. Results show that Hydra surpasses plug-in VLMs and state-of-the-art (SOTA) dehallucination methods, even without explicit adversarial defenses, demonstrating enhanced robustness and factual consistency. By bridging adversarial resistance and hallucination mitigation, Hydra provides a scalable, training-free solution for improving the reliability of VLMs in real-world applications.

Figures

Figures reproduced from arXiv: 2504.14395 by the authors.

Figure 1
Figure 1. An overview of Hydra improving VLM robustness by addressing hallucination errors and adversarial [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. The Hydra framework enhances VLM robustness through an iterative Action-Critique Loop, integrating [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The comparison of different dehallucination methods before and after applying adversarial attacks on POPE. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

32 extracted references · 3 canonical work pages

  1. [1]

    Seeing is deceiving: Exploitation of visual pathways in multi-modal language models

    Pete Janowczyk, Linda Laurier, Ave Giulietta, Arlo Octavia, and Meade Cleti. Seeing is deceiving: Exploitation of visual pathways in multi-modal language models. arXiv preprint arXiv:2411.05056, 2024

    arXiv 2024

  2. [2]

    A survey on hallucination in large vision-language models

    Hanchao Liu, Wenyuan Xue, Yifei Chen, Dapeng Chen, Xiutian Zhao, Ke Wang, Liping Hou, Rongjun Li, and Wei Peng. A survey on hallucination in large vision-language models. arXiv preprint arXiv:2402.00253, 2024

    arXiv 2024

  3. [3]

    Hallucina- tion of multimodal large language models: A survey

    Zechen Bai, Pichao Wang, Tianjun Xiao, Tong He, Zongbo Han, Zheng Zhang, and Mike Zheng Shou. Hallucina- tion of multimodal large language models: A survey. arXiv preprint arXiv:2404.18930, 2024

    arXiv 2024

  4. [4]

    On evaluating adversarial robustness of large vision-language models

    Yunqing Zhao, Tianyu Pang, Chao Du, Xiao Yang, Chongxuan Li, Ngai-Man Man Cheung, and Min Lin. On evaluating adversarial robustness of large vision-language models. Advances in Neural Information Processing Systems, 36:54111–54138, 2023

    2023

  5. [5]

    Adversarial illusions in{Multi-Modal} embeddings

    Eugene Bagdasaryan, Rishi Jha, Vitaly Shmatikov, and Tingwei Zhang. Adversarial illusions in{Multi-Modal} embeddings. In 33rd USENIX Security Symposium (USENIX Security 24), pages 3009–3025, 2024

    work page 2024

  6. [6]

    Woodpecker: Hallucination correction for multimodal large language models

    Shukang Yin, Chaoyou Fu, Sirui Zhao, Tong Xu, Hao Wang, Dianbo Sui, Yunhang Shen, Ke Li, Xing Sun, and Enhong Chen. Woodpecker: Hallucination correction for multimodal large language models. Science China Information Sciences, 67(12):220105, 2024

    2024

  7. [7]

    Logical closed loop: Uncovering object hallucinations in large vision-language models

    Junfei Wu, Qiang Liu, Ding Wang, Jinghao Zhang, Shu Wu, Liang Wang, and Tieniu Tan. Logical closed loop: Uncovering object hallucinations in large vision-language models. arXiv preprint arXiv:2402.11622, 2024

    arXiv 2024

  8. [8]

    React: Synergizing reasoning and acting in language models

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629, 2022

    arXiv 2022

Show all 32 references
  1. [9]

    Critic: Large language models can self-correct with tool-interactive critiquing

    Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Nan Duan, and Weizhu Chen. Critic: Large language models can self-correct with tool-interactive critiquing. arXiv preprint arXiv:2305.11738, 2023

    2023 arXiv

  2. [10]

    A study of the effect of jpg compression on adversarial images

    Gintare Karolina Dziugaite, Zoubin Ghahramani, and Daniel M Roy. A study of the effect of jpg compression on adversarial images. arXiv preprint arXiv:1608.00853, 2016

    2016 arXiv

  3. [11]

    Feature squeezing: Detecting adversarial examples in deep neural networks

    Weilin Xu, David Evans, and Yanjun Qi. Feature squeezing: Detecting adversarial examples in deep neural networks. arXiv preprint arXiv:1704.01155, 2017

    2017 arXiv

  4. [12]

    Adver- sarial attacks and defences: A survey

    Anirban Chakraborty, Manaar Alam, Vishal Dey, Anupam Chattopadhyay, and Debdeep Mukhopadhyay. Adver- sarial attacks and defences: A survey. arXiv preprint arXiv:1810.00069, 2018

    2018 arXiv

  5. [13]

    One prompt word is enough to boost adversarial robustness for pre-trained vision-language models

    Lin Li, Haoyan Guan, Jianing Qiu, and Michael Spratling. One prompt word is enough to boost adversarial robustness for pre-trained vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24408–24419, 2024

    2024

  6. [14]

    An interactive agent foundation model

    Zane Durante, Bidipta Sarkar, Ran Gong, Rohan Taori, Yusuke Noda, Paul Tang, Ehsan Adeli, Shrinidhi Kowshika Lakshmikanth, Kevin Schulman, Arnold Milstein, et al. An interactive agent foundation model. arXiv preprint arXiv:2402.05929, 2024

    2024 arXiv

  7. [15]

    Automatically correcting large language models: Surveying the landscape of diverse self-correction strategies

    Liangming Pan, Michael Saxon, Wenda Xu, Deepak Nathani, Xinyi Wang, and William Yang Wang. Automatically correcting large language models: Surveying the landscape of diverse self-correction strategies. arXiv preprint arXiv:2308.03188, 2023

    2023 arXiv

  8. [16]

    Doubly-universal adversarial perturbations: Deceiving vision-language models across both images and text with a single perturbation

    Hee-Seon Kim, Minbeom Kim, and Changick Kim. Doubly-universal adversarial perturbations: Deceiving vision-language models across both images and text with a single perturbation. arXiv preprint arXiv:2412.08108, 2024

    2024 arXiv

  9. [17]

    Vlattack: Multimodal adversarial attacks on vision-language tasks via pre-trained models

    Ziyi Yin, Muchao Ye, Tianrong Zhang, Tianyu Du, Jinguo Zhu, Han Liu, Jinghui Chen, Ting Wang, and Fenglong Ma. Vlattack: Multimodal adversarial attacks on vision-language tasks via pre-trained models. Advances in Neural Information Processing Systems, 36:52936–52956, 2023

    2023

  10. [18]

    Evaluating object hallucination in large vision-language models

    Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. Evaluating object hallucination in large vision-language models. arXiv preprint arXiv:2305.10355, 2023. 11 Chung-En (Johnny) Yu et al

    2023 arXiv

  11. [19]

    A survey of multimodel large language models

    Zijing Liang, Yanjie Xu, Yifan Hong, Penghui Shang, Qi Wang, Qiang Fu, and Ke Liu. A survey of multimodel large language models. In Proceedings of the 3rd International Conference on Computer, Artificial Intelligence and Control Engineering, pages 405–409, 2024

    2024

  12. [20]

    Amber: An llm-free multi-dimensional benchmark for mllms hallucination evaluation

    Junyang Wang, Yuhang Wang, Guohai Xu, Jing Zhang, Yukai Gu, Haitao Jia, Jiaqi Wang, Haiyang Xu, Ming Yan, Ji Zhang, et al. Amber: An llm-free multi-dimensional benchmark for mllms hallucination evaluation. arXiv preprint arXiv:2311.07397, 2023

    2023 arXiv

  13. [21]

    mplug-owl: Modularization empowers large language models with multimodality

    Qinghao Ye, Haiyang Xu, Guohai Xu, Jiabo Ye, Ming Yan, Yiyang Zhou, Junyang Wang, Anwen Hu, Pengcheng Shi, Yaya Shi, et al. mplug-owl: Modularization empowers large language models with multimodality. arXiv preprint arXiv:2304.14178, 2023

    2023 arXiv

  14. [22]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26296–26306, 2024

    2024

  15. [23]

    Minigpt-4: Enhancing vision-language understanding with advanced large language models

    Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592, 2023

    2023 arXiv

  16. [24]

    Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond. arXiv preprint arXiv:2308.12966, 2023

    2023 arXiv

  17. [25]

    End-to-end object detection with transformers

    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In European conference on computer vision , pages 213–229. Springer, 2020

    2020

  18. [26]

    Paligemma: A versatile 3b vlm for transfer

    Lucas Beyer, Andreas Steiner, André Susano Pinto, Alexander Kolesnikov, Xiao Wang, Daniel Salz, Maxim Neumann, Ibrahim Alabdulmohsin, Michael Tschannen, Emanuele Bugliarello, et al. Paligemma: A versatile 3b vlm for transfer. arXiv preprint arXiv:2407.07726, 2024

    2024 arXiv

  19. [27]

    Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In International conference on machine learning, pages 12888–12900. PMLR, 2022

    2022

  20. [28]

    Llama 3.2-11b vision

    Meta AI. Llama 3.2-11b vision. https://huggingface.co/meta-llama/Llama-3.2-11B-Vision , 2025. Accessed: 2025-03-07

    2025

  21. [29]

    Imagebind: One embedding space to bind them all

    Rohit Girdhar, Alaaeldin El-Nouby, Zhuang Liu, Mannat Singh, Kalyan Vasudev Alwala, Armand Joulin, and Ishan Misra. Imagebind: One embedding space to bind them all. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 15180–15190, 2023

    2023

  22. [30]

    One transformer fits all distributions in multi-modal diffusion at scale

    Fan Bao, Shen Nie, Kaiwen Xue, Chongxuan Li, Shi Pu, Yaole Wang, Gang Yue, Yue Cao, Hang Su, and Jun Zhu. One transformer fits all distributions in multi-modal diffusion at scale. In International Conference on Machine Learning, pages 1692–1717. PMLR, 2023

    2023

  23. [31]

    Microsoft coco captions: Data collection and evaluation server

    Xinlei Chen, Hao Fang, Tsung-Yi Lin, Ramakrishna Vedantam, Saurabh Gupta, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco captions: Data collection and evaluation server. arXiv preprint arXiv:1504.00325, 2015

    2015 arXiv

  24. [32]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 12

    2022

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.