Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

Retrieval Visual Contrastive Decoding to Mitigate Object Hallucinations in Large Vision-Language Models

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Train-free decoding with retrieved single-concept images claims to cut object hallucination in vision-language models.

desk verdict CHAIR results look real and the method is a genuine extension of VCD, but the POPE numbers are contaminated by YOLO answering for the LVLM, so the headline claims need re-scoping. read the letter →

arxiv 2505.20569 v2 pith:OCVW2SI6 submitted 2025-05-26 cs.CV cs.AIcs.LG

Jihoon Lee , Min Song This is my paper
classification cs.CVcs.AIcs.LG
keywords objecthallucinationlargevision-languagemodelscontrastivedecodingretrieval-augmentedCHAIRmetricPOPEevaluationimagecaptioningtraining-free
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

RVCD is a training-free decoding method that claims to substantially reduce object hallucination in large vision-language models while keeping caption quality. The idea is to let an object detector audit a first greedy caption, split the mentioned objects into those the detector confirms and those it does not, and then steer every subsequent decoding step with logits computed from retrieved images that each show one of those objects alone. The paper reports CHAIR_S dropping from 18.60 with the HALC baseline to 11.32 on LLaVA-1.5, and from 15.36 to 9.00 on MiniGPT-4, with BLEU close to baseline. The method is plug-and-play because it changes only the decoding rule, not the model weights.

What carries the argument

The carrying mechanism is the adjusted logit distribution of Eq. 5, which treats negative-image logits as a penalty term that suppresses tokens associated with objects the detector says are absent, and positive-image logits as a recovery term that keeps co-occurring ground-truth objects from being collateral damage; the hyperparameters $\alpha=1$ and $\beta=0.1$ balance the two. The support structure is the single-concept image database, one retrieved image per object word, which makes each penalty and recovery signal refer to a visually explicit concept.

What would settle it

Run RVCD on images where the object detector misses objects that are actually present and that a greedy caption correctly names; the method should suppress those correct mentions, which would appear as lower BLEU or lower POPE recall compared with running the same method on perfect human annotations. A sharper version is to compare detector-based object lists against annotation-based lists on the same sample, where the paper's Table 3 already shows CHAIR_S of 12.4 with the detector versus 8.44 with perfect annotations, predicting that improving detector recall monotonically closes that gap.

Watch

Extended reading notes

Core claim

On its own terms, RVCD establishes that the images used for visual contrastive decoding need not be transformations of the input image: external reference images that isolate a single concept can serve as stronger regulatory targets. The pipeline first writes a greedy draft caption and runs an object detector on the input; objects named in the draft but absent from the detector's list are labeled negative, and objects present in both are positive. For each object in those two lists, a single-concept image is retrieved from a database built by generating one image per word in the CHAIR dictionary and keeping only images whose captions agree with the generation prompt. At every token step the model computes logits from the original image, from each negative image, and from each positive image, and combines them as $f_{\mathrm{adjusted},t}(\cdot | v, x, y_{<t}) = (1+\alpha|N|-\beta|P|)\, f_\theta(\cdot | v, x, y_{<t}) - (\alpha \textstyle\sum_{i} N_{t,i} - \beta \textstyle\sum_{j} P_{t,j})$, taking the argmax of the adjusted softmax as the output token. The paper's evidence is that this lowers CHAIR_S to 11.32 on LLaVA-1.5 and 9.00 on MiniGPT-4, and lifts POPE accuracy to 88.54 and 85.96 respectively, while BLEU stays comparable to greedy decoding.

Load-bearing premise

The method treats the detector's object list as ground truth: any real object the detector misses is classified as a hallucination and actively suppressed.

Editorial extensions

If this is right

  • If the central claim holds, object hallucination in 7B-scale vision-language models can be cut by roughly one-third to one-half relative to the strongest prior decoding baseline on CHAIR_S, without retraining.
  • Because the method is train-free and decoding-only, it can be layered onto any compatible open vision-language model and combined with existing post-hoc corrections.
  • The dependency on detector accuracy means every improvement in open-vocabulary object detection should transfer directly into lower hallucination rates for the same vision-language model.
  • The latency cost, about six times greedy decoding but well below the HALC baseline, makes the method practical for offline captioning and feasible where real-time generation is not required.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the same retrieval-and-contrast recipe could generalize from object nouns to attribute, count, and position predicates by building single-concept databases for those categories; the paper's MME results already show gains on existence and color, but not yet consistently on position and count.
  • Beyond the paper, the method's ceiling is set by the closed CHAIR dictionary; moving to open-vocabulary detectors and generators would let the same logit surgery suppress arbitrary hallucinated concepts rather than only the 80 MSCOCO classes and their synonyms.
  • Beyond the paper, the fixed $\beta=0.1$ recovery strength could be made instance-dependent, increasing when the draft mentions many co-occurring objects, which the ablations suggest would trade BLEU against CHAIR more flexibly.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes RVCD, a training-free decoding method that reduces object hallucination in large vision-language models (LVLMs). The method first greedily decodes a draft caption, then uses YOLO to detect objects in the input image, dividing draft-mentioned objects into 'negative' (not detected) and 'positive' (detected) sets. For each set, it retrieves AI-generated single-concept reference images and computes contrastive logits that are combined with the original logits via Equation (5), where alpha and beta control the suppression of negative objects and the recovery of positive objects. The authors evaluate RVCD on CHAIR, BLEU, POPE, MME, and LLaVA-Bench across three 7B backbones (LLaVA-1.5, MiniGPT-4, mPLUG-Owl2) and report substantial CHAIR reductions (e.g., CHAIRS from 18.60 to 11.32 on LLaVA-1.5) and large POPE accuracy gains, along with latency measurements and ablations. The code and reference-image database are released.

Significance. If the CHAIR results are taken at face value, the paper offers a practical, train-free, plug-and-play intervention that reduces hallucination more than prior decoding baselines across three architectures, with code and data publicly released. The core idea of retrieving explicit reference images beyond transformations of the input is a welcome extension of visual contrastive decoding. A particular strength is that the beta=0 ablation in Table 3 (CHAIRS 12.40 on LLaVA-1.5 with YOLOv8x) shows the main effect does not depend on the tuned positive-logit parameter, so the central OH-reduction claim has independent grounding. However, several evaluation and reporting issues, most importantly the POPE protocol in Appendix D, prevent the paper from being accepted in its current form.

major comments (3)
  1. [Section 5.1, Appendix D] The POPE evaluation is contaminated by a detector-oracle channel. As stated in Appendix D, the full POPE query (which always names exactly one candidate object) is concatenated with the initially greedy-decoded yes/no answer and used as the draft caption for RVCD. Because the draft thus always contains the queried object, the method classifies that object as positive or negative solely from YOLO's detection on the original image (Section 4.2) and adjusts the logits accordingly. The final decoded answer therefore tracks YOLO's verdict rather than the LVLM's own visual judgment, so the large POPE accuracy gains in Table 2 (e.g., LLaVA-1.5 from 72.19/72.48 to 88.54) mostly measure how well RVCD relays YOLO to the language output. The comparison to HALC's protocol in Appendix D does not resolve this, because HALC uses the detection model only to ground the focal area of the query, not to substitute the final yes/no decision. Please re-run POPE under a protocol that does not insert the answer or the query object into the RVCD draft, or, if that is impossible, explicitly label Table 2 as an oracle/upper-bound study and base the general claim of improvement on CHAIR only. As it stands, the abstract's broad claim of 'substantial improvements over existing decoding-based methods' is partly supported by an artifact.
  2. [Section 4.2, Table 3] The method treats YOLO's detections as ground truth, and CHAIR does not penalize omission of objects. Objects in the draft caption that YOLO misses are defined as negative (Section 4.2) and their logits are suppressed by Equation (5). Consequently, part of the CHAIR reduction may come from making captions more conservative (simply not mentioning undetected objects) rather than from improving visual grounding. Table 3 itself shows the ceiling directly: with perfect annotations CHAIRs is 8.44, with YOLOv3 it is 12.84, and with YOLOv8x it is 12.40 on LLaVA-1.5, so detector recall bounds the gain. The authors should report an object-level recall or omission rate for the final RVCD captions, and analyze whether the CHAIR advantage over baselines persists when conditioning only on objects that YOLO correctly detects. Without this, the magnitude of the claimed improvement is hard to interpret beyond 'remove objects YOLO does not see.'
  3. [Section 6.2, Tables 11-12] The reported optimal hyperparameters (alpha=1, beta=0.1) are selected on the same MSCOCO validation splits used for the final reported numbers, with no held-out set. The improvement from beta=0 to beta=0.1 is small and within the reported standard deviations (e.g., CHAIRs 11.08±1.15 to 10.84±0.89 on LLaVA-1.5, overlapping in both directions). The central conclusion is robust to this issue because beta=0 already beats the baselines, but the 'optimal' claim and the final Table 1 numbers are subjected to selection bias. Please either report results with a fixed default set (e.g., beta=0) as the headline, or tune on a separate subset and report the validation-tuned and held-out results separately.
minor comments (6)
  1. [Section 5.1, Table 1] The claim that RVCD maintains 'comparable BLEU scores' is not supported by the numbers: RVCD's BLEU is consistently lower than HALC's by roughly 0.8-1.8 points (e.g., 15.48 vs 16.32 on LLaVA-1.5, 15.98 vs 17.83 on MiniGPT-4, 14.78 vs 15.63 on mPLUG-Owl2), with small standard deviations. Please soften the claim or provide a statistical significance test for the BLEU differences.
  2. [Section 4.1] The cited source for FLUX.1-dev is Yang et al. (2024a), which is a paper about 1.58-bit FLUX quantization, not the original FLUX model. Please cite the proper release of FLUX.1-dev and clarify the exact version used.
  3. [Equations (5)-(6)] The notation is confusing: in Equation (5), 'len(N)' and 'len(P)' are used but N and P were defined as lists; in Equation (6), the superscripts N^t_i and P^t_j are not clearly explained. Consider defining len(N)=m and len(P)=k and using consistent subscripts throughout.
  4. [Section 3.2, Figure 1] The caption of Figure 1 is ambiguous: 'Hal (·) shows the proportion of hallucinated objects from greedy-decoded captions detected by YOLO and LVLMs VQA that were true hallucinations.' Clarify the denominator and the exact comparison, and specify how the LVLM VQA answers were mapped to detection labels.
  5. [Section 6.3, Table 4] The text says RVCD 'achieves superior performance in both output quality and decoding efficiency,' but RVCD is slower than greedy, DoLa, and VCD (latencies 0.143-0.204 s/token vs 0.034-0.073 s/token). Please rephrase to say it is more efficient than OPERA and HALC while still being slower than the simpler baselines.
  6. [Appendix D] The analogy to HALC's POPE protocol is imprecise: HALC uses the detection model to ground the focal area of the query but does not directly substitute the final yes/no response. Please articulate the difference, or remove the analogy to avoid misleading the reader.

Circularity Check

1 steps flagged · score 6.0 of 10

POPE evaluation is a detector oracle: the queried object is inserted into the draft and Eq. 5 relays YOLO's verdict, so Table 2's gains reduce by construction; CHAIR results remain independent.

  1. other [Appendix D (Table 2), with Section 4.2 and Eq. 5]
    "In the POPE evaluation, to create a fair environment similar to previous studies (Chen et al., 2024b; Zhuang et al., 2024), we combined the entire query of POPE with an initially greedy decoded answer (yes/no) and used it as a draft caption for RVCD. Accordingly, the detector determines whether the object mentioned in the draft caption actually exists in the image and conveys this judgment to the LVLMs."

    In POPE, each question names exactly one candidate object, so that object is always present in the draft caption. Section 4.2 classifies an object as negative if YOLO does not detect it and positive if YOLO detects it; Eq. 5 then subtracts the negative-image logits or adds the positive-image logits at every decoding step. The final yes/no answer is therefore a relay of YOLO's detection verdict rather than an independent LVLM judgment. The large POPE accuracy gains over decoding baselines in Table 2 are thus produced by construction from the detector input, not by a general suppression of hallucination. The CHAIR evaluation does not have this structure because its draft is an open caption, and Table 3 honestly shows the method's sensitivity to detector accuracy.

full rationale

The central RVCD mechanism is not circular: it combines YOLO detections with externally generated single-concept images and adjusts logits via Eq. 5, and the CHAIR results are obtained from open-ended captions with no oracle channel. Table 3 independently establishes that even the beta=0 variant (alpha=1) lowers LLaVA-1.5 CHAIRs from the HALC baseline, so the central claim has content beyond any detector-relay artifact. No load-bearing self-citations or uniqueness arguments appear. The one significant circular component is the POPE protocol in Appendix D, where the candidate object is forced into the draft caption and Eq. 5's positive/negative logit adjustment is driven by YOLO's detection of that same object, making the benchmark answer a function of the detector by construction. This contaminates the headline POPE claim in Table 2 and the abstract's broad 'substantial improvements' claim to the extent it relies on POPE; the CHAIR-based improvement remains credible. Hyperparameter selection on the same MSCOCO validation set is a statistical overfitting concern, not a circularity, and is not scored as a circular step.

Assumptions & free parameters 4 free parameters · 4 assumptions · 1 invented entities

The central empirical claim rests on two tuned scalars (alpha, beta), a detector threshold, and the unstated assumption that YOLO detections define ground truth. The image database is an invented resource with independent existence, though its construction is tied to one LVLM's perception.

free parameters (4)
  • alpha (negative logit regulation factor) = 1
    Chosen in Section 6.2 and Table 11 as the value maximizing CHAIR reduction; effectively tuned on the MSCOCO validation set used for final reporting.
  • beta (positive logit recovery factor) = 0.1
    Tuned on MSCOCO validation set (Table 12, Figure 6) to maximize BLEU and CHAIR; final reported results use this setting.
  • YOLO confidence threshold = 0.25
    Set to the detector default (Table 10); this threshold determines which detected objects are considered positive objects.
  • Image database generation hyperparameters = not specified
    Figure 3 says guidance scale, inference steps, and random seed vary during database construction, but the values are not reported, so regeneration is not fully reproducible.
assumptions (4)
  • domain assumption YOLO object detections are a reliable proxy for objects truly present in the image.
    Section 4.2 defines any draft object not detected by YOLO as a negative (hallucinated) object; the whole method rests on this proxy.
  • domain assumption The CHAIR dictionary (80 MSCOCO classes plus synonyms) is sufficient to cover objects of interest in general captioning.
    Sections 4.1 and 5.1 evaluate only on this dictionary; objects outside it cannot be retrieved or contrasted, limiting applicability.
  • domain assumption AI-generated single-concept images filtered by llava-1.5 captions faithfully represent the target concept.
    Section 4.1 stores images only if llava-1.5's caption mentions the object, so the database is aligned with one evaluated LVLM's perception.
  • domain assumption Combining logits from multiple images via element-wise addition and scalar scaling is a valid way to control token probabilities.
    Eq. 5 and Eq. 6 assume logit-space arithmetic transfers contrastive signal without breaking the LVLM's output distribution; this is standard in prior contrastive decoding work.
invented entities (1)
  • Single-concept image database (RVCD reference database) independent evidence
    purpose: Provides explicit images representing single MSCOCO concepts for generating negative and positive logits at every decoding step.
    Released at the project GitHub; the images are inspectable and were manually checked (Ethics Statement), so the resource has external existence beyond the paper's claims.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Retrieval Visual Contrastive Decoding to Mitigate Object Hallucinations in Large Vision-Language Models." pith.science (2026). https://pith.science/paper/OCVW2SI6

@misc{pith2026250520569,
  author       = {Pith},
  title        = {Pith review of: Retrieval Visual Contrastive Decoding to Mitigate Object Hallucinations in Large Vision-Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OCVW2SI6}},
  note         = {Machine review of arXiv:2505.20569}
}
read the original abstract

Despite significant advancements in Large Vision-Language Models, Object Hallucination (OH) remains a persistent challenge. Building upon prior studies on contrastive decoding that address this issue without requiring additional model training, we introduce RVCD (Retrieval Visual Contrastive Decoding), an advanced method to suppress OH. RVCD leverages both negative and positive images at the logit level, explicitly referencing AI-generated images designed to represent a single concept. Our approach demonstrates substantial improvements over existing decoding-based methods.

Figures

Figures reproduced from arXiv: 2505.20569 by the authors.

Figure 1
Figure 1. Detection precision for YOLO and LVLM de [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Overall pipeline of our RVCD. x denotes the input prompt, and v denotes the input image. nvi and pvi are images retrieved from the image database, representing single-concept images for objects identified as hallucinations (appearing only in the draft caption) and ground truth (appearing in both the OD model and draft caption), respectively. Nt and Pt represent the sets of logits generated from nvi and pvi, respecti… view at source ↗
Figure 3
Figure 3. AI generated single concept image DB. we [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Top-5 token probabilities for each single con [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Comparison of different decoding baselines [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: BLEU and CHAIR scores based on the variation of the [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: LLaVA-Bench results comparing our RVCD and other methods with LLaVA-1.5 backbone. [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]
Figure 8
Figure 8. Figure 8: LLaVA-Bench results comparing our RVCD and other methods with MiniGPT-4 backbone. [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]
Figure 9
Figure 9. Figure 9: LLaVA-Bench results comparing our RVCD and other methods with MiniGPT-4 backbone. [PITH_FULL_IMAGE:figures/full_fig_p019_9.png]
Figure 10
Figure 10. Figure 10: LLaVA-Bench results comparing our RVCD and other methods with mPLUG-Owl2 backbone. [PITH_FULL_IMAGE:figures/full_fig_p020_10.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. MACD: Model-Aware Contrastive Decoding via Counterfactual Data

    cs.AI 2026-02 reject novelty 6.0 of 10

    MACD reduces Video-LLM hallucination by masking model-identified critical objects/frames via gradient ascent and using the masked video as a contrastive decoding reference.

Reference graph

Works this paper leans on

45 extracted references · 8 canonical work pages · cited by 1 Pith paper

  1. [1]

    Black Forest Labs . 2024. https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/LICENSE.md FLUX.1 License . Accessed on February 7, 2025

    work page 2024

  2. [2]

    Jun Chen, Deyao Zhu, Xiaoqian Shen, Xiang Li, Zechun Liu, Pengchuan Zhang, Raghuraman Krishnamoorthi, Vikas Chandra, Yunyang Xiong, and Mohamed Elhoseiny. 2023. Minigpt-v2: Large language model as a unified interface for vision-language multi-task learning https://arxiv.org/abs/2310.09478. arXiv:2310.09478

    arXiv 2023

  3. [3]

    Fouhey, and Joyce Chai

    Xuweiyi Chen, Ziqiao Ma, Xuejun Zhang, Sihan Xu, Shengyi Qian, Jianing Yang, David F. Fouhey, and Joyce Chai. 2024 a . https://doi.org/10.48550/arXiv.2407.06192 Multi-object hallucination in vision-language models . In Proceedings of the 38th Conference on Neural Information Processing Systems (NeurIPS). Accepted to NeurIPS 2024

    arXiv doi:10.48550/arxiv.2407.06192 2024

  4. [4]

    Zhaorun Chen, Zhuokai Zhao, Hongyin Luo, Huaxiu Yao, Bo Li, and Jiawei Zhou. 2024 b . HALC: Object hallucination reduction via adaptive focal-contrast decoding https://arxiv.org/abs/2403.00425. arXiv:2403.00425

    arXiv 2024

  5. [5]

    Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James Glass, and Pengcheng He. 2023. Dola: Decoding by contrasting layers improves factuality in large language models https://arxiv.org/abs/2309.03883. arXiv:2309.03883

    arXiv 2023

  6. [6]

    Chenhang Cui, Yiyang Zhou, Xinyu Yang, Shirley Wu, Linjun Zhang, James Zou, and Huaxiu Yao. 2023. Holistic analysis of hallucination in GPT-4V (ision): Bias and interference challenges https://arxiv.org/abs/2311.03287. arXiv:2311.03287

    arXiv 2023

  7. [7]

    Wenliang Dai, Zihan Liu, Ziwei Ji, Dan Su, and Pascale Fung. 2022. Plausible may not be faithful: Probing object hallucination in vision-language pre-training https://arxiv.org/abs/2210.07688. arXiv:2210.07688

    arXiv 2022

  8. [8]

    Alessandro Favero, Luca Zancato, Matthew Trager, Siddharth Choudhary, Pramuditha Perera, Alessandro Achille, Ashwin Swaminathan, and Stefano Soatto. 2024. https://arxiv.org/abs/2403.14003 Multi-modal hallucination control by visual information grounding . In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). ArXiv:24...

    arXiv 2024

Show all 45 references
  1. [9]

    Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, Yunsheng Wu, and Rongrong Ji. 2023. MME: A comprehensive evaluation benchmark for multimodal large language models https://arxiv.org/abs/2306.13394. arXiv:2306.13394

    2023 arXiv

  2. [10]

    Tianrui Guan, Fuxiao Liu, Xiyang Wu, Ruiqi Xian, Zongxia Li, Xiaoyu Liu, Xijun Wang, Lichang Chen, Furong Huang, Yaser Yacoob, Dinesh Manocha, and Tianyi Zhou. 2023. Hallusionbench: An advanced diagnostic suite for entangled language hallucination & visual illusion in large vi...

    2023

  3. [11]

    Anisha Gunjal, Jihan Yin, and Erhan Bas. 2023. Detecting and preventing hallucinations in large vision language models https://arxiv.org/abs/2308.06394. arXiv:2308.06394

    2023 arXiv

  4. [12]

    Qidong Huang, Xiaoyi Dong, Pan Zhang, Bin Wang, Conghui He, Jiaqi Wang, Dahua Lin, Weiming Zhang, and Nenghai Yu. 2023. Opera: Alleviating hallucination in multi-modal large language models via over-trust penalty and retrospection-allocation https://arxiv.org/abs/2311.17911. a...

    2023 arXiv

  5. [13]

    Liqiang Jing and Xinya Du. 2024. FGAIF: Aligning Large Vision-Language Models with Fine-grained AI Feedback https://arxiv.org/abs/2404.05046. arXiv:2404.05046. Submitted on 7 Apr 2024 (v1), last revised 6 May 2025 (v2)

    2024 arXiv

  6. [14]

    Sicong Leng, Hang Zhang, Guanzheng Chen, Xin Li, Shijian Lu, Chunyan Miao, and Lidong Bing. 2023. Mitigating object hallucinations in large vision-language models through visual contrastive decoding https://arxiv.org/abs/2311.16922. arXiv:2311.16922

    2023 arXiv

  7. [15]

    Liunian Harold Li, Mark Yatskar, Da Yin, Cho-Jui Hsieh, and Kai-Wei Chang. 2019. Visualbert: A simple and performant baseline for vision and language https://arxiv.org/abs/1908.03557. arXiv:1908.03557

    2019 arXiv

  8. [16]

    Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. 2023. Evaluating object hallucination in large vision-language models https://arxiv.org/abs/2305.10355. arXiv:2305.10355

    2023 arXiv

  9. [17]

    Lawrence Zitnick, and Piotr Dollár

    Tsung-Yi Lin, Michael Maire, Serge Belongie, Lubomir Bourdev, Ross Girshick, James Hays, Pietro Perona, Deva Ramanan, C. Lawrence Zitnick, and Piotr Dollár. 2014. Microsoft coco: Common objects in context. In Computer Vision--ECCV 2014, pages 740--755. Springer

    2014

  10. [18]

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2023 a . Improved baselines with visual instruction tuning https://arxiv.org/abs/2310.03744. arXiv:2310.03744

    2023 arXiv

  11. [19]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023 b . Visual instruction tuning https://arxiv.org/abs/2304.08485. arXiv:2304.08485

    2023 arXiv

  12. [20]

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, and Lei Zhang. 2023 c . Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection https://arxiv.org/abs/2303.05499. arXiv...

    2023 arXiv

  13. [21]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311--318

    2002

  14. [22]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning transferable visual models from natural language supervision. In Internation...

    2021

  15. [23]

    Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. 2015. You Only Look Once: Unified, Real-Time Object Detection https://arxiv.org/abs/1506.02640. arXiv:1506.02640

    2015 arXiv

  16. [24]

    Joseph Redmon and Ali Farhadi. 2018. YOLOv3: An Incremental Improvement https://arxiv.org/abs/1804.02767. arXiv:1804.02767. Tech Report

    2018 arXiv

  17. [25]

    Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kunchang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, Zhaoyang Zeng, Hao Zhang, Feng Li, Jie Yang, Hongyang Li, Qing Jiang, and Lei Zhang. 2024. Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks h...

    2024 arXiv

  18. [26]

    Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko. 2018. Object hallucination in image captioning https://arxiv.org/abs/1809.02156. arXiv:1809.02156

    2018 arXiv

  19. [27]

    Zhiqing Sun, Sheng Shen, Shengcao Cao, Haotian Liu, Chunyuan Li, Yikang Shen, Chuang Gan, Liang-Yan Gui, Yu-Xiong Wang, Yiming Yang, Kurt Keutzer, and Trevor Darrell. 2023. Aligning Large Multimodal Models with Factually Augmented RLHF https://arxiv.org/abs/2309.14525. arXiv:2...

    2023 arXiv

  20. [28]

    Haoqin Tu, Chenhang Cui, Zijun Wang, Yiyang Zhou, Bingchen Zhao, Junlin Han, Wangchunshu Zhou, Huaxiu Yao, and Cihang Xie. 2023. How many unicorns are in this image? A safety evaluation benchmark for vision LLMS https://arxiv.org/abs/2311.16101. arXiv:2311.16101

    2023 arXiv

  21. [29]

    Ultralytics. 2023. https://github.com/ultralytics/ultralytics Yolo by ultralytics . Accessed on January 24, 2025

    2023

  22. [30]

    Ultralytics. 2024. https://www.ultralytics.com/ko/blog/ultralytics-yolov8-turns-one-a-year-of-breakthroughs-and-innovations Ultralytics yolov8 turns one: A year of breakthroughs and innovations . Accessed on January 24, 2025

    2024

  23. [31]

    Xiyao Wang, Yuhang Zhou, Xiaoyu Liu, Hongjin Lu, Yuancheng Xu, Feihong He, Jaehong Yoon, Taixi Lu, Gedas Bertasius, Mohit Bansal, Huaxiu Yao, and Furong Huang. 2024. Mementos: A comprehensive benchmark for multimodal large language model reasoning over image sequences https://...

    2024 arXiv

  24. [32]

    Yun Xing, Yiheng Li, Ivan Laptev, and Shijian Lu. 2024. Mitigating Object Hallucination via Concentric Causal Attention https://arxiv.org/abs/2410.15926. arXiv:2410.15926. To appear at NeurIPS 2024, submitted on 21 Oct 2024

    2024 arXiv

  25. [33]

    Chenglin Yang, Celong Liu, Xueqing Deng, Dongwon Kim, Xing Mei, Xiaohui Shen, and Liang-Chieh Chen. 2024 a . https://doi.org/10.48550/arXiv.2412.18653 1.58-bit FLUX: The first successful approach to quantizing the state-of-the-art text-to-image generation model, FLUX.1-dev, us...

    2024 doi arXiv

  26. [34]

    Dingchen Yang, Bowen Cao, Guang Chen, and Changjun Jiang. 2024 b . Pensieve: Retrospect-then-Compare Mitigates Visual Hallucination https://arxiv.org/abs/2403.14401. arXiv:2403.14401. Submitted on 21 Mar 2024 (v1), last revised 1 Sep 2024 (v2)

    2024 arXiv

  27. [35]

    Qinghao Ye, Haiyang Xu, Jiabo Ye, Ming Yan, Anwen Hu, Haowei Liu, Qi Qian, Ji Zhang, Fei Huang, and Jingren Zhou. 2023. mplug-owl2: Revolutionizing multi-modal large language model with modality collaboration https://arxiv.org/abs/2311.04257. arXiv:2311.04257

    2023 arXiv

  28. [36]

    Shukang Yin, Chaoyou Fu, Sirui Zhao, Tong Xu, Hao Wang, Dianbo Sui, Yunhang Shen, Ke Li, Xing Sun, and Enhong Chen. 2023. Woodpecker: Hallucination correction for multimodal large language models https://arxiv.org/abs/2310.16045. arXiv:2310.16045

    2023 arXiv

  29. [37]

    Bohan Zhai, Shijia Yang, Xiangchen Zhao, Chenfeng Xu, Sheng Shen, Dongdi Zhao, Kurt Keutzer, Manling Li, Tan Yan, and Xiangjun Fan. 2023. Halle-switch: Controlling object hallucination in large vision language models. arXiv e-prints, pages arXiv--2310

    2023

  30. [38]

    Yiming Zhang, Zhuokai Zhao, Zhaorun Chen, Zhili Feng, Zenghui Ding, and Yining Sun. 2024. Rankclip: Ranking-consistent language-image pretraining https://arxiv.org/abs/2404.09387. arXiv:2404.09387

    2024 arXiv

  31. [39]

    Yiyang Zhou, Chenhang Cui, Rafael Rafailov, Chelsea Finn, and Huaxiu Yao. 2024 a . Aligning modalities in vision large language models via preference fine-tuning https://arxiv.org/abs/2402.11411. arXiv:2402.11411

    2024 arXiv

  32. [40]

    Yiyang Zhou, Chenhang Cui, Jaehong Yoon, Linjun Zhang, Zhun Deng, Chelsea Finn, Mohit Bansal, and Huaxiu Yao. 2023. Analyzing and mitigating object hallucination in large vision-language models https://arxiv.org/abs/2310.00754. arXiv:2310.00754

    2023 arXiv

  33. [41]

    Yiyang Zhou, Zhiyuan Fan, Dongjie Cheng, Sihan Yang, Zhaorun Chen, Chenhang Cui, Xiyao Wang, Yun Li, Linjun Zhang, and Huaxiu Yao. 2024 b . Calibrated self-rewarding vision language models https://arxiv.org/abs/2405.14622. arXiv:2405.14622

    2024 arXiv

  34. [42]

    Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. 2023. Minigpt-4: Enhancing vision-language understanding with advanced large language models https://arxiv.org/abs/2304.10592. arXiv:2304.10592

    2023 arXiv

  35. [43]

    Xianwei Zhuang, Zhihong Zhu, Zhanpeng Chen, Yuxin Xie, Liming Liang, , and Yuexian Zou. 2024. Game on tree: Visual hallucination mitigation via coarse-to-fine view tree and game theory. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, ...

    2024

  36. [44]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  37. [45]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.