Pith. sign in

REVIEW 5 major objections 4 minor 59 references

RICO: Improving Accuracy and Completeness in Image Recaptioning via Visual Reconstruction

T0 review · 5 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read RICO claims that recaptioning improves when each candidate caption is reconstructed into an image and the visible discrepancies between original and reconstruction drive the rewrite.

desk verdict Reconstruction-based caption revision is a genuine new mechanism, but the paper never isolates reconstruction from its own detailed prompt, so the central claim needs one more control before it lands. read the letter →

arxiv 2505.22613 v1 pith:7NGKZ2GT submitted 2025-05-28 cs.CV cs.AIcs.CL

Yuchi Wang , Yishuo Cai , Shuhuai Ren , Sihan Yang , Linli Yao , Yuanxin Liu , Yuanxing Zhang , Pengfei Wan
show 1 more author
Xu Sun
This is my paper
classification cs.CVcs.AIcs.CL
keywords imagerecaptioningvisualreconstructionmultimodallargelanguagemodelstext-to-imagegenerationdirectpreferenceoptimizationcaptionaccuracycompletenesshallucinationmitigation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

RICO sets out to prove that image recaptioning becomes more accurate and complete when the caption is forced back through a visual channel: a text-to-image model renders the current caption into an image, and an MLLM revises the caption by comparing that reconstruction with the original photograph. The paper reports that two iterations of this loop raise CapsBench accuracy from 42.0 to 59.0 on Qwen2-VL initial captions, with similar double-digit gains on relative position and shape, and about ten-point improvements over most recaptioning baselines on CapsBench and CompreCap. It also claims the improved captions transfer to training text-to-image models, giving gains on DPG-Bench and VQAScore. The significance, if the claim holds, is that converting text back into pixels makes the information lost by hallucinating or detail-dropping captioners visible and therefore correctable, a property neither stronger MLLMs nor human editors demonstrated in the paper's comparison.

What carries the argument

The load-bearing mechanism is the reconstruction–refinement loop, formalized as $v_i = T(c_{i-1})$ and $c_i = R(v_i, v_0, c_{i-1})$, where $T$ is the FLUX.1-dev text-to-image model and $R$ is GPT-4o prompted to contrast the original image with the reconstructed one. Reconstruction is what moves the comparison into a single modality: instead of asking whether text matches an image, the reviser looks at two images and names what differs. The prompt's eight focus aspects and its requirement to emit an analysis before the revised caption are what turn those named differences into concrete corrections and additions. For RICO-Flash, the mechanism is preference distillation: the ordered pair of initial and final captions from the loop becomes a DPO training signal that teaches one model to reproduce the effect of the loop in a single forward pass.

What would settle it

Take a probe set where the reconstruction model is known to fail on one attribute class, such as object count or small in-image text, and run RICO; if captions still gain accuracy on that class, the gains are not coming from the visual-discrepancy loop, while if they do not improve, the stated dependence on reconstruction power is confirmed.

Watch

Extended reading notes

Core claim

The central claim is that semantic alignment between image and caption should be bidirectional: the caption is faithful only if an image reconstructed from it agrees with the original. RICO operationalizes that test by alternating a text-to-image model, FLUX.1-dev, with an MLLM reviser, GPT-4o, applying $c_i = R(T(c_{i-1}), v_0, c_{i-1})$ for two steps. The reviser is prompted to attend to eight aspects—visual details, composition and layout, human attributes, perspective and style, text in the image, image quality, world knowledge, and color aesthetics—and to output an analysis before the revised caption. On the benchmarks used, the loop moves Qwen2-VL captions from 42.0 to 59.0 CapsBench accuracy and from 32.4 to 59.5 relative-position accuracy, while Amber's Cover minus CHAIR composite improves, indicating reduced hallucination. The paper further claims that a distilled single-pass model, RICO-Flash, trained with DPO on the loop's initial-to-final caption preferences, approaches the iterative version's performance without per-image reconstruction, and that a text-to-image model fine-tuned on RICO-refined captions outperforms one trained on the initial captions.

Load-bearing premise

The whole pipeline depends on the text-to-image model being strong enough to render fine-grained attributes and spatial relations from the caption; the paper says this in its limitations section, and if the reconstruction drops those details, the reviser sees no discrepancy and the completeness gains vanish.

Editorial extensions

If this is right

  • Recaptioning pipelines built on an MLLM can expect double-digit gains on fine-grained accuracy metrics simply by adding the reconstruction–refinement loop, with diminishing returns after two iterations.
  • Captions refined by the loop are better supervision for text-to-image training, since a FLUX model fine-tuned on RICO-refined captions beat one trained on initial captions on DPG-Bench and VQAScore.
  • The preference pairs produced by the loop are sufficient supervision for a distilled one-pass captioner, so the iterative cost does not have to be paid at inference time.
  • The method is model-agnostic on the captioning side: it improved initial captions from seven different captioning models and across several prompting schemes in the paper's experiments.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the paper's stated assumption is right, the loop's ceiling is set by the reconstruction model, not the reviser; swapping FLUX.1-dev for a stronger text-to-image generator should produce larger gains than swapping GPT-4o for another reviser, a claim the paper does not test.
  • The same discrepancy loop could be applied to other image-to-text products, such as structured scene-graph extraction or accessibility alt-text, where fine-grained omissions are costly; nothing in the mechanism restricts it to full-sentence captions.
  • A testable extension would measure whether details added by the reviser are ever true of neither image: if the reviser over-trusts its own analysis, precision of added tokens against the scene graph would decline even as coverage rises.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper presents RICO, a recaptioning framework that iteratively reconstructs a candidate caption with FLUX.1-dev and then asks GPT-4o to revise the caption by comparing the original image with the reconstructed image. After N=2 iterations, the authors report substantial gains on CapsBench, CompreCap, Amber, and CAPTURE relative to several initial captioning models and to existing recaptioning baselines. To reduce inference cost, RICO-Flash fine-tunes Qwen2-VL with DPO using the initial and final RICO captions as a preference pair. The paper also reports that a FLUX model fine-tuned on RICO-refined captions achieves higher DPG-Bench and VQAScore scores than one trained on the original captions.

Significance. If the reconstruction signal is genuinely the cause of the measured gains, RICO is a useful and well-motivated contribution to image recaptioning, with a clear mechanism and a practical distilled variant. The evaluation is broad, covering multiple initial MLLMs, multiple prompts, several benchmarks, and a downstream text-to-image generation task, and the code is released. However, the paper does not yet demonstrate that visual reconstruction itself, rather than the detailed refinement prompt, drives the improvements, and the evaluative overlap between the GPT-4o reviser and the CapsBench judge weakens the reliability of the reported effect sizes. These issues are addressable with additional experiments.

major comments (5)
  1. [§3.3, Table 2, Table 6, Table 12] The central claim is that the reconstructed image provides an error signal beyond the original image and caption, but no experiment isolates this signal. Table 6 ablates the aspect guidance ('wo/ tips') and the analysis output ('wo/ output analy.') while always providing both images; Table 2 compares RICO Edit against a GPT-4o baseline that receives only the short instruction in Table 7, not the full Table 12 prompt with eight focus aspects and forced CoT. Please add a control that runs GPT-4o with the complete RICO prompt but without the reconstructed image (original image plus caption only), at both N=1 and N=2, and also a GPT-4o Edit variant using the full prompt. If these controls match RICO's numbers, the reconstruction mechanism is not the active ingredient.
  2. [§3.3, §B.2, Table 1, Table 3] CapsBench is judged by GPT-4o (2024-08-06), the same model family used as the RICO reviser, so the measured gains may partly reflect the captions conforming to the reviser's preferences rather than improved accuracy or completeness. Please re-score a subset with an independent judge (e.g., Gemini 1.5 Pro or Qwen2-VL) and report per-item agreement with GPT-4o. In addition, all tables report only point estimates; please provide confidence intervals or significance tests, especially for small differences such as CompreCap Rel. 2.82 versus 2.84 in Table 1.
  3. [§3.4, §4.2] The RICO-Flash preference pairs are (initial Qwen2-VL caption, final RICO caption), where the final caption is produced by GPT-4o under the detailed RICO prompt. Without the no-reconstruction control in the first major comment, RICO-Flash may simply be a DPO distillation of GPT-4o's captioning behavior rather than evidence for the reconstruction loop. Please compare RICO-Flash against a model trained with DPO on preference pairs generated by GPT-4o using the full RICO prompt but no reconstructed image.
  4. [Table 2] The claim that RICO improves fine-grained spatial information is not consistently supported by this table: RICO Edit raises Shape from 20.41 to 49.51 but lowers Rel.Pos. from 39.45 to 34.04, below both the original caption (39.45) and GPT-4o Edit (44.04). Please address this drop explicitly and provide the same breakdown on the full CapsBench set, since the central claim includes completeness of spatial relations.
  5. [§E] The paper acknowledges that the pipeline assumes the text-to-image model is powerful enough to render fine-grained details, but it does not test this assumption. Please add a sensitivity analysis, for example by using a weaker reconstruction model or by measuring how often FLUX fails to reproduce attributes and relations that are annotated in the benchmark; otherwise it is unclear whether the completeness gains would survive with a different reconstruction model.
minor comments (4)
  1. [Appendix D] The DPO loss is written as -E log( pi_theta(y+)^beta / (pi_theta(y+)^beta + pi_theta(y-)^beta) ), which omits the reference policy and the sigmoid and is inconsistent with the formula in Section 3.4; please correct or remove this simplified derivation.
  2. [§4.2, §B.3] The comparison with human annotators uses 100 images and two annotators; please report inter-annotator agreement and note that this is a small sample when claiming that RICO surpasses humans.
  3. [Abstract, §1, Table 5] The statement of 'over 10 points' improvement on CapsBench is not true for all initial models in Table 5 (e.g., GPT-4o +8.1, Gemini 1.5 Pro +8.0); please qualify the claim to the models where it holds.
  4. [Figure 4, Table 1] The iteration-curve analysis and all headline tables would benefit from error bars or variance reporting; this is related to the statistical point in the second major comment.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: RICO's improvement is an empirical pipeline result evaluated on fixed external benchmarks, and the noted GPT-4o reviser/judge overlap is an evaluator confound rather than a definitional reduction.

full rationale

The paper's derivation chain is empirical rather than analytic: the central claim is that iterating a text-to-image reconstruction step (FLUX.1-dev) and a GPT-4o-based reviser improves caption accuracy and completeness, as measured on CapsBench, CompreCap, Amber, and CAPTURE. Nothing in the method defines the evaluation score in terms of the pipeline's own outputs or fitted parameters. The benchmark labels and scene graphs are externally fixed, and the revised captions are generated without access to those labels, so the reported gains are not constructed to equal an input. RICO-Flash is a DPO distillation of RICO outputs and is then evaluated on the same benchmarks; this is a standard distillation-fidelity check, not a circular derivation, because the preference pairs come from RICO's iterative outputs and the evaluation benchmarks are independent of that training signal. The two most serious validity concerns—the absence of an ablation that removes only the reconstructed image while holding the full Table 12 prompt fixed, and the use of GPT-4o as both reviser (§3.3) and CapsBench judge (§B.2)—are experimental confounds that could exaggerate the mechanism's contribution, but they do not reduce the result to its inputs by construction: there is no fitted parameter, self-citation chain, or definitional identity between the reconstruction step and the evaluation metric. The Limitations section explicitly flags the FLUX fidelity assumption, which is an external precondition of the method rather than a circular step. No load-bearing self-citations or imported uniqueness claims are present. Score 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the capability of off-the-shelf models (FLUX, GPT-4o) and on the validity of adopted benchmarks. No new physical or mathematical entities are introduced. The main free parameters are the iteration count and DPO hyperparameters, all chosen by hand.

free parameters (3)
  • Refinement iterations N = N=2
    Set in §4.1.1 based on empirical observation that two iterations balance performance and cost; Fig. 4 shows saturation after step 2. This is a hand-selected hyperparameter, not derived.
  • DPO preference scaling beta = 0.1
    Set in §4.1.1 and §C.2; no sensitivity analysis is shown.
  • DPO learning rate = 1.0e-5
    Set in §4.1.1 and §C.2; no sensitivity analysis is shown.
assumptions (4)
  • domain assumption The text-to-image model (FLUX.1-dev) can faithfully reconstruct enough fine-grained details from a caption for discrepancies to be informative.
    Explicitly acknowledged in §E: 'a key assumption... text-to-image model must be sufficiently powerful'. If false, the loop cannot recover omitted details.
  • domain assumption GPT-4o can reliably identify discrepancies between original and reconstructed images and convert them into correct caption revisions.
    The entire refinement step in §3.3 delegates this to GPT-4o with no verification beyond final benchmark scores.
  • domain assumption The final refined caption is consistently better than the initial caption, justifying the DPO preference pairs.
    §3.4 states 'based on our empirical observation that c_N consistently outperforms c_0', but no per-sample quality check or filtering is described.
  • domain assumption CapsBench, CompreCap, and Amber measure what the paper claims they measure.
    Benchmarks are adopted from prior work and not independently validated here.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RICO: Improving Accuracy and Completeness in Image Recaptioning via Visual Reconstruction." pith.science (2026). https://pith.science/paper/7NGKZ2GT

@misc{pith2026250522613,
  author       = {Pith},
  title        = {Pith review of: RICO: Improving Accuracy and Completeness in Image Recaptioning via Visual Reconstruction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7NGKZ2GT}},
  note         = {Machine review of arXiv:2505.22613}
}
read the original abstract

Image recaptioning is widely used to generate training datasets with enhanced quality for various multimodal tasks. Existing recaptioning methods typically rely on powerful multimodal large language models (MLLMs) to enhance textual descriptions, but often suffer from inaccuracies due to hallucinations and incompleteness caused by missing fine-grained details. To address these limitations, we propose RICO, a novel framework that refines captions through visual reconstruction. Specifically, we leverage a text-to-image model to reconstruct a caption into a reference image, and prompt an MLLM to identify discrepancies between the original and reconstructed images to refine the caption. This process is performed iteratively, further progressively promoting the generation of more faithful and comprehensive descriptions. To mitigate the additional computational cost induced by the iterative process, we introduce RICO-Flash, which learns to generate captions like RICO using DPO. Extensive experiments demonstrate that our approach significantly improves caption accuracy and completeness, outperforms most baselines by approximately 10% on both CapsBench and CompreCap. Code released at https://github.com/wangyuchi369/RICO.

Figures

Figures reproduced from arXiv: 2505.22613 by the authors.

Figure 1
Figure 1. Analysis of image captions generated by Qwen2-VL and its recaptioned variants. Despite the advanced [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Illustration of the motivation for introducing the visual reconstruction mechanism. Conventional [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Illustration of the iterative process of RICO. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Performance of the RICO pipeline under dif [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: An example demonstrating the iterative refinement process performed by our model, where red text [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: An example demonstrating the output analysis produced by our model, where green text highlights [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

59 extracted references · 4 canonical work pages

  1. [1]

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katie Millican, Malcolm Reynolds, Roman Ring, Eliza Rutherford, Serkan Cabi, Tengda Han, Zhitao Gong, Sina Samangooei, Marianne Monteiro, Jacob Menick, Sebastian Borgeaud, and 8 others. 2022. https://arxiv.org/abs/2204.14198 Flamingo: a visu...

    arXiv 2022

  2. [2]

    Peter Anderson, Basura Fernando, Mark Johnson, and Stephen Gould. 2016. https://arxiv.org/abs/1607.08822 Spice: Semantic propositional image caption evaluation . Preprint, arXiv:1607.08822

    arXiv 2016

  3. [3]

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, and 29 others. 2023 a . https://arxiv.org/abs/2309.16609 Qwen technical report . Preprint, arXiv:2309.16609

    arXiv 2023

  4. [4]

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023 b . https://arxiv.org/abs/2308.12966 Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond . Preprint, arXiv:2308.12966

    arXiv 2023

  5. [5]

    Zechen Bai, Pichao Wang, Tianjun Xiao, Tong He, Zongbo Han, Zheng Zhang, and Mike Zheng Shou. 2025. https://arxiv.org/abs/2404.18930 Hallucination of multimodal large language models: A survey . Preprint, arXiv:2404.18930

    arXiv 2025

  6. [6]

    James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, Wesam Manassra, Prafulla Dhariwal, Casey Chu, Yunxin Jiao, and Aditya Ramesh. 2023. https://cdn.openai.com/papers/dall-e-3.pdf Improving image generation with better captions

    work page 2023

  7. [7]

    Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Conghui He, Jiaqi Wang, Feng Zhao, and Dahua Lin. 2023. https://arxiv.org/abs/2311.12793 Sharegpt4v: Improving large multi-modal models with better captions . Preprint, arXiv:2311.12793

    arXiv 2023

  8. [8]

    Gonzalez, Ion Stoica, and Eric P

    Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023. https://lmsys.org/blog/2023-03-30-vicuna/ Vicuna: An open-source chatbot impressing gpt-4 with 90\

    2023

Show all 59 references
  1. [9]

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, and 181 others. 2025. https://arxiv.org/abs/2501.129...

    2025 arXiv

  2. [10]

    Hongyuan Dong, Jiawen Li, Bohong Wu, Jiacong Wang, Yuan Zhang, and Haoyuan Guo. 2024 a . https://arxiv.org/abs/2405.19092 Benchmarking and improving detail image caption . Preprint, arXiv:2405.19092

    2024 arXiv

  3. [11]

    Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Tianyu Liu, Baobao Chang, Xu Sun, Lei Li, and Zhifang Sui. 2024 b . https://arxiv.org/abs/2301.00234 A survey on in-context learning . Preprint, arXiv:2301.00234

    2024 arXiv

  4. [12]

    Lijie Fan, Dilip Krishnan, Phillip Isola, Dina Katabi, and Yonglong Tian. 2023. https://arxiv.org/abs/2305.20088 Improving clip training with language rewrites . Preprint, arXiv:2305.20088

    2023 arXiv

  5. [13]

    Taraneh Ghandi, Hamidreza Pourreza, and Hamidreza Mahyar. 2023. https://doi.org/10.1145/3617592 Deep learning approaches on image captioning: A review . ACM Computing Surveys, 56(3):1–39

    2023 doi

  6. [14]

    Dhruba Ghosh, Hanna Hajishirzi, and Ludwig Schmidt. 2023. https://arxiv.org/abs/2310.11513 Geneval: An object-focused framework for evaluating text-to-image alignment . Preprint, arXiv:2310.11513

    2023 arXiv

  7. [15]

    Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. 2022. https://arxiv.org/abs/2104.08718 Clipscore: A reference-free evaluation metric for image captioning . Preprint, arXiv:2104.08718

    2022 arXiv

  8. [16]

    Wenyi Hong, Weihan Wang, Ming Ding, Wenmeng Yu, Qingsong Lv, Yan Wang, Yean Cheng, Shiyu Huang, Junhui Ji, Zhao Xue, Lei Zhao, Zhuoyi Yang, Xiaotao Gu, Xiaohan Zhang, Guanyu Feng, Da Yin, Zihan Wang, Ji Qi, Xixuan Song, and 6 others. 2024. https://arxiv.org/abs/2408.16500 Cogv...

    2024 arXiv

  9. [17]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. https://arxiv.org/abs/2106.09685 Lora: Low-rank adaptation of large language models . Preprint, arXiv:2106.09685

    2021 arXiv

  10. [18]

    Xiwei Hu, Rui Wang, Yixiao Fang, Bin Fu, Pei Cheng, and Gang Yu. 2024. https://arxiv.org/abs/2403.05135 Ella: Equip diffusion models with llm for enhanced semantic alignment . Preprint, arXiv:2403.05135

    2024 arXiv

  11. [19]

    Kaiyi Huang, Chengqi Duan, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu. 2025. https://arxiv.org/abs/2307.06350 T2i-compbench++: An enhanced and comprehensive benchmark for compositional text-to-image generation . Preprint, arXiv:2307.06350

    2025 arXiv

  12. [20]

    Imagen-Team-Google, :, Jason Baldridge, Jakob Bauer, Mukul Bhutani, Nicole Brichtova, Andrew Bunner, Lluis Castrejon, Kelvin Chan, Yichang Chen, Sander Dieleman, Yuqing Du, Zach Eaton-Rosen, Hongliang Fei, Nando de Freitas, Yilin Gao, Evgeny Gladchenko, Sergio Gómez Colmenarej...

    2024

  13. [21]

    Black Forest Labs. 2024. Flux. https://github.com/black-forest-labs/flux

    2024

  14. [22]

    Zhengfeng Lai, Haotian Zhang, Bowen Zhang, Wentao Wu, Haoping Bai, Aleksei Timofeev, Xianzhi Du, Zhe Gan, Jiulong Shan, Chen-Nee Chuah, Yinfei Yang, and Meng Cao. 2024. https://arxiv.org/abs/2310.07699 Veclip: Improving clip training via visual-enriched captions . Preprint, ar...

    2024 arXiv

  15. [23]

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023 a . https://arxiv.org/abs/2301.12597 Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models . Preprint, arXiv:2301.12597

    2023 arXiv

  16. [24]

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. 2022. https://arxiv.org/abs/2201.12086 Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation . Preprint, arXiv:2201.12086

    2022 arXiv

  17. [25]

    Zhuang Li, Yuyang Chai, Terry Yue Zhuo, Lizhen Qu, Gholamreza Haffari, Fei Li, Donghong Ji, and Quan Hung Tran. 2023 b . https://arxiv.org/abs/2305.17497 Factual: A benchmark for faithful and consistent textual scene graph parsing . Preprint, arXiv:2305.17497

    2023 arXiv

  18. [26]

    Chin-Yew Lin. 2004. https://aclanthology.org/W04-1013/ ROUGE : A package for automatic evaluation of summaries . In Text Summarization Branches Out, pages 74--81, Barcelona, Spain. Association for Computational Linguistics

    2004

  19. [27]

    Zhiqiu Lin, Deepak Pathak, Baiqi Li, Jiayao Li, Xide Xia, Graham Neubig, Pengchuan Zhang, and Deva Ramanan. 2024. https://arxiv.org/abs/2404.01291 Evaluating text-to-visual generation with image-to-text generation . Preprint, arXiv:2404.01291

    2024 arXiv

  20. [28]

    Bingchen Liu, Ehsan Akhgari, Alexander Visheratin, Aleks Kamko, Linmiao Xu, Shivam Shrirao, Chase Lambert, Joao Souza, Suhail Doshi, and Daiqing Li. 2024 a . https://arxiv.org/abs/2409.10695 Playground v3: Improving text-to-image alignment with deep-fusion large language model...

    2024 arXiv

  21. [29]

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2024 b . https://arxiv.org/abs/2310.03744 Improved baselines with visual instruction tuning . Preprint, arXiv:2310.03744

    2024 arXiv

  22. [30]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. https://arxiv.org/abs/2304.08485 Visual instruction tuning . Preprint, arXiv:2304.08485

    2023 arXiv

  23. [31]

    Ilya Loshchilov and Frank Hutter. 2019. https://arxiv.org/abs/1711.05101 Decoupled weight decay regularization . Preprint, arXiv:1711.05101

    2019 arXiv

  24. [32]

    Fan Lu, Wei Wu, Kecheng Zheng, Shuailei Ma, Biao Gong, Jiawei Liu, Wei Zhai, Yang Cao, Yujun Shen, and Zheng-Jun Zha. 2025. https://arxiv.org/abs/2412.08614 Benchmarking large vision-language models via directed scene graph for comprehensive image captioning . Preprint, arXiv:...

    2025 arXiv

  25. [33]

    OpenAI, :, Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, Aleksander Mądry, Alex Baker-Whitcomb, Alex Beutel, Alex Borzunov, Alex Carney, Alex Chow, Alex Kirillov, and 401 others. 2024....

    2024 arXiv

  26. [34]

    OpenAI. 2023. https://cdn.openai.com/papers/GPTV_System_Card.pdf Gpt-4v(ision) system card

    2023

  27. [35]

    Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and...

    2022 arXiv

  28. [36]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. https://doi.org/10.3115/1073083.1073135 B leu: a method for automatic evaluation of machine translation . In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311--3...

    2002

  29. [37]

    Ruotian Peng, Haiying He, Yake Wei, Yandong Wen, and Di Hu. 2025. https://arxiv.org/abs/2504.06666 Patch matters: Training-free fine-grained image caption enhancement via local perception . Preprint, arXiv:2504.06666

    2025 arXiv

  30. [38]

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2024. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36

    2024

  31. [39]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2023. https://arxiv.org/abs/1910.10683 Exploring the limits of transfer learning with a unified text-to-text transformer . Preprint, arXiv:1910.10683

    2023 arXiv

  32. [40]

    Shuhuai Ren, Linli Yao, Shicheng Li, Xu Sun, and Lu Hou. 2024. https://arxiv.org/abs/2312.02051 Timechat: A time-sensitive multimodal large language model for long video understanding . Preprint, arXiv:2312.02051

    2024 arXiv

  33. [41]

    Yanpeng Sun, Jing Hao, Ke Zhu, Jiang-Jiang Liu, Yuxiang Zhao, Xiaofan Li, Gang Zhang, Zechao Li, and Jingdong Wang. 2025. https://arxiv.org/abs/2412.14233 Descriptive caption enhancement with visual specialists for multimodal perception . Preprint, arXiv:2412.14233

    2025

  34. [42]

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M. Dai, Anja Hauth, Katie Millican, David Silver, Melvin Johnson, Ioannis Antonoglou, Julian Schrittwieser, Amelia Glaese, Jilin Chen, Emily Pitler, Timothy Lil...

    2024 arXiv

  35. [43]

    Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, Soroosh Mariooryad, Yifan Ding, Xinyang Geng, Fred Alcober, Roy Frostig, Mark Omernick, Lexi Walker, Cosmin Paduraru, Christina Sorokin, a...

    2024 arXiv

  36. [44]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. https://arxiv.org/abs/2302.13971 Llama:...

    2023 arXiv

  37. [45]

    Lawrence Zitnick, and Devi Parikh

    Ramakrishna Vedantam, C. Lawrence Zitnick, and Devi Parikh. 2015. https://arxiv.org/abs/1411.5726 Cider: Consensus-based image description evaluation . Preprint, arXiv:1411.5726

    2015 arXiv

  38. [46]

    Junyang Wang, Yuhang Wang, Guohai Xu, Jing Zhang, Yukai Gu, Haitao Jia, Jiaqi Wang, Haiyang Xu, Ming Yan, Ji Zhang, and Jitao Sang. 2024 a . https://arxiv.org/abs/2311.07397 Amber: An llm-free multi-dimensional benchmark for mllms hallucination evaluation . Preprint, arXiv:2311.07397

    2024 arXiv

  39. [47]

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. 2024 b . https://arxiv.org/abs/2409.12191 Qwen...

    2024 arXiv

  40. [48]

    Weihan Wang, Qingsong Lv, Wenmeng Yu, Wenyi Hong, Ji Qi, Yan Wang, Junhui Ji, Zhuoyi Yang, Lei Zhao, Xixuan Song, Jiazheng Xu, Bin Xu, Juanzi Li, Yuxiao Dong, Ming Ding, and Jie Tang. 2024 c . https://arxiv.org/abs/2311.03079 Cogvlm: Visual expert for pretrained language model...

    2024 arXiv

  41. [49]

    Yuchi Wang, Shuhuai Ren, Rundong Gao, Linli Yao, Qingyan Guo, Kaikai An, Jianhong Bai, and Xu Sun. 2024 d . https://arxiv.org/abs/2404.10763 Ladic: Are diffusion models really inferior to autoregressive counterparts for image-to-text generation? Preprint, arXiv:2404.10763

    2024 arXiv

  42. [50]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. 2023. https://arxiv.org/abs/2201.11903 Chain-of-thought prompting elicits reasoning in large language models . Preprint, arXiv:2201.11903

    2023 arXiv

  43. [51]

    Hu Xu, Po-Yao Huang, Xiaoqing Ellen Tan, Ching-Feng Yeh, Jacob Kahn, Christine Jou, Gargi Ghosh, Omer Levy, Luke Zettlemoyer, Wen tau Yih, Shang-Wen Li, Saining Xie, and Christoph Feichtenhofer. 2024. https://arxiv.org/abs/2410.17251 Altogether: Image captioning via re-alignin...

    2024 arXiv

  44. [52]

    Le Xue, Manli Shu, Anas Awadalla, Jun Wang, An Yan, Senthil Purushwalkam, Honglu Zhou, Viraj Prabhu, Yutong Dai, Michael S Ryoo, Shrikant Kendre, Jieyu Zhang, Can Qin, Shu Zhang, Chia-Chih Chen, Ning Yu, Juntao Tan, Tulika Manoj Awalgaonkar, Shelby Heinecke, and 8 others. 2024...

    2024

  45. [53]

    Linli Yao, Weijing Chen, and Qin Jin. 2023. https://arxiv.org/abs/2211.09371 Capenrich: Enriching caption semantics for web images via cross-modal pre-trained knowledge . Preprint, arXiv:2211.09371

    2023 arXiv

  46. [54]

    Qinghao Ye, Xianhan Zeng, Fu Li, Chunyuan Li, and Haoqi Fan. 2025. https://arxiv.org/abs/2503.07906 Painting with words: Elevating detailed image captioning with benchmark and alignment learning . Preprint, arXiv:2503.07906

    2025 arXiv

  47. [55]

    Qiying Yu, Quan Sun, Xiaosong Zhang, Yufeng Cui, Fan Zhang, Yue Cao, Xinlong Wang, and Jingjing Liu. 2024. https://arxiv.org/abs/2310.20550 Capsfusion: Rethinking image-text data at scale . Preprint, arXiv:2310.20550

    2024 arXiv

  48. [56]

    Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. 2024. http://arxiv.org/abs/2403.13372 Llamafactory: Unified efficient fine-tuning of 100+ language models . In Proceedings of the 62nd Annual Meeting of the Association for Compu...

    2024 arXiv

  49. [57]

    Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. 2023. https://arxiv.org/abs/2304.10592 Minigpt-4: Enhancing vision-language understanding with advanced large language models . Preprint, arXiv:2304.10592

    2023 arXiv

  50. [58]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  51. [59]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.