Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Beyond Human Data: Aligning Multimodal Large Language Models by Iterative Self-Evolution

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that a multimodal large language model can improve its own visual question answering and description abilities using only unannotated images, with no human- or GPT-written questions, no ground-truth answers, and no…

desk verdict Credible fully annotation-free MLLM self-evolution recipe; broad gains are real, but test-set tuning and unvalidated preference pairs need cleanup before the 'first' claim is fully trusted. read the letter →

arxiv 2412.15650 v1 pith:CMK5435C submitted 2024-12-20 cs.LG

Wentao Tan , Qiong Cao , Yibing Zhan , Chao Xue , Changxing Ding This is my paper · ORCID
classification cs.LG
keywords multimodallargelanguagemodelsself-evolutionpreferenceoptimizationDPOhallucinationreductionunlabeleddataself-questioningvision-languagealignment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a multimodal large language model can improve its own visual understanding using nothing but unannotated images, eliminating the need for human- or GPT-written questions, ground-truth answers, or auxiliary models. It proposes SENA, a three-part self-evolution loop in which the model writes and checks its own questions, refines its own answers using image descriptions, and then optimizes itself with DPO plus an image-content alignment loss. On LLaVA-1.5-7B, three rounds of this loop raise LLaVA-W from 59.6 to 67.4, MM-Vet from 31.7 to 35.8, and AMBER discriminative F1 from 74.3 to 83.6 while cutting CHAIR hallucination from 7.6 to 4.9. If correct, preference alignment for MLLMs becomes dramatically cheaper and fully scalable to arbitrary image collections.

What carries the argument

The machinery is a three-component self-evolution loop: Image-driven Self-Questioning (SQ) generates a question from the image, asks the model whether it can answer it, and regenerates it if not; Answer Self-Enhancement (SE) writes a caption of the image to produce a descriptive answer, then refines both the descriptive and generated answers using that caption, making the chosen answer more detailed and precise than the rejected answer, which is generated from the same image corrupted by $T=600$ diffusion-noise steps; and Image Content Alignment (CA) adds a loss term maximizing the log-likelihood of the descriptive answer under the current model, anchoring the model to the actual image content. The preference pairs are then optimized with the standard DPO loss, Eq. (2), so the model learns to raise the probability of the enhanced answer and lower the probability of the noisy-image answer.

What would settle it

Sample a random set of the preference pairs SENA constructs, show the two answers to human annotators without revealing which is chosen or rejected, and measure how often the paper's chosen answer is judged better; if that win rate is not clearly above chance, the DPO update in Eq. (2) would be training on reversed preferences and the reported gains would not come from genuine preference alignment. A direct check of whether self-enhancement actually adds correct details rather than verbose hallucination would also settle the question.

Watch

Extended reading notes

Core claim

The central claim is that a single multimodal large language model can serve as its own data generator, labeler, and critic, so that human preference alignment can proceed without any labeled data. The framework, SENA, works in three steps: the model generates questions about an image and regenerates them if it judges them unanswerable or irrelevant; it produces a chosen answer from the original image and a rejected answer from the same image corrupted by diffusion noise, then enhances the chosen answer using a caption of the image; and it optimizes with DPO plus an image-content alignment loss that maximizes the likelihood of the caption, keeping the model's attention on the image. The authors report that three iterations of this loop improve LLaVA-1.5-7B on generative and discriminative benchmarks, roughly matching or exceeding methods that use human- or GPT-annotated data, ground-truth answers, or extra models, and that the gains extend to 13B and Qwen2-VL-7B base models.

Load-bearing premise

The load-bearing premise is that the model's self-generated preference signal is reliable: the answer refined from its own caption is consistently better than the answer produced from a diffusion-noised image, for both descriptive and generated questions.

Editorial extensions

If this is right

  • MLLM preference alignment can operate on arbitrary unannotated image collections, removing the need for question-answer annotation pipelines.
  • The same loop transfers to larger base models; the paper reports improvements on LLaVA-1.5-13B and Qwen2-VL-7B without changing the framework.
  • The image-content alignment loss reduces object hallucination, as measured by CHAIR and MMHal-Bench, which addresses a known failure mode of self-training.
  • Because no auxiliary model is used, the framework avoids the cost and possible drift of distilling from a larger teacher, making iterative self-improvement a closed loop.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The self-questioning step could serve as a general-purpose question filter for other self-training pipelines, since it removes unanswerable or off-image questions without human review.
  • The diffusion-noise schedule $T$ is currently global; an image-adaptive schedule, using less noise for simple images and more for complex ones, might extend the performance plateau the paper observes after three rounds.
  • The image-content alignment loss could be reused at inference time as a scoring function to select among candidate answers, turning the training objective into a lightweight hallucination filter.
  • A head-to-head test in which SENA's chosen answers are compared with its rejected answers on held-out images would quantify how much of the benchmark gain comes from preference optimization versus from the added descriptive knowledge in the alignment loss.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes SENA, an iterative self-evolution framework for multimodal large language models that uses only unlabeled images. It replaces human- or GPT-annotated questions with model-generated questions filtered by a self-questioning (SQ) mechanism, constructs chosen answers through an answer self-enhancement (SE) step that uses the model's own image descriptions, and builds rejected answers by prompting the model on diffusion-noised images. The model is then optimized with a DPO loss (Eq. 2) plus an image content alignment loss (Eq. 3) that maximizes the likelihood of the self-generated descriptions. Experiments on LLaVA-1.5-7B, LLaVA-1.5-13B, and Qwen2-VL-7B report consistent improvements across LLaVA-W, MM-Vet, MMHal-Bench, AMBER, and MMBench, and show competitive performance with methods that rely on annotated data or extra models.

Significance. The core idea is timely and the reported gains are large (e.g., +7.8 on LLaVA-W and +9.3 F1 on AMBER discriminative in Table 4). If the self-generated preference signal is reliable, the framework would be a notable step toward low-cost multimodal alignment without any labeled data, external reward models, or ground-truth annotations. The paper is generally well structured and includes a code release, ablations of each component (Tables 1-3), and supplementary experiments on model transfer (Tables 10-11). However, the current evidence does not yet establish the reliability of the self-referential training signal, and several evaluation-hygiene issues need to be addressed before the central claim can be accepted.

major comments (4)
  1. [Generate Preference Data, Eq. (2)] The DPO preference pairs are entirely self-generated: yse_w is the model's own self-enhanced output (Algorithm 1, lines 15-16) and y_l is the same model's output on a noised image (lines 14-15). The paper's only evidence that yse_w is actually preferable is a few qualitative examples and CLIP-score snapshots (Figs. 3 and 6), and the paper itself argues against CLIP-based quality selection when discussing CSR in 'Comparison with SOTA'. No human or external verification of the preference labels is provided. If the self-enhancement step only adds verbose hallucinated detail (as in the Fig. 3 change from 'a zebra' to 'three zebras' and 'lush green forest'), DPO will reinforce those hallucinations. Please include a systematic evaluation of the preference pairs, e.g., human ratings or win rates against a grounded reference on a random sample, and report the factual-error rate of yse_w versus y_l.
  2. [Image Content Alignment, Eq. (3)] The alignment loss maximizes the likelihood of yse_des, which is itself a model-generated, self-enhanced description. The paper claims this directs the model's attention to image content, but the target sequence is not connected to any verified ground truth. For instance, the enhanced description in Fig. 3 introduces 'three zebras' and 'lush green forest'; if these are incorrect, Eq. (3) actively trains the model to hallucinate. The authors should demonstrate on a held-out set that the CA loss reduces object-level hallucinations (e.g., CHAIR on a captioning task) relative to the base model, rather than only increasing the likelihood of the model's own descriptions.
  3. [Tables 7 and 8, with Table 4] The values M=6K and T=600 used in the main evaluation appear to be selected after inspecting performance on the same benchmark suite on which the final numbers are reported. This is a form of model selection on the test set and can inflate the apparent gains. Please either choose these hyperparameters on a validation split that is disjoint from the evaluation benchmarks, or report the full grid of M and T in the main text and show that the qualitative ranking against the baselines is stable across reasonable settings.
  4. [Implementation Details and Evaluation] The training images are sampled from LLaVA-665k, which includes COCO, GQA, and TextVQA images, while LLaVA-W and AMBER (and to some extent other benchmarks) also draw images from COCO. The paper does not analyze whether any of the 18K training images appear in the evaluation sets. Without this overlap analysis, part of the observed improvement could be due to memorization of specific test images. Please report the overlap (e.g., by image IDs and, if necessary, duplicate-image detection) and, if overlap exists, provide results on the subset of evaluation examples whose images were not seen during training.
minor comments (5)
  1. [Experiments] Tables 1-4 are based on a single training run with one random sampling of images (Algorithm 1, line 1). Given the stochasticity in image selection and training, reporting mean and standard deviation across at least three runs would make the improvements more convincing.
  2. [Comparison with SOTA] The table grouping 'with annotated data or extra models' includes CSR, but the main text does not specify what external information CSR uses; adding a sentence to clarify the comparison conditions would help the reader interpret the results.
  3. [Figure 3] The CLIP scores are presented as indicators of answer quality, but it should be stated explicitly that these scores are not used in the training objective, avoiding an apparent inconsistency with the criticism of CSR.
  4. [Conclusion] The phrase 'multi-model self-evolution framework' should read 'multimodal self-evolution framework' (typo).
  5. [Algorithm 1] The questions qgen and qsq_gen are generated by θ0 before the evolution loop, which means only answers (and not questions) evolve across iterations; the main text should state this explicitly to avoid overstating the scope of self-evolution.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: SENA's improvements are evaluated on external benchmarks, and no equation reduces its predictions to its fitted inputs or to a self-citation.

full rationale

The paper makes an empirical algorithmic claim—that iterated DPO plus a caption-likelihood regularizer on self-generated data improves LLaVA-1.5 on held-out benchmarks—not a formal derivation from first principles. I checked the load-bearing steps: question generation (Algorithm 1 lines 3–8) uses the model itself with self-questioning; chosen/rejected generation (lines 12–16) uses original and diffusion-noised images; optimization (Eq. 2) is standard DPO on those pairs; Eq. (3) maximizes likelihood of yse_des, the model's own enhanced caption. This last term is self-referential as a training target, but it is not an equation that reduces a predicted benchmark result to a fitted input, nor is it imported from a same-author uniqueness theorem. No parameter is fitted to a benchmark subset and then reported as a prediction on that same quantity. The self-generated preference signal is the framework's intended premise rather than a hidden circular reduction, and the paper candidly acknowledges limitations such as performance lagging behind annotated-data methods and a plateau after three iterations. The lack of an external validator for yse_w vs yl, and the use of CLIP scores as quality evidence while criticizing CLIP in CSR, are validity and robustness concerns, not circularity under the stated criteria. The central claims are therefore self-contained against external benchmarks.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The framework adds no invented entities and introduces four hand-set hyperparameters (T, M, beta, N). Its central assumptions are that self-generated preference pairs are reliable, that self-captioning reduces hallucination, that self-question filtering works, and that the LLaVA-665k image pool does not contaminate evaluation. The absence of external preference labels makes these assumptions load-bearing.

free parameters (4)
  • Diffusion noise steps T = 600
    Tuned by ablation on the evaluation benchmarks (Table 8) to maximize generative and discriminative scores; at T=600 performance peaks.
  • Number of images per iteration M = 6,000
    Selected from 2K/4K/6K/8K ablations (Table 7); larger M helps until 6K, then performance plateaus or drops.
  • DPO scale beta = 0.1
    Hand-set; no ablation reported, standard DPO range.
  • Evolution iterations N = 3
    Stopping point chosen by the paper based on observed performance plateau after three rounds; not justified independently.
assumptions (6)
  • standard math Autoregressive factorization of response likelihood (Eq. 1) correctly models generation
    Used to compute DPO ratio and alignment loss; standard for transformer LMs.
  • domain assumption DPO effectively aligns a model when preference pairs are reliable
    DPO (Eq. 2) is adopted from prior work; its guarantees are assumed, not rederived.
  • domain assumption Self-enhanced answers are consistently better than noisy-image answers
    Central to preference construction (Algorithm 1, steps 14-16); no external validation beyond examples and CLIP scores.
  • domain assumption Maximizing likelihood of self-generated descriptions (Eq. 3) reduces hallucinations
    Motivated by hallucination literature and supported only by ablation; self-generated captions may contain errors that get reinforced.
  • domain assumption Self-questioning prompt (psq) reliably detects unanswerable questions
    The SQ step assumes the model's self-assessment of question answerability is informative; no quantitative evaluation of filtered question quality.
  • domain assumption Training on LLaVA-665k images does not overlap with evaluation benchmarks in a way that inflates gains
    The paper samples 18K unlabeled images from the same pool used to train LLaVA-1.5 but does not check overlap with LLaVA-W, MM-Vet, MMHal, or AMBER.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Beyond Human Data: Aligning Multimodal Large Language Models by Iterative Self-Evolution." pith.science (2026). https://pith.science/paper/CMK5435C

@misc{pith2026241215650,
  author       = {Pith},
  title        = {Pith review of: Beyond Human Data: Aligning Multimodal Large Language Models by Iterative Self-Evolution},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CMK5435C}},
  note         = {Machine review of arXiv:2412.15650}
}
read the original abstract

Human preference alignment can greatly enhance Multimodal Large Language Models (MLLMs), but collecting high-quality preference data is costly. A promising solution is the self-evolution strategy, where models are iteratively trained on data they generate. However, current techniques still rely on human- or GPT-annotated data and sometimes require additional models or ground truth answers. To address these issues, we propose a novel multimodal self-evolution framework that enables the model to autonomously generate high-quality questions and answers using only unannotated images. First, we implement an image-driven self-questioning mechanism, allowing the model to create and evaluate questions based on image content, regenerating them if they are irrelevant or unanswerable. This sets a strong foundation for answer generation. Second, we introduce an answer self-enhancement technique, starting with image captioning to improve answer quality. We also use corrupted images to generate rejected answers, forming distinct preference pairs for optimization. Finally, we incorporate an image content alignment loss function alongside Direct Preference Optimization (DPO) loss to reduce hallucinations, ensuring the model focuses on image content. Experiments show that our framework performs competitively with methods using external information, offering a more efficient and scalable approach to MLLMs.

Figures

Figures reproduced from arXiv: 2412.15650 by the authors.

Figure 1
Figure 1. Comparisons between (a) traditional framework and (b) our framework. Our framework combines carefully designed [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Illustration of the Image-Driven Self-Questioning. SQ checks whether [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Illustration of the Answer Self-Enhancement techniques. SE analyzes the previous question-and-answer pairs with [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Comparison of outputs from various models on different visual tasks in MMHal-Bench. Best viewed in color. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: The General Descriptive Prompt Set Pdes. SENA enables the model to generate questions qgen about images. We find that qgen sometimes focuses only on a specific object in the image, neglecting other important in￾formation. To maximize the knowledge the model learns from…
Figure 6
Figure 6. Figure 6: The General Descriptive Prompt Set Pdes. The quality of randomly generated model answers is of￾ten similar, as shown in [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. From Answers to Rationales: Self-Aligning Multimodal Reasoning with Answer-Oriented Chain-of-Thought

    cs.CL 2025-07 conditional novelty 5.0 of 10

    Answer-oriented chain-of-thought prompts that generate both positive and negative reasoning data, combined with iterative DPO, improve multimodal LLM reasoning on several benchmarks.

Reference graph

Works this paper leans on

54 extracted references · 8 canonical work pages · cited by 1 Pith paper

  1. [1]

    L.; Almeida, D.; Altenschmidt, J.; Altman, S.; Anadkat, S.; et al

    Achiam, J.; Adler, S.; Agarwal, S.; Ahmad, L.; Akkaya, I.; Aleman, F. L.; Almeida, D.; Altenschmidt, J.; Altman, S.; Anadkat, S.; et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  2. [2]

    Ahn, D.; Choi, Y.; Kim, S.; Yu, Y.; Kang, D.; and Choi, J. 2024. i-SRT: Aligning Large Multimodal Models for Videos by Iterative Self-Retrospective Judgment. arXiv preprint arXiv:2406.11280

    arXiv 2024

  3. [3]

    Amirloo, E.; Fauconnier, J.-P.; Roesmann, C.; Kerl, C.; Boney, R.; Qian, Y.; Wang, Z.; Dehghan, A.; Yang, Y.; Gan, Z.; et al. 2024. Understanding Alignment in Multimodal LLMs: A Comprehensive Study. arXiv preprint arXiv:2407.02477

    arXiv 2024

  4. [4]

    Bai, J.; Bai, S.; Yang, S.; Wang, S.; Tan, S.; Wang, P.; Lin, J.; Zhou, C.; and Zhou, J. 2023. Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond. arXiv preprint arXiv:2308.12966

    arXiv 2023

  5. [5]

    A.; Richemond, P

    Calandriello, D.; Guo, D.; Munos, R.; Rowland, M.; Tang, Y.; Pires, B. A.; Richemond, P. H.; Lan, C. L.; Valko, M.; Liu, T.; et al. 2024. Human alignment of large language models through online preference optimisation. arXiv preprint arXiv:2403.08635

    arXiv 2024

  6. [6]

    Chen, K.; Zhang, Z.; Zeng, W.; Zhang, R.; Zhu, F.; and Zhao, R. 2023. Shikra: Unleashing multimodal llm's referential dialogue magic. arXiv preprint arXiv:2306.15195

    arXiv 2023

  7. [7]

    Chen, Z.; Deng, Y.; Yuan, H.; Ji, K.; and Gu, Q. 2024. Self-play fine-tuning converts weak language models to strong language models. arXiv preprint arXiv:2401.01335

    arXiv 2024

  8. [8]

    Dai, W.; Li, J.; Li, D.; Tiong, A. M. H.; Zhao, J.; Wang, W.; Li, B.; Fung, P.; and Hoi, S. 2023. InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning. arXiv:2305.06500

    arXiv 2023

Show all 54 references
  1. [9]

    Deng, Y.; Lu, P.; Yin, F.; Hu, Z.; Shen, S.; Zou, J.; Chang, K.-W.; and Wang, W. 2024. Enhancing Large Vision Language Models with Self-Training on Image Comprehension. arXiv preprint arXiv:2405.19716

    2024 arXiv

  2. [10]

    Dong, H.; Xiong, W.; Pang, B.; Wang, H.; Zhao, H.; Zhou, Y.; Jiang, N.; Sahoo, D.; Xiong, C.; and Zhang, T. 2024. Rlhf workflow: From reward modeling to online rlhf. arXiv preprint arXiv:2405.07863

    2024 arXiv

  3. [11]

    Guo, S.; Zhang, B.; Liu, T.; Liu, T.; Khalman, M.; Llinares, F.; Rame, A.; Mesnard, T.; Zhao, Y.; Piot, B.; et al. 2024. Direct language model alignment from online ai feedback. arXiv preprint arXiv:2402.04792

    2024 arXiv

  4. [12]

    Huang, Q.; Dong, X.; Zhang, P.; Wang, B.; He, C.; Wang, J.; Lin, D.; Zhang, W.; and Yu, N. 2024. Opera: Alleviating hallucination in multi-modal large language models via over-trust penalty and retrospection-allocation. In CVPR

    2024

  5. [13]

    A.; and Manning, C

    Hudson, D. A.; and Manning, C. D. 2019. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In CVPR

    2019

  6. [14]

    Jiang, C.; Xu, H.; Dong, M.; Chen, J.; Ye, W.; Yan, M.; Ye, Q.; Zhang, J.; Huang, F.; and Zhang, S. 2024. Hallucination augmented contrastive learning for multimodal large language model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 270...

    2024

  7. [15]

    A.; et al

    Krishna, R.; Zhu, Y.; Groth, O.; Johnson, J.; Hata, K.; Kravitz, J.; Chen, S.; Kalantidis, Y.; Li, L.-J.; Shamma, D. A.; et al. 2017. Visual genome: Connecting language and vision using crowdsourced dense image annotations. International journal of computer vision, 123: 32--73

    2017

  8. [16]

    Leng, S.; Zhang, H.; Chen, G.; Li, X.; Lu, S.; Miao, C.; and Bing, L. 2024. Mitigating object hallucinations in large vision-language models through visual contrastive decoding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 13872--13882

    2024

  9. [17]

    Li, J.; Li, D.; Savarese, S.; and Hoi, S. 2023 a . Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning, 19730--19742. PMLR

    2023

  10. [18]

    Li, L.; Xie, Z.; Li, M.; Chen, S.; Wang, P.; Chen, L.; Yang, Y.; Wang, B.; and Kong, L. 2023 b . Silkie: Preference distillation for large visual language models. arXiv preprint arXiv:2312.10665

    2023 arXiv

  11. [19]

    Lin, T.-Y.; Maire, M.; Belongie, S.; Hays, J.; Perona, P.; Ramanan, D.; Doll \'a r, P.; and Zitnick, C. L. 2014. Microsoft coco: Common objects in context. In Computer Vision--ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 1...

    2014

  12. [20]

    Liu, F.; Lin, K.; Li, L.; Wang, J.; Yacoob, Y.; and Wang, L. 2023 a . Mitigating hallucination in large multi-modal models via robust instruction tuning. In The Twelfth International Conference on Learning Representations

    2023

  13. [21]

    Liu, H.; Li, C.; Li, Y.; and Lee, Y. J. 2024 a . Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 26296--26306

    2024

  14. [22]

    Liu, H.; Li, C.; Li, Y.; Li, B.; Zhang, Y.; Shen, S.; and Lee, Y. J. 2024 b . LLaVA-NeXT: Improved reasoning, OCR, and world knowledge

    2024

  15. [23]

    Liu, H.; Li, C.; Wu, Q.; and Lee, Y. J. 2024 c . Visual instruction tuning. Advances in neural information processing systems, 36

    2024

  16. [24]

    Liu, Y.; Duan, H.; Zhang, Y.; Li, B.; Zhang, S.; Zhao, W.; Yuan, Y.; Wang, J.; He, C.; Liu, Z.; et al. 2023 b . Mmbench: Is your multi-modal model an all-around player? arXiv preprint arXiv:2307.06281

    2023 arXiv

  17. [25]

    Lu, J.; Zhong, W.; Huang, W.; Wang, Y.; Mi, F.; Wang, B.; Wang, W.; Shang, L.; and Liu, Q. 2023. Self: Language-driven self-evolution for large language model. arXiv preprint arXiv:2310.00533

    2023 arXiv

  18. [26]

    K.; and Chakraborty, A

    Mishra, A.; Shekhar, S.; Singh, A. K.; and Chakraborty, A. 2019. Ocr-vqa: Visual question answering by reading text in images. In 2019 international conference on document analysis and recognition (ICDAR), 947--952. IEEE

    2019

  19. [27]

    D.; Ermon, S.; and Finn, C

    Rafailov, R.; Sharma, A.; Mitchell, E.; Manning, C. D.; Ermon, S.; and Finn, C. 2024. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36

    2024

  20. [28]

    Rosset, C.; Cheng, C.-A.; Mitra, A.; Santacroce, M.; Awadallah, A.; and Xie, T. 2024. Direct nash optimization: Teaching language models to self-improve with general preferences. arXiv preprint arXiv:2404.03715

    2024 arXiv

  21. [29]

    Singh, A.; Natarajan, V.; Shah, M.; Jiang, Y.; Chen, X.; Batra, D.; Parikh, D.; and Rohrbach, M. 2019. Towards vqa models that can read. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 8317--8326

    2019

  22. [30]

    Sun, Z.; Shen, S.; Cao, S.; Liu, H.; Li, C.; Shen, Y.; Gan, C.; Gui, L.-Y.; Wang, Y.-X.; Yang, Y.; et al. 2023. Aligning large multimodal models with factually augmented rlhf. arXiv preprint arXiv:2309.14525

    2023 arXiv

  23. [31]

    S.; and Agarwal, A

    Swamy, G.; Dann, C.; Kidambi, R.; Wu, Z. S.; and Agarwal, A. 2024. A minimaximalist approach to reinforcement learning from human feedback. arXiv preprint arXiv:2401.04056

    2024 arXiv

  24. [32]

    Tan, W.; Ding, C.; Jiang, J.; Wang, F.; Zhan, Y.; and Tao, D. 2024. Harnessing the Power of MLLMs for Transferable Text-to-Image Person ReID. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 17127--17137

    2024

  25. [33]

    Tan, W.; Ding, C.; Wang, P.; Gong, M.; and Jia, K. 2023. Style interleaved learning for generalizable person re-identification. IEEE Transactions on Multimedia

    2023

  26. [34]

    Wang, G.; Ding, C.; Tan, W.; and Tan, M. 2024 a . Decoupled Prototype Learning for Reliable Test-Time Adaptation. arXiv preprint arXiv:2401.08703

    2024 arXiv

  27. [35]

    Wang, J.; Wang, Y.; Xu, G.; Zhang, J.; Gu, Y.; Jia, H.; Yan, M.; Zhang, J.; and Sang, J. 2023. An LLM-free Multi-dimensional Benchmark for MLLMs Hallucination Evaluation. arXiv preprint arXiv:2311.07397

    2023 arXiv

  28. [36]

    Wang, P.; Bai, S.; Tan, S.; Wang, S.; Fan, Z.; Bai, J.; Chen, K.; Liu, X.; Wang, J.; Ge, W.; Fan, Y.; Dang, K.; Du, M.; Ren, X.; Men, R.; Liu, D.; Zhou, C.; Zhou, J.; and Lin, J. 2024 b . Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution. ar...

    2024 arXiv

  29. [37]

    Wang, P.; Ding, C.; Tan, W.; Gong, M.; Jia, K.; and Tao, D. 2022. Uncertainty-aware clustering for unsupervised domain adaptive object re-identification. IEEE Transactions on Multimedia, 25: 2624--2635

    2022

  30. [38]

    Wang, X.; Chen, J.; Wang, Z.; Zhou, Y.; Zhou, Y.; Yao, H.; Zhou, T.; Goldstein, T.; Bhatia, P.; Huang, F.; et al. 2024 c . Enhancing Visual-Language Modality Alignment in Large Vision Language Models via Self-Improvement. arXiv preprint arXiv:2405.15973

    2024 arXiv

  31. [39]

    Xiong, W.; Dong, H.; Ye, C.; Wang, Z.; Zhong, H.; Ji, H.; Jiang, N.; and Zhang, T. 2024. Iterative preference learning from human feedback: Bridging theory and practice for rlhf under kl-constraint. In Forty-first International Conference on Machine Learning

    2024

  32. [40]

    Ye, Q.; Xu, H.; Ye, J.; Yan, M.; Hu, A.; Liu, H.; Qian, Q.; Zhang, J.; and Huang, F. 2024. mplug-owl2: Revolutionizing multi-modal large language model with modality collaboration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 13040--13051

    2024

  33. [41]

    Yu, T.; Yao, Y.; Zhang, H.; He, T.; Han, Y.; Cui, G.; Hu, J.; Liu, Z.; Zheng, H.-T.; Sun, M.; et al. 2024 a . Rlhf-v: Towards trustworthy mllms via behavior alignment from fine-grained correctional human feedback. In Proceedings of the IEEE/CVF Conference on Computer Vision an...

    2024

  34. [42]

    Yu, T.; Zhang, H.; Yao, Y.; Dang, Y.; Chen, D.; Lu, X.; Cui, G.; He, T.; Liu, Z.; Chua, T.-S.; et al. 2024 b . Rlaif-v: Aligning mllms through open-source ai feedback for super gpt-4v trustworthiness. arXiv preprint arXiv:2405.17220

    2024

  35. [43]

    Yu, W.; Yang, Z.; Li, L.; Wang, J.; Lin, K.; Liu, Z.; Wang, X.; and Wang, L. 2023. Mm-vet: Evaluating large multimodal models for integrated capabilities. arXiv preprint arXiv:2308.02490

    2023 arXiv

  36. [44]

    Y.; Cho, K.; Sukhbaatar, S.; Xu, J.; and Weston, J

    Yuan, W.; Pang, R. Y.; Cho, K.; Sukhbaatar, S.; Xu, J.; and Weston, J. 2024. Self-rewarding language models. arXiv preprint arXiv:2401.10020

    2024 arXiv

  37. [45]

    Zhang, D.; Zhang, H.; Tang, J.; Hua, X.-S.; and Sun, Q. 2020 a . Causal intervention for weakly-supervised semantic segmentation. Advances in Neural Information Processing Systems, 33: 655--666

    2020

  38. [46]

    Zhang, D.; Zhang, H.; Tang, J.; Wang, M.; Hua, X.; and Sun, Q. 2020 b . Feature pyramid transformer. In Computer Vision--ECCV 2020: 16th European Conference, Glasgow, UK, August 23--28, 2020, Proceedings, Part XXVIII 16, 323--339. Springer

    2020

  39. [47]

    A.; Hu, K.; Liu, S.; Zhang, Y.; Yang, J.; Li, C.; et al

    Zhang, K.; Li, B.; Zhang, P.; Pu, F.; Cahyono, J. A.; Hu, K.; Liu, S.; Zhang, Y.; Yang, J.; Li, C.; et al. 2024. Lmms-eval: Reality check on the evaluation of large multimodal models. arXiv preprint arXiv:2407.12772

    2024 arXiv

  40. [48]

    Zhou, Y.; Cui, C.; Rafailov, R.; Finn, C.; and Yao, H. 2024 a . Aligning modalities in vision large language models via preference fine-tuning. arXiv preprint arXiv:2402.11411

    2024 arXiv

  41. [49]

    Zhou, Y.; Cui, C.; Yoon, J.; Zhang, L.; Deng, Z.; Finn, C.; Bansal, M.; and Yao, H. 2023. Analyzing and mitigating object hallucination in large vision-language models. arXiv preprint arXiv:2310.00754

    2023 arXiv

  42. [50]

    Zhou, Y.; Fan, Z.; Cheng, D.; Yang, S.; Chen, Z.; Cui, C.; Wang, X.; Li, Y.; Zhang, L.; and Yao, H. 2024 b . Calibrated self-rewarding vision language models. arXiv preprint arXiv:2405.14622

    2024 arXiv

  43. [51]

    Zhu, D.; Chen, J.; Shen, X.; Li, X.; and Elhoseiny, M. 2023. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592

    2023 arXiv

  44. [52]

    Zhu, K.; Zhao, L.; Ge, Z.; and Zhang, X. 2024. Self-Supervised Visual Preference Alignment. arXiv preprint arXiv:2404.10501

    2024 arXiv

  45. [53]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all...

  46. [54]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.