Pith. sign in

REVIEW 11 cited by

MIA-DPO: Multi-Image Augmented Direct Preference Optimization For Large Vision-Language Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2410.17637 v1 pith:BTEE4D42 submitted 2024-10-23 cs.CV cs.AI

Ziyu Liu , Yuhang Zang , Xiaoyi Dong , Pan Zhang , Yuhang Cao , Haodong Duan , Conghui He , Yuanjun Xiong
show 2 more authors
Dahua Lin Jiaqi Wang
This is my paper · ORCID
classification cs.CVcs.AI
keywords multi-imagedatamia-dpopreferenceoptimizationrejectedvisualalignment
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Visual preference alignment involves training Large Vision-Language Models (LVLMs) to predict human preferences between visual inputs. This is typically achieved by using labeled datasets of chosen/rejected pairs and employing optimization algorithms like direct preference optimization (DPO). Existing visual alignment methods, primarily designed for single-image scenarios, struggle to effectively handle the complexity of multi-image tasks due to the scarcity of diverse training data and the high cost of annotating chosen/rejected pairs. We present Multi-Image Augmented Direct Preference Optimization (MIA-DPO), a visual preference alignment approach that effectively handles multi-image inputs. MIA-DPO mitigates the scarcity of diverse multi-image training data by extending single-image data with unrelated images arranged in grid collages or pic-in-pic formats, significantly reducing the costs associated with multi-image data annotations. Our observation reveals that attention values of LVLMs vary considerably across different images. We use attention values to identify and filter out rejected responses the model may have mistakenly focused on. Our attention-aware selection for constructing the chosen/rejected pairs without relying on (i) human annotation, (ii) extra data, and (iii) external models or APIs. MIA-DPO is compatible with various architectures and outperforms existing methods on five multi-image benchmarks, achieving an average performance boost of 3.0% on LLaVA-v1.5 and 4.3% on the recent InternLM-XC2.5. Moreover, MIA-DPO has a minimal effect on the model's ability to understand single images.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 11 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Probing Visual Language Priors in VLMs

    cs.CV 2024-12 conditional novelty 7.0 of 10

    ViLP shows that vision-language models often answer from text priors instead of image content, and an image-corruption DPO method partially fixes this.

  2. HUGIN: Enhancing Vision-Language Planning for Autonomous Logistics Sorting

    cs.AI 2026-08 conditional novelty 6.0 of 10

    HUGIN, a training framework combining fact-recombination augmentation and global-context ranking, raises sorting-plan accuracy on SortingBench, improving Qwen3-VL-8B from 63.6% to 78.8%.

  3. Direct Diffusion Score Preference Optimization via Stepwise Contrastive Policy-Pair Supervision

    cs.CV 2025-12 conditional novelty 6.0 of 10

    Diffusion image models can be aligned without human labels by supervising every denoising step with score targets from original versus degraded prompts.

  4. FantasyTalking2: Timestep-Layer Adaptive Preference Optimization for Audio-Driven Portrait Animation

    cs.CV 2025-08 unverdicted novelty 6.0 of 10

    A three-part system, Talking-Critic, Talking-NSQ, and TLPO, aligns diffusion portrait animation models to human preferences and improves lip-sync, motion naturalness, and visual quality.

  5. CheXPO: Preference Optimization for Chest X-ray VLMs with Counterfactual Rationale

    cs.CV 2025-07 conditional novelty 6.0 of 10

    A preference optimization strategy using confidence-based hard example mining, similarity retrieval, and synthetic counterfactual rationales improves chest X-ray VQA accuracy by 8.93% relative over supervised fine-tuning.

  6. ScaleCap: Inference-Time Scalable Image Captioning via Dual-Modality Debiasing

    cs.CV 2025-06 conditional novelty 6.0 of 10

    ScaleCap is an inference-time captioning pipeline that enriches captions with heuristic questions and filters hallucinations via offline contrastive sentence rating, yielding a 450K dataset that improves LVLM pretrain...

  7. Visual Agentic Reinforcement Fine-Tuning

    cs.CV 2025-05 conditional novelty 6.0 of 10

    Reinforcement fine-tuning with verifiable rewards enables open-source vision-language models to use web search and image-processing code, improving visual QA and multi-hop reasoning.

  8. InternLM-XComposer2.5-Reward: A Simple Yet Effective Multi-Modal Reward Model

    cs.CV 2025-01 conditional novelty 6.0 of 10

    IXC-2.5-Reward is an open-source multimodal reward model that achieves 70.0% macro accuracy on VL-RewardBench and improves LVLM chat via PPO.

  9. SymDPO: Boosting In-Context Learning of Large Multimodal Models with Symbol Demonstration Direct Preference Optimization

    cs.CV 2024-11 conditional novelty 6.0 of 10

    SymDPO replaces answer text in multimodal demonstrations with meaningless symbols during preference training, forcing models to use image context and improving in-context learning performance on five benchmarks.

  10. X-Prompt: Towards Universal In-Context Image Generation in Auto-Regressive Vision Language Foundation Models

    cs.CV 2024-12 conditional novelty 5.0 of 10

    X-Prompt compresses in-context image examples into a few learned tokens and adds text-description tasks, enabling a Chameleon-style autoregressive model to handle multiple image generation tasks in one framework.

  11. LeanPO: Lean Preference Optimization for Likelihood Alignment in Video-LLMs

    cs.CV 2025-06 conditional novelty 4.0 of 10

    LeanPO improves Video-LLM alignment by using a reference-free average-likelihood reward, self-generated winning/losing pairs, and dynamic label smoothing, yielding gains on six video benchmarks.

Pith tools