Pith. sign in

REVIEW 4 major objections 6 minor 62 references

Multi-reference image generation improves when references are tagged with identifiers and instructions point to those identifiers, rather than describing each reference in natural language.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 22:43 UTC pith:P3HVS6SD

load-bearing objection Clear, practical reformulation of multi-reference generation with a genuinely useful real-image dataset and benchmark; the identifier-format claim is plausible but the headline evaluation confounds format with extra training data. the 4 major comments →

arxiv 2607.15619 v2 pith:P3HVS6SD submitted 2026-07-17 cs.CV

StructGen: Disambiguating Multi-Reference Image Generation via Structured Context Modeling

Jianing Peng , Mengyu Wang , Henghui Ding , Zixiang Li , Ting Liu , Xiaochao Qu , Luoqi Liu , Yao Zhao
show 1 more author
Yunchao Wei
This is my paper
classification cs.CV
keywords multi-reference image generationstructured contextidentifier-based instructiondiffusion modelsdataset curationbenchmarkhuman-centric generationattribute-subject association
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to establish that the main bottleneck in multi-reference image generation is the input format, not just model capacity. It argues that natural-language instructions describing several reference images quickly become verbose and ambiguous, so the paper proposes StructGen, which encodes references in a dictionary-like structure in which each image is assigned a unique identifier (for example, , , ) and the instruction composes those identifiers explicitly. This converts cross-reference association from implicit reasoning into explicit searching, which the paper claims yields more accurate attribute–subject grounding and more consistent generation. To support this, the paper builds a real-image-based data curation pipeline, a training framework that fine-tunes a unified multimodal model on the structured format, and a benchmark with eight human-centric subsets; experiments report consistent gains in prompt following, subject consistency, and facial identity preservation, particularly as the number of references grows.

Core claim

The central claim is that replacing free-form natural language with a structured, identifier-based context — a reference dictionary paired with an identifier-based instruction — improves multi-reference image generation. Identifiers act as explicit anchors: they avoid entangling content and relations in ambiguous language, turning cross-reference association from implicit understanding into explicit searching. The paper further claims that its curation pipeline, which parses real-world images into structured fields rather than distilling synthetic compositions, preserves spatial diversity and human–environment interactions that prior datasets lack, and that this combination yields higher fid

What carries the argument

The load-bearing object is the structured context: a reference dictionary D_ref = {id_i : z_i} that maps each identifier (such as <human_1>, <obj_1>, <scene>) to either its encoded image tokens or its detailed caption, concatenated with an identifier-based instruction c_id. This replaces the fixed-order concatenation of image-then-text tokens that base models use, and it is what turns reference resolution into an explicit search over dictionary entries. Supporting it are a four-stage data pipeline (collection with face/human filtering, structured vision-language captioning, generative extraction with pose perturbation and identity-matched post-filtering, and LLM-based instruction synthesis)

Load-bearing premise

The evaluation attributes the performance gains to the structured context format, yet StructGen is trained on a larger, curated dataset than the baselines and receives identifier-tagged inputs where baselines receive natural language, so two variables change at once.

What would settle it

Fine-tune a strong baseline on the exact same combined training mixture (the 15,965 curated structured samples plus the prior multi-image training data, same number of training steps, same trainable layers) but in plain natural-language format, and evaluate on the same benchmark subsets; if scores match or exceed StructGen's, the structured format is not the source of the gains. Conversely, if ablating to natural language drops performance even with the full data mixture, the format claim is supported.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Adopting a dictionary-and-identifier context requires no architectural changes to interleaved multimodal models, so the format could transfer across model families as a drop-in training recipe.
  • Structured instructions are far more concise than natural-language equivalents (about 64 versus 101 tokens in the paper's example), reducing user effort and the model's reasoning load simultaneously.
  • Because references can be supplied as images or as captions interchangeably, the same framework covers both multi-image composition and text-only generation with precise attribute association.
  • Real-world image curation with identity post-filtering yields a training distribution that better reflects natural interactions and backgrounds, which likely explains gains in subject and scene consistency beyond the input-format effect.
  • The gains concentrate in the most complex settings — multi-human with scene references — where natural-language baselines nearly fail, so structured grounding appears most valuable precisely when association complexity is highest.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Implicit extension: the dictionary-plus-identifier pattern should generalize beyond human-centric data to any enumerable reference set (object composition, layout-controlled generation, multi-image editing), since nothing in the mechanism is specific to people or clothing; a larger-scale application would test this.
  • The paper's mechanistic claim predicts a testable artifact: in attention maps, identifier tokens should attend strongly to their dictionary entries during generation; measuring that would provide direct evidence for the 'explicit searching' story.
  • Because the evaluation changes input format and training data simultaneously relative to baselines, the cleanest inference is that the format and the data each contribute, but their independent contributions on the full data mixture are not yet isolated; a controlled data-matched comparison is the natural next step.
  • The dataset is small (~16k curated samples), so scaling the pipeline to more categories and broader scenes would show whether the advantage widens or saturates — a question the paper leaves open.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes StructGen, a framework for multi-reference image generation that replaces free-form natural-language instructions with a structured, dictionary-like context in which each reference image is assigned an identifier (e.g., <human_1>, <obj_1>, <scene>) and the instruction refers to these identifiers. The authors argue that this structured format reduces ambiguity and improves attribute–subject association and generation consistency. To support the approach, they construct a real-image-based training dataset via a four-stage curation pipeline (collection, captioning, extraction, instruction synthesis), fine-tune BAGEL-MICo on this dataset combined with MICo data, and introduce a new benchmark (StructGen Bench) with eight subcategories and three metrics (prompt following, subject consistency, facial ID). Experiments on OmniContext and the proposed benchmark report consistent improvements over OmniGen2, BAGEL, BAGEL-MICo, and Echo-4o.

Significance. If the central claim is established, the paper would make a useful contribution to multi-reference image generation by showing that a structured input format with explicit identifiers can improve grounding and consistency, and by providing a real-image-derived training pipeline and a dedicated benchmark. The strengths of the paper include: a clearly described data curation pipeline that addresses a genuine data-scarcity problem; a new benchmark with eight controlled subcategories; the use of multiple complementary metrics; and the commitment to release code. The paper also includes an ablation study, which is the right methodological instinct. However, the central claim — that the structured format itself, rather than the additional curated training data or the different input variant used during evaluation, is responsible for the gains — is not cleanly supported by the current experiments.

major comments (4)
  1. [Section 4.1, Tables 1 and 2, Table 4] The headline comparisons do not isolate the effect of the structured context. StructGen is fine-tuned from BAGEL-MICo on a combined dataset (10,665 MICo-De&Re samples plus 15,965 curated samples, per Table 4), while the principal baselines (BAGEL-MICo, Echo-4o, BAGEL, OmniGen2) are not trained on this combined data in the same format. Moreover, on OmniContext, StructGen receives a structured variant with identifiers while the baselines receive the natural-language version. Thus two variables — input format and training data — change simultaneously. The claim in Section 3.1 that identifier anchors themselves improve reference grounding and generation consistency is load-bearing; to support it, the authors should add a control that trains BAGEL-MICo (or a same-initialization model) on the same combined data with natural-language instructions, and evaluate StructGen's structured format agai
  2. [Section 4.3, Table 3] The ablation study is too narrow to support the paper's central claim. Table 3 reports only facial ID consistency on two subsets (single- and multi-human with scene reference under Outfit-Conditioned). The headline claims in Tables 1–2 also cover prompt following (PF) and subject consistency (SC), and the Section 3.1 argument is about attribute–subject association and generation consistency, not merely identity preservation. The authors should report PF and SC metrics for configurations (a)–(e), or justify why these are not needed. Without this, the conclusion that 'structured context modeling and our curated dataset each contribute independently' is only tested on one dimension.
  3. [Section 4.3, Table 3] The interpretation of configurations (a)–(d) as 'each isolates a single factor relative to the full configuration (e)' is not accurate. Configuration (a) is 'training on our curated dataset using natural language format' whereas (e) is the full configuration (curated + MICo data, structured format); so (a) versus (e) changes both the input format and the training data composition. Configuration (b) is 'training on MICo data while converted into our structured format,' which changes the dataset composition (MICo-only versus curated+MICo) as well. Thus, even on the ID metric, the individual contributions of 'structured format' and 'curated dataset' are confounded. The authors should add configurations such as: curated data + structured format, curated data + natural language, MICo data + structured format, and MICo data + natural language, all with matched training steps and data size, to
  4. [Section 3.3, StructGen Bench construction] The benchmark uses GPT-4.1 both to synthesize the instructions (including the identifier-based variants) and as the VLM judge for prompt following and subject consistency. Since the method is designed around the identifier format, and the judge is from the same model family that produced the instructions, there is a risk of evaluation bias favoring the format that the judge itself can more easily parse. The authors should provide a human-validation study on a subset of StructGen Bench (e.g., 100–200 samples) to confirm that the GPT-4.1 judgments correlate with human judgments for both input formats, or at least report the agreement rate. This is particularly important because the structured-vs-natural-language comparison is the core of the paper.
minor comments (6)
  1. [Abstract] Typo: 'planing out coherent spatial arrangement' should be 'planning out coherent spatial arrangement.'
  2. [Section 3.2, Reference Extraction] Typo: 'the croped view' should be 'the cropped view.'
  3. [Table 3] The table header 'ID: Single/Multi' is ambiguous; the caption should explicitly state that these are the single-human and multi-human subsets with scene reference under the Outfit-Conditioned task, as done in the text.
  4. [Tables 1 and 2] No confidence intervals or statistical significance tests are reported for the PF/SC/ID differences. Given that the gains on some subsets are modest (e.g., Table 2, Multi Human + Scene ID), providing error bars or a significance test would strengthen the conclusions.
  5. [Section 3.1, Eq. (3)] The notation D_ref = {id_i : z_ref^i} is informal: it is not a standard set-builder definition, since id_i is a token and z_ref^i is a sequence. Clarifying that this is a mapping from identifier tokens to token sequences would improve precision.
  6. [Section 4.1] The paper states that the mixed-sampling strategy samples each reference 'between either the image or text modality with equal probability,' but it is not clear whether this applies at training time only or also at inference. Clarifying this would help reproducibility.

Circularity Check

0 steps flagged

No circularity: the paper's central claim is an empirical format/data claim tested against external baselines, not a derivation that reduces to its inputs.

full rationale

StructGen is an empirical systems paper: there is no mathematical derivation chain whose output is equivalent to its input. The central claim—that structured identifier-based context improves multi-reference generation—is supported by fine-tuning BAGEL-MICo on a newly curated dataset and comparing with external baselines on both the public OmniContext benchmark and a newly built benchmark. No fitted parameter is relabeled as a prediction: the model is trained with a flow-matching loss and evaluated on held-out prompts/metrics. The ablation in Table 3 is partial (ID on two subsets), and the main tables co-vary input format with training data, but that is an experimental confound/validity threat, not a case where the result is equivalent to the input by construction. The paper contains a few peripheral self-citations ([14], [27], [40]) in related-work lists, but none is load-bearing: the structured format, data pipeline, and evaluation do not rest on these citations. The use of GPT-4.1 both to construct the benchmark and as VLM judge is a potential evaluation-bias risk, but it is not circular in the sense of a prediction reducing to its input. Under the stated rules, no circular step can be exhibited with a specific reduction, so the appropriate finding is no significant circularity.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 0 invented entities

The paper introduces no new physical entities. The structured context itself is a data representation, not an entity. The listed free parameters are engineering thresholds that influence training data quality, and the axioms are the background assumptions about the base model, the extraction model, the judge, and the fine-tuning scheme. None of these are fitted to the target metric in a way that would make the result tautological.

free parameters (3)
  • Face quality threshold (Q-Align score) = unknown
    Sources images are filtered by Q-Align face quality scores; the threshold is hand-chosen and affects the composition of the training set (Sec 3.2).
  • Face similarity threshold for identity matching = unknown
    Post-filtering keeps samples only when matched face pairs have high similarity scores; the similarity threshold is hand-picked (Sec 3.2).
  • Mixture ratio of MICo and curated data = proportional to sample counts
    Training mixes MICo and the newly curated data proportionally to sample counts, which is a design choice affecting the model's behavior (Sec 4.1, Table 4).
axioms (5)
  • standard math BAGEL's flow-matching loss and transformer attention function as intended for the structured context inputs.
    The training objective and attention mechanism are inherited from the base model (Sec 3.3, Sec C).
  • domain assumption Qwen-Image-Edit extraction produces reference images that preserve source identity and attributes closely enough to train the model.
    The whole data pipeline depends on the generative extractor producing consistent references (Sec 3.2).
  • domain assumption GPT-4.1 VLM-as-a-judge provides valid prompt-following and subject-consistency scores.
    Evaluation of PF and SC relies on GPT-4.1 judgments, which may carry their own preferences (Sec 3.3).
  • domain assumption Identifiers like <human_1> can be learned as anchors by the LLM backbone with only attention and normalization layers updated.
    The fine-tuning scheme assumes that a small set of trainable layers suffices to understand the new structured format (Sec 3.3, Sec C).
  • ad hoc to paper The predefined outfit catalog and pose-perturbation list are sufficiently diverse to prevent copy-paste bias while keeping identity.
    These hand-designed choices are introduced specifically to make the extraction pipeline work (Sec 3.2 and supplementary).

pith-pipeline@v1.3.0-alltime-deepseek · 21505 in / 8870 out tokens · 90197 ms · 2026-08-01T22:43:33.318483+00:00 · methodology

0 comments
read the original abstract

Multi-reference image generation aims to synthesize images by integrating attributes from multiple reference images under textual instructions. As the number of references increases, the task necessitates complex semantic comprehension, such as correctly associating attributes with the intended subjects and planing out coherent spatial arrangement between subjects and their environments. Existing approaches, which rely solely on natural language instruction, often fail to capture these complex intentions precisely, leading to semantic misalignment and inconsistent generation. We identify two key factors behind these limitations: natural language instructions are often verbose and ambiguous, and high-quality multi-reference data is scarce. To address these issues, we propose StructGen, which employs a structured, dictionary-like format to encode multiple reference images, thereby enabling explicit and unambiguous specification of generation intentions. To support this design, we construct a structured dataset based on high-quality real images and develop a corresponding training framework, along with a dedicated benchmark for challenging multi-reference scenarios. Extensive experiments on both public benchmarks and our proposed benchmark demonstrate that StructGen consistently outperforms existing methods on both semantic alignment and detailed reference-generation consistency, especially under complex instructions with multiple references. The code is available at https://jianingpeng0382.github.io/StructGen/

Figures

Figures reproduced from arXiv: 2607.15619 by Henghui Ding, Jianing Peng, Luoqi Liu, Mengyu Wang, Ting Liu, Xiaochao Qu, Yao Zhao, Yunchao Wei, Zixiang Li.

Figure 1
Figure 1. Figure 1: Qualitative comparison on a challenging example with multiple references. Through structured context modeling, [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Compared with previous datasets, ours preserves [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: StructGen consists of Structured Tuning (top) and Structured Data Curation (bottom). The data curation pipeline [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Statistics on the number of reference images per [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Overview of StructGen Bench. The benchmark is organized along three factors: task type, subject number, and scene [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Qualitative comparison on our proposed benchmark. StructGen demonstrates advantages on the consistency of human [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: StructGen performs generation with all references provided as textual descriptions, demonstrating the potential of [PITH_FULL_IMAGE:figures/full_fig_p008_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: (Top) Qwen3-VL [1] produces structured captions (oc_id: Outfit-Conditioned identifier; oc_detail: detail description). (Bottom) VLM spatial matching prompt for multi-human images [PITH_FULL_IMAGE:figures/full_fig_p013_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Reference Extraction prompts for all entity categories: Outfit-Conditioned human references (head-and-shoulders, [PITH_FULL_IMAGE:figures/full_fig_p014_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: (Top) Outfit-Conditioned identifier-based instruction synthesis prompt. The LLM rewrites the full-image caption by [PITH_FULL_IMAGE:figures/full_fig_p015_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: (Top) Qwen3-VL Human Detail Description Prompt. Qwen3-VL [ [PITH_FULL_IMAGE:figures/full_fig_p016_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Training data sample 1: single-person example with human, belonging, and scene references. oc_id: Outfit-Conditioned [PITH_FULL_IMAGE:figures/full_fig_p017_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Training data sample 2: a second single-person example. Field names follow the same convention as in Fig. 12. [PITH_FULL_IMAGE:figures/full_fig_p018_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Training data sample 3: multi-person example with two human subjects and multiple belonging references. Field [PITH_FULL_IMAGE:figures/full_fig_p019_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Training data sample 4: a second multi-person example. Field names follow the same convention as in Fig. 12. [PITH_FULL_IMAGE:figures/full_fig_p020_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: (Top) Scene prototype generation prompt: GPT-4.1 [ [PITH_FULL_IMAGE:figures/full_fig_p021_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: A representative benchmark sample from the multi-person Outfit-Retaining with scene subcategory of StructGen [PITH_FULL_IMAGE:figures/full_fig_p022_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: A representative benchmark sample from the multi-person Outfit-Conditioned with scene subcategory of StructGen [PITH_FULL_IMAGE:figures/full_fig_p023_18.png] view at source ↗
Figure 19
Figure 19. Figure 19: Qualitative comparisons with Echo-4o [21] on Outfit-Conditioned cases, where the model must compose a target image from separate human identity and clothing references. StructGen achieves more accurate attribute–subject association and stronger consistency with all references [PITH_FULL_IMAGE:figures/full_fig_p024_19.png] view at source ↗
Figure 20
Figure 20. Figure 20: Qualitative comparisons with Echo-4o [21] on Outfit-Retaining cases, where the model must faithfully preserve the full appearance of referenced human subjects including their original outfits. StructGen better preserves both facial identity and full-body appearance consistency [PITH_FULL_IMAGE:figures/full_fig_p025_20.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

62 extracted references · 17 linked inside Pith

  1. [1]

    Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhaohai Li, Mingsheng Li, Mei Li, Kaixin Li, Zicheng Lin, Junyang Lin, Xuejing Liu, Jiawei Liu, Chenglong Liu, Yang Liu, Dayiheng Liu, Shixuan ...

    Pith/arXiv arXiv 2025

  2. [2]

    Tim Brooks, Aleksander Holynski, and Alexei A Efros. 2023. InstructPix2Pix: Learning to Follow Image Editing Instructions. In2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society, 18392–18402

    2023

  3. [3]

    Bowen Chen, Haomiao Sun, Li Chen, Xu Wang, Daniel Kang Du, Xinglong Wu, et al. 2025. XVerse: Consistent Multi-Subject Control of Identity and Semantic Attributes via DiT Modulation. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems

    2025

  4. [4]

    Jiuhai Chen, Zhiyang Xu, Xichen Pan, Yushi Hu, Can Qin, Tom Goldstein, Lifu Huang, Tianyi Zhou, Saining Xie, Silvio Savarese, et al. 2025. Blip3-o: A family of fully open unified multimodal models-architecture, training and dataset.arXiv preprint arXiv:2505.09568(2025)

    Pith/arXiv arXiv 2025

  5. [5]

    Ruoxi Chen, Dongping Chen, Siyuan Wu, Sinan Wang, Shiyun Lang, Peter Sushko, Gaoyang Jiang, Yao Wan, and Ranjay Krishna. 2025. MultiRef: Controllable Image Generation with Multiple Visual References. InProceedings of the 33rd ACM International Conference on Multimedia. 13325–13331

    2025

  6. [6]

    Xiaokang Chen, Zhiyu Wu, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, and Chong Ruan. 2025. Janus-pro: Unified multimodal understanding and generation with data and model scaling.arXiv preprint arXiv:2501.17811 (2025)

    Pith/arXiv arXiv 2025

  7. [7]

    Yufeng Cui, Honghao Chen, Haoge Deng, Xu Huang, Xinghang Li, Jirong Liu, Yang Liu, Zhuoyan Luo, Jinsheng Wang, Wenxuan Wang, et al. 2025. Emu3. 5: Native multimodal models are world learners.arXiv preprint arXiv:2510.26583 (2025)

    Pith/arXiv arXiv 2025

  8. [8]

    Chaorui Deng, Deyao Zhu, Kunchang Li, Chenhui Gou, Feng Li, Zeyu Wang, Shu Zhong, Weihao Yu, Xiaonan Nie, Ziang Song, et al. 2025. Emerging properties in unified multimodal pretraining.arXiv preprint arXiv:2505.14683(2025)

    Pith/arXiv arXiv 2025

  9. [9]

    Jiankang Deng, Jia Guo, Xue Niannan, and Stefanos Zafeiriou. 2019. ArcFace: Additive Angular Margin Loss for Deep Face Recognition. InCVPR

    2019

  10. [10]

    Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit Haim Bermano, Gal Chechik, and Daniel Cohen-Or. 2023. An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion. InThe Eleventh International Conference on Learning Representations

    2023

  11. [11]

    2025.Nano banana

    Google. 2025.Nano banana. Technical Report. Google. https://developers. googleblog.com/en/introducing-gemini-2-5-flash-image

    2025

  12. [12]

    Runze He, Bo Cheng, Yuhang Ma, Qingxiang Jia, Shanyuan Liu, Ao Ma, Xiaoyu Wu, Liebucha Wu, Dawei Leng, and Yuhui Yin. 2025. Plangen: Towards unified layout planning and image generation in auto-regressive vision language models. InProceedings of the IEEE/CVF International Conference on Computer Vision. 18143–18154

    2025

  13. [13]

    Runze He, Kai Ma, Linjiang Huang, Shaofei Huang, Jialin Gao, Xiaoming Wei, Jiao Dai, Jizhong Han, and Si Liu. 2025. Freeedit: Mask-free reference-based image editing with multi-modal instruction.IEEE Transactions on Pattern Analysis and Machine Intelligence(2025)

    2025

  14. [14]

    Yihan Hu, Jianing Peng, Yiheng Lin, Ting Liu, Xiaochao Qu, Luoqi Liu, Yao Zhao, and Yunchao Wei. 2025. Dcedit: Dual-level controlled image editing via precisely localized semantics.arXiv preprint arXiv:2503.16795(2025)

    Pith/arXiv arXiv 2025

  15. [15]

    Hang Hua, Ziyun Zeng, Yizhi Song, Yunlong Tang, Liu He, Daniel Aliaga, Wei Xiong, and Jiebo Luo. 2025. MMIG-Bench: Towards Comprehensive and Explainable Evaluation of Multi-Modal Image Generation Models. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track

    2025

  16. [16]

    Kaiyi Huang, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu. 2023. T2i- compbench: A comprehensive benchmark for open-world compositional text-to- image generation.Advances in Neural Information Processing Systems36 (2023), 78723–78747

    2023

  17. [17]

    Mude Hui, Siwei Yang, Bingchen Zhao, Yichun Shi, Heng Wang, Peng Wang, Cihang Xie, and Yuyin Zhou. 2025. HQ-Edit: A High-Quality Dataset for Instruction-based Image Editing. (2025)

    2025

  18. [18]

    Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. 2017. Image-to-Image Translation with Conditional Adversarial Networks. In2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society, 5967–5976

    2017

  19. [19]

    Houcheng Jiang, Junfeng Fang, Ningyu Zhang, Mingyang Wan, Guojun Ma, Xiang Wang, Xiangnan He, and Tat-Seng Chua. 2025. AnyEdit: Edit Any Knowledge Encoded in Language Models. InInternational Conference on Machine Learning. PMLR, 27510–27533

    2025

  20. [20]

    2023.Ultralytics YOLO

    Glenn Jocher, Jing Qiu, and Ayush Chaurasia. 2023.Ultralytics YOLO. https: //github.com/ultralytics/ultralytics

    2023

  21. [21]

    Zihao Wang Leqi Zhu Zhenghao Hu Zilong Huang Jun He Zhiyuan Yan Jinghua Yu Hongsheng Li Conghui He Weijia Li Junyan Ye, Dongzhi Jiang. 2025. Echo- 4o: Harnessing the Power of GPT-4o Synthetic Images for Improved Image Generation.https://arxiv.org/abs/2508.09987(2025)

    Pith/arXiv arXiv 2025

  22. [22]

    Max Ku, Dongfu Jiang, Cong Wei, Xiang Yue, and Wenhu Chen. 2024. Viescore: Towards explainable metrics for conditional image synthesis evaluation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 12268–12290

    2024

  23. [23]

    Harold W Kuhn. 1955. The Hungarian method for the assignment problem. Naval research logistics quarterly2, 1-2 (1955), 83–97

    1955

  24. [24]

    Nupur Kumari, Bingliang Zhang, Richard Zhang, Eli Shechtman, and Jun-Yan Zhu. 2023. Multi-Concept Customization of Text-to-Image Diffusion. In2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 1931–1941

    2023

  25. [25]

    Baiqi Li, Zhiqiu Lin, Deepak Pathak, Jiayao Li, Yixin Fei, Kewen Wu, Tiffany Ling, Xide Xia, Pengchuan Zhang, Graham Neubig, et al. 2024. GenAI-Bench: Evaluating and Improving Compositional Text-to-Visual Generation.arXiv e-prints(2024), arXiv–2406

    2024

  26. [26]

    Dongxu Li, Junnan Li, and Steven Hoi. 2023. Blip-diffusion: Pre-trained subject representation for controllable text-to-image generation and editing.Advances in Neural Information Processing Systems36 (2023), 30146–30166

    2023

  27. [27]

    Zixiang Li, Haoyu Wang, Wei Wang, Chuangchuang Tan, Yunchao Wei, and Yao Zhao. 2025. DCI: Dual-Conditional Inversion for Boosting Diffusion-Based Image Editing.arXiv preprint arXiv:2506.02560(2025)

    Pith/arXiv arXiv 2025

  28. [28]

    Zhong-Yu Li, Ruoyi Du, Juncheng Yan, Le Zhuo, Zhen Li, Peng Gao, Zhanyu Ma, and Ming-Ming Cheng. 2025. Visualcloze: A universal image generation framework via visual in-context learning. InProceedings of the IEEE/CVF International Conference on Computer Vision. 18969–18979

    2025

  29. [29]

    Yiyang Ma, Xingchao Liu, Xiaokang Chen, Wen Liu, Chengyue Wu, Zhiyu Wu, Zizheng Pan, Zhenda Xie, Haowei Zhang, Xingkai Yu, et al. 2025. JanusFlow: Harmonizing Autoregression and Rectified Flow for Unified Multimodal Understanding and Generation. In2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 7739–7751

    2025

  30. [30]

    James Munkres. 1957. Algorithms for the assignment and transportation problems.Journal of the society for industrial and applied mathematics5, 1 (1957), 32–38

    1957

  31. [31]

    OpenAI. 2025. GPT-4-1. https://openai.com/index/gpt-4-1

    2025

  32. [32]

    OpenAI. 2025. GPT-4o. https://openai.com/index/introducing-4o-image- generation

    2025

  33. [33]

    Xichen Pan, Satya Narayan Shukla, Aashu Singh, Zhuokai Zhao, Shlok Kumar Mishra, Jialiang Wang, Zhiyang Xu, Jiuhai Chen, Kunpeng Li, Felix Juefei-Xu, Ji Hou, and Saining Xie. 2025. Transfer between modalities with metaqueries. arXiv preprint arXiv:2504.06256(2025). Jianing Peng, et al

    Pith/arXiv arXiv 2025

  34. [34]

    William Peebles and Saining Xie. 2023. Scalable Diffusion Models with Transformers. In2023 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 4172–4182

    2023

  35. [35]

    Yuang Peng, Yuxin Cui, Haomiao Tang, Zekun Qi, Runpei Dong, Jing Bai, Chunrui Han, Zheng Ge, Xiangyu Zhang, and Shu-Tao Xia. 2025. DreamBench++: A Human-Aligned Benchmark for Personalized Image Generation. InThe Thirteenth International Conference on Learning Representations

    2025

  36. [36]

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. 2023. DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation. In2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 22500–22510

    2023

  37. [37]

    Zhenxiong Tan, Songhua Liu, Xingyi Yang, Qiaochu Xue, and Xinchao Wang

  38. [38]

    Chameleon Team. 2024. Chameleon: Mixed-modal early-fusion foundation models.arXiv preprint arXiv:2405.09818(2024)

    Pith/arXiv arXiv 2024

  39. [39]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need.Advances in neural information processing systems30 (2017)

    2017

  40. [40]

    Mengyu Wang, Yuyao Huang, Henghui Ding, Xinlong Wang, Tiejun Huang, Yao Zhao, Yunchao Wei, and Shuicheng Yan. 2024. Region-native visual tokenization. InEuropean Conference on Computer Vision. Springer, 19–36

    2024

  41. [41]

    Xierui Wang, Siming Fu, Qihan Huang, Wanggui He, and Hao Jiang. 2025. MS- Diffusion: Multi-subject Zero-shot Image Personalization with Layout Guidance. InThe Thirteenth International Conference on Learning Representations

    2025

  42. [42]

    Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, et al. 2024. Emu3: Next-token prediction is all you need.arXiv preprint arXiv:2409.18869(2024)

    Pith/arXiv arXiv 2024

  43. [43]

    Xinyu Wei, Kangrui Cen, Hongyang Wei, Zhen Guo, Bairui Li, Zeqing Wang, Jinrui Zhang, and Lei Zhang. 2025. MICo-150K: A Comprehensive Dataset Advancing Multi-Image Composition.arXiv preprint arXiv:2512.07348(2025)

    Pith/arXiv arXiv 2025

  44. [44]

    Chengyue Wu, Xiaokang Chen, Zhiyu Wu, Yiyang Ma, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, Chong Ruan, et al . 2025. Janus: Decoupling visual encoding for unified multimodal understanding and generation. InProceedings of the Computer Vision and Pattern Recognition Conference. 12966–12977

    2025

  45. [45]

    Chenfei Wu, Jiahao Li, Jingren Zhou, Junyang Lin, Kaiyuan Gao, Kun Yan, Sheng ming Yin, Shuai Bai, Xiao Xu, Yilei Chen, Yuxiang Chen, Zecheng Tang, Zekai Zhang, Zhengyi Wang, An Yang, Bowen Yu, Chen Cheng, Dayiheng Liu, Deqing Li, Hang Zhang, Hao Meng, Hu Wei, Jingyuan Ni, Kai Chen, Kuan Cao, Liang Peng, Lin Qu, Minggang Wu, Peng Wang, Shuting Yu, Tingkun...

    Pith/arXiv arXiv 2025

  46. [46]

    Chenyuan Wu, Pengfei Zheng, Ruiran Yan, Shitao Xiao, Xin Luo, Yueze Wang, Wanli Li, Xiyan Jiang, Yexin Liu, Junjie Zhou, Ze Liu, Ziyi Xia, Chaofan Li, Haoge Deng, Jiahao Wang, Kun Luo, Bo Zhang, Defu Lian, Xinlong Wang, Zhongyuan Wang, Tiejun Huang, and Zheng Liu. 2025. OmniGen2: Exploration to Advanced Multimodal Generation.arXiv preprint arXiv:2506.18871(2025)

    Pith/arXiv arXiv 2025

  47. [47]

    Haoning Wu, Zicheng Zhang, Weixia Zhang, Chaofeng Chen, Liang Liao, Chunyi Li, Yixuan Gao, Annan Wang, Erli Zhang, Wenxiu Sun, et al . 2024. Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levels. In International Conference on Machine Learning. PMLR, 54015–54029

    2024

  48. [48]

    Mingrui Wu, Hang Liu, Jiayi Ji, Xiaoshuai Sun, and Rongrong Ji. 2026. MICON- Bench: Benchmarking and Enhancing Multi-Image Context Image Generation in Unified Multimodal Models.arXiv preprint arXiv:2602.19497(2026)

    arXiv 2026

  49. [49]

    Shaojin Wu, Mengqi Huang, Wenxu Wu, Yufeng Cheng, Fei Ding, and Qian He

  50. [50]

    Bin Xia, Bohao Peng, Yuechen Zhang, Junjia Huang, Jiyang Liu, Jingyao Li, Haoru Tan, Sitong Wu, Chengyao Wang, Yitong Wang, et al . 2025. Dreamomni2: Multimodal instruction-based editing and generation.arXiv preprint arXiv:2510.06679(2025)

    arXiv 2025

  51. [51]

    InProceedings of the IEEE/CVF International Conference on Computer Vision

    Less-to-more generalization: Unlocking more controllability by in-context generation. InProceedings of the IEEE/CVF International Conference on Computer Vision. 18682–18692

  52. [52]

    Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. 2025. Show-o: One Single Transformer to Unify Multimodal Understanding and Generation. InThe Thirteenth International Conference on Learning Representations

    2025

  53. [53]

    Shitao Xiao, Yueze Wang, Junjie Zhou, Huaying Yuan, Xingrun Xing, Ruiran Yan, Chaofan Li, Shuting Wang, Tiejun Huang, and Zheng Liu. 2025. OmniGen: Unified Image Generation. In2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 13294–13304

    2025

  54. [54]

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, ...

    Pith/arXiv arXiv 2025

  55. [55]

    Jinheng Xie, Zhenheng Yang, and Mike Zheng Shou. 2025. Show-o2: Improved native unified multimodal models.arXiv preprint arXiv:2506.15564(2025)

    Pith/arXiv arXiv 2025

  56. [56]

    Kai Zhang, Lingbo Mo, Wenhu Chen, Huan Sun, and Yu Su. 2023. Magicbrush: A manually annotated dataset for instruction-guided image editing.Advances in Neural Information Processing Systems36 (2023), 31428–31449

    2023

  57. [57]

    Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. 2023. Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models.arXiv preprint arXiv:2308.06721(2023)

    Pith/arXiv arXiv 2023

  58. [58]

    Chunting Zhou, LILI YU, Arun Babu, Kushal Tirumala, Michihiro Yasunaga, Leonid Shamis, Jacob Kahn, Xuezhe Ma, Luke Zettlemoyer, and Omer Levy. 2025. Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Model. InThe Thirteenth International Conference on Learning Representations

    2025

  59. [59]

    Haozhe Zhao, Xiaojian Ma, Liang Chen, Shuzheng Si, Rujie Wu, Kaikai An, Peiyu Yu, Minjia Zhang, Qing Li, and Baobao Chang. 2024. Ultraedit: Instruction-based fine-grained image editing at scale.Advances in Neural Information Processing Systems37, 3058–3093

    2024

  60. [61]

    Yupeng Zhou, Daquan Zhou, Ming-Ming Cheng, Jiashi Feng, and Qibin Hou

  61. [2024]

    source_description

    Storydiffusion: Consistent self-attention for long-range image and video generation.Advances in Neural Information Processing Systems37 (2024), 110315– 110340. StructGen: Disambiguating Multi-Reference Image Generation via Structured Context Modeling StructGen: Disambiguating Multi-Reference Image Generation via Structured Context Modeling Supplementary M...

    2024

  62. [2025]

    InProceedings of the IEEE/CVF International Conference on Computer Vision

    Ominicontrol: Minimal and universal control for diffusion transformer. InProceedings of the IEEE/CVF International Conference on Computer Vision. 14940–14950