Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

FocusDiff: Advancing Fine-Grained Text-Image Alignment for Autoregressive Visual Generation through RL

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read PairComp reveals and fixes a fine-grained alignment failure in autoregressive text-to-image models.

desk verdict FocusDiff is a genuinely novel paired-differential RL method with a useful benchmark and clean ablations, but its flagship PairComp result is confounded by using the same VLM as captioner, reward, and judge. read the letter →

arxiv 2506.05501 v1 pith:CNPKZWJA submitted 2025-06-05 cs.CV

Kaihang Pan , Wendong Bu , Yuruo Wu , Yang Wu , Kai Shen , Yunfei Li , Hang Zhao , Juncheng Li
show 2 more authors
Siliang Tang Yueting Zhuang
This is my paper
classification cs.CV
keywords fine-grainedtext-imagealignmentautoregressiveimagegenerationreinforcementlearningGroupRelativePolicyOptimizationpairedpromptsbenchmarkeditingdatacounterfactualgeometricmeanevaluation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that autoregressive text-to-image models, despite strong overall alignment, fail at fine-grained control: they cannot reliably turn word-level differences in a prompt into corresponding visual differences. To expose this, it introduces PairComp, whose test cases are pairs of prompts with similar syntax but different local semantics, scored by both arithmetic and geometric means of text-image consistency. The paper then proposes FocusDiff, a training scheme built from FocusDiff-Data, paired before-and-after edit images with captions differing only in a few words, and Pair-GRPO, a reinforcement-learning algorithm that compares generations across such prompt pairs. With Janus-Pro-7B as backbone, Janus-FocusDiff-7B raises PairComp average from 75.5/70.4 to 85.0/83.5 and improves GenEval, T2I-CompBench, and DPG-Bench. The intended upshot is that an autoregressive model can learn precise visual-token control without abandoning the next-token-prediction paradigm.

What carries the argument

The load-bearing objects are FocusDiff-Data, roughly 200,000 paired text-image tuples mined from image-editing datasets via caption generation and cross-verification, and Pair-GRPO, an extension of GRPO in which the group for relative advantage contains images generated from both prompts of a similar pair plus, with decaying probability, the ground-truth images paired with both prompts. The metric that carries the diagnosis is the geometric mean consistency score sg, which punishes the low-scoring prompt of each pair and thereby quantifies generation instability.

What would settle it

Evaluate Janus-FocusDiff-7B on PairComp using human raters or an independent judge such as a different MLLM; if the pairwise advantage over Janus-Pro-7B shrinks or disappears, the measured gains reflect optimization of the judge rather than genuine fine-grained alignment.

Watch

Extended reading notes

Core claim

FocusDiff's central claim is that fine-grained text-image alignment can be learned from paired examples: by training on two prompts whose captions share structure but differ in one to three words, together with the two corresponding images, the model can trace which visual tokens change when text tokens change. The Pair-GRPO algorithm operationalizes this by placing generations from both prompts in one group for advantage estimation, and by temporarily injecting the ground-truth images from FocusDiff-Data as both positive guides and cautionary counterexamples under a curriculum that shifts from exploitation to exploration. The paper reports that this yields stable, precise generation: Janus-FocusDiff-7B attains 85.0/83.5 arithmetic/geometric mean on PairComp versus 75.5/70.4 for Janus-Pro-7B, with the arithmetic-geometric gap shrinking from 5.1 to 1.5 points, and improves color-attribute T2I-CompBench from 63.6 to 83.0.

Load-bearing premise

The claim depends on InternVL2.5-26B giving faithful and unbiased yes/no judgments of fine-grained text-image consistency, because the same model generates the training captions, supplies the reinforcement-learning reward, and scores PairComp.

Editorial extensions

If this is right

  • If the central claim is right, autoregressive text-to-image models can reach fine-grained control that current diffusion baselines match or exceed, without changing the architecture or the next-token-prediction training paradigm.
  • Counterfactual prompts such as "square watermelon" become generable, indicating reduced hallucination from training-distribution bias.
  • The method scales to smaller backbones: Janus-FocusDiff-1B outperforms Janus-Pro-7B on GenEval and T2I-CompBench, which suggests paired differential data is an efficient training signal.
  • PairComp's geometric-mean metric gives future work a sensitive measure of fine-grained alignment, since it collapses when either prompt in a pair is generated poorly.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same paired-prompt recipe could be applied to diffusion backbones, replacing the discrete-token group formulation with a denoising-level advantage estimate; the paper only demonstrates the idea in autoregressive models.
  • Because InternVL2.5-26B serves as both reward model and judge, the reported gains partly measure how well the generator optimizes that particular judge; an independent human study or a different judge would clarify whether alignment itself improved.
  • A natural testable extension is to vary the semantic distance between paired prompts, predicting that intermediate distances best transfer to unseen fine-grained attributes.
  • The geometric-mean metric could be ported to other benchmarks by grouping prompts into minimal pairs, making instability visible where arithmetic means hide it.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper addresses fine-grained text-image alignment for autoregressive visual generation. It introduces PairComp, a benchmark of paired prompts that differ in subtle semantics; FocusDiff-Data, a dataset of paired text-image pairs derived from image-editing datasets; and Pair-GRPO, a reinforcement learning algorithm that extends GRPO by grouping responses across similar prompt pairs and by injecting ground-truth images with a decaying probability. Using Janus-Pro as the backbone, the authors report large gains on PairComp (e.g., Janus-FocusDiff-7B reaches average sa=85.0/sg=83.5 vs. 75.5/70.4 for Janus-Pro-7B) and improvements on GenEval, T2I-CompBench, and DPG-Bench.

Significance. If the reported results hold, the paper makes a useful contribution: the PairComp benchmark diagnoses a real weakness in existing text-to-image models, and the paired-data plus group-expanding RL recipe appears to improve fine-grained control. The method is evaluated on standard external benchmarks (GenEval, T2I-CompBench, DPG-Bench) with gains beyond the backbone, which partially supports the central claim. However, the headline PairComp numbers are potentially confounded because the same InternVL2.5-26B model is used for caption generation, reward modeling, and PairComp evaluation; without an independent judge or human evaluation, the benchmark claim is not yet established. The paper would be strengthened by reporting per-subtask sample sizes and uncertainty estimates, which are currently absent.

major comments (3)
  1. [Section 2; Section 3.2; Appendix D] The PairComp evaluation protocol (Section 2 and Appendix D) uses InternVL2.5-26B as the judge, and the Pair-GRPO reward model (Section 3.2 and Appendix C) is the same InternVL2.5-26B. FocusDiff-Data captions are also generated by InternVL2.5-26B (Section 3.1 and Appendix B). Because the RL objective is to maximize the probability that this exact judge answers 'yes' to the same question used in PairComp scoring, the reported PairComp gains may reflect overfitting to the judge's biases (e.g., color or layout priors) rather than genuine fine-grained alignment. Please provide an independent evaluation of PairComp using a different VLM judge (e.g., GPT-4o) or human raters, and report per-case agreement or correlation between the InternVL and independent judgments.
  2. [Section 2; Figure 2] PairComp statistics are not fully reported: the per-subtask number of test cases N is not given, and no variance, standard deviation, or confidence interval is provided for any PairComp score. With unknown N and no error bars, the significance of small differences (e.g., average sa 85.0 for Janus-FocusDiff-7B vs. 84.4 for SD3) cannot be assessed. Please report N per subtask and bootstrap confidence intervals or standard deviations for the main comparisons.
  3. [Section 4.4; Table 3] The ablation row 'w/o FocusDiff-Data' is not sufficiently specified. The text says 'we further generate a set of prompts commonly used in text-to-image to replace FocusDiff-Data for RL training with Vanilla GRPO,' but it is unclear whether these prompts are paired, what their source is, and how they match the distribution of FocusDiff-Data. This control is load-bearing for the claim that the paired structure of FocusDiff-Data is responsible for the gains; please describe the prompt generation procedure and, ideally, use a matched control that differs only in the pairing structure.
minor comments (5)
  1. [Throughout] The benchmark name is inconsistently capitalized: 'Paircomp' appears in Figure 2, Appendix A, and Section 2, while the rest of the paper uses 'PairComp'.
  2. [Appendix D] In the evaluation-protocol paragraph, the four generated images are listed as (I^{1,1}_i, T^{1,2}_i, I^{2,1}_i, I^{2,2}_i); the second entry should be I^{1,2}_i, not T^{1,2}_i.
  3. [Section 4.2] The relative improvement over Janus-Pro-7B on T2I-CompBench is reported as '45.6723%'; this appears to be a typo, likely '45.67%', and the specific subtask average used for this percentage should be stated.
  4. [Table 3] The row label 'w/oFocusDiff-Data' is missing a space and should read 'w/o FocusDiff-Data'.
  5. [Section 3.1] In the sentence 'as well as I 1 and I 1, are similar in overall expression', the second 'I 1' should be 'I 2'.

Circularity Check

1 steps flagged · score 6.0 of 10

Partial circularity: the headline PairComp benchmark is scored by the same InternVL2.5-26B model that supplies the Pair-GRPO reward and the training captions, so the reported PairComp gains partly measure optimization of the judge itself.

  1. fitted input called prediction [Section 2 (Evaluation Protocols), Section 3.1 (Data Perspective), Section 3.2 (QA-based Reward), Appendix C and D]
    "We use InternVL2.5-26B (Chen et al., 2024b) as the evaluation model to assess the semantic consistency between the generated images and the text prompts. ... the semantic consistency score calculated as S(I,T) = P_yes/(P_yes+P_no). ... we can employ a powerful visual comprehension model like InternVL2.5-26B (Chen et al., 2024b) to generate style-similar captions for the images. ..."

    The same InternVL2.5-26B model plays three roles: it generates and verifies the FocusDiff-Data captions (Sec 3.1), it is the QA reward model in Pair-GRPO (Sec 3.2, App C), and it is the official PairComp judge (Sec 2, App D). For short prompts, the PairComp score S(I,T)=P_yes/(P_yes+P_no) is exactly the reward score used in RL; for long prompts, the reward averages VQA scores from the same MLLM. Thus the headline PairComp improvement (85.0/83.5 vs. Janus-Pro-7B's 75.5/70.4) is measured by the very function the policy was trained to maximize. This is a fitted-input-called-prediction pattern: the claimed evidence for fine-grained alignment is not an independent test but reflects optimization against the benchmark judge.

full rationale

The paper's central PairComp claim is substantially confounded because the evaluation judge, the RL reward model, and the training-data caption generator are all InternVL2.5-26B. Optimizing a policy against a scorer and then reporting gains on that same scorer does not establish that the model genuinely distinguishes fine-grained semantic differences; it may exploit systematic VLM biases. I flag this as one concrete circular step with quoted evidence. However, the paper is not wholly circular: GenEval, T2I-CompBench, and DPG-Bench use independent metrics and show real improvements (e.g., GenEval overall 0.85 vs. 0.80 for Janus-Pro-7B; T2I-CompBench color-attribute 83.0 vs. 63.6). Those results provide external, non-circular support for the method's general value. The paper also cites prior work by the same authors (Janus-Pro-R1, T2I-R1) as baselines rather than as load-bearing justification, so no additional circularity score is warranted from self-citation. Overall score 6: one major evaluation-plus-reward loop partially compromises the headline benchmark, while external benchmarks retain independent content.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim does not posit new physical entities. It relies on several domain assumptions about data quality and evaluator validity, plus standard GRPO machinery and a set of hand-chosen hyperparameters. The most consequential assumption is that InternVL-based scoring reflects human alignment.

free parameters (4)
  • Group size G per prompt = Seven, expanded to fourteen when paired prompts are grouped, and to eighteen when ground-truth images are added.
    Hand-chosen in Table 4 reinforcement-learning setup; defines group-relative advantage in Pair-GRPO and is central to the method.
  • Ground-truth insertion probability p = Starts at 1.0 and decreases to 0.0 over 2200 training steps.
    Curriculum schedule that mixes labeled images into the advantage group; the transition from exploitation to exploration is a core design choice.
  • Learning rate schedule = Peak 1e-5, convert 2e-6 at step 300, minimum 2e-7.
    Authors report sensitivity to learning rate and use a custom linear-plus-cosine schedule; not derived from theory.
  • Inference guidance scale and top-k = Guidance scale 5.0 or 6.0, top-k 4096.
    Sampling choices for autoregressive visual tokens; affect output quality and alignment.
assumptions (4)
  • domain assumption InternVL2.5-26B judgments are a valid proxy for human fine-grained semantic consistency.
    The model generates captions, filters data, supplies QA rewards, and scores PairComp; no human agreement study is reported. Sections 3.1, 3.2, and Appendix D.
  • domain assumption Image editing datasets (AnyEdit, UltraEdit) provide high-quality before-and-after pairs that cover diverse attribute changes.
    FocusDiff-Data is derived from these sources after automatic filtering; biases in editing tasks transfer to the training pairs. Section 3.1 and Appendix B.
  • domain assumption Janus-Pro's autoregressive visual tokenization lets paired images and captions differ in only a few tokens.
    The differential-learning argument depends on sparse token differences between T1 and T2, and between I1 and I2; only Janus-Pro is tested. Section 3.1 and Figure 1c.
  • standard math GRPO advantage estimation and KL constraint remain valid for token-level image generation.
    Adopts the GRPO framework from Shao et al. 2024 with group-relative advantages; this is standard RL but applied to visual tokens.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FocusDiff: Advancing Fine-Grained Text-Image Alignment for Autoregressive Visual Generation through RL." pith.science (2026). https://pith.science/paper/CNPKZWJA

@misc{pith2026250605501,
  author       = {Pith},
  title        = {Pith review of: FocusDiff: Advancing Fine-Grained Text-Image Alignment for Autoregressive Visual Generation through RL},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CNPKZWJA}},
  note         = {Machine review of arXiv:2506.05501}
}
read the original abstract

Recent studies extend the autoregression paradigm to text-to-image generation, achieving performance comparable to diffusion models. However, our new PairComp benchmark -- featuring test cases of paired prompts with similar syntax but different fine-grained semantics -- reveals that existing models struggle with fine-grained text-image alignment thus failing to realize precise control over visual tokens. To address this, we propose FocusDiff, which enhances fine-grained text-image semantic alignment by focusing on subtle differences between similar text-image pairs. We construct a new dataset of paired texts and images with similar overall expressions but distinct local semantics, further introducing a novel reinforcement learning algorithm to emphasize such fine-grained semantic differences for desired image generation. Our approach achieves state-of-the-art performance on existing text-to-image benchmarks and significantly outperforms prior methods on PairComp.

Figures

Figures reproduced from arXiv: 2506.05501 by the authors.

Figure 1
Figure 1. (a) For Janus-Pro-7B, the geometric mean score in PairComp is significantly lower than the arithmetic [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Statistical information of PairComp and test case examples for each subtask. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. The framework of our Pair-GRPO. fine-grained subtle differences. See more details in Appendix B. 3.2 Training Perspective: Pair-GRPO With FocusDiff-Data, we first conduct a super￾vised text-to-image fine-tuning. Subsequently, we treat image generation as a Markovian decision process at the token level and perform reinforce￾ment learning based on an improved version of GRPO(Shao et al., 2024) ( [PITH_FULL_IMAGE:figu… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Qualitative Comparisons between Janus-Pro-7B and our Janus-FocusDiff on pairs of similar prompts. For [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: More qualitative Comparisons between Janus-Pro-7B and Janus-FocusDiff on pairs of similar prompts. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Examples of training data in FocusDiff-Data. A photo of a fish that can fly A photo of a square watermel on Janus-Pro-7B A photo of a volcano that erupts cherry blossoms A photo of a purple carrot and a yellow grape Ours Ours Janus-Pro-7B [PITH_FULL_IMAGE:figures/full…
Figure 7
Figure 7. Figure 7: Counterfactual image generation. itively demonstrate the dataset’s advantages. It is evident that the images and their corresponding prompts exhibit only region-level or word-level dif￾ferences. This design enables models to focus on learning fine-grained semantic alig…
Figure 8
Figure 8. Figure 8: The pipeline for constructing FocusDiff-Data P2 exhibit similar semantic structures; (2) Using the InternVL-8B model (Chen et al., 2024b), we verify whether P1 and I1, as well as P2 and I2, were semantically aligned. (3) We further leverage InternVL-8B to ensure that P…

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Towards Meta-Cognitive Knowledge Editing for Multimodal LLMs

    cs.AI 2025-09 conditional novelty 6.0 of 10

    CogEdit and MIND shift multimodal knowledge editing toward evaluating and enabling meta-cognitive skills: self-awareness, boundary monitoring, and noise robustness.

Reference graph

Works this paper leans on

50 extracted references · 8 canonical work pages · cited by 1 Pith paper

  1. [1]

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, and 29 others. 2023. Qwen technical report. arXiv preprint arXiv:2309.16609

    arXiv 2023

  2. [2]

    Yoshua Bengio, J \'e r \^o me Louradour, Ronan Collobert, and Jason Weston. 2009. Curriculum learning. In Proceedings of the 26th annual international conference on machine learning, pages 41--48

    2009

  3. [3]

    James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, and 1 others. 2023. Improving image generation with better captions. Computer Science. https://cdn. openai. com/papers/dall-e-3. pdf, 2(3):8

    2023

  4. [4]

    Dong Chen, Kaihang Pan, Guangyu Dai, Guoming Wang, Yueting Zhuang, Siliang Tang, and Mingliang Xu. 2024 a . Improving vision anomaly detection with the guidance of language modality. IEEE Transactions on Multimedia

    work page 2024

  5. [5]

    Jiuhai Chen, Zhiyang Xu, Xichen Pan, Yushi Hu, Can Qin, Tom Goldstein, Lifu Huang, Tianyi Zhou, Saining Xie, Silvio Savarese, and 1 others. 2025 a . Blip3-o: A family of fully open unified multimodal models-architecture, training and dataset. arXiv preprint arXiv:2505.09568

    arXiv 2025

  6. [6]

    Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, and Zhenguo Li. 2023. https://arxiv.org/abs/2310.00426 Pixart- : Fast training of diffusion transformer for photorealistic text-to-image synthesis . Preprint, arXiv:2310.00426

    arXiv 2023

  7. [7]

    Xiaokang Chen, Zhiyu Wu, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, and Chong Ruan. 2025 b . Janus-pro: Unified multimodal understanding and generation with data and model scaling. arXiv preprint arXiv:2501.17811

    arXiv 2025

  8. [8]

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, and 1 others. 2024 b . Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24185--24198

    2024

Show all 50 references
  1. [9]

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M \"u ller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, and 1 others. 2024. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first international confe...

    2024

  2. [10]

    Patrick Esser, Robin Rombach, and Bjorn Ommer. 2021. Taming transformers for high-resolution image synthesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12873--12883

    2021

  3. [11]

    Yuying Ge, Sijie Zhao, Ziyun Zeng, Yixiao Ge, Chen Li, Xintao Wang, and Ying Shan. 2023. Making llama see and draw with seed tokenizer. arXiv preprint arXiv:2310.01218

    2023 arXiv

  4. [12]

    Yuying Ge, Sijie Zhao, Jinguo Zhu, Yixiao Ge, Kun Yi, Lin Song, Chen Li, Xiaohan Ding, and Ying Shan. 2024. Seed-x: Multimodal models with unified multi-granularity comprehension and generation. arXiv preprint arXiv:2404.14396

    2024 arXiv

  5. [13]

    Dhruba Ghosh, Hannaneh Hajishirzi, and Ludwig Schmidt. 2023. Geneval: An object-focused framework for evaluating text-to-image alignment. Advances in Neural Information Processing Systems, 36:52132--52152

    2023

  6. [14]

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025 a . Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948

    2025 arXiv

  7. [15]

    Ziyu Guo, Renrui Zhang, Chengzhuo Tong, Zhizheng Zhao, Peng Gao, Hongsheng Li, and Pheng-Ann Heng. 2025 b . https://arxiv.org/abs/2501.13926 Can we generate images with cot? let's verify and reinforce image generation step by step . Preprint, arXiv:2501.13926

    2025 arXiv

  8. [16]

    Jian Han, Jinlai Liu, Yi Jiang, Bin Yan, Yuqi Zhang, Zehuan Yuan, Bingyue Peng, and Xiaobing Liu. 2024. Infinity: Scaling bitwise autoregressive modeling for high-resolution image synthesis. arXiv preprint arXiv:2412.04431

    2024 arXiv

  9. [17]

    Xiwei Hu, Rui Wang, Yixiao Fang, Bin Fu, Pei Cheng, and Gang Yu. 2024. Ella: Equip diffusion models with llm for enhanced semantic alignment. arXiv preprint arXiv:2403.05135

    2024 arXiv

  10. [18]

    Kaiyi Huang, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu. 2023. T2i-compbench: A comprehensive benchmark for open-world compositional text-to-image generation. Advances in Neural Information Processing Systems, 36:78723--78747

    2023

  11. [19]

    Dongzhi Jiang, Ziyu Guo, Renrui Zhang, Zhuofan Zong, Hao Li, Le Zhuo, Shilin Yan, Pheng-Ann Heng, and Hongsheng Li. 2025. T2i-r1: Reinforcing image generation with collaborative semantic-level and token-level cot. arXiv preprint arXiv:2505.00703

    2025 arXiv

  12. [20]

    Black Forest Labs. 2024. Flux. https://github.com/black-forest-labs/flux

    2024

  13. [21]

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, and 1 others. 2024. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326

    2024 arXiv

  14. [22]

    Juncheng Li, Kaihang Pan, Zhiqi Ge, Minghe Gao, Wei Ji, Wenqiao Zhang, Tat-Seng Chua, Siliang Tang, Hanwang Zhang, and Yueting Zhuang. 2023. Fine-tuning multimodal llms to follow zero-shot demonstrative instructions. arXiv preprint arXiv:2308.04152

    2023 arXiv

  15. [23]

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. 2022. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In International conference on machine learning, pages 12888--12900. PMLR

    2022

  16. [24]

    Wang Lin, Liyu Jia, Wentao Hu, Kaihang Pan, Zhongqi Yue, Wei Zhao, Jingyuan Chen, Fei Wu, and Hanwang Zhang. 2025. Reasoning physical video generation with diffusion timestep tokens via reinforcement learning. arXiv preprint arXiv:2504.15932

    2025 arXiv

  17. [25]

    Hao Liu, Wilson Yan, Matei Zaharia, and Pieter Abbeel. 2024 a . World model on million-length video and language with blockwise ringattention. arXiv preprint arXiv:2402.08268

    2024 arXiv

  18. [26]

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2024 b . Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26296--26306

    2024

  19. [27]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual instruction tuning. Advances in neural information processing systems, 36:34892--34916

    2023

  20. [28]

    Yiyang Ma, Xingchao Liu, Xiaokang Chen, Wen Liu, Chengyue Wu, Zhiyu Wu, Zizheng Pan, Zhenda Xie, Haowei Zhang, Xingkai yu, Liang Zhao, Yisong Wang, Jiaying Liu, and Chong Ruan. 2024. Janusflow: Harmonizing autoregression and rectified flow for unified multimodal understanding ...

    2024

  21. [29]

    OpenAI. 2023. Chatgpt. https://chat.openai.com

    2023

  22. [30]

    OpenAI. 2024. Introducing 4o image generation. https://openai.com/index/introducing-4o-image-generation/

    2024

  23. [31]

    Kaihang Pan, Zhaoyu Fan, Juncheng Li, Qifan Yu, Hao Fei, Siliang Tang, Richang Hong, Hanwang Zhang, and Qianru Sun. 2024 a . Towards unified multimodal editing with enhanced knowledge collaboration. Advances in Neural Information Processing Systems, 37:110290--110314

    2024

  24. [32]

    Kaihang Pan, Juncheng Li, Hongye Song, Jun Lin, Xiaozhong Liu, and Siliang Tang. 2023. Self-supervised meta-prompt learning with meta-gradient regularization for few-shot generalization. arXiv preprint arXiv:2303.12314

    2023 arXiv

  25. [33]

    Kaihang Pan, Wang Lin, Zhongqi Yue, Tenglong Ao, Liyu Jia, Wei Zhao, Juncheng Li, Siliang Tang, and Hanwang Zhang. 2025 a . Generative multimodal pretraining with discrete diffusion timestep tokens. arXiv preprint arXiv:2504.14666

    2025 arXiv

  26. [34]

    Kaihang Pan, Siliang Tang, Juncheng Li, Zhaoyu Fan, Wei Chow, Shuicheng Yan, Tat-Seng Chua, Yueting Zhuang, and Hanwang Zhang. 2024 b . Auto-encoding morph-tokens for multimodal llm. arXiv preprint arXiv:2405.01926

    2024 arXiv

  27. [35]

    Kaihang Pan, Yang Wu, Wendong Bu, Kai Shen, Juncheng Li, Yingting Wang, Yunfei Li, Siliang Tang, Jun Xiao, Fei Wu, Hang Zhao, and Yueting Zhuang. 2025 b . Unlocking aha moments via reinforcement learning: Advancing collaborative visual comprehension and generation. arXiv prepr...

    2025

  28. [36]

    Haiyi Qiu, Minghe Gao, Long Qian, Kaihang Pan, Qifan Yu, Juncheng Li, Wenjie Wang, Siliang Tang, Yueting Zhuang, and Tat-Seng Chua. 2024. Step: Enhancing video-llms' compositional reasoning by spatio-temporal graph-guided self-training. arXiv preprint arXiv:2412.00161

    2024 arXiv

  29. [37]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, and 1 others. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300

    2024 arXiv

  30. [38]

    Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. 2024. Autoregressive model beats diffusion: Llama for scalable image generation. arXiv preprint arXiv:2406.06525

    2024 arXiv

  31. [39]

    Jordan Vice, Naveed Akhtar, Richard Hartley, and Ajmal Mian. 2025. Exploring bias in over 100 text-to-image generative models. arXiv preprint arXiv:2503.08012

    2025 arXiv

  32. [40]

    Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, and 1 others. 2024. Emu3: Next-token prediction is all you need. arXiv preprint arXiv:2409.18869

    2024 arXiv

  33. [41]

    Yecheng Wu, Zhuoyang Zhang, Junyu Chen, Haotian Tang, Dacheng Li, Yunhao Fang, Ligeng Zhu, Enze Xie, Hongxu Yin, Li Yi, and 1 others. 2024. Vila-u: a unified foundation model integrating visual understanding and generation. arXiv preprint arXiv:2409.04429

    2024 arXiv

  34. [42]

    Enze Xie, Junsong Chen, Yuyang Zhao, Jincheng Yu, Ligeng Zhu, Chengyue Wu, Yujun Lin, Zhekai Zhang, Muyang Li, Junyu Chen, and 1 others. 2025. Sana 1.5: Efficient scaling of training-time and inference-time compute in linear diffusion transformer. arXiv preprint arXiv:2501.18427

    2025 arXiv

  35. [43]

    Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. 2024. Show-o: One single transformer to unify multimodal understanding and generation. arXiv preprint arXiv:2408.12528

    2024 arXiv

  36. [44]

    Yuanyang Yin, Yaqi Zhao, Yajie Zhang, Ke Lin, Jiahao Wang, Xin Tao, Pengfei Wan, Di Zhang, Baoqun Yin, and Wentao Zhang. 2024. Sea: Supervised embedding alignment for token-level visual-textual integration in mllms. arXiv preprint arXiv:2408.11813

    2024 arXiv

  37. [45]

    Qifan Yu, Wei Chow, Zhongqi Yue, Kaihang Pan, Yang Wu, Xiaoyang Wan, Juncheng Li, Siliang Tang, Hanwang Zhang, and Yueting Zhuang. 2024. Anyedit: Mastering unified high-quality image editing for any idea. arXiv preprint arXiv:2411.15738

    2024 arXiv

  38. [46]

    Haozhe Zhao, Xiaojian Shawn Ma, Liang Chen, Shuzheng Si, Rujie Wu, Kaikai An, Peiyu Yu, Minjia Zhang, Qing Li, and Baobao Chang. 2024 a . Ultraedit: Instruction-based fine-grained image editing at scale. Advances in Neural Information Processing Systems, 37:3058--3093

    2024

  39. [47]

    Yaqi Zhao, Yuanyang Yin, Lin Li, Mingan Lin, Victor Shea-Jay Huang, Siwei Chen, Weipeng Chen, Baoqun Yin, Zenan Zhou, and Wentao Zhang. 2024 b . Beyond sight: Towards cognitive alignment in lvlm via enriched visual knowledge. arXiv preprint arXiv:2411.16824

    2024 arXiv

  40. [48]

    Xianwei Zhuang, Yuxin Xie, Yufan Deng, Dongchao Yang, Liming Liang, Jinghan Ru, Yuguo Yin, and Yuexian Zou. 2025. Vargpt-v1. 1: Improve visual autoregressive large unified model via iterative instruction tuning and reinforcement learning. arXiv preprint arXiv:2504.02949

    2025 arXiv

  41. [49]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  42. [50]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.