pith. machine review for the scientific record. sign in

arxiv: 2604.26341 · v1 · submitted 2026-04-29 · 💻 cs.CV

Recognition: unknown

SpatialFusion: Endowing Unified Image Generation with Intrinsic 3D Geometric Awareness

Haiyi Qiu , Kaihang Pan , Jiacheng Li , Juncheng Li , Siliang Tang , Yueting Zhuang
Authors on Pith no claims yet

Pith reviewed 2026-05-07 13:41 UTC · model grok-4.3

classification 💻 cs.CV
keywords unified image generation3D geometric awarenessmetric depth mapsMixture-of-Transformersdiffusion backbonespatial coherenceimage editingMLLM augmentation
0
0 comments X

The pith

SpatialFusion adds a parallel spatial transformer to MLLMs that derives metric-depth maps from semantic context and feeds them to diffusion models for 3D-aware generation.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces SpatialFusion to fix the missing spatial understanding in unified image generation models that pair MLLMs for semantics with diffusion backbones for synthesis. It augments the MLLM via a Mixture-of-Transformers design so a dedicated spatial transformer shares self-attention and extracts metric-depth maps directly from semantic signals. These depth scaffolds pass through a depth adapter into the diffusion backbone to enforce geometric constraints during generation. A progressive two-stage training schedule then delivers stronger results on spatial benchmarks, better general text-to-image and editing performance, and almost no added inference cost. A reader would care because explicit 3D geometry inside the model can produce more consistent, realistic outputs without separate depth predictors.

Core claim

SpatialFusion internalizes 3D geometric awareness by employing a Mixture-of-Transformers architecture that augments the MLLM with a parallel spatial transformer; shared self-attention lets the spatial branch derive metric-depth maps of target images from rich semantic contexts. These explicit geometric scaffolds are injected into the diffusion backbone through a specialized depth adapter that supplies precise spatial constraints, and a progressive two-stage training strategy produces markedly better results on spatially-aware benchmarks while preserving gains on standard generation and editing tasks with negligible inference overhead.

What carries the argument

Mixture-of-Transformers (MoT) architecture in which a spatial transformer shares self-attention with the MLLM to derive metric-depth maps that are injected via a depth adapter into the diffusion backbone.

If this is right

  • Performance rises substantially on spatially-aware benchmarks.
  • The model outperforms leading unified systems such as GPT-4o on those tasks.
  • Generalized gains appear in both text-to-image generation and image editing.
  • Inference overhead stays negligible after the two-stage training.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Similar shared-attention branches could let other multimodal models acquire geometric understanding without dedicated depth networks.
  • The same scaffolds might improve consistency in video generation or novel-view synthesis by carrying depth across frames or viewpoints.
  • If semantic context alone drives accurate depth, the approach could be tested on scenes with heavy occlusion or ambiguous lighting to find its limits.
  • The low overhead opens the possibility of adding further geometric outputs such as surface normals without changing deployment cost.

Load-bearing premise

Sharing self-attention between the MLLM and the added spatial transformer is sufficient for the spatial branch to derive accurate metric-depth maps from semantic context alone, without explicit depth supervision or additional geometric losses.

What would settle it

If the spatial transformer produces low-accuracy metric-depth maps when compared against ground-truth depths on a held-out set of 3D-structured scenes, or if ablating the shared self-attention removes the reported gains on spatial benchmarks, the core mechanism would be refuted.

Figures

Figures reproduced from arXiv: 2604.26341 by Haiyi Qiu, Jiacheng Li, Juncheng Li, Kaihang Pan, Siliang Tang, Yueting Zhuang.

Figure 1
Figure 1. Figure 1: (a) MLLMs produce geometry-deficient hidden view at source ↗
Figure 2
Figure 2. Figure 2: Overview of SpatialFusion. The framework internalizes 3D awareness via: (1) Semantics-Guided Geometric Derivation, view at source ↗
Figure 4
Figure 4. Figure 4: (a) Ablation study on the shared attention sampling view at source ↗
Figure 3
Figure 3. Figure 3: Qualitative results of image synthesis guided by view at source ↗
Figure 5
Figure 5. Figure 5: Qualitative comparison of spatially-aware image generation. Our method achieves the best performance. view at source ↗
read the original abstract

Recent unified image generation models have achieved remarkable success by employing MLLMs for semantic understanding and diffusion backbones for image generation. However, these models remain fundamentally limited in spatially-aware tasks due to a lack of intrinsic spatial understanding and the absence of explicit geometric guidance during generation. In this paper, we propose SpatialFusion, a novel framework that internalizes 3D geometric awareness into unified image generation models. Specifically, we first employ a Mixture-of-Transformers (MoT) architecture to augment the MLLM with a parallel spatial transformer to enhance 3D geometric modeling capability. By sharing self-attention with the MLLM, the spatial transformer learns to derive metric-depth maps of target images from rich semantic contexts. These explicit geometric scaffolds are then injected into the diffusion backbone through a specialized depth adapter, providing precise spatial constraints for spatially-coherent image generation. Through a progressive two-stage training strategy, SpatialFusion significantly enhances performance on spatially-aware benchmarks, notably outperforming leading models such as GPT-4o. Additionally, it achieves generalized performance gains across both text-to-image generation and image editing scenarios, all while maintaining negligible inference overhead.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The manuscript proposes SpatialFusion, a framework to endow unified image generation models with intrinsic 3D geometric awareness. It augments an MLLM with a parallel spatial transformer in a Mixture-of-Transformers (MoT) architecture; the transformer shares self-attention to derive metric-depth maps from semantic context. These maps are injected into the diffusion backbone via a depth adapter to provide spatial constraints. A progressive two-stage training strategy is claimed to yield significant gains on spatially-aware benchmarks (outperforming GPT-4o), plus generalized improvements in text-to-image generation and image editing, all with negligible inference overhead.

Significance. If the depth maps produced by the spatial branch are metrically accurate and the adapter successfully transfers geometric constraints, the approach could meaningfully advance spatially coherent unified generation without added inference cost. The shared-attention design for geometric modeling is conceptually interesting and the low-overhead claim is attractive. However, the absence of any quantitative results, ablations, or supervision details in the abstract makes it impossible to judge whether the significance materializes.

major comments (2)
  1. [Abstract] Abstract: the manuscript asserts benchmark outperformance and superiority over GPT-4o yet supplies no quantitative numbers, ablation results, error bars, training details, or dataset information to support these claims.
  2. [Method] Method (architecture description): the central claim that the parallel spatial transformer derives accurate metric-depth maps from semantic features alone via shared self-attention is load-bearing, yet the text provides no depth regression loss, ground-truth depth supervision, or geometric regularizers during the two-stage training. Metric depth is scale-sensitive and underdetermined from 2D semantics; without explicit supervision the subsequent depth adapter cannot be guaranteed to supply reliable spatial constraints.
minor comments (1)
  1. [Abstract] Abstract: the phrase 'notably outperforming leading models such as GPT-4o' is imprecise without naming the specific benchmarks or reporting margins.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback. We address each major comment below and outline the revisions we will make to strengthen the manuscript.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the manuscript asserts benchmark outperformance and superiority over GPT-4o yet supplies no quantitative numbers, ablation results, error bars, training details, or dataset information to support these claims.

    Authors: We agree that the abstract would benefit from including a small number of key quantitative highlights to better support the claims within the available space. The body of the manuscript (Section 4) contains the full results, including specific benchmark scores, comparisons to GPT-4o, ablations, and dataset details. In the revised version we will add concise numerical examples and dataset references to the abstract. revision: yes

  2. Referee: [Method] Method (architecture description): the central claim that the parallel spatial transformer derives accurate metric-depth maps from semantic features alone via shared self-attention is load-bearing, yet the text provides no depth regression loss, ground-truth depth supervision, or geometric regularizers during the two-stage training. Metric depth is scale-sensitive and underdetermined from 2D semantics; without explicit supervision the subsequent depth adapter cannot be guaranteed to supply reliable spatial constraints.

    Authors: The referee is correct that the current manuscript text does not explicitly describe the supervision and loss used for the spatial transformer. The two-stage training procedure (detailed in Section 3.3) does include direct metric-depth supervision on the spatial branch using ground-truth depth maps; however, these details were omitted from the architecture description. We will revise the method section to add the depth regression loss (L1 + scale-invariant term), the ground-truth supervision sources, and any geometric regularizers applied during training, thereby clarifying how metric accuracy is enforced. revision: yes

Circularity Check

0 steps flagged

No significant circularity; derivation relies on novel architectural additions

full rationale

The paper introduces a Mixture-of-Transformers architecture augmenting an MLLM with a parallel spatial transformer that shares self-attention to produce metric-depth maps, which are then passed through a depth adapter to the diffusion backbone under a two-stage training regime. This chain is presented as an empirical construction of new components rather than any re-expression of target outputs in terms of fitted inputs or prior results. No equations, self-citations, uniqueness theorems, or ansatzes are shown reducing the claimed 3D awareness or benchmark gains to tautological mappings of the inputs. The derivation remains self-contained as an architectural proposal whose validity rests on external training and evaluation rather than internal definitional equivalence.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Only the abstract is available, so concrete free parameters, axioms, or invented entities cannot be enumerated. The approach appears to rest on standard assumptions of diffusion models and multimodal transformers plus the unstated premise that shared attention suffices for metric depth prediction.

pith-pipeline@v0.9.0 · 5517 in / 1178 out tokens · 88952 ms · 2026-05-07T13:41:16.661303+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

63 extracted references · 31 canonical work pages · 13 internal anchors

  1. [1]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. 2025. Qwen2.5-VL Technical Report.arXiv preprint arXiv:2502.13923(2025)

    work page internal anchor Pith review arXiv 2025

  2. [2]

    James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al . 2023. Improving im- age generation with better captions.Computer Science. https://cdn. openai. com/papers/dall-e-3. pdf2, 3 (2023), 8

    2023

  3. [3]

    Mahtab Bigverdi, Zelun Luo, Cheng-Yu Hsieh, Ethan Shen, Dongping Chen, Linda G Shapiro, and Ranjay Krishna. 2025. Perception tokens enhance visual reasoning in multimodal language models. InProceedings of the Computer Vision and Pattern Recognition Conference. 3836–3845

    2025

  4. [4]

    Tim Brooks, Aleksander Holynski, and Alexei A Efros. 2023. Instructpix2pix: Learning to follow image editing instructions. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition. 18392–18402

    2023

  5. [5]

    Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhong- dao Wang, James Kwok, Ping Luo, Huchuan Lu, et al. 2023. Pixart-alpha: Fast training of diffusion transformer for photorealistic text-to-image synthesis.arXiv preprint arXiv:2310.00426(2023)

    work page internal anchor Pith review arXiv 2023

  6. [6]

    Xiaokang Chen, Zhiyu Wu, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, and Chong Ruan. 2025. Janus-pro: Unified multimodal understanding and generation with data and model scaling.arXiv preprint arXiv:2501.17811 (2025)

    work page internal anchor Pith review arXiv 2025

  7. [7]

    Chaorui Deng, Deyao Zhu, Kunchang Li, Chenhui Gou, Feng Li, Zeyu Wang, Shu Zhong, Weihao Yu, Xiaonan Nie, Ziang Song, et al. 2025. Emerging properties in unified multimodal pretraining.arXiv preprint arXiv:2505.14683(2025)

    work page internal anchor Pith review arXiv 2025

  8. [8]

    Tianchen Deng, Yue Pan, Shenghai Yuan, Dong Li, Chen Wang, Mingrui Li, Long Chen, Lihua Xie, Danwei Wang, Jingchuan Wang, et al. 2025. What is the best 3d scene representation for robotics? from geometric to foundation models.arXiv preprint arXiv:2512.03422(2025)

    work page arXiv 2025

  9. [9]

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. 2024. Scaling rectified flow transformers for high-resolution image synthesis. InForty- first international conference on machine learning

    2024

  10. [10]

    Yu Gao, Lixue Gong, Qiushan Guo, Xiaoxia Hou, Zhichao Lai, Fanshi Li, Liang Li, Xiaochen Lian, Chao Liao, Liyang Liu, et al . 2025. Seedream 3.0 technical report.arXiv preprint arXiv:2504.11346(2025)

    work page internal anchor Pith review arXiv 2025

  11. [11]

    Dhruba Ghosh, Hannaneh Hajishirzi, and Ludwig Schmidt. 2023. Geneval: An object-focused framework for evaluating text-to-image alignment.Advances in Neural Information Processing Systems36 (2023), 52132–52152

    2023

  12. [12]

    Google. 2025. Gemini 2.0 Flash. https://aistudio.google.com/prompts/new_chat? model=gemini-2.0-flash-exp. Accessed: 2026-04-01

    2025

  13. [13]

    Sen He, Wentong Liao, Michael Ying Yang, Yongxin Yang, Yi-Zhe Song, Bodo Rosenhahn, and Tao Xiang. 2021. Context-aware layout to image generation with enhanced object appearance. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition. 15049–15058

    2021

  14. [14]

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising diffusion probabilistic models.Advances in neural information processing systems33 (2020), 6840–6851

    2020

  15. [15]

    Xiwei Hu, Rui Wang, Yixiao Fang, Bin Fu, Pei Cheng, and Gang Yu. 2024. Ella: Equip diffusion models with llm for enhanced semantic alignment.arXiv preprint arXiv:2403.05135(2024)

    work page internal anchor Pith review arXiv 2024

  16. [16]

    Zijing Hu, Yunze Tong, Fengda Zhang, Junkun Yuan, Jun Xiao, and Kun Kuang

  17. [17]

    Asynchronous Denoising Diffusion Models for Aligning Text-to-Image Generation.arXiv preprint arXiv:2510.04504(2025)

    work page arXiv 2025

  18. [18]

    Zijing Hu, Junkun Yuan, Kairong Han, Yunze Tong, Shengyu Zhang, Fei Wu, and Kun Kuang. 2026. Reinforcement Learning in Generative Multimodal AI: A Survey. (2026)

    2026

  19. [19]

    Zijing Hu, Fengda Zhang, Long Chen, Kun Kuang, Jiahui Li, Kaifeng Gao, Jun Xiao, Xin Wang, and Wenwu Zhu. 2025. Towards better alignment: Training dif- fusion models with reinforcement learning against sparse rewards. InProceedings of the Computer Vision and Pattern Recognition Conference. 23604–23614

    2025

  20. [20]

    Zijing Hu, Fengda Zhang, and Kun Kuang. 2025. D-fusion: Direct preference optimization for aligning diffusion models with visually consistent samples. arXiv preprint arXiv:2505.22002(2025)

    work page arXiv 2025

  21. [21]

    Kaiyi Huang, Chengqi Duan, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu. 2025. T2i-compbench++: An enhanced and comprehensive benchmark for compositional text-to-image generation.IEEE Transactions on Pattern Analysis and Machine Intelligence47, 5 (2025), 3563–3579

    2025

  22. [22]

    Mao Xun Huang, Brian J Chan, and Hen-Hsen Huang. 2025. SmartSpatial: enhancing 3D spatial awareness in stable diffusion with a novel evaluation framework. InProceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence. 10099–10107

    2025

  23. [23]

    Black Forest Labs. 2024. FLUX. https://github.com/black-forest-labs/flux

    2024

  24. [24]

    Black Forest Labs. 2025. FLUX.1-Kontext-dev. https://huggingface.co/black- forest-labs/FLUX.1-Kontext-dev

    2025

  25. [25]

    Yuheng Li, Haotian Liu, Qingyang Wu, Fangzhou Mu, Jianwei Yang, Jianfeng Gao, Chunyuan Li, and Yong Jae Lee. 2023. Gligen: Open-set grounded text-to- image generation. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition. 22511–22521

    2023

  26. [26]

    Weixin Liang, Lili Yu, Liang Luo, Srinivasan Iyer, Ning Dong, Chunting Zhou, Gargi Ghosh, Mike Lewis, Wen-tau Yih, Luke Zettlemoyer, et al. 2024. Mixture- of-transformers: A sparse and scalable architecture for multi-modal foundation models.arXiv preprint arXiv:2411.04996(2024)

    work page arXiv 2024

  27. [27]

    Bin Lin, Zongjian Li, Xinhua Cheng, Yuwei Niu, Yang Ye, Xianyi He, Shenghai Yuan, Wangbo Yu, Shaodong Wang, Yunyang Ge, et al. 2025. Uniworld-v1: High- resolution semantic encoders for unified visual understanding and generation. arXiv preprint arXiv:2506.03147(2025)

    work page internal anchor Pith review arXiv 2025

  28. [28]

    Shiyu Liu, Yucheng Han, Peng Xing, Fukun Yin, Rui Wang, Wei Cheng, Jiaqi Liao, Yingming Wang, Honghao Fu, Chunrui Han, et al . 2025. Step1x-edit: A practical framework for general image editing.arXiv preprint arXiv:2504.17761 (2025)

    work page internal anchor Pith review arXiv 2025

  29. [29]

    Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, and Ying Shan. 2024. T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. InProceedings of the AAAI conference on artificial intelligence, Vol. 38. 4296–4304

    2024

  30. [30]

    OpenAI. 2025. Introducing 4o Image Generation. https://openai.com/index/ introducing-4o-image-generation/. Accessed: 2026-03-31

    2025

  31. [31]

    Kaihang Pan, Wendong Bu, Yuruo Wu, Yang Wu, Kai Shen, Yunfei Li, Hang Zhao, Juncheng Li, Siliang Tang, and Yueting Zhuang. 2025. Focusdiff: Advancing fine-grained text-image alignment for autoregressive visual generation through rl.arXiv preprint arXiv:2506.05501(2025)

    work page arXiv 2025

  32. [32]

    Kaihang Pan, Weile Chen, Haiyi Qiu, Qifan Yu, Wendong Bu, Zehan Wang, Yun Zhu, Juncheng Li, and Siliang Tang. 2025. Wiseedit: Benchmarking cognition-and creativity-informed image editing.arXiv preprint arXiv:2512.00387(2025)

    work page arXiv 2025

  33. [33]

    Kaihang Pan, Wang Lin, Zhongqi Yue, Tenglong Ao, Liyu Jia, Wei Zhao, Juncheng Li, Siliang Tang, and Hanwang Zhang. 2025. Generative multimodal pretraining with discrete diffusion timestep tokens. InProceedings of the Computer Vision and Pattern Recognition Conference. 26136–26146

    2025

  34. [34]

    Kaihang Pan, Qi Tian, Jianwei Zhang, Weijie Kong, Jiangfeng Xiong, Yanxin Long, Shixue Zhang, Haiyi Qiu, Tan Wang, Zheqi Lv, et al. 2026. OmniWeaving: Towards Unified Video Generation with Free-form Composition and Reasoning. arXiv preprint arXiv:2603.24458(2026)

    work page arXiv 2026

  35. [35]

    Kaihang Pan, Yang Wu, Wendong Bu, Kai Shen, Juncheng Li, Yingting Wang, Yunfei Li, Siliang Tang, Jun Xiao, Fei Wu, et al. 2025. Janus-pro-r1: Advancing collaborative visual comprehension and generation via reinforcement learning. arXiv preprint arXiv:2506.01480(2025)

    work page arXiv 2025

  36. [36]

    Kiho Park, Yo Joong Choe, Yibo Jiang, and Victor Veitch. 2024. The geometry of categorical and hierarchical concepts in large language models.arXiv preprint arXiv:2406.01506(2024)

    work page arXiv 2024

  37. [37]

    William Peebles and Saining Xie. 2023. Scalable diffusion models with transform- ers. InProceedings of the IEEE/CVF international conference on computer vision. 4195–4205. Haiyi Qiu, Kaihang Pan, Jiacheng Li, Juncheng Li, Siliang Tang, and Yueting Zhuang

    2023

  38. [38]

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. 2023. Sdxl: Improving latent diffu- sion models for high-resolution image synthesis.arXiv preprint arXiv:2307.01952 (2023)

    work page internal anchor Pith review arXiv 2023

  39. [39]

    Can Qin, Shu Zhang, Ning Yu, Yihao Feng, Xinyi Yang, Yingbo Zhou, Huan Wang, Juan Carlos Niebles, Caiming Xiong, Silvio Savarese, et al . 2023. Unicontrol: A unified diffusion model for controllable visual generation in the wild.arXiv preprint arXiv:2305.11147(2023)

    work page arXiv 2023

  40. [40]

    René Ranftl, Alexey Bochkovskiy, and Vladlen Koltun. 2021. Vision transformers for dense prediction. InProceedings of the IEEE/CVF international conference on computer vision. 12179–12188

    2021

  41. [41]

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 10684–10695

    2022

  42. [42]

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. 2022. Photorealistic text-to-image diffusion models with deep language understanding.Advances in neural information processing systems35 (2022), 36479–36494

    2022

  43. [43]

    You Shen, Zhipeng Zhang, Yansong Qu, Xiawu Zheng, Jiayi Ji, Shengchuan Zhang, and Liujuan Cao. 2025. Fastvggt: Training-free acceleration of visual geometry transformer.arXiv preprint arXiv:2509.02560(2025)

    work page arXiv 2025

  44. [44]

    Weijia Shi, Xiaochuang Han, Chunting Zhou, Weixin Liang, Xi Victoria Lin, Luke Zettlemoyer, and Lili Yu. 2024. Lmfusion: Adapting pretrained language models for multimodal generation.arXiv preprint arXiv:2412.15188(2024)

    work page arXiv 2024

  45. [45]

    Yichun Shi, Peng Wang, and Weilin Huang. 2024. Seededit: Align image re- generation to image editing.arXiv preprint arXiv:2411.06686(2024)

    work page arXiv 2024

  46. [46]

    Zihan Su, Hongyang Wei, Kangrui Cen, Yong Wang, Guanhua Chen, Chun Yuan, and Xiangxiang Chu. 2026. Generation Enhances Understanding in Uni- fied Multimodal Models via Multi-Representation Generation.arXiv preprint arXiv:2601.21406(2026)

    work page arXiv 2026

  47. [47]

    Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rup- precht, and David Novotny. 2025. Vggt: Visual geometry grounded transformer. InProceedings of the Computer Vision and Pattern Recognition Conference. 5294– 5306

    2025

  48. [48]

    Zengbin Wang, Xuecai Hu, Yong Wang, Feng Xiong, Man Zhang, and Xiangxiang Chu. 2026. Everything in Its Place: Benchmarking Spatial Intelligence of Text-to- Image Models.arXiv preprint arXiv:2601.20354(2026)

    work page arXiv 2026

  49. [49]

    Zehan Wang, Jiayang Xu, Ziang Zhang, Tianyu Pang, Chao Du, Hengshuang Zhao, and Zhou Zhao. 2025. GenSpace: Benchmarking Spatially-Aware Image Generation.arXiv preprint arXiv:2505.24870(2025)

    work page arXiv 2025

  50. [50]

    Chenfei Wu, Jiahao Li, Jingren Zhou, Junyang Lin, Kaiyuan Gao, Kun Yan, Sheng- ming Yin, Shuai Bai, Xiao Xu, Yilei Chen, et al . 2025. Qwen-image technical report.arXiv preprint arXiv:2508.02324(2025)

    work page internal anchor Pith review arXiv 2025

  51. [51]

    Chenyuan Wu, Pengfei Zheng, Ruiran Yan, Shitao Xiao, Xin Luo, Yueze Wang, Wanli Li, Xiyan Jiang, Yexin Liu, Junjie Zhou, et al. 2025. Omnigen2: Exploration to advanced multimodal generation.arXiv preprint arXiv:2506.18871(2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  52. [52]

    Diankun Wu, Fangfu Liu, Yi-Hsin Hung, and Yueqi Duan. 2025. Spatial-mllm: Boosting mllm capabilities in visual-based spatial intelligence.arXiv preprint arXiv:2505.23747(2025)

    work page arXiv 2025

  53. [53]

    Shitao Xiao, Yueze Wang, Junjie Zhou, Huaying Yuan, Xingrun Xing, Ruiran Yan, Chaofan Li, Shuting Wang, Tiejun Huang, and Zheng Liu. 2025. Omnigen: Unified image generation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 13294–13304

    2025

  54. [54]

    Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. 2024. Show-o: One single transformer to unify multimodal understanding and generation.arXiv preprint arXiv:2408.12528(2024)

    work page internal anchor Pith review arXiv 2024

  55. [55]

    Zhen Xu, Hongyu Zhou, Sida Peng, Haotong Lin, Haoyu Guo, Jiahao Shao, Peis- han Yang, Qinglin Yang, Sheng Miao, Xingyi He, et al . 2026. Towards depth foundation models: Recent trends in vision-based depth estimation.Computa- tional Visual Media(2026)

    2026

  56. [56]

    Yang Ye, Xianyi He, Zongjian Li, Bin Lin, Shenghai Yuan, Zhiyuan Yan, Bohan Hou, and Li Yuan. 2025. Imgedit: A unified image editing dataset and benchmark. arXiv preprint arXiv:2505.20275(2025)

    work page internal anchor Pith review arXiv 2025

  57. [57]

    Qifan Yu, Wei Chow, Zhongqi Yue, Kaihang Pan, Yang Wu, Xiaoyang Wan, Juncheng Li, Siliang Tang, Hanwang Zhang, and Yueting Zhuang. 2025. Anyedit: Mastering unified high-quality image editing for any idea. InProceedings of the Computer Vision and Pattern Recognition Conference. 26125–26135

    2025

  58. [58]

    Jiaxin Zhang, Junjun Jiang, Haijie Li, Youyu Chen, Kui Jiang, and Dave Zhenyu Chen. 2026. GAP-MLLM: Geometry-Aligned Pre-training for Activating 3D Spatial Perception in Multimodal Large Language Models.arXiv preprint arXiv:2603.16461(2026)

    work page arXiv 2026

  59. [59]

    Kai Zhang, Lingbo Mo, Wenhu Chen, Huan Sun, and Yu Su. 2023. Magicbrush: A manually annotated dataset for instruction-guided image editing.Advances in Neural Information Processing Systems36 (2023), 31428–31449

    2023

  60. [60]

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. 2023. Adding conditional control to text-to-image diffusion models. InProceedings of the IEEE/CVF inter- national conference on computer vision. 3836–3847

    2023

  61. [61]

    Zechuan Zhang, Ji Xie, Yu Lu, Zongxin Yang, and Yi Yang. 2025. Enabling in- structional image editing with in-context generation in large scale diffusion transformer. InThe Thirty-ninth Annual Conference on Neural Information Pro- cessing Systems

    2025

  62. [62]

    Haozhe Zhao, Xiaojian Ma, Liang Chen, Shuzheng Si, Rujie Wu, Kaikai An, Peiyu Yu, Minjia Zhang, Qing Li, and Baobao Chang. 2024. Ultraedit: Instruction-based fine-grained image editing at scale.Advances in Neural Information Processing Systems37 (2024), 3058–3093

    2024

  63. [63]

    Guangcong Zheng, Xianpan Zhou, Xuewei Li, Zhongang Qi, Ying Shan, and Xi Li. 2023. Layoutdiffusion: Controllable diffusion model for layout-to-image generation. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition. 22490–22499

    2023