Pith

open record

sign in

arxiv: 2606.13345 · v1 · pith:5DOGPDOY · submitted 2026-06-11 · cs.CV

JointEdit3D: Feed-Forward 3D Scene Editing in a Unified Latent Space

Xinnan Zhu , Ruijie Xu , Jiayu Ying , Daoguo Dong , Jiachen Xu , Yuan Xie , Xin Tan This is my paper

Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel 2026-06-27 07:24 UTCgrok-4.3pith:5DOGPDOYrecord.jsonopen to challenge →

classification cs.CV
keywords 3D scene editingfeed-forward inferencelatent space inpaintingunified RGB-geometrySceneAnchor Branchpaired editing datasetasymmetric generation
0
0 comments X

The pith

JointEdit3D performs 3D scene editing in a single forward pass by asymmetrically inpainting a unified RGB-geometry latent space while anchoring to the source scene.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper aims to establish that 3D scene editing can be performed efficiently in one forward pass rather than through per-scene optimization or cascaded pipelines. It does so by adapting a shared latent space that already couples RGB appearance and geometry to the editing task, using only a single edited reference view to generate consistent outputs across views and geometry. A dedicated branch selectively injects source structure, and specialized losses keep edited parts faithful while protecting unchanged regions. The work also supplies a large paired dataset and benchmark to make evaluation of such methods more standardized. A sympathetic reader would care because this removes high test-time costs and reduces structural inconsistencies that plague current approaches.

Core claim

JointEdit3D adapts a unified RGB-geometry reconstruction-generation latent space to feed-forward 3D scene editing through asymmetric latent inpainting, where only a single edited RGB reference latent is observed, and the remaining RGB views along with edited geometry are generated under source-scene anchoring via a dedicated SceneAnchor Branch and edit/background-aware losses.

What carries the argument

Asymmetric latent inpainting performed in a unified RGB-geometry latent space, guided by a SceneAnchor Branch that injects source-scene structure without direct copying.

If this is right

  • 3D editing no longer requires iterative per-scene optimization at test time.
  • Edited regions show improved visual quality while overall 3D structural completeness increases.
  • Background content remains competitive in preservation quality with earlier methods.
  • Standardized comparison of 3D editing methods becomes possible through the released paired dataset and benchmark.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The feed-forward design could support interactive editing sessions where users make repeated changes without waiting for optimization.
  • Similar asymmetric anchoring might apply to related tasks such as editing dynamic 3D content or multi-object scenes.
  • If the latent space generalizes well, the approach could extend to editing scenes captured from consumer devices rather than controlled renders.

Load-bearing premise

The unified RGB-geometry latent space can be adapted to feed-forward editing via asymmetric inpainting and a SceneAnchor Branch without introducing structural inconsistencies.

What would settle it

Running the method on a scene with intricate geometry and observing whether the generated 3D structure in non-reference views matches the edited appearance without visible mismatches or holes.

Figures

Figures reproduced from arXiv: 2606.13345 by Daoguo Dong, Jiachen Xu, Jiayu Ying, Ruijie Xu, Xinnan Zhu, Xin Tan, Yuan Xie.

Figure 1
Figure 1. Figure 1: JointEdit3D pipeline. JointEdit3D performs RGB-geometry latent inpainting from a source [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: SceneEdit3D-15K data generation. From composable 3D indoor scenes, a VLM proposes [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Efficiency-quality plot. steps with batch size 10 on 8 NVIDIA H200 GPUs. Additional implementation details are provided in Appendix B.1. Baselines. We compare with representative optimization-based and feed-forward 3D/multi-view editing methods: SPIn-NeRF [21], Gaussian Grouping [38], GScream [31], GaussianEditor [6], MVInpainter [2], and Omni-3DEdit [4], and include SEVA [44] as a protocol-mismatched nove… view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative comparison across synthetic SceneEdit3D-Bench and real-world 360- [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Additional qualitative comparison on challenging operation types, including dynamic object [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Additional qualitative examples on ScanNet++ and casually captured real scenes. [PITH_FULL_IMAGE:figures/full_fig_p010_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Training-time response diagnostic for region-decomposed supervision. RGB and geometry [PITH_FULL_IMAGE:figures/full_fig_p014_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Edit-condition impact visualization. Heatmaps show the denoiser response difference [PITH_FULL_IMAGE:figures/full_fig_p015_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Threshold calibration for latent-difference edit/background decomposition. We sweep [PITH_FULL_IMAGE:figures/full_fig_p016_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Additional qualitative comparison for appearance editing. [PITH_FULL_IMAGE:figures/full_fig_p017_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Additional qualitative comparison for real-scene object removal. [PITH_FULL_IMAGE:figures/full_fig_p018_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Additional qualitative examples for real-scene multi-editing. [PITH_FULL_IMAGE:figures/full_fig_p019_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Expanded representative VLM prompt for proposing canonical renderer-executable tasks. [PITH_FULL_IMAGE:figures/full_fig_p021_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Representative prompt for converting rendered before/after evidence into natural user [PITH_FULL_IMAGE:figures/full_fig_p021_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Algorithmic view of the Blender execution protocol. Layout parsing, collision checks, [PITH_FULL_IMAGE:figures/full_fig_p022_15.png] view at source ↗
read the original abstract

Existing 3D scene editing methods typically rely on per-scene optimization over explicit 3D representations or cascaded edit-and-reconstruct pipelines, resulting in high test-time cost, limited 3D awareness, and structural inconsistencies. To couple appearance synthesis and geometry prediction during editing, we build on a unified RGB-geometry reconstruction-generation latent space and adapt it to feed-forward 3D scene editing. The resulting framework, \textbf{JointEdit3D}, performs asymmetric latent inpainting by observing only a single edited RGB reference latent and generating the remaining RGB views and edited geometry latent under source-scene anchoring. JointEdit3D introduces a dedicated SceneAnchor Branch to inject source-scene structure without forcing direct copying, and adopts edit/background-aware losses to balance edited-region fidelity with unedited-content preservation. To address the lack of paired resources for standardized 3D scene editing evaluation, we introduce SceneEdit3D-15K, a dataset with 15K paired editing samples and renderer-provided 3D annotations, together with SceneEdit3D-Bench, a curated 100-sample benchmark. Experiments show that JointEdit3D improves edited-region quality and 3D structural completeness over prior baselines while maintaining competitive background preservation.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

0 major / 1 minor

Summary. The manuscript introduces JointEdit3D, a feed-forward 3D scene editing framework that adapts a unified RGB-geometry reconstruction-generation latent space via asymmetric latent inpainting (observing only a single edited RGB reference latent while generating remaining views and edited geometry under source-scene anchoring). It adds a SceneAnchor Branch to inject source structure without direct copying, employs edit/background-aware losses, and releases the SceneEdit3D-15K dataset (15K paired samples with 3D annotations) plus SceneEdit3D-Bench (100-sample curated benchmark). Experiments claim improvements in edited-region quality and 3D structural completeness over baselines while preserving competitive background fidelity.

Significance. If the central claims hold, the work offers a practical advance by replacing per-scene optimization and cascaded pipelines with a single feed-forward pass in a shared latent space, which could lower test-time cost and reduce structural inconsistencies in 3D editing. The new paired dataset and benchmark address a clear evaluation gap and would benefit the community regardless of the method's ultimate performance.

minor comments (1)
  1. Abstract: reports experimental improvements but supplies no equations, loss formulations, architecture diagrams, or data splits, preventing verification of whether gains are supported by the implementation or affected by post-hoc choices.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for their review of our manuscript. The report provides a clear summary of JointEdit3D and recognizes the potential significance of replacing per-scene optimization with a feed-forward approach in a unified latent space, as well as the contribution of the new dataset and benchmark. The recommendation is listed as uncertain, yet the report contains no specific major comments. We address this below and remain available to clarify any aspects that may have contributed to the uncertainty.

Circularity Check

0 steps flagged

No significant circularity detected

full rationale

The paper presents an empirical method that adapts an existing unified RGB-geometry latent space (described as built upon, not derived within this work) via asymmetric inpainting, a SceneAnchor Branch, and specialized losses. No equations, derivations, or self-referential definitions are present that would reduce outputs to inputs by construction. Evaluation relies on a newly introduced dataset and benchmark with comparisons to baselines, making the central claims externally falsifiable rather than tautological. No load-bearing self-citations or fitted predictions masquerading as results are identifiable from the text.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on the effectiveness of adapting a pre-existing unified latent space and the utility of the new dataset; no free parameters, axioms, or invented entities are explicitly detailed in the abstract.

axioms (1)
  • domain assumption A unified RGB-geometry reconstruction-generation latent space can be adapted for editing tasks.
    Stated as the foundation for the feed-forward editing approach.

pith-pipeline@v0.9.1-grok · 5770 in / 1263 out tokens · 21456 ms · 2026-06-27T07:24:33.553824+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

58 extracted references · 6 canonical work pages · 3 internal anchors

  1. [1]

    Instructpix2pix: Learning to follow image editing instructions

    Tim Brooks, Aleksander Holynski, and Alexei A Efros. Instructpix2pix: Learning to follow image editing instructions. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 18392–18402, 2023

    2023

  2. [2]

    MVInpainter: Learning multi-view consistent inpainting to bridge 2d and 3d editing

    Chenjie Cao, Chaohui Yu, Fan Wang, Xiangyang Xue, and Yanwei Fu. MVInpainter: Learning multi-view consistent inpainting to bridge 2d and 3d editing. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URLhttps://openreview.net/forum?id=XIScpCMUse

    2024

  3. [3]

    Uni3c: Unifying precisely 3d-enhanced camera and human motion controls for video generation

    Chenjie Cao, Jingkai Zhou, Shikai Li, Jingyun Liang, Chaohui Yu, Fan Wang, Xiangyang Xue, and Yanwei Fu. Uni3c: Unifying precisely 3d-enhanced camera and human motion controls for video generation. In Proceedings of the SIGGRAPH Asia 2025 Conference Papers, pages 1–12, 2025

    2025

  4. [4]

    Omni-3DEdit: Generalized versatile 3D editing in one-pass

    Liyi Chen, Pengfei Wang, Guowen Zhang, Zhiyuan Ma, and Lei Zhang. Omni-3DEdit: Generalized versatile 3D editing in one-pass. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026

    2026

  5. [5]

    Dge: Direct gaussian 3d editing by consistent multi-view editing

    Minghao Chen, Iro Laina, and Andrea Vedaldi. Dge: Direct gaussian 3d editing by consistent multi-view editing. InEuropean conference on computer vision, pages 74–92. Springer, 2024

    2024

  6. [6]

    Gaussianeditor: Swift and controllable 3d editing with gaussian splatting

    Yiwen Chen, Zilong Chen, Chi Zhang, Feng Wang, Xiaofeng Yang, Yikai Wang, Zhongang Cai, Lei Yang, Huaping Liu, and Guosheng Lin. Gaussianeditor: Swift and controllable 3d editing with gaussian splatting. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 21476–21485, 2024

    2024

  7. [7]

    Scannet: Richly-annotated 3d reconstructions of indoor scenes

    Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 5828–5839, 2017

    2017

  8. [8]

    3d-front: 3d furnished rooms with layouts and semantics

    Huan Fu, Bowen Cai, Lin Gao, Ling-Xiao Zhang, Jiaming Wang, Cao Li, Qixun Zeng, Chengyue Sun, Rongfei Jia, Binqiang Zhao, et al. 3d-front: 3d furnished rooms with layouts and semantics. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 10933–10942, 2021

    2021

  9. [9]

    Effecterase: Joint video object removal and insertion for high-quality effect erasing

    Yang Fu, Yike Zheng, Ziyun Dai, and Henghui Ding. Effecterase: Joint video object removal and insertion for high-quality effect erasing. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026

    2026

  10. [10]

    Instruct- nerf2nerf: Editing 3d scenes with instructions

    Ayaan Haque, Matthew Tancik, Alexei A Efros, Aleksander Holynski, and Angjoo Kanazawa. Instruct- nerf2nerf: Editing 3d scenes with instructions. InProceedings of the IEEE/CVF international conference on computer vision, pages 19740–19750, 2023

    2023

  11. [11]

    Gen3r: 3d scene generation meets feed-forward reconstruction

    Jiaxin Huang, Yuanbo Yang, Bangbang Yang, Lin Ma, Yuewen Ma, and Yiyi Liao. Gen3r: 3d scene generation meets feed-forward reconstruction. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026

    2026

  12. [12]

    Fuse3d: Generating 3d assets controlled by multi-image fusion

    Xuancheng Jin, Rengan Xie, Wenting Zheng, Rui Wang, Hujun Bao, and Yuchi Huo. Fuse3d: Generating 3d assets controlled by multi-image fusion. InProceedings of the SIGGRAPH Asia 2025 Conference Papers, pages 1–12, 2025

    2025

  13. [13]

    3d gaussian splatting for real-time radiance field rendering.ACM Trans

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, George Drettakis, et al. 3d gaussian splatting for real-time radiance field rendering.ACM Trans. Graph., 42(4):139–1, 2023

    2023

  14. [14]

    Editsplat: Multi-view fusion and attention-guided optimization for view-consistent 3d scene editing with 3d gaussian splatting

    Dong In Lee, Hyeongcheol Park, Jiyoung Seo, Eunbyung Park, Hyunje Park, Ha Dam Baek, Sangheon Shin, Sangmin Kim, and Sangpil Kim. Editsplat: Multi-view fusion and attention-guided optimization for view-consistent 3d scene editing with 3d gaussian splatting. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 11135–11145, 2025

    2025

  15. [15]

    V oxhammer: Training-free precise and coherent 3d editing in native 3d space

    Lin Li, Zehuan Huang, Haoran Feng, Gengxiong Zhuang, Rui Chen, Chunchao Guo, and Lu Sheng. V oxhammer: Training-free precise and coherent 3d editing in native 3d space. InThirteenth International Conference on 3D Vision, 2026. URLhttps://openreview.net/forum?id=UhHNN5lW67

    2026

  16. [16]

    Director3d: Real-world camera trajectory and 3d scene generation from text.Advances in neural information processing systems, 37:75125–75151, 2024

    Xinyang Li, Zhangyu Lai, Linning Xu, Yansong Qu, Liujuan Cao, Shengchuan Zhang, Bo Dai, and Rongrong Ji. Director3d: Real-world camera trajectory and 3d scene generation from text.Advances in neural information processing systems, 37:75125–75151, 2024. 11

    2024

  17. [17]

    Dl3dv-10k: A large-scale scene dataset for deep learning-based 3d vision

    Lu Ling, Yichen Sheng, Zhi Tu, Wentian Zhao, Cheng Xin, Kun Wan, Lantao Yu, Qianyu Guo, Zixun Yu, Yawen Lu, et al. Dl3dv-10k: A large-scale scene dataset for deep learning-based 3d vision. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22160–22169, 2024

    2024

  18. [18]

    Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow matching for generative modeling. InThe Eleventh International Conference on Learning Representations, 2023. URLhttps://openreview.net/forum?id=PqvMRDCJT9t

    2023

  19. [19]

    Edit3r: Instant 3d scene editing from sparse unposed images.arXiv preprint arXiv:2512.25071, 2025

    Jiageng Liu, Weijie Lyu, Xueting Li, Yejie Guo, and Ming-Hsuan Yang. Edit3r: Instant 3d scene editing from sparse unposed images.arXiv preprint arXiv:2512.25071, 2025

    work page arXiv 2025

  20. [20]

    Nerf: Representing scenes as neural radiance fields for view synthesis.Communications of the ACM, 65 (1):99–106, 2021

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis.Communications of the ACM, 65 (1):99–106, 2021

    2021

  21. [21]

    Spin-nerf: Multiview segmentation and perceptual inpainting with neural radiance fields

    Ashkan Mirzaei, Tristan Aumentado-Armstrong, Konstantinos G Derpanis, Jonathan Kelly, Marcus A Brubaker, Igor Gilitschenski, and Alex Levinshtein. Spin-nerf: Multiview segmentation and perceptual inpainting with neural radiance fields. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20669–20679, 2023

    2023

  22. [22]

    Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, and Ying Shan. T2I-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models.Proceedings of the AAAI Conference on Artificial Intelligence, 38(5):4296–4304, 2024. doi: 10.1609/aaai.v38i5.28226

    work page doi:10.1609/aaai.v38i5.28226 2024

  23. [23]

    A benchmark dataset and evaluation methodology for video object segmentation

    Federico Perazzi, Jordi Pont-Tuset, Brian McWilliams, Luc Van Gool, Markus Gross, and Alexander Sorkine-Hornung. A benchmark dataset and evaluation methodology for video object segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 724–732, 2016

    2016

  24. [24]

    Editcast3d: Single-frame-guided 3d editing with video propagation and view selection

    Huaizhi Qu, Ruichen Zhang, Shuqing Luo, Luchao Qi, Zhihao Zhang, Xiaoming Liu, Roni Sengupta, and Tianlong Chen. Editcast3d: Single-frame-guided 3d editing with video propagation and view selection. arXiv preprint arXiv:2510.13652, 2025

    work page arXiv 2025

  25. [25]

    Common objects in 3d: Large-scale learning and evaluation of real-life 3d category reconstruction

    Jeremy Reizenstein, Roman Shapovalov, Philipp Henzler, Luca Sbordone, Patrick Labatut, and David Novotny. Common objects in 3d: Large-scale learning and evaluation of real-life 3d category reconstruction. InProceedings of the IEEE/CVF international conference on computer vision, pages 10901–10911, 2021

    2021

  26. [26]

    Gen3c: 3d-informed world-consistent video generation with precise camera control

    Xuanchi Ren, Tianchang Shen, Jiahui Huang, Huan Ling, Yifan Lu, Merlin Nimier-David, Thomas Müller, Alexander Keller, Sanja Fidler, and Jun Gao. Gen3c: 3d-informed world-consistent video generation with precise camera control. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6121–6132, 2025

    2025

  27. [27]

    Hypersim: A photorealistic synthetic dataset for holistic indoor scene understanding

    Mike Roberts, Jason Ramapuram, Anurag Ranjan, Atulit Kumar, Miguel Angel Bautista, Nathan Paczan, Russ Webb, and Joshua M Susskind. Hypersim: A photorealistic synthetic dataset for holistic indoor scene understanding. InProceedings of the IEEE/CVF international conference on computer vision, pages 10912–10922, 2021

    2021

  28. [28]

    The Replica Dataset: A Digital Replica of Indoor Spaces

    Julian Straub, Thomas Whelan, Lingni Ma, Yufan Chen, Erik Wijmans, Simon Green, Jakob J Engel, Raul Mur-Artal, Carl Ren, Shobhit Verma, et al. The replica dataset: A digital replica of indoor spaces.arXiv preprint arXiv:1906.05797, 2019

    work page internal anchor Pith review Pith/arXiv arXiv 1906

  29. [29]

    Wan: Open and Advanced Large-Scale Video Generative Models

    Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, et al. Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  30. [30]

    Vggt: Visual geometry grounded transformer

    Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. Vggt: Visual geometry grounded transformer. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 5294–5306, 2025

    2025

  31. [31]

    Learning 3d geometry and feature consistent gaussian splatting for object removal

    Yuxin Wang, Qianyi Wu, Guofeng Zhang, and Dan Xu. Learning 3d geometry and feature consistent gaussian splatting for object removal. InEuropean conference on computer vision, pages 1–17. Springer, 2024

    2024

  32. [32]

    Aurafusion360: Augmented unseen region alignment for reference-based 360deg unbounded scene inpainting

    Chung-Ho Wu, Yang-Jung Chen, Ying-Huan Chen, Jie-Ying Lee, Bo-Hsu Ke, Chun-Wei Tuan Mu, Yi- Chuan Huang, Chin-Yang Lin, Min-Hung Chen, Yen-Yu Lin, et al. Aurafusion360: Augmented unseen region alignment for reference-based 360deg unbounded scene inpainting. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 16366–16376, 2025. 12

    2025

  33. [33]

    Geometry forcing: Marrying video diffusion and 3d representation for consistent world modeling

    Haoyu Wu, Diankun Wu, Tianyu He, Junliang Guo, Yang Ye, Yueqi Duan, and Jiang Bian. Geometry forcing: Marrying video diffusion and 3d representation for consistent world modeling. InThe Fourteenth International Conference on Learning Representations, 2026. URL https://openreview.net/forum? id=ULXYZCms41

    2026

  34. [34]

    Gaussctrl: Multi-view consistent text-driven 3d gaussian splatting editing

    Jing Wu, Jia-Wang Bian, Xinghui Li, Guangrun Wang, Ian Reid, Philip Torr, and Victor Adrian Prisacariu. Gaussctrl: Multi-view consistent text-driven 3d gaussian splatting editing. InEuropean conference on computer vision, pages 55–71. Springer, 2024

    2024

  35. [35]

    Structured 3d latents for scalable and versatile 3d generation

    Jianfeng Xiang, Zelong Lv, Sicheng Xu, Yu Deng, Ruicheng Wang, Bowen Zhang, Dong Chen, Xin Tong, and Jiaolong Yang. Structured 3d latents for scalable and versatile 3d generation. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 21469–21480, 2025

    2025

  36. [36]

    Prometheus: 3d-aware latent diffusion models for feed-forward text-to-3d scene generation

    Yuanbo Yang, Jiahao Shao, Xinyang Li, Yujun Shen, Andreas Geiger, and Yiyi Liao. Prometheus: 3d-aware latent diffusion models for feed-forward text-to-3d scene generation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2857–2869, 2025

    2025

  37. [37]

    IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models

    Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models.arXiv preprint arXiv:2308.06721, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  38. [38]

    Gaussian grouping: Segment and edit anything in 3d scenes

    Mingqiao Ye, Martin Danelljan, Fisher Yu, and Lei Ke. Gaussian grouping: Segment and edit anything in 3d scenes. InEuropean conference on computer vision, pages 162–179. Springer, 2024

    2024

  39. [39]

    Scannet++: A high-fidelity dataset of 3d indoor scenes

    Chandan Yeshwanth, Yueh-Cheng Liu, Matthias Nießner, and Angela Dai. Scannet++: A high-fidelity dataset of 3d indoor scenes. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 12–22, 2023

    2023

  40. [40]

    Magicbrush: A manually annotated dataset for instruction-guided image editing.Advances in Neural Information Processing Systems, 36:31428–31449, 2023

    Kai Zhang, Lingbo Mo, Wenhu Chen, Huan Sun, and Yu Su. Magicbrush: A manually annotated dataset for instruction-guided image editing.Advances in Neural Information Processing Systems, 36:31428–31449, 2023

    2023

  41. [41]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. InProceedings of the IEEE/CVF international conference on computer vision, pages 3836–3847, 2023

    2023

  42. [42]

    3ditscene: Editing any scene via language-guided disentangled gaussian splatting

    Qihang Zhang, Yinghao Xu, Chaoyang Wang, Hsin-Ying Lee, Gordon Wetzstein, Bolei Zhou, and Ceyuan Yang. 3ditscene: Editing any scene via language-guided disentangled gaussian splatting. InThe Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum? id=iKDbLpVgQc

    2025

  43. [43]

    World-consistent video diffusion with explicit 3d modeling

    Qihang Zhang, Shuangfei Zhai, Miguel Angel Bautista Martin, Kevin Miao, Alexander Toshev, Joshua Susskind, and Jiatao Gu. World-consistent video diffusion with explicit 3d modeling. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 21685–21695, 2025

    2025

  44. [44]

    Stable virtual camera: Generative view synthesis with diffusion models

    Jensen Zhou, Hang Gao, Vikram V oleti, Aaryaman Vasishta, Chun-Han Yao, Mark Boss, Philip Torr, Christian Rupprecht, and Varun Jampani. Stable virtual camera: Generative view synthesis with diffusion models. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 12405– 12414, 2025

    2025

  45. [45]

    Imaginarium: Vision-guided high-quality 3d scene layout generation

    Xiaoming Zhu, Xu Huang, Qinghongbing Xie, Zhi Deng, Junsheng Yu, Yirui Guan, Zhongyuan Liu, Lin Zhu, Qijun Zhao, Ligang Liu, et al. Imaginarium: Vision-guided high-quality 3d scene layout generation. ACM Transactions on Graphics (TOG), 44(6):1–24, 2025

    2025

  46. [46]

    Dreameditor: Text-driven 3d scene editing with neural fields

    Jingyu Zhuang, Chen Wang, Liang Lin, Lingjie Liu, and Guanbin Li. Dreameditor: Text-driven 3d scene editing with neural fields. InSIGGRAPH Asia 2023 conference papers, pages 1–10, 2023

    2023

  47. [47]

    tasks”: [{“operation

    Jingyu Zhuang, Di Kang, Yan-Pei Cao, Guanbin Li, Liang Lin, and Ying Shan. Tip-editor: An accurate 3d editor following both text-prompts and image-prompts.ACM Transactions on Graphics (ToG), 43(4):1–12, 2024. 13 Source Image Edited Image RGB Latent RGB Mask RGB Region GEO Latent GEO Mask GEO Region Figure 7: Training-time response diagnostic for region-de...

    2024

  48. [48]

    Parse the JSON layout file L and scene metadata M to recover object identities, support relations, parent-child links, approximate object extents, and navigable room bounds

  49. [49]

    Resolve affected object roots from q, expand them with the parent-child graph, and run layout validity checks to reject structural, too-small, or unsupported targets

  50. [50]

    For relocation, sample candidate target positions and run collision detection, support-surface checks, room-bound checks, and free-space validation before accepting a placement

  51. [51]

    Generate a seeded camera trajectory P={(R i, ti)}N i=1 from fixed templates such as horizontal sweeps, diagonal motions, rising motions, and drop-forward motions, centered on the affected object set

  52. [52]

    Run visibility checks along P for the source and edited states; reject or repair tasks whose targets are not sufficiently visible or whose edit is hidden by accidental occlusion

  53. [53]

    Save the initial object states and camera state so both branches start from the same scene

  54. [54]

    For each branch b∈ {before,after} : restore the saved scene state, apply the branch-specific edit operatorE b q, attach the same camera pathP, and render RGB/depth frames

  55. [55]

    Render binary masks through Blender object-index passes

    Select branch-specific mask objects: deleted objects for deletion, moved objects in both relocation branches, and appearance-changed objects in both appearance branches. Render binary masks through Blender object-index passes

  56. [56]

    Export RGB videos, depth maps, masks, relative camera poses, intrinsics, task JSON, and completion sentinels

  57. [57]

    Derive inverse samples for deletion and relocation by swapping before/after branches and reversing frame, depth, mask, and pose order: delete→add and move→move-reverse

  58. [58]

    remove object chair_12

    Discard samples with missing files, empty masks, invisible targets, corrupted frames, invalid physical placement, or failed collision/visibility checks. Eb q(S0) =    hide(o), q= delete, b= after, move(o,∆), q= move, b= after, replace_mat(o, m′), q= appearance, b= after, identity,otherwise. Figure 15: Algorithmic view of the Blender execution pro...