REVIEW 4 major objections 4 minor 1 cited by
HiScene: Creating Hierarchical 3D Scenes with Isometric View Generation
T0 review · 4 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash
Pith's one-line read HiScene's central claim is that a full 3D scene can be created from a text prompt by generating the whole room as one isometric object, then decomposing it and regenerating every item as an independent, aligned 3D instance.
desk verdict Clever isometric-view pipeline with a strong amodal completion module, but the headline claim of separable complete objects is never directly measured and the scene-initialization keystone is unvalidated. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing representation is the isometric view taken as a scene-as-object projection: it makes a room behave like a single object to an object-centric 3D generator. Around that, the machinery is a three-stage pipeline: (1) TRELLIS native 3D generation creates an initial 3D Gaussian Splatting scene from the isometric image; (2) OmniSeg3D-GS contrastive segmentation, fed by EntitySeg instance masks, separates objects, and object-centric circular renders are screened by a vision-language model for occlusion; (3) a fine-tuned Stable Video Diffusion model performs amodal completion as a video transition, and the completed views go through sparse-view LRM whose voxelized geometry is encoded by TRELLIS's structure encoder and injected as a noisy latent initialization at intermediate timestep $t\in[0.2,0.4]$, anchoring regeneration to the original placement.
What would settle it
Take a set of isometric room images with known ground-truth object instances and run TRELLIS scene initialization followed by OmniSeg3D-GS with EntitySeg masks; if overlapping objects such as chairs or table and pillows merge into single Gaussian groups and cannot be separated into the known instances, the decomposition-and-regeneration cascade fails at its first step.
Extended reading notes
Core claim
The paper's central discovery is that a full 3D scene can be generated compositionally by treating a room as a hierarchical object under an isometric view. The room is first generated as a whole by a native object-level 3D generation model, then parsed into instances through 3D semantic segmentation, each instance is completed with a video-diffusion model that treats occlusion removal as a temporal dissolve transition, and finally each object is regenerated with a shape prior so it keeps the original scene's pose and scale. The authors report that this pipeline yields scenes with natural layouts, complete compositional instances, and physical plausibility, and outperforms LLM-layout baselines on aesthetic score, image-text alignment, and user preference.
Load-bearing premise
The single most load-bearing premise is that the object-centric native 3D generator TRELLIS, trained on single objects, also produces a clean, parseable 3D Gaussian scene when fed a room-level isometric image; the paper invokes this at scene initialization without validation at scene scale.
Editorial extensions
If this is right
- Users can generate editable 3D scenes from text without specifying a layout, bypassing LLM spatial planning and handcrafted placement rules.
- Each object in the generated scene is produced as a complete, separable 3D entity, enabling individual manipulation, replacement, and simulation without disturbing the rest of the room.
- Amodal completion formulated as a temporal video transition recovers occluded object shape and removes occlusion shadows better than static image inpainting or single-image diffusion, as measured on the COCO-A and BSDS-A benchmarks.
- Spatial shape prior injection keeps regenerated objects aligned with the original scene, avoiding the canonical-orientation drift and Janus artifacts that appear when native 3D regeneration runs without alignment.
- The full pipeline runs in about 12 minutes per scene, compared with one to two hours for SDS-optimization-based baselines, making interactive iteration practical.
Reading between the lines
- The same hierarchy could be applied recursively, so a generated object that itself contains smaller objects could be parsed and regenerated as a scene within a scene, provided the underlying native 3D generator can represent that level of detail.
- The video-transition framing of amodal completion is not tied to furniture; the same dissolve-occlusion training recipe could be applied to people, vehicles, or dense vegetation by constructing shadow-aware video pairs for those categories.
- Because the shape prior is injected as latent initialization rather than a model-specific constraint, the alignment mechanism could be ported to other native 3D generators beyond TRELLIS, which would make the pipeline more modular.
- A testable extension is to add PBR material estimation into the regeneration stage, directly addressing the paper's own stated limitation of baked lighting and opening the output to standard rendering pipelines.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes HiScene, a text-to-3D-scene generation system that treats a scene as a hierarchy of objects under isometric views. The pipeline first generates an isometric scene image from text, lifts it to a 3D Gaussian Splatting scene via the object-centric native 3D generator TRELLIS, and then parses the scene into instances using EntitySeg and OmniSeg3D-GS. Each occluded instance is completed with a video-diffusion-based amodal completion model trained on a new 1.32M-pair dataset that includes 468K shadow-aware synthetic pairs, and the completed instance is regenerated with a shape-prior injection from sparse-view LRM to preserve spatial alignment. The method is evaluated on 12 scenes with CLIP/ImageReward/Aesthetic scores and a 20-user study, and the amodal completion model is evaluated on COCO-A and BSDS-A. The paper also reports ablations on the video model, shadow data, and spatial alignment, and gives a runtime of about 12 minutes per scene.
Significance. If the central claim holds—that users can obtain complete, separable, spatially aligned 3D objects inside an editable scene—HiScene would be a practically valuable bridge between 2D text-to-image generation and 3D scene editing. The paper contributes a large amodal-completion dataset with shadow effects, a SVD-based video-diffusion completion model, and a shape-prior injection mechanism that enables regeneration with preserved alignment. The qualitative results are visually appealing, and the runtime advantage over SDS-based baselines is compelling. However, the current evaluation does not directly measure the paper's core promise of compositional completeness in 3D: scene-level scores are computed on 2D renderings of 12 scenes, and the amodal module is validated on 2D datasets. The keystone assumption that TRELLIS generalizes from single objects to full-room isometric views is asserted without scene-level validation. These are significant gaps for a journal submission, though they appear addressable with additional experiments.
major comments (4)
- [Section 3.2, Scene Initialization] The first stage feeds a full-room isometric image into TRELLIS, a model trained on object-centric assets, and the manuscript supplies no scene-level validation of the resulting 3DGS scene S0. Since the subsequent EntitySeg/OmniSeg3D-GS parsing operates on S0, any instance that is fused, truncated, or omitted at this stage cannot be recovered by the later amodal completion and shape-prior stages; the decomposition-and-regeneration cascade inherits every failure of initialization. The paper should report, on a modest set of scenes, instance-level parsing quality—for example, the number of detected instances versus the expected count, per-instance 3D overlap against manually annotated Gaussians, or at least a failure-case analysis—to demonstrate that S0 is parseable into cleanly separated objects.
- [Section 4.2, Table 2] The amodal completion model is validated on 2D datasets (COCO-A, BSDS-A) with mIoU on amodal masks, but the paper's central claim concerns 3D instance completeness: each object in the generated scene should be an intact, manipulatable 3D entity. Table 1 only reports 2D-rendered aesthetic/CLIP/ImageReward scores, and the user study asks about matching degree and overall quality, not about object completeness. The authors should directly measure per-instance 3D completeness—for instance, by comparing the regenerated object's geometry with the initially visible region, by measuring the fraction of instances whose regenerated 3DGS covers the amodal mask footprint across multiple views, or by asking users to judge whether each object remains intact under rotation.
- [Section 4.1, Table 1] The comparison to GALA3D and DreamScene is under-specified: the paper states that both baselines require predefined 3D layouts, but it does not describe how these layouts were provided (e.g., from an LLM, from the same text, or from a human). If the baselines were run without their expected layout input, the comparison is biased; if layouts were supplied, the protocol should be stated. Additionally, all results derive from only 12 scenes, and no per-scene breakdown or statistical significance is reported. Given the near-ceiling user scores (2.90 out of 3.00), the advantage over baselines should be supported with confidence intervals or per-item analysis.
- [Section 3.4 and Figure 9] The spatial alignment claim rests on a purely qualitative ablation (Figure 9). Spatial alignment is a quantitative geometric property; the paper should report a metric such as Chamfer distance, pose error, or scale error between the regenerated instance and its original placement in S0, averaged over scenes. The hand-selected timestep range t in [0.2, 0.4] is a design choice, and its sensitivity should be examined with at least a small sweep to show the full method is not fragile to this choice.
minor comments (4)
- [Supplementary B.3] The sentence containing 'thn illustration of the use' should read 'an illustration of the use'; please fix this typo.
- [References [24] and [25]] References [24] and [25] cite the same SAM paper (Kirillov et al., 2023) and should be merged into a single reference to avoid duplication and confusion.
- [Equation (1)] The notation in Equation (1) does not define the ranges of L, C, and N; please add explicit definitions for these quantities in the text.
- [Table 3] The table header 'Datasets & Method COCO-A BSDS-A I2I I2V I2I I2V' is difficult to parse; the caption should clarify that I2I and I2V refer to image-model and video-model variants, respectively, and that COCO-A and BSDS-A are the two evaluation datasets.
Circularity Check
No circularity found: HiScene's modules are trained on independently constructed data and evaluated on external benchmarks, and no derivation step reduces to its own inputs.
full rationale
The paper's derivation chain is self-contained against external evidence. The amodal completion model is trained on a newly constructed dataset (Objaverse-filtered synthetic shadow data plus Pix2gestalt-derived pairs, Sec. 3.3 and Supp. B) and evaluated on the held-out external benchmarks COCO-A and BSDS-A (Sec. 4.2, Table 2), where the reported mIoU values are measured outputs rather than fitted inputs. The video-model-versus-image-model ablation (Sec. 4.3, Table 3) compares two models under identical data settings using independent quality metrics, so its conclusion is not baked in by construction. Scene initialization (Sec. 3.2) does invoke TRELLIS on room-level isometric views, and this is an unvalidated distributional assumption that could undermine the method, but that is a correctness or robustness risk, not circularity: the paper does not define any quantity in terms of the claimed scene-quality result, and no equation reduces to its own input. The spatial alignment stage (Sec. 3.4) uses LRM reconstruction as a conditioning prior rather than as the predicted quantity; the regenerated object is produced by the generative model with the prior as initialization, so the output is not equivalent to the input. The Limitations section (Sec. 5) states that textures have baked lighting and lack PBR materials, which is an acknowledged scope limitation, not a circular step. There are no load-bearing self-citations: the external systems used (TRELLIS, InstantMesh, OmniSeg3D-GS, SVD) are cited as prior work, not as the present authors' own unverified results, and the central claims are supported by external benchmarks, user studies, and ablations. Consequently, the appropriate circularity score is 0.
Assumptions & free parameters
free parameters (2)
- Occlusion ratio threshold τ =
0.4
- Shape prior injection timestep t =
[0.2, 0.4]
assumptions (4)
- ad hoc to paper A full-room isometric view behaves like an object for TRELLIS, enabling scene-as-object generation.
- ad hoc to paper LRM geometry latents and TRELLIS structure latents are compatible enough that initializing TRELLIS sampling at t in [0.2, 0.4] with an LRM-derived voxel latent yields spatially aligned objects.
- domain assumption VLM-based occlusion detection and ratio estimation are sufficiently accurate for automatic pipeline decisions.
- domain assumption Synthetic shadows from Blender/Objaverse transfer to real occlusion scenarios.
Cite this review
Pith. "Pith review of HiScene: Creating Hierarchical 3D Scenes with Isometric View Generation." pith.science (2026). https://pith.science/paper/73K3HPC7
@misc{pith2026250413072,
author = {Pith},
title = {Pith review of: HiScene: Creating Hierarchical 3D Scenes with Isometric View Generation},
year = {2026},
howpublished = {\url{https://pith.science/paper/73K3HPC7}},
note = {Machine review of arXiv:2504.13072}
}
read the original abstract
Scene-level 3D generation represents a critical frontier in multimedia and computer graphics, yet existing approaches either suffer from limited object categories or lack editing flexibility for interactive applications. In this paper, we present HiScene, a novel hierarchical framework that bridges the gap between 2D image generation and 3D object generation and delivers high-fidelity scenes with compositional identities and aesthetic scene content. Our key insight is treating scenes as hierarchical "objects" under isometric views, where a room functions as a complex object that can be further decomposed into manipulatable items. This hierarchical approach enables us to generate 3D content that aligns with 2D representations while maintaining compositional structure. To ensure completeness and spatial alignment of each decomposed instance, we develop a video-diffusion-based amodal completion technique that effectively handles occlusions and shadows between objects, and introduce shape prior injection to ensure spatial coherence within the scene. Experimental results demonstrate that our method produces more natural object arrangements and complete object instances suitable for interactive applications, while maintaining physical plausibility and alignment with user inputs.
Figures
Figures from the paper (12 more)
Forward citations
Cited by 1 Pith paper
-
3D Scene Generation: A Survey
The paper surveys 3D scene generation and organizes methods into four paradigms, with datasets, evaluation metrics, applications, and future directions.
Reference graph
Works this paper leans on
-
[1]
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023. 14
arXiv 2023
-
[2]
Blended diffusion for text-driven editing of natural images
Omri Avrahami, Dani Lischinski, and Ohad Fried. Blended diffusion for text-driven editing of natural images. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 18208–18218, 2022. 3
2022
-
[3]
Blended latent diffusion
Omri Avrahami, Ohad Fried, and Dani Lischinski. Blended latent diffusion. ACM transactions on graphics (TOG), 42(4): 1–11, 2023. 3
2023
-
[4]
Iron- depth: Iterative refinement of single-view depth using surface normal and its uncertainty
Gwangbin Bae, Ignas Budvytis, and Roberto Cipolla. Iron- depth: Iterative refinement of single-view depth using surface normal and its uncertainty. arXiv preprint arXiv:2210.03676,
-
[5]
Image inpainting
Marcelo Bertalmio, Guillermo Sapiro, Vincent Caselles, and Coloma Ballester. Image inpainting. In Proceedings of the 27th annual conference on Computer graphics and interactive techniques, pages 417–424, 2000. 3
2000
-
[6]
Zoedepth: Zero-shot trans- fer by combining relative and metric depth
Shariq Farooq Bhat, Reiner Birkl, Diana Wofk, Peter Wonka, and Matthias M ¨uller. Zoedepth: Zero-shot trans- fer by combining relative and metric depth. arXiv preprint arXiv:2302.12288, 2023. 3
arXiv 2023
-
[7]
Stable video diffusion: Scaling latent video diffusion models to large datasets
Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023. 5, 16
arXiv 2023
-
[8]
Scenefactor: Factored latent 3d diffusion for control- lable 3d scene generation
Alexey Bokhovkin, Quan Meng, Shubham Tulsiani, and An- gela Dai. Scenefactor: Factored latent 3d diffusion for control- lable 3d scene generation. arXiv preprint arXiv:2412.01801,
Show all 94 references
-
[9]
Fan- tasia3d: Disentangling geometry and appearance for high- quality text-to-3d content creation
Rui Chen, Yongwei Chen, Ningxin Jiao, and Kui Jia. Fan- tasia3d: Disentangling geometry and appearance for high- quality text-to-3d content creation. In Proceedings of the IEEE/CVF international conference on computer vision , pages 22246–22256, 2023. 3
2023
-
[10]
Region-aware text-to-image generation via hard binding and soft refinement
Zhennan Chen, Yajie Li, Haofan Wang, Zhibo Chen, Zhengkai Jiang, Jun Li, Qian Wang, Jian Yang, and Ying Tai. Region-aware text-to-image generation via hard binding and soft refinement. arXiv preprint arXiv:2411.06558, 2024. 3
2024 arXiv
-
[11]
3dtopia-xl: Scaling high-quality 3d asset generation via primitive diffusion
Zhaoxi Chen, Jiaxiang Tang, Yuhao Dong, Ziang Cao, Fangzhou Hong, Yushi Lan, Tengfei Wang, Haozhe Xie, Tong Wu, Shunsuke Saito, et al. 3dtopia-xl: Scaling high-quality 3d asset generation via primitive diffusion. arXiv preprint arXiv:2409.12957, 2024. 3
2024 arXiv
-
[12]
Click-gaussian: Interactive segmenta- tion to any 3d gaussians
Seokhun Choi, Hyeonseop Song, Jaechul Kim, Taehyeong Kim, and Hoseok Do. Click-gaussian: Interactive segmenta- tion to any 3d gaussians. In European Conference on Com- puter Vision, pages 289–305. Springer, 2024. 14
2024
-
[13]
Luciddreamer: Domain-free gen- eration of 3d gaussian splatting scenes
Jaeyoung Chung, Suyoung Lee, Hyeongjin Nam, Jaerin Lee, and Kyoung Mu Lee. Luciddreamer: Domain-free gen- eration of 3d gaussian splatting scenes. arXiv preprint arXiv:2311.13384, 2023. 2, 3
2023 arXiv
-
[14]
Re- gion filling and object removal by exemplar-based image inpainting
Antonio Criminisi, Patrick P´erez, and Kentaro Toyama. Re- gion filling and object removal by exemplar-based image inpainting. IEEE Transactions on image processing, 13(9): 1200–1212, 2004. 3
2004
-
[15]
Objaverse: A universe of annotated 3d objects
Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated 3d objects. In Proceedings of the IEEE/CVF conference on computer vision and pattern rec...
2023
-
[16]
Text2room: Extracting textured 3d meshes from 2d text-to-image models
Lukas H ¨ollein, Ang Cao, Andrew Owens, Justin Johnson, and Matthias Nießner. Text2room: Extracting textured 3d meshes from 2d text-to-image models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7909–7920, 2023. 2, 3
2023
-
[17]
Lrm: Large reconstruction model for single image to 3d
Yicong Hong, Kai Zhang, Jiuxiang Gu, Sai Bi, Yang Zhou, Difan Liu, Feng Liu, Kalyan Sunkavalli, Trung Bui, and Hao Tan. Lrm: Large reconstruction model for single image to 3d. arXiv preprint arXiv:2311.04400, 2023. 3
2023 arXiv
-
[18]
Tracking any object amodally
Cheng-Yen Hsieh, Tarasha Khurana, Achal Dave, and Deva Ramanan. Tracking any object amodally. CoRR, 2023. 3
2023
-
[19]
Smarteraser: Remove anything from images using masked- region guidance
Longtao Jiang, Zhendong Wang, Jianmin Bao, Wengang Zhou, Dongdong Chen, Lei Shi, Dong Chen, and Houqiang Li. Smarteraser: Remove anything from images using masked- region guidance. arXiv preprint arXiv:2501.08279 , 2025. 3
2025 arXiv
-
[20]
Brushnet: A plug-and-play image inpainting model with decomposed dual-branch diffusion
Xuan Ju, Xian Liu, Xintao Wang, Yuxuan Bian, Ying Shan, and Qiang Xu. Brushnet: A plug-and-play image inpainting model with decomposed dual-branch diffusion. In European Conference on Computer Vision, pages 150–168. Springer,
-
[21]
Amodal completion and size constancy in natural scenes
Abhishek Kar, Shubham Tulsiani, Joao Carreira, and Jitendra Malik. Amodal completion and size constancy in natural scenes. In Proceedings of the IEEE international conference on computer vision, pages 127–135, 2015. 3
2015
-
[22]
Deep occlusion- aware instance segmentation with overlapping bilayers
Lei Ke, Yu-Wing Tai, and Chi-Keung Tang. Deep occlusion- aware instance segmentation with overlapping bilayers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4019–4028, 2021. 3
2021
-
[23]
3d gaussian splatting for real-time radiance field rendering
Bernhard Kerbl, Georgios Kopanas, Thomas Leimk¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 42(4):139–1, 2023. 2
2023
-
[24]
Segment any- thing
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. In Proceedings of the IEEE/CVF international confer- ence on computer vision, pages 4015–4026, 2023. 5, 15
2023
-
[25]
Segment any- thing
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. In Proceedings of the IEEE/CVF international confer- ence on computer vision, pages 4015–4026, 2023. 5, 7
2023
-
[26]
Open3dsg: Open-vocabulary 3d scene graphs from point clouds with queryable objects and open-set relationships
Sebastian Koch, Narunas Vaskevicius, Mirco Colosi, Pedro Hermosilla, and Timo Ropinski. Open3dsg: Open-vocabulary 3d scene graphs from point clouds with queryable objects and open-set relationships. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog...
2024
-
[27]
Black Forest Labs. Flux. https://github.com/ black-forest-labs/flux, 2024. 17
2024
-
[28]
Gaussiananything: Interactive point cloud latent diffusion for 3d generation
Yushi Lan, Shangchen Zhou, Zhaoyang Lyu, Fangzhou Hong, Shuai Yang, Bo Dai, Xingang Pan, and Chen Change Loy. Gaussiananything: Interactive point cloud latent diffusion for 3d generation. arXiv preprint arXiv:2411.08033, 2024. 3
2024 arXiv
-
[29]
Dreamscene: 3d gaussian-based text-to-3d scene generation via formation pattern sampling
Haoran Li, Haolin Shi, Wenli Zhang, Wenjun Wu, Yong Liao, Lin Wang, Lik-hang Lee, and Peng Yuan Zhou. Dreamscene: 3d gaussian-based text-to-3d scene generation via formation pattern sampling. In European Conference on Computer Vision, pages 214–230. Springer, 2024. 3, 6, 7
2024
-
[30]
Prototypical contrastive learning of unsupervised representa- tions
Junnan Li, Pan Zhou, Caiming Xiong, and Steven CH Hoi. Prototypical contrastive learning of unsupervised representa- tions. arXiv preprint arXiv:2005.04966, 2020. 14
2005 arXiv
-
[31]
Instant3d: Fast text-to-3d with sparse-view generation and large reconstruction model
Jiahao Li, Hao Tan, Kai Zhang, Zexiang Xu, Fujun Luan, Yinghao Xu, Yicong Hong, Kalyan Sunkavalli, Greg Shakhnarovich, and Sai Bi. Instant3d: Fast text-to-3d with sparse-view generation and large reconstruction model. arXiv preprint arXiv:2311.06214, 2023. 3
2023 arXiv
-
[32]
Craftsman: High-fidelity mesh generation with 3d native generation and interactive geometry refiner
Weiyu Li, Jiarui Liu, Rui Chen, Yixun Liang, Xuelin Chen, Ping Tan, and Xiaoxiao Long. Craftsman: High-fidelity mesh generation with 3d native generation and interactive geometry refiner. arXiv preprint arXiv:2405.14979, 2024. 3
2024 arXiv
-
[33]
Discene: Object decoupling and interaction modeling for complex scene generation
Xiao-Lei Li, Haodong Li, Hao-Xiang Chen, Tai-Jiang Mu, and Shi-Min Hu. Discene: Object decoupling and interaction modeling for complex scene generation. In SIGGRAPH Asia 2024 Conference Papers, pages 1–12, 2024. 3
2024
-
[34]
Luciddreamer: Towards high-fidelity text-to-3d generation via interval score matching
Yixun Liang, Xin Yang, Jiantao Lin, Haodong Li, Xiaogang Xu, and Yingcong Chen. Luciddreamer: Towards high-fidelity text-to-3d generation via interval score matching. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6517–6526, 2024. 3
2024
-
[35]
Instructscene: Instruction- driven 3d indoor scene synthesis with semantic graph prior
Chenguo Lin and Yadong Mu. Instructscene: Instruction- driven 3d indoor scene synthesis with semantic graph prior. arXiv preprint arXiv:2402.04717, 2024. 3
2024 arXiv
-
[36]
Magic3d: High-resolution text- to-3d content creation
Chen-Hsuan Lin, Jun Gao, Luming Tang, Towaki Takikawa, Xiaohui Zeng, Xun Huang, Karsten Kreis, Sanja Fidler, Ming- Yu Liu, and Tsung-Yi Lin. Magic3d: High-resolution text- to-3d content creation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognit...
2023
-
[37]
Variational amodal object completion
Huan Ling, David Acuna, Karsten Kreis, Seung Wook Kim, and Sanja Fidler. Variational amodal object completion. Ad- vances in Neural Information Processing Systems, 33:16246– 16257, 2020. 3
2020
-
[38]
Image inpainting via tractable steering of diffusion models
Anji Liu, Mathias Niepert, and Guy Van den Broeck. Image inpainting via tractable steering of diffusion models. arXiv preprint arXiv:2401.03349, 2023. 3
2023 arXiv
-
[39]
Pd-gan: Probabilistic diverse gan for image inpainting
Hongyu Liu, Ziyu Wan, Wei Huang, Yibing Song, Xintong Han, and Jing Liao. Pd-gan: Probabilistic diverse gan for image inpainting. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 9371–9381, 2021. 3
2021
-
[40]
One-2-3-45++: Fast single image to 3d ob- jects with consistent multi-view generation and 3d diffusion
Minghua Liu, Ruoxi Shi, Linghao Chen, Zhuoyang Zhang, Chao Xu, Xinyue Wei, Hansheng Chen, Chong Zeng, Jiayuan Gu, and Hao Su. One-2-3-45++: Fast single image to 3d ob- jects with consistent multi-view generation and 3d diffusion. In Proceedings of the IEEE/CVF conference on co...
2024
-
[41]
Zero-1-to-3: Zero-shot one image to 3d object
Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tokmakov, Sergey Zakharov, and Carl V ondrick. Zero-1-to-3: Zero-shot one image to 3d object. In Proceedings of the IEEE/CVF international conference on computer vision , pages 9298– 9309, 2023. 3
2023
-
[42]
Syncdreamer: Gener- ating multiview-consistent images from a single-view image
Yuan Liu, Cheng Lin, Zijiao Zeng, Xiaoxiao Long, Lingjie Liu, Taku Komura, and Wenping Wang. Syncdreamer: Gener- ating multiview-consistent images from a single-view image. arXiv preprint arXiv:2309.03453, 2023. 3
2023 arXiv
-
[43]
Wonder3d: Single im- age to 3d using cross-domain diffusion
Xiaoxiao Long, Yuan-Chen Guo, Cheng Lin, Yuan Liu, Zhiyang Dou, Lingjie Liu, Yuexin Ma, Song-Hai Zhang, Marc Habermann, Christian Theobalt, et al. Wonder3d: Single im- age to 3d using cross-domain diffusion. In Proceedings of the IEEE/CVF conference on computer vision and patt...
2024
-
[44]
A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics
David Martin, Charless Fowlkes, Doron Tal, and Jitendra Malik. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In Proceedings eighth IEEE international conference on computer vision. IC...
2001
-
[45]
Lt3sd: Latent trees for 3d scene diffusion
Quan Meng, Lei Li, Matthias Nießner, and Angela Dai. Lt3sd: Latent trees for 3d scene diffusion. arXiv preprint arXiv:2409.08215, 2024. 3
2024 arXiv
-
[46]
Latent-nerf for shape-guided generation of 3d shapes and textures
Gal Metzer, Elad Richardson, Or Patashnik, Raja Giryes, and Daniel Cohen-Or. Latent-nerf for shape-guided generation of 3d shapes and textures. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12663–12673, 2023. 3
2023
-
[47]
Ava: A large-scale database for aesthetic visual analysis
Naila Murray, Luca Marchesotti, and Florent Perronnin. Ava: A large-scale database for aesthetic visual analysis. In 2012 IEEE conference on computer vision and pattern recognition, pages 2408–2415. IEEE, 2012. 7, 8
2012
-
[48]
Sceneteller: Language-to-3d scene generation
Bas ¸ak Melis ¨Ocal, Maxim Tatarchenko, Sezer Karao˘glu, and Theo Gevers. Sceneteller: Language-to-3d scene generation. In European Conference on Computer Vision, pages 362–378. Springer, 2024. 3
2024
-
[49]
pix2gestalt: Amodal segmentation by synthesizing wholes
Ege Ozguroglu, Ruoshi Liu, D´ıdac Sur´ıs, Dian Chen, Achal Dave, Pavel Tokmakov, and Carl V ondrick. pix2gestalt: Amodal segmentation by synthesizing wholes. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3931–3940. IEEE Computer So- ciety...
2024
-
[50]
Atiss: Autoregres- sive transformers for indoor scene synthesis
Despoina Paschalidou, Amlan Kar, Maria Shugrina, Karsten Kreis, Andreas Geiger, and Sanja Fidler. Atiss: Autoregres- sive transformers for indoor scene synthesis. Advances in Neural Information Processing Systems , 34:12013–12026,
-
[51]
Gener- ating diverse structure for image inpainting with hierarchical vq-vae
Jialun Peng, Dong Liu, Songcen Xu, and Houqiang Li. Gener- ating diverse structure for image inpainting with hierarchical vq-vae. In Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition , pages 10775–10784,
-
[52]
11 Sdxl: Improving latent diffusion models for high-resolution image synthesis
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M¨uller, Joe Penna, and Robin Rombach. 11 Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023. 7
2023 arXiv
-
[53]
Dreamfusion: Text-to-3d using 2d diffusion
Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Mildenhall. Dreamfusion: Text-to-3d using 2d diffusion. arXiv preprint arXiv:2209.14988, 2022. 3
2022 arXiv
-
[54]
Amodal instance segmentation with kins dataset
Lu Qi, Li Jiang, Shu Liu, Xiaoyong Shen, and Jiaya Jia. Amodal instance segmentation with kins dataset. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3014–3023, 2019. 3
2019
-
[55]
High-quality entity segmentation
Lu Qi, Jason Kuen, Weidong Guo, Tiancheng Shen, Jiuxiang Gu, Jiaya Jia, Zhe Lin, and Ming-Hsuan Yang. High-quality entity segmentation. arXiv preprint arXiv:2211.05776, 2022. 5, 14
2022 arXiv
-
[56]
Learning transferable visual models from natural language supervi- sion
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...
2021
-
[57]
Walt: Watch and learn 2d amodal representation from time-lapse imagery
N Dinesh Reddy, Robert Tamburo, and Srinivasa G Narasimhan. Walt: Watch and learn 2d amodal representation from time-lapse imagery. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 9356–9366, 2022. 3
2022
-
[58]
High-resolution image synthesis with latent diffusion models
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 3
2022
-
[59]
Zero123++: a single image to consistent multi-view diffusion base model
Ruoxi Shi, Hansheng Chen, Zhuoyang Zhang, Minghua Liu, Chao Xu, Xinyue Wei, Linghao Chen, Chong Zeng, and Hao Su. Zero123++: a single image to consistent multi-view diffusion base model. arXiv preprint arXiv:2310.15110, 2023. 3
2023 arXiv
-
[60]
Mvdream: Multi-view diffusion for 3d gener- ation
Yichun Shi, Peng Wang, Jianglong Ye, Mai Long, Kejie Li, and Xiao Yang. Mvdream: Multi-view diffusion for 3d gener- ation. arXiv preprint arXiv:2308.16512, 2023. 3
2023 arXiv
-
[61]
Dreamgaussian: Generative gaussian splatting for effi- cient 3d content creation
Jiaxiang Tang, Jiawei Ren, Hang Zhou, Ziwei Liu, and Gang Zeng. Dreamgaussian: Generative gaussian splatting for effi- cient 3d content creation. arXiv preprint arXiv:2309.16653,
-
[62]
Lgm: Large multi-view gaussian model for high-resolution 3d content creation
Jiaxiang Tang, Zhaoxi Chen, Xiaokang Chen, Tengfei Wang, Gang Zeng, and Ziwei Liu. Lgm: Large multi-view gaussian model for high-resolution 3d content creation. In European Conference on Computer Vision, pages 1–18. Springer, 2024. 3
2024
-
[63]
Diffuscene: Denoising diffusion models for generative indoor scene synthesis
Jiapeng Tang, Yinyu Nie, Lev Markhasin, Angela Dai, Justus Thies, and Matthias Nießner. Diffuscene: Denoising diffusion models for generative indoor scene synthesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 20507–20518, 2024. 3
2024
-
[64]
Emergent correspondence from image diffusion
Luming Tang, Menglin Jia, Qianqian Wang, Cheng Perng Phoo, and Bharath Hariharan. Emergent correspondence from image diffusion. Advances in Neural Information Processing Systems, 36:1363–1389, 2023. 3
2023
-
[65]
Score jacobian chaining: Lifting pre- trained 2d diffusion models for 3d generation
Haochen Wang, Xiaodan Du, Jiahao Li, Raymond A Yeh, and Greg Shakhnarovich. Score jacobian chaining: Lifting pre- trained 2d diffusion models for 3d generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12619–12629, 2023. 3
2023
-
[66]
Architect: Generating vivid and interactive 3d scenes with hierarchical 2d inpainting
Yian Wang, Xiaowen Qiu, Jiageng Liu, Zhehuan Chen, Jiting Cai, Yufei Wang, Tsun-Hsuan Johnson Wang, Zhou Xian, and Chuang Gan. Architect: Generating vivid and interactive 3d scenes with hierarchical 2d inpainting. Advances in Neural Information Processing Systems, 37:67575–676...
2024
-
[67]
Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distilla- tion
Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distilla- tion. Advances in Neural Information Processing Systems , 36:8406–8441, 2023. 3
2023
-
[68]
Q-align: Teaching lmms for vi- sual scoring via discrete text-defined levels
Haoning Wu, Zicheng Zhang, Weixia Zhang, Chaofeng Chen, Liang Liao, Chunyi Li, Yixuan Gao, Annan Wang, Erli Zhang, Wenxiu Sun, et al. Q-align: Teaching lmms for vi- sual scoring via discrete text-defined levels. arXiv preprint arXiv:2312.17090, 2023. 7, 8
2023 arXiv
-
[69]
Blockfusion: Expandable 3d scene gener- ation using latent tri-plane extrapolation
Zhennan Wu, Yang Li, Han Yan, Taizhang Shang, Weixuan Sun, Senbo Wang, Ruikai Cui, Weizhe Liu, Hiroyuki Sato, Hongdong Li, et al. Blockfusion: Expandable 3d scene gener- ation using latent tri-plane extrapolation. ACM Transactions on Graphics (TOG), 43(4):1–17, 2024. 3
2024
-
[70]
Structured 3d latents for scalable and versatile 3d generation
Jianfeng Xiang, Zelong Lv, Sicheng Xu, Yu Deng, Ruicheng Wang, Bowen Zhang, Dong Chen, Xin Tong, and Jiaolong Yang. Structured 3d latents for scalable and versatile 3d generation. arXiv preprint arXiv:2412.01506, 2024. 2, 3, 4, 6
2024 arXiv
-
[71]
Sparp: Fast 3d object reconstruction and pose estimation from sparse views
Chao Xu, Ang Li, Linghao Chen, Yulin Liu, Ruoxi Shi, Hao Su, and Minghua Liu. Sparp: Fast 3d object reconstruction and pose estimation from sparse views. In European Con- ference on Computer Vision, pages 143–163. Springer, 2024. 3
2024
-
[72]
Instantmesh: Efficient 3d mesh generation from a single image with sparse-view large reconstruction models
Jiale Xu, Weihao Cheng, Yiming Gao, Xintao Wang, Shenghua Gao, and Ying Shan. Instantmesh: Efficient 3d mesh generation from a single image with sparse-view large reconstruction models. arXiv preprint arXiv:2404.07191 ,
-
[73]
Amodal com- pletion via progressive mixed context diffusion
Katherine Xu, Lingzhi Zhang, and Jianbo Shi. Amodal com- pletion via progressive mixed context diffusion. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9099–9109, 2024. 3
2024
-
[74]
Grm: Large gaussian reconstruction model for efficient 3d recon- struction and generation
Yinghao Xu, Zifan Shi, Wang Yifan, Hansheng Chen, Ceyuan Yang, Sida Peng, Yujun Shen, and Gordon Wetzstein. Grm: Large gaussian reconstruction model for efficient 3d recon- struction and generation. In European Conference on Com- puter Vision, pages 1–20. Springer, 2024. 3
2024
-
[75]
Scene synthesis via uncertainty-driven attribute syn- chronization
Haitao Yang, Zaiwei Zhang, Siming Yan, Haibin Huang, Chongyang Ma, Yi Zheng, Chandrajit Bajaj, and Qixing Huang. Scene synthesis via uncertainty-driven attribute syn- chronization. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5630–5640, 2021. 3
2021
-
[76]
Gaus- sian grouping: Segment and edit anything in 3d scenes
Mingqiao Ye, Martin Danelljan, Fisher Yu, and Lei Ke. Gaus- sian grouping: Segment and edit anything in 3d scenes. In European Conference on Computer Vision, pages 162–179. Springer, 2024. 14
2024
-
[77]
Gaussiandreamer: Fast generation from text to 3d gaus- sians by bridging 2d and 3d diffusion models
Taoran Yi, Jiemin Fang, Junjie Wang, Guanjun Wu, Lingxi Xie, Xiaopeng Zhang, Wenyu Liu, Qi Tian, and Xinggang Wang. Gaussiandreamer: Fast generation from text to 3d gaus- sians by bridging 2d and 3d diffusion models. In Proceedings 12 of the IEEE/CVF Conference on Computer Vis...
2024
-
[78]
Omniseg3d: Omniversal 3d segmentation via hierarchical contrastive learning
Haiyang Ying, Yixuan Yin, Jinzhi Zhang, Fan Wang, Tao Yu, Ruqi Huang, and Lu Fang. Omniseg3d: Omniversal 3d segmentation via hierarchical contrastive learning. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20612–20622, 2024. 2, 5, 14
2024
-
[79]
Wonderworld: Interactive 3d scene generation from a single image
Hong-Xing Yu, Haoyi Duan, Charles Herrmann, William T Freeman, and Jiajun Wu. Wonderworld: Interactive 3d scene generation from a single image. arXiv preprint arXiv:2406.09394, 2024. 3
2024 arXiv
-
[80]
Wonderjourney: Going from anywhere to everywhere
Hong-Xing Yu, Haoyi Duan, Junhwa Hur, Kyle Sargent, Michael Rubinstein, William T Freeman, Forrester Cole, De- qing Sun, Noah Snavely, Jiajun Wu, et al. Wonderjourney: Going from anywhere to everywhere. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern R...
2024
-
[81]
Commonscenes: Generating commonsense 3d indoor scenes with scene graph diffusion
Guangyao Zhai, Evin Pınar ¨Ornek, Shun-Cheng Wu, Yan Di, Federico Tombari, Nassir Navab, and Benjamin Busam. Commonscenes: Generating commonsense 3d indoor scenes with scene graph diffusion. Advances in Neural Information Processing Systems, 36:30026–30038, 2023. 3
2023
-
[82]
Echoscene: Indoor scene generation via information echo over scene graph diffusion
Guangyao Zhai, Evin Pınar ¨Ornek, Dave Zhenyu Chen, Ruo- tong Liao, Yan Di, Nassir Navab, Federico Tombari, and Benjamin Busam. Echoscene: Indoor scene generation via information echo over scene graph diffusion. In European Conference on Computer Vision, pages 167–184. Springer,
-
[83]
Amodal ground truth and completion in the wild
Guanqi Zhan, Chuanxia Zheng, Weidi Xie, and Andrew Zis- serman. Amodal ground truth and completion in the wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 28003–28013, 2024. 3
2024
-
[84]
Self-supervised scene de-occlusion
Xiaohang Zhan, Xingang Pan, Bo Dai, Ziwei Liu, Dahua Lin, and Chen Change Loy. Self-supervised scene de-occlusion. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3784–3792, 2020. 7
2020
-
[85]
3dshape2vecset: A 3d shape representation for neural fields and generative diffusion models
Biao Zhang, Jiapeng Tang, Matthias Niessner, and Peter Wonka. 3dshape2vecset: A 3d shape representation for neural fields and generative diffusion models. ACM Transactions On Graphics (TOG), 42(4):1–16, 2023. 3
2023
-
[86]
Text2nerf: Text-driven 3d scene generation with neu- ral radiance fields
Jingbo Zhang, Xiaoyu Li, Ziyu Wan, Can Wang, and Jing Liao. Text2nerf: Text-driven 3d scene generation with neu- ral radiance fields. IEEE Transactions on Visualization and Computer Graphics, 30(12):7749–7762, 2024. 3
2024
-
[87]
Clay: A controllable large-scale generative model for creating high- quality 3d assets
Longwen Zhang, Ziyu Wang, Qixuan Zhang, Qiwei Qiu, Anqi Pang, Haoran Jiang, Wei Yang, Lan Xu, and Jingyi Yu. Clay: A controllable large-scale generative model for creating high- quality 3d assets. ACM Transactions on Graphics (TOG), 43 (4):1–20, 2024. 3
2024
-
[88]
Towards text-guided 3d scene composition
Qihang Zhang, Chaoyang Wang, Aliaksandr Siarohin, Peiye Zhuang, Yinghao Xu, Ceyuan Yang, Dahua Lin, Bolei Zhou, Sergey Tulyakov, and Hsin-Ying Lee. Towards text-guided 3d scene composition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , ...
2024
-
[89]
Large scale image comple- tion via co-modulated generative adversarial networks
Shengyu Zhao, Jonathan Cui, Yilun Sheng, Yue Dong, Xiao Liang, Eric I Chang, and Yan Xu. Large scale image comple- tion via co-modulated generative adversarial networks. arXiv preprint arXiv:2103.10428, 2021. 3
2021 arXiv
-
[90]
Michelangelo: Conditional 3d shape generation based on shape-image-text aligned latent representation
Zibo Zhao, Wen Liu, Xin Chen, Xianfang Zeng, Rui Wang, Pei Cheng, Bin Fu, Tao Chen, Gang Yu, and Shenghua Gao. Michelangelo: Conditional 3d shape generation based on shape-image-text aligned latent representation. Advances in neural information processing systems, 36:73969–73982,
-
[91]
Hunyuan3d 2.0: Scaling diffusion mod- els for high resolution textured 3d assets generation
Zibo Zhao, Zeqiang Lai, Qingxiang Lin, Yunfei Zhao, Haolin Liu, Shuhui Yang, Yifei Feng, Mingxin Yang, Sheng Zhang, Xianghui Yang, et al. Hunyuan3d 2.0: Scaling diffusion mod- els for high resolution textured 3d assets generation. arXiv preprint arXiv:2501.12202, 2025. 3
2025 arXiv
-
[92]
Dreamscene360: Unconstrained text-to-3d scene generation with panoramic gaussian splatting
Shijie Zhou, Zhiwen Fan, Dejia Xu, Haoran Chang, Pradyumna Chari, Tejas Bharadwaj, Suya You, Zhangyang Wang, and Achuta Kadambi. Dreamscene360: Unconstrained text-to-3d scene generation with panoramic gaussian splatting. In European Conference on Computer Vision, pages 324–342...
2024
-
[93]
Gala3d: Towards text-to-3d complex scene generation via layout-guided generative gaussian splatting
Xiaoyu Zhou, Xingjian Ran, Yajiao Xiong, Jinlin He, Zhiwei Lin, Yongtao Wang, Deqing Sun, and Ming-Hsuan Yang. Gala3d: Towards text-to-3d complex scene generation via layout-guided generative gaussian splatting. arXiv preprint arXiv:2402.07207, 2024. 3, 6, 7
2024 arXiv
-
[94]
1": "A wooden nightstand with a lamp
Yan Zhu, Yuandong Tian, Dimitris Metaxas, and Piotr Doll´ar. Semantic amodal segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1464–1472, 2017. 7 13 HiScene: Creating Hierarchical 3D Scenes with Isometric View Generation Supp...
2017
Reviewed August 16, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.