Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

SemLayoutDiff: Semantic Layout Generation with Diffusion Model for Indoor Scene Synthesis

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read SemLayoutDiff claims a single categorical diffusion model over top-down semantic maps can generate indoor layouts that respect architectural masks better than prior diffusion baselines.

desk verdict A solid, honest empirical paper whose central method is plausible, but whose 'outperforming previous methods' claim rests on a partially manual baseline checkpoint choice and a custom evaluation renderer. read the letter →

arxiv 2508.18597 v2 pith:5BMJZXVQ submitted 2025-08-26 cs.GR cs.CV

Xiaohao Sun , Divyam Goel , Angel X. Chang This is my paper · ORCID
classification cs.GRcs.CV
keywords indoorscenesynthesissemanticlayoutgenerationcategoricaldiffusionmultinomialtop-downmaparchitecturalconditioningconnectedcomponentanalysisattributeprediction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SemLayoutDiff sets out to show that 3D indoor scene synthesis can be treated as an image-generation problem: first draw the room's top-down semantic map, then read the furniture attributes off that map. The paper claims that a single categorical diffusion model, conditioned on an architectural mask (floor, or floor plus doors and windows) and a room type, produces layouts that respect boundaries and openings better than prior diffusion baselines, which typically need separate models per room type and often place objects outside the room or through walls. If the claim holds, one trained model can generate bedrooms, living rooms, and dining rooms in unconditional, floor-conditioned, and full-architecture-conditioned modes. The evidence is a set of distribution-match and physical-plausibility metrics plus user rankings in which the proposed scenes are preferred.

What carries the argument

The load-bearing object is the top-down semantic map at a fixed scale of $0.01$ meters per pixel, generated by a multinomial (categorical) diffusion model: a discrete denoising process in which each pixel is a one-hot vector over $K=38$ classes and noise is added and removed via categorical distributions. The map carries the argument because it encodes object category, horizontal position, and horizontal size in pixel space, so objects cannot overlap at image level and the floor boundary is part of the generated image. Conditioning enters through two additive embeddings: the room mask is embedded and added to the noisy map embedding, and the room type is embedded and added to the timestep embedding. A second module, the attribute prediction model, takes the generated map and instance masks extracted by connected-component analysis, treats the layout feature as a query and the mask feature as key and value in a cross-attention layer, and predicts the vertical size, vertical position, and orientation class for each instance; object retrieval then picks the closest available asset by size.

What would settle it

Render every room in the training and test data as a top-down semantic map at the $0.01$ m/pixel scale and compare connected components against the annotated object instances: if a substantial share of real instances merge into a single connected component or are hidden by occlusion, the representation cannot express the ground truth. A second direct check is to sample many arch-conditioned scenes for a fixed doorway configuration and count how often a furniture box intersects the door opening; the paper reports low blocking rates without an explicit door-blocking loss, so the rate must be reproducible across seeds to confirm the claim.

Watch

Extended reading notes

Core claim

The central claim is that representing a scene as a top-down semantic map—each pixel one of $K=38$ classes, 34 object types plus floor, door, window, and void—and generating that map with a multinomial diffusion model, then extracting connected components as object instances and predicting each instance's vertical size, vertical offset, and four-way orientation, yields spatially coherent 3D layouts. The diffusion model is conditioned by adding a room-mask embedding to the noise input and a room-type embedding to the timestep embedding, which lets one network handle all room types and all three conditioning modes. Compared with two diffusion baselines under the same unified setting, the paper reports lower FID, KID, and category KL on the test distribution, and lower scene- and object-level out-of-bounds ratios, lower collision rates, and higher navigability; under arch-mask conditioning the reported FID is $71.06$ versus $88.47$ and $93.51$ for the baselines. User studies rank its scenes first about 80% of the time. The paper also shows that the same architecture can generate the room itself when no mask is given, and that its layouts can be handed to a separate object generator to produce textured scenes.

Load-bearing premise

The load-bearing assumption is that a single top-down semantic map with one category per pixel, plus per-instance vertical attributes, can faithfully represent a 3D indoor layout; any scene with vertically stacked or overlapping objects, such as a shelf over a desk or a chair under a table, cannot be expressed, and the whole pipeline inherits that ceiling.

Editorial extensions

If this is right

  • One model can serve all room types and conditioning modes; mixed-condition training uses a single network instead of one model per room type and mask type.
  • Architecture conditioning improves distribution match, with FID dropping from 93.93 with no mask to 71.06 with the arch mask in the per-masktype setting, showing doors and windows are informative layout constraints.
  • Out-of-bounds and object-object collision failures are reduced at the representation level because the generated layout is a single layer of category labels inside the room mask.
  • The predicted layouts can be passed to a separate 3D object generator to produce textured scenes, so the method slots into a two-stage generation pipeline.
  • After layout generation, attribute prediction is the main remaining failure source: retrieved objects can have wrong orientation or swapped length and width even when the bounding boxes fit the room.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the semantic map assigns exactly one category per pixel, the collision-avoidance gains are partly built into the representation, not learned; an ablation that generates overlapping 2D boxes from the same diffusion backbone would isolate how much of the improvement comes from the map itself.
  • Connected-component instance extraction will merge objects that touch in top-down view; counting how often annotated instances merge in the training data would directly quantify how often the representation loses an object.
  • The paper's own metric-sensitivity study shows FID, KID, and SCA shift with palette, floor inclusion, and camera zoom, so reported method margins may be tied to the chosen rendering; a standardized rendering protocol would make future comparisons more portable.
  • The method is limited to one horizontal layer per room; moving to layered or voxel semantic maps would be the natural extension, and the paper itself points to semantic voxel grids as one such direction.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces SemLayoutDiff, a two-stage generative model for 3D indoor scenes: a categorical (multinomial) diffusion model generates a top-down semantic layout map, optionally conditioned on a room mask (floor or full architecture) and room type; a cross-attention attribute prediction module then estimates per-object vertical size, vertical position, and orientation for bounding-box layout; object retrieval produces the final textured scene. The model supports unconditional generation, where it also generates the room architecture (floor, doors, windows). The authors evaluate on 3D-FRONT with three room types, comparing against DiffuScene and MiDiffusion under three conditioning modes. Main tables report distribution-matching metrics (FID, KID, SCA, CKL) and physical plausibility metrics (OOB, collision, navigability), plus user studies. They also present ablations: per-masktype vs mixed-condition training, per-roomtype models, direct layout evaluation, and a comparison with PhyScene for the living room.

Significance. If the comparative claims are supported, the paper makes a useful contribution: it shows that a single categorical diffusion model over top-down semantic maps can jointly generate room architecture and furniture layouts, a capability neither DiffuScene nor MiDiffusion provides, and it reports substantially lower collision and out-of-bounds rates. The mixed-condition training experiment is a genuine effort toward a genuinely unified model, and the direct layout evaluation in App. D.3 isolates layout quality from object retrieval. The paper is also unusually transparent about its evaluation choices (App. C), its attribute-prediction failure modes (Fig. D.5), and its representation limits (Sec. 5.4). The main caveat is that the headline comparison against MiDiffusion under architecture conditioning rests on a manually selected checkpoint (App. D.5), and the authors' own rendering ablations show that FID/KID/SCA are highly sensitive to evaluation protocol.

major comments (3)
  1. [App. D.5; Tables 1-2] The Arch-conditioned MiDiffusion baseline is selected by manual inspection of intermediate checkpoints rather than a fixed, outcome-independent rule. The paper states that the standard validation-loss checkpoints for the adapted architecture-plan bedroom model produced scenes 'filled almost exclusively with kid beds', and that the authors then chose a checkpoint based on qualitative criteria ('plausible distance from room centre', 'varied, realistic mix of bedroom items'). Because Tables 1 and 2 report SemLayoutDiff as outperforming MiDiffusion under Arch conditioning, this manual curation directly supports the headline claim. The selection may be conservative, but it is not reproducible and it is not a fixed protocol. Please re-run the Arch-conditioned comparison either using the standard validation-loss checkpoint for all rooms, or report both the standard and manually chosen checkpoints, and discuss the sensitivity of the tables to this choice.
  2. [Sec. 5.2; App. C] The evaluation metrics FID, KID, and SCA are computed with the authors' custom renderer, custom color palette, and specific floor/arch rendering choices. App. C itself shows that these choices substantially change the metrics: adding a floor, changing the palette, switching renderers, or changing the zoom level can move FID by tens of points and KID and SCA by large relative amounts. Since no experiment is reported that fixes the protocol of prior work and still shows SemLayoutDiff winning, the reader cannot tell whether the reported ranking is a property of the models or a property of the new rendering protocol. Please either report the comparison under the prior protocol used by DiffuScene/MiDiffusion, or add an experiment showing that the relative ranking of the three methods is stable across the rendering choices studied in App. C.
  3. [Sec. 5.4; Fig. D.5] The paper acknowledges that the top-down semantic-map representation cannot handle vertically stacked or overlapping objects, and that the attribute prediction and retrieval stage is the main source of residual errors (incorrect orientation, object sliding out of bounds, L-shaped sofa distortion). The abstract's claim of 'outperforming previous methods' should be scoped accordingly: the comparison is meaningful for layouts that are expressible as a single top-down semantic map, but the method cannot reproduce scenes with hierarchical or vertically occluded object arrangements. Please quantify how prevalent such cases are in the 3D-FRONT test set, or explicitly restrict the claim in the abstract and conclusion to the scenes representable by the method.
minor comments (5)
  1. [Sec. 4.1, equation for q(xt|x0)] The formula as printed is missing a closing parenthesis: it should read something like q(xt|x0) = C(xt | ᾱ_t x0 + (1 − ᾱ_t)/K · 1). Please correct the notation.
  2. [Sec. 5.2, last paragraph before Sec. 5.3] The text says SemLayoutDiff reduces OOBS from approximately 55% (DiffuScene) and 60% (MiDiffusion) to 13.8% in bedrooms, but Table 13 reports MiDiffusion bedroom OOBS as 65.25%, not 60%. Please align the text with the reported numbers.
  3. [App. C.2, Table 10] The sentence saying that switching to the zoomed-in view leaves FID/KID 'shift only marginally' is contradicted by Table 10, where KID for the bedroom square-floor case changes from 1.45 to 5.08. Please soften or correct this description.
  4. [App. C.3] The paper concludes that SCA is 'an unreliable metric when renderings vary', yet SCA is still reported as a headline metric in Tables 1-3, 12, 13, 15, 16, 18, and 20. Please state explicitly which metrics are load-bearing for the comparison and which are diagnostic, and avoid over-interpreting SCA in the main text.
  5. [Table 3 and App. D.5] The per-roomtype row for SemLayoutDiff in Table 3 reports FID 107.99 for Arch, which matches App. D.5, but the table is labeled 'Arch' while Table 18 refers to the same setting as 'per-roomtype models conditioned on architecture plan'. Please unify the terminology for readability.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: SemLayoutDiff's semantic-map diffusion and attribute prediction are self-contained; the App. D.5 MiDiffusion checkpoint caveat is an evaluation-fairness concern, not a circular reduction.

full rationale

The paper's derivation chain is a standard two-stage generative pipeline: a multinomial diffusion model generates a 2D semantic map conditioned on room mask and room type, connected components define object instances, and an attribute prediction network outputs vertical size, vertical position, and orientation. None of these stages is defined in terms of the final evaluation metrics or the headline claim. Horizontal size and position are read directly from the generated map pixels, but that is the chosen representation, not a tautology: the map itself is a generated output, and the vertical attributes are not present in the map. The category-specific instance filters in Sec. 4.3 are computed from training statistics and applied during inference, but they are post-processing filters rather than fitted quantities renamed as predictions; they do not force the reported FID/KID/CKL/OOB/collision numbers. The only potentially load-bearing caveat is in App. D.5, where the authors state that MiDiffusion's standard validation-loss checkpoint selection 'breaks down' for arch-conditioned bedrooms and that they 'inspected several intermediate checkpoints and chose the one' with plausible, varied bedroom items. This is an evaluation-fairness weakness in the baseline comparison, not a circularity: it does not make SemLayoutDiff's output equal to its input or derive the headline from a fitted parameter. The self-citations (e.g., SceneEval [38] and the top-down image prior [41]) are used for metrics and related work, not as load-bearing justification of the method's predictions. No equation, loss, or inference step reduces a predicted quantity to a training statistic or to a self-citation, so the paper's central claims are not circular.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

The ledger captures the main fitted or chosen values the central claim depends on: category-specific filtering thresholds, orientation discretization, diffusion step count, room construction parameters, and resolution choices. The axioms cover the adopted multinomial diffusion framework, the sufficiency of the 2D semantic map representation, connected-component instance extraction, axis-aligned orientation, dataset representativeness, and the ad hoc thresholds. No new physical entities are introduced.

free parameters (6)
  • Category-specific instance size thresholds = Not reported numerically; computed as minimum pixel ratio per object type from the training set
    Used in Sec. 4.3 to filter connected components before attribute prediction; acts as a post hoc filter that directly affects OOB and collision metrics.
  • Orientation classes = 4 classes (0, 90, 180, 270 degrees)
    Chosen because 97% of 3D-FRONT orientations are axis-aligned; reduces a continuous regression to a 4-way classification.
  • Diffusion steps = 4000
    Inference uses 4000 steps versus 1000 for the baselines; more steps likely improve sample quality at higher latency (18s vs 11s and 25s).
  • Room construction parameters = wall height 3m, door height 2m, window vertical offset 0.5m to 2m
    Assumed for building 3D geometry from the semantic map; affects visual plausibility and navigability evaluation.
  • Pixel scale and image padding = 0.01 m per pixel, padded to 1200x1200 (12m x 12m)
    Resolution and padding choices for the semantic map; fixed for all rooms and used in both training and inference.
  • Unified category set size K = K=38 (34 object types, floor, door, window, void)
    Combines object categories across room types from ATISS/DiffuScene; the category set is a design choice that defines the output space.
assumptions (6)
  • standard math Multinomial diffusion forward process and KL objective from Hoogeboom et al. (2021) are a valid generative model for categorical pixel maps.
    The semantic layout stage directly adopts this framework; the paper does not re-derive it.
  • domain assumption A single top-down semantic map with one category per pixel is sufficient to represent a 3D indoor layout for the evaluation settings.
    Core representation choice; the method predicts only vertical attributes and cannot handle vertical overlap or hierarchical arrangements, as acknowledged in Sec. 5.4.
  • domain assumption Connected components in the generated semantic map correspond to distinct object instances.
    Used for instance extraction in Sec. 4.3; the paper notes that residual noise can create spurious instances, causing redundant objects (App. D.7).
  • domain assumption Object orientations are approximately axis-aligned, so predicting one of four orientation classes is sufficient.
    Based on the 97% statistic from 3D-FRONT; objects with non-axis-aligned orientations will be poorly represented.
  • domain assumption The filtered 3D-FRONT training split used by ATISS and DiffuScene is a representative distribution for indoor scenes.
    Training and evaluation rely on this dataset; generalization to other datasets is not tested.
  • ad hoc to paper Category-specific pixel ratio thresholds derived from the training set are appropriate for cleaning generated maps.
    No external justification beyond matching training statistics; affects final scene composition and plausibility metrics.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SemLayoutDiff: Semantic Layout Generation with Diffusion Model for Indoor Scene Synthesis." pith.science (2026). https://pith.science/paper/5BMJZXVQ

@misc{pith2026250818597,
  author       = {Pith},
  title        = {Pith review of: SemLayoutDiff: Semantic Layout Generation with Diffusion Model for Indoor Scene Synthesis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5BMJZXVQ}},
  note         = {Machine review of arXiv:2508.18597}
}
read the original abstract

We present SemLayoutDiff, a unified model for synthesizing diverse 3D indoor scenes across multiple room types. The model introduces a scene layout representation combining a top-down semantic map and attributes for each object. Unlike prior approaches, which cannot condition on architectural constraints, SemLayoutDiff employs a categorical diffusion model capable of conditioning scene synthesis explicitly on room masks. It first generates a coherent semantic map, followed by a cross-attention-based network to predict furniture placements that respect the synthesized layout. Our method also accounts for architectural elements such as doors and windows, ensuring that generated furniture arrangements remain practical and unobstructed. Experiments on the 3D-FRONT dataset show that SemLayoutDiff produces spatially coherent, realistic, and varied scenes, outperforming previous methods.

Figures

Figures reproduced from arXiv: 2508.18597 by the authors.

Figure 1
Figure 1. SEMLAYOUTDIFF generates 3D scenes conditioned on an architectural map or unconditionally. Left (full pipeline): With architectural conditioning and room type label, SEMLAYOUTDIFF synthesizes a 2D semantic layout map, predicts 3D attributes to form bounding box layouts, and retrieves objects to construct a final scene. Right: In the unconditional setting, SEMLAYOUTDIFF generates the architecture map and the semantic … view at source ↗
Figure 2
Figure 2. Semantic map representation example. We represent the [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. SemLayoutDiff overview. From left: (a) is the unified diffusion model that is conditioned on the room mask, and room type croom. During the denoising process, the archmask or floormask embedding is added to the noise input embedding. The room type embedding is added to the timestep embedding. (b) is the object attribute prediction model with a semantic layout map as input. si, pi, ri indicate the ith instance’s size… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Example textured synthesized scenes with unconditional [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Comparison of scenes generated by prior methods and S [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Comparison of generated scenes using different methods with [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 8
Figure 8. Figure 8: Example of our synthesized scenes with TRELLIS [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. GOPI: Generation-Oriented 3D Pose Inference for Furniture Insertion from Single-View RGB-D Indoor Scenes

    cs.CV 2026-08 conditional novelty 6.0 of 10

    A pose-first furniture insertion framework infers 3D placement from masked RGB-D input and uses its image-plane projection to condition diffusion, improving geometric feasibility on a synthetic 3D-FRONT benchmark.

Reference graph

Works this paper leans on

60 extracted references · 40 canonical work pages · cited by 1 Pith paper

  1. [1]

    Open-universe indoor scene generation using LLM program synthesis and uncurated object databases

    Rio Aguina-Kang, Maxim Gumin, Do Heon Han, Stew- art Morris, Seung Jean Yoo, Aditya Ganeshan, R Kenny Jones, Qiuhong Anna Wei, Kailiang Fu, and Daniel Ritchie. Open-universe indoor scene generation using LLM program synthesis and uncurated object databases. arXiv preprint arXiv:2403.09675, 2024. 1

    arXiv 2024

  2. [2]

    Demystifying MMD GANs

    Mikołaj Bi ´nkowski, Danica J Sutherland, Michael Arbel, and Arthur Gretton. Demystifying MMD GANs. InInternational Conference on Learning Representations, 2018. 5

    work page 2018

  3. [3]

    I-design: Personalized LLM interior designer

    Ata C ¸ elen, Guo Han, Konrad Schindler, Luc Van Gool, Iro Armeni, Anton Obukhov, and Xi Wang. I-design: Personalized LLM interior designer. arXiv preprint arXiv:2404.02838, 2024. 2

    arXiv 2024

  4. [4]

    Lay- outDM: Transformer-based diffusion model for layout gen- eration

    Shang Chai, Liansheng Zhuang, and Fengying Yan. Lay- outDM: Transformer-based diffusion model for layout gen- eration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 18349– 18358, 2023. 2

    work page 2023

  5. [5]

    Set-the-scene: Global-local train- ing for generating controllable nerf scenes

    Dana Cohen-Bar, Elad Richardson, Gal Metzer, Raja Giryes, and Daniel Cohen-Or. Set-the-scene: Global-local train- ing for generating controllable nerf scenes. arXiv preprint arXiv:2303.13450, 2023. 2

    arXiv 2023

  6. [6]

    ProcTHOR: Large-scale embodied AI using procedural generation

    Matt Deitke, Eli VanderBilt, Alvaro Herrasti, Luca Weihs, Kiana Ehsani, Jordi Salvador, Winson Han, Eric Kolve, Aniruddha Kembhavi, and Roozbeh Mottaghi. ProcTHOR: Large-scale embodied AI using procedural generation. Ad- vances in Neural Information Processing Systems, 35:5982– 5994, 2022. 1

    work page 2022

  7. [7]

    BlenderProc2: A procedural pipeline for photorealistic ren- dering

    Maximilian Denninger, Dominik Winkelbauer, Martin Sundermeyer, Wout Boerdijk, Markus Wendelin Knauer, Klaus H Strobl, Matthias Humt, and Rudolph Triebel. BlenderProc2: A procedural pipeline for photorealistic ren- dering. Journal of Open Source Software, 8(82):4901, 2023. 3, 12

    work page 2023

  8. [8]

    LayoutGPT: Compositional visual planning and generation with large language models

    Weixi Feng, Wanrong Zhu, Tsu-jui Fu, Varun Jampani, Ar- jun Akula, Xuehai He, Sugato Basu, Xin Eric Wang, and William Yang Wang. LayoutGPT: Compositional visual planning and generation with large language models. arXiv preprint arXiv:2305.15393, 2023. 6, 17, 18

    arXiv 2023

Show all 60 references
  1. [9]

    Example-based synthesis of 3D object arrangements

    Matthew Fisher, Daniel Ritchie, Manolis Savva, Thomas Funkhouser, and Pat Hanrahan. Example-based synthesis of 3D object arrangements. ACM Transactions on Graph- ics (TOG), 31(6):1–11, 2012. 1, 2

    2012

  2. [10]

    3D-Front: 3D furnished rooms with layouts and semantics

    Huan Fu, Bowen Cai, Lin Gao, Ling-Xiao Zhang, Jiaming Wang, Cao Li, Qixun Zeng, Chengyue Sun, Rongfei Jia, Binqiang Zhao, et al. 3D-Front: 3D furnished rooms with layouts and semantics. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision , pages 10933–...

    2021

  3. [11]

    GraphDreamer: Compositional 3D scene synthesis from scene graphs, 2023

    Gege Gao, Weiyang Liu, Anpei Chen, Andreas Geiger, and Bernhard Sch ¨olkopf. GraphDreamer: Compositional 3D scene synthesis from scene graphs, 2023. 2

    2023

  4. [12]

    SceneHGN: Hierarchical graph net- works for 3D indoor scene generation with fine-grained ge- ometry

    Lin Gao, Jia-Mu Sun, Kaichun Mo, Yu-Kun Lai, Leonidas J Guibas, and Jie Yang. SceneHGN: Hierarchical graph net- works for 3D indoor scene generation with fine-grained ge- ometry. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(7):8902–8919, 2023. 2

    2023

  5. [13]

    GANs trained by a two time-scale update rule converge to a local Nash equi- librium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs trained by a two time-scale update rule converge to a local Nash equi- librium. Advances in neural information processing systems, 30, 2017. 5

    2017

  6. [14]

    Denoising diffu- sion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models. Advances in Neural Information Processing Systems, 33:6840–6851, 2020. 2

    2020

  7. [15]

    Argmax flows and multinomial dif- fusion: Learning categorical distributions

    Emiel Hoogeboom, Didrik Nielsen, Priyank Jaini, Patrick Forr´e, and Max Welling. Argmax flows and multinomial dif- fusion: Learning categorical distributions. Advances in Neu- ral Information Processing Systems, 34:12454–12465, 2021. 2, 3

    2021

  8. [16]

    Mixed diffusion for 3D indoor scene synthesis

    Siyi Hu, Diego Martin Arroyo, Stephanie Debats, Fabian Manhardt, Luca Carlone, and Federico Tombari. Mixed diffusion for 3D indoor scene synthesis. arXiv preprint arXiv:2405.21066, 2024. 1, 2, 3, 4, 5, 7, 13, 15, 20, 21, 22, 23

    2024 arXiv

  9. [17]

    SceneCraft: An LLM agent for synthesizing 3D scene as Blender code

    Ziniu Hu, Ahmet Iscen, Aashi Jain, Thomas Kipf, Yisong Yue, David A Ross, Cordelia Schmid, and Alireza Fathi. SceneCraft: An LLM agent for synthesizing 3D scene as Blender code. In ICLR 2024 Workshop on Large Language Model (LLM) Agents, 2024. 2

    2024

  10. [18]

    LayoutDM: Discrete diffusion model for controllable layout generation

    Naoto Inoue, Kotaro Kikuchi, Edgar Simo-Serra, Mayu Otani, and Kota Yamaguchi. LayoutDM: Discrete diffusion model for controllable layout generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10167–10176, 2023. 2

    2023

  11. [19]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations, 2015. 13

    2015

  12. [20]

    GRAINS: Generative re- cursive autoencoders for indoor scenes

    Manyi Li, Akshay Gadi Patil, Kai Xu, Siddhartha Chaudhuri, Owais Khan, Ariel Shamir, Changhe Tu, Baoquan Chen, Daniel Cohen-Or, and Hao Zhang. GRAINS: Generative re- cursive autoencoders for indoor scenes. ACM Transactions on Graphics (TOG), 38(2):1–16, 2019. 2

    2019

  13. [21]

    InstructScene: Instruction- driven 3D indoor scene synthesis with semantic graph prior

    Chenguo Lin and Yadong Mu. InstructScene: Instruction- driven 3D indoor scene synthesis with semantic graph prior. In International Conference on Learning Representations ,

  14. [22]

    CompoNeRF: Text-guided multi- object compositional nerf with editable 3D scene layout

    Yiqi Lin, Haotian Bai, Sijia Li, Haonan Lu, Xiaodong Lin, Hui Xiong, and Lin Wang. CompoNeRF: Text-guided multi- object compositional nerf with editable 3D scene layout. arXiv preprint arXiv:2303.13843, 2023. 2

    2023 arXiv

  15. [23]

    End-to-end optimization of scene layout

    Andrew Luo, Zhoutong Zhang, Jiajun Wu, and Joshua B Tenenbaum. End-to-end optimization of scene layout. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 3754–3763, 2020. 2

    2020

  16. [24]

    DeBaRA: Denoising-based 3D room arrangement generation

    L ´eopold Maillard, Nicolas Sereyjol-Garros, Tom Durand, and Maks Ovsjanikov. DeBaRA: Denoising-based 3D room arrangement generation. Advances in Neural Information Processing Systems, 2024. 2

    2024

  17. [25]

    Interactive furniture layout using in- terior design guidelines

    Paul Merrell, Eric Schkufza, Zeyang Li, Maneesh Agrawala, and Vladlen Koltun. Interactive furniture layout using in- terior design guidelines. ACM transactions on graphics (TOG), 30(4):1–10, 2011. 1, 2

    2011

  18. [26]

    COFS: Controllable furniture layout synthesis

    Wamiq Reyaz Para, Paul Guerrero, Niloy Mitra, and Peter Wonka. COFS: Controllable furniture layout synthesis. In ACM SIGGRAPH Conference Proceedings, 2023. 2

    2023

  19. [27]

    ATISS: Autore- gressive transformers for indoor scene synthesis

    Despoina Paschalidou, Amlan Kar, Maria Shugrina, Karsten Kreis, Andreas Geiger, and Sanja Fidler. ATISS: Autore- gressive transformers for indoor scene synthesis. Advances in Neural Information Processing Systems, 34:12013–12026,

  20. [28]

    Compositional 3D scene generation using locally conditioned diffusion

    Ryan Po and Gordon Wetzstein. Compositional 3D scene generation using locally conditioned diffusion. arXiv preprint arXiv:2303.12218, 2023. 2

    2023 arXiv

  21. [29]

    Infinite photore- alistic worlds using procedural generation

    Alexander Raistrick, Lahav Lipson, Zeyu Ma, Lingjie Mei, Mingzhe Wang, Yiming Zuo, Karhan Kayan, Hongyu Wen, Beining Han, Yihan Wang, Alejandro Newell, Hei Law, Ankit Goyal, Kaiyu Yang, and Jia Deng. Infinite photore- alistic worlds using procedural generation. In Proceedings ...

    2023

  22. [30]

    Fast and flex- ible indoor scene synthesis via deep convolutional genera- tive models

    Daniel Ritchie, Kai Wang, and Yu-an Lin. Fast and flex- ible indoor scene synthesis via deep convolutional genera- tive models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6182– 6190, 2019. 2

    2019

  23. [31]

    Deep unsupervised learning using nonequilibrium thermodynamics

    Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International Confer- ence on Machine Learning, pages 2256–2265. PMLR, 2015. 2

    2015

  24. [32]

    Denois- ing diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denois- ing diffusion implicit models. In International Conference on Learning Representations, 2021

    2021

  25. [33]

    Improved techniques for training score-based generative models

    Yang Song and Stefano Ermon. Improved techniques for training score-based generative models. Advances in neural information processing systems, 33:12438–12448, 2020. 2

    2020

  26. [34]

    CHOrD: Generation of collision-free, house-scale, and organized digital twins for 3D indoor scenes with controllable floor plans and optimal layouts

    Chong Su, Yingbin Fu, Zheyuan Hu, Jing Yang, Param Hanji, Shaojun Wang, Xuan Zhao, Cengiz ¨Oztireli, and Fangcheng Zhong. CHOrD: Generation of collision-free, house-scale, and organized digital twins for 3D indoor scenes with controllable floor plans and optimal layouts. arXiv...

    2025 arXiv

  27. [35]

    LayoutVLM: Differentiable optimization of 3D layout via vision-language models

    Fan-Yun Sun, Weiyu Liu, Siyi Gu, Dylan Lim, Goutam Bhat, Federico Tombari, Manling Li, Nick Haber, and Jiajun Wu. LayoutVLM: Differentiable optimization of 3D layout via vision-language models. arXiv preprint arXiv:2412.02193,

    arXiv

  28. [36]

    RelTriple: Learning plausible indoor layouts by integrating relationship triples into the diffusion process

    Kaifan Sun, Bingchen Yang, Peter Wonka, Jun Xiao, and Haiyong Jiang. RelTriple: Learning plausible indoor layouts by integrating relationship triples into the diffusion process. arXiv preprint arXiv:2503.20289, 2025. 2

    2025

  29. [37]

    Forest2Seq: Revitalizing order prior for sequential in- door scene synthesis

    Qi Sun, Hang Zhou, Wengang Zhou, Li Li, and Houqiang Li. Forest2Seq: Revitalizing order prior for sequential in- door scene synthesis. In European Conference on Computer Vision, 2024. 2

    2024

  30. [38]

    SceneEval: Evaluating semantic coherence in text-conditioned 3D indoor scene syn- thesis

    Hou In Ivan Tam, Hou In Derek Pun, Austin T Wang, An- gel X Chang, and Manolis Savva. SceneEval: Evaluating semantic coherence in text-conditioned 3D indoor scene syn- thesis. arXiv preprint arXiv:2503.14756, 2025. 6, 13

    2025

  31. [39]

    DiffuScene: Scene graph denoising diffusion probabilistic model for generative indoor scene synthesis

    Jiapeng Tang, Yinyu Nie, Lev Markhasin, Angela Dai, Jus- tus Thies, and Matthias Nießner. DiffuScene: Scene graph denoising diffusion probabilistic model for generative indoor scene synthesis. arXiv preprint arXiv:2303.14207, 2023. 1, 2, 3, 4, 5, 7, 12, 13, 15, 16, 17, 18, 20,...

    2023 arXiv

  32. [40]

    CG3D: Compositional generation for text-to-3D via gaussian splatting

    Alexander Vilesov, Pradyumna Chari, and Achuta Kadambi. CG3D: Compositional generation for text-to-3D via gaussian splatting. arXiv preprint arXiv:2311.17907, 2023. 2

    2023 arXiv

  33. [41]

    Deep convolutional priors for indoor scene syn- thesis

    Kai Wang, Manolis Savva, Angel X Chang, and Daniel Ritchie. Deep convolutional priors for indoor scene syn- thesis. ACM Transactions on Graphics (TOG), 37(4):1–14,

  34. [42]

    PlanIT: Planning and in- stantiating indoor scenes with relation graph and spatial prior networks

    Kai Wang, Yu-An Lin, Ben Weissmann, Manolis Savva, An- gel X Chang, and Daniel Ritchie. PlanIT: Planning and in- stantiating indoor scenes with relation graph and spatial prior networks. ACM Transactions on Graphics (TOG), 38(4):1– 15, 2019. 1, 2

    2019

  35. [43]

    Sceneformer: Indoor scene generation with transformers

    Xinpeng Wang, Chandan Yeshwanth, and Matthias Nießner. Sceneformer: Indoor scene generation with transformers. In 2021 International Conference on 3D Vision (3DV) , pages 106–115. IEEE, 2021. 1, 2

    2021

  36. [44]

    Lego-net: Learning regular rearrangements of ob- jects in rooms

    Qiuhong Anna Wei, Sijie Ding, Jeong Joon Park, Rahul Sajnani, Adrien Poulenard, Srinath Sridhar, and Leonidas Guibas. Lego-net: Learning regular rearrangements of ob- jects in rooms. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19...

    2023

  37. [45]

    Any- Home: Open-vocabulary generation of structured and tex- tured 3D homes

    Zehao Wen, Zichen Liu, Srinath Sridhar, and Rao Fu. Any- Home: Open-vocabulary generation of structured and tex- tured 3D homes. arXiv preprint arXiv:2312.06644, 2023. 2

    2023 arXiv

  38. [46]

    Structured 3D latents for scalable and versatile 3D generation

    Jianfeng Xiang, Zelong Lv, Sicheng Xu, Yu Deng, Ruicheng Wang, Bowen Zhang, Dong Chen, Xin Tong, and Jiaolong Yang. Structured 3D latents for scalable and versatile 3D generation. In IEEE Conference on Computer Vision and Pattern Recognition, 2025. 8

    2025

  39. [47]

    Constraint- based automatic placement for scene composition

    Ken Xu, James Stewart, and Eugene Fiume. Constraint- based automatic placement for scene composition. InGraph- ics Interface, pages 25–34, 2002. 2

    2002

  40. [48]

    Indoor scene generation from a collection of semantic- segmented depth images

    Ming-Jia Yang, Yu-Xiao Guo, Bin Zhou, and Xin Tong. Indoor scene generation from a collection of semantic- segmented depth images. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15203– 15212, 2021. 8

    2021

  41. [49]

    PhyScene: Physically interactable 3D scene synthe- sis for embodied AI

    Yandan Yang, Baoxiong Jia, Peiyuan Zhi, and Siyuan Huang. PhyScene: Physically interactable 3D scene synthe- sis for embodied AI. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 16262–16272, 2024. 2, 5, 23

    2024

  42. [50]

    Holodeck: Language guided gener- ation of 3D embodied AI environments

    Yue Yang, Fan-Yun Sun, Luca Weihs, Eli VanderBilt, Alvaro Herrasti, Winson Han, Jiajun Wu, Nick Haber, Ranjay Kr- ishna, Lingjie Liu, et al. Holodeck: Language guided gener- ation of 3D embodied AI environments. InIEEE Conference on Computer Vision and Pattern Recognition, pag...

    2024

  43. [51]

    CommonScenes: Generating commonsense 3D in- door scenes with scene graphs

    Guangyao Zhai, Evin Pinar ¨Ornek, Shun-Cheng Wu, Yan Di, Federico Tombari, Nassir Navab, and Benjamin Busam. CommonScenes: Generating commonsense 3D in- door scenes with scene graphs. Advances in Neural Informa- tion Processing Systems, 2023. 2

    2023

  44. [52]

    Echoscene: Indoor scene generation via information echo over scene graph diffusion

    Guangyao Zhai, Evin Pınar ¨Ornek, Dave Zhenyu Chen, Ruo- tong Liao, Yan Di, Nassir Navab, Federico Tombari, and Benjamin Busam. Echoscene: Indoor scene generation via information echo over scene graph diffusion. In European Conference on Computer Vision , pages 167–184. Springer,

  45. [53]

    Deep generative modeling for scene synthesis via hybrid represen- tations

    Zaiwei Zhang, Zhenpei Yang, Chongyang Ma, Linjie Luo, Alexander Huth, Etienne V ouga, and Qixing Huang. Deep generative modeling for scene synthesis via hybrid represen- tations. ACM Transactions on Graphics (TOG), 39(2):1–21,

  46. [54]

    SceneGraphNet: Neural message passing for 3D indoor scene augmentation

    Yang Zhou, Zachary While, and Evangelos Kalogerakis. SceneGraphNet: Neural message passing for 3D indoor scene augmentation. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pages 7384–7392,

  47. [57]

    bedroom, living room, dining room) and fil- ter the data following DiffuScene [39]

    We first extract rooms from scenes based on their room types (e.g. bedroom, living room, dining room) and fil- ter the data following DiffuScene [39]. Specifically, we remove rooms with unnatural dimensions (e.g., overly large sizes or extreme heights), discard infrequent ob- ...

  48. [58]

    For the unified dataset, we combine all room types and split the processed data into 70% training, 10% validation, and 20% test following prior work [27, 39]

    After this filtering process, the dataset includes 4041 bedrooms, 900 living rooms, and 813 dining rooms. For the unified dataset, we combine all room types and split the processed data into 70% training, 10% validation, and 20% test following prior work [27, 39]

  49. [59]

    We render the top-down 2D semantic layout map us- ing orthographic projection, where the camera is posi- tioned at the center top of the room. To ensure that each pixel on the map represents a consistent physi- cal unit s, we compute the desired image dimensions in pixels (wim...

  50. [60]

    The offset is determined by measuring the maximum distance from the object’s bounding box to the floor

    We then extract the objects’ category, size, vertical po- sition (offset from floor), and orientation based on the annotations. The offset is determined by measuring the maximum distance from the object’s bounding box to the floor. 1The Blender orthographic scale specifies the...

  51. [2019]

    A), additional information about our experimental setup (App

    2 SemLayoutDiff: Semantic Layout Generation with Diffusion Model for Indoor Scene Synthesis Supplementary Material In this supplement, we present model details (App. A), additional information about our experimental setup (App. B), preliminary experiments to investigate the im...

  52. [2021]

    1, 2, 3, 4, 5, 12, 13, 15, 17, 18, 21

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.