Pith. sign in

REVIEW 3 major objections 6 minor 53 references

MMGDreamer: Mixed-Modality Graph for Geometry-Controllable 3D Indoor Scene Generation

T0 review · 3 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read MMGDreamer claims that a mixed-modality scene graph—where object nodes can be text, images, or both—lets a dual-branch diffusion model control object geometry in generated 3D indoor scenes, outperforming prior graph-based generators on…

desk verdict The mixed-modality graph is a genuine extension of EchoScene, but the evaluation is confounded because test-time images come from the ground-truth objects, so the SOTA claims don't hold as presented. read the letter →

arxiv 2502.05874 v3 pith:7VTBUYKU submitted 2025-02-09 cs.CV cs.AIcs.LG

Zhifei Yang , Keyang Lu , Chao Zhang , Jiaxing Qi , Hanqi Jiang , Ruifei Ma , Shenglin Yin , Yifan Xu
show 4 more authors
Mingzhe Xing Zhen Xiao Jieyi Long Guangyao Zhai
This is my paper · ORCID
classification cs.CVcs.AIcs.LG
keywords 3Dindoorscenegenerationmixed-modalitygraphgraphsdiffusionmodelsgeometrycontrolCLIPfeatureslayoutshape
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that letting scene-graph nodes carry images as well as text gives a 3D indoor scene generator precise control over object geometry, something text-only graphs cannot do. It introduces a Mixed-Modality Graph in which each object node may be a text label, an image, or both, and edges can be omitted, and a dual-branch diffusion model that turns this graph into a bounding-box layout plus per-object signed-distance-field shapes. Two supporting modules carry the load: a visual enhancement module that synthesizes a visual feature for text-only nodes by quantizing their CLIP text embedding, and a relation predictor that fills in missing edges so layouts stay coherent. If the central claim holds, a user could supply a photo of a particular chair or nightstand and the model would place a faithful version of that object into a realistic room.

What carries the argument

The central object is the Mixed-Modality Graph (MMG), a directed scene graph whose object nodes can be text-only, image-only, or both, and whose edges are optional relationship labels. The argument runs through three mechanisms: a visual enhancement module that uses a VQ-VAE-style encoder, codebook, and decoder to manufacture a visual feature from a text-only node's CLIP text embedding; a relation predictor, a GCN with MLP heads, that classifies missing edges and turns the graph into a fully connected mixed-enhanced graph; and a dual-branch latent diffusion model whose layout and shape denoisers are both conditioned on node representations produced by a triplet-GCN graph encoder with an echo mechanism. The shape branch is where geometric control is supposed to happen: its 3D-UNet denoiser receives per-node visual features, so the output signed-distance-field shape should track the geometry implied by the input image.

What would settle it

Generate the same scene graph with the same text node but two reference images of the same object category that differ clearly in geometry, such as an armchair and a straight-backed chair, and compare the produced shapes; if the generated shapes are statistically indistinguishable under MMD or Chamfer distance, the visual features are not carrying geometric control.

Watch

Extended reading notes

Core claim

The central claim is that a scene graph with mixed-modality nodes, combined with a dual-branch latent diffusion generator, achieves state-of-the-art geometry-controllable 3D indoor scene generation. In the paper's construction, each node carries category text, a CLIP image feature, or both, and missing relationships are either inferred by a GCN-based relation predictor or represented by zero padding; the layout branch denoises bounding-box parameters while the shape branch denoises truncated-signed-distance-field latents, with both branches conditioned on graph-encoded node representations that include the visual features. The paper reports that on the SG-FRONT dataset this design outperforms prior text-only graph-to-3D generators on scene-level realism (FID, FID-CLIP, KID), object-level shape fidelity (MMD, COV, 1-NNA), and scene-graph consistency, with the improvements attributed to the visual conditioning and to the relation predictor's role in layout coherence.

Load-bearing premise

The load-bearing premise is that CLIP image features, and the text-derived visual features the enhancement module fabricates, carry enough geometric information about an object to steer the shape denoiser; if they only capture semantic category, the paper's claimed geometry control would collapse.

Editorial extensions

If this is right

  • A user can describe some objects in a room with text and others with photos, and omit relationships between them; the model fills in the gaps rather than rejecting the input.
  • Giving the shape branch a node's image feature should make the generated furniture follow the reference photo's geometry, including proportions and details such as chair backrests.
  • Because the relation predictor supplies missing edges, the layout branch can still arrange objects coherently even when the user provides no explicit relations.
  • Scene-graph editing operations—changing a relation or adding a node—propagate to the generated scene, preserving spatial relations such as front/behind and left/right better than text-only baselines in the paper's measurements.
  • The reported object-level metrics on SG-FRONT improve for most furniture categories, which follows if the visual conditioning is genuinely shaping the generated object geometry.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If CLIP features encode mostly semantic category rather than metric geometry, the apparent geometry control may be a category-conditioned shape prior in disguise; swapping the reference image within a category is a direct way to test this.
  • The relation predictor is trained on masked SG-FRONT relations, so its inferred edges are limited to the fifteen relationship types in that dataset; novel user-specified relationships outside that vocabulary would have to be mapped to synonyms or would be ignored.
  • The visual enhancement module is effectively a text-to-visual-feature translator; replacing it with a stronger image prior (e.g., generated reference views) would probably sharpen geometry control for text-only nodes.
  • Since the paper deliberately excludes texture and material, the same graph structure could later be extended to carry appearance features, giving joint geometry-and-texture control without changing the graph representation.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. MMGDreamer proposes a dual-branch diffusion model for 3D indoor scene generation from a Mixed-Modality Graph, where nodes can carry text, images, or both, and edges may be partially provided. The method encodes the graph with CLIP and an embedding layer, applies a visual enhancement module (VQ-VAE) to synthesize visual features for text-only nodes, and a relation predictor to infer missing edges. A triplet-GCN graph encoder with an echo mechanism conditions separate layout and shape diffusion branches. Experiments on SG-FRONT report improved FID, KID, MMD, COV, and 1-NNA over Graph-to-3D, CommonScenes, and EchoScene.

Significance. The idea of a mixed-modality scene graph is a useful extension of current text-only graph-based scene generation, and the dual-branch architecture is a reasonable integration of existing components. If the evaluation were fair, the method could offer a practical interface for geometry control. However, the current test protocol gives the model access to ground-truth object images, so the reported state-of-the-art numbers are not trustworthy as evidence of generalizable geometric control. The paper does not provide reproducible code or data, and the reported differences lack statistical validation.

major comments (3)
  1. [Experimental Settings (Evaluation Dataset) and Table 2] The evaluation protocol is confounded by the test-time use of ground-truth object images. The authors state that they 'extracted corresponding images from the 3D-FUTURE dataset (Fu et al. 2021) based on node IDs to construct a Full-Modality Graph.' At test time, this means the image condition for a node is a render of the exact object instance whose shape serves as the ground truth for the object-level metrics (MMD, COV, 1-NNA) in Table 2. The text-only baselines (Graph-to-3D, CommonScenes, EchoScene) have no access to this identity-preserving signal, so the large reported gains in Tables 1 and 2 can be explained by instance copying rather than by the proposed mixed-modality graph or the visual enhancement module. The ablation in Table 3 does not resolve this issue because every configuration still uses the same ground-truth images for image-conditioned nodes. To support the central claim of state-of-the-art geometric control, the authors need to evaluate with held-out or novel user-provided images (e.g., images not corresponding to any node ID in the test graph) and compare against an image-conditioned baseline that receives the same visual input.
  2. [Tables 1, 2, and 3] All quantitative results are reported without error bars or multiple runs. Several improvements over EchoScene are small (e.g., FID_CLIP in bedroom: 4.26 vs 3.84; KID in bedroom: 1.77 vs 1.72), and without variance estimates or a significance test it is unclear whether these differences are meaningful. The paper also does not release code or data, which makes it impossible to verify the results. The authors should report means and standard deviations over at least three random seeds and consider a statistical test for the main comparisons.
  3. [Visual Enhancement Module and Table 3] The visual enhancement module is only evaluated in the full mixed-modality setting, where image-conditioned nodes still carry ground-truth image features. The paper does not isolate the module's contribution on text-only graphs, which is the scenario the module is designed for. It would be informative to report results for (i) text-only graphs with and without the visual enhancement module and (ii) image-only graphs with and without the relation predictor. Without these ablations, it is unclear whether the claimed 'enrichment of visual fidelity' actually improves geometry control beyond the direct conditioning on ground-truth images.
minor comments (6)
  1. [Related Work] The method name 'CommonScenes' is misspelled as 'CommoScenes' in the sentence 'EchoScene (Zhai et al. 2024b) advances the CommonScenes by incorporating...'; please fix this typo and check the supplementary for the same misspelling.
  2. [Shape and Layout Branch] The text says 'For the shape branch, as shown in Fig. 2.C.2' but in Fig. 2, C.2 is the Layout Branch and C.3 is the Shape Branch; the figure reference should be corrected.
  3. [Table 3 and Ablation Study] The metric 'mSG' (mean scene graph consistency) is used in Table 3 but is not defined in the main text; please define it in the caption or in the 'Evaluation Metrics' paragraph.
  4. [Supplementary Material (GPT-4V Prompt)] The prompt in Fig. 5 states that 'only these twelve relationships are allowed' but then lists fifteen relationships; the number should be corrected to match the list.
  5. [Implementation Details] The main text does not specify which CLIP model is used; the supplementary mentions ViT-B/32, but this should be stated in the main text for completeness.
  6. [Ablation Study and Scene Graph Consistency] In the supplementary Table 1, under 'Relationship change' mode, MMGDreamer's 'symmetrical' score (0.53) is lower than CommonScenes (0.59), yet the text claims 'a clear superiority in the symmetrical metric compared to other methods'; this overstatement should be corrected.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: MMGDreamer's method is a set of trained modules benchmarked externally; no prediction reduces to its own inputs.

full rationale

The paper makes no first-principles derivation of its claimed geometry control; it presents trained modules (CLIP-encoded graph nodes, a VQ-VAE-based visual enhancement module, a GCN-based relation predictor, and a dual-branch diffusion model) and evaluates them against external baselines (Graph-to-3D, CommonScenes, EchoScene) on SG-FRONT. The self-citations to EchoScene and CommonScenes by co-author Guangyao Zhai are used as baseline methods and as a component source (the echo mechanism in the graph encoder), but the central claim of geometry controllability does not rest on an unverified uniqueness theorem or on a self-citation chain. The relation predictor is trained with a cross-entropy loss on masked relationships, and the visual enhancement module is trained with an ELBO objective on text-visual pairs; neither reduces to the reported metrics by construction. Although the evaluation protocol supplies ground-truth 3D-FUTURE images to the image-conditioned variants, that is the intended image-to-shape control task rather than a circular derivation; any generalization or comparison confound is an experimental-design concern, not circularity. No equation in the paper reduces to its own input, and no fitted parameter is renamed as a prediction. The work is therefore self-contained against external benchmarks and exhibits no significant circularity.

Assumptions & free parameters 7 free parameters · 4 assumptions · 0 invented entities

The method depends on standard deep learning components and two dataset-specific assumptions about image-geometry correspondence and CLIP's geometric informativeness. These assumptions are not validated outside the paper, and no code or data is provided to check them.

free parameters (7)
  • Loss weights alpha1, alpha2 = 1.0, 1.0
    Set to equal weighting for layout and shape branches; chosen by hand, not optimized.
  • Codebook size and dimension = 64 x 512
    Codebook for the visual enhancement module; these values are chosen hyperparameters.
  • Number of nearest codebook entries n = 4
    Selects n=4 closest entries in Eq. (9); chosen empirically.
  • Masking ratio for relation predictor training = 0.5
    50% of relations are masked during training; chosen, not swept.
  • Network hidden dimensions = GCN hidden 256, MLP 256/128
    Architecture sizes for the relation predictor; chosen by hand.
  • Learning rates = 1e-4 for diffusion, 5e-3 for relation predictor
    AdamW learning rates; chosen without reported tuning.
  • Training epochs = 1000 for VEM/RP, 2050 for diffusion
    Training durations; chosen and not ablated.
assumptions (4)
  • domain assumption CLIP textual and visual features are sufficient to control 3D object geometry.
    The shape branch conditions its denoiser on node features derived from CLIP; if CLIP features do not encode geometry, the method cannot achieve the claimed geometric control. Entered in 'Shape and Layout Branch'.
  • domain assumption The 3D-FUTURE images paired with SG-FRONT objects faithfully represent the geometry of those objects.
    The full-modality graph is built by extracting images from 3D-FUTURE based on node IDs; if the image-to-mesh correspondence is weak, the visual conditioning will mislead generation. Entered in 'Evaluation Dataset'.
  • domain assumption The scene graph is a sufficient representation for generating complete indoor scenes.
    The method discards any information not captured by nodes and edges; this assumption follows from prior scene-graph generation work and is not independently validated. Entered in 'Scene Graph Representation'.
  • standard math Standard diffusion and GCN mathematics are correct.
    The method relies on latent diffusion and triplet-GCN updates; these are taken as background knowledge.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MMGDreamer: Mixed-Modality Graph for Geometry-Controllable 3D Indoor Scene Generation." pith.science (2026). https://pith.science/paper/7VTBUYKU

@misc{pith2026250205874,
  author       = {Pith},
  title        = {Pith review of: MMGDreamer: Mixed-Modality Graph for Geometry-Controllable 3D Indoor Scene Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7VTBUYKU}},
  note         = {Machine review of arXiv:2502.05874}
}
read the original abstract

Controllable 3D scene generation has extensive applications in virtual reality and interior design, where the generated scenes should exhibit high levels of realism and controllability in terms of geometry. Scene graphs provide a suitable data representation that facilitates these applications. However, current graph-based methods for scene generation are constrained to text-based inputs and exhibit insufficient adaptability to flexible user inputs, hindering the ability to precisely control object geometry. To address this issue, we propose MMGDreamer, a dual-branch diffusion model for scene generation that incorporates a novel Mixed-Modality Graph, visual enhancement module, and relation predictor. The mixed-modality graph allows object nodes to integrate textual and visual modalities, with optional relationships between nodes. It enhances adaptability to flexible user inputs and enables meticulous control over the geometry of objects in the generated scenes. The visual enhancement module enriches the visual fidelity of text-only nodes by constructing visual representations using text embeddings. Furthermore, our relation predictor leverages node representations to infer absent relationships between nodes, resulting in more coherent scene layouts. Extensive experimental results demonstrate that MMGDreamer exhibits superior control of object geometry, achieving state-of-the-art scene generation performance. Project page: https://yangzhifeio.github.io/project/MMGDreamer.

Figures

Figures reproduced from arXiv: 2502.05874 by the authors.

Figure 1
Figure 1. MMGDreamer processes a Mixed-Modality Graph to generate a 3D indoor scene, where object geometry can be precisely controlled. Starting from the fifth type of input (Mixed-Modality) shown in module A as an example, the framework utilizes a vision-language model (B) to produce a Mixed-Modality Graph (C). This graph is further refined by the Generation Module (D) to create a coherent and precise 3D scene (E). struct vi… view at source ↗
Figure 2
Figure 2. Overview of MMGDreamer. Our pipeline consists of the Latent Mixed-Modality Graph, the Graph Enhancement Module, and the Dual-Branch Diffusion Model. During inference, MMGDreamer initiates with the Latent Mixed-Modality Graph, which undergoes enhancement via the Visual Enhancement Module and the Relation Predictor, resulting in the formation of a Visual-Enhanced Graph and a Mixed-Enhanced Graph. The Mixed-Enhanced Gr… view at source ↗
Figure 2
Figure 2. C.2, we use Truncated Signed Distance Field (Cur [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figures from the paper (7 more)
Figure 3
Figure 3. Figure 3: In comparison across different room types, our [PITH_FULL_IMAGE:figures/full_fig_p006_3.png]
Figure 3
Figure 3. Figure 3: Qualitative comparison with other methods. The first column shows the input mixed-modality graph, which visual￾izes only the most critical edges in the scene. Red rectangles denote areas of inconsistency in the generated scenes, while green rectangles signify regions o…
Figure 1
Figure 1. Figure 1: Failure case. The dashed box on the left is a top￾down view rendered using the ground truth, while the result on the right is generated scene by MMGDreamer. each with a dimension of 512. The Visual Enhancement Module is trained for 1,000 epochs with a batch size of 128…
Figure 2
Figure 2. Figure 2: More qualitative comparison on scene generation. The first row shows the input mixed-modality graph, which visualizes only the most critical edges in the scene. Red rectangles denote areas of inconsistency in the generated scenes, while green rectangles signify regions…
Figure 3
Figure 3. Figure 3: Qualitative results on object generation. The figure is divided into three sections by dashed lines. In each section, the top row shows the input images of various furniture items, the middle row displays the corresponding generated objects in the scenes, and the botto…
Figure 4
Figure 4. Figure 4: Qualitative Results On Relation-Free Scene Generation. The figure is divided into two sections by dashed lines. In each section, the dashed boxes represent the input mixed-modality graphs, where nodes are depicted either as text or images, without any explicit relation…
Figure 5
Figure 5. Figure 5: Prompt template for Mixed-Modality Graph Generation with GPT-4V [PITH_FULL_IMAGE:figures/full_fig_p017_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

53 extracted references · 29 canonical work pages

  1. [1]

    L.; Almeida, D.; Altenschmidt, J.; Altman, S.; Anadkat, S.; et al

    Achiam, J.; Adler, S.; Agarwal, S.; Ahmad, L.; Akkaya, I.; Aleman, F. L.; Almeida, D.; Altenschmidt, J.; Altman, S.; Anadkat, S.; et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  2. [2]

    CIC-BART-SSA: Controllable Image Captioning with Structured Semantic Augmentation

    Basioti, K.; Abdelsalam, M. A.; Fancellu, F.; Pavlovic, V.; and Fazly, A. 2024. CIC-BART-SSA: Controllable Image Captioning with Structured Semantic Augmentation. arXiv preprint arXiv:2407.11393

    work page Pith review arXiv 2024

  3. [3]

    A.; Guo, P.; Abnar, S.; Talbott, W.; Toshev, A.; Chen, Z.; Dinh, L.; Zhai, S.; Goh, H.; Ulbricht, D.; et al

    Bautista, M. A.; Guo, P.; Abnar, S.; Talbott, W.; Toshev, A.; Chen, Z.; Dinh, L.; Zhai, S.; Goh, H.; Ulbricht, D.; et al. 2022. Gaudi: A neural architect for immersive 3d scene generation. Advances in Neural Information Processing Systems, 35: 25102--25116

    work page 2022

  4. [4]

    J.; Arbel, M.; and Gretton, A

    Bi \'n kowski, M.; Sutherland, D. J.; Arbel, M.; and Gretton, A. 2018. Demystifying mmd gans. arXiv preprint arXiv:1801.01401

    arXiv 2018

  5. [5]

    C elen, A.; Han, G.; Schindler, K.; Van Gool, L.; Armeni, I.; Obukhov, A.; and Wang, X. 2024. I-design: Personalized llm interior designer. arXiv preprint arXiv:2404.02838

    arXiv 2024

  6. [6]

    G.; and Gui, L.-Y

    Cheng, Y.-C.; Lee, H.-Y.; Tulyakov, S.; Schwing, A. G.; and Gui, L.-Y. 2023. Sdfusion: Multimodal 3d shape completion, reconstruction, and generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 4456--4465

    work page 2023

  7. [7]

    Cong, Y.; Yi, J.; Rosenhahn, B.; and Yang, M. Y. 2023. Ssgvs: Semantic scene graph-to-video synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2555--2565

    work page 2023

  8. [8]

    Curless, B.; and Levoy, M. 1996. A volumetric method for building complex models from range images. In Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, 303--312

    1996

Show all 53 references
  1. [9]

    Dhamo, H.; Manhardt, F.; Navab, N.; and Tombari, F. 2021. Graph-to-3d: End-to-end generation and manipulation of 3d scenes using scene graphs. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 16352--16361

    2021

  2. [10]

    Engelmann, F.; Rematas, K.; Leibe, B.; and Ferrari, V. 2021. From points to multi-object 3D reconstruction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 4588--4597

    2021

  3. [11]

    A.; and Holynski, A

    Epstein, D.; Poole, B.; Mildenhall, B.; Efros, A. A.; and Holynski, A. 2024. Disentangled 3d scene generation with layout learning. arXiv preprint arXiv:2402.16936

    2024 arXiv

  4. [12]

    Fan, H.; Su, H.; and Guibas, L. J. 2017. A point set generation network for 3d object reconstruction from a single image. In Proceedings of the IEEE conference on computer vision and pattern recognition, 605--613

    2017

  5. [13]

    Fang, C.; Hu, X.; Luo, K.; and Tan, P. 2023. Ctrl-room: Controllable text-to-3d room meshes generation with layout constraints. arXiv preprint arXiv:2310.03602

    2023

  6. [14]

    Fu, H.; Jia, R.; Gao, L.; Gong, M.; Zhao, B.; Maybank, S.; and Tao, D. 2021. 3d-future: 3d furniture shape with texture. International Journal of Computer Vision, 129: 3313--3337

    2021

  7. [15]

    Hara, T.; and Harada, T. 2024. MaGRITTe: Manipulative and Generative 3D Realization from Image, Topview and Text. arXiv preprint arXiv:2404.00345

    2024 arXiv

  8. [16]

    Heusel, M.; Ramsauer, H.; Unterthiner, T.; Nessler, B.; and Hochreiter, S. 2017. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30

    2017

  9. [17]

    H \"o llein, L.; Cao, A.; Owens, A.; Johnson, J.; and Nie ner, M. 2023. Text2room: Extracting textured 3d meshes from 2d text-to-image models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 7909--7920

    2023

  10. [18]

    A.; Schmid, C.; and Fathi, A

    Hu, Z.; Iscen, A.; Jain, A.; Kipf, T.; Yue, Y.; Ross, D. A.; Schmid, C.; and Fathi, A. 2024. SceneCraft: An LLM Agent for Synthesizing 3D Scenes as Blender Code. In Forty-first International Conference on Machine Learning

    2024

  11. [19]

    Jang, E.; Gu, S.; and Poole, B. 2016. Categorical reparameterization with gumbel-softmax. arXiv preprint arXiv:1611.01144

    2016 arXiv

  12. [20]

    Johnson, J.; Gupta, A.; and Fei-Fei, L. 2018. Image generation from scene graphs. In Proceedings of the IEEE conference on computer vision and pattern recognition, 1219--1228

    2018

  13. [21]

    A.; Durand, T.; He, J.; Sigal, L.; and Mori, G

    Jyothi, A. A.; Durand, T.; He, J.; Sigal, L.; and Mori, G. 2019. Layoutvae: Stochastic scene layout generation from a label set. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 9895--9904

    2019

  14. [22]

    Koch, S.; Hermosilla, P.; Vaskevicius, N.; Colosi, M.; and Ropinski, T. 2024. Lang3DSG: Language-based contrastive pre-training for 3D Scene Graph prediction. In 2024 International Conference on 3D Vision (3DV), 1037--1047. IEEE

    2024

  15. [23]

    Liao, R.; Erler, M.; Wang, H.; Zhai, G.; Zhang, G.; Ma, Y.; and Tresp, V. 2024 a . VideoINSTA: Zero-shot Long Video Understanding via Informative Spatial-Temporal Reasoning with LLMs. arXiv preprint arXiv:2409.20365

    2024 arXiv

  16. [24]

    Liao, R.; Jia, X.; Li, Y.; Ma, Y.; and Tresp, V. 2024 b . GenTKG: Generative Forecasting on Temporal Knowledge Graph with Large Language Models. In Findings of the Association for Computational Linguistics: NAACL 2024, 4303--4317

    2024

  17. [25]

    Lin, C.; and Mu, Y. 2024. Instructscene: Instruction-driven 3d indoor scene synthesis with semantic graph prior. arXiv preprint arXiv:2402.04717

    2024 arXiv

  18. [26]

    Liu, M.; Shi, R.; Chen, L.; Zhang, Z.; Xu, C.; Wei, X.; Chen, H.; Zeng, C.; Gu, J.; and Su, H. 2024. One-2-3-45++: Fast single image to 3d objects with consistent multi-view generation and 3d diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern R...

    2024

  19. [27]

    J.; and Wonka, P

    Para, W.; Guerrero, P.; Kelly, T.; Guibas, L. J.; and Wonka, P. 2021. Generative layout modeling using constraint graphs. In Proceedings of the IEEE/CVF international conference on computer vision, 6690--6700

    2021

  20. [28]

    J.; Florence, P.; Straub, J.; Newcombe, R.; and Lovegrove, S

    Park, J. J.; Florence, P.; Straub, J.; Newcombe, R.; and Lovegrove, S. 2019. Deepsdf: Learning continuous signed distance functions for shape representation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 165--174

    2019

  21. [29]

    T.; and Mildenhall, B

    Poole, B.; Jain, A.; Barron, J. T.; and Mildenhall, B. 2022. Dreamfusion: Text-to-3d using 2d diffusion. arXiv preprint arXiv:2209.14988

    2022 arXiv

  22. [30]

    W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al

    Radford, A.; Kim, J. W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning, 8748--8763. PMLR

    2021

  23. [31]

    Ren, X.; Huang, J.; Zeng, X.; Museth, K.; Fidler, S.; and Williams, F. 2024. Xcube: Large-scale 3d generative modeling using sparse voxel hierarchies. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 4209--4219

    2024

  24. [32]

    Rombach, R.; Blattmann, A.; Lorenz, D.; Esser, P.; and Ommer, B. 2022. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 10684--10695

    2022

  25. [33]

    Ronneberger, O.; Fischer, P.; and Brox, T. 2015. U-net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer-assisted intervention--MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part II...

    2015

  26. [34]

    Rosinol, A.; Gupta, A.; Abate, M.; Shi, J.; and Carlone, L. 2020. 3D dynamic scene graphs: Actionable spatial perception with places, objects, and humans. arXiv preprint arXiv:2002.06289

    2020 arXiv

  27. [35]

    Schult, J.; Tsai, S.; H \"o llein, L.; Wu, B.; Wang, J.; Ma, C.-Y.; Li, K.; Wang, X.; Wimbauer, F.; He, Z.; et al. 2024. Controlroom3d: Room generation using semantic proxy rooms. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 6201--6210

    2024

  28. [36]

    Strader, J.; Hughes, N.; Chen, W.; Speranzon, A.; and Carlone, L. 2024. Indoor and outdoor 3d scene graph generation via language-enabled spatial ontologies. IEEE Robotics and Automation Letters

    2024

  29. [37]

    Van Den Oord, A.; Vinyals, O.; et al. 2017. Neural discrete representation learning. Advances in neural information processing systems, 30

    2017

  30. [38]

    Wald, J.; Dhamo, H.; Navab, N.; and Tombari, F. 2020. Learning 3d semantic scene graphs from 3d indoor reconstructions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 3961--3970

    2020

  31. [39]

    C.; and Liu, Z

    Wang, G.; Wang, P.; Chen, Z.; Wang, W.; Loy, C. C.; and Liu, Z. 2023. Perf: Panoramic neural radiance field from a single panorama. arXiv preprint arXiv:2310.16831

    2023 arXiv

  32. [40]

    Wang, X.; Yeshwanth, C.; and Nie ner, M. 2021. Sceneformer: Indoor scene generation with transformers. In 2021 International Conference on 3D Vision (3DV), 106--115. IEEE

    2021

  33. [41]

    Wu, Y.; Wei, P.; and Lin, L. 2023. Scene graph to image synthesis via knowledge consensus. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, 2856--2865

    2023

  34. [42]

    Xu, Y.; Chai, M.; Shi, Z.; Peng, S.; Skorokhodov, I.; Siarohin, A.; Yang, C.; Shen, Y.; Lee, H.-Y.; Zhou, B.; et al. 2023. Discoscene: Spatially disentangled generative radiance fields for controllable 3d-aware scene synthesis. In Proceedings of the IEEE/CVF conference on comp...

    2023

  35. [43]

    Yan, H.; Li, Y.; Wu, Z.; Chen, S.; Sun, W.; Shang, T.; Liu, W.; Chen, T.; Dai, X.; Ma, C.; et al. 2024. Frankenstein: Generating Semantic-Compositional 3D Scenes in One Tri-Plane. arXiv preprint arXiv:2403.16210

    2024 arXiv

  36. [44]

    Yang, G.; Huang, X.; Hao, Z.; Liu, M.-Y.; Belongie, S.; and Hariharan, B. 2019. Pointflow: 3d point cloud generation with continuous normalizing flows. In Proceedings of the IEEE/CVF international conference on computer vision, 4541--4550

    2019

  37. [45]

    Yang, Y.; Jia, B.; Zhi, P.; and Huang, S. 2024. Physcene: Physically interactable 3d scene synthesis for embodied ai. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 16262--16272

    2024

  38. [46]

    Zhai, G.; Cai, X.; Huang, D.; Di, Y.; Manhardt, F.; Tombari, F.; Navab, N.; and Busam, B. 2024 a . Sg-bot: Object rearrangement via coarse-to-fine robotic imagination on scene graphs. In 2024 IEEE International Conference on Robotics and Automation (ICRA), 4303--4310. IEEE

    2024

  39. [47]

    P.; Chen, D

    Zhai, G.; \"O rnek, E. P.; Chen, D. Z.; Liao, R.; Di, Y.; Navab, N.; Tombari, F.; and Busam, B. 2024 b . EchoScene: Indoor Scene Generation via Information Echo over Scene Graph Diffusion. arXiv preprint arXiv:2405.00915

    2024 arXiv

  40. [48]

    P.; Wu, S.-C.; Di, Y.; Tombari, F.; Navab, N.; and Busam, B

    Zhai, G.; \"O rnek, E. P.; Wu, S.-C.; Di, Y.; Tombari, F.; Navab, N.; and Busam, B. 2024 c . Commonscenes: Generating commonsense 3d indoor scenes with scene graphs. Advances in Neural Information Processing Systems, 36

    2024

  41. [49]

    Zhang, Y.; Huang, H.; Xiong, Z.; Shen, Z.; Lin, G.; Wang, H.; and Vun, N. 2024. Style-Consistent 3D Indoor Scene Synthesis with Decoupled Objects. arXiv preprint arXiv:2401.13203

    2024 arXiv

  42. [50]

    Zheng, L.; Chiang, W.-L.; Sheng, Y.; Zhuang, S.; Wu, Z.; Zhuang, Y.; Lin, Z.; Li, Z.; Li, D.; Xing, E.; et al. 2024. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36

    2024

  43. [51]

    Zhou, Y.; While, Z.; and Kalogerakis, E. 2019. Scenegraphnet: Neural message passing for 3d indoor scene augmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 7384--7392

    2019

  44. [52]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all...

  45. [53]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.