{"id":"b71509d6-5ac0-400f-bdf8-a4360642dfa8","arxiv_id":"2608.01106","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Feeding scene-graph embeddings into a frozen LLM through a two-stage alignment and LoRA pipeline improves layout accuracy in relation-dense scenes, at the cost of small losses on simple two-object layouts.","lead":"SG-Layout trains an LLM to generate object layouts by feeding it structured scene graphs along with text instructions, using a two-stage alignment and LoRA tuning pipeline. It shows gains over the Qwen3-8B backbone in complex indoor scenes, though simple two-object image layouts are handled better by text-only LoRA tuning.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"PSA and graph-derived metrics may reward input graph quality rather than learned spatial reasoning; graph-error ablation is missing.","rationale":"The reader's weakest assumption is exactly the point that matters most: the scene graph parser's quality and its role in both conditioning and evaluation. My stress-test pass looked for internal inconsistencies in the method description, loss functions, and tables; the method itself is coherent (two-stage training, frozen backbone, LoRA, RGT encoder, cross-entropy loss in Eq. 12 is standard). The main internal tension I found is that the abstract and conclusion state a blanket improvement, while Table 4 shows Qwen3+LoRA outperforming SG-Layout at K=2 and K=4 (92.58 vs 89.85 and 73.0 vs 71.5). The authors themselves nuance this in §4.2, saying SG-Layout becomes more beneficial as relation density increases, so the strongest claim should be read as qualified. The deeper load-bearing issue remains the parser/PSA confound, which is not addressed by any ablation or error analysis. I do not see a fatal flaw; the conditional verdict is right. The concrete check I propose is targeted: human-verified gold graphs and graph-noise injection would separate input quality from model capability. That experiment is feasible with the described datasets and would settle whether the PSA gain is a real learning effect or a parser-consistency artifact.","tokens_in":11502,"tokens_out":2379,"duration_ms":18524,"concrete_test":"Build a gold set of 100 test instructions with human-verified scene graphs and human-verified spatial relations. Evaluate SG-Layout under three conditions: (a) human-correct graphs, (b) parser graphs, (c) parser graphs with 10-30% random edge flips or node substitutions. Compute PSA against the human-verified relations for all conditions. If SG-Layout's PSA drops sharply with graph noise, or if its advantage over Qwen3+LoRA disappears under human-correct graphs, the measured advantage is largely an artifact of parser self-consistency rather than learned spatial reasoning. Also report parser node/edge precision and recall against the human labels.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that SG-Layout improves spatial reasoning accuracy and geometric consistency over the compact open-source backbone, with clear advantages in relation-dense scenes. The reader flags that the scene graphs used for training and inference, generated by GPT-4o and a VLM with manual correction, must be accurate enough to serve as ground-truth relational structure. I agree, and I want to sharpen the concern: the headline metric PSA (Eq. 2, §4.1) measures the percentage of ground-truth directional relations preserved in the predicted layout. If the inference-time scene graph comes from the same or similar parser family as the ground-truth annotation (a VLM or GPT-4o derivation), then PSA can reward parser self-consistency rather than true spatial semantics: the model is trained and evaluated on the parser's interpretation of relations, while the text-only baseline must reconstruct that relational structure from language alone. Table 3's 66.2% vs 56.7% PSA gap may thus reflect the parser's own consistency with its annotations, not a learned spatial-reasoning gain. The paper provides no graph-quality measurement (no parser accuracy against human labels) and no error-injection ablation (no experiments degrading or corrupting the input graph). Without these, the claim that graph conditioning itself improves spatial reasoning is under-supported. This is not a fatal flaw, but it is the load-bearing weak point.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SG-Layout, a two-stage framework that injects structured scene-graph information into a frozen Qwen3-8B LLM. In Stage 1 a relational graph encoder and projector are trained to align graph embeddings with the LLM's token space; in Stage 2 LoRA adapters are tuned for instruction-driven layout generation while the graph modules stay frozen. The method is evaluated on 2D image layout generation, 3D indoor scene synthesis, and object rearrangement, with controlled ablations against Qwen3 and Qwen3+LoRA and reference comparisons against LayoutGPT, SKE-Layout, and other systems. The central claim is that graph conditioning improves spatial reasoning accuracy and geometric consistency, with the largest gains in relation-dense and compositionally complex scenes.","tokens_in":11792,"tokens_out":5250,"duration_ms":45085,"significance":"If the central claim holds, the result is valuable: it demonstrates a parameter-efficient way to inject explicit relational structure into an LLM for layout generation, a task where text-only LLMs are known to be weak. The controlled experimental design is a strength: comparisons against Qwen3+LoRA under the same backbone isolate the effect of graph conditioning, and the metrics (PSA, IoU, OOB, collision, position/rotation error, F1) are external geometric quantities rather than model-generated judgments. The graph-encoder ablation in Table 5 is also a useful internal check. The paper's main limitation is that the reliability of the scene graphs themselves is never measured or stress-tested, and the reported differences lack statistical support, so the size of the claimed advantage is not yet firmly established.","major_comments":[{"comment":"The load-bearing premise of the central claim is that the scene graphs used at training and inference time are accurate enough to serve as ground-truth relational structure. Section 3.2 states that training graphs are produced by a VLM 'followed by manual verification,' while Section 3.3 states that inference graphs are generated by GPT-4o 'under rule guidance'; the paper reports no parser accuracy against human labels, no graph-error statistics, and no corruption/error-injection ablation. Because PSA (Eq. 2, §4.1) measures preservation of 'ground-truth directional relations,' and because those relations are the same relations that the graph tokenizer exposes to the model at inference, the 66.2% vs 56.7% PSA gap in Table 3 could partly reflect the model copying relations from the input graph rather than learning spatial reasoning. Please add a graph-quality evaluation on a human-verified sample and an ablation with degraded or randomly corrupted graphs (e.g., edge removal, node substitution) to show that the method degrades gracefully and that the reported advantage is not an artifact of parser self-consistency.","section":"§3.2, §3.3, §4.1 (Table 3)"},{"comment":"All comparisons are single-run point estimates without error bars, significance tests, or multiple-seed reporting. This matters because the controlled advantages are not uniform: in Table 4, Qwen3+LoRA outperforms SG-Layout at K=2 (92.58 vs 89.85) and K=4 (73.0 vs 71.5), and in Table 3 the collision rates are effectively tied (19.0% vs 19.2%). The paper should report mean and standard deviation over at least three runs and, if feasible, a paired significance test over matched prompts; without this, the claim that graph conditioning 'provides the largest gains in relation-dense scenes' is not statistically supported.","section":"§4.2 (Tables 3 and 4)"},{"comment":"The two-stage training paradigm is presented as a core contribution, but no ablation isolates the contribution of Stage 1 (graph-language feature alignment). The only architectural ablation in the paper is the graph-encoder study in Table 5. Please add an experiment training Stage 2 without Stage 1 (e.g., with a randomly initialized or untrained projector) and, ideally, a quantitative evaluation of the Stage-1 scene-description objective. Without this, the claim that the alignment stage specifically is necessary for the observed gains is not tested.","section":"§3.5, §4.2"},{"comment":"The manuscript does not report hyperparameters, optimizer settings, learning rates, LoRA rank and alpha, projector architecture, graph-token length and sequence position, or the train/validation splits for MSCOCO-Subset, 3D-FRONT/FUTURE, and SK-Dataset, and it does not release code. Because the contribution is a training paradigm, these details are necessary to reproduce or adjudicate the results. Please include a full implementation appendix or release code and checkpoints.","section":"§3.2, §4.1"}],"minor_comments":[{"comment":"The figure block in Section 4.3 appears to contain repeated example panels and near-identical captions for Fig. 4 and Fig. 5 in the provided text; please check the final PDF and ensure each figure shows the intended distinct 2D and 3D examples.","section":"§4.3 (Figures)"},{"comment":"There are textual typos in the example prompts, most noticeably 'a table and a chair are on the left front of of the bed' and 'left front of of the bed'; these should be corrected.","section":"Fig. 1 and §4.3"},{"comment":"For DiffuScene and InstructScene, several entries are marked '–' (OOB, collision, Pos., Rot., F1) without explanation; please state explicitly which metrics were not reported by these baselines, or provide the missing numbers if available.","section":"Table 3"},{"comment":"NSR-1K is listed in Table 2 with 39,436 examples, but no corresponding result table or experiment is presented in Section 4.2; please clarify its exact role in the evaluation.","section":"§4.1, Table 2"},{"comment":"The table lists AUC only for K=2 for LayoutGPT (GPT-3.5) and omits K=4, K=6, and K=8; the reader should be told whether those values were not reported or not computed.","section":"Table 4"}],"recommendation":"major_revision","confidential_remarks":"The evaluation uses SK-Dataset, introduced in the authors' own SKE-Layout paper, and the scene-graph annotations are generated by a VLM/GPT-4o pipeline from the same group. This is not evidence of misconduct, but it does make the parser-self-consistency concern more salient. I would encourage the editor to treat the requested graph-quality evaluation and error-injection ablation as essential for the revised version."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Know this paper for one thing: it is a clean, controlled demonstration that feeding scene-graph tokens into a frozen Qwen3-8B, via a relational graph encoder plus projector plus LoRA, improves layout generation—especially when scenes are relation-dense. That is a useful incremental result, not a breakthrough.\n\nWhat is actually new: the two-stage pipeline (first align graph embeddings with language by training the encoder/projector to describe scenes, then LoRA-tune the LLM for layout output) and the empirical finding that graph conditioning helps most at 6/8 objects in image layout and in 3D room synthesis. The controlled ablations against Qwen3 and Qwen3+LoRA are the right design; comparing to GPT-4-based systems is honestly labeled as reference, not head-to-head.\n\nCredit where due: Table 3 shows SG-Layout beats the strong Qwen3+LoRA baseline on PSA, OOB, Pos., Rot., and F1, and the encoder ablation (R-GCN vs GAT vs RGT) shows the choice matters. The paper is clearly written and does not oversell against GPT-4 systems.\n\nNow the soft spots. The biggest one is the missing analysis of input graph quality. The pipeline parses user instructions with GPT-4o at inference and builds training graphs with a VLM plus manual correction. There is no parser accuracy number and no error-injection ablation, so we do not know how much the gains depend on correct parsing or whether the model is robust to noisy or wrong edges. I read the PSA metric as less circular than it might appear: if the inference graph is wrong relative to ground truth, satisfying it should hurt PSA, not help. But the softer version of the concern stands—graph conditioning could be outsourcing relation recognition to the parser, and the paper does not measure that. This needs an ablation or at least parser accuracy reporting.\n\nSecond, Table 4 is genuinely mixed: at K=2 and K=4, text-only LoRA has higher AUC; graph wins at K=6/8 along with IoU and CLIP. The abstract's blanket \"improves over the backbone\" is too broad without the complexity caveat. Third, no error bars, significance tests, code, or exact dataset construction protocol; replication would require contacting authors. That is a real limitation for a paper whose main evidence is empirical.\n\nWho is this for? People working on LLM-based layout generation, indoor scene synthesis, or spatial reasoning in VLMs. It deserves peer review; it is a serious, honest empirical study with a clear experimental design. I would not cite it in my own work this year, but I would send it to a competent referee, mainly to push for the graph-error analysis.","headline":"A credible, incremental graph-conditioned layout generation paper; the core result holds up best in relation-dense scenes, and the main gaps are missing robustness analysis and artifacts, not fatal correctness.","tokens_in":12302,"tokens_out":2981,"would_cite":false,"duration_ms":26709,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SG-Layout claims that injecting scene-graph tokens into a frozen LLM yields more spatially accurate and physically consistent layouts than text-only tuning of the same backbone.","keywords":["scene graphs","layout generation","large language models","spatial reasoning","graph-language alignment","LoRA","indoor scene synthesis","object rearrangement"],"falsifier":"Re-run the indoor scene synthesis and rearrangement evaluations with a separately verified gold set of scene graphs, and also with randomly shuffled graph edges at inference. If shuffled or corrected graphs produce the same PSA and out-of-bound numbers as the original graphs, then the gain is not caused by the graph content, and the two-stage alignment claim is falsified.","tokens_in":11294,"feed_emoji":"🛋️","tokens_out":4933,"duration_ms":39689,"temperature":0.7,"pith_summary":"SG-Layout claims that explicit structured spatial knowledge, fed into a frozen instruction-tuned LLM as scene-graph tokens, produces more spatially accurate and physically consistent layouts than tuning the same LLM on text alone. The paper argues that natural-language instructions are inherently ambiguous about object relations, and that scene graphs supply the missing compositional structure. It reports consistent gains on image layout generation, 3D indoor scene synthesis, and robotic object rearrangement, with the largest improvements in relation-dense scenes. A sympathetic reader would take this as evidence that graph-conditioned token injection is a viable parameter-efficient route to spatial reasoning in compact open-source LLMs.","feed_headline":"Scene-graph tokens lift frozen-LLM layout accuracy","feed_subtitle":"On 3D bedroom synthesis, relation preservation rises from 56.7% to 66.2% over text-only LoRA.","key_machinery":"The load-bearing mechanism is the scene graph token, produced by a relational graph transformer (RGT) encoder followed by a projector that maps graph node and edge embeddings into the LLM's token-embedding space. These graph tokens are prepended to the text tokens of the user instruction, and the concatenated sequence is decoded by Qwen3-8B with LoRA adapters. A two-stage training schedule aligns the graph encoder and projector first by asking the frozen LLM to describe a room from graph tokens, then fine-tunes only the LoRA parameters to emit serialized layouts. The design keeps the backbone frozen throughout, so any gain is attributed to the injected graph structure rather than to re-training the language model.","core_discovery":"The central claim is that a relational graph encoder plus a learned projector can map scene-graph embeddings into the linguistic latent space of a frozen LLM, and that once aligned, those graph tokens condition layout decoding better than text-only instruction tuning. On the indoor scene synthesis benchmark, SG-Layout raises relation-preservation accuracy (PSA) to 66.2% versus 56.7% for Qwen3+LoRA and 23.5% for the bare backbone, while cutting the out-of-bound rate from 57.2% to 43.5%. In 2D image layout it beats both Qwen3 variants at six and eight objects and improves IoU and CLIP scores. The authors frame the result as a controlled comparison within the Qwen3 family, treating GPT-4-based and diffusion baselines as reference points rather than matched training protocols.","pith_inferences":["The authors' reliance on GPT-4o and a vision-language model with manual correction to build training scene graphs means the method's ceiling is partly set by parser accuracy; a natural stress test is training with noisy or automatically parsed graphs and measuring how much of the PSA gain survives.","The same graph-token injection could transfer to other geometry-conditioned generation tasks, such as floorplan synthesis or the millimeter-tolerant surgical-tray layouts the authors list as future work.","Because scene graphs are a controllable input, editing nodes or edges at inference should yield predictable layout changes; that would make SG-Layout a testable interface for interactive layout editing, a property the paper does not directly evaluate."],"forward_implications":["At six and eight objects in image layout generation, graph conditioning yields higher spatial-relation accuracy than text-only LoRA on the same frozen backbone.","In 3D indoor scene synthesis, SG-Layout improves relation preservation and reduces out-of-bound placements relative to text-only LoRA tuning.","The advantage of graph conditioning grows as relation density and scene complexity increase, while simple two-object prompts can still favor text-only tuning.","Because the backbone stays frozen, the two-stage alignment recipe can be transferred to other instruction-following LLMs without full fine-tuning."],"supporting_citations":[{"why":"Supplies LoRA, the low-rank adaptation method used to fine-tune the frozen backbone.","marker":"[9]"},{"why":"Supplies the relational graph transformer (RGT) used as the graph encoder.","marker":"[3]"},{"why":"Supplies Qwen3-8B, the frozen backbone LLM in all controlled comparisons.","marker":"[33]"},{"why":"Provides LayoutGPT as a reference baseline and the NSR-1K prompts for spatial reasoning evaluation.","marker":"[5]"},{"why":"Provides SKE-Layout as a retrieval-augmented reference baseline and the SK-Dataset for rearrangement evaluation.","marker":"[30]"},{"why":"Provides MSCOCO, the source of the curated 2D image-layout subset.","marker":"[16]"},{"why":"Provides the 3D-FRONT/3D-FUTURE datasets used for indoor scene synthesis.","marker":"[6]"},{"why":"Provides DiffuScene as a reference baseline for 3D indoor scene synthesis.","marker":"[27]"},{"why":"Provides LLM-GROP as a reference baseline for object rearrangement.","marker":"[2]"},{"why":"Provides InstructScene as a graph-conditioned reference baseline for 3D scene synthesis.","marker":"[15]"}],"fun_headline_variants":["Scene-graph alignment lifts relation accuracy to 66%","Graph-aligned frozen LLMs master dense scene layouts","Scene graphs give frozen LLMs spatial layout sense","From text to scene graphs: LLMs place objects better","Scene graphs inject geometric smarts into frozen LLMs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the scene graphs used for training and evaluation—generated by GPT-4o or a vision-language model with manual correction—are accurate enough to serve as ground truth; if these graphs contain wrong nodes or edges, the model is trained on incorrect relations and the reported advantage reflects graph quality rather than learned spatial reasoning.","fun_headline_variants_meta":{"raw":{"variants":["Scene-graph alignment lifts relation accuracy to 66%","Graph-aligned frozen LLMs master dense scene layouts","Scene graphs give frozen LLMs spatial layout sense","From text to scene graphs: LLMs place objects better","Scene graphs inject geometric smarts into frozen LLMs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001868,"raw_usage":{"total_tokens":7310,"prompt_tokens":904,"completion_tokens":6406,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":520,"completion_tokens_details":{"reasoning_tokens":6329}},"tokens_in":520,"tokens_out":6406,"duration_ms":36688,"temperature":1.0,"reasoning_tokens":6329,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T15:11:44.184925+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the indoor scene synthesis and rearrangement evaluations with a separately verified gold set of scene graphs, and also with randomly shuffled graph edges at inference. If shuffled or corrected graphs produce the same PSA and out-of-bound numbers as the original graphs, then the gain is not caused by the graph content, and the two-stage alignment claim is falsified.","supporting_citations":[{"cited_title":"In: 2023 IEEE/RSJ International Con- ference on Intelligent Robots and Systems (IROS)","cited_arxiv_id":null,"evidence_quote":"Provides LLM-GROP as a reference baseline for object rearrangement."},{"cited_title":"In: Proceedings of the Computer Vision and Pattern Recognition Conference","cited_arxiv_id":null,"evidence_quote":"Provides SKE-Layout as a retrieval-augmented reference baseline and the SK-Dataset for rearrangement evaluation."}],"review_version":2}