{"id":"555ff93c-0dcd-4f6d-9cbe-629e2406c827","arxiv_id":"2506.02697","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Retrieval-augmented generation, where retrieved layout templates guide a flow-matching model, improves conditional layout generation on RICO and PubLayNet.","lead":"This paper proposes LayoutRAG, a conditional layout generation model that retrieves similar layout examples from a training database based on user conditions, then uses those examples as references to guide a flow-matching generation process. The method is evaluated on the RICO and PubLayNet datasets and reports improved generation quality over several existing state-of-the-art models.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Retrieval similarity in Eq. (1)-(2) is undefined for class-only and no-condition tasks, so Table 1's C->S+P and U-Cond results do not support the main claim as written.","rationale":"The paper presents a coherent retrieval-augmented framework, and the raw-retrieval ablation in Table 2 is a valuable sanity check. However, the central claim of across-the-board improvement depends on retrieval being well-defined for every evaluated condition type. The reader's weakest-assumption analysis correctly identifies that Eq. (1)-(2) require bounding boxes that are absent in C->S+P and U-Cond. I checked the full method section and found no alternative similarity definition for these cases. This is load-bearing because two of the four headline tasks rely on it, and Section 4.3's coarse retrieval-coverage statistics do not resolve how the remaining cases are handled. A precise specification or code release, followed by a retrieval-on versus retrieval-off comparison, would settle whether the reported gains are due to the proposed mechanism or to the underlying LayoutFlow baseline. Since the reader already conditioned on this ambiguity, I do not see a reason to change the verdict: it remains CONDITIONAL rather than ACCEPT, but no stronger rejection is warranted by the available text.","tokens_in":12242,"tokens_out":4542,"duration_ms":51912,"concrete_test":"Ask the authors to release or precisely specify the retrieval procedure for C->S+P and U-Cond: if no boxes are available, state whether candidates are ranked by count-only keys, random selection, or a feature-based similarity, and how U-Cond constructs a query at all. Then re-run Table 1's C->S+P and U-Cond rows with retrieval disabled (using the LayoutFlow baseline branch) and with the specified retrieval. If the specified retrieval cannot be implemented as written, or if its removal leaves the reported FID and mIoU essentially unchanged, the headline outperformance for these tasks is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that retrieval-augmented generation outperforms state-of-the-art methods across all four conditional layout tasks. In Section 3.1, after count-based narrowing, candidate ranking is defined by Eq. (1)-(2) using IoU(e_i, e_hat_j) between matched query and candidate elements. For C->S+P, the query elements have class labels but no bounding boxes; for U-Cond, the query is empty, so the IoU term is undefined and the bipartite matching maximization cannot be evaluated. The paper never specifies an alternative similarity measure for these box-less cases. Section 4.3 also reports that only 66% of RICO test cases are retrievable by category-based retrieval, with no per-task breakdown or description of the fallback when no reference is found. Consequently, the reported FID and mIoU gains for two of the four benchmark tasks cannot be attributed to the described retrieval mechanism. This is an internal gap between the method description and the experiments, not a disagreement with external consensus. The conditional verdict is appropriate until the retrieval procedure for class-only and no-condition inputs is specified and shown to reproduce Table 1.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces LayoutRAG, a retrieval-augmented framework for content-agnostic conditional layout generation. Given a partial layout condition (class, size, or location attributes), the method retrieves similar layouts from the training database using a category-count index followed by IoU-based bipartite matching, then feeds the retrieved layout as a reference into a flow-matching generator built on LayoutFlow through a Condition-Modulated Attention module. Experiments on RICO and PubLayNet cover four tasks (C->S+P, C+S->P, Completion, U-Cond) and report FID, mIoU, Alignment, and Overlap, claiming state-of-the-art FID in most settings. Ablations analyze retrieval quality, reference quality, and fusion designs, and the paper promises code release upon acceptance.","tokens_in":12539,"tokens_out":5594,"duration_ms":54526,"significance":"If the claims hold, the paper is a useful and timely contribution: it is, to my knowledge, the first application of retrieval augmentation to content-agnostic conditional layout generation, it provides a transparent mechanism for injecting dataset knowledge at inference time, and it reports consistent FID improvements over strong flow/diffusion baselines on two standard benchmarks. The paper also includes extensive comparisons, a raw-retrieval analysis, ablations on reference quality and attention design, and a code-release commitment. The main weakness is that the retrieval similarity measure is not defined for conditions that lack bounding boxes, which affects three of the four benchmark tasks; until that gap is closed, the empirical basis for the headline claim is incomplete.","major_comments":[{"comment":"The retrieval similarity measure is undefined for the C->S+P, C+S->P, and U-Cond tasks. In C->S+P the query contains class labels but no bounding boxes; in C+S->P it contains class and size but no location; and in U-Cond the query is empty. For these cases the term IoU(e_i, e_hat_j) in Eq. (2) cannot be evaluated, so the bipartite matching problem in Eq. (1) has no well-defined input. The paper does not specify an alternative similarity or ranking for these box-less conditions, yet Table 1 reports LayoutRAG results for all three tasks and Table 2 reports Raw Retrieval results for U-Cond and C->S+P. Please specify exactly how references are retrieved, ranked, and counted for conditions without bounding boxes, or restrict the performance claims to tasks where the described similarity is actually computable.","section":"Section 3.1, Eq. (1)-(2); Table 1"},{"comment":"The description of retrieval coverage is incomplete. The text states that only 66% of RICO test cases and 99% of PubLayNet test cases can be retrieved by category-based retrieval, but no per-task breakdown is provided, and the inference-time fallback when no reference is found is never described. Since Table 1 reports aggregate metrics over the full test set, it is unclear whether non-retrievable cases are generated by the base LayoutFlow branch without any reference, or by something else. Without specifying this fallback, the reported numbers cannot be reproduced or properly interpreted. Please report per-task retrieval rates and state explicitly what the model does when retrieval fails.","section":"Section 4.3, Retrieval Result Analyses"},{"comment":"Table 2 compares Raw Retrieval with generative models only on the retrievable subset of the test data, and the LayoutRAG rows are empty for U-Cond and C->S+P. This makes the table difficult to interpret: it does not show whether LayoutRAG benefits from retrieval on tasks whose queries have no bounding boxes, nor does it explain why those entries are missing. Please either fill these entries or state explicitly that the retrieval mechanism is not applied to those tasks; otherwise the reader cannot tell which of the results in Table 1 actually exercise the proposed retrieval component.","section":"Section 4.3, Table 2"}],"minor_comments":[{"comment":"The summation indices in the two bipartite-matching constraints appear to be swapped: the first constraint should sum over i for each j, and the second should sum over j for each i, with the index ranges stated consistently.","section":"Section 3.1, Eq. (1)"},{"comment":"There are several typos that should be fixed: 'donates' should be 'denotes' in the description of C->S+P; 'content-agonistic' should be 'content-agnostic' in Section 2.3; 'poct-hoc' should be 'post-hoc' in the Conclusion; and 'yeilds' should be 'yields' in Section 4.3.","section":"Section 4.1 and elsewhere"},{"comment":"The naming of the unconditional task is inconsistent: the main text uses 'U-Cond' but Table 2 uses 'Uncond'. Please unify the terminology.","section":"Table 2"},{"comment":"The acronym mIoU is described as 'Maximum Interaction over Union', which is an unusual name for this metric. Please clarify whether it is the mean Intersection over Union over elements or the maximum-matching IoU used in earlier layout-generation papers, and define it precisely.","section":"Section 4.1, Evaluation Metrics"}],"recommendation":"major_revision","confidential_remarks":"The paper fits the journal's scope and the central idea is promising. The main issue is internal consistency rather than novelty: the similarity measure in Eq. (1)-(2) is only defined when query elements have bounding boxes, which excludes most of the evaluated tasks, and the fallback behavior for non-retrievable cases is unspecified. I would not reject on the current evidence; a major revision that clarifies these points and, if necessary, re-runs or re-frames the affected experiments would address my concerns."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This is a sensible and genuinely new idea: first to apply retrieval-augmented generation in the content-agnostic layout setting, using partial layout attributes as retrieval keys. The count-based index and the condition-modulated attention are reasonable components, and the reported FID gains over LayoutFlow on RICO are large enough to take seriously. The paper does well in showing that reference quality directly affects generation quality (Table 3), which supports the central mechanism.\n\nThe real soft spot is exactly what the stress-test flagged. The similarity measure in Eq. (1)-(2) is defined through IoU between query and candidate elements. For C->S+P, the query has only class labels, so IoU is undefined. For U-Cond, the query is empty, same issue. Table 2 even reports raw retrieval results for these tasks, so retrieval must have happened somehow, but the paper never explains how. That is not a trivial omission: it affects two of the four tasks in Table 1. Also, the 66% retrieval rate on RICO is not broken down by task, and there is no description of what the model does when no reference is retrieved. A referee will need that specification.\n\nOther weaknesses are more minor: no error bars, code \"will be released\" but not yet, and the raw-retrieval baseline is surprisingly competitive, which makes it hard to know how much of the gain comes from the reference itself versus the generative model. But the authors are honest about the raw-retrieval comparison, and the monotone relation in Table 3 suggests the mechanism is doing real work. Circularity is not a concern; retrieval from the training set is external to the held-out test evaluation.\n\nOverall, this is a solid paper with an incomplete method description, not a flawed central idea. The results probably hold up, but the missing retrieval details prevent full verification of the main claim on two of the four tasks. It deserves a serious peer review, with a request to specify the retrieval procedure for box-less and empty queries, break down the retrieval statistics by task, and release code.","headline":"A genuinely new retrieval-augmented approach to conditional layout generation with strong results, but the retrieval similarity is undefined for two of the four benchmark tasks, so the paper needs a clarifying revision before the claims are fully supported.","tokens_in":12973,"tokens_out":2739,"would_cite":true,"duration_ms":30390,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that conditional layout generation is improved by first retrieving a similar complete layout from a database and then using that reference, through Condition-Modulated Attention, to guide a flow-matching generator, and…","keywords":["layout generation","retrieval-augmented generation","conditional layout generation","flow matching","condition-modulated attention","RICO dataset","PubLayNet dataset","UI design"],"falsifier":"Run the retrieval step exactly as written on the class-only and unconditional tasks and check whether a query element without a bounding box can be passed through Eq. (2); the reported 66% (RICO) and 99% (PubLayNet) retrieval rates and the associated FID gains would require some unspecified substitute similarity for these queries.","tokens_in":12011,"feed_emoji":"🎨","tokens_out":10702,"duration_ms":98243,"temperature":0.7,"pith_summary":"LayoutRAG tries to improve controllable layout generation — arranging the bounding boxes of UI screens or document pages under partial user constraints — by making the generator look up similar complete layouts before it draws. The paper's claim is that retrieving layouts compatible with the given conditions and feeding them as references into a flow-matching generator yields layouts closer to real designs, as measured by FID and mIoU, than existing generative models on RICO and PubLayNet. If true, this matters because it turns an existing layout database into reusable memory: the condition is enriched with how similar designs actually arranged the missing attributes, rather than leaving the model to guess them. The design also allows the database to be swapped after training, so the generator can be adapted to new layout knowledge without retraining.","feed_headline":"Retrieved layouts lift layout-generation quality past prior models","feed_subtitle":"Using a layout database as reference memory, the model beats prior generators on RICO and PubLayNet.","key_machinery":"The load-bearing component is Condition-Modulated Attention (CMA), a cross-attention layer that fuses three inputs at each generation step: the current intermediate layout, the retrieved reference layout, and the user condition. CMA projects these features, combines them through matrix dot products, and applies scale and shift regressed from the time embedding, so the model can selectively borrow from the reference rather than copying it wholesale. The retrieval stage is the other essential piece: a per-category count index prunes the candidate set, and Kuhn-Munkres bipartite matching with IoU weights (Eqs. 1-2) ranks the survivors. Together they let the generator use a retrieved example as a concrete hint about how the unknown attributes are usually arranged, instead of treating the condition as the only source of information.","core_discovery":"The central claim is that conditional layout generation should be factored into two steps: retrieval and reference-guided generation. Given any subset of layout attributes (class, size, position), the method first narrows a database of training layouts using per-category element counts, then ranks candidates by an IoU-weighted bipartite match between query elements and candidate elements. The best template is passed, together with the user condition, into a Condition-Modulated Attention module that steers the flow-matching vector-field predictor, and in cases where the retrieved template already fits, it can be returned with slight modification. On RICO and PubLayNet the paper reports that this pipeline improves FID over state-of-the-art conditional layout generators on three of the four tested tasks and takes a close second on the remaining one, with higher mIoU on RICO, which it presents as evidence that retrieval is an effective conditioning mechanism.","pith_inferences":["Editorial inference: the same retrieve-then-reference recipe could transfer to other structured generation problems that have a corpus of exemplars and partial specifications, such as floor-plan design, presentation slide layouts, or scene-graph arrangement, though the paper does not test these settings.","Editorial inference: since the stated similarity metric needs bounding boxes on both sides, a class-only or unconditional query requires an unspecified fallback; the reported gains for C->S+P and U-Cond therefore rely on an implementation detail the paper does not describe.","Editorial inference: the authors' observation that irrelevant references degrade gracefully toward the base model suggests a learnable quality gate that decides between retrieval, slight modification, and pure generation could further extend the method; this is not proposed in the paper."],"forward_implications":["The threshold ablation indicates that generation quality tracks retrieval quality: on C+S->P for PubLayNet, FID improves from 1.24 when only low-similarity references are kept to 0.77 when all retrieved references are used, so better databases or retrieval should yield better layouts.","Because references come from an external database, the model can be adapted after training by replacing or extending that database, which the paper presents as a way to fix failure cases without retraining.","The raw retrieved layout itself already achieves FID comparable to or better than generative baselines on retrievable test data, so much of the method's value lies in the retrieval stage and not only in the generation stage.","The gains are largest on tasks with richer conditions (class plus size, and completion), suggesting retrieval guidance is most useful when the user's specification strongly constrains what the unknown attributes should look like."],"supporting_citations":[{"why":"supplies the base flow-matching generator, its transformer architecture, and the training objective that LayoutRAG extends with retrieval guidance.","marker":"[10]"},{"why":"provides the RICO UI-layout dataset used for training, as the retrieval database, and as an evaluation benchmark.","marker":"[8]"},{"why":"provides the PubLayNet document-layout dataset used for training, retrieval, and evaluation.","marker":"[43]"},{"why":"a state-of-the-art discrete diffusion baseline whose evaluation split and FID feature network the comparison adopts.","marker":"[40]"},{"why":"a state-of-the-art diffusion baseline using guidance-based conditioning, which the paper contrasts with retrieval-augmented conditioning.","marker":"[15]"},{"why":"a state-of-the-art autoregressive conditional baseline that contributes the data split used for training and evaluation.","marker":"[16]"},{"why":"defines the conditional flow-matching objective from which the retrieval-augmented loss in Eq. (5) is derived.","marker":"[22]"},{"why":"supplies the weighted bipartite matching formulation that the retrieval stage adapts to measure layout similarity in Eqs. (1)-(2).","marker":"[30]"}],"fun_headline_variants":["Retrieval-augmented layout gen beats SOTA on RICO and PubLayNet","LayoutRAG: Borrowing layout templates to guide generation","Retrieve then generate: new SOTA for conditional layouts","Conditional layout generation gets a retrieval boost","Retrieval guides flow-matching for better layout generation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The approach depends on being able to find a similar layout from the database even when the condition gives no bounding boxes, but the paper's stated similarity measure compares bounding boxes and is undefined when either side has none to compare.","fun_headline_variants_meta":{"raw":{"variants":["Retrieval-augmented layout gen beats SOTA on RICO and PubLayNet","LayoutRAG: Borrowing layout templates to guide generation","Retrieve then generate: new SOTA for conditional layouts","Conditional layout generation gets a retrieval boost","Retrieval guides flow-matching for better layout generation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000379,"raw_usage":{"total_tokens":2011,"prompt_tokens":935,"completion_tokens":1076,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":551,"completion_tokens_details":{"reasoning_tokens":990}},"tokens_in":551,"tokens_out":1076,"duration_ms":7651,"temperature":1.0,"reasoning_tokens":990,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T11:17:46.831551+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the retrieval step exactly as written on the class-only and unconditional tasks and check whether a query element without a bounding box can be passed through Eq. (2); the reported 66% (RICO) and 99% (PubLayNet) retrieval rates and the associated FID gains would require some unspecified substitute similarity for these queries.","supporting_citations":[{"cited_title":"Layout- flow: flow matching for layout generation","cited_arxiv_id":null,"evidence_quote":"supplies the base flow-matching generator, its transformer architecture, and the training objective that LayoutRAG extends with retrieval guidance."},{"cited_title":"Rico: A mobile app dataset for building data-driven design applications","cited_arxiv_id":null,"evidence_quote":"provides the RICO UI-layout dataset used for training, as the retrieval database, and as an evaluation benchmark."},{"cited_title":"Publaynet: largest dataset ever for document layout analysis","cited_arxiv_id":null,"evidence_quote":"provides the PubLayNet document-layout dataset used for training, retrieval, and evaluation."},{"cited_title":"Layoutdiffusion: Improving graphic layout generation by discrete diffusion probabilistic models","cited_arxiv_id":null,"evidence_quote":"a state-of-the-art discrete diffusion baseline whose evaluation split and FID feature network the comparison adopts."},{"cited_title":"Layoutdm: Discrete diffusion model for controllable layout generation","cited_arxiv_id":null,"evidence_quote":"a state-of-the-art diffusion baseline using guidance-based conditioning, which the paper contrasts with retrieval-augmented conditioning."},{"cited_title":"Layoutformer++: Conditional graphic layout generation via constraint serialization and decoding space restriction","cited_arxiv_id":null,"evidence_quote":"a state-of-the-art autoregressive conditional baseline that contributes the data split used for training and evaluation."},{"cited_title":"Optimal correction cost for object detection evaluation","cited_arxiv_id":null,"evidence_quote":"supplies the weighted bipartite matching formulation that the retrieval stage adapts to measure layout similarity in Eqs. (1)-(2)."}],"review_version":1}