{"id":"95da88d2-3d17-4358-8f1f-9cfda50d3880","arxiv_id":"2412.13195","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A data-curation engine plus a token-order attention injection module raises spatial accuracy of Stable Diffusion and FLUX models on standard benchmarks.","lead":"This paper introduces CoMPaSS, a two-part method that makes text-to-image AI models place objects in the correct positions described by text prompts. It curates clean spatial training data from COCO and injects token-order information into attention layers, improving spatial benchmarks by large margins.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Benchmark gains may reflect distribution matching: SCOP is built from COCO categories and binary viewer-centric relations, the same distribution as VISOR/GenEval/T2I-CompBench, while Sec. 5 concedes context-dependent and object-centric spatial language is not handled.","rationale":"The reader's weakest assumption identifies the same load-bearing concern: benchmark improvements may not generalize beyond the SCOP training distribution. This is the single most important threat to the paper's central claim because the claimed gains are exactly on benchmarks whose object vocabulary and spatial predicates match SCOP. Independent support exists: the ablations show both SCOP and TENOR contribute, the data-efficiency experiment is convincing, and the code is released. Yet these strengths do not address the generalization question. The paper itself flags in Sec. 5 that context-dependent and object-centric spatial relations are outside its scope, so the general 'spatial understanding' phrasing overstates what is demonstrated. The absence of error bars and the tuning of SCOP thresholds on the evaluation benchmarks further weaken the quantitative margin. A concrete OOD evaluation with non-COCO categories and more varied spatial language would settle whether the method captures a transferable spatial prior or mainly matches the curated distribution. Since the reader already conditions acceptance on this issue, my read does not change the verdict.","tokens_in":22266,"tokens_out":3699,"duration_ms":36226,"concrete_test":"Build an out-of-distribution spatial evaluation set using (1) object categories not in COCO (e.g., 'diamond ring', 'fishing rod', 'cactus'), (2) context-dependent prepositions such as 'near', 'inside', and 'between', and (3) object-centric phrases like 'to the person's right'. Generate with FLUX.1 and FLUX.1+CoMPaSS under identical sampling seeds, score with the same VISOR/GenEval pipeline, and report the per-category breakdown. If the relative gain on non-COCO categories or on context-dependent terms is near zero, the general spatial-understanding claim is not supported; if the gain persists, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that CoMPaSS enhances spatial understanding in T2I models rests on large relative gains on VISOR (+98%), T2I-CompBench Spatial (+67%), and GenEval Position (+131%). These benchmarks test almost exactly the distribution SCOP was curated from: COCO's 80 object categories (SCOP uses COCO training-split bounding boxes) and simple binary viewer-centric relations (left/right/above/below, plus diagonals). SCOP emits 8 spatial tokens, and its decoding templates match the linguistic form of the benchmark prompts. The paper's own Sec. 5 admits it does not handle context-dependent spatial language (e.g., size-disparity-affected 'above'/'beside') or object-centric frames ('to the child's right hand side'). Thus the gains are consistent with matching the curated training distribution rather than acquiring a general spatial competence. This is compounded by the absence of error bars and by hyperparameter selection (Sec. 4.3, Tab. 5) performed on these same benchmarks, so part of the reported margin could reflect selection bias. The appendix's three-object evaluation is still COCO-only and uses the same relational vocabulary, so it does not resolve the out-of-distribution question.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper identifies two causes of poor spatial-relation generation in text-to-image (T2I) diffusion models: ambiguous spatial captions in existing image-text datasets and loss of token-ordering information in text encoders. It proposes CoMPaSS, composed of SCOP, a constraint-based data engine applied to the COCO training split that extracts object pairs satisfying visual significance, semantic distinction, spatial clarity, minimal overlap, and size balance, and TENOR, a parameter-free module that injects positional encodings into the text-image attention keys and queries. Experiments on SD1.4, SD1.5, SD2.1, and FLUX.1 report large gains on VISOR (+98%), T2I-CompBench Spatial (+67%), and GenEval Position (+131%), together with improved overall scores on DPG-Bench and fidelity metrics, at low computational overhead and with promising data efficiency.","tokens_in":22564,"tokens_out":5962,"duration_ms":57160,"significance":"If the results hold, CoMPaSS is a practical and lightweight recipe for improving spatial compliance of open-weight T2I models: it adds no trainable parameters, requires only a brief fine-tuning phase, works across UNet and MMDiT architectures, and its data engine is simple and reproducible. The strengths of the paper are the systematic threshold-based curation pipeline, the sensible diagnostic in Table 1 showing that text encoders fail to rank logically equivalent spatial paraphrases as most similar, and the ablations in Tables 5 and 6 showing that both components contribute and that performance scales with training data. However, because SCOP and the headline benchmarks share the COCO category vocabulary and a small set of binary viewer-centric spatial relations, the evidence as presented is stronger for distribution-matched spatial compliance than for a general spatial-understanding capability. The broad claim of generalization needs an out-of-distribution evaluation and error bars before the conclusion is fully supported.","major_comments":[{"comment":"The evidence for the paper's main claim that CoMPaSS enhances spatial understanding generally is currently confined to the same distribution used to build SCOP. SCOP curates pairs from the COCO training split and encodes only eight spatial tokens (<left>, <right>, <above>, <below>, and four diagonals), while the three headline benchmarks evaluate simple binary viewer-centric relations over essentially the same object vocabulary. The paper's own Sec. 5 lists context-dependent spatial language and object-centric frames as unsupported. Without an evaluation on categories and relation types outside this closed vocabulary, the large relative gains are equally consistent with distribution matching. Please add such an out-of-distribution test, or revise the conclusion to claim improved spatial compliance on this benchmark distribution rather than general spatial understanding.","section":"Sec. 3.1, Sec. A.1, Tab. 2"},{"comment":"The SCOP thresholds tau_v, tau_u, tau_o, and tau_s are selected by grid search on the same benchmarks that produce the headline SOTA numbers, and no repeated-seed or bootstrap intervals are reported for any accuracy result in Tabs. 2, 4, 5, 6, or A8-A11. This makes it impossible to quantify how much of the reported margin is selection bias and leaves the true improvement over baselines uncertain. Please provide confidence intervals and a validation split that is not used for threshold selection.","section":"Sec. 4.3, Tab. 5"},{"comment":"The abstract and Sec. 4.2 state that gains are achieved 'without compromising general generation capabilities,' but the per-task breakdown in Tab. A11 shows several non-spatial tasks degrading: SD2.1+CoMPaSS drops GenEval Color from 0.85 to 0.71 and Count from 0.44 to 0.20, and SD1.5+CoMPaSS drops DPG-Bench Other from 67.81 to 60.80. Since overall scores can mask these trade-offs, the no-compromise claim should be made conditional on aggregate metrics or accompanied by a per-task analysis of which capabilities are preserved and which are not.","section":"Tab. A11 vs. abstract/Sec. 4.2"}],"minor_comments":[{"comment":"The thresholds are introduced as 'principled constraints' but are free parameters; please justify the chosen values or soften the terminology, and state how the resulting dataset size varies with each threshold.","section":"Sec. 3.1, Eqs. (1)-(5)"},{"comment":"The proxy task tests nearest-neighbor ranking among four prompt variations; please clarify how ties are handled and report per-relation results, since 'above'/'below' may behave differently from 'left'/'right'.","section":"Tab. 1"},{"comment":"The human validation reports an 85.2% agreement rate but does not state the number of annotators, the number of items judged, or the exact instructions given; please add this information for reproducibility.","section":"Sec. 3.1"},{"comment":"The size-disparity fine-tuning experiment is described only with one qualitative example; provide the training protocol and quantitative results, or label it explicitly as preliminary.","section":"Sec. 5, Fig. 7"},{"comment":"The latency overhead table reports mean +/- SD but not the number of measurement repetitions or the hardware conditions; please state the measurement protocol so the overhead numbers can be reproduced.","section":"Appendix B"}],"recommendation":"major_revision","confidential_remarks":"This is a useful engineering contribution with a plausible mechanism and reproducible code. The main risk is overclaiming: the benchmark gains may be substantially explained by distribution overlap between SCOP and the evaluation benchmarks. I would not treat that overlap as fatal, but I would require an out-of-distribution evaluation and error bars before accepting the general 'spatial understanding' claim. The per-task trade-offs in Tab. A11 should also be acknowledged in the final version."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about this paper. First, the combination of a constraint-based data-cleaning step and a parameter-free token-order injection is genuinely new, and the experiments are careful enough to show that both pieces matter. Second, the headline gains (VISOR +98%, T2I-CompBench Spatial +67%, GenEval Position +131% on FLUX.1) are large, but they likely reflect distribution matching as much as genuine understanding: SCOP is built from the COCO category vocabulary and the same binary left/right/above/below relations those benchmarks evaluate.\n\nWhat the paper does well: the SCOP engine is simple and principled—filtering COCO pairs on visual significance, semantic distinction, spatial clarity, minimal overlap, and size balance yields 28k clean pairs, and the 85.2% human agreement check is a nice sanity signal. TENOR is a small idea (inject positional encodings into cross-attention keys, and queries for MMDiT), but it is parameter-free, adds negligible overhead, and the ablations show it improves beyond SCOP alone, especially on unseen prompts. The data-efficiency comparison against SPRIGHT (500 images beating SPRIGHT's 500) is a strong practical argument. The Table 1 encoder diagnostic—CLIP and T5 almost never retrieve the logically equivalent rephrasing—is a useful observation on its own.\n\nThe soft spots are real but not fatal. The benchmark overlap is the main one: SCOP is COCO-only and the decoding templates produce prompts like \"a cup on top of a couch,\" which is the same distribution as VISOR, GenEval Position, and T2I-CompBench Spatial. The paper's own limitations section concedes that context-dependent relations (size-disparity \"above\") and object-centric frames (\"to the child's right hand side\") are not handled. So the general spatial-understanding claim should be read as \"enhanced performance on this specific relational vocabulary\" unless out-of-distribution evaluation is added. Second, the SCOP thresholds were grid-searched on these same benchmarks (Sec. 4.3, Tab. 5), and no error bars are reported, so some margin could be selection noise; the ablation shows robustness across nearby values, which softens this. Third, the UNet results incorporate attention supervision from TokenCompose, so the full recipe is SCOP+TENOR+that prior supervision.\n\nAll in all, this is a competent, honest paper with a credible practical contribution. It deserves a serious referee. I would ask the authors for out-of-distribution evaluation (new categories, object-centric or context-dependent prompts) and for multiple seeds or error bars before accepting the generalization claim at face value. For the narrower claim—cheap and effective spatial-compliance finetuning—the evidence is solid.","headline":"A well-executed recipe for improving spatial compliance on COCO-style benchmarks, but the headline gains track the training distribution more than they prove a general spatial understanding.","tokens_in":23066,"tokens_out":2777,"would_cite":true,"duration_ms":26077,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CoMPaSS claims that spatial failures in text-to-image diffusion models are fixable by pairing curated spatial data with token-order reinjection, achieving up to +131% relative gains on GenEval Position across four open-weight models.","keywords":["text-to-image diffusion models","spatial relationships","data curation","text encoder limitations","token ordering","cross-attention injection","COCO","benchmark evaluation"],"falsifier":"Evaluate CoMPaSS against its base models on a held-out benchmark built from non-COCO object categories (for example, 'a wrench to the left of a screwdriver') or object-centric spatial language ('to the child's right hand side'); if accuracy returns to baseline levels, the gains are distribution matching and the claim of enhanced general spatial understanding fails.","tokens_in":22112,"feed_emoji":"🎯","tokens_out":6945,"duration_ms":61019,"temperature":0.7,"pith_summary":"The paper claims that text-to-image diffusion models fail at spatial relations for two fixable reasons: existing datasets contain ambiguous or contradictory spatial language, and standard text encoders (CLIP, T5) collapse prompts that differ only in token order into nearly identical embeddings. CoMPaSS attacks both causes with a data engine, SCOP, that mines only visually clear, size-balanced object pairs from COCO images, and a parameter-free module, TENOR, that re-injects positional ordering into every text-image attention layer. Applied to SD1.4, SD1.5, SD2.1, and FLUX.1, it reports state-of-the-art spatial accuracy, with relative gains of +98% on VISOR, +67% on T2I-CompBench Spatial, and +131% on GenEval Position, while also improving overall alignment and image fidelity. If the results hold, spatial control is a data-and-conditioning problem solvable by light fine-tuning of existing open-weight models rather than by architectural redesign.","feed_headline":"Spatial accuracy of AI image generators up 131 percent","feed_subtitle":"A data-filtering step and token-order injection fix left/right/above/below in four open diffusion models.","key_machinery":"Two components carry the argument. SCOP (Spatial Constraints-Oriented Pairing) is a data engine that enumerates object pairs in an image and keeps only those passing five geometric constraints—visual significance, semantic distinction, spatial clarity, minimal overlap, and size balance—then decodes the surviving pairs into image crops paired with templated spatial captions. TENOR (Token ENcoding ORdering) is a parameter-free module that adds sinusoidal positional encodings to the key vectors in UNet cross-attention and to the text query/key vectors in MMDiT blocks, making token order visible at every attention step so structurally different prompts produce different conditioning signals.","core_discovery":"CoMPaSS establishes that injecting token-order information into the text-image attention of diffusion models, in combination with training on a small set of spatially unambiguous image-text pairs, makes both UNet-based and MMDiT-based text-to-image models substantially better at rendering left, right, above, and below relations. The paper's central claim is that the two interventions are complementary: SCOP supplies clean spatial supervision that was missing from web-scale training data, and TENOR provides the structural signal that lets the model tell 'A left of B' from 'B left of A', which standard encoders fail to preserve. On FLUX.1 the combination lifts VISOR from 37.96 to 75.17, T2I-CompBench Spatial from 0.18 to 0.30, and GenEval Position from 0.26 to 0.60, with no trainable parameters added at inference time.","pith_inferences":["Every reported benchmark shares SCOP's own COCO vocabulary and binary relation set, so the true test of general spatial understanding would be an out-of-distribution probe with non-COCO objects or context-dependent spatial language; the paper does not provide one.","The same token-order blindness that scrambles left/right also plausibly degrades attribute binding and other order-sensitive compositions, so TENOR may transfer to color, size, and count tasks—an untested implication of the paper's analysis.","The authors' listed limitations (extreme size disparities, object-centric frames) suggest concrete next experiments: building SCOP-style pairs that include size-contrast or object-centric annotations should extend the method toward fuller Qualitative Spatial Relations coverage, and their Fig. 7 shows a preliminary positive result for size.","The 85.2% human-agreement check validates the SCOP captions, but the paper does not decompose how much of the benchmark gain comes from the crop-and-template decoding versus the geometric filtering itself."],"forward_implications":["Any existing UNet- or MMDiT-based text-to-image model can be upgraded for spatial accuracy with a short fine-tuning phase that adds no parameters at inference and only about 3% latency.","A random 500-image subset of SCOP already lifts GenEval Position from 0.26 to 0.56 on FLUX.1, so the recipe is data-efficient enough for settings without access to web-scale datasets.","The model trained only on two-object pairs improves three-object spatial accuracy (e.g., FLUX.1 'any' accuracy from 30.12 to 52.44), indicating the token-order signal transfers beyond the training template.","The improvements are not confined to spatial metrics: overall GenEval, DPG-Bench, FID, and CMMD all improve, suggesting that cleaning spatial supervision also helps general prompt following.","The ablations assign distinct roles to the two components: SCOP alone raises spatial accuracy substantially, and TENOR adds generalization to unseen prompt structures."],"supporting_citations":[{"why":"Supplies the COCO training split from which SCOP curates its 28k spatially unambiguous object pairs.","marker":"[43]"},{"why":"Training-based spatial-consistency baseline (SPRIGHT) that CoMPaSS must beat; compared on SD2.1 performance and training overhead.","marker":"[6]"},{"why":"VISOR benchmark: the main spatial-understanding evaluation where CoMPaSS reports the largest relative gains.","marker":"[26]"},{"why":"T2I-CompBench: benchmark providing the Spatial and overall compositional metrics reported in the main results.","marker":"[35]"},{"why":"GenEval: benchmark providing Position and overall alignment scores, including the +131% relative gain.","marker":"[25]"},{"why":"ELLA/DPG-Bench: dense-prompt benchmark used to show general generation capability and relation-specific scores.","marker":"[33]"},{"why":"Transformer positional encodings that TENOR injects into text-image attention to preserve token order.","marker":"[68]"},{"why":"CLIP ViT-L text encoder analyzed in the proxy task and used by the diffusion models being enhanced.","marker":"[55]"},{"why":"T5-XXL text encoder analyzed in the proxy task and used by FLUX.1-style models.","marker":"[56]"},{"why":"TokenCompose-style attention supervision used in the UNet fine-tuning to help TENOR converge.","marker":"[71]"}],"fun_headline_variants":["Diffusion models learn left from right with CoMPaSS fix","Spatial relations in T2I: 131% gain with token order","New spatial engine makes AI image generators place objects correctly","Token ordering + clean data lifts T2I spatial scores up to 131%","CoMPaSS injects order into diffusion models to fix spatial reasoning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported gains generalize beyond the exact training distribution: SCOP pairs come only from COCO object categories with eight spatial tags, and the benchmarks test the same categories and the same binary left/right/above/below relations, so the improvements could reflect distribution matching rather than general spatial understanding.","fun_headline_variants_meta":{"raw":{"variants":["Diffusion models learn left from right with CoMPaSS fix","Spatial relations in T2I: 131% gain with token order","New spatial engine makes AI image generators place objects correctly","Token ordering + clean data lifts T2I spatial scores up to 131%","CoMPaSS injects order into diffusion models to fix spatial reasoning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000848,"raw_usage":{"total_tokens":3692,"prompt_tokens":951,"completion_tokens":2741,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":567,"completion_tokens_details":{"reasoning_tokens":2648}},"tokens_in":567,"tokens_out":2741,"duration_ms":16632,"temperature":1.0,"reasoning_tokens":2648,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T13:19:47.384356+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Evaluate CoMPaSS against its base models on a held-out benchmark built from non-COCO object categories (for example, 'a wrench to the left of a screwdriver') or object-centric spatial language ('to the child's right hand side'); if accuracy returns to baseline levels, the gains are distribution matching and the claim of enhanced general spatial understanding fails.","supporting_citations":[{"cited_title":"Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll ´ar, and C","cited_arxiv_id":null,"evidence_quote":"Supplies the COCO training split from which SCOP curates its 28k spatially unambiguous object pairs."},{"cited_title":"Getting it right: Improving spatial consis- tency in text-to-image models","cited_arxiv_id":null,"evidence_quote":"Training-based spatial-consistency baseline (SPRIGHT) that CoMPaSS must beat; compared on SD2.1 performance and training overhead."},{"cited_title":"T2i-compbench: A comprehensive benchmark for open-world compositional text-to-image generation","cited_arxiv_id":null,"evidence_quote":"T2I-CompBench: benchmark providing the Spatial and overall compositional metrics reported in the main results."},{"cited_title":"Geneval: An object-focused framework for evaluating text- to-image alignment","cited_arxiv_id":null,"evidence_quote":"GenEval: benchmark providing Position and overall alignment scores, including the +131% relative gain."},{"cited_title":"Gomez, Lukasz Kaiser, and Illia Polosukhin","cited_arxiv_id":null,"evidence_quote":"Transformer positional encodings that TENOR injects into text-image attention to preserve token order."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"T5-XXL text encoder analyzed in the proxy task and used by FLUX.1-style models."},{"cited_title":"Tokencompose: Text-to-image diffusion with token-level supervision","cited_arxiv_id":null,"evidence_quote":"TokenCompose-style attention supervision used in the UNet fine-tuning to help TENOR converge."}],"review_version":1}