{"id":"40c90508-b635-4eb2-b01f-7790464bd6b3","arxiv_id":"2501.12173","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"ComposeAnyone generates human images by conditioning a diffusion model on hand-drawn color-block layouts together with decoupled text or reference-image descriptions for each body part.","lead":"ComposeAnyone generates realistic human images from a simple color-block sketch of the body, combined with text or reference photos for each part like the top, pants, or shoes. It is worth reading because it unifies layout, text, and image control in a single model, which could simplify virtual fashion design and try-on tools.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim rests on an uncontrolled comparison: ComposeAnyone trains on the evaluation datasets while baselines are zero-shot, and the primary VLM Rate metric uses the same CogVLM2 used to create training captions.","rationale":"The reader's weakest_assumption focuses on whether hand-drawn layouts match the fitted ellipses and rectangles; that is a real deployment-level concern but secondary to the evidence for the central quantitative claim. The evaluation-protocol concern is more load-bearing: if the comparisons are not controlled and the primary metric is circular, none of the headline numbers support the abstract's 'better alignment' claim. The paper does have independent supporting elements: a released dataset, a code URL, and qualitative examples, and the limitation paragraph acknowledges VLM/segmentation inaccuracies. However, those do not address the train/test overlap or the choice of judge. A single hold-out experiment with an independent judge would settle whether the central claim survives, so the safest verdict remains REJECT rather than CONDITIONAL, because the current evidence is insufficient as presented.","tokens_in":12299,"tokens_out":5239,"duration_ms":54335,"concrete_test":"Hold out one of the three evaluation datasets completely from ComposeAnyone training (e.g., train only on VITON-HD and DressCode, then evaluate on the DeepFashion test set with the same hand-drawn layouts, prompts, and reference images used in Table 3). Re-run Tables 1 and 2 in this zero-shot protocol and recompute VLM Rate with an independent judge such as GPT-4V or LLaVA. If ComposeAnyone's FID/KID/VLM lead persists against the same off-the-shelf baselines, the evaluation-fairness objection is answered; if the gap collapses, the reported advantage is dataset familiarity and prompt-judge circularity rather than demonstrated method superiority.","verdict_should_be":"REJECT","load_bearing_attack":"Section 4.1 says the ComposeHuman training set is sampled from VITON-HD, DressCode, and DeepFashion, and evaluation is performed on test splits of those same three datasets. ComposeAnyone is therefore in-domain, while the layout baselines in Table 1 (GLIGEN, DenseDiffusion, MultiDiffusion, InstanceDiffusion, MIGC) and subject-driven baselines in Table 2 (IP-Adapter, lambda-ECLIPSE, ELITE, CustomNet, AnyDoor) are used off-the-shelf. The large FID/KID gaps (e.g., FID 18.33 vs 87.10 in Table 1; 11.62 vs 32.53 in Table 2) may reflect familiarity with the target distribution rather than the proposed architecture. The headline VLM Rate is also circular: CogVLM2 generates the component captions in Section 3.2 and CogVLM2 is the judge in Section 4.3, so training explicitly fits the judge's notion of alignment. Table 3 adds an internal tension: adding cross-attention modulation improves VLM Rate (80.61 vs 80.35) but clearly worsens FID (20.24 vs 14.85), KID, and LPIPS, so the claim that CA markedly enhances control is not consistently supported by the paper's own metrics. The central claim of better alignment therefore lacks a controlled, non-circular quantitative basis as written.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ComposeAnyone, a diffusion-based framework for generating human images from hand-drawn layouts (color-blocked ellipses/rectangles), component-level text descriptions, and reference images, in a decoupled or mixed-modality manner. It introduces the ComposeHuman dataset, built from VITON-HD, DressCode, and DeepFashion, with CogVLM2-generated captions, SAM/SCHP-extracted component references, and fitted layout shapes. The method concatenates reference and layout latents along spatial dimensions and applies cross-attention modulation at inference. Quantitative comparisons on layout-guided and subject-driven tasks are reported, claiming superior alignment and image quality.","tokens_in":12595,"tokens_out":6382,"duration_ms":57496,"significance":"The proposed framework has the useful goal of unifying text-only, image-only, and mixed-modality human generation with spatial layout control in a single diffusion model, and the ComposeHuman dataset could be a resource for future work. However, the paper's central quantitative claim is not supported by its current experimental protocol. The evaluation is in-domain for the proposed model but zero-shot for the baselines, the headline VLM metric is computed with the same model used to generate training annotations, and the ablation results show a trade-off between alignment and fidelity rather than a clear win. The hand-drawn layout interaction is not validated with real drawings. Thus the idea has merit, but the empirical evidence as presented is insufficient.","major_comments":[{"comment":"The quantitative comparisons in Tables 1 and 2 are confounded by the training/evaluation protocol. The ComposeHuman training set is sampled from the training splits of VITON-HD, DressCode, and DeepFashion (Section 4.1), and evaluation is performed on the test splits of these same datasets. All baseline methods (GLIGEN, DenseDiffusion, MultiDiffusion, InstanceDiffusion, MIGC in Table 1; IP-Adapter, lambda-ECLIPSE, ELITE, CustomNet, AnyDoor in Table 2) are used off-the-shelf without fine-tuning. Consequently, ComposeAnyone is in-domain while the baselines are zero-shot, which can account for a substantial portion of the large FID/KID differences (e.g., FID 18.33 vs 87.10 in Table 1; 11.62 vs 32.53 in Table 2). The central claim that ComposeAnyone achieves better alignment and quality is therefore not supported by the evidence as presented. The authors should either fine-tune the baselines on the ComposeHuman training set or evaluate all methods under a zero-shot protocol on datasets not seen during training.","section":"Section 4.1; Tables 1-2"},{"comment":"The VLM Rate metric is circular with respect to the training data. The component-level textual annotations in the ComposeHuman dataset are generated by CogVLM2 (Section 3.2, Figure 3), and the VLM Rate in Section 4.3 uses CogVLM2 as the judge to evaluate alignment between generated images and the same style of component descriptions. Since the model is explicitly trained to match CogVLM2's annotations, a high VLM Rate partly reflects fitting to the judge rather than true multimodal alignment. The authors should use an independent VLM (or human raters) for the evaluation metric, and should report results with both CogVLM2 and an alternative judge to demonstrate robustness.","section":"Section 3.2; Section 4.3"},{"comment":"The ablation results do not consistently support the claim that cross-attention modulation (CA) markedly enhances controllability. In Table 3, comparing '-CA+CFG' and '+CA+CFG' shows that CA improves VLM Rate by only 0.26 points (80.35 to 80.61) and Spatial Accuracy by 1.82 points (67.76 to 69.58), but worsens FID from 14.85 to 20.24, KID from 1.87 to 6.37, and LPIPS from 0.0735 to 0.1242. The same pattern holds when CFG is off. This trade-off between alignment metrics and image fidelity should be acknowledged and analyzed, rather than presenting CA as an unqualified improvement in controllability.","section":"Table 3; Section 4.6"},{"comment":"The 'hand-drawn layout' capability is not validated with actual hand-drawn inputs. All layouts used in training and evaluation are fitted ellipses and rectangles derived from SCHP segmentation masks via Eqs. (3)-(6). Real user drawings are likely to involve looser, overlapping, or irregular shapes, and the model's performance on such inputs is unknown. The paper should either include a user study with free-form drawings or an evaluation with layout perturbations to substantiate the claim that arbitrary color-block drawings work as described.","section":"Section 3.2; Eqs. (3)-(6); Section 4"}],"minor_comments":[{"comment":"The ComposeHuman dataset is not released in the paper; its size after cleaning and the distribution of components are not reported, which hampers reproducibility.","section":"Section 4.1"},{"comment":"In Eq. (18), the binary mask Mi is applied directly to the cross-attention map; the paper should clarify whether Mi is resized to the attention map resolution and whether it is normalized to preserve the total attention mass.","section":"Section 3.3, Eq. (18)"},{"comment":"The description of the 'Text or Image' random selection in Section 3.2 is brief; the probability of dropping text or image, and the handling of the corresponding prompts, should be specified.","section":"Section 3.2"},{"comment":"The references are incomplete in some entries (e.g., [2] and [7] lack publication venue and page numbers).","section":"References"}],"recommendation":"reject","confidential_remarks":"The paper's main quantitative evidence is compromised by the in-domain evaluation and the circular VLM metric, and the ablation does not support the claimed benefit of cross-attention modulation. The dataset contribution is derived from existing try-on datasets and is not fully described. These issues are too central to be addressed by minor polishing; a reject seems appropriate unless the authors completely rework the evaluation protocol, which would constitute a new submission."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick read of ComposeAnyone. The core idea is genuinely new: let the user draw color-block ellipses and rectangles for body parts and condition each part independently with text, a reference image, or a mix. That fills a real gap between layout-to-image and subject-driven methods. The ComposeHuman dataset is a concrete artifact — 62k+ images with component-level captions, reference crops, and fitted layout shapes — and the SAM/SCHP cross-validation for component extraction is more careful than most. I would want the dataset even if the method fell through.\n\nThe empirical case, though, is not as clean as the prose. Section 4.1 trains on VITON-HD, DressCode, and DeepFashion, then evaluates on test splits of those same sets. The layout baselines in Table 1 and subject-driven baselines in Table 2 are off-the-shelf. So the large FID/KID gaps are at least partly familiarity with the target distribution, not architecture. That alone makes the “outperforms” claim unproven.\n\nThe bigger issue is the VLM Rate metric. CogVLM2 writes the component captions in Section 3.2 and CogVLM2 judges alignment in Section 4.3. Training explicitly fits the judge’s notion of correctness, so part of the measured gain is baked in by construction. They need an independent evaluator — a different VLM, or human raters.\n\nThere is also an internal tension: Table 3 shows cross-attention modulation improves VLM Rate from 80.35 to 80.61 but pushes FID from 14.85 to 20.24 and KID from 1.87 to 6.37. The paper says CA “markedly enhances” control, but its own image-quality metrics contradict that. That should be discussed head-on, not glossed over.\n\nOne smaller worry: the hand-drawn layout claim rests on ellipses and rectangles fitted to SCHP contours. Real users draw messier shapes. That is a fair limitation but secondary to the comparison issues.\n\nWho is this for: researchers in controllable human generation and virtual try-on. The dataset and task formulation deserve referee time. As written, the central quantitative claim is not supported, but a fair re-evaluation with fine-tuned baselines and an independent judge could change that. My call: send to peer review with major revision, not desk reject. I would not cite the empirical numbers yet, but I would cite the dataset.","headline":"The decoupled conditioning formulation and ComposeHuman dataset are real contributions, but the empirical claims rest on an in-domain versus zero-shot comparison and a circular VLM judge, so the paper needs a fair re-run before the numbers can be trusted.","tokens_in":13145,"tokens_out":2397,"would_cite":true,"duration_ms":24533,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A single diffusion model turns color-block sketches, text, and reference photos into aligned human images.","keywords":["controllable human image generation","layout-to-image generation","decoupled multimodal conditions","hand-drawn layout","diffusion models","subject-driven generation","virtual try-on","ComposeHuman dataset"],"falsifier":"Collect a set of genuine free-hand color-block layouts drawn by non-expert users (with overlaps and irregular shapes) and measure the paper's own Spatial Accuracy (Jaccard, Dice, SSIM weighted 0.25/0.25/0.50 between GroundingDINO-detected parts and the layout) as well as the VLM rate. If these scores drop substantially relative to layouts fitted from SCHP masks on the same prompts, the advertised hand-drawn layout control is not robust to real user input. A simpler targeted version: generate with a deliberately irregular layout (a star-shaped or heavily overlapping color block) and check whether the cross-attention modulation in Eq. (18) keeps the text description inside the intended region.","tokens_in":12035,"feed_emoji":"🎨","tokens_out":7914,"duration_ms":65916,"temperature":0.7,"pith_summary":"ComposeAnyone claims a single diffusion model can generate a human image from a hand-drawn color-block layout in which each body part (face, top, bottom, shoes) is described separately, either by text or by a reference photo, mixed freely across parts. This removes the usual pairing requirement between text and image conditions, so text-only, image-only, and mixed-modality prompts all work in one framework. The paper builds the ComposeHuman dataset with decoupled per-component text and reference-image annotations, and reports that the resulting model aligns better with layout, text, and reference images than existing layout-guided and subject-driven baselines on VITON-HD, DressCode, and DeepFashion. A sympathetic reader would take the core claim as: spatial control plus per-part multimodal conditioning can be combined without sacrificing fidelity or consistency.","feed_headline":"Color-block sketch plus text or photo yields a full human image","feed_subtitle":"ComposeAnyone mixes layout, text, and reference photos in one diffusion pass, keeping every component aligned.","key_machinery":"The load-bearing mechanism is the decoupled multimodal condition injection: per-component text embeddings from CLIP are concatenated along the channel dimension, while VAE latents of the layout and of pixel-level concatenated reference images are concatenated along the spatial dimension, letting a single U-Net see all modalities without a dedicated fusion network. The second piece is inference-time cross-attention modulation, where the cross-attention map $A^{(i)}$ of each component description is multiplied elementwise by the binary mask $M_i$ of its color block in the hand-drawn layout ($A^*_{(i)} = A_{(i)} \\odot M_i$), forcing each description's semantics to concentrate inside its spatial region. Training uses an SNR-weighted MSE denoising loss, and the color-block layout is constructed by fitting ellipses and rectangles to SCHP human-parsing masks.","core_discovery":"On its own terms, the paper's discovery is that decoupling text and image conditions per human component, and aligning each with a color-blocked layout, yields controllable layout-to-human generation across three modes. The input is a hand-drawn layout of ellipses and rectangles, one color block per component; each block can be paired with a text description, a reference image, or both. The model is a Stable Diffusion U-Net, initialized from InstructPix2Pix, in which the VAE latents of the layout and of concatenated reference images are injected by spatial concatenation and CLIP text embeddings by channel concatenation, with no extra feature network. At inference, the cross-attention map for each component's text is modulated by the binary mask of its color block (Eq. 18), which the ablations credit with sharper spatial and semantic alignment. Experiments on VITON-HD, DressCode, and DeepFashion report better alignment and image quality than GLIGEN, DenseDiffusion, MultiDiffusion, InstanceDiffusion, MIGC for layout guidance and IP-Adapter, λ-ECLIPSE, ELITE, CustomNet, AnyDoor for subject-driven generation.","pith_inferences":["The same decoupled conditioning recipe could transfer to non-human subjects (animals, vehicles, furniture) wherever component-level segmentation exists, because nothing in the mechanism is person-specific.","A testable extension: collect free-hand layouts from real users and compare generation fidelity against SCHP-fitted layouts; if the gap is large, the next version should train on synthetically perturbed or genuinely hand-drawn layouts.","Since each color block is independently conditioned, the method suggests a part-localized editing protocol where modifying one block leaves other components untouched, an effect that could be verified by measuring per-region drift before and after editing.","The stochastic 'text or image' drop strategy may also serve as a general recipe for other multi-condition generation tasks, reducing the need for paired text-image training data."],"forward_implications":["A single trained model covers text-only, image-only, and mixed-modality human generation, since text and image conditions are sampled independently per component during training.","Users can add or remove accessories such as hats and bags by editing the hand-drawn layout instead of rewriting prompts, because each color block maps to a distinct component.","Multiple reference images can be supplied at once via pixel-level concatenation, avoiding sequential per-image conditioning.","Ablations show cross-attention modulation (CA) improves spatial accuracy and text fidelity, implying that mask-based attention steering is a working alternative to extra feature networks.","The ComposeHuman dataset provides decoupled component-level text and image annotations, giving a reusable training resource for layout-to-human generation."],"supporting_citations":[{"why":"Supplies the Stable Diffusion backbone and U-Net cross/self-attention layers that ComposeAnyone fine-tunes.","marker":"[32]"},{"why":"Pretrained InstructPix2Pix weights initialize the model backbone.","marker":"[4]"},{"why":"VAE encoder maps human, reference, and layout images to latent codes for spatial concatenation.","marker":"[22]"},{"why":"CLIP text encoder produces per-component prompt embeddings injected via channel concatenation.","marker":"[31]"},{"why":"SCHP human-parsing masks are the source of component regions fitted into hand-drawn layouts.","marker":"[25]"},{"why":"SAM segmentation is cross-validated against SCHP via SSIM to extract component reference images.","marker":"[23]"},{"why":"CogVLM2 generates the fine-grained component-level text descriptions used in ComposeHuman.","marker":"[17]"},{"why":"VITON-HD provides training pairs and the test set for layout-guided quantitative comparisons.","marker":"[10]"},{"why":"DressCode provides training pairs and the test set for subject-driven quantitative comparisons.","marker":"[29]"},{"why":"DeepFashion provides training pairs and the test set used for ablation studies.","marker":"[14]"}],"fun_headline_variants":["Sketch blocks + text or photo = human image","Multimodal human generation from colored layout sketch","One diffusion pass: color sketch, text, and photo control humans","Turn color-block sketches into humans with per-part text and photo","Blend text and photos onto color-block layouts to draw people"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The layout control depends on the assumption that a user's free-hand color-block drawing will resemble the ellipse-and-rectangle layouts that the model saw during training, which were fitted automatically from SCHP segmentation masks; if real users draw looser, overlapping, or irregular shapes, the spatial alignment and the Eq. (18) mask modulation may degrade.","fun_headline_variants_meta":{"raw":{"variants":["Sketch blocks + text or photo = human image","Multimodal human generation from colored layout sketch","One diffusion pass: color sketch, text, and photo control humans","Turn color-block sketches into humans with per-part text and photo","Blend text and photos onto color-block layouts to draw people"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000892,"raw_usage":{"total_tokens":3874,"prompt_tokens":1002,"completion_tokens":2872,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":618,"completion_tokens_details":{"reasoning_tokens":2790}},"tokens_in":618,"tokens_out":2872,"duration_ms":18262,"temperature":1.0,"reasoning_tokens":2790,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T17:26:03.523686+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Collect a set of genuine free-hand color-block layouts drawn by non-expert users (with overlaps and irregular shapes) and measure the paper's own Spatial Accuracy (Jaccard, Dice, SSIM weighted 0.25/0.25/0.50 between GroundingDINO-detected parts and the layout) as well as the VLM rate. If these scores drop substantially relative to layouts fitted from SCHP masks on the same prompts, the advertised hand-drawn layout control is not robust to real user input. A simpler targeted version: generate with a deliberately irregular layout (a star-shaped or heavily overlapping color block) and check whether the cross-attention modulation in Eq. (18) keeps the text description inside the intended region.","supporting_citations":[{"cited_title":"High-resolution image syn- thesis with latent diffusion models, 2021","cited_arxiv_id":null,"evidence_quote":"Supplies the Stable Diffusion backbone and U-Net cross/self-attention layers that ComposeAnyone fine-tunes."},{"cited_title":"Viton-hd: High-resolution virtual try-on via misalignment-aware normalization","cited_arxiv_id":null,"evidence_quote":"VITON-HD provides training pairs and the test set for layout-guided quantitative comparisons."},{"cited_title":"Dress code: High- resolution multi-category virtual try-on, 2022","cited_arxiv_id":null,"evidence_quote":"DressCode provides training pairs and the test set for subject-driven quantitative comparisons."},{"cited_title":"A versatile benchmark for de- tection, pose estimation, segmentation and re-identification of clothing images","cited_arxiv_id":null,"evidence_quote":"DeepFashion provides training pairs and the test set used for ablation studies."}],"review_version":1}