{"id":"9dca915a-747f-4f3c-8e5b-283b5bfb55d3","arxiv_id":"2508.17614","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A mask-free diffusion transformer for virtual try-on, trained with a self-generated and manually curated triplet dataset, achieves state-of-the-art scores on DressCode and competitive results on VITON-HD.","lead":"JCo-MVTON is a virtual try-on model that changes a person's outfit in a photo without needing a body mask, by feeding both the person and the target garment into the attention layers of a diffusion transformer. The authors also build a large synthetic training set by generating garment and try-on images with other models and manually filtering them, and they report strong results on DressCode, VITON-HD, and against commercial systems.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Benchmark gains may stem from test-set leakage or evaluation-protocol mismatch, since the synthetic training data is built from the same benchmark families used for evaluation.","rationale":"The reader's weakest_assumption focused on synthetic data fidelity (Try-Off accuracy, FLUX-Fill correctness, manual-filtering faithfulness). My concern is related but distinct and more directly load-bearing for the specific headline numbers: the synthetic corpus is constructed from the same benchmark families used for evaluation, and the paper provides no disjointness statement. Even if each synthetic triplet is individually realistic, the benchmark numbers can be inflated if the test images overlap, are near-duplicates, or share systematic distributional biases with the training pool. The reader already flagged the missing explicit split-disjointness statement, so I agree rather than partially agree. The correct verdict remains CONDITIONAL: the architecture and data pipeline are plausible, the DressCode gains are large, and the concern is testable by an audit rather than a demonstrated flaw. I do not reject or mark unverified because the evidence is substantially in favor and the concern is a protocol gap, not an internal inconsistency. A concrete near-duplicate audit and a re-run with cleaned training data would settle whether the benchmark claim survives.","tokens_in":14782,"tokens_out":1728,"duration_ms":16741,"concrete_test":"Request or independently run a split-disjointness audit: (1) Compute image-level near-duplicate similarity (e.g., p-hash or DINOv2 feature cosine) between every training triplet image and every VITON-HD/DressCode test image; if any test image has a near-duplicate in the training set, re-run the evaluation excluding it and report the change in FID/KID/SSIM/LPIPS. (2) Re-train a small-scale version of JCo-MVTON on the same architecture but training only on the 141,734 triplets with all images that are near-duplicates of test sets removed; if the unpaired FID on VITON-HD shifts materially above 9.561 or the DressCode FIDp gains shrink, the benchmark claims are not robust. (3) For the human evaluation, release the full protocol: number of annotators, whether they were blinded to model identity, and inter-annotator agreement; recompute the radar-chart scores with 95% confidence intervals.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is state-of-the-art benchmark performance (Table 1 and Table 2) plus superiority over commercial systems in the wild. The most load-bearing condition is that the reported benchmark numbers measure generalization rather than memorization or distributional overlap with the training data. Section 4.2.1 explicitly starts from VITON and DressCode and adds crawled Internet images; Section 4.2.2 defines iterative regeneration (II.a–II.c) of triplets that are then used to train JCo-MVTON. However, the paper never states whether the benchmark test splits are disjoint from this synthetic corpus, nor whether the paired ground-truth images used for evaluation appear in any form in the generated reference images R. Because the pipeline can memorize or bias toward the same garment and person distributions present in VITON-HD and DressCode, the unpaired FID/KID gains and the strong DressCode FIDp reductions in Table 2 could be inflated by train/evaluation overlap or by evaluation on images that effectively came from the training distribution. The human evaluation against commercial systems (Section 2) has no methodological details—participant count, blinding, statistical test—so it cannot independently corroborate the benchmark claim. This is not an accusation of intentional leakage; it is an unaddressed protocol gap that determines whether the core quantitative claim is sound. Section 4.2.1 also notes the recovered pairs {G,P} are themselves derived from the same benchmarks, so even the Try-Off and FLUX-Fill steps propagate benchmark-specific biases into the training data.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes JCo-MVTON, a mask-free virtual try-on method built on a Multi-Modal Diffusion Transformer (MM-DiT) backbone. The authors inject reference-person and garment-image latents into the self-attention layers of FLUX through separate QKV branches, use mutually exclusive attention masks to prevent cross-attention between conditional branches, and introduce a concatenation-based positional encoding scheme. To create training data, they design a two-stage pipeline: Stage I starts from VITON and DressCode plus crawled Internet images, recovers garment images with a 'Try-Off' model and produces reference images with mask-based FLUX-Fill; Stage II applies human filtering, IC-LoRA-based domain expansion, and iterative model-assisted triplet regeneration. Experiments on VITON-HD and DressCode report strong unpaired FID/KID results and strong DressCode metrics, and human evaluation against Kling, OutfitAnyone, and GPT-4o is claimed to show superiority in real-world scenarios. The paper includes ablations for the conditional branch and positional encoding, and provides a project page and code link.","tokens_in":15003,"tokens_out":2790,"duration_ms":29886,"significance":"If the quantitative claims hold, the paper would be a useful contribution to mask-free virtual try-on: the architectural changes are concrete and reproducible, the data pipeline addresses a real scarcity of triplet training data, and the method is evaluated on standard public benchmarks plus qualitative wild-image comparisons. Strengths include a clearly described MM-DiT conditioning mechanism with an attention mask and positional-encoding design, an explicit two-stage data-generation protocol, ablations on VITON-HD, and the release of code and a demo API. The unpaired VITON-HD FID/KID improvement and the DressCode improvements across all three garment categories are notable. However, the headline 'state-of-the-art / significantly outperforming' claim is not consistently supported by the paired VITON-HD numbers, and the evaluation protocol has open questions about train/test overlap and the methodology of the human study. These issues are load-bearing for the central claim that the method generalizes better than prior art and commercial systems.","major_comments":[{"comment":"The abstract states that JCo-MVTON 'achieves state-of-the-art performance on public benchmarks including DressCode, significantly outperforming existing methods,' but on VITON-HD paired evaluation the method trails GP-VTON in SSIM (0.8601 vs. 0.8701) and LPIPS (0.0891 vs. 0.0585). The paper should qualify the claim to specify that the advantage is primarily in unpaired FID/KID and on DressCode, and should discuss why paired perceptual similarity is worse despite better distributional metrics.","section":"Abstract and Table 1"},{"comment":"The training corpus is built from the same benchmark families used for evaluation: Stage I starts from VITON and DressCode, and Stage II iteratively regenerates triplets with a model trained on that corpus. The paper never states whether the VITON-HD and DressCode test splits are disjoint from the synthetic training triplets, nor whether ground-truth images from those test sets appear in any form in the generated reference images R. This is a load-bearing protocol gap: without an explicit overlap check or a held-out external test set, the unpaired FID/KID gains in Table 1 and the DressCode FIDp reductions in Table 2 could reflect training/evaluation distributional overlap rather than generalization. The authors should report a near-duplicate analysis between the training corpus and both test splits, or evaluate on a truly external benchmark.","section":"Section 4.2.1 (I.a, I.b, I.c) and Section 4.2.2 (II.c)"},{"comment":"The human evaluation against commercial systems is presented as evidence of real-world superiority, but the methods section lacks essential details: the number of participants, whether they were blind to model identity, the number and selection criteria of test images, the exact scoring scale, and any statistical significance test. Without these details, the radar-chart comparison in Figure 2 cannot be independently assessed, and the statement that JCo-MVTON 'markedly outperforms competitors on all assessed metrics' is not verifiable. The authors should either provide a complete evaluation protocol or downgrade the claim to a qualitative observation.","section":"Section 2 (Human Evaluation)"},{"comment":"The synthetic triplet generation assumes that the Try-Off model recovers accurate garment images, that FLUX-Fill produces correct try-on references, and that manual filtering removes systematic artifacts. Since the model is trained on these synthetic triplets, any systematic bias in garment texture, lighting, or folds could be learned as ground truth. The paper should provide a quantitative analysis of the synthetic data quality, e.g., human agreement rates in the filtering step, comparison of Try-Off outputs against real garment crops, and evidence that iterative regeneration does not amplify artifacts.","section":"Section 4.2.1 (I.b and I.c) and Section 4.2.2 (II.c)"}],"minor_comments":[{"comment":"There are several typos and inconsistent terms: 'Vitual' in the Section 4 title, 'Jonit' in Section 4.4, 'Ominicontrol' for OmniControl in Section 3.1, and inconsistent use of 'IC-LoRA' vs 'IC-LoRa' and 'MMDiT' vs 'MM-DiT'. These should be fixed in a revision.","section":"Throughout"},{"comment":"No error bars, confidence intervals, or significance tests are reported for any metric. Since some comparisons are close (e.g., DressCode upper SSIM 0.9123 vs. IDM-VTON 0.9125), the authors should report standard deviations over multiple evaluation runs or at least multiple random seeds.","section":"Table 1 and Table 2"},{"comment":"The bullet for the Dress scenario contains a formatting error: '(d) Dress: (c) Lower: JCo-MVTON tops all metrics; attains the best performance on all metrics except a slight lag behind Kling in Detail Consistency.' This should be rewritten for clarity.","section":"Section 2.1"},{"comment":"The paper says 'tens of millions of e-commerce data' were collected, but the final training set is 141,734 triplets. The manuscript should clarify the relationship between these numbers, e.g., the filtering ratio and whether the e-commerce data overlaps with the VITON/DressCode-derived synthetic corpus.","section":"Section 5, Implementation details and Datasets"}],"recommendation":"major_revision","confidential_remarks":"This paper's central technical contribution is plausible and the architectural description is detailed, but the evaluation section needs substantial strengthening before the claims are supportable. The train/test overlap concern is not an accusation of misconduct; it is a protocol gap that must be closed. I would also ask the editor to ensure the authors address the discrepancy between the abstract's 'significantly outperforming' and the paired VITON-HD results, since that discrepancy directly affects the paper's framing."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick read of JCo-MVTON. The system is a new combination of known pieces: FLUX MM-DiT backbone, dual-branch QKV injection for reference and garment, a mutually exclusive attention mask between condition branches, concatenation-based positional encoding, and a two-stage synthetic data pipeline that alternates Try-Off and mask-based try-on with human filtering. The architecture makes sense for mask-free VTON, and the DressCode numbers are consistently strong across upper, lower, and dress. The ablation shows the full-parameter branch clearly beats LoRA, which is a useful data point. The bidirectional data construction is the most original part; using a Try-Off model to recover garments and a mask-based model to create reference images, then iterating with the mask-free model itself, is a genuinely practical engineering contribution.\n\nThe soft spots are real but not fatal. The biggest one is an unaddressed protocol gap: the training triples are generated starting from VITON and DressCode images, and the paper never states that the benchmark test splits are disjoint from that synthetic corpus. If the generated references or garments overlap the test distribution, the FID/KID gains are inflated. I don't think this is deliberate, but the paper must say explicitly how overlap was prevented. The abstract says 'significantly outperforming' when on VITON-HD paired it trails GP-VTON on SSIM (0.8601 vs 0.8701) and LPIPS (0.0891 vs 0.0585). That overclaim should be fixed. There are no error bars or significance tests on any benchmark number, which matters for FID/KID with small test sets. The human evaluation against Kling and OutfitAnyone reports no participant count, no blinding, no inter-annotator agreement, and no statistical test; the radar charts alone don't support the strong conclusion. Also, the code link exists but no commit hash or dataset, so reproducibility is limited.\n\nNone of these flaws sink the central idea. The system is plausible, the DressCode results suggest real gains, and the data pipeline is worth studying. It deserves a serious referee, but a careful one who asks for the split-overlap statement, error bars, and a rewrite of the abstract and human eval. I'd bring it to a reading group to discuss the data-construction loop, though I wouldn't cite it until the overlap question is answered.","headline":"Plausible mask-free VTON system with strong DressCode numbers, but the benchmark claims need an explicit train/test-overlap statement and an honest abstract.","tokens_in":15640,"tokens_out":2641,"would_cite":false,"duration_ms":25643,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"JCo-MVTON, a mask-free multi-modal diffusion transformer for virtual try-on, reports state-of-the-art scores on VITON-HD and DressCode and beats commercial systems in the wild.","keywords":["virtual try-on","mask-free","diffusion transformer","MM-DiT","multi-modal conditioning","synthetic triplet dataset","self-attention fusion","garment transfer"],"falsifier":"Take the released model and run it on a held-out set of real e-commerce person-garment pairs that were never used in training, with manually checked ground-truth try-on images; if the LPIPS or FID gap versus a mask-based method on that set is much larger than the gap on VITON-HD or DressCode, the claimed wild generalization would not hold. A cheaper check: inspect whether the Try-Off model's recovered garments contain warped textures or borrowed background pixels, since those defects would propagate into the final try-on outputs.","tokens_in":14509,"feed_emoji":"👗","tokens_out":12081,"duration_ms":107644,"temperature":0.7,"pith_summary":"The paper claims that virtual try-on does not need a human-body segmentation mask at inference time if the model is built as a multi-modal diffusion transformer that attends jointly to the person image and the target garment. If true, this makes virtual try-on simpler to deploy in real-world shopping applications, where reliable body parsing is often unavailable. The paper argues that mask-based pipelines are brittle because mask or warping errors cascade, while mask-free models were held back mainly by a lack of aligned person-garment-result training triplets. Its fix is a bidirectional data-generation loop: a Try-Off model recovers garment images from person photos, a mask-based model creates reference try-on images, human annotators filter, and the model itself regenerates sharper references over three rounds. The resulting system reports best or near-best scores on VITON-HD and DressCode without any mask at inference, and beats commercial systems in human evaluations on in-the-wild images.","feed_headline":"A mask-free try-on transformer tops VITON-HD and DressCode","feed_subtitle":"Fusing the person and the garment in one diffusion transformer skips segmentation masks and beats commercial tools in the wild.","key_machinery":"The load-bearing mechanism is the conditional MM-DiT block. The Multi-Modal Diffusion Transformer (MM-DiT) is a transformer architecture that fuses text and image tokens in shared self-attention; JCo-MVTON turns it into a multi-condition try-on model by tokenizing the noisy image, text prompt, reference person image, and garment image, concatenating them into one sequence, and processing them through three parallel query-key-value projection branches inside self-attention. A binary attention mask zeroes out any attention between reference tokens and garment tokens, so each condition models itself and attends to the noise and text stream without interfering with the other condition. A 'joint positional encoding' gives the noise and reference identical coordinates and places the garment at horizontally appended coordinates, preserving the background geometry of the source person photo. On the data side, the load-bearing step is the synthetic triplet loop: Try-Off extracts garments from person photos, a mask-based model builds initial reference images, human annotators filter, IC-LoRA expands style coverage, and the model itself regenerates sharper references. This loop is what makes the mask-free model trainable at scale.","core_discovery":"JCo-MVTON's central claim is that jointly injecting the reference person image and the target garment image into the self-attention layers of an MM-DiT diffusion transformer is enough for high-quality mask-free try-on, provided the two condition branches are kept from attending to each other and share a positional encoding that aligns the noise latent with the reference image. The paper also claims that the scarcity of aligned person-garment-result triplets can be overcome by a bidirectional generation loop: a self-supervised Try-Off model produces garment images from person photos, a mask-based model produces reference images, and manual filtering plus iterative regeneration with the model itself yields about 120K high-quality triplets. On VITON-HD, the method reports the best unpaired FID (9.561) and KID (2.700), two distribution-similarity scores, among compared methods, together with strong paired scores; on DressCode it reports the best scores across upper, lower, and dress categories on most metrics. In a five-metric human evaluation on real-world images, it is claimed to outperform three commercial try-on systems.","pith_inferences":["An extension not explored in the paper is to train a control model on only the original public paired data, skipping the synthetic loop; if the synthetic-trained model wins mainly through overlap with its own generators, the wild-generalization claim would need qualification.","Because the mutual-exclusion mask forbids reference and garment branches from attending to each other entirely, allowing a small learned amount of cross-attention in later layers could improve fine-grained alignment of folds, seams, and shadows, a cheap ablation the authors leave open.","The same architecture and bidirectional loop should be able to run the inverse try-off task, removing a garment from a person, since the Try-Off generator already exists inside the data pipeline; a public try-off benchmark would make that capability measurable.","Full-parameter fine-tuning of the duplicated QKV branch beats LoRA in the paper, but at notable training cost; comparing higher-rank adapters or selective layer unfreezing under a fixed compute budget is a natural next experiment."],"forward_implications":["Deploying virtual try-on in the wild no longer requires a segmentation or parsing step; only a person photo and a garment image are needed at inference, so the pipeline is shorter and less prone to cascading mask errors.","The bidirectional generation loop, Try-Off for garments, mask-based try-on for references, human filtering, then model-based regeneration, can be iterated to grow training data across garment types and styles; the paper reports three rounds suffice.","Because the attention mask is defined by condition labels, additional condition branches beyond person and garment can be added without retraining the base backbone.","The reported numbers imply that mask-free models can top benchmark tables on unpaired and most category-specific metrics, not merely match mask-based competitors.","On VITON-HD, the method reports the best unpaired FID and KID among compared methods, and on DressCode most category metrics are best, showing mask-free models can lead rather than merely match."],"supporting_citations":[{"why":"Supplies the FLUX.1-dev diffusion-transformer backbone whose parameters are frozen and whose QKV branches are duplicated for the conditional try-on model.","marker":"Labs, 2024"},{"why":"Defines the MM-DiT architecture and rectified-flow training objective that the conditional try-on model extends.","marker":"Esser et al., 2024"},{"why":"Provides the Try-Off diffusion model used in Stage I to recover garment images from person photos.","marker":"Velioglu et al., 2024"},{"why":"IC-LoRA drives the domain-expansion step that generates style-diverse synthetic pairs for training.","marker":"Huang et al., 2024"},{"why":"CatVTON supplies the concatenation-based positional encoding idea for placing garment tokens next to person tokens and serves as a mask-free baseline.","marker":"Chong et al., 2025"},{"why":"DressCode is one of the two benchmark datasets on which the paper reports state-of-the-art scores.","marker":"Morelli et al., 2022"},{"why":"VITON-HD is the paired/unpaired benchmark used for the main quantitative comparison.","marker":"Choi et al., 2021"},{"why":"BiRefNet extracts garment foregrounds from paired person-garment images to create training data for the Try-Off model.","marker":"Zheng et al., 2024"},{"why":"One of the two public benchmarks whose paired person-garment images seed the synthetic data pipeline.","marker":"Han et al., 2018a"}],"fun_headline_variants":["Joint diffusion transformer eliminates masks for try-on, beats benchmarks","Try-on without masks: joint person-garment diffusion bests commercial tools","Self-supervised data loop powers mask-free try-on that outperforms SOTA","JCo-MVTON: one diffusion transformer, no masks, top scores on DressCode","Mask-free try-on via joint diffusion beats benchmarks and commercial systems"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole approach rests on the assumption that the synthetic triplets, garments recovered by Try-Off, references produced by a mask-based model, and then human-filtered, are faithful enough to serve as ground truth for real try-on, so any systematic artifact in those generated images would be learned as correct behavior.","fun_headline_variants_meta":{"raw":{"variants":["Joint diffusion transformer eliminates masks for try-on, beats benchmarks","Try-on without masks: joint person-garment diffusion bests commercial tools","Self-supervised data loop powers mask-free try-on that outperforms SOTA","JCo-MVTON: one diffusion transformer, no masks, top scores on DressCode","Mask-free try-on via joint diffusion beats benchmarks and commercial systems"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001009,"raw_usage":{"total_tokens":4314,"prompt_tokens":1041,"completion_tokens":3273,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":657,"completion_tokens_details":{"reasoning_tokens":3176}},"tokens_in":657,"tokens_out":3273,"duration_ms":21791,"temperature":1.0,"reasoning_tokens":3176,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T17:03:04.299186+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the released model and run it on a held-out set of real e-commerce person-garment pairs that were never used in training, with manually checked ground-truth try-on images; if the LPIPS or FID gap versus a mask-based method on that set is much larger than the gap on VITON-HD or DressCode, the claimed wild generalization would not hold. A cheaper check: inspect whether the Try-Off model's recovered garments contain warped textures or borrowed background pixels, since those defects would propagate into the final try-on outputs.","supporting_citations":[{"cited_title":"Dress Code: High-Resolution Multi-Category Virtual Try-On","cited_arxiv_id":"2204.08532","evidence_quote":"DressCode is one of the two benchmark datasets on which the paper reports state-of-the-art scores."}],"review_version":2}