{"id":"4a313d13-48c7-41e1-99c5-9ca43a930bb2","arxiv_id":"2411.10499","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"FitDiT applies a customized Diffusion Transformer to image-based virtual try-on, adding a garment feature evolution stage, a frequency-domain loss, and a relaxed mask strategy to improve texture and size fidelity.","lead":"FitDiT is a new virtual try-on system built on Diffusion Transformers that preserves fine garment details like patterns and text while adapting clothing length to the person. It reports better quality scores than recent baselines on standard benchmarks and shows a 4.57 second inference time for a 1024x768 image.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Frequency loss in Eq. (6) uses the strict ground-truth garment mask m_g while the model is trained with dilated-relaxed masks, potentially reintroducing the shape leakage that the mask strategy is designed to prevent.","rationale":"The reader identified the same load-bearing concern: the frequency loss in Eq. (6) relies on a strict garment mask that is inconsistent with the dilated-relaxed mask used elsewhere in training. This is the most serious issue because it directly threatens the paper's distinctive size-aware fitting contribution and, by extension, the claim of being state of the art for cross-category virtual try-on. The concern is not a formal inconsistency or a disagreement with consensus; it is an unanalyzed interaction between two of the three proposed components. It is addressable by a focused ablation, which is exactly the kind of condition that a conditional acceptance should require. The paper otherwise presents reasonable quantitative results and a plausible DiT-based architecture, but the missing code, dataset, and error bars further support keeping the verdict conditional rather than moving to acceptance. I do not see grounds to reject outright, because the concern, while plausible, has not been demonstrated; it could be that the frequency loss weight is small enough that the dilated-relaxed mask dominates, or that the strict mask region still contains enough of the garment shape for the loss to be beneficial. The concrete test proposed would settle this. The reader's conditional verdict remains appropriate, so no change to the overall decision is recommended, but the condition should explicitly include this mask-mismatch check.","tokens_in":13198,"tokens_out":4446,"duration_ms":45961,"concrete_test":"Train three FitDiT variants on DressCode (or VITON-HD): (A) full model with Eq. (6) using strict m_g; (B) identical training but with Eq. (6) computed using the same dilated-relaxed mask used for the inpainting input; (C) training without the frequency loss. Evaluate on a held-out cross-category unpaired test set with explicit size mismatches, measuring a shape-aware metric such as IoU between the parsed garment mask of the output and the ground-truth garment mask of the target garment, plus FID/LPIPS for texture fidelity. If variant (A) shows significantly worse shape IoU than (B) while (B) retains the texture gains, the concern is confirmed.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's size-aware fitting claim rests on the dilated-relaxed mask strategy (Sec. 3.3), which deliberately avoids strict human-parsing masks to prevent the model from learning to fill the entire agnostic region. However, the frequency loss in Eq. (6) is computed as the spectral distance between the predicted try-on image and the original person image, both masked by m_g, where m_g is the strict garment segmentation mask of the target person. This creates a direct training signal that encourages the generated garment to match the original garment's shape and extent inside m_g. For cross-category or size-mismatched try-on, the desired garment may be longer or shorter than the original, so the strict-mask frequency loss either ignores the pixels outside m_g (where the new garment shape must be learned) or penalizes the model for generating a garment that does not exactly fill m_g. This conflicts with the dilated-relaxed mask objective, which asks the model to infer the true garment length. The paper never analyzes this interaction, and no ablation isolates the frequency loss from the mask strategy. If the frequency loss dominates, it could systematically bias the model toward the original garment shape, undermining the reported improvement in size-aware fitting and the SOTA claim on cross-category try-on.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FitDiT, a diffusion-transformer-based virtual try-on method with three technical contributions: a garment priors evolution stage that fine-tunes a garment feature extractor, a frequency-domain loss in pixel space intended to improve high-frequency texture fidelity, and a dilated-relaxed mask strategy intended to improve size-aware fitting. The method is evaluated on VITON-HD, DressCode, and a self-collected Complex Virtual Dressing Dataset (CVDD), with quantitative comparisons against several recent baselines, an ablation study, and an inference-time analysis. The paper claims state-of-the-art results on the tested datasets and reports a 4.57-second inference time for a 1024x768 image.","tokens_in":13438,"tokens_out":4148,"duration_ms":44425,"significance":"If the reported results hold, FitDiT would be a meaningful advance for image-based virtual try-on: it is, to the authors' knowledge, the first DiT-specific customization for this task, it reports consistent quantitative improvements over strong baselines on two public benchmarks, and it addresses the practically important cross-category and size-mismatch scenario. The paper also provides a useful architectural analysis of attention parameter allocation across U-Net and DiT backbones, and the inference-time comparison is a concrete practical contribution. However, the strength of the central claims is moderated by three gaps: the frequency loss and the dilated-relaxed mask strategy interact in an unanalyzed way, the ablations are limited to the self-collected CVDD without error bars, and the CVDD itself is not yet released and its construction is not fully specified. These issues are addressable and do not invalidate the approach, but they need to be resolved before the state-of-the-art claim can be considered fully supported.","major_comments":[{"comment":"The one-step estimate of z_0 in Eq. (5) involves division by (1-t), which becomes numerically unstable for timesteps close to t=1. The paper does not mention how such timesteps are handled during training. Please clarify the clipping or rescaling strategy used in practice.","section":"§3.4, Eq. (5)"}],"minor_comments":[{"comment":"The limitation statement regarding hand and finger details is honest and useful, but it should be mirrored in the main text's conclusion so that readers of the main paper are aware of the primary failure mode.","section":"Supplementary Material, Sec. 13"}],"recommendation":"major_revision","confidential_remarks":"The paper is technically sound in its core diffusion-transformer adaptation and the reported public-benchmark numbers are strong, but the unresolved interaction between the strict-mask frequency loss and the relaxed-mask training strategy is a genuine correctness risk for the size-aware fitting claim. The lack of error bars and the reliance on an unreleased self-collected dataset for ablations also make the state-of-the-art claim hard to verify as reported. I would encourage the editor to require the experiments suggested in the major comments before acceptance. The 'first DiT for virtual try-on' claim should also be checked against concurrent work, since the field is moving quickly."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First DiT-based virtual try-on, and the numbers are genuinely strong. The paper does three things well: it replaces the U-Net with an SD3-derived DiT and makes a convincing case that the DiT's attention allocation suits detail-heavy try-on; it adds a garment priors evolution stage that fine-tunes the garment encoder; and it introduces a dilated-relaxed mask that plausibly reduces shape leakage. The quantitative gains over IDM-VTON, OOTDiffusion, and CatVTON on DressCode and VITON-HD are consistent, and the ablation on their self-collected CVDD supports the texture components. The paper is clearly written and the engineering is sensible.\n\nThe soft spots are real but addressable. The frequency-spectra loss in Eq. (6) computes a spectral distance between the predicted try-on and the original person image, both masked by the strict garment segmentation mask mg. That is the same strict-mask signal the dilated-relaxed strategy is designed to avoid. If the loss weight is non-trivial, the model is being pulled to reproduce the original garment's shape and texture inside mg, which directly conflicts with the claim that the relaxed mask lets the model infer correct garment length in cross-category try-on. The paper never analyzes this interaction, and the ablation doesn't cross the frequency loss with the mask strategy. That's a genuine hole in the size-fitting claim.\n\nAlso missing: error bars or significance tests on any metric; ablations only on the self-collected CVDD, not on the public benchmarks; code and the CVDD dataset are promised but not linked. These are standard referee asks, not fatal flaws. The 'first DiT for try-on' claim is plausible, but it's a novelty statement rather than a numerically verified one.\n\nWho is this for? People working on image-based try-on or garment-conditioned generation will want to read it; it's an applied contribution, not a conceptual shift. It deserves a serious referee. I'd like the referee to push on the frequency-loss/mask interaction, ask for error bars, and request the code release. I recommend sending it to peer review with a request for revision.","headline":"First DiT-based virtual try-on with strong numbers, but the frequency loss's strict-mask term likely conflicts with the relaxed-mask training strategy and needs scrutiny before the size-fitting claim holds.","tokens_in":13981,"tokens_out":2290,"would_cite":true,"duration_ms":22113,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FitDiT customizes a Diffusion Transformer for virtual try-on, beating U-Net baselines on texture and fit.","keywords":["virtual try-on","diffusion transformer","garment texture preservation","frequency-domain loss","size-aware fitting","dilated-relaxed mask","latent diffusion","image inpainting"],"falsifier":"Run a controlled ablation on cross-category pairs: train FitDiT with (a) dilated-relaxed masks and no frequency loss, (b) dilated-relaxed masks plus the strict-mask frequency loss as in Eq. (6), and (c) dilated-relaxed masks plus a frequency loss computed on the same loosely expanded mask used for denoising. If variant (b) does not improve or worsens garment-length accuracy measured against human-annotated garment boundaries compared with (a), the frequency loss is undermining size-aware fitting; if (c) matches or exceeds (b), the exact-mask choice in Eq. (6) is not the right one.","tokens_in":12990,"feed_emoji":"👕","tokens_out":6819,"duration_ms":56452,"temperature":0.7,"pith_summary":"This paper tries to show that image-based virtual try-on gets a large quality boost by replacing the usual U-Net diffusion backbone with a Diffusion Transformer (DiT) tailored to clothing. On the model's own benchmarks, the resulting system, FitDiT, preserves fine garment textures such as stripes, logos, and printed text better than prior diffusion try-on methods, and it renders the correct garment length instead of flooding the whole masked region. If the claims hold, this addresses two bottlenecks that matter for real online shopping: authentic detail transfer and size-aware fit. The paper also reports a single-image inference time of 4.57 seconds at 1024x768, competitive with or faster than earlier approaches.","feed_headline":"FitDiT beats prior virtual try-on on texture and fit","feed_subtitle":"Switching from U-Net to a slimmed diffusion transformer keeps logos, stripes, and correct garment length.","key_machinery":"The load-bearing object is a customized Stable Diffusion 3-style Diffusion Transformer (a transformer-based denoising network) split into a frozen GarmentDiT that extracts clothing features and a DenoisingDiT that generates the try-on. The argument rests on three mechanisms: hybrid attention concatenation of garment keys and values into the denoising branch; a frequency loss $L_f$ that applies a 2D discrete Fourier transform to the predicted and ground-truth images masked by the garment region; and the dilated-relaxed mask, a coarse rectangular mask randomly expanded so the model cannot infer garment length from the mask boundary. The paper also uses an attention-parameter-ratio analysis to argue that DiT allocates far more parameters to high-resolution latent features than U-Net backbones, which is why it suits detail-heavy tasks like try-on.","core_discovery":"FitDiT is presented as the first customization of a Diffusion Transformer for virtual try-on. The authors adapt Stable Diffusion 3 by removing the text encoder (about 72 percent parameter savings), modulating DiT blocks with garment image embeddings instead of text, and injecting garment key/value features from a frozen GarmentDiT into the denoising branch through hybrid attention. Three additions carry the performance claims: a garment priors evolution stage that fine-tunes the garment extractor on garment data, a frequency-spectra distance loss computed in pixel space after a one-step denoising estimate that pushes high-frequency detail toward the ground truth, and a dilated-relaxed mask strategy that trains the model with loosely expanded masks so it learns garment shape rather than filling the mask. In paired and unpaired tests on VITON-HD, DressCode, and a self-collected complex-texture dataset (CVDD), FitDiT reports better SSIM, LPIPS, FID, and KID than the compared baselines, with particular gains on complex textures and cross-category outfits.","pith_inferences":["The frequency loss in Eq. (6) is computed on the exact garment mask of the target person, while training uses deliberately loose masks; this mismatch is never ablated, so a natural next test is whether the strict-mask frequency term partially reintroduces the garment-shape leakage the loose masks are meant to remove.","The reported gains on CVDD, a self-collected dataset of 516 challenging pairs, may not transfer to standard benchmarks without that extra fine-tuning; the paper's strongest KID reductions are shown on CVDD and unpaired VITON-HD.","If the attention-parameter-ratio argument is the real cause, the same customization should improve other exemplar-guided inpainting tasks, such as object retexturing or furniture try-on, which could be tested by applying the three mechanisms to a non-fashion dataset."],"forward_implications":["Rich textures such as text, logos, stripes, and patterns transfer to the target person with fewer artifacts than previous U-Net diffusion try-on methods, according to the reported metrics.","Cross-category try-on produces garments of the correct length rather than filling the entire inpainting mask, thanks to the dilated-relaxed mask strategy.","The slimmed DiT reaches 4.57 seconds per 1024x768 image, about 27 percent faster than StableVITON and 54 percent faster than IDM-VTON, and can run below 6GB GPU memory with CPU offload.","The DiT backbone's high-resolution attention allocation makes it a viable base for other detail-preserving image generation tasks beyond try-on."],"supporting_citations":[{"why":"Stable Diffusion 3, the DiT backbone whose structure is slimmed and customized for try-on.","marker":"[10]"},{"why":"Latent diffusion models, the generative framework the method builds on.","marker":"[36]"},{"why":"IDM-VTON, a U-Net diffusion baseline with garment feature injection that FitDiT compares against and extends.","marker":"[7]"},{"why":"OOTDiffusion, a baseline that uses garment labels as conditioning and is compared for cross-category try-on.","marker":"[44]"},{"why":"CatVTON, a concatenation-based diffusion baseline used for comparison.","marker":"[8]"},{"why":"AnyFit, a parsing-free mask baseline whose keypoint-only masks are contrasted with the dilated-relaxed mask.","marker":"[23]"},{"why":"Rectified flow, the forward process used to estimate the clean image for the frequency loss.","marker":"[25]"},{"why":"VITON-HD, one of the two public datasets used for training and evaluation.","marker":"[6]"},{"why":"DressCode, the multi-category public dataset used for training and evaluation.","marker":"[29]"}],"fun_headline_variants":["FitDiT: Slimmed DiT try-on nails texture and fit","FitDiT: First DiT for virtual try-on with accurate garment details","FitDiT: DiT try-on keeps logos, stripes, and exact length","FitDiT: Fast and faithful virtual try-on with slimmed DiT","FitDiT: Diffusion transformer try-on with sharp garment detail"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the dilated-relaxed mask prevents garment-shape leakage while the frequency loss, computed on the exact garment mask, does not leak shape back in and does not conflict with the loose-mask training; the paper provides no ablation isolating that interaction.","fun_headline_variants_meta":{"raw":{"variants":["FitDiT: Slimmed DiT try-on nails texture and fit","FitDiT: First DiT for virtual try-on with accurate garment details","FitDiT: DiT try-on keeps logos, stripes, and exact length","FitDiT: Fast and faithful virtual try-on with slimmed DiT","FitDiT: Diffusion transformer try-on with sharp garment detail"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001462,"raw_usage":{"total_tokens":5915,"prompt_tokens":1010,"completion_tokens":4905,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":626,"completion_tokens_details":{"reasoning_tokens":4807}},"tokens_in":626,"tokens_out":4905,"duration_ms":31298,"temperature":1.0,"reasoning_tokens":4807,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T19:57:51.977495+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a controlled ablation on cross-category pairs: train FitDiT with (a) dilated-relaxed masks and no frequency loss, (b) dilated-relaxed masks plus the strict-mask frequency loss as in Eq. (6), and (c) dilated-relaxed masks plus a frequency loss computed on the same loosely expanded mask used for denoising. If variant (b) does not improve or worsens garment-length accuracy measured against human-annotated garment boundaries compared with (a), the frequency loss is undermining size-aware fitting; if (c) matches or exceeds (b), the exact-mask choice in Eq. (6) is not the right one.","supporting_citations":[{"cited_title":"Scaling recti- fied flow transformers for high-resolution image synthesis","cited_arxiv_id":null,"evidence_quote":"Stable Diffusion 3, the DiT backbone whose structure is slimmed and customized for try-on."},{"cited_title":"Improving diffusion models for vir- tual try-on","cited_arxiv_id":null,"evidence_quote":"IDM-VTON, a U-Net diffusion baseline with garment feature injection that FitDiT compares against and extends."},{"cited_title":"Viton-hd: High-resolution virtual try-on via misalignment-aware normalization","cited_arxiv_id":null,"evidence_quote":"VITON-HD, one of the two public datasets used for training and evaluation."},{"cited_title":"Dress code: High- resolution multi-category virtual try-on","cited_arxiv_id":null,"evidence_quote":"DressCode, the multi-category public dataset used for training and evaluation."}],"review_version":1}