{"id":"374c8980-b673-495b-9945-4e26034cc877","arxiv_id":"2504.17414","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A diffusion video try-on framework that uses animated textured 3D meshes as frame-level guidance, plus a new high-resolution benchmark, achieves stronger temporal consistency and garment fidelity than two released baselines.","lead":"This paper introduces a video try-on system that builds an animated textured 3D model of the person wearing the target garment and feeds it into a diffusion model, improving garment consistency across frames. It also releases a new high-resolution benchmark of 130 videos for evaluating video try-on.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The state-of-the-art claim rests on the accuracy of the animated textured 3D guidance, yet guidance fidelity is never measured; Supplementary C concedes the direct animation path distorts texture, so the central mechanism is not validated.","rationale":"The reader's weakest_assumption identifies exactly this risk, and the paper's own Supplementary C corroborates it, so I agree. The method is internally coherent and the ablations are suggestive, but because the guidance-fidelity link is never measured and the comparison to baselines is weakened by mask differences and missing significance tests, CONDITIONAL remains appropriate. The concrete test would either confirm that guidance inaccuracy does not harm consistency or force the authors to weaken the Section 5 claim.","tokens_in":16054,"tokens_out":3392,"duration_ms":37958,"concrete_test":"On a subset of HR-VVT videos with a distinct, trackable garment pattern, register the rendered animated 3D guidance to the source video frames and measure per-frame silhouette IoU and pattern keypoint drift relative to the ground-truth garment over the sequence. Then correlate this drift with per-video VFID and with the user-study preference scores. If high drift videos are not outliers in VFID or preference, the concern is mitigated; if drift predicts failures or if drift exceeds a small threshold (e.g., >5 px mean keypoint error) on consistent results, the central mechanism is unvalidated.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"Section 5's claim that textured 3D guidance produces state-of-the-art video try-on presumes that the mesh generated in Section 3.1 carries the target garment's appearance through the video. That premise is insecure for three connected reasons. (1) Reconstruction freezes SMPL-X pose θ during refinement (Eq. 2 optimizes only β, t, s), so the reconstructed mesh is aligned to an image-based pose estimate, not to the video motion; Supplementary C states that directly animating with video-based SMPL poses causes 'texture distortion and deformation' and requires a rigging workaround. (2) Self-occluded regions are infilled with SMPL-X body geometry and 'normals as the texture,' so any garment area hidden in the keyframe has no true texture; when the animation reveals these regions, guidance may show synthetic fabric rather than the target garment. (3) Table 4 shows a metric gain when textured 3D guidance is added, but it does not ablate guidance fidelity; the gain could come from the extra conditioning stream rather than from accurate texture motion. If the animated mesh visibly drifts or supplies wrong texture, the diffusion model may copy that error, and the claimed consistency benefit reverses.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes 3DV-TON, a diffusion-based video try-on framework that uses an animatable textured 3D mesh as explicit frame-level guidance. Given a source video and a target garment image, the method selects a keyframe, performs image-based try-on on that frame, reconstructs a clothed 3D human mesh (based on ECON with a modified SMPL-X refinement that freezes pose), and animates the mesh using video-based SMPL poses via KNN skinning. The animated textured mesh is fed, together with the clothing image and the try-on image, into a pseudo-3D UNet with temporal attention and classifier-free guidance. A rectangular masking strategy is introduced to prevent clothing leakage. The authors also introduce HR-VVT, a 130-video high-resolution benchmark, and report quantitative metrics (SSIM, LPIPS, VFID), user studies, and qualitative comparisons against ViViD and CatV2TON, claiming state-of-the-art performance.","tokens_in":16249,"tokens_out":3261,"duration_ms":30290,"significance":"If the central mechanism holds, the paper makes a useful contribution by introducing explicit textured 3D guidance into video try-on, a task where prior diffusion-based methods are criticized for prioritizing appearance fidelity over motion coherence. The idea of linking single-image 3D reconstruction with video-based SMPL animation to provide a consistent garment-texture reference is reasonable and could benefit the community. The introduction of the HR-VVT benchmark, targeting high-resolution and diverse scenarios, is a positive step for evaluation, provided the dataset is released with appropriate documentation. The paper also makes an honest disclosure of the mask-size confound in the main comparison, which is commendable. However, the strength of the empirical claim is currently limited by the lack of validation of the guidance fidelity and by the non-standard evaluation protocol in the key tables.","major_comments":[{"comment":"The central mechanism is the textured 3D guidance, yet its fidelity is never measured. The reconstruction optimizes Eq. (2) over β, t, and s only, freezing θ, and animation uses video-based SMPL poses through KNN skinning; Supplementary C concedes that direct animation causes \"texture distortion and deformation\" and requires a rigging workaround. No quantitative metric (e.g., garment-region texture similarity, silhouette IoU between the rendered guidance and the source video, or temporal consistency of the texture) is reported for the guidance itself. Table 4 shows that adding textured 3D guidance improves SSIM/LPIPS/VFID, but it does not distinguish a benefit from accurate texture motion from a benefit of simply adding an extra conditioning stream; without a guidance-fidelity measurement or a control using corrupted texture, the load-bearing claim that consistent texture motion drives the gain is not established. The robustness axiom stated in Supplementary C (\"diffusion models do not require high precision for the pose accuracy\") is asserted without evidence.","section":"§3.1, Eq. (2); Supplementary C"},{"comment":"The quantitative state-of-the-art claim depends on a non-default evaluation setup. In Table 1, with the default rectangular mask, 3DV-TON has SSIM 0.8681 and LPIPS 0.0707, both worse than CatV2TON (0.8727 and 0.0639, respectively); the better results (0.8992, 0.0521) are obtained only under 3DV-TON* with the baseline's mask. The text honestly discloses this, but the conclusion \"state-of-the-art performance\" is not supported by the default setting. Table 2 has no mask-matched condition at all: 3DV-TON uses its larger mask while ViViD and CatV2TON use their own masks, making the reported VFID/LPIPS advantages confounded by mask size. The authors should report mask-matched comparisons as the primary table or explicitly quantify the effect of mask size on all metrics.","section":"§4.4, Tables 1 and 2"},{"comment":"The ablation in Table 4 is not comparable to the main tables: the VFID values (e.g., 5.236 and 2.381 for the no-3D and full model) are an order of magnitude lower than the VFID values in Tables 1 and 2 (which are in the 13–20 range), suggesting a different test subset, shorter clips, or a different protocol. The dataset and evaluation protocol for the ablation are not specified, so the reader cannot determine whether the improvements generalize to the benchmark conditions. Please specify the exact subset, number of clips, and metric computation used in the ablation.","section":"§4.5, Table 4"}],"minor_comments":[{"comment":"\"at the expanse of motion coherence\" should read \"at the expense of motion coherence\".","section":"Abstract"},{"comment":"There is a typo: \"varialtional\" should be \"variational\".","section":"§3.2"},{"comment":"The caption contains typos: \"inconrrect gaments\" should be \"incorrect garments\".","section":"Figure 6"},{"comment":"In the Textured 3D Guidance paragraph, \"motion conherence\" should be \"motion coherence\".","section":"§4.5"},{"comment":"The loss in Eq. (5) is missing the subscript on the norm; it should be ∥·∥₂ for consistency with Eq. (4).","section":"Eq. (5)"},{"comment":"The heading \"Disccusion\" is a typo for \"Discussion\".","section":"Supplementary B"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid engineering contribution, but the abstract and conclusion currently overstate the evidence. I would not accept without the mask-matched primary comparison and a guidance-fidelity measurement. The HR-VVT benchmark is useful, though the provenance statement in Supplementary A is thin; the authors should confirm that the data collection complies with platform terms and privacy norms before release."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the textured 3D guidance is a real and useful idea, and the paper is honest about its main comparison caveat, but the 'state-of-the-art' headline runs ahead of the evidence. The explicit animated garment-texture mesh as frame-level diffusion conditioning is new relative to geometric-only 3D priors (Champ, RealisDance) and to flow-warping baselines. The HR-VVT benchmark (130 high-res videos, three garment categories) is a practical addition. Credit where due: the authors disclose that their rectangular mask enlarges the generated area, making the default comparison unfair to them, and they report the numbers under the baseline's mask as well. That is the right way to handle a known mismatch.\n\nThe ablations in Table 4 show SMPL guidance helps and textured 3D guidance helps on top across SSIM, LPIPS, and both VFID variants. That is reasonable evidence the extra conditioning stream is doing something. The user study is heavily in their favor, which at minimum says the outputs look competitive.\n\nThe soft spots, in order of importance. First, 'SOTA' is established against two code-released video baselines plus frame-wise image try-ons; that is a thin field. Second, no error bars or significance tests appear anywhere, so the numeric leads in Tables 1 and 2 could be noise. Third—and this is the load-bearing one—the animated textured mesh itself is never validated. Supplementary C admits that direct animation with video-pose SMPL causes texture distortion and deformation, which is why they rig the reconstructed human with the video-based body; and self-occluded regions are infilled with body geometry and normals-as-texture. For any garment area hidden in the keyframe, the 'consistent texture reference' is synthetic until the diffusion model decides what to do. Table 4 shows a gain from adding the stream, but not that the stream carries true garment texture through the motion. The mechanism is plausible and the qualitative results support it, but the core claim is supported less than the paper's tone suggests.\n\nAlso: no code, no variance, and a noticeable number of typos. All fixable.\n\nNet: read this for the 3D-guidance idea and the benchmark. I would send it to peer review—the idea is solid and the authors are honest—but I would ask for a guidance-fidelity check (e.g., how well the animated mesh preserves garment texture across poses) and a rerun of all comparisons under identical masks before treating the SOTA claim as established.","headline":"Real contribution in textured 3D guidance for video try-on, honestly reported, but the SOTA claim needs more baselines, error bars, and a direct check of whether the animated mesh actually carries garment texture.","tokens_in":16852,"tokens_out":3170,"would_cite":true,"duration_ms":30442,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Textured 3D meshes, not pixels, keep video try-on fabric consistent","keywords":["video try-on","textured 3D guidance","temporal consistency","diffusion models","3D human reconstruction","garment identity","SMPL animation","HR-VVT"],"falsifier":"Take a garment with a large printed logo, have the person rotate 360 degrees with the back initially hidden, and compare output frames before and after the back becomes visible; the animated mesh's hidden-surface texture is filled from normals rather than pixels, so if that infill misrepresents the garment, the final video should show the logo changing identity or blurring exactly at the frames where the back is revealed. A second check is to rerun the pipeline with the video-based body poses deliberately perturbed and watch VFID degrade if the animated 3D mesh is genuinely carrying the consistency.","tokens_in":1394,"feed_emoji":"👕","tokens_out":2456,"duration_ms":66788,"temperature":0.7,"pith_summary":"The paper tries to establish that consistent video try-on can be achieved by giving the generative model a textured 3D reference, not just warped 2D images or body geometry. It builds a pipeline that turns one try-on frame into an animatable clothed 3D human, moves it with the video's poses, and feeds the rendered texture sequence into a diffusion model as explicit frame-level guidance. If true, this solves the central failure of prior video try-on: preserving fine garment patterns while the person moves, without blurring or texture drift. The paper also contributes a high-resolution benchmark, HR-VVT, with 130 videos, and reports state-of-the-art results across quality and consistency metrics.","feed_headline":"Textured 3D meshes, not pixels, keep video try-on fabric consistent","feed_subtitle":"Diffusion video try-on with an animated 3D garment guide preserves logos and fabric motion across poses.","key_machinery":"The central object is the animatable textured 3D mesh of the person wearing the target garment. It is built by running image try-on on one keyframe, reconstructing the clothed body with single-image clothed-human reconstruction, and rigging that mesh to SMPL, a parametric body model used for posing and skinning, using poses estimated from the original video. This mesh is rendered as a per-frame texture reference that gives the diffusion model a consistent appearance-level motion cue, and a rectangular masking strategy prevents the original garment's pixels from leaking into the masked generation area.","core_discovery":"3DV-TON claims that explicit textured 3D guidance resolves the trade-off in video try-on between appearance fidelity and motion coherence. The pipeline selects a keyframe, performs image try-on on it, reconstructs an animatable textured 3D mesh of the clothed person, and animates that mesh with video-based poses from a parametric body model. The rendered mesh sequence is then presented to a denoising diffusion UNet as a frame-level reference, alongside the garment image and try-on image, so that the garment's texture moves consistently with the body rather than being smoothed or distorted by temporal averaging. The paper reports state-of-the-art VFID and user-preference results on existing benchmarks and on its new HR-VVT benchmark, and its ablations show that textured guidance outperforms geometric-only body guidance, especially in ambiguous poses such as leg crossing.","pith_inferences":["If the 3D guidance is truly what carries consistency, the same construction should work for any moving object whose appearance must persist, not only human clothing; logos on bags, vehicles, or animals would be a direct stress test.","The paper's own supplementary note that animating with video-based poses can distort texture implies a testable ceiling: consistency gains should shrink when image-based and video-based pose estimates disagree, so measuring that disagreement could predict failures.","Replacing the roughly 30-second reconstruction with a feed-forward animatable reconstruction, which the limitation section points to as future work, would turn the method from an offline pipeline into a near-real-time one; if that swap succeeds, 3D-guided try-on becomes practical for interactive applications."],"forward_implications":["Video try-on should preserve garment logos and patterns under lateral rotation and leg-crossing poses, where flow-warping and appearance-only diffusion baselines tend to fail.","Temporal consistency metrics such as VFID should improve without sacrificing image-level quality, because the 3D guide supplies appearance instead of relying on temporal smoothing.","The 3D guidance pipeline can be integrated into existing dual-UNet or diffusion-transformer try-on architectures without task-specific retraining, since the guidance is injected as an additional frame-level condition.","The rectangular masking strategy reduces artifact propagation from original-clothing information leaking through dynamic human and garment motion.","The new HR-VVT benchmark provides 130 high-resolution videos across upper-body, lower-body, and dress categories, offering a more demanding evaluation setting for future video try-on work."],"supporting_citations":[{"why":"Motivates the core problem: pixel-reconstruction objectives bias video models toward appearance fidelity and away from motion coherence.","marker":"[4]"},{"why":"Image try-on result on the keyframe supplies the garment appearance that the 3D reconstruction starts from.","marker":"[6]"},{"why":"Alternative image try-on method used to construct the keyframe try-on condition.","marker":"[61]"},{"why":"Provides a training video dataset and a strong diffusion-based baseline for comparison.","marker":"[13]"},{"why":"Supplies the temporal attention module that gives the denoising network motion modeling.","marker":"[20]"},{"why":"Video-based pose estimation used to animate the reconstructed mesh in sync with the source motion.","marker":"[51]"},{"why":"Alternative video-based pose estimator used for the same animation step.","marker":"[52]"},{"why":"Base single-image clothed-human reconstruction method that the 3D guidance pipeline builds on.","marker":"[60]"},{"why":"Transformer-based video try-on baseline compared on both benchmarks.","marker":"[8]"}],"fun_headline_variants":["3D garment guide keeps video try-on fabric consistent","Animated 3D mesh anchors video try-on textures","Diffusion try-on with 3D mesh reference beats pixel-only","New HR-VVT benchmark and 3D-guided video try-on","Textured 3D guidance fixes video try-on motion drift"],"cache_read_input_tokens":18944,"weakest_assumption_plain":"The method assumes the textured 3D mesh built from one keyframe, once animated with video-estimated poses, still represents the garment accurately enough to guide every frame; if that mesh's texture distorts or its unseen regions are filled in wrongly, the diffusion model is guided by a false appearance.","fun_headline_variants_meta":{"raw":{"variants":["3D garment guide keeps video try-on fabric consistent","Animated 3D mesh anchors video try-on textures","Diffusion try-on with 3D mesh reference beats pixel-only","New HR-VVT benchmark and 3D-guided video try-on","Textured 3D guidance fixes video try-on motion drift"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000233,"raw_usage":{"total_tokens":1494,"prompt_tokens":950,"completion_tokens":544,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":566,"completion_tokens_details":{"reasoning_tokens":459}},"tokens_in":566,"tokens_out":544,"duration_ms":4850,"temperature":1.0,"reasoning_tokens":459,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T10:40:24.262110+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a garment with a large printed logo, have the person rotate 360 degrees with the back initially hidden, and compare output frames before and after the back becomes visible; the animated mesh's hidden-surface texture is filled from normals rather than pixels, so if that infill misrepresents the garment, the final video should show the logo changing identity or blurring exactly at the frames where the back is revealed. A second check is to rerun the pipeline with the video-based body poses deliberately perturbed and watch VFID degrade if the animated 3D mesh is genuinely carrying the consistency.","supporting_citations":[{"cited_title":"Animatediff: Animate your personalized text-to- image diffusion models without specific tuning","cited_arxiv_id":null,"evidence_quote":"Supplies the temporal attention module that gives the denoising network motion modeling."},{"cited_title":"Wham: Reconstructing world-grounded humans with accu- rate 3d motion","cited_arxiv_id":null,"evidence_quote":"Alternative video-based pose estimator used for the same animation step."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Base single-image clothed-human reconstruction method that the 3D guidance pipeline builds on."}],"review_version":1}