{"id":"6153b59a-ff13-4864-a82e-ba5d1a69ace4","arxiv_id":"2412.04433","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"PBDyG reconstructs animatable human avatars from video by simulating loose clothing with physics, estimating fabric properties from the recorded motion.","lead":"This paper proposes PBDyG, a system that builds a 3D animated human avatar from multi-camera video, with clothing like skirts and coats that swing and fold naturally because their motion is simulated with physics. A generalist might care because this is a step toward digital humans for virtual reality, gaming, and film that move like real people, not just rigid puppets.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claim that physical mass and stiffness are estimated from video is not established: LPBD (Eq.","rationale":"The reader's REJECT verdict is, in my view, correct, and my stress-test does not change it. I chose to focus not on the tracking-drift risk (the reader's stated weakest assumption) but on a more downstream, equally load-bearing gap: even with perfect 6-DOF trajectories, the PBD parameter fit in Sec. 3.3 is underdetermined and unvalidated. The loss LPBD (Eq. 14) is the sole signal for mass and stiffness; it fits tens of thousands of parameters to a short single-motion sequence. Overfitting is the default outcome unless the parameters are regularized or tested on held-out motion. The paper offers qualitative reanimation examples, but no evidence that the fitted values are physically meaningful or that dynamics improve generalization over pose-dependent skinning. The newly introduced HF metrics in Table 1 are not validated and do not address this. I agree with the reader's overall rejection; I partially agree with their reasoning because their rationale already notes that physical parameters are 'fitted to tracking data rather than independently verified,' though their formal weakest assumption is about tracking drift.","tokens_in":12900,"tokens_out":4618,"duration_ms":47441,"concrete_test":"Construct a synthetic cloth sequence: simulate a skirt with XPBD using known per-vertex masses and per-edge stiffnesses, then (a) run PBDyG's parameter optimization on the first 20 frames and compare the recovered M and alpha fields to the ground-truth values (report correlation and relative error), and (b) use the recovered parameters to re-simulate a held-out motion and compare against the ground-truth XPBD result. If the recovered parameters deviate substantially or the held-out simulation error is not clearly smaller than an LBS-only baseline, the physical-parameter estimation claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central contribution is the recovery of physically meaningful mass and material stiffness from RGB video (Abstract; Sec. 3.3). The only supervision for this recovery is LPBD in Eq. 14, which minimizes the error between PBD-predicted positions and Dynamic 3D Gaussian tracks over the training frames. The optimized parameters are per-point masses M and per-edge compliances alpha on a 10,000-point tetrahedral mesh (Sec. A.1), i.e., tens of thousands of free variables fitted to roughly 30 frames of a single motion. Nothing in the formulation constrains this fit to correspond to true material properties: the loss can be driven to zero by overfitting the training trajectory, and the paper provides no held-out motion evaluation, no comparison to ground-truth parameters, no cross-subject consistency check, and no ablation isolating the benefit of the PBD layer over pure LBS on novel poses. Thus the statement that 'physical properties including mass and material stiffness are estimated' is unsupported even if the Dynamic Gaussian tracking is perfectly drift-free. The reader's tracking-drift concern is secondary: it would compound the problem, but the identifiability/validation gap is sufficient by itself to undermine the central claim. The method may still produce plausible renderings, but the physical-parameter recovery, which differentiates PBDyG from pose-dependent baselines, is not demonstrated.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces PBDyG, a method for learning animatable clothed human avatars from multiview RGB video. The approach first reconstructs a static 3D Gaussian Splatting model, tracks it over time with Dynamic 3D Gaussians, refines an SMPL body using the tracked Gaussians, and then embeds the Gaussian cloud in a tetrahedral mesh driven by extended Position Based Dynamics (XPBD). The authors claim that per-point masses and per-edge stiffnesses are estimated from video, enabling movement-dependent cloth deformation for loose garments such as skirts and coats. Experiments on the DNA-Rendering dataset compare against Animatable Gaussians using standard metrics plus two newly proposed high-frequency metrics, together with ablations of the AirMesh constraint and substep strategy.","tokens_in":13095,"tokens_out":3831,"duration_ms":46070,"significance":"If the central claim were fully supported, the paper would make a useful contribution: it proposes a holistic representation of body and clothing with contact preserved through a common tetrahedral mesh, and it attempts to recover physical parameters from video rather than from pre-scanned geometry. The use of XPBD with AirMesh constraints and the explicit handling of Gaussian anisotropy during deformation are sensible engineering choices, and the ablation studies on the AirMesh constraint and substep strategy provide some evidence about the internal workings of the method. The planned release of code and data is also a strength. However, the paper's headline contribution, the estimation of physically meaningful mass and stiffness from RGB video, is not validated by the current experiments. The quantitative evaluation is inconclusive because the baseline outperforms the proposed method on standard metrics and the newly introduced metrics are not independently justified. As a result, the significance of the work as presented remains uncertain.","major_comments":[{"comment":"The optimization of per-point masses M and per-edge compliances alpha is supervised only by LPBD, the mean squared error between XPBD-predicted positions and the Dynamic Gaussian tracking positions on the training frames. With roughly 10,000 sampled points (Sec. A.1) and about 30 training frames, this is a high-dimensional fit with no regularization and no independent check that the resulting parameters correspond to real material properties. The paper provides no experiment that transfers the trained avatar to a genuinely novel motion and compares the simulated cloth trajectory against ground truth, no synthetic test with known mass/stiffness values, and no cross-subject consistency check. Consequently, the abstract's claim that 'physical properties including mass and material stiffness are estimated from the RGB videos' is not established. Please add such experiments or temper the claim.","section":"Sec. 3.3, Eq. (14)"},{"comment":"On the standard metrics (PSNR, SSIM, LPIPS), Animatable Gaussians is better in four of the six reported comparisons, and the proposed method is better only on the newly introduced HF-SSIM and HF-PSNR in some rows. These new metrics are introduced without any validation that they align with perceptual quality, and no confidence intervals or significance tests are reported. The statement that the qualitative results 'prove otherwise' is not an adequate substitute for statistical evidence. Please validate the new metrics against human judgments, report per-frame error distributions, and provide significance tests.","section":"Sec. 4, Table 1 and Figs. 3-4"},{"comment":"Both the SMPL refinement loss Ltrack and the physical parameter loss LPBD depend entirely on the trajectories produced by Dynamic 3D Gaussians. The paper does not analyze how tracking errors or drift on highly deformable garments such as skirts and coats propagate into the estimated masses and compliances. If the tracker locks onto the wrong surface point or drifts over time, the fitted PBD parameters are meaningless even if the renderings appear plausible. Please quantify tracking accuracy on the test sequences, or use synthetic sequences with known ground-truth motion to isolate this dependence.","section":"Sec. 3.1, Eqs. (7) and (14)"},{"comment":"The related work section identifies PhysAvatar, Gaussian Garments, and AniDress as the most similar approaches for recovering physical or garment-aware avatars, but none of these is included in the experimental comparison. Since the claimed advantage of PBDyG is specifically the estimation of physical properties and the handling of loose garments, the absence of any comparison against these methods leaves the contribution undemonstrated. Please add at least a qualitative comparison or a clear explanation of why a quantitative comparison is not feasible.","section":"Sec. 4, comparison set"}],"minor_comments":[{"comment":"There is a typo: 'Quantatative' should be 'Quantitative'.","section":"Table 1 caption"},{"comment":"The word 'convinient' should be 'convenient'.","section":"Sec. 5, Conclusion"},{"comment":"The phrase 'track the set of Gaussians G that model the avatar clothese across all frames' contains a typo: 'clothese' should be 'clothes'.","section":"Sec. 3.1, paragraph 3"},{"comment":"The description of the training/test split is ambiguous: it says the last 30 frames are used as test data, but also that training data consists of a selected frame and the following 30 frames. Please clarify whether the test frames overlap with the training frames or are strictly held out.","section":"Sec. 4.1, experimental setup"},{"comment":"The caption reads 'Training PoseNovel Pose1Novel Pose2' without spaces; please separate the column labels for readability.","section":"Fig. 5 caption"},{"comment":"The regularization losses are written as Lpreg = sum(||theta||^2) and Lsreg = sum(||beta||^2); please make the summation indices explicit for clarity.","section":"Sec. 3.2, Eq. (5)"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is borderline. The central claim about physical parameter recovery is plausible but currently unsupported by the experimental design; however, the gap is addressable through additional experiments (synthetic ground-truth validation, cross-motion transfer tests, perceptual validation of the new metrics, and comparison with closer baselines). I would not reject outright, but the revision needs to be substantial."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe core idea is worth a look: they rig 3D Gaussians to an SMPL body and drive loose garments with XPBD, optimizing per-point masses and per-edge compliances from multi-view video. That integration is new, and the qualitative result—skirts and coats that swing with momentum rather than just pose—is exactly what the avatar crowd has been missing. The SMPL refinement step that uses the Gaussians to pull the body fit toward the observations is also useful in its own right.\n\nBut the paper does not establish what it claims on the physical-parameter front. Eq. 14 is the only supervision for M and alpha, and it minimizes distance between PBD predictions and the Dynamic 3D Gaussian tracks over the training frames. Those tracks are the thing being fit, so the optimized parameters are a best-fit to that trajectory, not a validated material characterization. There is no held-out motion evaluation, no ground-truth parameter comparison, no cross-subject consistency, and no ablation showing PBD beats pure LBS on novel poses. The stress-test note is right: the tracking-drift worry compounds, but the identifiability gap is enough on its own.\n\nThe quantitative comparison is another soft spot. They compare only against Animatable Gaussians, and on PSNR/SSIM/LPIPS they mostly lose. They then introduce HF-SSIM and HF-PSNR after the fact to argue their results are better. That is post hoc metric design without validation, and it does not inspire confidence. The paper also never describes how Gaussians are split into body and cloth—a load-bearing detail, since the whole simulation depends on which points are flexed.\n\nOn the positive side, the method is honestly positioned relative to prior work: AniDress, DiffAvatar, PhysAvatar, and Gaussian Garments are all cited, and the combination is indeed new. The ablations of the AirMesh constraint and the substep strategy are a start, though not sufficient.\n\nWho should read this? Anyone working on animatable avatars or physics-based clothing reconstruction. It is a credible research direction with a clearly premature evaluation. A revised version with rigorous held-out tests, a proper baseline set, and a described segmentation could change the picture.\n\nI would send it to review rather than desk-reject—the idea is fresh and the flaws are fixable—but I expect major revisions and a much stronger evaluation before acceptance.","headline":"Novel XPBD+3DGS avatar pipeline, but physical-parameter recovery is unvalidated and the eval is too thin to support the central claim.","tokens_in":13667,"tokens_out":2723,"would_cite":false,"duration_ms":96575,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Clothed human avatars with movement-dependent garments can be reconstructed from multiview RGB video by coupling Dynamic 3D Gaussians tracking with Position Based Dynamics and optimizing per-point mass and stiffness.","keywords":["3D Gaussian Splatting","clothed human avatar","position based dynamics","cloth simulation","physical parameter estimation","dynamic human reconstruction","SMPL","animatable avatar"],"falsifier":"Track a multiview sequence of a person whose garment's true mass and stiffness are known, then simulate the same garment in a held-out, more energetic motion and compare the simulated cloth positions against video; if the recovered parameters fail to reproduce the deformation at the new motion scale, the claim of recovering physical properties is falsified.","tokens_in":12654,"feed_emoji":"👗","tokens_out":4342,"duration_ms":42536,"temperature":0.7,"pith_summary":"This paper claims that the physical properties of a person's clothing, including local mass and material stiffness, can be recovered from ordinary multiview RGB video and used to reanimate a clothed human avatar whose loose garments deform with body motion, not just with pose. The key move is to model the whole clothed person as 3D Gaussians split into a rigid SMPL body and flexible cloth, then drive the cloth Gaussians with position-based dynamics instead of learned pose-conditioned deformation. If correct, the method turns passive video into a simulation-ready avatar that can be posed arbitrarily while skirts and coats swing, wrinkle, and settle under the recovered physics. The authors demonstrate this on subjects wearing highly deformable garments and introduce high-frequency evaluation metrics to argue that their reconstructions preserve clothing detail better than conventional metrics indicate.","feed_headline":"Physics learned from video makes avatar skirts swing naturally","feed_subtitle":"Gaussians become a simulation-ready avatar whose skirts and coats move with momentum, not just pose.","key_machinery":"Position Based Dynamics (PBD) and its compliant variant XPBD form the engine of the method. The paper represents rigid body deformation with linear blend skinning on SMPL vertices and body Gaussians, and non-rigid cloth deformation with PBD acting on a sampled tetrahedral mesh: Delaunay triangulation builds connectivity, distance constraints plus AirMesh unilateral constraints keep the mesh from collapsing, and a ten-substep splitting prevents gravitational sag. The parameters optimized from video are the mass of each point and the compliance, or stiffness, of each constraint edge, with a custom interpolation rule carrying the deformation to unsampled Gaussians while preserving their anisotropy.","core_discovery":"The central discovery is that movement-dependent cloth deformation can be extracted from video by coupling Dynamic 3D Gaussians tracking with Position Based Dynamics, and that the simulation parameters, per-point mass and constraint compliance, can be optimized so that the simulated Gaussians reproduce the tracked trajectories. The avatar is built by rigging reconstructed Gaussians to an SMPL body, sampling about 10,000 points into a tetrahedral mesh, applying linear blend skinning to body points and PBD to cloth points, and minimizing the mean squared error between predicted and tracked cloth positions. This yields an animatable model in which cloth behavior is a function of body motion, not only skeletal pose.","pith_inferences":["The same body-plus-deformable-skin coupling could extend beyond clothing to hair, capes, or accessories by attaching a PBD layer to any skeleton-driven template.","If the tracking truly supplies six-degree-of-freedom trajectories, the recovered masses and stiffnesses could be compared across subjects and used as data for learning general cloth priors.","A testable extension is to check whether the estimated parameters predict cloth motion on faster or more energetic versions of the recorded motions, which would indicate that the parameters are physical rather than merely per-sequence fitting.","The high-frequency metrics could be applied to existing avatar methods, since lower conventional scores may hide genuinely better cloth geometry."],"forward_implications":["People wearing loose garments can be reconstructed as avatars whose cloth reacts to motion, enabling reanimation to movement sequences the recorded person never performed.","Physical parameters recovered from the video allow the avatar to be dropped into downstream physics engines for simulation-ready use.","SMPL fits improve by referencing the Gaussian reconstruction, a refinement that is useful independently of the full pipeline.","The proposed high-frequency metrics HF-SSIM and HF-PSNR suggest that standard PSNR and SSIM can miss real gains in cloth-detail fidelity in dynamic avatar reconstruction."],"supporting_citations":[{"why":"Supplies the 6-DOF trajectories of every Gaussian that serve as the reference for SMPL refinement and for the PBD parameter fit.","marker":"[15]"},{"why":"Provides the 3D Gaussian Splatting representation and rendering used to reconstruct the person and to later reanimate the simulated Gaussians.","marker":"[5]"},{"why":"Provides the SMPL parametric body model that the cloth Gaussians are rigged to and whose pose drives the simulation.","marker":"[14]"},{"why":"Introduces Position Based Dynamics, the simulation method used for non-rigid cloth deformation.","marker":"[20]"},{"why":"Introduces XPBD, whose compliance-based stiffness handling lets the paper optimize physically meaningful cloth stiffness.","marker":"[16]"},{"why":"Supplies the AirMesh constraint that prevents the tetrahedral mesh from collapsing during PBD parameter optimization.","marker":"[21]"},{"why":"Serves as the main state-of-the-art avatar reconstruction baseline for quantitative and qualitative comparison.","marker":"[12]"}],"fun_headline_variants":["Video physics makes avatar skirts swing naturally","PBDyG: Cloth physics learned from RGB video for avatars","Avatars that move cloth by physics, not just pose","Motion-aware avatars via Position-Based Dynamic Gaussians","Physics from video: dynamic Gaussians simulate cloth"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes the Dynamic 3D Gaussian tracker supplies drift-free, six-degree-of-freedom trajectories for every cloth Gaussian across all frames, so that the SMPL refinement and the PBD parameter fit have trustworthy reference positions to match.","fun_headline_variants_meta":{"raw":{"variants":["Video physics makes avatar skirts swing naturally","PBDyG: Cloth physics learned from RGB video for avatars","Avatars that move cloth by physics, not just pose","Motion-aware avatars via Position-Based Dynamic Gaussians","Physics from video: dynamic Gaussians simulate cloth"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000267,"raw_usage":{"total_tokens":1574,"prompt_tokens":861,"completion_tokens":713,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":477,"completion_tokens_details":{"reasoning_tokens":635}},"tokens_in":477,"tokens_out":713,"duration_ms":7128,"temperature":1.0,"reasoning_tokens":635,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T21:23:49.854842+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Track a multiview sequence of a person whose garment's true mass and stiffness are known, then simulate the same garment in a held-out, more energetic motion and compare the simulated cloth positions against video; if the recovered parameters fail to reproduce the deformation at the new motion scale, the claim of recovering physical properties is falsified.","supporting_citations":[{"cited_title":"Dynamic 3d gaussians: Tracking by per- sistent dynamic view synthesis","cited_arxiv_id":null,"evidence_quote":"Supplies the 6-DOF trajectories of every Gaussian that serve as the reference for SMPL refinement and for the PBD parameter fit."},{"cited_title":"3d gaussian splatting for real-time radiance field rendering","cited_arxiv_id":null,"evidence_quote":"Provides the 3D Gaussian Splatting representation and rendering used to reconstruct the person and to later reanimate the simulated Gaussians."},{"cited_title":"Smpl: A skinned multi- person linear model","cited_arxiv_id":null,"evidence_quote":"Provides the SMPL parametric body model that the cloth Gaussians are rigged to and whose pose drives the simulation."},{"cited_title":"Position based dynamics","cited_arxiv_id":null,"evidence_quote":"Introduces Position Based Dynamics, the simulation method used for non-rigid cloth deformation."},{"cited_title":"Xpbd: position-based simulation of compliant constrained dynamics","cited_arxiv_id":null,"evidence_quote":"Introduces XPBD, whose compliance-based stiffness handling lets the paper optimize physically meaningful cloth stiffness."},{"cited_title":"Air meshes for robust collision handling","cited_arxiv_id":null,"evidence_quote":"Supplies the AirMesh constraint that prevents the tetrahedral mesh from collapsing during PBD parameter optimization."},{"cited_title":"Ani- matable gaussians: Learning pose-dependent gaussian maps for high-fidelity human avatar modeling","cited_arxiv_id":null,"evidence_quote":"Serves as the main state-of-the-art avatar reconstruction baseline for quantitative and qualitative comparison."}],"review_version":1}