{"id":"ae0e8e05-85d1-4b22-b211-c5ed229c65d1","arxiv_id":"2602.02402","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"SoMA couples robot joint actions, environmental forces, and learned Gaussian-splat dynamics into a single neural simulator, improving resimulation and generalization on real robot soft-body manipulation by about 20% over prior methods.","lead":"SoMA is a neural simulator that learns to reproduce the motion of soft objects (rope, cloth, dolls, T-shirts) from real robot videos, using 3D Gaussian splats as the internal representation. It is designed to make training robot manipulation policies cheaper by simulating the object's response to robot actions without hand-written physics.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper never tests whether robot actions actually drive the simulation; without an action-ablation or action-shuffle check, the central claim of action-conditioned control is unsupported.","rationale":"The reader's weakest assumption is the accuracy of the R2S calibration (§4.2.1). I agree that miscalibrated scale/transform/gravity would corrupt the action-conditioned forces. However, the more fundamental gap is that the paper provides no evidence that the action signal is causally used at all. This is more load-bearing because even with perfect calibration, the neural simulator could ignore R_t and still produce the reported resimulation/generalization results from visual state history alone. The architecture includes action inputs, so the concern is not that the method is wrong, but that the central claim is unverified. A single action-shuffle/ablation experiment would settle it. This concern is consistent with the reader's CONDITIONAL verdict: acceptance should require this verification. I would not move the verdict because the reader already conditioned acceptance on similar validation concerns; my read reinforces that condition rather than changing it.","tokens_in":14054,"tokens_out":5086,"duration_ms":51900,"concrete_test":"Run the trained SoMA on held-out test sequences with three action inputs: (i) the ground-truth robot joint states R_t; (ii) R_t shuffled across time steps or across sequences; (iii) zeroed actions. Compute PSNR, SSIM, LPIPS, and depth metrics on the same object-masked regions as Table 1. If (ii) and (iii) are statistically indistinguishable from (i), the dynamics do not depend on actions, falsifying the action-conditioning claim. Additionally, add a 'w/o action' variant to Table 3 by removing R_t from Eq. (4) and Eq. (11); if generalization does not degrade, the central claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that SoMA is an action-conditioned simulator: Eq. (4) defines G_t = φ_θ(G_{t−1}, G_{t−2}, R_t), and Eq. (11) computes robot forces from control points and gripper state. But no experiment isolates the effect of R_t on the output. The ablations (Table 3) remove supervision and multi-resolution training, but never remove or corrupt the action input. If the learned dynamics are dominated by visual state history, the reported generalization could be achieved without true action conditioning, and the 'robot joint states → control' claim collapses. The R2S calibration in §4.2.1 is a necessary precondition — if the scale or transform in Eq. (6)–(8)/(14) is wrong, the action signal is corrupted — but even perfect calibration does not guarantee the network uses actions. The paper's metrics (Table 1) compare against state-based GausSim and physics-based PhysTwin, but these baselines differ in architecture and supervision, so they do not isolate the contribution of action conditioning. Without a direct test of causal action influence, the central 'action-conditioned neural simulator' claim rests on an untested assumption.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"SoMA proposes a real-to-sim neural simulator for deformable objects manipulated by a robot. It reconstructs the object as hierarchical 3D Gaussian splats, maps robot joint states and gripper state into a unified simulation coordinate system via forward kinematics and a similarity transform, and trains a graph-neural-network dynamics model that predicts future Gaussian splat states conditioned on past states and on robot-induced forces. Training uses multi-view RGB videos with an occlusion-aware masked image loss and a momentum-consistency regularizer, in a coarse-to-fine temporal schedule. The method is evaluated on a new real-world dataset (rope, doll, cloth, T-shirt) for resimulation and generalization against PhysTwin and GausSim, with additional results on the PhysTwin dataset. The paper reports best average PSNR/SSIM/LPIPS/AbsRel/RMSE and claims a 20% improvement.","tokens_in":14425,"tokens_out":5632,"duration_ms":50863,"significance":"If the action-conditioning result is substantiated, SoMA is a meaningful step toward controllable neural simulators for soft-body manipulation: it explicitly couples joint-space actions to learned Gaussian-splat dynamics and demonstrates stable long-horizon rollouts on cloth folding. Strengths of the paper include end-to-end supervision from real multi-view images (with held-out generalization, so the result is not fitted into existence), a new real-world dataset, multi-view qualitative results, and a clear architectural separation of environmental and robot forces. The main risk is that the central action-conditioning claim is not directly tested; the missing action-ablation study, lack of uncertainty quantification, and an unsupported 20% headline prevent the current version from establishing the claim at the level required by a journal paper.","major_comments":[{"comment":"The headline \"20% improvement\" is not directly supported by the reported numbers. In Table 1, SoMA's PSNR improvement over GausSim is 5.7% on resimulation and 5.1% on generalization; over PhysTwin it is 16.5% and 23.9%. Depth Abs Rel improvements range from 12% to 22%. The paper does not define which metric, baseline, or aggregation yields the 20% figure. Please provide the exact definition and report relative improvements with error bars.","section":"Abstract; §5.2, Table 1"},{"comment":"The central claim is that SoMA is an action-conditioned simulator, but no experiment isolates the effect of the robot action R_t. Eq. (4) includes R_t in the state recurrence and Eq. (11) computes robot forces, yet the ablations in Table 3 remove supervision or multi-resolution training, never actions. A direct test is needed: e.g., shuffle or zero out R_t while keeping the same visual history, or compare against a no-action variant. Without it, the reported generalization could be largely driven by visual-state history rather than by joint-state control.","section":"§4.2.2, Eq. (11); §5.2, Table 3"},{"comment":"The robot-to-simulation mapping relies on a scale factor s and rigid transform (R,t) recovered from the mounted-camera pose and a fitted table plane. No sensitivity analysis is provided for this calibration. If s, R, or t is mis-estimated, the robot force f_rob in Eq. (11) is applied in the wrong reference frame, so the learned dynamics cannot be causally correct regardless of network capacity. Please report calibration error or an ablation that perturbs s/R/t and measures the effect on rollout metrics.","section":"§4.2.1, Eq. (6)–(8), Eq. (14)"},{"comment":"Quantitative claims lack uncertainty information. Tables 1, 2, and 7 report averages with no standard deviations, number of sequences, or significance tests. Some differences are small (e.g., SSIM 0.971 vs 0.947; LPIPS 0.055 vs 0.086), so it is unclear whether improvements are consistent across sequences. Please provide per-sequence distributions or error bars and paired significance tests where possible.","section":"§5.1; Tables 1, 2, 7; Appendix B.3"},{"comment":"Baselines are adapted \"for fair comparison (details in the supplementary material)\", but no supplementary material is included with this version. In particular, GausSim cannot consume action inputs and is evaluated without actions in the generalization setting; this is a different task setting rather than a controlled comparison. Please include the baseline implementation details in the main text or appendix and, ideally, a variant of GausSim or another neural simulator augmented with the same action conditioning.","section":"§5.1; Appendix B.3"}],"minor_comments":[{"comment":"\"Problem Defination\" should be \"Problem Definition\".","section":"§4.1"},{"comment":"The heading \"IMMIGRATED SUPERVISION\" appears to be a typo; it should likely read \"Blended Supervision\".","section":"§4.2.4"},{"comment":"Typos: \"Genral\" should be \"General\"; \"w/o MRF\" should be \"w/o MRT\"; the references to \"Sec. 4.4\" and \"Sec. 4.3\" are inconsistent with the actual numbering (§4.2.4 and §4.2.3).","section":"Table 3"},{"comment":"The caption says \"Quantitative results on the cloth folding task\" but the table reports clustering parameters for all datasets. The caption does not match the content.","section":"Table 5"},{"comment":"\"More Reults\" should be \"More Results\"; \"phystwin datatests\" should be \"PhysTwin datasets\".","section":"Fig. 4"},{"comment":"The RMSE values are not monotonically consistent with PSNR (e.g., k=5 gives lower RMSE but lower PSNR than k=10). Please clarify whether these are averages over different rollouts or computed on the same set of frames.","section":"Table 6"}],"recommendation":"major_revision","confidential_remarks":"The paper has a solid architecture and a useful dataset, but the evaluation gap is significant: the action-conditioning claim needs a dedicated action-ablation, the 20% headline needs to be substantiated, and the quantitative comparisons need uncertainty statistics. These are fixable within the scope of the manuscript, so I would not recommend rejection, but I would require the additional experiments before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: SoMA is a well-engineered extension of Gaussian-splat neural dynamics to robot manipulation, and the action-conditioned R2S mapping plus force-driven interaction graph is a genuine step beyond GausSim and GS-Dynamics. But the headline \"20% improvement\" is not backed by the table numbers, and the central claim—that the robot action input actually influences the learned dynamics—is never directly ablated. I don't think that's fatal, but it's a gap that needs closing.\n\nWhat I liked: The design is thoughtful. Modeling robot forces via control points and an interaction graph is a real improvement over state-based regression. The two-stage temporal training is a sensible way to handle long horizons, and the occlusion-aware masked loss with the momentum regularizer is well motivated. The T-shirt folding results look qualitatively stable, and the new real-world dataset with rope, cloth, doll, and T-shirt is a useful resource.\n\nWhere I'd push back: (1) No error bars or significance tests anywhere; the ablation table only shows single numbers. (2) The abstract's 20% is the average of some aggregated metrics, but Table 1 shows 5.7% PSNR over GausSim and 16.5% over PhysTwin. The 20% might come from depth metrics, but it's not spelled out. (3) The missing action ablation is the big one. Without a 'no-action' or action-shuffle variant, you can't show that R_t is doing causal work. Generalization to unseen actions is suggestive, but a strong state-based baseline could plausibly do the same on short horizons. The R2S calibration is also assumed exact; if the scale or transform is biased, every action force is corrupted, and that error isn't propagated into the metrics.\n\nThe paper is coherent and honest about its limitations, and the method is detailed enough to be reproduced. The stress-test concern is valid, but it's addressable with one additional experiment rather than a fundamental flaw.\n\nRecommended call: This deserves peer review. Send it out with a request for an action ablation, error bars, and a precise definition of the 20% metric. If the authors can provide that, the paper would be a solid contribution to real-to-sim robotics.","headline":"A credible action-conditioned GS simulator, but the paper never directly tests whether the action input drives the simulation.","tokens_in":14840,"tokens_out":2997,"would_cite":false,"duration_ms":28783,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Gaussian-splat simulator reproduces robot soft-body motion without physics","keywords":["Gaussian splatting","real-to-sim simulation","deformable objects","robot manipulation","neural dynamics","action-conditioned simulation","soft-body dynamics","long-horizon stability"],"falsifier":"In a scene with a deliberately introduced calibration error (for instance, rotating the robot-to-sim transform by a few degrees or fitting the table plane from a misaligned point cloud), measure rollout PSNR and depth RMSE; if the simulator's accuracy degrades sharply or the object drifts off the observed trajectory, that would confirm the alignment step is load-bearing.","tokens_in":14004,"feed_emoji":"🤖","tokens_out":3592,"duration_ms":35055,"temperature":0.7,"pith_summary":"SoMA claims that soft-body manipulation can be simulated from real video by learning a force-driven dynamics model directly on 3D Gaussian splats, conditioned on robot joint actions. The paper argues this unifies object, environment, and robot in one latent space, avoiding predefined physical parameters. If right, it gives a controllable real-to-sim simulator that stays stable over long horizons and generalizes to unseen actions, supporting policy learning without fresh real-world data. The authors report roughly 20% improvement over physics-based and state-based neural baselines on real robot manipulation benchmarks.","feed_headline":"Gaussian-splat simulator reproduces robot soft-body motion without physics","feed_subtitle":"Learned neural dynamics match real deformable-object videos over long horizons and beat physics-based simulators by 20%.","key_machinery":"The central mechanism is the action-conditioned force-driven Gaussian-splat dynamics: a hierarchical graph network that, at each step, predicts linear and angular velocities for Gaussian splats from their history and a total interaction force. Environment forces (gravity, table support) are computed analytically; robot forces are predicted from an interaction graph between robot control points and splats, after aligning robot, camera, and reconstruction frames with a similarity transformation and a resolved gravity direction.","core_discovery":"In the paper's own terms, SoMA establishes that robot-conditioned, force-driven dynamics over hierarchical Gaussian splats can serve as an end-to-end real-to-sim simulator for deformable objects. It reconstructs the scene, maps it to a unified simulation frame via a scale factor, rigid transform, and gravity direction recovered from the mounted camera and fitted table plane, then predicts per-splat forces (gravity, support, and robot contact) through a graph network. Training with multi-resolution temporal sampling and a momentum-conservation regularizer yields long-horizon stability and generalization to unseen manipulation sequences, demonstrated on rope, cloth, doll, and a full T-shirt fo","pith_inferences":["Because forces are modeled as explicit environment terms plus a learned robot-contact channel, the approach could plausibly be extended to other external influences (air drag, multiple robots, liquid interaction) by adding analogous force terms; this is a testable extension the paper does not pursue.","The alignment step (scale, rigid transform, gravity) is hand-set rather than learned and its error is not propagated into the reported metrics; a stress-test that deliberately perturbs the calibrated transform would directly measure how much the action-conditioned dynamics depend on this mapping.","The momentum-consistency regularizer is a self-supervised physical constraint; ablating it and evaluating view-consistency across the multiple cameras would clarify whether it stabilizes occluded dynamics or merely improves visible-region fitting."],"forward_implications":["SoMA provides a real-to-sim backend that can be controlled directly by joint-space actions, enabling open-loop simulation of manipulation strategies without predefined physics.","The method generalizes to unseen actions and contact configurations on held-out test sequences, suggesting data-driven simulation can transfer beyond memorized trajectories.","Long-horizon stability on T-shirt folding indicates the simulator can handle self-contact and large deformations, supporting task-level analysis and policy development.","The same simulator, driven by external motion trajectories rather than robot joint states, still outperforms baselines, showing the action-conditioning mechanism is not limited to robot kinematics."],"fun_headline_variants":["SoMA neural sim beats physics in soft-body resimulation","Gaussian splat neural sim predicts robot soft-body motion","Robot-conditioned neural simulator for deformable objects","Learned sim reconstructs real soft-body manipulation 20% better","End-to-end Gaussian splat simulator for long-horizon cloth folding"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The robot-to-simulation mapping (scale factor, rigid alignment, and gravity direction) is assumed to be recovered accurately from the mounted-camera pose and fitted table plane; if any of these is off, every action-conditioned force is applied in the wrong frame, and the learned dynamics cannot be causally correct regardless of network capacity.","fun_headline_variants_meta":{"raw":{"variants":["SoMA neural sim beats physics in soft-body resimulation","Gaussian splat neural sim predicts robot soft-body motion","Robot-conditioned neural simulator for deformable objects","Learned sim reconstructs real soft-body manipulation 20% better","End-to-end Gaussian splat simulator for long-horizon cloth folding"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000151,"raw_usage":{"total_tokens":998,"prompt_tokens":663,"completion_tokens":335,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":407,"completion_tokens_details":{"reasoning_tokens":251}},"tokens_in":407,"tokens_out":335,"duration_ms":4107,"temperature":1.0,"reasoning_tokens":251,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T05:20:49.491106+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"In a scene with a deliberately introduced calibration error (for instance, rotating the robot-to-sim transform by a few degrees or fitting the table plane from a misaligned point cloud), measure rollout PSNR and depth RMSE; if the simulator's accuracy degrades sharply or the object drifts off the observed trajectory, that would confirm the alignment step is load-bearing.","supporting_citations":[],"review_version":1}