{"id":"46ca3f27-b5a7-4872-b72b-013ce639ff77","arxiv_id":"2506.18779","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"DefFusionNet learns a conditional diffusion model over goal point clouds for deformable shape servoing, enabling diverse multimodal goals and outperforming DefGoalNet with fewer demonstrations.","lead":"This paper introduces DefFusionNet, a diffusion-based neural network that learns a distribution of valid goal shapes for deformable object manipulation from a small number of demonstrations, instead of predicting one averaged shape. The authors report that this multimodal goal generation improves simulated and physical surgical retraction and manufacturing packaging tasks over the deterministic DefGoalNet baseline.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Physical-robot validation is missing; simulation metrics do not establish that DefFusionNet's sampled goals are reachable by the frozen DeformerNet on real hardware.","rationale":"The reader's weakest_assumption identified the same coupling between goal validity and DeformerNet reachability, so I partially agree. However, I do not find an internal inconsistency in the diffusion model or a deliberately biased comparison; the simulation results, including the closed-loop evaluation of DeformerNet, support the core methodological contribution of modeling multimodal goal distributions. The main gap is physical-robot evidence: the manuscript mentions hardware experiments but provides no quantitative evaluation, so the strongest real-world claim is not yet verified. The reader's CONDITIONAL verdict is appropriate, and no change to that verdict is needed based on this stress-test pass.","tokens_in":510,"tokens_out":4945,"duration_ms":99414,"concrete_test":"Run a physical-robot evaluation on at least 10 novel configurations per task, sampling 3 goals from DefFusionNet per configuration and executing the closed-loop DeformerNet policy to convergence. Report per-trial success based on physical measurements (e.g., collision detection and point-cloud coverage). If the end-to-end success rate is substantially lower than in simulation, or if failures concentrate on particular sampled goals, the paper should limit its claim to simulation and either fine-tune DeformerNet or constrain the sampled goals to a reachable set.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that DefFusionNet produces goal shapes that lead to successful task execution. The pipeline relies on a frozen DeformerNet (Sec. IV-F) to execute every sampled goal, but quantitative success is only reported in simulation: final point clouds are scored with geometric criteria such as collision avoidance, success percentage, and coverage. The physical-robot experiments are mentioned in Sec. V but no concrete metrics, number of trials, task success definitions, or failure cases are given. Because DeformerNet is not fine-tuned, there is no evidence that the distribution learned by DefFusionNet lies within the set of goals DeformerNet can actually realize on hardware. If many sampled goals are unreachable or cause collisions only in the real world, the headline result that 10 demonstrations outperform DefGoalNet may not transfer from simulation to physical robots. This is not a criticism of the diffusion formulation itself, which is plausible, but of the evidence supporting the real-world component of the central claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DefFusionNet, a diffusion-based conditional generative model that learns a distribution over valid goal point clouds for deformable-object manipulation, conditioned on the current object point cloud and a task-context point cloud. The generated goals are passed to a frozen DeformerNet controller for goal-conditioned shape servoing. The authors evaluate the approach in simulation on a surgical tissue-retraction task and an object-packaging task, comparing against their prior deterministic DefGoalNet baseline across several data-set sizes (10, 100, and 1000 demonstrations). They report that DefFusionNet with as few as 10 demonstrations outperforms DefGoalNet trained on 1000 demonstrations on collision avoidance, success percentage, coverage, and Chamfer distance, and they claim to demonstrate the approach on physical dVRK and KUKA robot hardware.","tokens_in":14687,"tokens_out":5694,"duration_ms":74781,"significance":"If the central claims hold, the paper addresses a real limitation of deterministic goal generation for deformable-object manipulation: multi-modality is common in practice, and averaging over modes can produce physically infeasible goals. The 10-demonstration result is striking because it suggests that a probabilistic model of goal distributions can be substantially more sample-efficient than a deterministic one for this problem. The authors also commit to releasing code and data, which is a concrete strength. However, the significance is tempered by the absence of quantitative physical-robot results, the lack of a second baseline, and a technical mismatch between the stated probabilistic sampling formulation and the actual inference procedure.","major_comments":[{"comment":"The abstract and Section V state that the method is demonstrated on a physical dVRK and on two KUKA iiwa arms, but the experiments subsections (V-B) report only simulation metrics, with no number of physical trials, task-success criteria, quantitative success rates, or failure cases for either hardware setup. Because the pipeline uses a frozen DeformerNet without fine-tuning, the simulation results do not establish that goals sampled by DefFusionNet are reachable by DeformerNet on real hardware. This gap is load-bearing for the claim of physical-robot validation and should be addressed by either providing the missing hardware results or by clearly limiting the paper's claim to simulation.","section":"Sec. V (intro) and Sec. IV-F"},{"comment":"The generative model is defined probabilistically in Eq. (3) as a Gaussian transition with variance β_t I, and Eq. (4) expresses the full generative distribution as a product of such transitions. Algorithm 2, however, performs a deterministic update that subtracts the predicted noise and never samples from those Gaussian transitions. This is a DDIM-like deterministic sampler, not the DDPM sampler implied by the formulation, and the discrepancy is not discussed. The diversity of the generated goals, which is central to the paper's contribution, depends on the actual sampling procedure, so the authors should either sample from Eq. (3) or explicitly state and justify the deterministic sampler.","section":"Sec. IV-E, Algorithm 2, and Eqs. (3)-(4)"},{"comment":"Equation (1) defines the conditional goal distribution as a product over points, p_theta(X|c) = ∏_i p_theta(x_i|c), and the noise-predictor description in Section IV-C suggests that h is a series of fully-connected layers applied pointwise, with the global context injected only as a per-point gate and bias. If the denoising network has no cross-point interactions, then the joint structure of the generated point cloud is captured only through the single latent vector z and the global context features. The paper should clarify whether h contains any cross-point or attention layers; if it does not, it should justify why independent per-point generation is sufficient to represent coherent deformable-object goal shapes, since that is the core modeling claim of the paper.","section":"Sec. IV-B, Eq. (1), and Sec. IV-C"}],"minor_comments":[{"comment":"The figures should state the number of trials per box and define the box elements (median, quartiles, whiskers). In addition, the headline comparison of DefFusionNet-1000 versus DefGoalNet-1000 is based on a single training run; reporting variance across multiple seeds for all dataset sizes would strengthen the claim.","section":"Figs. 8-10 and 14-15"},{"comment":"The success percentage metric is computed only for the subset of scenarios that already avoid collisions. This conditional metric can be misleading because a method that rarely avoids collisions may still show a high success percentage on its remaining runs; the authors should report joint success or report both components over all test scenarios.","section":"Sec. V-B.1"},{"comment":"The coverage percentage is described as the percentage of object volume contained by the container, but the manuscript does not specify how volume is estimated from the final point cloud (e.g., convex hull, occupancy grid, or another discretization). This should be clarified for reproducibility.","section":"Sec. V-B.2"},{"comment":"The symbol T is used both for the task T in Section III and for the number of diffusion timesteps in Section IV-A. Disambiguating these notations (for example, using script T or a subscript) would prevent confusion.","section":"Sec. IV-A and Sec. III"},{"comment":"The claim that this is the first generative model for diverse multi-modal deformable-object goals should be supported by a more systematic comparison with prior goal-generation methods, including image- and language-conditioned generative goal models, rather than only a contrast with Diffusion Policy and SVBP.","section":"Sec. I and Related Work"},{"comment":"The KL divergence loss is scaled by 1/T, where T is the diffusion timesteps. This scaling is not standard in VAE training and is not motivated; a brief explanation of the choice would be helpful.","section":"Eq. (7)"}],"recommendation":"major_revision","confidential_remarks":"The core idea is solid and the simulation results support the main technical narrative, so the paper is worth revising rather than rejecting. The most important issue is the mismatch between the claimed physical-robot demonstration and the absence of any hardware results; if the hardware experiments cannot be reported, the authors should explicitly restrict the claims to simulation. Please also ask the authors to reconcile the probabilistic formulation with the deterministic sampler and to clarify whether the noise predictor has any cross-point interactions."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: this is a sensible and useful extension of DefGoalNet, and the 10-demonstration result is genuinely striking. The diffusion-plus-VAE architecture is not conceptually new—it borrows directly from Luo and Hu and Ho et al.—but using it to sample a distribution over goal point clouds conditioned on current and context clouds, inside a shape servoing pipeline, is a real contribution. The authors also ship code and data, which makes the empirical claims checkable.\n\nWhat the paper does well: the problem is well motivated. DefGoalNet's averaged goals are a real failure mode, and the paper demonstrates it clearly in both tasks. The controlled two-mode surgical setup is a fair way to show the model captures multimodality. The quantitative comparison—DefFusionNet with 10 demonstrations beating DefGoalNet with 1000—is the kind of result that matters for practice, and the effect is consistent across two tasks and multiple metrics. The Chamfer distance plots are a bit redundant with the success metrics, but they support the claim that the sampled goals are actually closer to ground-truth goals.\n\nThe soft spots are real but not disqualifying. The biggest one is the physical-robot section. The manuscript says experiments were conducted on the dVRK and KUKA arms, but gives no number of trials, no success definition, no quantitative results, no failure cases. That means the central real-world claim—that this works on hardware—is currently supported only by assertion. I would not publish the paper without those numbers.\n\nSecond, the evaluation compares only against DefGoalNet. There are no generative baselines (e.g., a conditional VAE or CVAE goal generator, or a diffusion model without the latent). That makes it hard to know how much of the gain is from diffusion per se and how much from the overall probabilistic formulation.\n\nThird, the reliance on terminal point clouds from scripted policies as ground-truth goals is a mild circularity: the model learns to imitate the scripted goals, and success is measured by whether the frozen DeformerNet can drive the object to those goals. If the scripted policies miss valid solutions, the model will too. The authors acknowledge the multi-modality is intentionally scripted, so this is a reasonable experimental design, not a hidden flaw.\n\nOn the citation pattern: prior work is cited appropriately, and the self-citations (DefGoalNet, DeformerNet) are load-bearing because those are the actual baselines and controller. That is legitimate.\n\nWho should read this: anyone working on deformable object manipulation or goal-conditioned shape servoing. It is a solid incremental contribution, not a paradigm shift.\n\nRecommendation: send it to a serious robotics venue, but the physical-robot results must be added or the claims must be explicitly limited to simulation. I would accept it for peer review with that expectation.","headline":"A credible, incremental application of conditional diffusion to deformable-object goal generation, with a striking 10-demonstration result in simulation, but the physical-robot evidence is currently asserted rather than shown.","tokens_in":15182,"tokens_out":2078,"would_cite":true,"duration_ms":20972,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"DefFusionNet replaces deterministic goal prediction with a diffusion model that samples from the distribution of valid goal shapes, producing multimodal, physically plausible goals from as few as 10 demonstrations.","keywords":["deformable object manipulation","shape servoing","diffusion probabilistic model","goal shape generation","multimodal distribution","learning from demonstration","point cloud","robot manipulation"],"falsifier":"Run the full pipeline on a task with two clearly separated valid goal modes and track whether the robot's final shape matches the particular goal that was sampled. If for many samples the end shape is closer to the average of the two modes than to the sampled mode, or if task success does not depend on which goal was sampled, then the model either does not capture distinct modes or the controller cannot realize them. A concrete version: for 100 sampled goals from a bimodal context, measure the Chamfer distance between the reached cloud and the sampled goal versus the distance to the other mode; a median comparable to the cross-mode distance would falsify the claim that sampled goals are executed faithfully.","tokens_in":14324,"feed_emoji":"🤖","tokens_out":7610,"duration_ms":70529,"temperature":0.7,"pith_summary":"Shape servoing for deformable objects requires a goal shape, but specifying one is usually impractical. This paper argues that the goal should be treated as a sample from a learned distribution of all shapes that would complete the task, not as a unique deterministic output. DefFusionNet learns the conditional distribution $p_\\theta(P_g \\mid P_c, P_T)$ with a diffusion probabilistic model, conditioned on the current object point cloud and a task context. The paper shows that in simulated surgical retraction and object packaging, the generative model produces realistic, bimodal goals from as few as 10 demonstrations, outperforming the deterministic DefGoalNet trained on 1000, and it reports success on a physical robot.","feed_headline":"Diffusion model learns diverse robot goal shapes from 10 demos","feed_subtitle":"DefFusionNet samples many valid target shapes for soft-object tasks, where deterministic models average into unusable ones.","key_machinery":"The central machinery is a conditional denoising diffusion probabilistic model: DefFusionNet. A PointNet encoder turns the current and contextual point clouds into feature vectors, a VAE-style encoder compresses the goal cloud into a latent vector $z$, and a noise-predictor network, structured with LSTM-style gates, estimates the noise added to a goal cloud at each diffusion step. Training minimizes a noise-reconstruction MSE plus a KL divergence that keeps the latent close to a unit Gaussian. At inference, reverse diffusion, conditioned on $z$ sampled from a Gaussian and on the encoded context, converts pure noise into a goal point cloud $P_g$, which is then executed by the separately trained DeformerNet controller without fine-tuning.","core_discovery":"On its own terms, the paper establishes that a diffusion-based generative model can replace deterministic goal prediction for deformable object manipulation. Given the current object point cloud $P_c$ and a task-specific contextual point cloud $P_T$, DefFusionNet learns the conditional distribution $p_\\theta(P_g \\mid P_c, P_T)$ over goal point clouds that lead to task success. At inference it samples a goal from this distribution and passes it to the pretrained DeformerNet controller, which carries out the deformation in a closed loop. In the experiments, DefFusionNet trained on 10 demonstrations achieves a median collision-avoidance rate above 70% and success percentage above 95% on simulated tissue retraction, beating DefGoalNet trained on 1000 demonstrations; it also outperforms DefGoalNet on object packaging. The authors describe this as the first generative model to produce a diverse, multimodal set of deformable object goals for real-world robotic applications.","pith_inferences":["The decoupling of goal sampling from control suggests a testable extension: evaluate whether DeformerNet actually reaches the exact sampled goal, not just whether the task succeeds; if it does not, the effective goal distribution may be narrower than the modeled one and closed-loop goal correction would be needed.","Because all demonstrations are scripted and only terminal point clouds become goals, the learned distribution is anchored to the scripted policies; transferring to human demonstrations, which are noisier and more varied, could either produce richer multimodality or expose sensitivity to demonstration quality, an untested regime.","The idea of modeling a distribution over valid goals rather than a single action or state is not limited to point clouds; it could be applied to other goal representations such as target end-effector poses or mesh deformations, and to other deformable-object tasks like folding or cable routing, provided a conditional generator is available.","The 'first generative model' claim is specifically about deformable object goals in real-world robotic pipelines; earlier generative models for goal images or videos exist, so the novelty rests on applying generative point-cloud modeling within a shape-servoing loop, which is a narrow but concrete contribution."],"forward_implications":["Deployment of shape servoing no longer requires a manually engineered or manually demonstrated goal shape: the robot only needs the current object point cloud and a contextual cloud at runtime.","Tasks with multiple equally valid solutions, such as retracting tissue left or right, become tractable because the predictor outputs a distribution rather than an average that is physically meaningless.","Goal generation becomes sample-efficient: in these experiments 10 demonstrations suffice to beat a deterministic baseline trained on 1000, which lowers the cost of collecting demonstration data.","Dividing the problem into goal generation and goal-conditioned control lets each part be trained independently, with the controller trained at scale on cheap data and the goal generator on a small curated set.","The same approach should extend to continuous, high-variance goal distributions, as suggested by the packaging task where rotation angles are sampled uniformly over $[-\\pi, \\pi]$."],"supporting_citations":[{"why":"DefGoalNet, the deterministic goal-prediction baseline whose averaging failure DefFusionNet is designed to fix; also the source of the demonstration-data pipeline.","marker":"[7]"},{"why":"DeformerNet, the pretrained shape-servoing controller that converts sampled goal point clouds into robot actions, loaded without fine-tuning.","marker":"[6]"},{"why":"Denoising diffusion probabilistic models, the generative framework that gives DefFusionNet its ability to model arbitrary multimodal distributions.","marker":"[8]"},{"why":"The VAE framework whose latent encoding z regularizes the goal distribution and is sampled at inference to add diversity.","marker":"[40]"},{"why":"PointNet, the encoder architecture used to produce feature vectors from the current, contextual, and goal point clouds.","marker":"[41]"},{"why":"Diffusion probabilistic models for 3D point cloud generation, used together with [8] as the base for generating goal clouds.","marker":"[39]"}],"fun_headline_variants":["Diffusion model generates diverse robot goals from just 10 demos","DefFusionNet samples multiple valid goals for deformable object tasks","10 demos, diffusion beats 1000-demo deterministic model","Diffusion-based goal generation avoids averaging in deformable manipulation","Diffusion model yields multimodal soft-robot goals from few demos"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the terminal point clouds of the scripted demonstrations are genuinely valid goal shapes, and that the separately trained DeformerNet can deform the real object to whatever goal $P_g$ DefFusionNet samples, especially on physical hardware.","fun_headline_variants_meta":{"raw":{"variants":["Diffusion model generates diverse robot goals from just 10 demos","DefFusionNet samples multiple valid goals for deformable object tasks","10 demos, diffusion beats 1000-demo deterministic model","Diffusion-based goal generation avoids averaging in deformable manipulation","Diffusion model yields multimodal soft-robot goals from few demos"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000545,"raw_usage":{"total_tokens":2639,"prompt_tokens":1009,"completion_tokens":1630,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":625,"completion_tokens_details":{"reasoning_tokens":1541}},"tokens_in":625,"tokens_out":1630,"duration_ms":12949,"temperature":1.0,"reasoning_tokens":1541,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T23:15:11.342338+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the full pipeline on a task with two clearly separated valid goal modes and track whether the robot's final shape matches the particular goal that was sampled. If for many samples the end shape is closer to the average of the two modes than to the sampled mode, or if task success does not depend on which goal was sampled, then the model either does not capture distinct modes or the controller cannot realize them. A concrete version: for 100 sampled goals from a bimodal context, measure the Chamfer distance between the reached cloud and the sampled goal versus the distance to the other mode; a median comparable to the cross-mode distance would falsify the claim that sampled goals are executed faithfully.","supporting_citations":[{"cited_title":"Defgoalnet: Contextual goal learning from demonstrations for deformable object manipulation,","cited_arxiv_id":null,"evidence_quote":"DefGoalNet, the deterministic goal-prediction baseline whose averaging failure DefFusionNet is designed to fix; also the source of the demonstration-data pipeline."},{"cited_title":"Deformernet: Learning bimanual manipulation of 3d deformable objects,","cited_arxiv_id":null,"evidence_quote":"DeformerNet, the pretrained shape-servoing controller that converts sampled goal point clouds into robot actions, loaded without fine-tuning."},{"cited_title":"Diffusion probabilistic models for 3d point cloud generation,","cited_arxiv_id":null,"evidence_quote":"Diffusion probabilistic models for 3D point cloud generation, used together with [8] as the base for generating goal clouds."}],"review_version":1}