{"id":"0fa9fc11-49f2-4cf9-9f62-66fa5b688918","arxiv_id":"2506.18825","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"SViP shows that sequencing a visuomotor policy with TAMP-planned primitives, via learned switching conditions and feasibility checks, improves generalization from few demonstrations in bimanual manipulation.","lead":"SViP combines learned bimanual visuomotor policies with classical task and motion planning (TAMP), using scene graphs to split demonstrations into learned contact-rich segments and scripted motion primitives. It reports strong success rates with only 20 demonstrations on real bimanual tasks, especially under out-of-distribution object placements where pure imitation learning fails.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The SafeBiOp feasibility validator assumes collision risk during a reactive visuomotor policy is determined by the initiation configuration; since the policy is closed-loop, this is an unverified structural assumption, and Table II shows only 70 percent success in the unsafe setup.","rationale":"SViP has genuine strengths: the object-centric decomposition is sensible, the simulation and real-world tables provide consistent improvements over ACT and DP, and the authors honestly list limitations such as manual PDDL definitions and single-view depth. I focused on the feasibility validator rather than the point-cloud issue because the point-cloud limitation is explicitly acknowledged and still leaves SViP outperforming DP in all reported setups, whereas SafeBiOp is the component that is supposed to make learned-policy sequencing safe and reliable in the central claim. The paper's own Table II shows 55 percent and 70 percent success in the unreachable and unsafe setups, so the guarantee is not strong. A misspecified safety predicate does not merely reduce performance; it undermines the TAMP constraint semantics and the paper's claim of complying with motion constraints. The missing equation, missing error bars, and absent quantitative results for the novel-goal tasks are real reporting gaps, but they are secondary to a structural assumption that, if false, affects every deployment. I therefore retain the reader's conditional verdict: the paper should be accepted only if the SafeBiOp input-output relationship is validated on scenes where the closed-loop policy varies while the initiation inputs are fixed, or the claim is weakened to a heuristic filter. The proposed experiment is directly implementable in the existing MuJoCo environment.","tokens_in":11816,"tokens_out":8615,"duration_ms":99857,"concrete_test":"In the MuJoCo peg-in-hole environment of Section IV-A, hold fixed the SafeBiOp input triple (q, p, ν) for a pole obstacle and vary only the pose of the peg/socket (or the grasp pose in G_pre) across 20 rollouts of the same learned policy π, while recording the actual pole-collision rate and the validator's predicted probability. If the collision rate varies across scenes with identical validator inputs, SafeBiOp is not a function of the initiation configuration and collision filtering in TAMP is unsound; if it does not vary, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section III-C trains the feasibility validator ξ as an MLP that maps initiation conditions (q, obstacle position p, obstacle feature ν) to a collision probability, using data assembled as {q_t, p, ν, δ}. The declared role of SafeBiOp is to certify that a bimanual policy π is free from collision with other objects. This is load-bearing for the central claim that SViP can sequence visuomotor policies with motion primitives reliably in OOD and novel setups. The assumption is that the collision outcome of a closed-loop policy can be predicted from the robot configuration and static obstacle description at initiation. However, π is conditioned on point-cloud observations at execution time; trajectories, and hence collision outcomes, can differ across scenes that share the same q, p, ν whenever the manipulated-object pose, grasp, or partial occlusion differs. The paper does not establish that π is deterministic or approximately deterministic given these inputs, nor does it include object-state features in the dataset. The empirical evidence is consistent with this gap: in the unsafe scenario (Table II) SViP succeeds in only 14 of 20 trials, and the failures are attributed to inaccurate point clouds, not to the validator's missing variables. The validator can at best encode a heuristic; it cannot provide the certified collision-freeness the TAMP constraint claims.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SViP, a hybrid imitation-learning/TAMP system for bimanual manipulation. Human demonstrations are segmented into scene graphs; contact-rich segments are treated as black-box bimanual skills executed by learned visuomotor policies, while pick/place/transit operations are handled by parameterized object-centric motion primitives. A DDPM-based switching condition generator predicts start/end configurations and object-centric trajectories, and an MLP feasibility validator is used to enforce a SafeBiOp collision-avoidance constraint. The system is evaluated in an ALOHA-style simulated peg-in-hole task and in three real-world tasks using 20 demonstrations per task, with comparisons against ACT and Diffusion Policy, plus qualitative novel-goal demonstrations.","tokens_in":12089,"tokens_out":4164,"duration_ms":44535,"significance":"If the claims hold, SViP makes a useful contribution to data-efficient bimanual manipulation by combining the generalization of TAMP with the contact-rich robustness of learned visuomotor policies, without requiring 6-DoF object pose estimators. The data efficiency (20 real demonstrations), use of SE(3)-equivariant encoders, and automatic extraction of symbolic descriptions from scene graphs are genuine strengths. The main significance risk is that the claimed 'certified' collision safety rests on an unvalidated structural assumption about the feasibility validator, and the quantitative evidence for novel-goal composition is thin.","major_comments":[{"comment":"The SafeBiOp feasibility validator is trained on triples (q, p, ν) with δ as the closest distance of the robot to the obstacle during an unrolled execution. The paper does not establish that the collision outcome of a closed-loop visuomotor policy is determined by the initiation configuration and a static obstacle descriptor. Since the policy is conditioned on point-cloud observations at execution time, trajectories and collision outcomes can differ across scenes that share the same (q, p, ν) whenever the manipulated-object pose, grasp, or partial occlusion differs. Table II shows only 70% success in the unsafe setup, and the text attributes failures to inaccurate point clouds rather than to the validator's missing variables. The predicate SafeBiOp is therefore not 'certified' as stated; at best it encodes a learned heuristic. I recommend including object-state and point-cloud features in the validation dataset, validating the predictor against ground-truth collision labels across repeated rollouts, and either relaxing the 'certified' language or providing quantitative evidence that the predictor is conservative.","section":"Section III-C"},{"comment":"The unreachable and unsafe experiments are the central evidence for constraint handling (Q2), but they are based on only 20 trials per setup with no error bars, no confidence intervals, and no decomposition of failures into planning failures versus execution failures. The implementation of IsReachable is also not described ('scripted using distance-related metrics' is vague), which makes it difficult to interpret the 55% success rate in the unreachable setup. Please report per-trial outcomes, a failure-mode breakdown, and more trials or bootstrap intervals so the reader can assess the significance of the reported rates.","section":"Table II and Section IV-A3"},{"comment":"The novel-goal experiments (Table-to-Bin Clearance, Cup-sleeve Insertion and Reconfiguration, and Multiple Instructions) are reported only through illustrative screenshots; no success rates, completion statistics, or quantitative goal-satisfaction metrics are provided. Since Q3 ('Can SViP complete tasks with novel goals?') is a headline claim, the current evidence is insufficient. Please add quantitative results for these tasks, including number of rollouts, success criteria, and where possible a comparison against a non-compositional baseline.","section":"Section IV-B3"},{"comment":"Reproducibility is hampered by missing details. The simulation environment is deferred to the ALOHA repository, and the switching condition generator is described only at a high level without network architecture, DDPM hyperparameters, training loss, or the exact point-cloud preprocessing used in the real-world experiments. The real-world comparison also uses only Diffusion Policy as a baseline, while ACT results are reported in simulation but not on real hardware. Reporting these details would substantially strengthen confidence in the 100%/88% simulation claims and the real-world comparisons.","section":"Section IV-A and Section IV-B"}],"minor_comments":[{"comment":"There are typos: 'Seqeuncing' in the abstract, 'Furthurmore' in the introduction, and inconsistent spelling of 'bimanual' as 'bimannual' in Section III-A; please proofread the final version.","section":"Abstract and Section I"},{"comment":"Figure 13 is captioned 'The leaky scenarios in three real-world experiments,' but the term 'leaky' is never defined or discussed in the text; please either define it or rename the figure.","section":"Section IV-B2"},{"comment":"The manuscript contains apparent layout artifacts: figure numbers jump from Fig. 5 to Figs. 9-13, and duplicate text blocks appear in the real-world experiment section and in Section E. These should be cleaned up before publication.","section":"Section IV-B and Section E (including Figs. 9-13)"},{"comment":"The real-world success rates in Table III are reported without confidence intervals or per-trial variance over 20 rollouts; for small differences (e.g., 80% vs 75%) this makes it hard to assess whether the superiority is meaningful. Please add confidence intervals or raw trial data.","section":"Table III"},{"comment":"Equation (2) introduces the notation \\hat{\\epsilon}(V|C_{o_1},...,C_{o_M}) without explicitly defining how it relates to the standard noise-prediction objective of the DDPM; please clarify the notation and the training loss used for the switching condition generator.","section":"Section III-B"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for a robotics venue and the main idea is promising. The load-bearing issue is the SafeBiOp feasibility validator: the claimed certification is not supported by the current formulation or by the 70% success rate in the unsafe setup. This is fixable with additional validation and careful rewording, so I recommend major revision rather than rejection. I did not identify concerns about novelty disclosure or citation behavior beyond the usual expectation that the comparison with recent hybrid TAMP/IL works be expanded."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nShort version: this paper deserves a serious referee. It is not a paradigm shift, but it is a real, working combination: it takes bimanual demonstrations, segments them into contact-rich and object-centric phases using scene graphs, and lets a TAMP planner interleave a learned policy with scripted primitives. The switching condition generator is learned from point clouds, which removes the explicit 6-DoF pose estimator that prior hybrid systems (Mandlekar et al., SkillMimicGen) rely on. That is a genuine step.\n\nThe experiments support the main claim directionally. In simulation, SViP keeps high success under XY-OOD and XYH-OOD where ACT and DP collapse. Real-world results on Object Handoff, Screwdriver Packing, and Cup-sleeve Insertion are consistently better than Diffusion Policy across ID, OOD, and Tilted setups. Twenty real demos per task is a fair demonstration of data efficiency. The novel-task experiments (Table-to-Bin, Multiple Instructions) show composition that is not just scripting.\n\nNow the soft spots, in proportion. First, the SafeBiOp feasibility validator. The paper calls it a way to certify collision-freeness, but it is an MLP trained to predict collision probability from the initiation configuration and obstacle pose. The learned policy is closed-loop and reactive to point clouds; nothing in the training data captures object pose, grasp quality, or perceptual variation at execution time. So \"certified\" is overclaiming. It is a heuristic, and the 70% success in the unsafe scenario is consistent with that. This is not fatal to the core idea—the system still works—but the authors should either soften the language or add features that account for the closed-loop trajectory.\n\nSecond, the evaluation has holes. Twenty rollouts per real-world cell with no error bars makes the margin between SViP and DP less convincing than the table suggests. The simulation environment is borrowed from ALOHA, which is fine, but details are deferred to that repository. Equation 3 is referenced but I could not find it in the manuscript; hyperparameters for the generators and validator are largely unspecified. And there is no direct comparison with the closest prior hybrid systems, Mandlekar et al. or SkillMimicGen, even in simplified form.\n\nThe citation pattern is fair. The authors build on the right prior work and are explicit about the manual PDDL and single-view depth limitations in the conclusion.\n\nWho is this for? Anyone working on hybrid learned-scripted manipulation, compositional skills, or data-efficient bimanual IL. It deserves peer review. I would suggest major revision: fix the SafeBiOp claim, report error bars or more rollouts, include the missing equation and hyperparameters, and add at least one direct hybrid baseline.\n\nMy verdict: conditional accept, leaning positive.","headline":"A genuinely useful hybrid system that shows learned visuomotor policies can be stitched into TAMP with point-cloud-based switching, but the feasibility validator is a heuristic and the evaluation has holes.","tokens_in":12599,"tokens_out":2187,"would_cite":true,"duration_ms":21601,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SViP shows that 20 demonstrations plus task-and-motion planning can replace hundreds of demos for bimanual manipulation.","keywords":["bimanual manipulation","imitation learning","task and motion planning","visuomotor policy","diffusion model","scene graph","out-of-distribution generalization","motion primitives"],"falsifier":"Collect a test set of cup-sleeve insertion rollouts where the overhead camera is moved to a new viewpoint or a second object partially occludes the sleeve, then run SViP; if success does not fall sharply relative to the 75 percent out-of-distribution result, then point-cloud distribution mismatch is not the limiting factor. Alternatively, log the feasibility validator's predicted collision probability for many random starting configurations and compare with actual collisions measured during execution; any large divergence would show that initiation-configuration-only prediction cannot certify SafeBiOp.","tokens_in":11625,"feed_emoji":"🤖","tokens_out":8776,"duration_ms":78402,"temperature":0.7,"pith_summary":"Visuomotor policies trained by imitation tend to fail when the scene shifts a few centimeters from the training distribution, and their errors accumulate over long horizons. SViP argues that the fix is not more data but a division of labor: let a learned policy own only the contact-rich bimanual moments, and hand everything else—picking, placing, transiting, transferring—to parameterized scripted motion primitives that a task-and-motion planner can re-sequence at run time. The paper claims that with only 20 real demonstrations, this hybrid achieves 100 percent success on in-distribution and XY out-of-distribution bimanual insertion in simulation, 88 percent when object headings also shift, and consistently beats a diffusion-policy baseline across three real-world tasks. It also reports that the planner can discover longer sequences—such as relocating an obstacle or an unreachable object—to satisfy reachability and collision constraints, and can compose several learned skills to reach novel goals like clearing the table into a bin.","feed_headline":"20 demos + task planning beat pure imitation on bimanual robots","feed_subtitle":"SViP lets a planner handle everything but the contact-rich moments, keeping 88-100 percent success in hard OOD setups.","key_machinery":"The scene graph is the load-bearing abstraction: it turns a raw demonstration into a symbolic description in which bimanual contact-rich segments are black-box skills with preconditions and effects, and unimanual segments are planner-controllable primitives. The switching-condition generator is a denoising diffusion model that learns the distribution of decision variables—starting and ending joint angles $q$ and object-centric trajectories $\\tau_o$—conditioned on point clouds $C_o$, with the score function factorized as $\\hat{\\varepsilon}(V|C_{o_1},...,C_{o_M}) = \\varepsilon_\\theta(q,k) + \\sum_i \\varepsilon_\\theta(\\tau_{o_i},k|C_{o_i})$; SE(3)-equivariance is achieved through a Vector Neuron architecture and an equivariant point-cloud encoder, letting the same generator work for objects at arbitrary poses without pose estimators. The feasibility validator is an MLP that predicts collision probability from the initiation configuration of a bimanual skill, which lets the planner enforce the constraint $\\mathrm{SafeBiOp}(a,h_l,h_r,q_l,q_r)$ without simulating the reactive policy's future trajectory. The planning integration uses PDDLStream-style stream functions and an adaptive sampling-based planner to interleave learned and scripted actions.","core_discovery":"SViP's central claim is that a learned visuomotor policy should be treated as one black-box subroutine inside a larger planning problem rather than as the whole controller. The paper segments human demonstrations into an event-driven scene graph whose nodes are grippers, objects, and table regions and whose edges carry continuous variables such as joint configurations and grip poses. Contact-rich bimanual segments become skills defined by pre- and post-scene graphs; the surrounding unimanual segments become object-centric motion primitives. A switching-condition generator, trained as a diffusion model with an SE(3)-equivariant point-cloud encoder, predicts the joint configurations and object-relative trajectories that let these primitives start and end exactly where the learned policy needs them. A feasibility validator predicts collision risk for a bimanual skill from its initiation configuration, and both are wrapped into a task-and-motion planner that computes a constraint-satisfying sequence. The result, the paper argues, is a data-efficient system that generalizes across out-of-distribution initial conditions, obeys reachability and safety constraints, and composes skills for goals not seen in training.","pith_inferences":["The same scene-graph abstraction could likely be applied to unimanual contact-rich skills such as screwing or wiping, since the formulation does not hinge on having two arms; a natural extension is to replace some scripted primitives with a library of learned unimanual skills and let the planner choose among them.","If the switching-condition generator is the bottleneck, then improving point-cloud completeness—for example, fusing multiple depth views or online reconstruction, which the authors name as future work—should raise the tilted cup-sleeve success from 40 percent toward the other tasks' levels; that is a testable prediction.","The framework's wall-clock limits are set by the planner's exponential growth in object count; for scenes with many objects, the paper's 7 to 27 second planning times would grow, so a hierarchical or learned proposal for plan skeletons would be a natural next step.","Because the feasibility validator reasons only from the initiation configuration, its predictions may fail when a learned policy reacts differently to visual perturbations mid-execution; testing this would require comparing validator predictions against measured collisions in out-of-distribution rollouts."],"forward_implications":["With only 20 demonstrations per task, SViP reports 100 percent success on in-distribution and XY-shifted bimanual peg insertion, and 88 percent when object heading also shifts; the compared end-to-end imitation policies drop to 2 to 42 percent in the same shifted setups.","In real-world object handoff, screwdriver packing, and cup-sleeve insertion, SViP reaches 65 to 100 percent success across in-distribution, out-of-distribution, and tilted setups, while the diffusion-policy baseline ranges from 0 to 90 percent.","Because the planner can insert extra scripted primitives, SViP can satisfy reachability and collision constraints—for example, regrasping an out-of-reach socket or moving a pole out of the way—where end-to-end policies fail outright.","Novel goals such as table-to-bin clearance and a 19-step multiple-instruction task are achievable by re-sequencing learned bimanual skills with primitives, without collecting new demonstrations for the new goal.","The method does not require object pose estimators, which matters for symmetric objects such as pegs, cups, and sleeves where pose estimation is unstable."],"supporting_citations":[{"why":"supplies the adaptive sampling-based TAMP solver that interleaves learned skills and scripted primitives","marker":"[10]"},{"why":"provides the PDDLStream formalism into which the scene graphs are parsed for symbolic planning","marker":"[11]"},{"why":"the video segmentation tool the monitor uses to track objects and grippers when constructing event-driven scene graphs","marker":"[19]"},{"why":"the action-chunking imitation baseline and the dual-arm teleoperation platform used for data collection and comparison","marker":"[27]"},{"why":"the diffusion-policy baseline that SViP is compared against in simulation and on the real robot","marker":"[5]"},{"why":"provides the MuJoCo bimanual peg-in-hole simulation and the scripted demonstration generator used for the OOD benchmarks","marker":"[28]"},{"why":"the earlier interleaved scripted-and-learned TAMP system that SViP extends by replacing pose estimators with learned switching conditions from point clouds","marker":"[18]"},{"why":"the Vector Neuron architecture that gives the object-centric trajectory generator its SE(3)-equivariant structure","marker":"[7]"}],"fun_headline_variants":["Planning plus 20 demos beats pure imitation for bimanual tasks","SViP: 20 demos and task planning for OOD bimanual generalization","Bimanual robots generalize OOD with just 20 demos via SViP","Task planning turns 20 demos into long-horizon bimanual policies","SViP: planner-assisted visuomotor policy from 20 demonstrations"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that deployment point clouds from one overhead depth camera look like those in the 20 demonstrations, so the switching-condition generator and feasibility validator trained on those clouds keep working; the paper's own tilted cup-sleeve result (40 percent) shows what happens when that premise weakens.","fun_headline_variants_meta":{"raw":{"variants":["Planning plus 20 demos beats pure imitation for bimanual tasks","SViP: 20 demos and task planning for OOD bimanual generalization","Bimanual robots generalize OOD with just 20 demos via SViP","Task planning turns 20 demos into long-horizon bimanual policies","SViP: planner-assisted visuomotor policy from 20 demonstrations"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000225,"raw_usage":{"total_tokens":1500,"prompt_tokens":1017,"completion_tokens":483,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":633,"completion_tokens_details":{"reasoning_tokens":378}},"tokens_in":633,"tokens_out":483,"duration_ms":4837,"temperature":1.0,"reasoning_tokens":378,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:41:04.257032+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Collect a test set of cup-sleeve insertion rollouts where the overhead camera is moved to a new viewpoint or a second object partially occludes the sleeve, then run SViP; if success does not fall sharply relative to the 75 percent out-of-distribution result, then point-cloud distribution mismatch is not the limiting factor. Alternatively, log the feasibility validator's predicted collision probability for many random starting configurations and compare with actual collisions measured during execution; any large divergence would show that initiation-configuration-only prediction cannot certify SafeBiOp.","supporting_citations":[{"cited_title":"Sampling-based methods for factored task and motion planning","cited_arxiv_id":null,"evidence_quote":"supplies the adaptive sampling-based TAMP solver that interleaves learned skills and scripted primitives"},{"cited_title":"Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware","cited_arxiv_id":null,"evidence_quote":"the action-chunking imitation baseline and the dual-arm teleoperation platform used for data collection and comparison"},{"cited_title":"Human-in-the-loop task and motion planning for imitation learning","cited_arxiv_id":null,"evidence_quote":"the earlier interleaved scripted-and-learned TAMP system that SViP extends by replacing pose estimators with learned switching conditions from point clouds"},{"cited_title":"Vec- tor Neurons: A general framework for SO(3)-equivariant networks","cited_arxiv_id":null,"evidence_quote":"the Vector Neuron architecture that gives the object-centric trajectory generator its SE(3)-equivariant structure"}],"review_version":2}