{"id":"396bfe2a-ee74-4228-9b40-fb3192585d56","arxiv_id":"2509.04645","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"A hybrid A* search over SE(3) point cloud transforms, with learned suggesters proposing which object to move and where, solves multi-object rearrangement without discretizing actions.","lead":"SPOT plans multi-object rearrangement by searching over continuous object transformations in point cloud space, guided by learned object and placement suggesters. The system reports higher task success than imitation policy baselines on both simulated and real pick-and-place tasks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Transition model in Sec. 4 assumes exact rigid SE(3) object motion; search states are analytic children, not physics, so planning success can overstate executable plans when contact/toppling occurs.","rationale":"The paper is a solid systems contribution: A* search is genuinely used, the ablations show search guidance matters, and the execution-success numbers provide real evidence that the approach can produce executable plans. The reader correctly identifies the transition model as the weakest assumption. My concern is not that the authors hid the issue; they report execution gaps and list failure modes. However, the gap between planning success and execution success (88% vs 63% in block stacking) is exactly what one would expect if the search graph uses a rigid-transform model that does not match true contact dynamics. The MDE helps but only as a soft cost, so it cannot repair a wrong transition: once A* commits to a child node, the node's point cloud is the one generated by the analytic transform, not by the physical outcome. Thus the central claim 'generates successful plans' is only established relative to the approximate model unless the plans are checked against real dynamics. This does not invalidate the paper; the execution results and limitations keep the verdict conditional. I would keep the reader's CONDITIONAL verdict rather than moving to accept or reject, because the evidence is positive but the transition-model assumption remains untested as a causal bottleneck.","tokens_in":18887,"tokens_out":5866,"duration_ms":69074,"concrete_test":"Re-run the 23 block-stacking initial configurations with SPOT, but generate candidate child states in two modes: (A) the current analytic rigid-transform model, and (B) a one-step PyBullet physics rollout of the same action applied to the true parent state. Execute the resulting plans with the same robot primitives and compare execution success. If mode B does not materially improve execution success, the rigid-transform assumption is not the main bottleneck; if mode B substantially improves success, the assumption is load-bearing and planning success overstates executable performance.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Sec. 4 defines p(o_{t+1}|o_t,a_t) as a rigid SE(3) transform applied to object x_t, and Sec. 4.4 derives each child node by applying that transform to the parent point cloud. This makes every internal search state an analytic construction, not a true next observation. In stacking/unstacking tasks the real transition can include toppling, sliding, or objects moving with the grasped object (e.g., moving a plate with a cup on it). The MDE (Sec. 4.3, C.1) only adds a cost C_d(n)=δ(o,a) to the A* f-value; it never corrects or replaces the child state. Consequently, the reported 'planning success' (Table 1: 88% block stacking, 100% table bussing) is success against the same approximate model that may be wrong; execution success (63% and 86%) is the only physics-grounded number. The paper's own Limitations section admits MDE can output small deviation for bad actions, but this is a symptom of the deeper issue: the search graph is not embedded in the true dynamics. The claim 'generates successful plans' is load-bearing on the validity of this transition model for the particular plans found.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SPOT, a hybrid learning-and-search planner for multi-object rearrangement. SPOT operates on segmented point clouds, samples candidate object and placement transformations from learned suggesters, and runs A* search over object-wise SE(3) transformations until a goal function is satisfied. A learned model-deviation estimator (MDE) biases the search away from actions whose predicted next state is unlikely to match execution. The output plan is executed with pick-and-place primitives. Experiments in simulation (block stacking, constrained packing) and the real world (table bussing) report planning and execution success, with comparisons to 3D Diffusion Policy, beam search, random rollouts, and an object-suggester ablation.","tokens_in":19220,"tokens_out":6877,"duration_ms":71105,"significance":"If the results hold, SPOT offers a useful alternative to symbolic task planning and latent-space planners: it avoids discretizing actions or object relationships and plans directly in point-cloud space using task-agnostic demonstration priors. The real-world table-bussing experiments and the comparison to a strong policy-learning baseline (DP3) are valuable evidence that search over continuous object transformations can be practical. The paper is transparent about its failure modes, which strengthens the presentation. The main caveat is that planning success is measured against an idealized rigid-transform dynamics model; execution success is the physically grounded number and is reported separately. The stress-test concern about the transition model lands and is the paper's most important limitation.","major_comments":[{"comment":"The search transition model treats p(o_{t+1}|o_t,a_t) as the exact application of an SE(3) transform to one object, leaving all other points unchanged (Sec. 4 and 4.4). The MDE only adds a penalty C_d(n)=δ(o,a) to f(n); it never replaces the analytic child state with a physically predicted state. Consequently, planning success in Table 1 (88%, 100%) and Figure 5 (100%) is success against the planner's internal model, not necessarily against the physical environment. Execution success is lower (63%, 86%, 84%). The Limitations section explicitly acknowledges that the MDE can output a small deviation for a bad action and thereby cause execution failures; that is a symptom of the same issue. The central claim that SPOT 'generates successful plans' is load-bearing on the fidelity of this transition model. The paper should either qualify planning success as model-relative or provide evidence (","section":"Sec. 4.4 / Sec. 4.3 / Sec. 7"},{"comment":"The paper introduces the model deviation estimator as a core component (Sec. 4.3, Sec. E.2) and lists its failure modes in Sec. 7, but no experiment ablates or even varies the MDE. Tables 1-2 and Figure 5 only ablate the object suggester and the search component. Since the MDE is the only mechanism intended to compensate for the approximate rigid-transform transition model, its contribution to both planning and execution success is unmeasured. Please add an ablation with C_d removed (or a weight sweep) and report both planning and execution success for that variant.","section":"Sec. 4.3 / Sec. 5.3"},{"comment":"The Points2Plans comparison is not numerically substantiated. The text states that SPOT 'slightly outperforms' Points2Plans and references Figure 5, but no numerical results for Points2Plans, eRDTransformer, or Pairwise-RD appear in the text or in a table. Appendix F.3 also changes the success metric from Points2Plans' relational predicates to a non-disturbance check, which complicates direct comparison. To support the stated claim, report the baseline success rates on the same 500 evaluation runs and discuss how the modified metric affects the comparison.","section":"Sec. 5.3.1 / Appendix F.3"},{"comment":"The comparison with 3D Diffusion Policy is confounded by goal conditioning. DP3 is not goal-conditioned and was trained on 23 demonstrations of a single goal, while SPOT receives the goal function at test time (Appendix F.4). The claim that SPOT 'outperforms a policy-learning approach' is therefore only established against a non-goal-conditioned baseline with limited data. Adding a goal-conditioned policy baseline or restricting the claim accordingly would make the comparison more convincing.","section":"Sec. 5.3.1 / Appendix F.4 / Table 2"}],"minor_comments":[{"comment":"The definition of planning success should state explicitly that it is evaluated on the search states obtained by applying the planned transformations, not on states after physical execution. This would make the planning/execution distinction unambiguous.","section":"Sec. 5.1"},{"comment":"The 'Beam Search' ablation uses beam width 1, which is equivalent to greedy search. Consider renaming it or explaining why a greedy search is the intended ablation.","section":"Sec. 5.2 / Appendix F.5"},{"comment":"For the real-world table-bussing results, it would help to report the number of trials per configuration and whether the 86% execution success corresponds to 12 of 14 configurations. Reporting only percentages for a 14-configuration set makes confidence intervals difficult to assess.","section":"Table 1"},{"comment":"The MDE training data for table bussing is collected by rolling out plans 'until either task execution success or failure.' If plans often terminate early, the MDE may not cover the state distribution that the planner actually encounters. Please discuss or provide evidence that the training distribution is sufficiently broad.","section":"Appendix C.2"}],"recommendation":"major_revision","confidential_remarks":"The core idea is promising and the execution results are encouraging. The main risk is that planning success is model-relative because the search graph is built on exact rigid transformations, and the MDE only penalizes, never corrects, the child states. I would like to see an MDE ablation and a clearer DP3 comparison before recommending acceptance. There are no citation or novelty concerns."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea is genuinely new and the paper delivers on it: A* search over object-wise SE(3) transformations of a segmented point cloud, with learned object and placement suggesters guiding node expansion. No symbolic state, no latent embedding, no given plan skeleton. That is a real departure from Points2Plans, eRDTransformer, and Simeonov et al. The experiments support the central claim: SPOT beats random rollouts, beam search, and DP3 on block stacking, and gets 100% planning success on real table bussing (86% execution). The ablations show search itself matters, not just node budget.\n\nWhat the paper does well: the system is put together carefully. The suggesters are trained on demonstrations that are not task-specific, with goals that differ from the demos, and success is judged by an external goal function. The limitations section is unusually honest—it explicitly says the MDE can output small deviation for a bad action. The search metrics and qualitative graphs are useful. This is a solid systems paper.\n\nNow the soft spots, in proportion. The stress-test note is right: the transition model assumes a rigid SE(3) transform of one object with everything else unchanged, so every search state is an analytic construct, not a true next observation. The MDE only adds a cost; it never corrects the child state. So the 88% block-stacking and 100% table-bussing planning success are success against that approximate model. The execution numbers (63% and 86%) are the physics-grounded ones. The paper even acknowledges this failure mode in Limitations, which is credit to the authors, but it is a load-bearing simplification. I would expect a revision to either soften the planning-success claim or provide a version of the search that uses the MDE to reject or correct states.\n\nOther issues are fixable and more minor: no ablation of the MDE or of the placement suggester against random sampling, several key hyperparameters (cost weights, node budget, thresholds) not reported, the Points2Plans baseline uses a modified metric and controller, and the real-world results lack error bars. The circularity concern is minor—the suggesters are trained on demos, but the goal configurations differ, the initial scenes are held out, and the goal function is external.\n\nWho is this for? Anyone working on learning-based manipulation planning or rearrangement. It deserves a serious referee. I would send it to review, expecting revisions that add ablations, report the missing hyperparameters, and frame planning success with the transition-model caveat.","headline":"SPOT is a genuinely new recipe—A* search over continuous SE(3) point-cloud transformations with learned samplers—and the experiments back it up, though the search's transition model is an approximation and 'planning success' should be read with that in mind.","tokens_in":19699,"tokens_out":2052,"would_cite":true,"duration_ms":21015,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SPOT claims that long-horizon multi-object rearrangement can be planned by A* search over continuous SE(3) transformations of segmented point clouds, guided by learned suggesters, with no discretization of actions or object relationships.","keywords":["robot manipulation","point cloud planning","A* search","multi-object rearrangement","SE(3) transformations","learned priors","task planning","manipulation"],"falsifier":"Run SPOT on the real-world 4-step table-bussing configuration with a cup nested inside a bowl on a plate, execute the returned plan, and check each step: if any step moves a supporting object while another object rests on it and the resting object's observed cloud diverges from the rigid-transform prediction by more than the MDE's predicted deviation, the core transition model is falsified. A cheaper surrogate: hold out transitions from the MDE dataset and compare predicted versus actual object-wise Chamfer deviations; if the MDE's ranking does not separate safe from unsafe actions (e.g., a 'b","tokens_in":18811,"feed_emoji":"🤖","tokens_out":12316,"duration_ms":97710,"temperature":0.7,"pith_summary":"SPOT's central claim is that long-horizon multi-object rearrangement can be planned directly in point-cloud space: the state is a segmented, partially observed point cloud, an action is a continuous SE(3) transformation applied to one object, and an A* search over these transformations finds a sequence that satisfies a goal condition on the cloud. The paper argues that search over this high-dimensional continuous space becomes tractable when two learned models propose which object to move and where to place it, and a learned model-deviation estimator biases search toward actions whose real-world effects match the rigid-transform prediction. The paper reports that this hybrid of learned priors and search outperforms an end-to-end policy-learning baseline: on simulated block stacking, execution success is 78%/58%/30% for 2-/3-/4-step tasks versus 13%/0%/0% for 3D Diffusion Policy, and on real-world table bussing SPOT reaches 86% execution success. If the claim holds, rearrangement planning no longer requires a hand-built symbolic ontology of objects, relationships, and actions, and goal-agnostic demonstrations are enough to learn the priors the search needs.","feed_headline":"Search stacks blocks where imitation fails: 78% vs 13%","feed_subtitle":"Planning as continuous object transformations, with no symbolic discretization, beats imitation on multi-step stacking.","key_machinery":"The central object is the search node n = [o_n, x_n, T_n, n_p]: a segmented scene point cloud plus the action (object and rigid transformation) that produced it from its parent. Three learned components make the search feasible: an object suggester scoring which object to move; a placement suggester (TAXPose-D, a conditional VAE over a discrete latent space) sampling multimodal task-relevant transformations; and a model deviation estimator predicting the Chamfer deviation between the rigidly transformed scene and the observed next scene, whose output enters the cost g(n) = C_a + w_cC_c + w_dC_d + w_pC_p. The load-bearing identity is the transition model—p(o_{t+1}|o_t,a_t) is exactly a rigid","core_discovery":"SPOT solves multi-object rearrangement as search over scene configurations in point-cloud space. The state is a partially observed, segmented point cloud; an action applies a rigid SE(3) transformation to one object's points and leaves all others fixed; a goal function decides when a cloud satisfies the task. A* expands the root cloud by sampling an object and candidate transformations from learned suggesters. Node cost combines a constant action cost, a voxel collision cost, deviation predicted by a learned model-deviation estimator, and a probability cost from the suggesters. The output plan is a sequence of transformations executed by pick-and-place primitives. Headline results: 88±2% pla","pith_inferences":["Because the transition model fixes all but one object, the method's reach is bounded by the “one rigid object at a time, no knock-on effects” regime; tasks with toppling, sliding, or deformable contact would require the deviation estimator to carry more weight than it does, or a richer dynamics model. This is my inference, not a paper claim.","The reported search metrics (1 to 49 seconds and 33 to 1711 generated nodes from 1- to 4-step tasks) suggest that, in its current form, cost grows steeply with plan length; the paper's own future-work list (Partial Expansion A*, MCTS, replanning) points to the same bottleneck.","The “no discretization” claim applies to actions and object relationships, not to task specification: the goal function and heuristic are still hand-designed per task, so full autonomy would additionally require learning those components.","A testable extension follows directly from the training setup: since the suggesters are goal-agnostic, SPOT's learned priors should transfer to novel goal functions in the same environment with only the goal function and heuristic swapped; the paper does not evaluate this."],"forward_implications":["Rearrangement planning can be done with no symbolic task ontology: no hand-defined predicates over object relationships and no discretized action set, only a goal function and learned priors from demonstrations.","Goal-agnostic demonstrations suffice: the suggesters are trained on transitions that need not share the planner's goal, so the same learned priors can serve different goals in the same environment.","Search-based planning is the source of the gains: the beam-search ablation, which picks a single child node myopically, collapses to 43%/7%/0% execution success at 2/3/4 steps, so the improvement is not merely a matter of node count.","SPOT can find paths shorter than the demonstrations it trained on, because the suggesters capture a goal-agnostic distribution of feasible moves and search explores alternatives, e.g., moving a cup aside to stack bowls directly instead of the demonstrator's extra step.","Planning success exceeds execution success in both environments (88% vs 63% block stacking; 100% vs 86% table bussing), so the remaining gap isolates the difference between the rigid-transform model and real dynamics.","The cost function itself is task-agnostic; only the heuristic and goal function are hand-designed per task, and the paper reports them as the pieces left for future work to learn."],"supporting_citations":[{"why":"TAXPose-D supplies the placement suggester: a conditional VAE over a discrete latent space that samples multimodal, task-relevant transformations for a given object and scene.","marker":"[23]"},{"why":"Points2Plans provides the constrained-packing environment and its 8000-transition demonstration dataset, and is the relational-planning baseline SPOT is compared against in that task.","marker":"[7]"},{"why":"3D Diffusion Policy is the end-to-end imitation policy baseline that SPOT must outperform on block-stacking execution success.","marker":"[33]"},{"why":"Supplies the model-deviation-estimator training recipe (offline transition labeling from executed rollouts) that SPOT adapts for its deviation cost.","marker":"[4]"},{"why":"Contact-GraspNet generates the grasp poses used to execute the planned transformations in the real world.","marker":"[32]"},{"why":"PointNet++ is the architecture used for the object suggester and the model deviation estimator.","marker":"[36]"},{"why":"CoTracker3 extracts 2D object tracks from RGB-D human-demonstration videos, from which the real-world table-bussing transition dataset is built.","marker":"[37]"},{"why":"Pairwise-RD is a relational-dynamics baseline in the constrained-packing comparison that only captures pairwise object interactions.","marker":"[8]"}],"fun_headline_variants":["SPOT plans in point-cloud space, no discretization, beats imitation 78-13","Search over point clouds nails multi-object moves where imitation fails","Replace symbols with point clouds: SPOT outperforms imitation on stacking","Continuous-action search from point clouds trumps imitation on rearrangement","From raw clouds to plans: SPOT beats imitation 78% to 13% on stacking"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The load-bearing premise is that moving one object leaves every other point in the scene exactly where it was—an action is precisely a rigid SE(3) transform of one object's point cloud; if real dynamics make other objects shift, topple, or slide (a cup falling off a moved plate), the child states the search plans through are not the states that occur at execution.","fun_headline_variants_meta":{"raw":{"variants":["SPOT plans in point-cloud space, no discretization, beats imitation 78-13","Search over point clouds nails multi-object moves where imitation fails","Replace symbols with point clouds: SPOT outperforms imitation on stacking","Continuous-action search from point clouds trumps imitation on rearrangement","From raw clouds to plans: SPOT beats imitation 78% to 13% on stacking"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001315,"raw_usage":{"total_tokens":5178,"prompt_tokens":714,"completion_tokens":4464,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":458,"completion_tokens_details":{"reasoning_tokens":4376}},"tokens_in":458,"tokens_out":4464,"duration_ms":30525,"temperature":1.0,"reasoning_tokens":4376,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T05:56:27.768240+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run SPOT on the real-world 4-step table-bussing configuration with a cup nested inside a bowl on a plate, execute the returned plan, and check each step: if any step moves a supporting object while another object rests on it and the resting object's observed cloud diverges from the rigid-transform prediction by more than the MDE's predicted deviation, the core transition model is falsified. A cheaper surrogate: hold out transitions from the MDE dataset and compare predicted versus actual object-wise Chamfer deviations; if the MDE's ranking does not separate safe from unsafe actions (e.g., a 'b","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"TAXPose-D supplies the placement suggester: a conditional VAE over a discrete latent space that samples multimodal, task-relevant transformations for a given object and scene."},{"cited_title":"Points2Plans: From Point Clouds to Long-Horizon Plans with Composable Relational Dynamics","cited_arxiv_id":"2408.14769","evidence_quote":"Points2Plans provides the constrained-packing environment and its 8000-transition demonstration dataset, and is the relational-planning baseline SPOT is compared against in that task."},{"cited_title":"Learning Model Preconditions for Planning with Multiple Models","cited_arxiv_id":"2206.05573","evidence_quote":"Supplies the model-deviation-estimator training recipe (offline transition labeling from executed rollouts) that SPOT adapts for its deviation cost."},{"cited_title":"Sundermeyer, A","cited_arxiv_id":null,"evidence_quote":"Contact-GraspNet generates the grasp poses used to execute the planned transformations in the real world."},{"cited_title":"Karaev, I","cited_arxiv_id":null,"evidence_quote":"CoTracker3 extracts 2D object tracks from RGB-D human-demonstration videos, from which the real-world table-bussing transition dataset is built."},{"cited_title":"Predicting Stable Configurations for Semantic Placement of Novel Objects","cited_arxiv_id":"2108.12062","evidence_quote":"Pairwise-RD is a relational-dynamics baseline in the constrained-packing comparison that only captures pairwise object interactions."}],"review_version":1}