Pith. sign in

REVIEW 4 major objections 4 minor 40 references

SPOT claims that long-horizon multi-object rearrangement can be planned by A* search over continuous SE(3) transformations of segmented point clouds, guided by learned suggesters, with no discretization of actions or object relationships.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A hybrid A* search over SE(3) point cloud transforms, with learned suggesters proposing which object to move and where, solves multi-object rearrangement without discretizing actions.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection SPOT is a genuinely new recipe—A* search over continuous SE(3) point-cloud transformations with learned samplers—and the experiments back it up, though the search's transition model is an approximation and 'planning success' should be read with that in mind. the 4 major comments →

arxiv 2509.04645 v1 pith:MR4B6ERG submitted 2025-09-04 cs.RO

Planning from Point Clouds over Continuous Actions for Multi-object Rearrangement

Kallol Saha , Amber Li , Angela Rodriguez-Izquierdo , Lifan Yu , Ben Eisner , Maxim Likhachev , David Held This is my paper
classification cs.RO
keywords robot manipulationpoint cloud planningA* searchmulti-object rearrangementSE(3) transformationslearned priorstask planningmanipulation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SPOT's central claim is that long-horizon multi-object rearrangement can be planned directly in point-cloud space: the state is a segmented, partially observed point cloud, an action is a continuous SE(3) transformation applied to one object, and an A* search over these transformations finds a sequence that satisfies a goal condition on the cloud. The paper argues that search over this high-dimensional continuous space becomes tractable when two learned models propose which object to move and where to place it, and a learned model-deviation estimator biases search toward actions whose real-world effects match the rigid-transform prediction. The paper reports that this hybrid of learned priors and search outperforms an end-to-end policy-learning baseline: on simulated block stacking, execution success is 78%/58%/30% for 2-/3-/4-step tasks versus 13%/0%/0% for 3D Diffusion Policy, and on real-world table bussing SPOT reaches 86% execution success. If the claim holds, rearrangement planning no longer requires a hand-built symbolic ontology of objects, relationships, and actions, and goal-agnostic demonstrations are enough to learn the priors the search needs.

Core claim

SPOT solves multi-object rearrangement as search over scene configurations in point-cloud space. The state is a partially observed, segmented point cloud; an action applies a rigid SE(3) transformation to one object's points and leaves all others fixed; a goal function decides when a cloud satisfies the task. A* expands the root cloud by sampling an object and candidate transformations from learned suggesters. Node cost combines a constant action cost, a voxel collision cost, deviation predicted by a learned model-deviation estimator, and a probability cost from the suggesters. The output plan is a sequence of transformations executed by pick-and-place primitives. Headline results: 88±2% pla

What carries the argument

The central object is the search node n = [o_n, x_n, T_n, n_p]: a segmented scene point cloud plus the action (object and rigid transformation) that produced it from its parent. Three learned components make the search feasible: an object suggester scoring which object to move; a placement suggester (TAXPose-D, a conditional VAE over a discrete latent space) sampling multimodal task-relevant transformations; and a model deviation estimator predicting the Chamfer deviation between the rigidly transformed scene and the observed next scene, whose output enters the cost g(n) = C_a + w_cC_c + w_dC_d + w_pC_p. The load-bearing identity is the transition model—p(o_{t+1}|o_t,a_t) is exactly a rigid

Load-bearing premise

The load-bearing premise is that moving one object leaves every other point in the scene exactly where it was—an action is precisely a rigid SE(3) transform of one object's point cloud; if real dynamics make other objects shift, topple, or slide (a cup falling off a moved plate), the child states the search plans through are not the states that occur at execution.

What would settle it

Run SPOT on the real-world 4-step table-bussing configuration with a cup nested inside a bowl on a plate, execute the returned plan, and check each step: if any step moves a supporting object while another object rests on it and the resting object's observed cloud diverges from the rigid-transform prediction by more than the MDE's predicted deviation, the core transition model is falsified. A cheaper surrogate: hold out transitions from the MDE dataset and compare predicted versus actual object-wise Chamfer deviations; if the MDE's ranking does not separate safe from unsafe actions (e.g., a 'b

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Rearrangement planning can be done with no symbolic task ontology: no hand-defined predicates over object relationships and no discretized action set, only a goal function and learned priors from demonstrations.
  • Goal-agnostic demonstrations suffice: the suggesters are trained on transitions that need not share the planner's goal, so the same learned priors can serve different goals in the same environment.
  • Search-based planning is the source of the gains: the beam-search ablation, which picks a single child node myopically, collapses to 43%/7%/0% execution success at 2/3/4 steps, so the improvement is not merely a matter of node count.
  • SPOT can find paths shorter than the demonstrations it trained on, because the suggesters capture a goal-agnostic distribution of feasible moves and search explores alternatives, e.g., moving a cup aside to stack bowls directly instead of the demonstrator's extra step.
  • Planning success exceeds execution success in both environments (88% vs 63% block stacking; 100% vs 86% table bussing), so the remaining gap isolates the difference between the rigid-transform model and real dynamics.
  • The cost function itself is task-agnostic; only the heuristic and goal function are hand-designed per task, and the paper reports them as the pieces left for future work to learn.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Because the transition model fixes all but one object, the method's reach is bounded by the “one rigid object at a time, no knock-on effects” regime; tasks with toppling, sliding, or deformable contact would require the deviation estimator to carry more weight than it does, or a richer dynamics model. This is my inference, not a paper claim.
  • The reported search metrics (1 to 49 seconds and 33 to 1711 generated nodes from 1- to 4-step tasks) suggest that, in its current form, cost grows steeply with plan length; the paper's own future-work list (Partial Expansion A*, MCTS, replanning) points to the same bottleneck.
  • The “no discretization” claim applies to actions and object relationships, not to task specification: the goal function and heuristic are still hand-designed per task, so full autonomy would additionally require learning those components.
  • A testable extension follows directly from the training setup: since the suggesters are goal-agnostic, SPOT's learned priors should transfer to novel goal functions in the same environment with only the goal function and heuristic swapped; the paper does not evaluate this.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes SPOT, a hybrid learning-and-search planner for multi-object rearrangement. SPOT operates on segmented point clouds, samples candidate object and placement transformations from learned suggesters, and runs A* search over object-wise SE(3) transformations until a goal function is satisfied. A learned model-deviation estimator (MDE) biases the search away from actions whose predicted next state is unlikely to match execution. The output plan is executed with pick-and-place primitives. Experiments in simulation (block stacking, constrained packing) and the real world (table bussing) report planning and execution success, with comparisons to 3D Diffusion Policy, beam search, random rollouts, and an object-suggester ablation.

Significance. If the results hold, SPOT offers a useful alternative to symbolic task planning and latent-space planners: it avoids discretizing actions or object relationships and plans directly in point-cloud space using task-agnostic demonstration priors. The real-world table-bussing experiments and the comparison to a strong policy-learning baseline (DP3) are valuable evidence that search over continuous object transformations can be practical. The paper is transparent about its failure modes, which strengthens the presentation. The main caveat is that planning success is measured against an idealized rigid-transform dynamics model; execution success is the physically grounded number and is reported separately. The stress-test concern about the transition model lands and is the paper's most important limitation.

major comments (4)
  1. [Sec. 4.4 / Sec. 4.3 / Sec. 7] The search transition model treats p(o_{t+1}|o_t,a_t) as the exact application of an SE(3) transform to one object, leaving all other points unchanged (Sec. 4 and 4.4). The MDE only adds a penalty C_d(n)=δ(o,a) to f(n); it never replaces the analytic child state with a physically predicted state. Consequently, planning success in Table 1 (88%, 100%) and Figure 5 (100%) is success against the planner's internal model, not necessarily against the physical environment. Execution success is lower (63%, 86%, 84%). The Limitations section explicitly acknowledges that the MDE can output a small deviation for a bad action and thereby cause execution failures; that is a symptom of the same issue. The central claim that SPOT 'generates successful plans' is load-bearing on the fidelity of this transition model. The paper should either qualify planning success as model-relative or provide evidence (
  2. [Sec. 4.3 / Sec. 5.3] The paper introduces the model deviation estimator as a core component (Sec. 4.3, Sec. E.2) and lists its failure modes in Sec. 7, but no experiment ablates or even varies the MDE. Tables 1-2 and Figure 5 only ablate the object suggester and the search component. Since the MDE is the only mechanism intended to compensate for the approximate rigid-transform transition model, its contribution to both planning and execution success is unmeasured. Please add an ablation with C_d removed (or a weight sweep) and report both planning and execution success for that variant.
  3. [Sec. 5.3.1 / Appendix F.3] The Points2Plans comparison is not numerically substantiated. The text states that SPOT 'slightly outperforms' Points2Plans and references Figure 5, but no numerical results for Points2Plans, eRDTransformer, or Pairwise-RD appear in the text or in a table. Appendix F.3 also changes the success metric from Points2Plans' relational predicates to a non-disturbance check, which complicates direct comparison. To support the stated claim, report the baseline success rates on the same 500 evaluation runs and discuss how the modified metric affects the comparison.
  4. [Sec. 5.3.1 / Appendix F.4 / Table 2] The comparison with 3D Diffusion Policy is confounded by goal conditioning. DP3 is not goal-conditioned and was trained on 23 demonstrations of a single goal, while SPOT receives the goal function at test time (Appendix F.4). The claim that SPOT 'outperforms a policy-learning approach' is therefore only established against a non-goal-conditioned baseline with limited data. Adding a goal-conditioned policy baseline or restricting the claim accordingly would make the comparison more convincing.
minor comments (4)
  1. [Sec. 5.1] The definition of planning success should state explicitly that it is evaluated on the search states obtained by applying the planned transformations, not on states after physical execution. This would make the planning/execution distinction unambiguous.
  2. [Sec. 5.2 / Appendix F.5] The 'Beam Search' ablation uses beam width 1, which is equivalent to greedy search. Consider renaming it or explaining why a greedy search is the intended ablation.
  3. [Table 1] For the real-world table-bussing results, it would help to report the number of trials per configuration and whether the 86% execution success corresponds to 12 of 14 configurations. Reporting only percentages for a 14-configuration set makes confidence intervals difficult to assess.
  4. [Appendix C.2] The MDE training data for table bussing is collected by rolling out plans 'until either task execution success or failure.' If plans often terminate early, the MDE may not cover the state distribution that the planner actually encounters. Please discuss or provide evidence that the training distribution is sufficiently broad.

Circularity Check

0 steps flagged

No significant circularity: SPOT's planning success is measured against an externally specified goal function, and execution success provides an independent ground-truth check. The only notable self-citation (TAXPose-D) is an implementation choice for the placement suggester, not a load-bearing uniqueness claim.

full rationale

The paper's derivation chain is not circular. SPOT defines a transition model p(o_{t+1}|o_t,a_t) as a rigid SE(3) transform applied to one object (Sec. 4), and search nodes are generated by applying that transform. Planning success is then evaluated by checking an externally specified goal function G(o) on the resulting point cloud. This is a standard planning formulation, not a case of a fitted parameter being renamed as a prediction: G is hand-defined and independent of the learned suggesters, and the search can fail to find a goal even under the model. The learned object suggester, placement suggester, and model deviation estimator are trained on demonstrations and real-robot rollouts, respectively, and they guide but do not define task success. Execution success is measured by physically executing the plan in simulation or the real world, providing an independent check that the planning result is not merely self-consistent. The transition-model assumption (exact rigid motion, no toppling/sliding) is an explicit modeling limitation; the paper's own MDE and Limitations section acknowledge that bad actions can cause deviation, and execution failures are reported. This is a correctness risk, not circularity. The only self-citation of note is TAXPose-D [23], which shares a co-author (D. Held) and is used as the implementation of the placement suggester. However, it is not used to justify the core planning claim, it is a published external method, and its outputs are separately evaluated (Table 5) and ablated (e.g., random rollouts). No uniqueness theorem or ansatz is imported from the cited prior work. Therefore, no prediction in the paper reduces by construction to the learned inputs or to a self-citation chain.

Axiom & Free-Parameter Ledger

7 free parameters · 6 axioms · 0 invented entities

The paper contributes no new physical entities. The central claim rests on task-specific demonstrations, hand-designed cost weights (some unreported), and the rigid-transform transition model. The learned suggesters and MDE are trained modules, not invented physical quantities.

free parameters (7)
  • k (candidate transformations per object) = 10 (block stacking), 5 (constrained packing), 3 (table bussing)
    Number of child transformations sampled from the placement suggester per object at each A* expansion; controls search breadth and coverage of goal-relevant actions (Appendix E.1).
  • m (goal count for multi-goal A*) = 1 (simulation), 10 (real-world)
    Search continues until m goal nodes are found; affects which plan is returned and how much search time is spent (Appendix E.4).
  • Cost weights wc, wd, wp = not reported
    Weights in the A* cost g(n)=Ca+wc*Cc+wd*Cd+wp*Cp (Sec 4.4.1); the paper never gives their numeric values, which is a significant unstated detail for reproduction.
  • Action cost Ca = 0.01
    Constant step cost used in the cost function (Appendix E.2).
  • Node expansion budget = not reported (200 for random rollouts)
    A* terminates when a maximum node expansion limit is exceeded (Sec 4.4), but the actual budget used for SPOT is not stated.
  • MDE label clipping (clip max) and epsilon = 3.2/5000 and 1/0.01
    Outlier clipping and stabilization constants in the Chamfer-distance deviation label (Eq. 1, Table 4).
  • Heuristic and goal thresholds = not reported
    Alignment tolerance in block stacking and XY-distance threshold in table bussing that determine h(n) and G(o) (Appendix E.3).
axioms (6)
  • domain assumption Objects are rigid and move independently; the next point cloud is exactly the current point cloud with a single object transformed by T (Sec 4 Markovian model).
    This is the transition model used to generate child nodes in A*. If contacts cause toppling or non-rigid effects, the search states are invalid; the MDE only penalizes such actions, it does not correct the state.
  • domain assumption Demonstrations D (non-task-specific) provide enough coverage that learned suggesters can propose goal-relevant transformations for novel goals (Sec 3, 4.2).
    SPOT's success hinges on the placement suggester proposing at least one transform per object, per step, that lies on a path to the goal; if a required placement is absent from the demo distribution, the search cannot find it.
  • domain assumption Segmentation and semantic labeling of the partial point cloud is accurate (SAM2/Grounding DINO); object identities and masks are correct (Sec 3).
    All downstream components depend on the object-wise masks; errors propagate to the suggesters and to the goal and heuristic evaluation.
  • domain assumption The goal function G and heuristic h are given and correctly evaluate point clouds (Sec 3, E.3).
    The planner optimizes toward G; if G is mis-specified or the heuristic is poor, search success does not correspond to task success. The paper acknowledges these are task-specific.
  • standard math A* search with a finite branching factor and a bounded expansion budget terminates and finds a goal when one exists in the generated tree (standard A* properties).
    The paper uses A* in the standard sense; the admissibility of h is asserted, but the search does not need to prove optimality because it stops at the first goal(s) and selects by score.
  • domain assumption The MDE predicts execution deviation well enough to keep the search away from physically invalid actions (Sec 4.3).
    No ablation of the MDE is reported, but the paper's own limitations state that incorrect MDE predictions can cause both planning and execution failures.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Planning from Point Clouds over Continuous Actions for Multi-object Rearrangement." pith.science (2026). https://pith.science/paper/MR4B6ERG

@misc{pith2026250904645,
  author       = {Pith},
  title        = {Pith review of: Planning from Point Clouds over Continuous Actions for Multi-object Rearrangement},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MR4B6ERG}},
  note         = {Machine review of arXiv:2509.04645}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Long-horizon planning for robot manipulation is a challenging problem that requires reasoning about the effects of a sequence of actions on a physical 3D scene. While traditional task planning methods are shown to be effective for long-horizon manipulation, they require discretizing the continuous state and action space into symbolic descriptions of objects, object relationships, and actions. Instead, we propose a hybrid learning-and-planning approach that leverages learned models as domain-specific priors to guide search in high-dimensional continuous action spaces. We introduce SPOT: Search over Point cloud Object Transformations, which plans by searching for a sequence of transformations from an initial scene point cloud to a goal-satisfying point cloud. SPOT samples candidate actions from learned suggesters that operate on partially observed point clouds, eliminating the need to discretize actions or object relationships. We evaluate SPOT on multi-object rearrangement tasks, reporting task planning success and task execution success in both simulation and real-world environments. Our experiments show that SPOT generates successful plans and outperforms a policy-learning approach. We also perform ablations that highlight the importance of search-based planning.

Figures

Figures reproduced from arXiv: 2509.04645 by Amber Li, Angela Rodriguez-Izquierdo, Ben Eisner, David Held, Kallol Saha, Lifan Yu, Maxim Likhachev.

Figure 1
Figure 1. Figure 1: SPOT: Search over Point cloud Object Transformations. Our method solves multi-object rear￾rangement tasks by planning directly in point cloud space, without using any privileged ground-truth informa￾tion such as the ground-truth object states. Starting from a segmented point cloud, it performs A* search over object-wise SE(3) transformations until a goal configuration is found. The output plan is then exec… view at source ↗
Figure 2
Figure 2. Figure 2: Learned object and placement suggesters. Left: The object suggester predicts a probability dis￾tribution over which objects in the scene can be feasibly moved, given a point cloud observation of the scene. Right: Given an object and the scene point cloud, the placement suggester samples candidate transformations indicating where the object could be moved next. point cloud plan P = (a1, a2, . . . aT −1) usi… view at source ↗
Figure 3
Figure 3. Figure 3: System overview. Our method takes an RGB-D image, object names, and a goal function as input. It generates a segmented point cloud and uses A* search to plan over continuous actions (object transforma￾tions) until a goal-satisfying point cloud is found. Node expansion occurs by sampling an object to move from the object suggester, then sampling a corresponding transformation from the placement suggester. T… view at source ↗
Figure 4
Figure 4. Figure 4: Initial configurations. We use initial configurations of varying complexity for each task. In a 4- step table bussing task, the robot first has to remove the cup from the plate before stacking all objects. For block stacking, where the goal is to arrange blocks in the order red, green, blue from top to bottom, a 4-step task involves unstacking all blocks before restacking them in the correct order. The con… view at source ↗
Figure 5
Figure 5. Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Efficient Path-finding. SPOT finds a more efficient plan than the video demonstration by first moving the cup beside the plate, creating space to stack the bowls directly. In contrast, the demonstration spends an extra step to remove the cup from the plate. some success on 2-step tasks, it is unable to complete any 3 or 4-step tasks, while SPOT has a much higher success rate. 5.3.2 Real-world Results We sh… view at source ↗
Figure 7
Figure 7. Figure 7: Real World Data Collection. Our demonstration data consists of RGB-D videos of a human moving the objects in the scene. We use Grounded Segment Anything [31] and CoTracker3 [37] to extract 2D tracks from which we extract a 2D velocity-time graph for each object. For each detected object movement, RANSAC￾based SVD computes the best rigid transformation (4×4 matrices) representing that movement. point cloud … view at source ↗
Figure 8
Figure 8. Figure 8: Real-world setup. Left: Our real-world setup of the table bussing environment with the RGB-D camera, Franka arm, and a set of objects. Right: All of the objects (plates, bowls, and cups) seen in the table bussing environment. F.3 Points2Plans Comparison Constrained Packing. We define success in the constrained packing task to be when all of the objects are arranged on top of the shelf surface without being… view at source ↗
Figure 9
Figure 9. Figure 9: Task execution success rate as a function of task complexity in the simulation block stacking envi [PITH_FULL_IMAGE:figures/full_fig_p022_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Task execution success rate as a function of task complexity in the real-world table bussing environ [PITH_FULL_IMAGE:figures/full_fig_p023_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Expanded graph example 1. The graph represents the expanded nodes for a 2-step table bussing configuration where a bowl is on a plate and a cup is inside a bowl on the table. A plan is found that first moves the cup to the table, then stacks the bowls, and places the cup on the plate. The plan found is marked with green colored edges 23 [PITH_FULL_IMAGE:figures/full_fig_p023_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Expanded graph example 2. The graph represents the expanded nodes for a 4-step table bussing configuration where a cup is initially placed on a plate. A plan is found that first moves the cup to the table, then stacks both the bowls, and then moves the cup back. The plan found is marked with green colored edges [PITH_FULL_IMAGE:figures/full_fig_p024_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Expanded graph example 3. The graph represents the expanded nodes for a 3-step table bussing configuration where a cup is initially placed inside a bowl on top of the plate, and a separate bowl is placed on the table. A plan is found that first moves the cup to the table, stacks the remaining bowl from the table on top of the bowl already on the plate, then moves the cup back inside the bowls. The plan fo… view at source ↗
Figure 14
Figure 14. Figure 14: Plan graph example 1. The graph represents all of the plans and goals found for a 3-step table bussing configuration with multi-goal A* where 2 plates, a bowl and a cup are placed separately on the table. We see in this graph that A* search finds multimodal paths to achieve the goal. It may either produce a plan that places the cup on the plate first and then stacks the bowls, or it may stack the bowls fi… view at source ↗
Figure 15
Figure 15. Figure 15: Plan graph example 2. The graph represents all of the plans and goals found for a 3-step table bussing configuration where a plate is placed on the table, alongside a cup inside a bowl, with another bowl placed separately on the table. We see in this graph that A* search finds multimodal paths to achieve the goal. It may either move the cup inside the bowl to the table first, and then stack the bowls befo… view at source ↗
Figure 16
Figure 16. Figure 16: Plan graph example 3. The graph represents all of the plans and goals found for a 3-step table bussing configuration where a cup is placed inside a bowl on a plate, alongside another bowl placed separately on the table. We see in this graph that A* search finds multimodal paths of varying lengths to achieve the goal. The cup may be first moved to the table, then the bowls can be stacked, before moving the… view at source ↗
Figure 17
Figure 17. Figure 17: Plan graph example 4. This graphs shows a plan found for a 4-step block stacking initial configu￾ration. Since we set m = 1 for multi-goal search, the graph contains only one plan, which is the first plan that was found during search. 26 [PITH_FULL_IMAGE:figures/full_fig_p026_17.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

40 extracted references · 30 canonical work pages · 10 internal anchors

  1. [1]

    L. P. Kaelbling and T. Lozano-P´erez. Integrated task and motion planning in belief space. The International Journal of Robotics Research, 32(9-10):1194–1227, 2013

    2013

  2. [2]

    H. M. Pasula, L. S. Zettlemoyer, and L. P. Kaelbling. Learning symbolic models of stochastic domains. Journal of Artificial Intelligence Research, 29:309–352, jul 2007. doi:10.1613/jair

    doi:10.1613/jair 2007

  3. [3]

    Plaku and G

    E. Plaku and G. D. Hager. Sampling-based motion and symbolic action planning with geo- metric and differential constraints. In 2010 IEEE International Conference on Robotics and Automation, pages 5002–5008, 2010. doi:10.1109/ROBOT.2010.5509563

    work page arXiv 2010

  4. [4]

    Learning Model Preconditions for Planning with Multiple Models

    A. LaGrassa and O. Kroemer. Learning model preconditions for planning with multiple mod- els, 2022. URL https://arxiv.org/abs/2206.05573

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  5. [5]

    McConachie, T

    D. McConachie, T. Power, P. Mitrano, and D. Berenson. Learning when to trust a dynamics model for planning in reduced state spaces. IEEE Robotics and Automation Letters , 5(2): 3540–3547, Apr. 2020. ISSN 2377-3774. doi:10.1109/lra.2020.2972858. URL http://dx. doi.org/10.1109/LRA.2020.2972858

    work page arXiv 2020

  6. [6]

    Keep it Simple: Data-efficient Learning for Controlling Complex Systems with Simple Models

    T. Power and D. Berenson. Keep it simple: Data-efficient learning for controlling complex systems with simple models, 2021. URL https://arxiv.org/abs/2102.02493

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  7. [7]

    Points2Plans: From Point Clouds to Long-Horizon Plans with Composable Relational Dynamics

    Y . Huang, C. Agia, J. Wu, T. Hermans, and J. Bohg. Points2plans: From point clouds to long-horizon plans with composable relational dynamics. arXiv preprint arXiv:2408.14769, 2024. 11

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  8. [8]

    Predicting Stable Configurations for Semantic Placement of Novel Objects

    C. Paxton, C. Xie, T. Hermans, and D. Fox. Predicting stable configurations for semantic placement of novel objects, 2021. URL https://arxiv.org/abs/2108.12062

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  9. [9]

    X. Lin, C. Qi, Y . Zhang, Z. Huang, K. Fragkiadaki, Y . Li, C. Gan, and D. Held. Planning with spatial-temporal abstraction from point clouds for deformable object manipulation, 2023. URL https://arxiv.org/abs/2210.15751

    Pith/arXiv arXiv 2023

  10. [11]

    Ichter and M

    B. Ichter and M. Pavone. Robot motion planning in learned latent spaces. IEEE Robotics and Automation Letters, 4(3):2407–2414, 2019. doi:10.1109/LRA.2019.2901898

    work page arXiv 2019

  11. [12]

    Latent Space Planning for Multi-Object Manipulation with Environment-Aware Relational Classifiers

    Y . Huang, N. C. Taylor, A. Conkey, W. Liu, and T. Hermans. Latent space planning for multi-object manipulation with environment-aware relational classifiers, 2024. URL https: //arxiv.org/abs/2305.10857

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  12. [13]

    Simeonov, Y

    A. Simeonov, Y . Du, B. Kim, F. R. Hogan, J. Tenenbaum, P. Agrawal, and A. Rodriguez. A long horizon planning framework for manipulating rigid pointcloud objects. In Confer- ence on Robot Learning (CoRL) , 2020. URL https://anthonysimeonov.github.io/ rpo-planning-framework/

    work page 2020

  13. [14]

    W. Liu, C. Paxton, T. Hermans, and D. Fox. Structformer: Learning spatial structure for language-guided semantic rearrangement of novel objects. In ICRA 2022, 2022

    work page 2022

  14. [15]

    C. Agia, T. Migimatsu, J. Wu, and J. Bohg. Stap: Sequencing task-agnostic policies. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 7951–7958, 2023. doi:10.1109/ICRA48891.2023.10160220

    arXiv 2023

  15. [16]

    K. Lin, C. Agia, T. Migimatsu, M. Pavone, and J. Bohg. Text2motion: from natural language instructions to feasible plans. Autonomous Robots, Nov 2023. ISSN 1573-7527. doi:10.1007/ s10514-023-10131-7. URL https://doi.org/10.1007/s10514-023-10131-7

    doi:10.1007/s10514-023-10131-7 2023

  16. [17]

    U. A. Mishra, S. Xue, Y . Chen, and D. Xu. Generative skill chaining: Long-horizon skill planning with diffusion models. In 7th Annual Conference on Robot Learning , 2023. URL https://openreview.net/forum?id=HtJE9ly5dT

    work page 2023

  17. [18]

    T. Xue, A. Razmjoo, S. Shetty, and S. Calinon. Logic-skill programming: An optimization- based approach to sequential skill planning, 2024. URL https://arxiv.org/abs/2405. 04082

    work page 2024

  18. [19]

    J. O. de Haro, V . N. Hartmann, O. S. Oguz, and M. Toussaint. Learning efficient constraint graph sampling for robotic sequential manipulation. CoRR, abs/2011.04828, 2020. URL https://arxiv.org/abs/2011.04828

    work page internal anchor Pith review Pith/arXiv arXiv 2011

  19. [20]

    J. Tang, Z. Ye, Y . Yan, Z. Zheng, T. Gao, and Y . Jin. Zero-shot robotic manipulation with language-guided instruction and formal task planning, 2025. URL https://arxiv.org/ abs/2501.15214

    work page arXiv 2025

  20. [21]

    K. Rana, J. Haviland, S. Garg, J. Abou-Chakra, I. Reid, and N. Suenderhauf. Sayplan: Ground- ing large language models using 3d scene graphs for scalable robot task planning, 2023. URL https://arxiv.org/abs/2307.06135

    Pith/arXiv arXiv 2023

  21. [22]

    Planning for Multi-Object Manipulation with Graph Neural Network Relational Classifiers

    Y . Huang, A. Conkey, and T. Hermans. Planning for multi-object manipulation with graph neural network relational classifiers, 2023. URL https://arxiv.org/abs/2209.11943

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  22. [23]

    J. Wang, O. Donca, and D. Held. Learning distributional demonstration spaces for task-specific cross-pose estimation. In International Conference on Robotics and Automation (ICRA), 2024. 12

    work page 2024

  23. [24]

    Simeonov, A

    A. Simeonov, A. Goyal, L. Manuelli, L. Yen-Chen, A. Sarmiento, A. Rodriguez, P. Agrawal, and D. Fox. Shelving, stacking, hanging: Relational pose diffusion for multi-modal rearrange- ment. Conference on Robot Learning, 2023

    work page 2023

  24. [25]

    Jiang, Y

    B. Jiang, Y . Wu, W. Zhou, C. Paxton, and D. Held. Hacman++: Spatially-grounded motion primitives for manipulation. 2024

    work page 2024

  25. [26]

    W. Zhou, B. Jiang, F. Yang, C. Paxton, and D. Held. Hacman: Learning hybrid actor-critic maps for 6d non-prehensile manipulation. 2023

    work page 2023

  26. [27]

    B. Fu, S. K. Leong, X. Lian, and X. Ji. 6d robotic assembly based on rgb-only object pose estimation, 2022. URL https://arxiv.org/abs/2208.12986

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  27. [28]

    Q. Yu, C. Hao, J. Wang, W. Liu, L. Liu, Y . Mu, Y . You, H. Yan, and C. Lu. Manipose: A comprehensive benchmark for pose-aware object manipulation in robotics, 2024. URLhttps: //arxiv.org/abs/2403.13365

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  28. [29]

    N. Ravi, V . Gabeur, Y .-T. Hu, R. Hu, C. Ryali, T. Ma, H. Khedr, R. R ¨adle, C. Rolland, L. Gustafson, E. Mintun, J. Pan, K. V . Alwala, N. Carion, C.-Y . Wu, R. Girshick, P. Doll ´ar, and C. Feichtenhofer. Sam 2: Segment anything in images and videos, 2024. URL https: //arxiv.org/abs/2408.00714

    Pith/arXiv arXiv 2024

  29. [30]

    S. Liu, Z. Zeng, T. Ren, F. Li, H. Zhang, J. Yang, Q. Jiang, C. Li, J. Yang, H. Su, J. Zhu, and L. Zhang. Grounding dino: Marrying dino with grounded pre-training for open-set object detection, 2024. URL https://arxiv.org/abs/2303.05499

    Pith/arXiv arXiv 2024

  30. [31]

    T. Ren, S. Liu, A. Zeng, J. Lin, K. Li, H. Cao, J. Chen, X. Huang, Y . Chen, F. Yan, Z. Zeng, H. Zhang, F. Li, J. Yang, H. Li, Q. Jiang, and L. Zhang. Grounded sam: Assembling open- world models for diverse visual tasks, 2024

    work page 2024

  31. [32]

    Sundermeyer, A

    M. Sundermeyer, A. Mousavian, R. Triebel, and D. Fox. Contact-graspnet: Efficient 6-dof grasp generation in cluttered scenes. 2021

    work page 2021

  32. [33]

    Y . Ze, G. Zhang, K. Zhang, C. Hu, M. Wang, and H. Xu. 3d diffusion policy: Generalizable visuomotor policy learning via simple 3d representations. InProceedings of Robotics: Science and Systems (RSS), 2024

    2024

  33. [34]

    Felner, M

    A. Felner, M. Goldenberg, G. Sharon, R. Stern, T. Beja, N. Sturtevant, J. Schaeffer, and R. Holte. Partial-expansion a* with selective node generation.Proceedings of the AAAI Confer- ence on Artificial Intelligence, 26(1):471–477, Sep. 2021. doi:10.1609/aaai.v26i1.8137. URL https://ojs.aaai.org/index.php/AAAI/article/view/8137

    work page doi:10.1609/aaai.v26i1.8137 2021

  34. [35]

    E. Cai, O. Donca, B. Eisner, and D. Held. Non-rigid relative placement through 3d dense diffusion. In 8th Annual Conference on Robot Learning, 2024. URL https://arxiv.org/ abs/2410.19247

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  35. [36]

    C. R. Qi, L. Yi, H. Su, and L. J. Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. arXiv preprint arXiv:1706.02413, 2017

    Pith/arXiv arXiv 2017

  36. [37]

    Karaev, I

    N. Karaev, I. Makarov, J. Wang, N. Neverova, A. Vedaldi, and C. Rupprecht. CoTracker3: Simpler and better point tracking by pseudo-labelling real videos. 2024

    work page 2024

  37. [38]

    query mask

    E. Coumans and Y . Bai. Pybullet, a python module for physics simulation for games, robotics and machine learning. http://pybullet.org, 2016–2023. 13 Appendix Table of Contents A Object Suggester 14 A.1 Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 A.2 Implementation Details . . . . . . . . . . . . . . . . . . . ...

    work page 2016

  38. [40]

    Plans found inside this graph are marked via green edges

    An expanded graph that marks all the nodes expanded during search. Plans found inside this graph are marked via green edges. Figures 11 through 13 show expanded graphs for three different table bussing initial configurations

    work page

  39. [41]

    Figures 14 through 16 show plan graphs for three different table bussing initial configura- tions

    A plan graph that visualizes different paths to goal configurations found during multi-goal A* search. Figures 14 through 16 show plan graphs for three different table bussing initial configura- tions. Figure 11: Expanded graph example 1. The graph represents the expanded nodes for a 2-step table bussing configuration where a bowl is on a plate and a cup ...

    work page

  40. [2113]

    URL https://doi.org/10.1613%2Fjair.2113

    work page

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.