Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

XPG-RL: Reinforcement Learning with Explainable Priority Guidance for Efficiency-Boosted Mechanical Search

T0 review · 3 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A reinforcement learning agent that switches among prioritized primitives—grasp, clear occlusion, next-best view—via two learned thresholds retrieves hidden objects with higher success and up to 4.5× fewer motions than hierarchical or…

desk verdict The priority-threshold idea is neat and the ablations are honest, but the headline 4.5x efficiency claim is undercut by a perception confound that the paper never controls for. read the letter →

arxiv 2504.20969 v2 pith:YMJ3FO2U submitted 2025-04-29 cs.RO cs.LG

Yiting Zhang , Shichen Li , Elena Shrestha This is my paper · ORCID
classification cs.ROcs.LG
keywords mechanicalsearchreinforcementlearningactionprioritizationnext-best-viewclutteredmanipulationPPOthresholdsocclusionremovalrobotgrasping
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that adding a fixed, explainable priority order over a robot's action primitives makes reinforcement learning for mechanical search—finding and grasping an occluded target object—substantially more successful and efficient than learning without that structure. The agent, XPG-RL, evaluates three primitives in priority order: grasp the target, remove an occluding object, or move the camera to a next-best view. A PPO-trained policy produces two thresholds that decide when to switch from a higher-priority primitive to a lower one, so only the switching boundaries are learned while the ordering is given. In simulated scenes with 5 to 20 objects, the paper reports that XPG-RL beats flat, hierarchical, and heuristic baselines in both success rate and average motions per task, reaching a relative efficiency of 6.9 in the 20-object case versus 1.5 for the best baseline—a 4.5× gain. Real-world trials with 9-object tabletop scenes, including one fully occluded target, show the same advantage. A sympathetic reader would care because it suggests that a small, human-interpretable inductive bias can rescue long-horizon RL in data-poor manipulation settings.

What carries the argument

The load-bearing mechanism is Algorithm 1, the priority-guided decision policy: a nested conditional that checks the target grasp score against learned threshold $\tau_1$, then the occlusion-removal score against $\tau_2$, and only then initiates a next-best-view camera move. The thresholds are the output of a PPO policy $\pi(\tau_t \mid o_t)$, so the agent learns when to switch primitives but cannot reorder them. The next-best-view routine constructs a Truncated Signed Distance Function (TSDF) of the scene, simulates candidate camera poses, scores each by the predicted grasp quality of the target, and moves to the best one; it fires only when both thresholds fail, which keeps viewpoint changes from being wasted on easy scenes.

What would settle it

Train XPG-RL on scenes where the target is fully invisible at the initial viewpoint and occluders are only removable from a side viewpoint, and compare it with a variant whose action order is learned or randomized: if the flexible-order variant matches or exceeds XPG-RL's success rate and motion efficiency, the fixed ordering is not the source of the reported gains. A cheaper check is to randomize the primitive order at each training episode; if performance does not degrade, the specific priority order is not load-bearing.

Watch

Extended reading notes

Core claim

The central discovery is that a fixed priority hierarchy over manipulation primitives—grasp the target, then remove an occluder, then adjust the viewpoint—combined with a learned threshold policy for advancing down that hierarchy, outperforms flat reinforcement learning, hierarchical reinforcement learning, and heuristic-driven high-level selection in mechanical search. Concretely, the policy maps a two-channel observation (a target mask produced by SAM and PoseCNN, and an object-level depth map from the proposed Object Depth Mapping module) to thresholds $\tau_1$ and $\tau_2$ in $[0,1]$; if the predicted success probability of grasping the target exceeds $\tau_1$, the agent grasps, otherwise it checks whether removing an occluder is predicted to succeed above $\tau_2$, and otherwise it executes an next-best-view camera move. The reported simulation success rates fall from 81% at 5 objects to 64% at 20 objects, while the strongest baseline (MPGNet) drops from 76% to 22%, and XPG-RL's average motion number stays at or below 4.94 even in the most cluttered setting. The paper takes this as evidence that the priority guidance reduces redundant or misaligned actions without sacrificing adaptability.

Load-bearing premise

The fixed priority order (grasp target, then remove occlusion, then adjust viewpoint) is hard-coded, so the learned policy can only tune the thresholds that decide when to move down the hierarchy; if optimal behavior in some states requires moving the camera before removing an occluder, the policy class cannot express that.

Editorial extensions

If this is right

  • Deployed in warehouses or homes, a robot using priority-guided thresholds would retrieve occluded items with fewer pokes and camera sweeps, cutting cycle time and reducing disturbance of surrounding objects.
  • Because the high-level decision is just two scalar thresholds, the robot's behavior is auditable: an operator can see why a grasp was attempted or deferred, which helps in safety-critical or human-shared spaces.
  • The method's efficiency advantage grows with clutter (relative efficiency 5.0 at 10 objects, 6.9 at 20 objects), suggesting the fixed priority order is most valuable exactly where flat policies collapse.
  • The separation of perception (target mask plus object-level depth) from decision (thresholds) implies the framework could be ported to a different gripper or sensor suite by adapting the perception module while keeping the priority structure and PPO policy unchanged.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension is to replace the three hand-picked primitives with domain-specific ones for other long-horizon manipulation tasks (e.g., assembly, sorting, or search-and-rescue), reusing Algorithm 1 unchanged and learning only the thresholds.
  • The paper's results suggest a testable hypothesis about inductive bias: in data-limited manipulation, imposing the correct action hierarchy transfers more value than increasing network capacity or exploration, a claim one could verify by measuring the performance gap as training data shrinks.
  • The Object Depth Mapping representation discards per-pixel depth within objects; an ablation that feeds raw depth to the same policy would isolate whether the simplification itself contributes to learning efficiency, beyond the priority guidance.
  • The next-best-view criterion scores candidate views only by predicted target grasp quality; scoring views by expected improvement in both target and occluder visibility might yield better viewpoint choices in scenes where the target is deeply buried.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes XPG-RL, a reinforcement learning method for mechanical search that couples a fixed priority ordering over action primitives (grasp target, remove occluder, adjust viewpoint) with a PPO policy that outputs two thresholds governing when to switch among these primitives. Perception is built from SAM, PoseCNN, an object-level depth mapping module, and AnyGrasp grasp-quality scores. The method is evaluated in Isaac Lab simulations and on a real Kinova arm across four tabletop scenes, reporting higher task success rates and lower average motion numbers than three baselines, with ablations for the priority guidance and the next-best-view strategy.

Significance. If the empirical claims hold, the paper offers a practical and modular way to inject domain knowledge into RL-based mechanical search, and the explicit priority structure gives a degree of interpretability that is rare in this area. The strengths are the clear Algorithm 1, the two-component ablation, the inclusion of both simulation and real-world trials, and the short-horizon efficiency table. However, the significance depends on the comparison being fair: the central contribution is an engineered decision module, so the evidence must rule out the pretrained perception and scoring pipeline as the source of the reported advantage. That isolation is currently missing, and the absence of variance reporting makes it difficult to assess how robust the improvements are.

major comments (3)
  1. [III-A, III-B; Tables I and II] The claimed efficiency advantage is not cleanly attributable to the proposed decision policy because XPG-RL's action scores come from AnyGrasp (§III-B) and its observations depend on SAM, PoseCNN, and ODM (§III-A), whereas the baselines (Target-Oriented VPG, Hierarchical Policy Learning, MPGNet; §IV) use their own learned Q-functions and do not receive these pretrained modules. The 20-object 'No Priority Guidance' ablation still achieves 42% success versus MPGNet's 22% (Table II), suggesting that a large part of the gap may come from the perception/scoring pipeline rather than from the priority scheme, but the paper does not state whether that ablation uses AnyGrasp scores. To support the headline 'up to 4.5x' claim, the authors should either run the baselines with the same perception module or remove AnyGrasp from XPG-RL in an ablation, so that the comparison isolates the decision-making component.
  2. [Tables I-IV, Fig. 4; IV-A, IV-B] All simulation results are reported as point estimates without standard deviations, confidence intervals, or the number of seeds. For stochastic RL training and task success metrics, differences such as 81% versus 76% at 5 objects may lie within run-to-run noise, and the phrase 'consistently outperforms' is not supported by a single run per condition. The real-world results are based on only 5 repetitions per scene (Table III), so differences like 4/5 versus 3/5 are not statistically meaningful. Please report means and standard deviations over at least 5 seeds, give per-trial data for the real-robot experiments, and temper the consistency claims accordingly.
  3. [Algorithm 1; III-B] The fixed ordering (grasp target, then remove occluder, then adjust viewpoint) means the learned policy can only tune two thresholds and cannot reorder or skip primitives. If moving the camera to a better viewpoint is the most efficient first action when the target is occluded but an occluder has a high grasp score, XPG-RL cannot express that behavior because it will always remove the occluder first. This expressivity limitation is never discussed or ablated, despite being central to the 'context-aware switching' claim. The authors should either justify the fixed order with additional experiments (for example, an alternative priority order or an action space in which NBV is available at every priority level) or explicitly state this limitation as a boundary of the method.
minor comments (6)
  1. [III-A and III-B] There are small textual errors: 'contrained' in Section III, and 'AnyGrasp [6]. representing' has a misplaced period. Please proofread the method section.
  2. [IV, Metrics] The definition of 'average motion number' should clarify whether it is averaged over all episodes or only successful episodes, and what action count is recorded for failed episodes. This matters because the efficiency metric divides success rate by this average.
  3. [Fig. 4 and Table I] The relative efficiency values in Figure 4 require the reader to reconstruct the calculation from Table I. Please state explicitly that the reference is Target-Oriented VPG, show the formula, and include uncertainty estimates on the relative efficiency.
  4. [V and Table III] The conclusion states that XPG-RL 'completed all five trials' in fully occluded scenes, but Table III shows 4/5 in Scenes 1 and 4; the sentence should refer only to Scene 3 or be rephrased to avoid overstating the real-world results.
  5. [Appendix A] The training description says '10,000 steps' for PPO; please specify whether these are environment steps or gradient steps, include the random seed policy, and report the reward scale details with the exact discount factor used in the reported runs.
  6. [Table IV] The short-horizon table compares only MPGNet and XPG-RL; adding Hierarchical Policy Learning and Target-Oriented VPG would make the short-horizon evidence consistent with the rest of the paper.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: learned thresholds and external baseline comparisons are not equivalent to the paper's inputs.

full rationale

XPG-RL's central claim is an empirical efficiency advantage over external baselines. The thresholds tau1,t and tau2,t are outputs of a PPO policy trained on the reward in Eq. (1) and evaluated on held-out simulated and real scenes; they are not fitted values renamed as predictions. The priority ordering in Algorithm 1 is an explicit design choice, and the reward function is a normal RL objective, so aligning test success rate and motion number with that objective is expected training behavior, not a circular derivation. The reported comparisons are against Target-Oriented VPG, Hierarchical Policy Learning, and MPGNet, which are external methods; no equation in the paper reduces another by construction. The self-citations ([2], [17], [19], [38]) appear in related-work or method-inspiration contexts and are not load-bearing for the claimed result: the task formulation is attributed to [36], NBV inspiration to [37], and both are external. There is no imported uniqueness theorem and no definition of the method's inputs in terms of its outputs. The perception-suite mismatch noted by the skeptic is a fairness or confounding concern about attribution of the performance gap; it is a correctness or validity issue, not circularity. Overall, the derivation chain is self-contained: learned decision thresholds plus external benchmark evaluation.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The central claim rests on hand-chosen reward weights, a fixed priority ordering, and the reliability of pretrained perception and grasping modules (SAM, PoseCNN, AnyGrasp). No new physical entities or forces are introduced. The action thresholds are learned by the RL policy and are not manually fitted to the benchmark numbers.

free parameters (1)
  • Reward coefficients (1000, -100, -1)
    Hand-chosen weights in the reward function (Eq. 1) that define the trade-off between success, infeasible actions, and per-step cost. These shape the learned policy and are not derived from data.
assumptions (3)
  • domain assumption The fixed priority hierarchy (grasp target, then remove occlusion, then adjust viewpoint) is a suitable structural prior for mechanical search.
    Imposed in Algorithm 1 and Section III-B; the policy can only adjust thresholds, not reorder or skip the primitives, so the hierarchy is assumed to be beneficial across all states.
  • domain assumption AnyGrasp quality scores for the target and occluding objects are informative enough to serve as decision variables for threshold-based action selection.
    Used in Algorithm 1 to compare against learned thresholds; if the scores were unreliable or poorly calibrated, the threshold policy would have no useful signal to act on.
  • domain assumption The simulation environment (Isaac Lab with YCB objects) and the perception pipeline transfer to the real world without domain adaptation or fine-tuning.
    The policy is trained in simulation and deployed on the real robot, as described in Section IV, so sim-to-real fidelity of both dynamics and perception is assumed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of XPG-RL: Reinforcement Learning with Explainable Priority Guidance for Efficiency-Boosted Mechanical Search." pith.science (2026). https://pith.science/paper/YMJ3FO2U

@misc{pith2026250420969,
  author       = {Pith},
  title        = {Pith review of: XPG-RL: Reinforcement Learning with Explainable Priority Guidance for Efficiency-Boosted Mechanical Search},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YMJ3FO2U}},
  note         = {Machine review of arXiv:2504.20969}
}
abstract

Mechanical search (MS) in cluttered environments remains a significant challenge for autonomous manipulators, requiring long-horizon planning and robust state estimation under occlusions and partial observability. In this work, we introduce XPG-RL, a reinforcement learning framework that enables agents to efficiently perform MS tasks through explainable, priority-guided decision-making based on raw sensory inputs. XPG-RL integrates a task-driven action prioritization mechanism with a learned context-aware switching strategy that dynamically selects from a discrete set of action primitives such as target grasping, occlusion removal, and viewpoint adjustment. Within this strategy, a policy is optimized to output adaptive threshold values that govern the discrete selection among action primitives. The perception module fuses RGB-D inputs with semantic and geometric features to produce a structured scene representation for downstream decision-making. Extensive experiments in both simulation and real-world settings demonstrate that XPG-RL consistently outperforms baseline methods in task success rates and motion efficiency, achieving up to 4.5$\times$ higher efficiency in long-horizon tasks. These results underscore the benefits of integrating domain knowledge with learnable decision-making policies for robust and efficient robotic manipulation. The project page for XPG-RL is https://yitingzhang1997.github.io/xpgrl/.

Figures

Figures reproduced from arXiv: 2504.20969 by the authors.

Figure 1
Figure 1. We propose XPG-RL, a reinforcement learning framework for mechanical search tasks. XPG-RL leverages [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of the XPG-RL framework. The framework consists of two main components: (1) a perception pipeline, which processes fused RGB-D images to extract semantic and geometric context and builds a compact scene representation; and (2) an RL-based decision-making module, which takes this representation as input and learns a policy to predict adaptive thresholds. These thresholds guide the selection among priority-st… view at source ↗
Figure 3
Figure 3. Priority-guided action candidates. The action space consists of three discrete primitives—target grasping, occlusion removal, and viewpoint adjustment—ranked from highest to lowest priority. Learned thresholds govern switching between these actions, enabling the agent to make efficient, interpretable decisions by sequentially evaluating actions in priority order. grasp with the highest quality score on the target ob… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Efficiency scaling with complexity. XPG-RL outperforms baselines across all clutter levels, with relative efficiency gains widening as object count increases. Across all methods, we observe a general trend: the task suc￾cess rate decreases and the average motion number…
Figure 5
Figure 5. Figure 5: Real-world experiment scenes. Four tabletop setups with 9 objects each, illustrating different occlusion conditions, including a fully occluded target in Scene 3. target further. XPG-RL avoids such failures through priority￾guided decision-making that defers risky acti…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Certifiably Safe Manipulation of Deformable Linear Objects via Joint Shape and Tension Prediction

    cs.RO 2025-05 reject novelty 5.0 of 10

    A learned LSTM that predicts cable shape and tension is combined with a zonotope-based trajectory optimizer to enforce collision and tension safety in simulated wire harness assembly.

Reference graph

Works this paper leans on

40 extracted references · 30 canonical work pages · cited by 1 Pith paper

  1. [1]

    Bench- marking in manipulation research: The ycb object and model set and benchmarking protocols

    Berk Calli, Aaron Walsman, Arjun Singh, Siddhartha Srinivasa, Pieter Abbeel, and Aaron M Dollar. Bench- marking in manipulation research: The ycb object and model set and benchmarking protocols. arXiv preprint arXiv:1502.03143, 2015

    arXiv 2015

  2. [2]

    Differentiable discrete elastic rods for real-time mod- eling of deformable linear objects

    Yizhou Chen, Yiting Zhang, Zachary Brei, Tiancheng Zhang, Yuzhen Chen, Julie Wu, and Ram Vasudevan. Differentiable discrete elastic rods for real-time mod- eling of deformable linear objects. arXiv preprint arXiv:2406.05931, 2024

    arXiv 2024

  3. [3]

    Mechanical search: Multi-step retrieval of a target object occluded by clutter

    Michael Danielczuk, Andrey Kurenkov, Ashwin Bal- akrishna, Matthew Matl, David Wang, Roberto Mart ´ın- Mart´ın, Animesh Garg, Silvio Savarese, and Ken Gold- berg. Mechanical search: Multi-step retrieval of a target object occluded by clutter. In 2019 International Confer- ence on Robotics and Automation (ICRA) , pages 1614–

    work page 2019

  4. [4]

    The cluttered environment picking benchmark (cepb) for advanced warehouse automation: evaluating the perception, planning, control, and grasping of manipulation systems

    Salvatore D’Avella, Matteo Bianchi, Ashok M Sun- daram, Carlo Alberto Avizzano, M ´aximo A Roa, and Paolo Tripicchio. The cluttered environment picking benchmark (cepb) for advanced warehouse automation: evaluating the perception, planning, control, and grasping of manipulation systems. IEEE Robotics & Automation Magazine, 2023

    work page 2023

  5. [5]

    A planning framework for non-prehensile manipulation under clutter and uncertainty

    Mehmet R Dogar and Siddhartha S Srinivasa. A planning framework for non-prehensile manipulation under clutter and uncertainty. Autonomous Robots, 33:217–236, 2012

    work page 2012

  6. [6]

    Anygrasp: Robust and efficient grasp perception in spatial and temporal domains

    Hao-Shu Fang, Chenxi Wang, Hongjie Fang, Minghao Gou, Jirong Liu, Hengxu Yan, Wenhai Liu, Yichen Xie, and Cewu Lu. Anygrasp: Robust and efficient grasp perception in spatial and temporal domains. IEEE Transactions on Robotics , 2023

    2023

  7. [7]

    Prehensile and non-prehensile robotic pick-and- place of objects in clutter using deep reinforcement learning

    Muhammad Babar Imtiaz, Yuansong Qiao, and Brian Lee. Prehensile and non-prehensile robotic pick-and- place of objects in clutter using deep reinforcement learning. Sensors, 23(3):1513, 2023

    work page 2023

  8. [8]

    Hierar- chical task and motion planning in the now

    Leslie Pack Kaelbling and Tom ´as Lozano-P´erez. Hierar- chical task and motion planning in the now. In2011 IEEE International Conference on Robotics and Automation , pages 1470–1477. IEEE, 2011

    2011

Show all 40 references
  1. [9]

    Segment anything

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 4015–4026, 2023

    2023

  2. [10]

    A review of robot learning for manipulation: Challenges, representations, and algorithms

    Oliver Kroemer, Scott Niekum, and George Konidaris. A review of robot learning for manipulation: Challenges, representations, and algorithms. Journal of machine learning research, 22(30):1–82, 2021

    2021

  3. [11]

    Learning multi-step robotic manipulation policies from visual ob- servation of scene and q-value predictions of previous action

    Sulabh Kumra, Shirin Joshi, and Ferat Sahin. Learning multi-step robotic manipulation policies from visual ob- servation of scene and q-value predictions of previous action. In 2022 International Conference on Robotics and Automation (ICRA) , pages 8245–8251. IEEE, 2022

    2022

  4. [12]

    Visuomotor mechanical search: Learning to retrieve target objects in clutter

    Andrey Kurenkov, Joseph Taglic, Rohun Kulkarni, Mar- cus Dominguez-Kuhne, Animesh Garg, Roberto Mart ´ın- Mart´ın, and Silvio Savarese. Visuomotor mechanical search: Learning to retrieve target objects in clutter. In 2020 IEEE/RSJ International Conference on Intelligent Robots...

    2020

  5. [13]

    Semantic and geometric modeling with neural message passing in 3d scene graphs for hierarchical mechanical search

    Andrey Kurenkov, Roberto Mart ´ın-Mart´ın, Jeff Ich- nowski, Ken Goldberg, and Silvio Savarese. Semantic and geometric modeling with neural message passing in 3d scene graphs for hierarchical mechanical search. In 2021 IEEE International Conference on Robotics and Automation (...

    2021

  6. [14]

    Hierarchical primi- tive composition: Simultaneous activation of skills with inconsistent action dimensions in multiple hierarchies

    Jeong-Hoon Lee and Jongeun Choi. Hierarchical primi- tive composition: Simultaneous activation of skills with inconsistent action dimensions in multiple hierarchies. IEEE Robotics and Automation Letters , 7(3):7581–7588, 2022

    2022

  7. [15]

    Mpgnet: Learning move-push- grasping synergy for target-oriented grasping in occluded scenes

    Dayou Li, Chenkun Zhao, Shuo Yang, Ran Song, Xiaolei Li, and Wei Zhang. Mpgnet: Learning move-push- grasping synergy for target-oriented grasping in occluded scenes. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5064–5071. IEEE, 2024

    2024

  8. [16]

    Towards practical multi-object manipulation using relational reinforcement learning

    Richard Li, Allan Jabri, Trevor Darrell, and Pulkit Agrawal. Towards practical multi-object manipulation using relational reinforcement learning. In 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 4051–4058. IEEE, 2020

    2020

  9. [17]

    Multi-modal fusion of in-situ video data and process parameters for online forecasting of cookie drying readiness

    Shichen Li and Chenhui Shao. Multi-modal fusion of in-situ video data and process parameters for online forecasting of cookie drying readiness. arXiv preprint arXiv:2504.15599, 2025

    2025 arXiv

  10. [18]

    Synergistic task and motion planning with rein- forcement learning-based non-prehensile actions

    Gaoyuan Liu, Joris De Winter, Denis Steckelmacher, Roshan Kumar Hota, Ann Nowe, and Bram Vander- borght. Synergistic task and motion planning with rein- forcement learning-based non-prehensile actions. IEEE Robotics and Automation Letters, 8(5):2764–2771, 2023

    2023

  11. [19]

    Meta-learning-based domain generalization for cost-effective tool condition monitor- ing in ultrasonic metal welding

    Yuquan Meng, Zhiqiao Dong, Kuan-Chieh Lu, Shichen Li, and Chenhui Shao. Meta-learning-based domain generalization for cost-effective tool condition monitor- ing in ultrasonic metal welding. IEEE Transactions on Industrial Informatics, 2024

    2024

  12. [20]

    Orbit: A unified simulation framework for interactive robot learning environments

    Mayank Mittal, Calvin Yu, Qinxi Yu, Jingzhou Liu, Nikita Rudin, David Hoeller, Jia Lin Yuan, Ritvik Singh, Yunrong Guo, Hammad Mazhar, Ajay Mandlekar, Buck Babich, Gavriel State, Marco Hutter, and Animesh Garg. Orbit: A unified simulation framework for interactive robot learni...

    2023

  13. [21]

    Fast and resilient manipulation planning for object retrieval in cluttered and confined environments

    Changjoo Nam, Sang Hun Cheong, Jinhwi Lee, Dong Hwan Kim, and ChangHwan Kim. Fast and resilient manipulation planning for object retrieval in cluttered and confined environments. IEEE Transactions on Robotics, 37(5):1539–1552, 2021

    2021

  14. [22]

    Aug- menting reinforcement learning with behavior primitives for diverse manipulation tasks

    Soroush Nasiriany, Huihan Liu, and Yuke Zhu. Aug- menting reinforcement learning with behavior primitives for diverse manipulation tasks. In 2022 International Conference on Robotics and Automation (ICRA) , pages 7477–7484. IEEE, 2022

    2022

  15. [23]

    Stable-baselines3: Reliable reinforcement learning imple- mentations

    Antonin Raffin, Ashley Hill, Adam Gleave, Anssi Kanervisto, Maximilian Ernestus, and Noah Dormann. Stable-baselines3: Reliable reinforcement learning imple- mentations. Journal of Machine Learning Research , 22 (268):1–8, 2021

    2021

  16. [24]

    Task priority grasping and locomotion control of modular robot

    Gionata Salvietti, HX Zhang, J Gonzalez-Gomez, Domenico Prattichizzo, and JW Zhang. Task priority grasping and locomotion control of modular robot. In 2009 IEEE International Conference on Robotics and Biomimetics (ROBIO), pages 1069–1074. IEEE, 2009

    2009

  17. [25]

    Exploring the limits of hierarchical world models in reinforcement learning

    Robin Schiewer, Anand Subramoney, and Laurenz Wiskott. Exploring the limits of hierarchical world models in reinforcement learning. Scientific Reports, 14 (1):26856, 2024

    2024

  18. [26]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 , 2017

    2017 arXiv

  19. [27]

    Learn- ing to combine primitive skills: A step towards versa- tile robotic manipulation

    Robin Strudel, Alexander Pashevich, Igor Kalevatykh, Ivan Laptev, Josef Sivic, and Cordelia Schmid. Learn- ing to combine primitive skills: A step towards versa- tile robotic manipulation. In 2020 IEEE International Conference on Robotics and Automation (ICRA) , pages 4637–464...

    2020

  20. [28]

    Selective object rearrangement in clutter

    Bingjie Tang and Gaurav S Sukhatme. Selective object rearrangement in clutter. In Conference on Robot Learn- ing, pages 1001–1010. PMLR, 2023

    2023

  21. [29]

    Multi-stage reinforcement learn- ing for non-prehensile manipulation

    Dexin Wang, Chunsheng Liu, Faliang Chang, Hengqiang Huan, and Kun Cheng. Multi-stage reinforcement learn- ing for non-prehensile manipulation. IEEE Robotics and Automation Letters, 2024

    2024

  22. [30]

    Hierarchical visual policy learning for long-horizon robot manipulation in densely cluttered scenes

    Hecheng Wang, Lizhe Qi, Bin Fang, and Yunquan Sun. Hierarchical visual policy learning for long-horizon robot manipulation in densely cluttered scenes. arXiv preprint arXiv:2312.02697, 2023

    2023 arXiv

  23. [31]

    I2hrl: interactive influence-based hierarchical reinforcement learning

    Rundong Wang, Runsheng Yu, Bo An, and Zinovi Ra- binovich. I2hrl: interactive influence-based hierarchical reinforcement learning. In Proceedings of the Twenty- Ninth International Conference on International Joint Conferences on Artificial Intelligence, pages 3131–3138, 2021

    2021

  24. [32]

    Posecnn: A convolutional neural network for 6d object pose estimation in cluttered scenes

    Yu Xiang, Tanner Schmidt, Venkatraman Narayanan, and Dieter Fox. Posecnn: A convolutional neural network for 6d object pose estimation in cluttered scenes. In Robotics: Science and Systems , 2018

    2018

  25. [33]

    Hierarchical reinforcement learning with universal policies for multi- step robotic manipulation

    Xintong Yang, Ze Ji, Jing Wu, Yu-Kun Lai, Changyun Wei, Guoliang Liu, and Rossitza Setchi. Hierarchical reinforcement learning with universal policies for multi- step robotic manipulation. IEEE Transactions on Neural Networks and Learning Systems, 33(9):4727–4741, 2021

    2021

  26. [34]

    Learn- ing synergies between pushing and grasping with self- supervised deep reinforcement learning

    Andy Zeng, Shuran Song, Stefan Welker, Johnny Lee, Alberto Rodriguez, and Thomas Funkhouser. Learn- ing synergies between pushing and grasping with self- supervised deep reinforcement learning. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) ...

    2018

  27. [35]

    Transporter networks: Rearranging the visual world for robotic manipulation

    Andy Zeng, Pete Florence, Jonathan Tompson, Stefan Welker, Jonathan Chien, Maria Attarian, Travis Arm- strong, Ivan Krasin, Dan Duong, Vikas Sindhwani, et al. Transporter networks: Rearranging the visual world for robotic manipulation. In Conference on Robot Learning , pages 7...

    2021

  28. [36]

    Hierarchical policy learning for mechanical search

    Oussama Zenkri, Ngo Anh Vien, and Gerhard Neumann. Hierarchical policy learning for mechanical search. In 2022 International Conference on Robotics and Automa- tion (ICRA), pages 1954–1960. IEEE, 2022

    2022

  29. [37]

    Affordance-driven next-best-view planning for robotic grasping

    Xuechao Zhang, Dong Wang, Sun Han, Weichuang Li, Bin Zhao, Zhigang Wang, Xiaoming Duan, Chongrong Fang, Xuelong Li, and Jianping He. Affordance-driven next-best-view planning for robotic grasping. arXiv preprint arXiv:2309.09556, 2023

    2023 arXiv

  30. [38]

    Certifiably safe manipu- lation of deformable linear objects via joint shape and tension prediction

    Yiting Zhang and Shichen Li. Certifiably safe manipu- lation of deformable linear objects via joint shape and tension prediction. arXiv preprint arXiv:2505.13889 , 2025. APPENDIX A. Training Details

    2025 arXiv

  31. [39]

    The policy is optimized with a learning rate of 3×10−4, a discount factor (γ) of 0.99, and a generalized advantage estimation (GAE) parameter (λ) of 0.95

    Policy Training Hyperparameters: The RL agent is trained for a total of 10,000 steps using the PPO algorithm. The policy is optimized with a learning rate of 3×10−4, a discount factor (γ) of 0.99, and a generalized advantage estimation (GAE) parameter (λ) of 0.95. The training...

  32. [40]

    These masked images are then processed by PoseCNN [32], which utilizes a 13-layer VGG16-style convolutional backbone for feature extraction

    Network Architectures: The object recognition compo- nent of the perception pipeline begins with object detection using the Segment Anything Model (SAM) [9], which gener- ates instance-level masks for each object in the scene. These masked images are then processed by PoseCNN ...

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.