{"id":"8b54a275-d1e2-42a4-b313-5391c4ca9cfc","arxiv_id":"2504.20969","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"XPG-RL learns adaptive thresholds for switching between grasping, occlusion removal, and viewpoint adjustment, improving mechanical search efficiency by up to 4.5x over baselines.","lead":"This paper introduces XPG-RL, a reinforcement learning method that helps robots find and grasp occluded objects in cluttered scenes. It combines a fixed action priority list with learned decision thresholds, reporting higher success rates and fewer motions than prior baselines in simulation and real-world tests.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Empirical comparison confounded by perception module: XPG-RL uses AnyGrasp/SAM/PoseCNN while baselines do not, so the claimed 4.5x efficiency gain may not be attributable to the proposed priority-guided RL.","rationale":"The reader's weakest_assumption focused on the fixed priority ordering (Algorithm 1) preventing camera-before-occluder-removal policies. That is a legitimate expressivity concern, but it does not directly threaten the paper's central empirical claim: even a suboptimal fixed ordering could still outperform baselines in practice. The more load-bearing issue is that the experimental comparison changes multiple variables at once. XPG-RL benefits from pretrained SAM, PoseCNN, and AnyGrasp, while the baselines use end-to-end learned Q-functions. Table II's 'No Priority Guidance' variant, which still uses the new perception pipeline, achieves 42% success at 20 objects versus MPGNet's 22%, indicating a large perception-driven advantage regardless of the priority mechanism. If the ablation uses AnyGrasp scores, then the flat policy benefits from the same scores; if it does not, the ablation changes two variables (priority scheme and scoring) and cannot isolate priority guidance. In either case, the paper lacks a comparison that controls for the perception/scoring component. The claimed 4.5x efficiency boost in long-horizon tasks (Section IV-A, Fig. 4) is therefore not cleanly attributable to the XPG-RL decision module. We recommend keeping the conditional decision: acceptance should require an updated comparison with perception-matched baselines, plus reporting of seed variance and code/data release as the reader already requested. No evidence of fabrication or internal inconsistency was found.","tokens_in":10414,"tokens_out":6842,"duration_ms":67247,"concrete_test":"Re-run the simulation experiments in Table I with baselines upgraded to use the same perception and grasp-scoring pipeline as XPG-RL (SAM masks, PoseCNN target mask, ODM depth map, and AnyGrasp Qtarget/Qocclude), keeping their high-level action-selection logic unchanged. Compare success rates and average motion numbers at 5/10/15/20 objects. If the success-rate gap at 20 objects drops from 42 percentage points (64 vs 22) to roughly the 22-point gap seen in the No-Priority-Guidance ablation (64 vs 42), the headline 4.5x claim is substantially due to perception, not the proposed decision mechanism.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (abstract, §IV) is that XPG-RL's priority-guided switching and NBV decisions yield 'up to 4.5× higher efficiency' than baselines. The evidence for this is weakened by a missing control: XPG-RL's action selection relies on grasp-quality scores computed by AnyGrasp (§III-B), and its observation inputs depend on SAM and PoseCNN (§III-A). The baselines (Target-Oriented VPG, Hierarchical Policy Learning, MPGNet; §IV, Table I) use their own learned Q-functions and do not have access to these pretrained perception/scoring modules. The ablation 'No Priority Guidance' (Table II) removes only the priority scheme, yet at 20 objects it reaches 42% success compared to MPGNet's 22%, suggesting that a large fraction of the gap is due to the perception/scoring pipeline rather than the proposed decision algorithm. Since the paper does not report whether the flat-policy ablation uses AnyGrasp scores, the effect is not fully isolated. Without perception-matched baselines, the headline efficiency comparison cannot be attributed to XPG-RL's algorithmic contributions. The fixed priority ordering (Algorithm 1) is an expressivity limitation, but it is secondary to this confounding.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes XPG-RL, a reinforcement learning method for mechanical search that couples a fixed priority ordering over action primitives (grasp target, remove occluder, adjust viewpoint) with a PPO policy that outputs two thresholds governing when to switch among these primitives. Perception is built from SAM, PoseCNN, an object-level depth mapping module, and AnyGrasp grasp-quality scores. The method is evaluated in Isaac Lab simulations and on a real Kinova arm across four tabletop scenes, reporting higher task success rates and lower average motion numbers than three baselines, with ablations for the priority guidance and the next-best-view strategy.","tokens_in":10592,"tokens_out":5825,"duration_ms":62969,"significance":"If the empirical claims hold, the paper offers a practical and modular way to inject domain knowledge into RL-based mechanical search, and the explicit priority structure gives a degree of interpretability that is rare in this area. The strengths are the clear Algorithm 1, the two-component ablation, the inclusion of both simulation and real-world trials, and the short-horizon efficiency table. However, the significance depends on the comparison being fair: the central contribution is an engineered decision module, so the evidence must rule out the pretrained perception and scoring pipeline as the source of the reported advantage. That isolation is currently missing, and the absence of variance reporting makes it difficult to assess how robust the improvements are.","major_comments":[{"comment":"The claimed efficiency advantage is not cleanly attributable to the proposed decision policy because XPG-RL's action scores come from AnyGrasp (§III-B) and its observations depend on SAM, PoseCNN, and ODM (§III-A), whereas the baselines (Target-Oriented VPG, Hierarchical Policy Learning, MPGNet; §IV) use their own learned Q-functions and do not receive these pretrained modules. The 20-object 'No Priority Guidance' ablation still achieves 42% success versus MPGNet's 22% (Table II), suggesting that a large part of the gap may come from the perception/scoring pipeline rather than from the priority scheme, but the paper does not state whether that ablation uses AnyGrasp scores. To support the headline 'up to 4.5x' claim, the authors should either run the baselines with the same perception module or remove AnyGrasp from XPG-RL in an ablation, so that the comparison isolates the decision-making component.","section":"III-A, III-B; Tables I and II"},{"comment":"All simulation results are reported as point estimates without standard deviations, confidence intervals, or the number of seeds. For stochastic RL training and task success metrics, differences such as 81% versus 76% at 5 objects may lie within run-to-run noise, and the phrase 'consistently outperforms' is not supported by a single run per condition. The real-world results are based on only 5 repetitions per scene (Table III), so differences like 4/5 versus 3/5 are not statistically meaningful. Please report means and standard deviations over at least 5 seeds, give per-trial data for the real-robot experiments, and temper the consistency claims accordingly.","section":"Tables I-IV, Fig. 4; IV-A, IV-B"},{"comment":"The fixed ordering (grasp target, then remove occluder, then adjust viewpoint) means the learned policy can only tune two thresholds and cannot reorder or skip primitives. If moving the camera to a better viewpoint is the most efficient first action when the target is occluded but an occluder has a high grasp score, XPG-RL cannot express that behavior because it will always remove the occluder first. This expressivity limitation is never discussed or ablated, despite being central to the 'context-aware switching' claim. The authors should either justify the fixed order with additional experiments (for example, an alternative priority order or an action space in which NBV is available at every priority level) or explicitly state this limitation as a boundary of the method.","section":"Algorithm 1; III-B"}],"minor_comments":[{"comment":"There are small textual errors: 'contrained' in Section III, and 'AnyGrasp [6]. representing' has a misplaced period. Please proofread the method section.","section":"III-A and III-B"},{"comment":"The definition of 'average motion number' should clarify whether it is averaged over all episodes or only successful episodes, and what action count is recorded for failed episodes. This matters because the efficiency metric divides success rate by this average.","section":"IV, Metrics"},{"comment":"The relative efficiency values in Figure 4 require the reader to reconstruct the calculation from Table I. Please state explicitly that the reference is Target-Oriented VPG, show the formula, and include uncertainty estimates on the relative efficiency.","section":"Fig. 4 and Table I"},{"comment":"The conclusion states that XPG-RL 'completed all five trials' in fully occluded scenes, but Table III shows 4/5 in Scenes 1 and 4; the sentence should refer only to Scene 3 or be rephrased to avoid overstating the real-world results.","section":"V and Table III"},{"comment":"The training description says '10,000 steps' for PPO; please specify whether these are environment steps or gradient steps, include the random seed policy, and report the reward scale details with the exact discount factor used in the reported runs.","section":"Appendix A"},{"comment":"The short-horizon table compares only MPGNet and XPG-RL; adding Hierarchical Policy Learning and Target-Oriented VPG would make the short-horizon evidence consistent with the rest of the paper.","section":"Table IV"}],"recommendation":"major_revision","confidential_remarks":"The core concern is experimental isolation rather than a theoretical error. With perception-matched baselines or a clean ablation, proper seed statistics, and a discussion of the fixed priority ordering, the paper could become acceptable. I see no citation or ethics concerns beyond the missing comparison details noted in the report."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the priority-threshold mechanism is a genuinely sensible idea, and the ablation study is the right kind of analysis, but the experimental comparison is not clean enough to support the headline efficiency claim. This paper deserves a rigorous review, but the authors need to fix the baseline comparison and report variance.\n\nWhat's new: instead of learning a flat high-level policy or relying on fixed heuristics, the agent learns two thresholds that decide when to grasp the target, remove an occluder, or move the camera, given AnyGrasp quality scores. That combination is not in the prior work I've seen. The Object Depth Mapping (averaging depth per segmented object and shading by distance) is a simple, effective way to compress RGB-D into something the policy can actually use. The NBV is triggered only when both grasp scores are low, which makes sense and seems to help in fully occluded scenes. The ablation study removing priority guidance and removing NBV is exactly the right kind of control; it shows both components matter.\n\nSoft spots: the biggest one is the perception confound. XPG-RL uses AnyGrasp to compute Q_target and Q_occlude, plus SAM and PoseCNN for segmentation. The baselines—Target-Oriented VPG, Hierarchical Policy Learning, MPGNet—learn their own Q-functions and do not get AnyGrasp scores. So when XPG-RL beats MPGNet by 64% to 22% at 20 objects, a large chunk of that gap could come from the stronger grasp-quality estimator rather than the priority-switching policy. The 'No Priority Guidance' ablation still uses the same perception stack and hits 42%, nearly double MPGNet, which strongly suggests the perception pipeline alone explains a big part of the advantage. The paper never reports a perception-matched baseline (e.g., a flat policy with the same AnyGrasp scores, or baselines augmented with the same perception), so the claimed efficiency gain cannot be cleanly attributed to the algorithm.\n\nSecond, there are no standard deviations or number of seeds anywhere in the simulation results, and the real-world trials use only 5 repetitions per scene. That makes it hard to judge whether the differences are meaningful. No code or data release is mentioned, which further limits reproducibility. The fixed priority ordering (grasp > remove > move) is a design choice; it is clearly stated and the ablations show it helps, so I do not see that as a fatal flaw—just an expressivity limitation worth noting.\n\nOverall: the core idea is plausible and the experiments are directionally consistent, but the headline claim needs better controls. This is a paper for robotic manipulation researchers, especially those working on mechanical search and hierarchical RL. I would send it to peer review rather than desk reject, but I would insist on perception-matched baselines, error bars, and at least a promise of code before acceptance.","headline":"The priority-threshold idea is neat and the ablations are honest, but the headline 4.5x efficiency claim is undercut by a perception confound that the paper never controls for.","tokens_in":11154,"tokens_out":2690,"would_cite":false,"duration_ms":27978,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A reinforcement learning agent that switches among prioritized primitives—grasp, clear occlusion, next-best view—via two learned thresholds retrieves hidden objects with higher success and up to 4.5× fewer motions than hierarchical or…","keywords":["mechanical search","reinforcement learning","action prioritization","next-best-view","cluttered manipulation","PPO thresholds","occlusion removal","robot grasping"],"falsifier":"Train XPG-RL on scenes where the target is fully invisible at the initial viewpoint and occluders are only removable from a side viewpoint, and compare it with a variant whose action order is learned or randomized: if the flexible-order variant matches or exceeds XPG-RL's success rate and motion efficiency, the fixed ordering is not the source of the reported gains. A cheaper check is to randomize the primitive order at each training episode; if performance does not degrade, the specific priority order is not load-bearing.","tokens_in":10160,"feed_emoji":"🤖","tokens_out":7785,"duration_ms":70950,"temperature":0.7,"pith_summary":"This paper tries to establish that adding a fixed, explainable priority order over a robot's action primitives makes reinforcement learning for mechanical search—finding and grasping an occluded target object—substantially more successful and efficient than learning without that structure. The agent, XPG-RL, evaluates three primitives in priority order: grasp the target, remove an occluding object, or move the camera to a next-best view. A PPO-trained policy produces two thresholds that decide when to switch from a higher-priority primitive to a lower one, so only the switching boundaries are learned while the ordering is given. In simulated scenes with 5 to 20 objects, the paper reports that XPG-RL beats flat, hierarchical, and heuristic baselines in both success rate and average motions per task, reaching a relative efficiency of 6.9 in the 20-object case versus 1.5 for the best baseline—a 4.5× gain. Real-world trials with 9-object tabletop scenes, including one fully occluded target, show the same advantage. A sympathetic reader would care because it suggests that a small, human-interpretable inductive bias can rescue long-horizon RL in data-poor manipulation settings.","feed_headline":"Priority-guided RL finds hidden objects 4.5x more efficiently","feed_subtitle":"Learned switching over grasp, clear, and viewpoint moves beats hierarchical baselines in cluttered scenes.","key_machinery":"The load-bearing mechanism is Algorithm 1, the priority-guided decision policy: a nested conditional that checks the target grasp score against learned threshold $\\tau_1$, then the occlusion-removal score against $\\tau_2$, and only then initiates a next-best-view camera move. The thresholds are the output of a PPO policy $\\pi(\\tau_t \\mid o_t)$, so the agent learns when to switch primitives but cannot reorder them. The next-best-view routine constructs a Truncated Signed Distance Function (TSDF) of the scene, simulates candidate camera poses, scores each by the predicted grasp quality of the target, and moves to the best one; it fires only when both thresholds fail, which keeps viewpoint changes from being wasted on easy scenes.","core_discovery":"The central discovery is that a fixed priority hierarchy over manipulation primitives—grasp the target, then remove an occluder, then adjust the viewpoint—combined with a learned threshold policy for advancing down that hierarchy, outperforms flat reinforcement learning, hierarchical reinforcement learning, and heuristic-driven high-level selection in mechanical search. Concretely, the policy maps a two-channel observation (a target mask produced by SAM and PoseCNN, and an object-level depth map from the proposed Object Depth Mapping module) to thresholds $\\tau_1$ and $\\tau_2$ in $[0,1]$; if the predicted success probability of grasping the target exceeds $\\tau_1$, the agent grasps, otherwise it checks whether removing an occluder is predicted to succeed above $\\tau_2$, and otherwise it executes an next-best-view camera move. The reported simulation success rates fall from 81% at 5 objects to 64% at 20 objects, while the strongest baseline (MPGNet) drops from 76% to 22%, and XPG-RL's average motion number stays at or below 4.94 even in the most cluttered setting. The paper takes this as evidence that the priority guidance reduces redundant or misaligned actions without sacrificing adaptability.","pith_inferences":["A natural extension is to replace the three hand-picked primitives with domain-specific ones for other long-horizon manipulation tasks (e.g., assembly, sorting, or search-and-rescue), reusing Algorithm 1 unchanged and learning only the thresholds.","The paper's results suggest a testable hypothesis about inductive bias: in data-limited manipulation, imposing the correct action hierarchy transfers more value than increasing network capacity or exploration, a claim one could verify by measuring the performance gap as training data shrinks.","The Object Depth Mapping representation discards per-pixel depth within objects; an ablation that feeds raw depth to the same policy would isolate whether the simplification itself contributes to learning efficiency, beyond the priority guidance.","The next-best-view criterion scores candidate views only by predicted target grasp quality; scoring views by expected improvement in both target and occluder visibility might yield better viewpoint choices in scenes where the target is deeply buried."],"forward_implications":["Deployed in warehouses or homes, a robot using priority-guided thresholds would retrieve occluded items with fewer pokes and camera sweeps, cutting cycle time and reducing disturbance of surrounding objects.","Because the high-level decision is just two scalar thresholds, the robot's behavior is auditable: an operator can see why a grasp was attempted or deferred, which helps in safety-critical or human-shared spaces.","The method's efficiency advantage grows with clutter (relative efficiency 5.0 at 10 objects, 6.9 at 20 objects), suggesting the fixed priority order is most valuable exactly where flat policies collapse.","The separation of perception (target mask plus object-level depth) from decision (thresholds) implies the framework could be ported to a different gripper or sensor suite by adapting the perception module while keeping the priority structure and PPO policy unchanged."],"supporting_citations":[{"why":"Supplies the VPG quality-map paradigm and the Target-Oriented VPG baseline that XPG-RL is compared against.","marker":"[34]"},{"why":"Provides the mechanical-search task formulation and the Hierarchical Policy Learning HRL baseline; also cited for the observation that RL efficiency drops sharply as object count grows.","marker":"[36]"},{"why":"MPGNet is the strongest heuristic-based baseline, jointly learning move, push, and grasp with a fixed high-level logic that XPG-RL must beat.","marker":"[15]"},{"why":"AnyGrasp produces the predicted grasp success scores $Q_{\\text{target}}$ and $Q_{\\text{occlude}}$ on which the threshold comparison in Algorithm 1 acts.","marker":"[6]"},{"why":"SAM generates instance-level masks for all objects, the foundation of the target-specific mask in the perception pipeline.","marker":"[9]"},{"why":"PoseCNN adds semantic context and target-specific segmentation, complementing SAM to produce the target mask.","marker":"[32]"},{"why":"PPO is the RL algorithm that optimizes the threshold policy $\\pi(\\tau_t \\mid o_t)$ in the decision module.","marker":"[26]"},{"why":"Affordance-based NBV planning inspires the conditional next-best-view routine that simulates candidate viewpoints and scores them by grasp quality.","marker":"[37]"}],"fun_headline_variants":["Explainable priority RL finds hidden objects 4.5x faster","Priority-guided policy speeds up mechanical search 4.5x","RL agent learns when to grasp, clear, or view to find objects faster","Learned thresholds boost object search efficiency by 4.5x","Adaptive switching over action primitives accelerates mechanical search"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The fixed priority order (grasp target, then remove occlusion, then adjust viewpoint) is hard-coded, so the learned policy can only tune the thresholds that decide when to move down the hierarchy; if optimal behavior in some states requires moving the camera before removing an occluder, the policy class cannot express that.","fun_headline_variants_meta":{"raw":{"variants":["Explainable priority RL finds hidden objects 4.5x faster","Priority-guided policy speeds up mechanical search 4.5x","RL agent learns when to grasp, clear, or view to find objects faster","Learned thresholds boost object search efficiency by 4.5x","Adaptive switching over action primitives accelerates mechanical search"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001581,"raw_usage":{"total_tokens":6351,"prompt_tokens":1031,"completion_tokens":5320,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":647,"completion_tokens_details":{"reasoning_tokens":5231}},"tokens_in":647,"tokens_out":5320,"duration_ms":37876,"temperature":1.0,"reasoning_tokens":5231,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T05:14:11.853509+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train XPG-RL on scenes where the target is fully invisible at the initial viewpoint and occluders are only removable from a side viewpoint, and compare it with a variant whose action order is learned or randomized: if the flexible-order variant matches or exceeds XPG-RL's success rate and motion efficiency, the fixed ordering is not the source of the reported gains. A cheaper check is to randomize the primitive order at each training episode; if performance does not degrade, the specific priority order is not load-bearing.","supporting_citations":[{"cited_title":"Learn- ing synergies between pushing and grasping with self- supervised deep reinforcement learning","cited_arxiv_id":null,"evidence_quote":"Supplies the VPG quality-map paradigm and the Target-Oriented VPG baseline that XPG-RL is compared against."},{"cited_title":"Hierarchical policy learning for mechanical search","cited_arxiv_id":null,"evidence_quote":"Provides the mechanical-search task formulation and the Hierarchical Policy Learning HRL baseline; also cited for the observation that RL efficiency drops sharply as object count grows."},{"cited_title":"Mpgnet: Learning move-push- grasping synergy for target-oriented grasping in occluded scenes","cited_arxiv_id":null,"evidence_quote":"MPGNet is the strongest heuristic-based baseline, jointly learning move, push, and grasp with a fixed high-level logic that XPG-RL must beat."},{"cited_title":"Posecnn: A convolutional neural network for 6d object pose estimation in cluttered scenes","cited_arxiv_id":null,"evidence_quote":"PoseCNN adds semantic context and target-specific segmentation, complementing SAM to produce the target mask."}],"review_version":1}