REVIEW 3 major objections 6 minor 23 references
A hierarchical reinforcement learning policy that separates obstacles before grasping can harvest occluded strawberries with 71.7–88.3% success after sim-to-real transfer.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-02 03:38 UTC pith:42NIBETU
load-bearing objection A useful HRL push-before-grasp system for strawberry harvesting, but the real-world statistical evidence for 'consistently outperforms' is weaker than the text suggests. the 3 major comments →
Vision-Based Obstacle Separation for Strawberry Harvesting in Clusters Using Hierarchical Reinforcement Learning
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The paper's central claim is that decomposing clustered strawberry harvesting into sequential obstacle separation and target grasping, and coordinating those stages with a hierarchical reinforcement learning loop, makes selective harvesting feasible under heavy occlusion. VGPA's high level selects an option (push or grasp) with the help of a vision-guided module that estimates occlusion and provides curriculum-style supervision early in training; its low level learns continuous pushing and grasping via DDPG with a Progressive Adaptive Exploration Strategy that adaptively mixes random, guided, and policy-driven actions based on stage success rates. The authors report 96.7% high-level success
What carries the argument
The central object is VGPA, a hierarchical goal-conditioned reinforcement learning framework. At the high level, a vision-guided module uses visual detection of the target and surrounding fruit to estimate occlusion and supervise option selection (push vs. grasp), with the fraction of vision-guided episodes decaying as the policy's own success rate rises. At the low level, the Progressive Adaptive Exploration Strategy (PAES) regulates exploration by mixing uniform random actions, guided actions from a previously mastered stage, and the current policy's actions, with probabilities and noise scales updated by smoothed stage success. Hindsight-experience relabeling is used to improve sample eff
Load-bearing premise
The real-world evaluation uses only 20 hand-constructed scenes per occlusion level with three repeats each (60 trials per condition) and no statistical significance testing, so the claimed consistent outperformance over direct picking assumes those scenes faithfully represent the range of cluster geometries in practice.
What would settle it
A larger field study with, say, 100 or more naturally occurring clusters per occlusion condition, reporting per-trial success rates and confidence intervals; if the success advantage over direct picking shrinks to within noise under moderate or dense occlusion, the central claim would be weakened.
If this is right
- Robotic harvesters can succeed on fruits that are visually inaccessible by actively clearing obstacles first, rather than only avoiding them.
- A vision-guided high-level policy converges faster and more stably than a hierarchy without such guidance, as shown by the ablation comparisons.
- Policies trained in simulation can transfer to a real robot without fine-tuning when trained in end-effector pose space, simplifying deployment.
- The method offers a better success-versus-time trade-off than a vision-language-model planner: 80.0% average success in 4.15 s versus 59.7% in 2.93 s for direct picking, and 7.23 s for the VLM baseline.
- In dense occlusion the approach still fails in about 28% of trials, often because the swallowing motion picks both the obstacle and the target fruit together.
Where Pith is reading between the lines
- The same push-then-grasp decomposition could transfer to other cluster-growing crops such as tomatoes, grapes, or cherries, where the obstacle is also unripe fruit or foliage and the reward structure is analogous.
- The vision-guided module acts as a learned curriculum: using a separate perception signal to supervise option selection early in training is a general recipe for long-horizon manipulation tasks where a clear visual gating signal exists.
- The residual failures under dense occlusion suggest that a more dexterous end-effector (e.g., suction or compliant fingers) combined with the learned pushing policy could push success rates higher than the reported 71.7%.
- Because the real-world results came from 20 researcher-built scenes per condition in a single greenhouse, the natural next test is a diverse field trial with naturally varying cluster geometries and statistical confidence intervals.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes VGPA, a hierarchical reinforcement learning framework for selective strawberry harvesting in clusters. The high-level policy selects between obstacle separation and grasping options; a vision-guided (VG) module injects image-based occlusion information to accelerate high-level learning, while a Progressive Adaptive Exploration Strategy (PAES) adaptively balances random, guided, and policy-driven low-level actions. The low-level policies are trained with DDPG and HER. The authors report a 96.7% high-level success rate in PyBullet simulation and, after sim-to-real transfer on a custom parallel robot, real-world success rates of 88.3%, 80.0%, and 71.7% under slight, moderate, and dense occlusion, compared with 75.7%, 58.3%, and 45.0% for direct picking. The paper also includes ablations of VG and PAES and a comparison against HAC and a VLM-based planner.
Significance. If the reported results hold, the paper makes a practically relevant contribution to agricultural robotics by showing that a push-before-grasp hierarchical policy can improve harvesting in clustered strawberry environments and transfer from simulation to a real robot. The hierarchical decomposition, vision-guided option supervision, and adaptive exploration are sensible design choices, and the inclusion of real-world deployment on a self-built robot is a strength. The paper also provides a clear qualitative failure analysis and an honest acknowledgment of environmental simplifications. However, the central real-world superiority claim rests on a small, clustered sample with no statistical reporting, and the HAC baseline comparison is not explained well enough to establish that the proposed method is genuinely better rather than better tuned or differently inductive-biased. The work is therefore promising but not yet fully convincing as stated.
major comments (3)
- [IV-B3, Table I] The claim that VGPA 'consistently outperforms direct picking under different occlusion conditions' is not statistically established. Each reported rate is based on 60 trials (20 scenes × 3 repeated trials), the three trials per scene are not independent, and no confidence intervals, per-scene breakdowns, or statistical tests are provided. For the slight-occlusion condition, 88.3% (53/60) vs 75.7% (45/60) is only marginally significant even under a naive two-proportion test (z≈1.8, p≈0.07), and clustering further reduces the effective sample size. The moderate and dense occlusion differences are larger, but the blanket claim covers all conditions. The authors should report cluster-adjusted confidence intervals and a proper hypothesis test, and specify whether the 20 scenes were randomly sampled or fixed.
- [IV-B1, Fig. 5] The HAC comparison reports a striking gap (96.7% vs 7.77% high-level success) without explaining why HAC essentially fails to learn. The manuscript does not describe HAC's hyperparameters, network architecture, exploration schedule, reward structure, or whether the same epoch/episode budget and random seeds were used. Because VGPA uses vision-guided demonstrations and a stage-aware exploration schedule, the comparison may not be apples-to-apples. The authors should provide a fair HAC configuration, report multiple-seed means with variance, and ideally run HAC with the same number of environment interactions and a comparable exploration mechanism to isolate the contribution of the proposed components.
- [III-B, Algorithm 1] The vision-guided module directly supplies the option in 'VG episodes,' with the fraction of such episodes governed by Eq. (3). This means part of the high-level training signal comes from a hand-designed occlusion rule, not from the learned policy. The reported 96.7% high-level success rate may therefore conflate the learned decision policy with the injected heuristic, especially if VG episodes or VG-informed option selection remain active during evaluation. Please state explicitly whether VG supervision is disabled at evaluation and report high-level performance with and without the vision-guided option injection.
minor comments (6)
- [IV-B3, Table I vs text] The text states direct picking's average grasping time is 2.93 s, but Table I reports 2.63 s. Please make the numbers consistent.
- [III-D, Eqs. (8)-(10)] The reward thresholds in Eqs. (8)-(10) are presented without motivation. For example, the pushing reward requires (obs-to-tar)_x to lie in a 5 mm band; please define the coordinate frame and explain why this interval was chosen. Also, Eq. (7) uses the symbol g⟨ which is never defined.
- [III-B, Eq. (3)] Eq. (3) contains several schedule constants (α_max, α_min, k, τ0, λ) but no sensitivity analysis is given. Since PAES and VG are central contributions, the sensitivity of the final result to these constants should be discussed or at least bounded.
- [III, Algorithm 1] The high-level policy is said to be optimized using 'DIOL,' but DIOL is neither expanded nor cited. Please define it in the text or replace with the full algorithm name and reference.
- [IV-B2, Fig. 6] The ablation curves in Fig. 6 show only one trajectory per setting; reporting multiple seeds with mean and standard deviation would substantially strengthen the claim that VG and PAES improve convergence and stability.
- [IV-B3, VLM comparison] The VLM baseline is described as 'developed in our group' but no implementation details or citation are provided beyond [23]. Since this is an internal comparison, it should be positioned as such and the differences in action space, perception, and planning should be described.
Circularity Check
No significant circularity: reported success rates are measured experimental outcomes, not quantities derived from fitted parameters or forced by self-citation.
full rationale
The paper's central claims are empirical: VGPA achieves 96.7% high-level success in simulation and 88.3%/80.0%/71.7% real-world success under increasing occlusion, compared with 75.7%/58.3%/45.0% for direct picking. These are measured by executing the trained policy in test scenarios, not by algebraically deriving the result from the method's parameters. The vision-guided module (Eq. 3, Algorithm 1) is a training-time curriculum that adjusts the proportion of hand-specified option demonstrations; it is not a post-hoc fit to the reported success rates, and the paper explicitly reduces its influence as the high-level success rate rises. PAES (Eqs. 4–6) adapts exploration during training; it does not produce the final success rates by construction. The self-citations ([5], [19], [22], [23]) describe a prior obstacle-separation strategy, a detection/ripeness method, the robotic arm platform, and a VLM baseline; none is invoked as a uniqueness theorem or as the justification that the proposed method's success rates must have particular values. The small real-world sample size and absence of statistical testing are correctness/evidence concerns, not circularity. No step in the derivation chain reduces to its own inputs.
Axiom & Free-Parameter Ledger
free parameters (3)
- Vision-guided demonstration ratio schedule (α_max, α_min, k, τ_0, λ) =
not fully reported
- PAES bounds (c_α, c_σ, c_g, γ_min, γ_max, κ) =
not fully reported
- Low-level reward weights and thresholds (0.7/0.3, distance thresholds 0.045/0.01, pose ±10°) =
0.7/0.3 weight, 0.04–0.045 push distance, 0.0–0.01 grasp distance, ±10°
axioms (4)
- domain assumption YOLOv11 detection and depth back-projection produce sufficiently accurate 3D fruit positions for policy input.
- domain assumption The PyBullet simulation with SolidWorks strawberry models captures enough physics (rigid push interactions, similar friction) for the learned push behavior to transfer.
- domain assumption The benchmark HAC baseline was implemented with comparable hyperparameter and training effort to VGPA.
- domain assumption The 20-scene occlusion taxonomy (slight/moderate/dense) is representative of real clustered strawberry canopies.
invented entities (1)
-
No new physical entities (forces, objects, or mediators) are introduced.
no independent evidence
Cite this review
Pith. "Pith review of Vision-Based Obstacle Separation for Strawberry Harvesting in Clusters Using Hierarchical Reinforcement Learning." pith.science (2026). https://pith.science/paper/42NIBETU
@misc{pith2026260713799,
author = {Pith},
title = {Pith review of: Vision-Based Obstacle Separation for Strawberry Harvesting in Clusters Using Hierarchical Reinforcement Learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/42NIBETU}},
note = {Machine review of arXiv:2607.13799}
}
read the original abstract
Selective harvesting in clustered strawberry environments is challenging because ripe fruits are often occluded by surrounding unripe fruits, making direct grasping unreliable. To address this problem, this paper proposes a hierarchical reinforcement learning framework, termed VGPA, which integrates a vision-guided decision mechanism and a Progressive Adaptive Exploration Strategy (PAES) for vision-based obstacle separation and harvesting. The task was decomposed into two sequential stages: obstacle separation and target grasping. At the high level, the vision-guided mechanism improved option selection and accelerated policy convergence. At the low level, PAES improved exploration efficiency and training stability during continuous control learning. In simulation experiments, the learned policy achieved a success rate of 96.7%. In addition, sim-to-real transfer experiments on a self-developed parallel robot showed that the proposed method achieved success rates ranging from 71.7% to 88.3%, outperforming direct picking while requiring only 1.22~s more average harvesting time. These results verified the effectiveness, generalization ability, and practical potential of the proposed method for robotic harvesting in complex clustered environments.
Figures
Reference graph
Works this paper leans on
-
[1]
Machine learning in agriculture domain: A state-of-art survey.Artificial Intelligence in the Life Sciences, 1:100010, 2021
Vishal Meshram, Kailas Patil, Vidula Meshram, Dinesh Hanchate, and SD Ramkteke. Machine learning in agriculture domain: A state-of-art survey.Artificial Intelligence in the Life Sciences, 1:100010, 2021
2021
-
[2]
Fruit localization and environment perception for strawberry harvest- ing robots.IEEE access, 7:147642–147652, 2019
Yuanyue Ge, Ya Xiong, Gabriel Lins Tenorio, and P ˚al Johan From. Fruit localization and environment perception for strawberry harvest- ing robots.IEEE access, 7:147642–147652, 2019
2019
-
[3]
Peduncle collision-free grasping based on deep reinforcement learning for tomato harvesting robot.Computers and Electronics in Agriculture, 216:108488, 2024
Yajun Li, Qingchun Feng, Yifan Zhang, Chuanlang Peng, Yuhang Ma, Cheng Liu, Mengfei Ru, Jiahui Sun, and Chunjiang Zhao. Peduncle collision-free grasping based on deep reinforcement learning for tomato harvesting robot.Computers and Electronics in Agriculture, 216:108488, 2024
2024
-
[4]
Reaching pruning locations in a vine using a deep rein- forcement learning policy
Francisco Yandun, Tanvir Parhar, Abhisesh Silwal, David Clifford, Zhiqiang Yuan, Gabriella Levine, Sergey Yaroshenko, and George Kantor. Reaching pruning locations in a vine using a deep rein- forcement learning policy. In2021 IEEE International Conference on Robotics and Automation (ICRA), pages 2400–2406. IEEE, 2021
2021
-
[5]
An improved obsta- cle separation method using deep learning for object detection and tracking in a hybrid visual control loop for fruit picking in clusters
Ya Xiong, Yuanyue Ge, and P ˚al Johan From. An improved obsta- cle separation method using deep learning for object detection and tracking in a hybrid visual control loop for fruit picking in clusters. Computers and Electronics in Agriculture, 191:106508, 2021
2021
-
[6]
Emlyn Williams and Athanasios Polydoros. Zero-shot sim-to- real reinforcement learning for fruit harvesting.arXiv preprint arXiv:2505.08458, 2025
Pith/arXiv arXiv 2025
-
[7]
Efficient learning of goal-oriented push-grasping synergy in clutter
Kechun Xu, Hongxiang Yu, Qianen Lai, Yue Wang, and Rong Xiong. Efficient learning of goal-oriented push-grasping synergy in clutter. IEEE Robotics and Automation Letters, 6(4):6337–6344, 2021
2021
-
[8]
Learning bifunctional push-grasping synergistic strategy for goal- agnostic and goal-oriented tasks
Dafa Ren, Shuang Wu, Xiaofan Wang, Yan Peng, and Xiaoqiang Ren. Learning bifunctional push-grasping synergistic strategy for goal- agnostic and goal-oriented tasks. In2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 2909–
-
[9]
Learning extrinsic dexterity with parameterized manipu- lation primitives
Shih-Min Yang, Martin Magnusson, Johannes A Stork, and Todor Stoyanov. Learning extrinsic dexterity with parameterized manipu- lation primitives. In2024 IEEE International Conference on Robotics and Automation (ICRA), pages 5404–5410. IEEE, 2024
2024
-
[10]
Between mdps and semi-mdps: A framework for temporal abstraction in reinforce- ment learning.Artificial intelligence, 112(1-2):181–211, 1999
Richard S Sutton, Doina Precup, and Satinder Singh. Between mdps and semi-mdps: A framework for temporal abstraction in reinforce- ment learning.Artificial intelligence, 112(1-2):181–211, 1999
1999
-
[11]
The option-critic architecture
Pierre-Luc Bacon, Jean Harb, and Doina Precup. The option-critic architecture. InProceedings of the AAAI conference on artificial intelligence, volume 31, 2017
2017
-
[12]
Jesse Zhang, Haonan Yu, and Wei Xu. Hierarchical reinforce- ment learning by discovering intrinsic options.arXiv preprint arXiv:2101.06521, 2021
Pith/arXiv arXiv 2021
-
[13]
Hierarchical reinforcement learning with universal policies for multistep robotic manipulation.IEEE Transac- tions on Neural Networks and Learning Systems, 33(9):4727–4741, 2021
Xintong Yang, Ze Ji, Jing Wu, Yu-Kun Lai, Changyun Wei, Guoliang Liu, and Rossitza Setchi. Hierarchical reinforcement learning with universal policies for multistep robotic manipulation.IEEE Transac- tions on Neural Networks and Learning Systems, 33(9):4727–4741, 2021
2021
-
[14]
Hierarchical visual policy learning for long-horizon robot manipulation in densely cluttered scenes
Hecheng Wang, Lizhe Qi, Ziheng Wang, Jiankun Ren, Wei Li, and Yunquan Sun. Hierarchical visual policy learning for long-horizon robot manipulation in densely cluttered scenes. In2025 IEEE International Conference on Robotics and Automation (ICRA), pages 1149–1155. IEEE, 2025
2025
-
[15]
Impedance primitive-augmented hierarchical reinforcement learning for sequential tasks
Amin Berjaoui Tahmaz, Ravi Prakash, and Jens Kober. Impedance primitive-augmented hierarchical reinforcement learning for sequential tasks. In2025 IEEE International Conference on Robotics and Automation (ICRA), pages 10973–10979. IEEE, 2025
2025
-
[16]
Context- aware grasp generation in cluttered scenes
Dinh-Cuong Hoang, Johannes A Stork, and Todor Stoyanov. Context- aware grasp generation in cluttered scenes. In2022 International Conference on Robotics and Automation (ICRA), pages 1492–1498. IEEE, 2022
2022
-
[17]
Learning to grasp the ungraspable with emergent extrinsic dexterity
Wenxuan Zhou and David Held. Learning to grasp the ungraspable with emergent extrinsic dexterity. InConference on Robot Learning, pages 150–160. PMLR, 2023
2023
-
[18]
Continuous control with deep reinforcement learning.arXiv preprint arXiv:1509.02971, 2015
Timothy P Lillicrap, Jonathan J Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. Continuous control with deep reinforcement learning.arXiv preprint arXiv:1509.02971, 2015
Pith/arXiv arXiv 2015
-
[19]
Light- resilient visual regression of strawberry ripeness for robotic harvesting
Meili Sun, Shimin Hu, Chunjiang Zhao, and Ya Xiong. Light- resilient visual regression of strawberry ripeness for robotic harvesting. Computers and Electronics in Agriculture, 241:111169, 2026
2026
-
[20]
Pybullet quickstart guide.ed: PyBullet Quickstart Guide
Erwin Coumans and Yunfei Bai. Pybullet quickstart guide.ed: PyBullet Quickstart Guide. https://docs. google. com/document/u/1/d, 2021
2021
-
[21]
Learning multi-level hierarchies with hindsight.arXiv preprint arXiv:1712.00948, 2017
Andrew Levy, George Konidaris, Robert Platt, and Kate Saenko. Learning multi-level hierarchies with hindsight.arXiv preprint arXiv:1712.00948, 2017
Pith/arXiv arXiv 2017
-
[22]
Design and control of a novel six-degree-of-freedom hybrid robotic arm
Yang Chen, Zhonghua Miao, Yuanyue Ge, Sen Lin, Liping Chen, and Ya Xiong. Design and control of a novel six-degree-of-freedom hybrid robotic arm. In2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 3597–3604. IEEE, 2024
2024
-
[23]
Active obstacle separation: Vision-language model (vlm) driven clearing decisions for robotic harvesting.Smart Agricultural Technology, page 102037, 2026
Quan Zhao, Yuanyue Ge, Hui Zhao, and Ya Xiong. Active obstacle separation: Vision-language model (vlm) driven clearing decisions for robotic harvesting.Smart Agricultural Technology, page 102037, 2026
2026
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.