Pith. sign in

REVIEW 4 major objections 5 minor 17 references

Learning to Grasp from 2.5D images: a Deep Reinforcement Learning Approach

T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A deep reinforcement learning policy trained on 2.5D depth images alone can regress a suction-grasp tool pose with negligible position and orientation errors, according to this paper.

desk verdict A plausible depth-only DRL proof-of-concept whose headline 'negligible errors' claim is not backed by the reported experiments. read the letter →

arxiv 1908.03440 v1 pith:YVRPISUE submitted 2019-08-08 cs.RO cs.LGstat.ML

Alessia Bertugli , Paolo Galeone This is my paper · ORCID
classification cs.ROcs.LGstat.ML
keywords deepreinforcementlearninggrasping2.5DimagesdepthcameravacuumgrippercurriculumPPOrobotmanipulation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a deep reinforcement learning agent can grasp flat blocks with a vacuum gripper using only 2.5D depth images as input, and that this is the first demonstration of a DRL policy regressing 3D tool coordinates from such images alone. The authors built a simulated robot setup with a fixed depth camera and compared PPO, TRPO, and DDPG. They report that PPO combined with a 19-lesson curriculum lands the suction tooltip on the block surface with position error near 1 cm and yaw error near 2 degrees, while the other algorithms underperform. If true, the result matters because depth-only perception is cheap and robust to illumination changes, making flexible suction grasping easier to deploy in structured settings.

What carries the argument

The carrying mechanism is the pairing of dense reward shaping with a 19-lesson curriculum on a PPO policy. The state is a single simulated depth image, with intensity scaled to a 0.4-2 m camera range and Gaussian noise added; the action is a continuous tool pose. Rewards grant +0.1 for touching the target, -0.1 for undesired collisions, +0.5 for reaching the position tolerance around the block centroid, and another +0.5 for matching the block's yaw, plus two progressive rewards that rate the direction of motion and end-effector orientation. The curriculum narrows the position tolerance from 0.1 m toward 0.01 m and the yaw tolerance from 10 degrees to 2 degrees across lessons, pushing the policy toward precision.

What would settle it

Evaluate the trained PPO-plus-curriculum policy on a held-out set of random block positions, orientations, and shapes, recording the physical outcome of the suction attempt (whether the block lifts and stays attached) together with the measured tooltip pose error. If the policy frequently enters the rewarded tolerance zone without achieving a grasp, the dense-reward proxy is the weak link; if it misses the zone, the pose-regression claim fails.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that a Proximal Policy Optimization (PPO) agent can solve a continuous suction-grasping task when fed a single 2.5D depth image and asked to output an end-effector pose. The final curriculum lesson demands the tooltip be within 0.01 m of the block centroid in each positional coordinate and within 2 degrees of the block's yaw rotation, and the paper reports that the policy reaches the object surface with negligible position and orientation errors. The same experiments show DDPG performing poorly, TRPO being unstable, and PPO without curriculum learning not reaching the same precision, which the authors attribute to the staged tightening of the reward region.

Load-bearing premise

The reward thresholds (tooltip within 0.1 m in x and y, 0.01-0.02 m in z, and within 10 degrees of the block yaw, tightened to 0.01 m and 2 degrees in the final lesson) are treated as the definition of a successful grasp, but the paper never measures whether a tool configuration that meets them actually lifts the block.

Editorial extensions

If this is right

  • A robot can solve this suction-grasping task with no RGB input, needing only a depth camera and a learned policy.
  • The curriculum schedule, with 19 lessons tightening position and yaw thresholds, is the component that produces precision, so reward shaping plus staged difficulty appears to be the key recipe.
  • DDPG and TRPO are not competitive on this task within the tested steps, suggesting that policy-gradient methods with staged curricula are better suited to continuous pose regression from images.
  • Sparse rewards do not generate learning here, so dense, continuously informative rewards are necessary for this kind of manipulation task.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper stops at cumulative reward curves; a held-out pose-error table or physical grasp test would directly test the 'negligible errors' claim.
  • If the behavior transfers to a real depth camera, a single low-cost depth sensor plus this policy could form a complete suction-grasping front end, with no color camera and no hand-crafted vision pipeline.
  • The same graduated tightening of a reward region could be applied to other precision pose-regression tasks, such as peg insertion or tool alignment, where the end effector must converge to a tight tolerance.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a deep reinforcement learning (DRL) approach for controlling a simulated Kuka KR16 robot equipped with a vacuum gripper, using 2.5D depth images as the only visual input. The goal is to regress a 3D end-effector pose that brings the suction head to the surface of blocks of varying shape, size, position, and orientation. The authors compare PPO, TRPO, and DDPG, and also study curriculum learning, different CNN architectures, and input configurations (depth only, depth plus RGB, frame grab). The reported results are cumulative reward curves during training. The abstract claims that the method reaches the object surface with "negligible position and orientation errors" and that this is "the first successful attempt" of using 2.5D images only as input to a DRL algorithm for grasping with 3D world-coordinate regression.

Significance. If the central claim were supported, the work would be a useful demonstration that a DRL policy can regress a precise suction-grasp pose from depth images alone, potentially enabling low-cost visual grasping in structured environments. The paper also provides a simulated environment and a comparison of popular DRL algorithms on this task, which could be a starting point for further study. However, the significance as currently established is low: the only evidence presented consists of training reward curves, with no quantitative measurement of pose error, grasp success, or physical validity of the learned behavior. The claimed novelty and effectiveness are therefore not demonstrated by the experiments.

major comments (4)
  1. [Abstract and Section V] The central claim of "negligible position and orientation errors" is not supported by any quantitative measurement. All results shown in Figure 5 are cumulative reward curves during training, and no held-out evaluation reports tooltip error, percentage of successful grasps, contact events, or whether the object was lifted. A rising training reward is an optimization signal, not a performance metric, so the abstract's claim is unsubstantiated by the experimental section.
  2. [Section III-C and Section III-D] The reward function defines success as reaching hand-set thresholds: ±0.1 m in x/y, 0.01–0.02 m in z, and ±10 degrees in y-rotation, tightened in the curriculum to ±0.01 m and 2 degrees. The paper never validates that these thresholds correspond to a physically successful suction grasp. For a vacuum gripper, success requires proper surface contact and normal alignment, and the z window in particular is not tied to any measured suction or lift outcome. Thus a high cumulative reward only shows that the policy satisfied the proxy, not that the proxy is faithful.
  3. [Section V] The experimental protocol is not rigorous enough to support the paper's conclusions. The authors state that "tests were interrupted when the result seemed satisfactory," and no seed variance, multiple runs, or statistical analysis is reported. With a single run and subjective stopping criteria, the comparison between algorithms (PPO good, DDPG poor, TRPO unstable) is anecdotal rather than quantitative. The paper should report held-out metrics with confidence intervals or at least multiple seeds.
  4. [Abstract and Section I] The claim of being "the first successful attempt" is unsupported because no comparison to existing grasp methods or baselines is provided, and there is no quantitative evidence that the proposed method succeeds at grasping. The novelty claim cannot be evaluated from the reported experiments.
minor comments (5)
  1. [Section III-A, Eq. (1)] Equation (1) uses 'actualv' without defining how this value is obtained from the raw sensor reading; clarify the notation and units.
  2. [Section V] The reference "stable baselines [?]" is missing its citation; please add the appropriate reference.
  3. [Throughout] There are numerous typos and grammatical errors, e.g., "as of the input" in the abstract, "particularity adapt" in Section V-C, and "starting form" in Section III-D. A careful proofreading is needed.
  4. [Figure 5] The caption of Figure 5 lists subfigures (a)–(k) but does not describe the content of each subfigure; the reader must infer which configuration each curve corresponds to from the text, which is difficult.
  5. [Section III-C] The total reward function is not explicitly defined as a sum of the sparse terms (0.1, -0.1, 0.5) and the dense terms r_fmt and r_fft; the combination rule should be stated explicitly.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the results are an empirical RL evaluation, with no fitted parameter or self-citation masquerading as a prediction.

full rationale

The paper is an empirical reinforcement-learning study, not a derivation chain. The dense reward design in Section III-C, including the hand-set position and rotation thresholds and the shaping constants k1=0.03 and k2=0.01, defines the task objective; the experiments then show that PPO with curriculum learning maximizes cumulative reward. No quantity that is claimed as a prediction is obtained by fitting to itself: k1 and k2 are reward-shaping coefficients, not parameters tuned to reproduce the reported outcome, and the curriculum's final goal box of 0.01 m and 2 degrees is the target the policy is trained to reach, not an independent result derived from the trained policy. There are no self-citations, no imported uniqueness theorems, and no renamed external results. The main weakness is an evaluation gap: the abstract's claim of 'negligible position and orientation errors' is inferred from cumulative-reward curves and manually defined reward thresholds, and Section V states that tests 'were interrupted when the result seemed satisfactory' rather than reporting held-out tool-pose errors or physical grasp successes. That is a correctness risk because the reward proxy may not equal a physically successful suction grasp, but it is not circularity: the reward is the training objective, not a fitted input renamed as a prediction.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the validity of the simulation and the reward thresholds as proxies for real grasping. The free parameters are limited to reward shaping constants and success thresholds; no new physical entities or forces are introduced.

free parameters (2)
  • reward scaling constants k1, k2 = 0.03, 0.01
    Hand-selected constants in the progressive reward functions (Eq. 2 and 4); they control reward magnitude and are not derived from data.
  • dense reward thresholds (position and orientation success criteria) = 0.1 m x/y, 0.01-0.02 m z, 10 degrees; final curriculum lesson 0.01 m and 2 degrees
    These thresholds define the success zone for the end-effector. They are set by the authors and effectively encode the claimed precision target.
assumptions (4)
  • domain assumption Unity 3D with ML-Agents accurately simulates the robot, physics, and camera.
    The policy is trained and evaluated only in simulation; the fidelity of the simulator for real grasping is unverified.
  • domain assumption Gaussian noise and value clamping to [0.4,2] m faithfully model a real depth camera (Orbbec Astra S).
    Used to define the visual input; no real camera images are used to validate this model.
  • domain assumption The dense reward signal is a valid objective for grasp success.
    Success is defined by reaching the hand-set position and rotation intervals; the paper assumes this is equivalent to a successful suction grasp.
  • domain assumption The static, single-frame formulation (one configuration per episode, frame grab) is representative of the target grasping task.
    The agent does not need temporal information; this simplifies the problem and may not generalize to dynamic scenes.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Learning to Grasp from 2.5D images: a Deep Reinforcement Learning Approach." pith.science (2026). https://pith.science/paper/YVRPISUE

@misc{pith2026190803440,
  author       = {Pith},
  title        = {Pith review of: Learning to Grasp from 2.5D images: a Deep Reinforcement Learning Approach},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YVRPISUE}},
  note         = {Machine review of arXiv:1908.03440}
}
read the original abstract

In this paper, we propose a deep reinforcement learning (DRL) solution to the grasping problem using 2.5D images as the only source of information. In particular, we developed a simulated environment where a robot equipped with a vacuum gripper has the aim of reaching blocks with planar surfaces. These blocks can have different dimensions, shapes, position and orientation. Unity 3D allowed us to simulate a real-world setup, where a depth camera is placed in a fixed position and the stream of images is used by our policy network to learn how to solve the task. We explored different DRL algorithms and problem configurations. The experiments demonstrated the effectiveness of the proposed DRL algorithm applied to grasp tasks guided by visual depth camera inputs. When using the proper policy, the proposed method estimates a robot tool configuration that reaches the object surface with negligible position and orientation errors. This is, to the best of our knowledge, the first successful attempt of using 2.5D images only as of the input of a DRL algorithm, to solve the grasping problem regressing 3D world coordinates.

Figures

Figures reproduced from arXiv: 1908.03440 by the authors.

Figure 1
Figure 1. Simulated environment with Kuka KR16 equipped [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 3
Figure 3. Blocks with different shapes and dimensions. [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 2
Figure 2. Depth camera view with Gaussian noise. (a) Block with parallelepiped shape. (b) Block with L shape. (c) Block with U shape [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: View from RGB camera placed over the support [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Cumulative reward of all tests [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

17 extracted references · 11 canonical work pages

  1. [1]

    Unity: A general platform for intelligent agents,

    A. Juliani, V .-P. Berges, Y . G. Esh Vckay, H. Henry, M. Mattar, and D. Lange, “Unity: A general platform for intelligent agents,” 2018. [Online]. Available: arXivpreprintarXiv:1809.02627

    arXiv 2018

  2. [2]

    Playing atari with deep reinforcement learning,

    V . Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller, “Playing atari with deep reinforcement learning,” 2013

    2013

  3. [3]

    Deeploco: Dynamic locomotion skills using hierarchical deep reinforcement learning,

    X. B. Peng, G. Berseth, K. Yin, and M. V . D. Panne, “Deeploco: Dynamic locomotion skills using hierarchical deep reinforcement learning,” ACM Transactions on Graphics (TOG), vol. 36, no. 41, p. Issue 4, Jul. 2017

    work page 2017

  4. [4]

    Gibson env: Real-world perception for embod- ied agents,

    F. Xia, A. R. Zamir, Z.-Y . He, A. Sax, J. Malik, and S. Savarese, “Gibson env: Real-world perception for embod- ied agents,” 2018

    work page 2018

  5. [5]

    Deep reinforcement learning for vision-based robotic grasping: A simulated comparative evaluation of off- policy methods,

    D. Quillen, E. Jang, C. F. Ofir Nachum, J. Ibarz, and S. Levine, “Deep reinforcement learning for vision-based robotic grasping: A simulated comparative evaluation of off- policy methods,” 2018

    work page 2018

  6. [6]

    Learning complex dexterous manipulation with deep reinforcement learning and demonstrations,

    A. Rajeswaran, V . Kumar, G. V . Abhishek Gupta, J. Schulman, E. Todorov, and S. Levine, “Learning complex dexterous manipulation with deep reinforcement learning and demonstrations,” 2018. [Online]. Available: arXivpreprintarXiv:1709.10087

    arXiv 2018

  7. [7]

    A natural policy gradient,

    S. Kakade, “A natural policy gradient,” 2002

    work page 2002

  8. [8]

    Learning to fly by combining reinforcement learning with behavioural cloning,

    E. F. Morales and C. Sammut, “Learning to fly by combining reinforcement learning with behavioural cloning,” inProceed- ing ICML ’04 Proceedings of the twenty-first international conference on Machine learning , 2004, p. 76

    work page 2004

Show all 17 references
  1. [9]

    Asymmetric actor critic for image-based robot learning,

    L. Pinto, M. Andrychowicz, P. Welinder, W. Zaremba, and P. Abbeel, “Asymmetric actor critic for image-based robot learning,” 2018

    2018

  2. [10]

    Sim-to-real robot learning from pixels with progressive nets,

    A. A. Rusu, M. Vecerk, T. Rothrl, N. Heess, R. Pascanu, and R. Hadsell, “Sim-to-real robot learning from pixels with progressive nets,” 2018. [Online]. Available: arXivpreprintarXiv:1610.04286

    2018 arXiv

  3. [11]

    Actor-critic algorithms,

    V . R. Konda and J. N. Tsitsiklis, “Actor-critic algorithms,” in Advances in neural information processing systems , 2000, p. 10081014

    2000

  4. [12]

    Continuous control with deep reinforcement learning,

    T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y . Tassa, D. Silver, and D. Wierstra, “Continuous control with deep reinforcement learning,” 2016

    2016

  5. [13]

    Deep reinforcement learning that matters,

    P. Henderson, R. Islam, P. Bachman, J. Pineau, D. Precup, and D. Meger, “Deep reinforcement learning that matters,” 2017

    2017

  6. [14]

    Deep reinforcement learning: A brief survey,

    K. Arulkumaran, M. P. Deisenroth, M. Brundage, and A. A. Bharath, “Deep reinforcement learning: A brief survey,”IEEE Signal Processing Magazine , vol. 34, no. 6, pp. 26–38, Nov. 2017

    2017

  7. [15]

    Trust region policy optimization,

    J. Schulman, S. Levine, P. Moritz, M. I. Jordan, and P. Abbeel, “Trust region policy optimization,” 2017. [Online]. Available: arXivpreprintarXiv:1502.05477

    2017 arXiv

  8. [16]

    Proximal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” 2017. [Online]. Available: arXivpreprintarXiv:1707.06347

    2017 arXiv

  9. [17]

    Cur- riculum learning,

    Y . Bengio, J. Louradour, R. Collobert, and J. Weston, “Cur- riculum learning,” 2009. (a) Cumulative reward of sparse rewards test. (b) Cumulative reward of test with two cameras: depth plus RGB. (c) Cumulative reward frame grab test. (d) Cumulative reward of 32x32 images test...

    2009

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.