Interactive Shaping of Granular Media Using Reinforcement Learning

Anny Ripke; Benedikt Kreis; Malte Mosbach; Maren Bennewitz; Muhammad Ehsan Ullah; Sven Behnke

REVIEW 3 major objections 5 minor 41 references

Interactive Shaping of Granular Media Using Reinforcement Learning

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A simulation-trained reinforcement learning policy carves sand into target shapes with 3.4 mm mean error, and transfers zero-shot to a real robot.

desk verdict Solid sim evaluation of reward and observation design for sand shaping; the real-world transfer claim outruns the evidence. read the letter →

arxiv 2509.06469 v2 pith:RXSEISP5 submitted 2025-09-08 cs.RO

Benedikt Kreis , Malte Mosbach , Anny Ripke , Muhammad Ehsan Ullah , Sven Behnke , Maren Bennewitz This is my paper · ORCID

classification cs.RO

keywords reinforcementlearninggranularmediamanipulationheightmaprepresentationsim-to-realtransferrewardshapingTruncatedQuantileCriticsroboticsand

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a reinforcement-learned policy can shape granular media such as sand into arbitrary target structures with millimeter-level accuracy, provided the task is represented compactly. The authors show that a difference height map between the current and goal sand surface, combined with binary masks for the end-effector and goal area and a two-part dense reward, lets a visual policy trained entirely in simulation learn effective shaping behaviors. Their best policy, trained with the 'delta' reward and the Truncated Quantile Critics algorithm, reaches a mean absolute cell height difference of $3.4\,\text{mm}$ over 100 evaluation episodes, outperforming a progressive-reward policy, a coverage-planning baseline, and random actions. The same policy, deployed zero-shot on a real robot with a stereo camera, forms a rectangle in a real sandbox, indicating that the height-map representation transfers across the sim-to-real gap.

What carries the argument

The load-bearing mechanism is the pairing of a compact observation encoder with a dense two-part reward. The observation stack combines the difference height map $H_d = H_g - H_c$ (with current heights truncated at the initial flat level) with a binary end-effector mask; a sigmoid-gated goal mask modulates the CNN-extracted features to focus on the goal area, yielding a 64-dimensional feature vector that is concatenated with the end-effector's position observations. The shaping reward $r_{\text{delta}} = \alpha_c(\hat{d}_{t-1} - \hat{d}_t)$ rewards reductions in mean absolute cell height difference, while the movement reward $r_m = -\tanh(\alpha_m d_m) + \mathbb{1}_{\text{reached}}$ pulls the end-effector toward the goal area; an alternative 'progressive' variant rewards progress relative to the best configuration achieved in the episode. Training runs on a height-map sand simulation built on an angle-of-repose collapse model, where excess sand is redistributed between adjacent cells until the local slope condition is met, which keeps the simulation fast enough for online RL.

What would settle it

Run the trained DELTA policy on the real robot for repeated episodes (e.g., 30 trials) across several goal shapes and measure the mean absolute cell height difference in the goal area; if the real-world mean error exceeds roughly twice the simulated $3.4\,\text{mm}$, or if non-rectangle shapes are unrecognizable, the zero-shot sim-to-real claim is falsified. Alternatively, compare the simulated angle-of-repose collapse against real sand for the same end-effector trajectory; a systematic mismatch in how far sand redistributes would falsify the simulator premise.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes that interactive shaping of granular media can be formulated as a goal-conditioned reinforcement learning problem solvable from visual observations. The central claim is that a compact observation—the per-cell difference between goal and current truncated height maps, the end-effector's projected footprint mask, and a goal-area mask—together with a dense reward that rewards reductions in mean absolute height difference $\hat{d}$ and steers the end-effector toward the goal area, enables a Truncated Quantile Critics policy to carve shapes despite collapsing sand dynamics. In 100 simulated evaluation episodes, the DELTA policy achieves a mean error of $3.4\,\text{mm}$ within the goal area, significantly lower than the progressive-reward policy ($4.5\,\text{mm}$), a Boustrophedon coverage planner ($4.8\,\text{mm}$), and random actions ($7.2\,\text{mm}$). The authors further claim zero-shot transfer to a real robot: the simulation-trained visual policy, observing only reconstructed depth-based height maps, produces a rectangle in a real sandbox without additional training.

Load-bearing premise

The load-bearing assumption is that the angle-of-repose height-map simulation is faithful enough that a policy trained only in simulation, on reconstructed depth-based height maps, generalizes zero-shot to real sand; the supporting real-world evidence is a single qualitative rectangle demo with no repeated trials or quantitative accuracy measurement.

Editorial extensions

If this is right

Simulation-trained visual policies for granular manipulation can be deployed zero-shot on real hardware when observations are represented as height maps rather than raw depth images.
A compact state representation paired with dense, goal-directed rewards makes RL tractable in a configuration space where random actions mostly increase distance to the goal, as shown by the NO-M ablation that avoids manipulation entirely without the movement reward.
The choice of RL algorithm matters: TQC converges and outperforms SAC, while TD3 fails to reach positive rewards, indicating that distributional critics help in this setting.
Because the policy outputs Cartesian end-effector increments rather than joint positions, the same trained policy is transferable to different robotic arms.
Coverage planning that changes every goal cell (B-CPP) is less accurate than the learned DELTA policy, implying that exhaustive coverage is less important than adapting to collapse dynamics.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Our inference: if the angle-of-repose model is the sole sand dynamics, the framework may transfer to other granular materials only insofar as their repose angle matches the training material; retraining or randomizing over repose angles would be a natural testable extension.
Our inference: the single qualitative real-robot rectangle is weak evidence for the zero-shot claim; a systematic real-world study with multiple goal shapes and repeated quantitative trials is the obvious next step.
Our inference: because the observation is a $32\times 32$ grid with $1\,\text{cm}$ cells and heights bounded by $20\,\text{cm}$, the method is resolution-limited; finer or larger-scale shaping would require tiling, higher-resolution grids, or hierarchical policies.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this is a well-executed simulation study with a real-world demo that is too thin for the claim attached to it. The reward design (delta vs progressive), the height-map gating encoder, and free-form end-effector control for arbitrary goal reliefs are genuinely new relative to the excavation and trenching prior work. The quantitative simulation results are credible: 100 evaluation episodes, standard deviations reported, Mann-Whitney U tests against two baselines plus ablations, and the ablations show that the design choices matter. So the core claim—that compact height-map observations plus a dense delta reward train a visual policy to shape granular media in simulation—holds up.\n\nThe soft spots are in proportion. First, zero-shot real-world transfer rests on one qualitative rectangle episode (Fig. 5b). No repeated trials, no quantitative height error, no comparison to baselines. That is not enough for the abstract's phrase 'including their real-world deployment' as a demonstrated result. It is a promising existence proof, not a measured result. Second, the evaluation metric d_hat is the same quantity the delta reward optimizes. That doesn't invalidate the comparison—the agent still has to learn actions that reduce the difference—but the metric is aligned with training by design; a second metric like volume error or shape overlap would strengthen the evaluation. Third, no code artifact or commit hash appears in the manuscript; the paper website is mentioned, but without code the reproducibility is limited. Fourth, the simulator is the Kim et al. angle-of-repose collapse heuristic. Training entirely in that sim and transferring zero-shot is plausible but unquantified; the model ignores compaction, moisture, and 3D flow around the cubic tool. A single demo cannot rule out that the policy overfits to simulator artifacts.\n\nNone of these are fatal to the central simulation contribution. The baselines are a bit weak—random actions and a coverage planner—but the coverage planner is a reasonable traditional approach for this task, and the ablations (gating mechanism, depth vs height map, TQC vs SAC vs TD3) are properly done. The citation pattern looks fine; the prior work is covered adequately.\n\nWho this is for: robotics researchers working on deformable or granular manipulation, and RL practitioners interested in observation and reward shaping for high-dimensional state spaces. I would bring it to a reading group and I would cite it for the reward formulation if I were working on sand shaping. This deserves a serious referee; I would accept it conditionally, asking for more real-world trials with quantitative metrics and a code release. The sim contribution is solid enough to justify the revision effort.

Referee Report

3 major / 5 minor

Summary. The paper proposes an RL framework for shaping granular media with a UR5e arm and a cubic end-effector, using a stereo camera and height-map representations. The agent is trained in a height-map simulator based on an angle-of-repose collapse model, with observations consisting of the difference between goal and current height maps, Boolean EE and goal masks, and EE proprioception. The reward combines a delta-shaped reduction of the mean absolute height difference with a goal-area movement term. Experiments compare the delta reward against a progressive reward and ablations, benchmark against random and coverage-path-planning baselines, evaluate a feature-extractor ablation, and compare TQC with SAC and TD3. The authors report statistically significant improvements in simulation and one qualitative zero-shot real-robot rectangle demo, and conclude that the approach enables effective training and real-world deployment.

Significance. If the simulation results are reproducible, the paper makes a useful contribution to RL-based granular media manipulation: the compact height-map observation with gated goal-area features and the dense delta reward with a goal-area movement term are shown to outperform two baselines and several ablations on a set of 400 goal maps, with Mann-Whitney U tests and reported standard deviations. The public code release and the inclusion of algorithm and feature-extractor ablations are strengths. The main caveats are that the headline zero-shot real-world claim rests on a single qualitative demo, and that the primary evaluation metric is the same quantity used in the dense reward, which limits the strength of the conclusions drawn from the simulation comparisons.

major comments (3)

[Sec. IV-C, Fig. 5b] The zero-shot real-world transfer claim, stated in the abstract and conclusion, rests on a single qualitative rectangle episode with no repeated trials, no quantitative height-map error, and no baseline comparison. Because the training simulator is a height-map angle-of-repose collapse heuristic (Sec. II-A) that does not model 3D flow around the 2x2x15 cm end-effector, compaction, or moisture effects, a single favorable rollout is insufficient to establish that the policy generalizes rather than succeeding on one convenient configuration. Please add quantitative real-robot evaluations with repeated trials across multiple shapes and report a measured height-error metric, or substantially weaken the abstract and conclusion claims to describe an illustrative transfer demonstration.
[Eq. (2) and Table III] The quantity d_hat defined in Eq. (2) is not explicitly restricted to the goal area, and the text says N_cell is the number of grid cells, while the evaluation metric in Table III is described as being computed within the goal area. Please clarify whether the reward and the evaluation metric use the same set of cells. If they differ, the reported comparison is not aligned with the training objective; if they are identical, the Height Diff. metric is exactly the quantity optimized by r_delta in Eq. (1), and the evaluation should be supplemented by an independent accuracy metric (e.g., full-height-map error, volume error, or a shape-distance measure) to avoid assessing the method primarily on its own reward signal.
[Sec. IV-C, Tabs. II and III] The main quantitative results in Table III do not state how many training seeds or policy instances they are based on; the text mentions three seeds only for the algorithm comparison in Fig. 6. If the 100 evaluation episodes come from a single training run, the reported standard deviations and Mann-Whitney tests characterize episode-level variability, not variability due to RL training, so the robustness claims about stable convergence are not supported for the main policy. Please report the number of seeds, provide per-seed results or a multi-seed aggregate, and state which specific policy instance was used for the real-robot demo.

minor comments (5)

[Abstract and Fig. 4] There are typographical errors: 'sucessfully' in the abstract and 'examplary' in the Fig. 4 caption should be 'successfully' and 'exemplary', respectively.
[Eq. (3)] The notation for the progressive reward is ambiguous: please define d_hat_closest and d_hat_o_furthest precisely and state whether the first term uses the goal-area distance or the global distance, since the text says 'closest and furthest reached distances inside the current episode' while the second term explicitly uses the outside-goal-area distance d_hat_o.
[Sec. II-A] Several occurrences of 'Kimet al.' lack a space before the citation, and the sentence 'Kimet al. [24] as well as Pavlov and Johnson [20] go one step further' would read more clearly if the specific contribution of each work were separated syntactically.
[Table III] The table would be clearer with explicit column-group headers distinguishing the H_R (reconstructed height map) and H_P (privileged height map) conditions; currently only the caption explains the split, and the rightmost columns are easy to misread as separate methods.
[Fig. 5b caption] The caption says 'the reconstructed 3D scene in simulation' for the real-robot row; please clarify whether this is a visualization generated from the real depth stream or a separate simulation render, since the text emphasizes zero-shot deployment.

Circularity Check

0 steps flagged · score 0.0 of 10

No meaningful circularity; the reward/evaluation overlap is a standard objective alignment, and the real-world transfer claim has a support gap but no circular derivation.

full rationale

The paper's contribution is an empirical RL system, not a derivation of a prediction from an input. The evaluation metric d_hat (Eq. 2) is indeed the same quantity whose temporal decrease forms the DELTA reward (Eq. 1), so the policy is optimized against the headline metric. This is a genuine alignment but not circularity: the agent must learn effective end-effector actions in a simulated collapse dynamics, the NO-M ablation shows that the reward does not trivially solve the task, and the PROG and baseline comparisons are all scored on the same d_hat, preserving fairness. No parameter is fitted and then renamed as a prediction. The zero-shot transfer claim (Sec. IV-C, Fig. 5b) is supported only by one qualitative rectangle episode without repeated trials or quantitative real-world height-map error; this is a limitation of evidence and a correctness risk, not a circularity. The simulator is based on an external cited model (Kim et al., angle-of-repose collapse); no load-bearing self-citation or imported uniqueness theorem appears, and the RePAIR dataset is used only as a source of goal shapes. The central comparison (DELTA vs. PROG, NO-M, B-CPP, RAND) stands as an independent empirical result.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central claim rests on standard RL assumptions, a simplified sand simulation, and hand-chosen reward weights. No new physical entities are postulated.

free parameters (4)

alpha_c = 5000
Scaling factor for the delta and progressive shaping rewards (Eq. 1, 3); hand-chosen, directly controls reward magnitude and affects training dynamics.
alpha_f = 1000
Penalty scale for moving outside the goal area in the PROG reward (Eq. 3); hand-chosen.
alpha_m = 10
Steepness of the goal-area distance penalty (Eq. 4); hand-chosen.
episode_length N_ep = 40
Number of steps per episode; limits how much shaping can be achieved and affects the target shapes chosen.

assumptions (3)

domain assumption Kim et al. angle-of-repose height-map model reproduces the essential dynamics of real sand for this task.
Sec. II-A bases the simulation on [24]; the zero-shot transfer claim in Sec. IV-C depends on this model.
domain assumption The reconstructed height map from depth images is an adequate state representation despite occlusions.
Sec. III-C and the visual policy experiments assume occlusion-free or recoverable elevation information from depth images.
standard math The Markov property holds for the height-map transition under the agent's actions.
The MDP formulation in Sec. III-C treats the height map plus EE positions as Markov state.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Interactive Shaping of Granular Media Using Reinforcement Learning." pith.science (2026). https://pith.science/paper/RXSEISP5

@misc{pith2026250906469,
  author       = {Pith},
  title        = {Pith review of: Interactive Shaping of Granular Media Using Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RXSEISP5}},
  note         = {Machine review of arXiv:2509.06469}
}

read the original abstract

Autonomous manipulation of granular media, such as sand, is crucial for applications in construction, excavation, and additive manufacturing. However, shaping granular materials presents unique challenges due to their high-dimensional configuration space and complex dynamics, where traditional rule-based approaches struggle without extensive engineering efforts. Reinforcement learning (RL) offers a promising alternative by enabling agents to learn adaptive manipulation strategies through trial and error. In this work, we present an RL framework that enables a robotic arm with a cubic end-effector and a stereo camera to shape granular media into desired target structures. We show the importance of compact observations and concise reward formulations for the large configuration space, validating our design choices with an ablation study. Our results demonstrate the effectiveness of the proposed approach for the training of visual policies that manipulate granular media including their real-world deployment, significantly outperforming two baseline approaches in terms of target shape accuracy.

Figures

Figures reproduced from arXiv: 2509.06469 by the authors.

**Figure 2.** Overview of our approach: We employ a training process to enable agents to manipulate granular media using sensory inputs. We train a visual [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** We employ the distance dm of the EE position pEE to the closest point belonging to the goal area pg to guide the agent towards regions that require manipulation through a dense reward signal (see Sec. III-D). space as the height map elevation data. To achieve that, we first stack the difference height map with the EE mask. Then we extract feature vectors of size 64 from both, the stacked observation and the goal mas… view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: From left to right, we show the reconstructed 3D scene in simulation, [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]

**Figure 5.** Figure 5: Manipulation of the granular medium by a visual, goal-conditioned RL agent in simulation (top) and on the real robotic system (bottom). [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]

**Figure 6.** Figure 6: Using three different seeds, the RL training with TQC [28] results [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

41 extracted references · 41 canonical work pages

[1]

Learning robotic manipulation of granular media,

C. Schenck, J. Tompson, S. Levine, and D. Fox, “Learning robotic manipulation of granular media,” inProc. of Conf. on Robot Learning (CoRL), 2017

work page 2017
[2]

Learning excavation of rigid objects with offline reinforcement learning,

S. Jin, Z. Ye, and L. Zhang, “Learning excavation of rigid objects with offline reinforcement learning,”arXiv preprint, 2023

work page 2023
[3]

Few-shot adaptation for manipulating granular materials under domain shift,

Y . Zhu, P. Thangeda, M. Ornik, and K. Hauser, “Few-shot adaptation for manipulating granular materials under domain shift,” inProc. of Robotics: Science and Systems (RSS), 2023

work page 2023
[4]

Robotic embankment: Free-form autonomous formation in terrain with heap,

D. Jud, I. Hurkxkens, C. Girot, and M. Hutter, “Robotic embankment: Free-form autonomous formation in terrain with heap,”Construction Robotics, vol. 5, no. 2, 2021

work page 2021
[5]

Soil- adaptive excavation using reinforcement learning,

P. Egli, D. Gaschen, S. Kerscher, D. Jud, and M. Hutter, “Soil- adaptive excavation using reinforcement learning,”IEEE Robotics and Automation Letters (RA-L), vol. 7, no. 4, 2022

work page 2022
[6]

Deep reinforcement learning with adversarial training for automated excavation using depth images,

T. Osa and M. Aizawa, “Deep reinforcement learning with adversarial training for automated excavation using depth images,”IEEE Access, vol. 10, 2022

work page 2022
[7]

Au- tomated excavator based on reinforcement learning and multibody system dynamics,

I. Kurinov, G. Orzechowski, P. Hamalainen, and A. Mikkola, “Au- tomated excavator based on reinforcement learning and multibody system dynamics,”IEEE Access, vol. 8, 2020

work page 2020
[8]

Excavation reinforcement learning using geometric representation,

Q. Lu, Y . Zhu, and L. Zhang, “Excavation reinforcement learning using geometric representation,”IEEE Robotics and Automation Letters (RA- L), vol. 7, no. 2, 2022

work page 2022

Show all 41 references

[9]

Model- free vision-based shaping of deformable plastic materials,

A. Cherubini, V . Ortenzi, A. Cosgun, R. Lee, and P. Corke, “Model- free vision-based shaping of deformable plastic materials,”Intl. Jour- nal of Robotics Research (IJRR), vol. 39, no. 14, 2020

2020
[10]

Sim-to-real reinforcement learning for deformable object manipulation,

J. Matas, S. James, and A. J. Davison, “Sim-to-real reinforcement learning for deformable object manipulation,” inProc. of Conf. on Robot Learning (CoRL), 2018

2018
[11]

A system for general in-hand object re-orientation,

T. Chen, J. Xu, and P. Agrawal, “A system for general in-hand object re-orientation,” inProc. of Conf. on Robot Learning (CoRL), 2022

2022
[12]

Deep reinforcement learning for active flow control in a turbulent separation bubble,

B. Font, F. Alcántara-Ávila, J. Rabault, R. Vinuesa, and O. Lehmkuhl, “Deep reinforcement learning for active flow control in a turbulent separation bubble,”Nature Communications, vol. 16, no. 1, 2025

2025
[13]

A data-driven approach for fast simulation of robot locomotion on granular media,

Y . Zhu, L. Abdulmajeid, and K. Hauser, “A data-driven approach for fast simulation of robot locomotion on granular media,” inProc. of the IEEE Intl. Conf. on Robotics & Automation (ICRA), 2019

2019
[14]

Real-time remodeling of granular terrain for robot locomotion,

A. Karsai, D. Kerimoglu, D. Soto, S. Ha, T. Zhang, and D. I. Goldman, “Real-time remodeling of granular terrain for robot locomotion,” Advanced Intelligent Systems, vol. 4, no. 12, 2022

2022
[15]

Learning manipulation of steep granular slopes for fast mini rover turning,

D. Kerimoglu, D. Soto, M. L. Hemsley, J. Brunner, S. Ha, T. Zhang, and D. I. Goldman, “Learning manipulation of steep granular slopes for fast mini rover turning,” inProc. of the IEEE Intl. Conf. on Robotics & Automation (ICRA), 2024

2024
[16]

Tactile-based object retrieval from granular media,

J. Xu, Y . Jia, D. Yang, P. Meng, X. Zhu, Z. Guo, S. Song, and M. Ciocarlie, “Tactile-based object retrieval from granular media,” arXiv preprint, 2024

2024
[17]

Towards autonomous grading in the real world,

Y . Miron, C. Ross, Y . Goldfracht, C. Tessler, and D. Di Castro, “Towards autonomous grading in the real world,” inProc. of the IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems (IROS), 2022

2022
[18]

Material-agnostic shaping of granular materials with optimal transport,

N. Alatur, O. Andersson, R. Siegwart, and L. Ott, “Material-agnostic shaping of granular materials with optimal transport,” inProc. of the IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems (IROS), 2023

2023
[19]

Neural field dynamics model for granular object piles manipulation,

S. Xue, S. Cheng, P. Kachana, and D. Xu, “Neural field dynamics model for granular object piles manipulation,” inProc. of Conf. on Robot Learning (CoRL), 2023

2023
[20]

Soil displacement terramechanics for wheel-based trenching with a planetary rover,

C. Pavlov and A. M. Johnson, “Soil displacement terramechanics for wheel-based trenching with a planetary rover,” inProc. of the IEEE Intl. Conf. on Robotics & Automation (ICRA), 2019

2019
[21]

Localized graph-based neural dynamics models for terrain manipulation,

C. Liu, Y . Li, and K. Hauser, “Localized graph-based neural dynamics models for terrain manipulation,”arXiv preprint, 2025

2025
[22]

Robotic landscapes: Designing formation processes for large scale autonomous earth moving,

I. Hurkxkens, A. Mirjan, F. Gramazio, M. Kohler, and C. Girot, “Robotic landscapes: Designing formation processes for large scale autonomous earth moving,” inImpact: Design With All Senses, 2020

2020
[23]

Manipulation of granular materials by learning particle interactions,

N. Tuomainen, D. Blanco-Mulero, and V . Kyrki, “Manipulation of granular materials by learning particle interactions,”IEEE Robotics and Automation Letters (RA-L), vol. 7, no. 2, 2022

2022
[24]

Developing a simple model for sand-tool interaction and autonomously shaping sand,

W. Kim, C. Pavlov, and A. M. Johnson, “Developing a simple model for sand-tool interaction and autonomously shaping sand,”arXiv preprint, 2019

2019
[25]

Heap - the autonomous walking excava- tor,

D. Jud, S. Kerscher, M. Wermelinger, E. Jelavic, P. Egli, P. Leemann, G. Hottiger, and M. Hutter, “Heap - the autonomous walking excava- tor,”Automation in Construction, vol. 129, 2021

2021
[26]

Toward autonomous excavation planning,

L. Terenzi and M. Hutter, “Toward autonomous excavation planning,” IEEE Transactions on Field Robotics (T-FR), vol. 1, 2024

2024
[27]

Learning granular media avalanche behavior for indirectly manipulating obstacles on a granular slope,

H. Hu, F. Qian, and D. Seita, “Learning granular media avalanche behavior for indirectly manipulating obstacles on a granular slope,” in Proc. of Conf. on Robot Learning (CoRL), 2025

2025
[28]

Con- trolling overestimation bias with truncated mixture of continuous distributional quantile critics,

A. Kuznetsov, P. Shvechikov, A. Grishin, and D. P. Vetrov, “Con- trolling overestimation bias with truncated mixture of continuous distributional quantile critics,” inProc. of the Intl. Conf. on Machine Learning, 2020

2020
[29]

A unified approach for motion and force control of robot manipulators: The operational space formulation,

O. Khatib, “A unified approach for motion and force control of robot manipulators: The operational space formulation,”IEEE Journal of Robotics and Automation, vol. 3, no. 1, 1987

1987
[30]

Dexpbt: Scaling up dexterous manipulation for hand-arm systems with population based training,

A. Petrenko, A. Allshire, G. State, A. Handa, and V . Makoviychuk, “Dexpbt: Scaling up dexterous manipulation for hand-arm systems with population based training,” inProc. of Robotics: Science and Systems (RSS), 2023

2023
[31]

Coverage of Known Spaces: The Boustrophedon Cellular Decomposition,

H. Choset, “Coverage of Known Spaces: The Boustrophedon Cellular Decomposition,”Autonomous Robots, vol. 9, no. 3, 2000

2000
[32]

Approximate algorithms for the traveling salesperson problem,

D. J. Rosenkrantz, R. E. Stearns, and P. M. Lewis, “Approximate algorithms for the traveling salesperson problem,” inProc. of the Annual Symposion on Switching and Automata Theory (SWAT), 1974

1974
[33]

Stable-baselines3: Reliable reinforcement learning implemen- tations,

A. Raffin, A. Hill, A. Gleave, A. Kanervisto, M. Ernestus, and N. Dor- mann, “Stable-baselines3: Reliable reinforcement learning implemen- tations,”Journal of Machine Learning Research (JMLR), vol. 22, no. 268, 2021

2021
[34]

Openai gym,

G. Brockman, V . Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba, “Openai gym,”arXiv preprint, 2016

2016
[35]

robosuite: A modular sim- ulation framework and benchmark for robot learning,

Y . Zhu, J. Wong, A. Mandlekar, R. Martín-Martín, A. Joshi, K. Lin, A. Maddukuri, S. Nasiriany, and Y . Zhu, “robosuite: A modular sim- ulation framework and benchmark for robot learning,”arXiv preprint, 2020

2020
[36]

Mujoco: A physics engine for model-based control,

E. Todorov, T. Erez, and Y . Tassa, “Mujoco: A physics engine for model-based control,” inProc. of the IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems (IROS), 2012

2012
[37]

Not your grandmother’s toolbox - the robotics toolbox reinvented for python,

P. Corke and J. Haviland, “Not your grandmother’s toolbox - the robotics toolbox reinvented for python,” inProc. of the IEEE Intl. Conf. on Robotics & Automation (ICRA), 2021

2021
[38]

Re-assembling the past: The repair dataset and benchmark for real world 2d and 3d puzzle solving,

T. Tsesmeliset al., “Re-assembling the past: The repair dataset and benchmark for real world 2d and 3d puzzle solving,” inProc. of the Conf. on Neural Information Processing Systems (NIPS), 2025

2025
[39]

On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other,

H. B. Mann and D. R. Whitney, “On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other,”The Annals of Mathematical Statistics, vol. 18, no. 1, 1947

1947
[40]

Soft actor-critic: Off- policy maximum entropy deep reinforcement learning with a stochastic actor,

T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, “Soft actor-critic: Off- policy maximum entropy deep reinforcement learning with a stochastic actor,” inProc. of the Intl. Conf. on Machine Learning, 2018

2018
[41]

Addressing function approxi- mation error in actor-critic methods,

S. Fujimoto, H. Hoof, and D. Meger, “Addressing function approxi- mation error in actor-critic methods,” inProc. of the Intl. Conf. on Machine Learning, 2018

2018

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

Learning robotic manipulation of granular media,

C. Schenck, J. Tompson, S. Levine, and D. Fox, “Learning robotic manipulation of granular media,” inProc. of Conf. on Robot Learning (CoRL), 2017

work page 2017

[2] [2]

Learning excavation of rigid objects with offline reinforcement learning,

S. Jin, Z. Ye, and L. Zhang, “Learning excavation of rigid objects with offline reinforcement learning,”arXiv preprint, 2023

work page 2023

[3] [3]

Few-shot adaptation for manipulating granular materials under domain shift,

Y . Zhu, P. Thangeda, M. Ornik, and K. Hauser, “Few-shot adaptation for manipulating granular materials under domain shift,” inProc. of Robotics: Science and Systems (RSS), 2023

work page 2023

[4] [4]

Robotic embankment: Free-form autonomous formation in terrain with heap,

D. Jud, I. Hurkxkens, C. Girot, and M. Hutter, “Robotic embankment: Free-form autonomous formation in terrain with heap,”Construction Robotics, vol. 5, no. 2, 2021

work page 2021

[5] [5]

Soil- adaptive excavation using reinforcement learning,

P. Egli, D. Gaschen, S. Kerscher, D. Jud, and M. Hutter, “Soil- adaptive excavation using reinforcement learning,”IEEE Robotics and Automation Letters (RA-L), vol. 7, no. 4, 2022

work page 2022

[6] [6]

Deep reinforcement learning with adversarial training for automated excavation using depth images,

T. Osa and M. Aizawa, “Deep reinforcement learning with adversarial training for automated excavation using depth images,”IEEE Access, vol. 10, 2022

work page 2022

[7] [7]

Au- tomated excavator based on reinforcement learning and multibody system dynamics,

I. Kurinov, G. Orzechowski, P. Hamalainen, and A. Mikkola, “Au- tomated excavator based on reinforcement learning and multibody system dynamics,”IEEE Access, vol. 8, 2020

work page 2020

[8] [8]

Excavation reinforcement learning using geometric representation,

Q. Lu, Y . Zhu, and L. Zhang, “Excavation reinforcement learning using geometric representation,”IEEE Robotics and Automation Letters (RA- L), vol. 7, no. 2, 2022

work page 2022