{"id":"830078ad-359c-4c64-a356-452efd151d47","arxiv_id":"2509.06469","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A visual reinforcement-learning policy with a compact height-map observation and a dense 'delta' reward shapes granular media into target reliefs, outperforming heuristic baselines and transferring to a real robot in a single demonstration.","lead":"This paper trains a robotic arm with reinforcement learning to sculpt sand into target shapes, using a compact height-map view and a dense progress reward. The method beats two simple baselines in simulation and is shown on a real robot in one demonstration.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Zero-shot real-world transfer rests on a single qualitative rectangle demo; the angle-of-repose sim-to-real gap is untested quantitatively.","rationale":"The reader's weakest_assumption correctly identifies the simulator fidelity and the thin real-world validation. This is the most load-bearing concern because the paper's distinct claim beyond simulated RL is zero-shot sim-to-real transfer (Sec. IV-C 'Real World Transfer'). The simulated comparisons (DELTA vs. PROG, B-CPP, RAND) are statistically supported with 100 episodes and appropriate tests, so the simulated performance claim is not in question. However, the real-world claim rests on a single qualitative episode, and the angle-of-repose model is a simplified surface-collapse heuristic that may not capture the physics of a cubic tool dragged through sand. If the real-world transfer does not hold up quantitatively, the paper's contribution narrows to a simulation-only result, which is weaker than claimed. A multi-trial quantitative real-world evaluation would settle this directly. Since the concern is exactly the reader's weakest assumption and the requested additional evidence is feasible, the conditional verdict remains appropriate without change.","tokens_in":11302,"tokens_out":3220,"duration_ms":30055,"concrete_test":"Run at least 10 (ideally 20) real-world evaluation episodes with the DELTA policy, using randomly placed rectangle goals and at least one L-shape. Record depth images before and after each episode, reconstruct height maps with the same pipeline, and compute the Height Diff. d̂ within the goal area using Eq. (2). Compare the resulting distribution to the simulated 3.4±1.1 mm. Also run the same trials with RAND or B-CPP as a control. If the real-world mean d̂ is within roughly 2 mm of the simulated mean and significantly better than RAND (e.g., Mann-Whitney U, p<0.05), the transfer claim is supported; otherwise it fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's headline contribution includes 'real-world deployment' (Abstract, Sec. V), but the only evidence is one qualitative rectangle episode (Sec. IV-C, Fig. 5b) with no repeated trials, no quantitative height-map error, and no comparison to baselines. The simulation (Sec. II-A) is a height-map angle-of-repose collapse heuristic (Kim et al.) that does not model bulk tool-sand interaction forces, compaction, or 3D flow around the 2x2x15 cm cubic end-effector. Since the policy is trained entirely in this simulator and observes reconstructed height maps, any systematic mismatch between the simulator's collapse dynamics and real sand (e.g., moisture, cohesion, drag) could invalidate the learned actions. A single favorable demo cannot rule out that the policy overfits to simulator artifacts or that the shown trial was cherry-picked. Thus, the strongest claim's real-world component is unsupported by the provided evidence.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an RL framework for shaping granular media with a UR5e arm and a cubic end-effector, using a stereo camera and height-map representations. The agent is trained in a height-map simulator based on an angle-of-repose collapse model, with observations consisting of the difference between goal and current height maps, Boolean EE and goal masks, and EE proprioception. The reward combines a delta-shaped reduction of the mean absolute height difference with a goal-area movement term. Experiments compare the delta reward against a progressive reward and ablations, benchmark against random and coverage-path-planning baselines, evaluate a feature-extractor ablation, and compare TQC with SAC and TD3. The authors report statistically significant improvements in simulation and one qualitative zero-shot real-robot rectangle demo, and conclude that the approach enables effective training and real-world deployment.","tokens_in":11539,"tokens_out":6467,"duration_ms":63243,"significance":"If the simulation results are reproducible, the paper makes a useful contribution to RL-based granular media manipulation: the compact height-map observation with gated goal-area features and the dense delta reward with a goal-area movement term are shown to outperform two baselines and several ablations on a set of 400 goal maps, with Mann-Whitney U tests and reported standard deviations. The public code release and the inclusion of algorithm and feature-extractor ablations are strengths. The main caveats are that the headline zero-shot real-world claim rests on a single qualitative demo, and that the primary evaluation metric is the same quantity used in the dense reward, which limits the strength of the conclusions drawn from the simulation comparisons.","major_comments":[{"comment":"The zero-shot real-world transfer claim, stated in the abstract and conclusion, rests on a single qualitative rectangle episode with no repeated trials, no quantitative height-map error, and no baseline comparison. Because the training simulator is a height-map angle-of-repose collapse heuristic (Sec. II-A) that does not model 3D flow around the 2x2x15 cm end-effector, compaction, or moisture effects, a single favorable rollout is insufficient to establish that the policy generalizes rather than succeeding on one convenient configuration. Please add quantitative real-robot evaluations with repeated trials across multiple shapes and report a measured height-error metric, or substantially weaken the abstract and conclusion claims to describe an illustrative transfer demonstration.","section":"Sec. IV-C, Fig. 5b"},{"comment":"The quantity d_hat defined in Eq. (2) is not explicitly restricted to the goal area, and the text says N_cell is the number of grid cells, while the evaluation metric in Table III is described as being computed within the goal area. Please clarify whether the reward and the evaluation metric use the same set of cells. If they differ, the reported comparison is not aligned with the training objective; if they are identical, the Height Diff. metric is exactly the quantity optimized by r_delta in Eq. (1), and the evaluation should be supplemented by an independent accuracy metric (e.g., full-height-map error, volume error, or a shape-distance measure) to avoid assessing the method primarily on its own reward signal.","section":"Eq. (2) and Table III"},{"comment":"The main quantitative results in Table III do not state how many training seeds or policy instances they are based on; the text mentions three seeds only for the algorithm comparison in Fig. 6. If the 100 evaluation episodes come from a single training run, the reported standard deviations and Mann-Whitney tests characterize episode-level variability, not variability due to RL training, so the robustness claims about stable convergence are not supported for the main policy. Please report the number of seeds, provide per-seed results or a multi-seed aggregate, and state which specific policy instance was used for the real-robot demo.","section":"Sec. IV-C, Tabs. II and III"}],"minor_comments":[{"comment":"There are typographical errors: 'sucessfully' in the abstract and 'examplary' in the Fig. 4 caption should be 'successfully' and 'exemplary', respectively.","section":"Abstract and Fig. 4"},{"comment":"The notation for the progressive reward is ambiguous: please define d_hat_closest and d_hat_o_furthest precisely and state whether the first term uses the goal-area distance or the global distance, since the text says 'closest and furthest reached distances inside the current episode' while the second term explicitly uses the outside-goal-area distance d_hat_o.","section":"Eq. (3)"},{"comment":"Several occurrences of 'Kimet al.' lack a space before the citation, and the sentence 'Kimet al. [24] as well as Pavlov and Johnson [20] go one step further' would read more clearly if the specific contribution of each work were separated syntactically.","section":"Sec. II-A"},{"comment":"The table would be clearer with explicit column-group headers distinguishing the H_R (reconstructed height map) and H_P (privileged height map) conditions; currently only the caption explains the split, and the rightmost columns are easy to misread as separate methods.","section":"Table III"},{"comment":"The caption says 'the reconstructed 3D scene in simulation' for the real-robot row; please clarify whether this is a visualization generated from the real depth stream or a separate simulation render, since the text emphasizes zero-shot deployment.","section":"Fig. 5b caption"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the journal's scope and the simulation study is internally consistent, but the real-robot transfer is a listed contribution and is currently supported only by one qualitative demo. I recommend requiring either a small quantitative real-robot evaluation or a clear reduction of the transfer claim in the abstract and conclusion. I also suggest asking the authors to state the seed protocol for the main results, since Table III does not indicate whether the reported means and standard deviations come from one or multiple training runs."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a well-executed simulation study with a real-world demo that is too thin for the claim attached to it. The reward design (delta vs progressive), the height-map gating encoder, and free-form end-effector control for arbitrary goal reliefs are genuinely new relative to the excavation and trenching prior work. The quantitative simulation results are credible: 100 evaluation episodes, standard deviations reported, Mann-Whitney U tests against two baselines plus ablations, and the ablations show that the design choices matter. So the core claim—that compact height-map observations plus a dense delta reward train a visual policy to shape granular media in simulation—holds up.\\n\\nThe soft spots are in proportion. First, zero-shot real-world transfer rests on one qualitative rectangle episode (Fig. 5b). No repeated trials, no quantitative height error, no comparison to baselines. That is not enough for the abstract's phrase 'including their real-world deployment' as a demonstrated result. It is a promising existence proof, not a measured result. Second, the evaluation metric d_hat is the same quantity the delta reward optimizes. That doesn't invalidate the comparison—the agent still has to learn actions that reduce the difference—but the metric is aligned with training by design; a second metric like volume error or shape overlap would strengthen the evaluation. Third, no code artifact or commit hash appears in the manuscript; the paper website is mentioned, but without code the reproducibility is limited. Fourth, the simulator is the Kim et al. angle-of-repose collapse heuristic. Training entirely in that sim and transferring zero-shot is plausible but unquantified; the model ignores compaction, moisture, and 3D flow around the cubic tool. A single demo cannot rule out that the policy overfits to simulator artifacts.\\n\\nNone of these are fatal to the central simulation contribution. The baselines are a bit weak—random actions and a coverage planner—but the coverage planner is a reasonable traditional approach for this task, and the ablations (gating mechanism, depth vs height map, TQC vs SAC vs TD3) are properly done. The citation pattern looks fine; the prior work is covered adequately.\\n\\nWho this is for: robotics researchers working on deformable or granular manipulation, and RL practitioners interested in observation and reward shaping for high-dimensional state spaces. I would bring it to a reading group and I would cite it for the reward formulation if I were working on sand shaping. This deserves a serious referee; I would accept it conditionally, asking for more real-world trials with quantitative metrics and a code release. The sim contribution is solid enough to justify the revision effort.","headline":"Solid sim evaluation of reward and observation design for sand shaping; the real-world transfer claim outruns the evidence.","tokens_in":11990,"tokens_out":1699,"would_cite":true,"duration_ms":15854,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A simulation-trained reinforcement learning policy carves sand into target shapes with 3.4 mm mean error, and transfers zero-shot to a real robot.","keywords":["reinforcement learning","granular media manipulation","height map representation","sim-to-real transfer","reward shaping","Truncated Quantile Critics","robotic shaping","sand manipulation"],"falsifier":"Run the trained DELTA policy on the real robot for repeated episodes (e.g., 30 trials) across several goal shapes and measure the mean absolute cell height difference in the goal area; if the real-world mean error exceeds roughly twice the simulated $3.4\\,\\text{mm}$, or if non-rectangle shapes are unrecognizable, the zero-shot sim-to-real claim is falsified. Alternatively, compare the simulated angle-of-repose collapse against real sand for the same end-effector trajectory; a systematic mismatch in how far sand redistributes would falsify the simulator premise.","tokens_in":11072,"feed_emoji":"🏗️","tokens_out":8360,"duration_ms":63068,"temperature":0.7,"pith_summary":"The paper claims that a reinforcement-learned policy can shape granular media such as sand into arbitrary target structures with millimeter-level accuracy, provided the task is represented compactly. The authors show that a difference height map between the current and goal sand surface, combined with binary masks for the end-effector and goal area and a two-part dense reward, lets a visual policy trained entirely in simulation learn effective shaping behaviors. Their best policy, trained with the 'delta' reward and the Truncated Quantile Critics algorithm, reaches a mean absolute cell height difference of $3.4\\,\\text{mm}$ over 100 evaluation episodes, outperforming a progressive-reward policy, a coverage-planning baseline, and random actions. The same policy, deployed zero-shot on a real robot with a stereo camera, forms a rectangle in a real sandbox, indicating that the height-map representation transfers across the sim-to-real gap.","feed_headline":"RL carves sand to 3.4 mm accuracy, transfers to a real robot","feed_subtitle":"Compact height-map observations and dense rewards let a simulation-trained visual policy shape sand zero-shot.","key_machinery":"The load-bearing mechanism is the pairing of a compact observation encoder with a dense two-part reward. The observation stack combines the difference height map $H_d = H_g - H_c$ (with current heights truncated at the initial flat level) with a binary end-effector mask; a sigmoid-gated goal mask modulates the CNN-extracted features to focus on the goal area, yielding a 64-dimensional feature vector that is concatenated with the end-effector's position observations. The shaping reward $r_{\\text{delta}} = \\alpha_c(\\hat{d}_{t-1} - \\hat{d}_t)$ rewards reductions in mean absolute cell height difference, while the movement reward $r_m = -\\tanh(\\alpha_m d_m) + \\mathbb{1}_{\\text{reached}}$ pulls the end-effector toward the goal area; an alternative 'progressive' variant rewards progress relative to the best configuration achieved in the episode. Training runs on a height-map sand simulation built on an angle-of-repose collapse model, where excess sand is redistributed between adjacent cells until the local slope condition is met, which keeps the simulation fast enough for online RL.","core_discovery":"On its own terms, the paper establishes that interactive shaping of granular media can be formulated as a goal-conditioned reinforcement learning problem solvable from visual observations. The central claim is that a compact observation—the per-cell difference between goal and current truncated height maps, the end-effector's projected footprint mask, and a goal-area mask—together with a dense reward that rewards reductions in mean absolute height difference $\\hat{d}$ and steers the end-effector toward the goal area, enables a Truncated Quantile Critics policy to carve shapes despite collapsing sand dynamics. In 100 simulated evaluation episodes, the DELTA policy achieves a mean error of $3.4\\,\\text{mm}$ within the goal area, significantly lower than the progressive-reward policy ($4.5\\,\\text{mm}$), a Boustrophedon coverage planner ($4.8\\,\\text{mm}$), and random actions ($7.2\\,\\text{mm}$). The authors further claim zero-shot transfer to a real robot: the simulation-trained visual policy, observing only reconstructed depth-based height maps, produces a rectangle in a real sandbox without additional training.","pith_inferences":["Our inference: if the angle-of-repose model is the sole sand dynamics, the framework may transfer to other granular materials only insofar as their repose angle matches the training material; retraining or randomizing over repose angles would be a natural testable extension.","Our inference: the single qualitative real-robot rectangle is weak evidence for the zero-shot claim; a systematic real-world study with multiple goal shapes and repeated quantitative trials is the obvious next step.","Our inference: because the observation is a $32\\times 32$ grid with $1\\,\\text{cm}$ cells and heights bounded by $20\\,\\text{cm}$, the method is resolution-limited; finer or larger-scale shaping would require tiling, higher-resolution grids, or hierarchical policies."],"forward_implications":["Simulation-trained visual policies for granular manipulation can be deployed zero-shot on real hardware when observations are represented as height maps rather than raw depth images.","A compact state representation paired with dense, goal-directed rewards makes RL tractable in a configuration space where random actions mostly increase distance to the goal, as shown by the NO-M ablation that avoids manipulation entirely without the movement reward.","The choice of RL algorithm matters: TQC converges and outperforms SAC, while TD3 fails to reach positive rewards, indicating that distributional critics help in this setting.","Because the policy outputs Cartesian end-effector increments rather than joint positions, the same trained policy is transferable to different robotic arms.","Coverage planning that changes every goal cell (B-CPP) is less accurate than the learned DELTA policy, implying that exhaustive coverage is less important than adapting to collapse dynamics."],"supporting_citations":[{"why":"Supplies the angle-of-repose height-map sand model that underpins the simulation environment.","marker":"[24]"},{"why":"Truncated Quantile Critics, the RL algorithm that the policy is trained with.","marker":"[28]"},{"why":"Source of the progressive reward-shaping idea adapted into one of the tested reward formulations.","marker":"[30]"},{"why":"Basis for the Boustrophedon coverage path planning baseline compared against.","marker":"[26]"},{"why":"Boustrophedon cellular decomposition used by the coverage baseline.","marker":"[31]"},{"why":"RL algorithm implementations used for training in the experimental setup.","marker":"[33]"},{"why":"Robosuite framework based on MuJoCo used for the simulation and training.","marker":"[35]"},{"why":"MuJoCo physics engine that simulates the robot and sand interaction.","marker":"[36]"}],"fun_headline_variants":["RL carves sand to 3.4 mm, transfers zero-shot to real robot","Sand shaping with RL: 3.4 mm accuracy, real-world transfer","RL agent shapes granular media to 3.4 mm on real robot","Zero-shot sand sculpting by RL hits 3.4 mm accuracy","Reinforcement learning carves sand: 3.4 mm, real robot deployment"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that the angle-of-repose height-map simulation is faithful enough that a policy trained only in simulation, on reconstructed depth-based height maps, generalizes zero-shot to real sand; the supporting real-world evidence is a single qualitative rectangle demo with no repeated trials or quantitative accuracy measurement.","fun_headline_variants_meta":{"raw":{"variants":["RL carves sand to 3.4 mm, transfers zero-shot to real robot","Sand shaping with RL: 3.4 mm accuracy, real-world transfer","RL agent shapes granular media to 3.4 mm on real robot","Zero-shot sand sculpting by RL hits 3.4 mm accuracy","Reinforcement learning carves sand: 3.4 mm, real robot deployment"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000184,"raw_usage":{"total_tokens":1302,"prompt_tokens":912,"completion_tokens":390,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":528,"completion_tokens_details":{"reasoning_tokens":287}},"tokens_in":528,"tokens_out":390,"duration_ms":3923,"temperature":1.0,"reasoning_tokens":287,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T16:15:24.017383+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the trained DELTA policy on the real robot for repeated episodes (e.g., 30 trials) across several goal shapes and measure the mean absolute cell height difference in the goal area; if the real-world mean error exceeds roughly twice the simulated $3.4\\,\\text{mm}$, or if non-rectangle shapes are unrecognizable, the zero-shot sim-to-real claim is falsified. Alternatively, compare the simulated angle-of-repose collapse against real sand for the same end-effector trajectory; a systematic mismatch in how far sand redistributes would falsify the simulator premise.","supporting_citations":[{"cited_title":"Mujoco: A physics engine for model-based control,","cited_arxiv_id":null,"evidence_quote":"MuJoCo physics engine that simulates the robot and sand interaction."},{"cited_title":"robosuite: A modular sim- ulation framework and benchmark for robot learning,","cited_arxiv_id":null,"evidence_quote":"Robosuite framework based on MuJoCo used for the simulation and training."},{"cited_title":"Developing a simple model for sand-tool interaction and autonomously shaping sand,","cited_arxiv_id":null,"evidence_quote":"Supplies the angle-of-repose height-map sand model that underpins the simulation environment."},{"cited_title":"Con- trolling overestimation bias with truncated mixture of continuous distributional quantile critics,","cited_arxiv_id":null,"evidence_quote":"Truncated Quantile Critics, the RL algorithm that the policy is trained with."},{"cited_title":"Dexpbt: Scaling up dexterous manipulation for hand-arm systems with population based training,","cited_arxiv_id":null,"evidence_quote":"Source of the progressive reward-shaping idea adapted into one of the tested reward formulations."},{"cited_title":"Toward autonomous excavation planning,","cited_arxiv_id":null,"evidence_quote":"Basis for the Boustrophedon coverage path planning baseline compared against."},{"cited_title":"Coverage of Known Spaces: The Boustrophedon Cellular Decomposition,","cited_arxiv_id":null,"evidence_quote":"Boustrophedon cellular decomposition used by the coverage baseline."},{"cited_title":"Stable-baselines3: Reliable reinforcement learning implemen- tations,","cited_arxiv_id":null,"evidence_quote":"RL algorithm implementations used for training in the experimental setup."}],"review_version":1}