{"id":"719ba7e4-61a9-4d35-8a5d-fcc45ac90405","arxiv_id":"2502.05595","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A model-based reinforcement learning framework, MC-PILOT, learns accurate pick-and-throw policies for a Franka Emika Panda robot from a few dozen throws by combining Gaussian process dynamics with explicit release-delay estimation.","lead":"The authors extend a model-based reinforcement learning method, MC-PILOT, to teach a Franka Panda robot to throw objects into targets using only a handful of practice throws. The system learns a Gaussian process model of the object's flight, estimates the gripper's release delay, and optimizes a throwing policy on that model, reaching near-perfect accuracy in tests.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The data-efficiency claim is supported only against a supervised regression baseline, not a Model-Free RL algorithm, so the paper's central comparison is not yet established.","rationale":"The reader's weakest assumption (the instantaneous-release delay model) is real, but it is not the most load-bearing part of the central claim. The paper's own simulation shows the estimated delay differs from the true U(0.01,0.02) while MC-PILOT still reaches near-100% accuracy, suggesting the fitted (a,b) act as calibration terms rather than physical delays. That weakens the 'delay estimation' subclaim, but not the demonstrated throwing accuracy. The comparison baseline is more central because the title and Section 7 make a comparative claim about data efficiency versus Model-Free RL, and the only model-free baseline in the paper is a supervised feedforward regressor, not an RL algorithm. A true MFRL comparison could change the headline conclusion about data efficiency. I therefore keep the reader's CONDITIONAL verdict but shift the focus to the missing MFRL baseline; if the authors supply that comparison, the verdict should be revisited accordingly.","tokens_in":25808,"tokens_out":12934,"duration_ms":138940,"concrete_test":"In the same Gazebo setup as Section 6.2, train a genuine MFRL algorithm (e.g., Tossingbot-style policy or DDPG/PPO with the same RBF policy class and the same saturated-distance reward) for 1, 10, 100, and 1000 trials, and measure test-target accuracy on the same 100-location batches. If MC-PILOT's single-trial accuracy is matched only after more than an order of magnitude more MFRL trials, the central data-efficiency claim is confirmed; if MFRL reaches comparable accuracy within a similar budget, the claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Sections 6.2 and 6.3.3 compare MC-PILOT to a 'Neural Network policy' trained by minimizing MSE between predicted and applied velocity on random throws (Section 6.2, eq. (36)); this baseline has no reward function, no policy-gradient update, and no RL exploration loop. Section 7 nevertheless states that MC-PILOT was compared with a 'Model-Free RL solution' and concludes that it is more data-efficient. The real-system comparison (Fig. 13) is the same supervised-regression baseline retrained on its own landings. A genuine MFRL algorithm (e.g., Tossingbot, DDPG, or PPO with the same policy class and reward) could be substantially more sample-efficient than this regression baseline, so the 'data efficient' advantage over MFRL is unverified. This does not disprove MC-PILOT's absolute throwing accuracy, but it removes the comparative load from the central claim: the paper currently demonstrates data-efficient learning relative to an analytical baseline and a supervised inverse model, not relative to Model-Free RL.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces MC-PILOT, a model-based reinforcement learning method for robotic pick-and-throw with a Franka Emika Panda. It learns a Gaussian process model of free-flight dynamics from motion-capture trajectories, estimates a uniform release-delay distribution via Bayesian optimization on landing predictions, and optimizes a squashed radial-basis-function policy by Monte Carlo particle simulation. Experiments in Gazebo and on the real system report near-100% target accuracy from a single trial (with Nexp=10 in the real setup) and generalization to unseen target locations and to several object shapes, comparing against a ballistic baseline and a neural-network regression baseline.","tokens_in":26072,"tokens_out":4168,"duration_ms":41724,"significance":"If the empirical claims hold, this is a valuable demonstration that model-based policy search can solve a dynamic manipulation task with very little real-world interaction. The paper's concrete strengths are the real-hardware validation with multiple object shapes, the explicit treatment of the gripper release delay inside the policy optimization loop, and the use of GP predictive uncertainty in a particle-based policy gradient scheme. However, the headline comparison against 'Model-Free RL' is not supported by the experiments as reported, because the baseline is a supervised regression network rather than a reinforcement learning algorithm. This limits the force of the data-efficiency claim, although the absolute throwing accuracy achieved by MC-PILOT is reported consistently across simulation and hardware.","major_comments":[{"comment":"The 'Model-Free' baseline is not a Model-Free RL algorithm. In §6.2 the neural-network policy is trained by minimizing the mean squared error between applied velocities and landing positions on collected throws, with no reward signal, no policy-gradient update, and no exploration loop. Section 7 nevertheless states that MC-PILOT was compared with a 'Model-Free RL solution' and concludes that it is more data-efficient. This mismatch is load-bearing because the central claim is data efficiency relative to Model-Free RL. Please either add a genuine MFRL baseline (e.g., PPO or DDPG with the same policy class and reward) or restrict the comparative claims to the supervised regression baseline and the analytical baseline actually evaluated.","section":"§6.2, §6.3.3, §7"},{"comment":"The simulation validation of the delay estimator is not convincing. The true release delay is U(0.01, 0.02) s, while the estimated distribution is a = 0.021 ± 0.057 s and b = 0.003 ± 0.004 s. The estimated interval [0.021, 0.024] does not match the true support, and the large standard deviation on a means that 'the optimization does predict the presence of a delay statistically' is not supported by the numbers. Since the paper presents delay estimation as a separate contribution, please report a calibration or coverage measure of the estimated distribution against the true delay, or explicitly discuss what quantity the estimator is capturing if it is not the physical release delay.","section":"§6.2 and Table 2 (left)"},{"comment":"The release model assumes that the robot tracks the reference trajectory closely enough that the release state is obtained by forward kinematics at the sampled release time, but no tracking-error quantification is provided, despite Figure 4 showing actual and reference joint trajectories. The simulation result above suggests that the fitted (a, b) parameters may be absorbing model mismatch and tracking error rather than representing a physical delay. To support the delay-estimation contribution, please quantify the tracking error at release and analyze how it affects the estimated delay distribution.","section":"§3.2, eq. (24)"}],"minor_comments":[{"comment":"The delay distribution is defined inconsistently: §5 first says td ∼ U(a, b), but eq. (29) and the surrounding text use td ∼ U(a, a + b). Please choose one convention and use it throughout.","section":"§5"},{"comment":"The true simulation delay is written with a comma as the decimal separator ('[0,01 s, 0,02 s]'); use consistent decimal notation and notation matching Table 2.","section":"§6.2"},{"comment":"The neural-network baseline is called 'unsupervised' in the text, but it is trained by supervised regression on collected landing and velocity data. Rename it to avoid confusion.","section":"§6.3.3"},{"comment":"The simulation-time variable is t′ in §4 but t in the update rule in eq. (25); unify the notation.","section":"§4.2.2"},{"comment":"References [14] and [31] are the same paper; the duplicate should be removed.","section":"References"},{"comment":"The caption says 'actual trajectory recorded on the robot,' but the text does not discuss the magnitude of tracking error; either add a quantitative comparison or remove the word 'actual' if the curves are only references.","section":"Figure 4"}],"recommendation":"major_revision","confidential_remarks":"The core MBRL method and the hardware results appear sound and interesting, but the paper currently overstates the comparison with Model-Free RL. The revision path is clear: either add a true MFRL baseline or scope the claims, and strengthen the delay-estimation validation. The manuscript is an incremental but substantial extension of the authors' CoDIT 2023 paper; the addition of delay estimation and the multi-object hardware study is sufficient for a journal-level contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a solid, incremental MBRL paper for robotic throwing. The core additions—a target-conditioned policy over a continuous target domain and a release-delay distribution estimated by Bayesian optimization inside the policy loop—are real, and the real-hardware results are genuinely useful. The main thing to know is that the data-efficiency claim against Model-Free RL is not backed by the experiments: the \"Model-Free\" baseline is a supervised regression network, not any RL algorithm.\n\nWhat the paper does well: it extends MC-PILCO to throwing by conditioning the GP model and policy on target location and by inserting an estimated release-delay distribution into the particle-based policy optimization. Simulation uses 10 seeds with near-100% accuracy, clearly above the analytic baseline and the regression network. Real experiments on a Franka Panda show systematic improvement for a ball, cube, and tennis ball, including generalization to held-out targets and adaptation to a new target domain (a bin) without re-exploration. The delay-estimation procedure is a nice idea, and the paper is honest in places: it acknowledges that the estimated delay in simulation does not match the true U(0.01,0.02) delay, and it shows that delay parameters are object-dependent.\n\nWhere it is soft: first, the baseline mislabeling. Section 6.2 describes a feedforward network trained by minimizing MSE between predicted and applied velocity on random throws. There is no reward, no policy gradient, no RL loop. Section 7 nevertheless calls it a \"Model-Free RL solution\" and concludes that MC-PILOT is more data-efficient. That comparative claim is unverified. A genuine MFRL baseline (DDPG, PPO, or Tossingbot-style) might be substantially more sample-efficient than this regression baseline. This does not damage the absolute throwing accuracy results, but it removes the comparative load from the central claim. Second, the delay estimate is best treated as a calibration term absorbing model mismatch, not a true physical delay; the paper's own simulation result shows this. The authors do acknowledge it, but the interpretation should be framed accordingly. Third, no code or data is released, which limits reproducibility checks. These are all addressable.\n\nOverall, the algorithm appears sound, the real-hardware evaluation is meaningful, and the writing is clear. I would send this to peer review; the referee should ask for a relabeled or genuine MFRL baseline, clearer sample sizes, and ideally artifacts. The paper is for robotics researchers working on pick-and-throw or on sample-efficient MBRL for manipulation. I would bring it to a reading group and would cite the delay-estimation idea in related work, though not the data-efficiency comparison.","headline":"Competent, incremental MBRL paper for robot throwing, but the data-efficiency claim against Model-Free RL is not supported by the experiments because the baseline is supervised regression, not RL.","tokens_in":26586,"tokens_out":2252,"would_cite":true,"duration_ms":24843,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T40","68T05"],"pacs":[],"model":"deepseek-v4-flash","headline":"A model-based RL algorithm, MC-PILOT, learns to throw objects to nearly every target from a single exploratory trial.","keywords":["pick-and-throw","model-based reinforcement learning","Gaussian process regression","release delay estimation","data efficiency","policy generalization","Franka Emika Panda","Monte Carlo policy optimization"],"falsifier":"In simulation the true delay is uniform on $[0.01, 0.02]$ s, but the estimated distribution is $U(0.021, 0.024)$ s, already showing the estimator is not recovering the physical delay; on the real arm, a high-speed camera or instrumented gripper that records the actual separation event across velocities and objects would settle whether one uniform scalar delay exists, and if the fitted $(a,b)$ must change with object and speed to keep landings accurate, the release model is a calibration term rather than the stated mechanism.","tokens_in":1552,"feed_emoji":"🎯","tokens_out":1602,"duration_ms":73740,"temperature":0.7,"pith_summary":"MC-PILOT is a model-based reinforcement learning method for pick-and-throw: a robot grasps an object and throws it into a target bin rather than carrying it there. The paper's central claim is that by fitting a Gaussian-process model of the object's free flight and an estimated release-delay distribution, the algorithm learns a throwing policy that hits nearly every target in simulation and on a Franka Emika Panda arm using only a handful of exploratory throws. The policy maps any target position in a region to a release speed, so new target locations are hit without collecting new data. Compared with an analytic ballistic baseline and with model-free policies trained on growing datasets, MC-PILOT reaches higher accuracy with far less interaction. If this holds, it makes pick-and-throw practical for industrial cells where targets change frequently and interaction time is expensive.","feed_headline":"One trial of model-based RL teaches a robot to throw to ~100% accuracy","feed_subtitle":"A Gaussian-process model plus estimated release delay lets a Panda arm hit training and unseen targets from a handful of throws.","key_machinery":"The central object is the particle-based rollout with a delayed-release initial state. Each particle samples a target $P^{(m)}$, a release delay $t_d^{(m)}\\sim U(a,a+b)$, and starts from $x_0^{(m)} = [f_{\\mathrm{kin}}(q_{\\tilde t_r}^{(m)}), J_a(q_{\\tilde t_r}^{(m)})\\dot q_{\\tilde t_r}^{(m)}]^T$, then propagates through one-step GP Gaussian transitions. This makes the policy optimization aware of both model uncertainty and timing uncertainty. A second mechanism is the delay estimator: Bayesian optimization minimizes the distance between GP-simulated landing points and measured landings as a function of $(a,b)$, using the same GP model. The speed-integration GP model with a squared-exponential kernel carries the free-flight dynamics.","core_discovery":"MC-PILOT inserts a GP model of the free-flight dynamics and a uniform release-delay distribution $t_d \\sim U(a,a+b)$ into the initial condition of each Monte Carlo particle; the policy is a squashed radial-basis network $\\pi_\\theta(P)$ that outputs release speed from target $P$, and optimization propagates particles through the GP to minimize expected saturated landing distance. In simulation with $N_{\\mathrm{exp}}=5$, $N_a=0$ and one outer-loop trial, the paper reports almost 100% accuracy on batches of 100 sampled targets, whereas the ballistic baseline misses the far targets and a neural-network policy needs many more training throws. On the real Franka Emika Panda with $N_{\\mathrm{exp}}=10$, $N_a=2$, a single trial yields policies that hit training and unseen test targets within a 0.1 m radius, and the bin-target experiment reaches 100% accuracy with three objects while the baseline reaches roughly 50%. The release-delay distribution is estimated by Bayesian optimization so that GP-simulated landings match observed landings, and it is re-estimated per object.","pith_inferences":["The estimated $(a,b)$ likely functions as a lumped calibration that absorbs tracking error, drag, and release mechanics rather than as a literal physical delay; a natural test is to compare the fitted distribution against high-speed video of actual object separation.","The fixed-release-geometry assumption, a constant height, angle, and radial distance, is what makes the policy a scalar speed from target; extending to variable release angle or height would stress whether the GP can absorb those extra degrees of freedom.","The same delay-estimation loop could be repurposed for other latency-sensitive dynamic manipulation skills, such as catching, hitting, or handover, where the arm and end-effector are not synchronized.","The rotational data augmentation and target-augmented state exploit symmetry around the robot's vertical axis; deployment in asymmetric environments, such as wind or uneven ground, would require revisiting those choices."],"forward_implications":["A single trial of about ten exploration throws suffices to hit a target region, so pick-and-throw becomes usable in settings where hundreds of practice throws are not affordable.","Because the policy is a function of target position, generalization to unseen targets follows from the same trial without additional exploration.","Modeling the release delay inside the particle simulation is the difference between near-100% accuracy and baseline-like misses; ignoring it leaves far targets unreachable.","When task requirements change, such as replacing ground targets with a bin of different height, the same model supports a new policy by re-running optimization only, in about 15 minutes on a laptop GPU.","Delay parameters are object-dependent and must be re-estimated when the thrown object changes, while the GP model itself can be reused across objects."],"supporting_citations":[{"why":"Supplies the GP speed-integration one-step model, the particle-based Monte Carlo policy gradient, and the dropout exploration that MC-PILOT adapts to throwing.","marker":"[19]"},{"why":"The principal model-free baseline; its hundreds-to-thousands-of-trials convergence is the data-efficiency contrast for MC-PILOT.","marker":"[16]"},{"why":"Establishes the earlier version of the algorithm that this paper extends with delay estimation and real-world validation.","marker":"[21]"},{"why":"Provides the throwing-motion pattern and the skill-learning comparison class with limited real target changes.","marker":"[12]"},{"why":"Underpins the delay-distribution estimation procedure that minimizes simulated-versus-actual landing error.","marker":"[40]"},{"why":"Supplies the empirical drag law used in the simulated environment, which the GP must learn to match.","marker":"[47]"}],"fun_headline_variants":["Robot throws with near-perfect accuracy after one model-based RL trial","One-shot learning: model-based RL robot throw accuracy","Model-based RL enables data-efficient robotic throwing","MC-PILOT: robot throwing from a single trial","Robot learns to throw accurately from just five throws"],"cache_read_input_tokens":28800,"weakest_assumption_plain":"The whole scheme assumes that the object lets go at a single random instant and that the robot's real motion at that instant matches the planned motion, so the release position and velocity can be taken straight from the plan.","fun_headline_variants_meta":{"raw":{"variants":["Robot throws with near-perfect accuracy after one model-based RL trial","One-shot learning: model-based RL robot throw accuracy","Model-based RL enables data-efficient robotic throwing","MC-PILOT: robot throwing from a single trial","Robot learns to throw accurately from just five throws"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00034,"raw_usage":{"total_tokens":1898,"prompt_tokens":992,"completion_tokens":906,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":608,"completion_tokens_details":{"reasoning_tokens":829}},"tokens_in":608,"tokens_out":906,"duration_ms":9077,"temperature":1.0,"reasoning_tokens":829,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T18:42:14.220166+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"In simulation the true delay is uniform on $[0.01, 0.02]$ s, but the estimated distribution is $U(0.021, 0.024)$ s, already showing the estimator is not recovering the physical delay; on the real arm, a high-speed camera or instrumented gripper that records the actual separation event across velocities and objects would settle whether one uniform scalar delay exists, and if the fitted $(a,b)$ must change with object and speed to keep landings accurate, the release model is a calibration term rather than the stated mechanism.","supporting_citations":[{"cited_title":"Amadio, A","cited_arxiv_id":null,"evidence_quote":"Supplies the GP speed-integration one-step model, the particle-based Monte Carlo policy gradient, and the dropout exploration that MC-PILOT adapts to throwing."},{"cited_title":"Turcato, A","cited_arxiv_id":null,"evidence_quote":"Establishes the earlier version of the algorithm that this paper extends with delay estimation and real-world validation."},{"cited_title":"Pahi ˇc, Z","cited_arxiv_id":null,"evidence_quote":"Provides the throwing-motion pattern and the skill-learning comparison class with limited real target changes."},{"cited_title":"Almedeij, Drag coe fficient of flow around a sphere: Matching asymp- totically the wide trend, Powder Technology 186 (3) (2008) 218–223","cited_arxiv_id":null,"evidence_quote":"Supplies the empirical drag law used in the simulated environment, which the GP must learn to match."}],"review_version":1}