{"id":"7ee48786-7444-41fc-9174-941983124105","arxiv_id":"2607.10892","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"One diffusion policy trained via energy-guided RL solves multi-shape block pushing without demos and transfers zero-shot to real robots under varied conditions.","lead":"A single diffusion policy trained from scratch with online RL learns multi-shape block pushing in sparse-reward simulation. The same controller transfers zero-shot to a real robot under changes in goal pose, block shape, weight, and surface friction.","discovery_kind":"new_application","skeptic_critique":{"model":"grok-4.5","headline":"Real-world zero-shot claim rests on a tiny, incompletely reported hardware sample whose success cannot be independently verified from the text.","rationale":"The Reader correctly flags the friction/stability assumption in §IV-A as a soft spot and correctly assigns CONDITIONAL. That assumption, however, is secondary: even if friction stays inside the safe region, the paper still needs a verifiable hardware sample large enough to support the multi-variation zero-shot statement. The present text supplies only aggregate claims and a 9-trial SAC comparison. Strengthening the hardware evidence (or explicitly scoping the claim to the logged trials) is therefore the single most load-bearing fix; once that is done the Reader’s remaining concerns (code, hyper-parameters, language) become ordinary revision items rather than verdict-changing ones. Hence I keep CONDITIONAL and only partially agree with the Reader’s weakest-assumption ranking.","tokens_in":13560,"tokens_out":555,"duration_ms":7381,"concrete_test":"Release the exact real-world trial log (36+ episodes): for each trial record shape, measured mass, surface type, initial/goal poses, success/failure, steps, path length, and max pose-estimation error. Recompute Table III success rates and distances from that log; if any condition drops below 2/3 success or if the external videos cannot be matched to the log, the zero-shot claim must be narrowed or withdrawn.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim (Abstract; §I c; §V-B; Table III) is not merely that ESM works in simulation, but that the same policy transfers zero-shot to a real UR5e under variations in goal pose, shape, weight and surface friction. That hardware claim is supported by only 9 tasks for the SAC comparison (3 shapes × 3 trials) and a stated total of 36 tasks whose per-condition success counts, failure modes, pose-estimation error statistics, and exact friction/weight values are never tabulated. Videos are promised on an external site that is not part of the archival record. Because the paper itself argues (§IV-A, §VII) that transfer succeeds only when contact dynamics remain inside the stabilizing region of the multi-modal policy, an unquantified or non-reproducible hardware sample leaves the decisive empirical premise uncheckable. Simulation tables (I–II) are strong; the zero-shot half of the strongest claim is not yet on equally firm ground.","agreement_with_reader":"partial"},"referee_report":{"model":"grok-4.5","summary":"The paper proposes training a single diffusion policy from scratch via online RL for multi-task sparse-reward block pushing across shapes (T/C/L). It introduces an energy-guided score matching (ESM) loss that reweights the standard DDPM ELBO by exp(Q/λ) so that the diffusion model targets the closed-form policy-mirror-descent update (Eq. 2, Prop. 1, Appendix A). Exploration is aided by reverse curriculum generation (expanding start–goal distance) and objective-centric observations/actions expressed in the block frame plus a simple 4-D shape code. In simulation the method reaches 100 % success with short episodes (Tables I–II), substantially outperforming SAC, PPO, TD3 and several recent diffusion-RL baselines; ablations confirm the value of curriculum and frame design (Fig. 7). The same policy is claimed to transfer zero-shot to a UR5e under variations of goal pose, shape, weight and surface friction (Abstract, §I, §V-B, Table III).","tokens_in":13851,"tokens_out":1082,"duration_ms":17212,"significance":"If the claims hold, the work supplies a practical, demonstration-free route to multi-modal policies for contact-rich multi-task manipulation and shows that diffusion policies can be trained end-to-end with online RL while retaining zero-shot sim-to-real robustness under the tested conditions. The derivation that ESM recovers the mirror-descent target (Appendix A) is clean and avoids the baseline-function or truncation issues of related weighted-score methods. Strong simulation tables, clear ablations, and an explicit multi-shape generalization test (including an unseen I-block) constitute solid empirical evidence for the algorithmic contribution. The real-world transfer result, if fully substantiated, would be a useful data point for the community on when multi-modal policies can absorb unmodeled friction without domain randomization.","major_comments":[{"comment":"The central zero-shot claim (Abstract; §I-c; §V-B) rests on 36 real-world tasks whose per-condition success counts, distances, steps, failure modes, pose-estimation error statistics, and exact friction/weight values are never tabulated. Table III reports only a 9-task subset (3 shapes × 3 trials) for the SAC comparison; the remaining 27 tasks and all quantitative metrics appear only as qualitative statements and an external website. Because the paper itself argues (§IV-A, §VII) that transfer succeeds only when contact dynamics remain inside the stabilizing region of the multi-modal policy, an incompletely reported hardware sample leaves the decisive empirical premise unverifiable from the archival record.","section":"§V-B, Table III"},{"comment":"No quantitative characterization of the residual sim-to-real gap is supplied (measured friction coefficients, contact stiffness, pose-estimation RMSE under occlusion, etc.). The argument in §IV-A that “the sim-to-real gap will not pose a significant challenge \tau as long as it does not affect system stability” is therefore an untested modeling assumption rather than a measured fact; without those numbers the zero-shot success cannot be attributed to policy multi-modality versus simply a small gap.","section":"§IV-A, §VII"}],"minor_comments":[{"comment":"The citation for catastrophic forgetting is left as “[?]” in the introduction.","section":"§I"},{"comment":"Shape-encoding vector is described as living in [0.0,2.0]^4 yet never specified numerically for the four training shapes; a short table or equation would aid reproducibility.","section":"§III-B, Fig. 2"},{"comment":"Curriculum parameters M=10, N=50 and the exact spatial expansion schedule appear only in Appendix B; they should be stated in the main text or a hyper-parameter table.","section":"§III-C, Appendix B"},{"comment":"Figure 7 ablation curves lack error bars or seed-wise variance, making it hard to judge whether the “acceptable success rates” after ablation are statistically reliable.","section":"Fig. 7"},{"comment":"The point-cloud registration pipeline (§IV-B) is only sketched; a short quantitative accuracy number (e.g., mean rotational error under partial occlusion) would strengthen the claim that state-based inputs enable easy transfer.","section":"§IV-B"}],"recommendation":"major_revision","confidential_remarks":"The external website is not part of the archival record; if the journal requires self-contained evidence, the authors must move the full 36-task table and representative failure videos into the supplement. The algorithmic core (ESM + curriculum + frame design) is publishable even if the hardware claim is softened; the present over-claim on zero-shot transfer is the main obstacle."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The useful takeaway is simple: one diffusion policy trained purely online (no demos) with an exp(Q)-reweighted ELBO, reverse curriculum, and block-frame observations hits 100% multi-shape sparse-reward Push-T variants in sim and appears to transfer zero-shot to a UR5e under the friction/weight/goal variations they tried. That combination is what is actually new; the individual pieces (online diffusion RL, reverse curriculum, goal-centric frames) already exist.\n\nWhat they do well is the empirical pipeline. Table I is clean: ESM succeeds where SAC is partial and PPO/TD3/QVPO/DACER/QSM collapse. Per-shape and unseen I-block numbers (Table II) and the ablations (Fig. 7) make a fair case that expressiveness plus curriculum plus local frames matter. Appendix A’s equivalence argument for ESM is coherent and avoids the baseline estimation some related methods need. They also stay relatively honest in the limitations about when zero-shot contact transfer can fail.\n\nSoft spots, in proportion: the hardware half of the headline claim is the weak link. Real evaluation is 36 tasks with only a 9-task SAC comparison tabulated; per-condition counts, pose-error stats, and exact friction/weight values are not in the paper, and videos live off-archive. That does not kill the claim, but it means the decisive sim-to-real premise is not independently checkable from the text. Curriculum (M, N, expansion) and λ are free design choices; no code release. Language about “universal” policies is a bit broader than the tested tabletop suite.\n\nThis is for people already working on diffusion policies or contact-rich multi-task RL who want a practical online recipe and a hardware stress test, not a theory paper. Math and citation pattern look solid enough; data are strong in sim and provisional on hardware. I would send it to peer review—ask for fuller hardware tables, hyperparameters, and code—rather than desk-reject. Worth engaging if you care about online diffusion for manipulation.","headline":"Strong sim multi-task diffusion-from-scratch result with a clean reweighted ELBO; zero-shot hardware claim is real but thinly documented.","tokens_in":14498,"tokens_out":521,"would_cite":true,"duration_ms":12816,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"A single diffusion policy trained from scratch with online RL solves multi-shape block pushing and transfers zero-shot to real robots under tested variations.","keywords":["diffusion policy","online reinforcement learning","block pushing","sim-to-real transfer","multi-task manipulation","curriculum learning","objective-centric representation","energy-guided score matching"],"falsifier":"Run the identical zero-shot policy on a surface whose friction (or block mass) lies outside the range already tested and record whether success rate collapses while pose estimation remains accurate; if it does, the zero-shot claim fails for that gap size.","tokens_in":14410,"feed_emoji":"🤖","tokens_out":849,"duration_ms":8260,"temperature":0.7,"pith_summary":"The paper claims that one diffusion policy can learn multi-task, contact-rich block pushing from sparse rewards without expert demonstrations. It trains the policy from scratch with online RL by reweighting the usual diffusion training loss with the exponential of the action-value function, then pairs that objective with reverse curriculum generation and objective-centric state and action representations. In simulation the resulting controller reaches 100 percent success on T-, C-, and L-shaped blocks and still succeeds on an unseen I-shape, while Gaussian-policy baselines largely fail. The same policy, using only pose estimates, is then run on a real robot and succeeds under changes in goal pose, block shape, block weight, and surface friction. A sympathetic reader cares because the result suggests that the multi-modality of diffusion policies can carry both multi-task competence and zero-shot sim-to-real transfer for contact-rich manipulation when the right curriculum and observation design remove the exploration barrier.","feed_headline":"One diffusion policy pushes many block shapes, real robot included","feed_subtitle":"Trained from scratch with RL and curriculum, it transfers zero-shot under friction, weight and goal changes.","key_machinery":"Energy-guided score matching (ESM): the ordinary diffusion evidence-lower-bound loss reweighted by the exponential of the current action-value function, which is exactly the closed-form policy-mirror-descent update and can be used as a drop-in policy module inside any actor-critic RL algorithm.","core_discovery":"A single diffusion policy, trained from scratch by online reinforcement learning with an energy-guided score-matching loss, reverse curriculum generation, and objective-centric state representations, solves multi-shape sparse-reward block-pushing tasks at 100 percent success in simulation and transfers zero-shot to real-world block pushing under the tested variations in goal pose, shape, weight, and surface friction.","pith_inferences":["If the multi-modality of the diffusion policy is what preserves a stabilizing skill under friction mismatch, the same pipeline may transfer to other planar contact tasks (e.g., sliding assembly) without domain randomization.","The simple shape encoding already yields partial zero-shot generalization to an unseen I-block; richer image- or geometry-based encodings could extend the same controller to open-set object sets.","Because the method only needs pose estimates, any improvement in marker-free registration immediately upgrades the real-world success envelope without retraining the policy."],"forward_implications":["One multi-modal policy class can replace separate single-task controllers for families of contact-rich pushing tasks when curriculum and relative coordinates are used.","Gaussian-policy RL methods remain inadequate for the same multi-task sparse-reward setting even after identical curriculum and representation design.","Accurate pose estimation plus relative coordinates is sufficient for zero-shot transfer of diffusion policies when contact dynamics stay inside the stabilizing region of the learned policy.","The same ESM loss can serve as a fine-tuning module for existing behavior-cloned diffusion policies without redesigning the network."],"fun_headline_variants":["Single RL diffusion policy pushes multi-shape blocks zero-shot to real","One diffusion policy solves multi-task block pushing with zero-shot transfer","Diffusion policy from RL handles multi-shape blocks and transfers zero-shot","RL-trained diffusion controller multi-tasks block pushes to real robot","Multi-shape sparse-reward block pushing by one diffusion policy zero-shot"],"cache_read_input_tokens":128,"weakest_assumption_plain":"The main sim-to-real gap (surface friction and related contact forces) does not push the closed-loop system outside the region where the learned multi-modal policy remains stabilizing, so domain randomization or residual adaptation can be skipped.","fun_headline_variants_meta":{"raw":{"variants":["Single RL diffusion policy pushes multi-shape blocks zero-shot to real","One diffusion policy solves multi-task block pushing with zero-shot transfer","Diffusion policy from RL handles multi-shape blocks and transfers zero-shot","RL-trained diffusion controller multi-tasks block pushes to real robot","Multi-shape sparse-reward block pushing by one diffusion policy zero-shot"]},"model":"grok-4.5","effort":"low","cost_usd":0.005468,"raw_usage":{"total_tokens":1476,"prompt_tokens":751,"num_sources_used":0,"completion_tokens":79,"cost_in_usd_ticks":54680000,"prompt_tokens_details":{"text_tokens":751,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":646,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":751,"tokens_out":79,"duration_ms":5502,"temperature":1.0,"reasoning_tokens":646,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-14T08:29:09.051110+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Run the identical zero-shot policy on a surface whose friction (or block mass) lies outside the range already tested and record whether success rate collapses while pose estimation remains accurate; if it does, the zero-shot claim fails for that gap size.","supporting_citations":[],"review_version":1}