{"id":"cfb88a93-efcd-453d-bca7-3f1a54f39a3a","arxiv_id":"2507.05674","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A diffusion policy pretrained on offline gait data and then finetuned with PPO achieves robust language-conditioned quadruped control with 50 Hz onboard inference.","lead":"DMLoco is a two-stage system that first trains a diffusion-based quadruped policy on offline demonstrations of four gaits, then finetunes it with online reinforcement learning to enable smooth gait changes and language-guided commands. It runs onboard at 50 Hz on a Jetson computer, and the paper reports improved success and tracking over several existing methods.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The two-term PPO finetuning reward in Sec. IV.3 has no gait-preservation or policy-regularization term, so the policy may drift to one robust gait; per-gait success rates in Table I do not verify gait identity, leaving the multi-gait claim unsupported.","rationale":"The reader's weakest assumption, that the two-term PPO finetuning preserves the pretrained skills and produces robust transitions, is exactly the load-bearing point. I sharpen it by noting the objective contains no mechanism to preserve gait identity, and the success metrics cannot detect a collapse to one gait. This is not a claim that the authors are wrong; it is a missing control. The paper has independent support: DDIM and TensorRT latency are concrete, and the three-seed learning curve in Figure 5(a) is a good reproducibility signal. But those do not address gait identity. The proposed check is cheap and decisive: if the four gait commands still yield four distinct contact-phase signatures after finetuning, the concern is retired and the central claim stands. Since the reader already conditions acceptance on resolving this area, no verdict change is needed.","tokens_in":9813,"tokens_out":9255,"duration_ms":100418,"concrete_test":"On the released DMLoco checkpoints (or a retrained identical model), run the pretrained and finetuned policies in simulation and on the real robot under the four gait commands (trot, bound, pace, pronk) at matched speeds. Record hind-leg and fore-leg phase offsets plus duty factors from joint encoders or contact sensors. Apply a simple classifier, for example phase lag between left-front and right-front and between front and hind pairs, to both policy sets. If the finetuned policy's four commanded conditions do not produce four distinguishable gait signatures, for instance if trot and pace become statistically indistinguishable, the multi-gait claim fails. Report this analysis alongside Table I and Figure 5(b).","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that DMLoco enables a quadruped to execute four learned gaits depends on Section IV.3's online finetuning. The finetuning reward contains only speed-tracking error and fall detection, and the text states that a relatively high diffusion noise scheduling is used to encourage exploration. There is no KL constraint toward the pretrained policy, no auxiliary behavior-cloning loss, and no term that rewards matching the commanded gait label. Under a high-noise PPO update, the policy can therefore drift from the four pretrained modes toward a single robust gait (for instance, trotting) that tracks most velocity commands and avoids falls. The reported metrics do not rule this out: success rate in Table I is defined as the robot correctly follows the instruction and remains upright, and tracking error is MSE velocity error; neither measures foot-contact pattern or gait phase. Figure 5(b) reports only transition success rates. Consequently, a finetuned policy that has collapsed to one gait could still attain high success and low tracking error, while the paper's signature multi-task and transition claims would be false. The absence of any gait-identity measurement is the single most load-bearing gap in the evidence.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DMLoco, a framework for quadruped robot locomotion that combines diffusion-based multi-task imitation pretraining with online PPO finetuning. The policy is first trained on an offline dataset of four gaits (trotting, bounding, pacing, pronking) with structured velocity/gait commands or natural-language embeddings; it is then finetuned in simulation with a simple two-term reward (speed-tracking error and fall detection) to improve robustness and enable gait transitions. The authors report high velocity-tracking success and low tracking error in simulation, transition-success results on a real Unitree Aliengo robot, and real-time 50 Hz onboard inference via DDIM and TensorRT. The central claims are that DMLoco enables language-conditioned multi-gait control and robust gait transitions without explicit transition data.","tokens_in":10060,"tokens_out":7240,"duration_ms":79642,"significance":"If the central claims hold, DMLoco offers a practical recipe for converting offline diffusion policies for legged locomotion into robust, deployable controllers with a minimal reward design, and it demonstrates language-conditioned control plus real-time onboard execution. The paper provides a code/video link and includes both simulation and real-robot experiments, which is valuable for reproducibility. The main technical risk is that the finetuning procedure may destroy the multi-gait structure of the pretrained policy, and the current evaluation metrics do not resolve this uncertainty. The contribution is incremental but potentially useful; the evidence as presented does not yet fully support the multi-gait and transition claims.","major_comments":[{"comment":"The finetuning reward contains only speed-tracking error and fall detection, with no term that rewards matching the commanded gait, no KL constraint toward the pretrained policy, and no auxiliary behavior-cloning loss. The paper states that a relatively high diffusion noise schedule is used to encourage exploration, which further increases the risk of mode collapse. The evaluation metrics in Table I (success rate defined as 'correctly follows the instruction and remains upright' and MSE velocity error) do not measure foot-contact pattern, gait phase, or any other gait-identity quantity. Consequently, the reported 100% success rates and low tracking errors are compatible with a policy that has collapsed to a single robust gait (e.g., trotting) while the velocity command is tracked. The paper's central claims of multi-gait execution and task transitions therefore require either an explicit gait-preservation mechanism in the finetuning objective or an evaluation that verifies the executed gait matches the command; without one of these, the evidence in Table I and Figure 4 does not support the claims.","section":"Section V-A and Figure 4"},{"comment":"The gait-transition experiment is underpowered and incompletely reported. The text says 'Each transition type is repeated 5 times, resulting in 20 trials in total,' but the experiment is run at three speeds (0.1, 0.5, 1.0 m/s) and four transition types, which would require 60 trials if five repetitions per condition are used; the discrepancy is unexplained. Figure 4 shows no error bars or statistical tests, and two baselines (BeT and DMLoco-lang) are excluded with only a brief textual justification. The reader cannot assess whether the reported transition-success differences are significant or whether the selected baselines would change the comparison. The paper should clarify the trial count, report per-speed and per-transition breakdowns with confidence intervals, and either include all baselines or justify their exclusion with quantitative results.","section":"Section III-C"},{"comment":"The paper claims to use PPO for finetuning but only shows a REINFORCE-style gradient: ∇θ J = E[∑ ∇θ log π̄θ r̄]. The clipped surrogate objective, advantage estimation (GAE), and the treatment of the two-layer MDP for diffusion policies are not specified. Since the integration of online PPO finetuning is one of the paper's main contributions, the missing derivation makes the method difficult to reproduce and obscures whether the implementation deviates from standard PPO. The authors should provide the full PPO objective as applied to their diffusion policy, or explicitly refer to the derivation in [8] and state any modifications.","section":"Section V-C and Figure 5(a)"},{"comment":"The learning curve in Figure 5(a) shows the success rate increasing from about 0.6 to nearly 1.0 during finetuning, but the paper does not specify the evaluation protocol used to compute this curve. If the success rate is measured on the same velocity-tracking task as Table I, it contradicts the near-perfect performance of the pretrained policy (DiffuseLoco, which is described as similar to the pretraining stage) and the high success rates reported in Table II for the pretrained policy with DDIM. If the evaluation during finetuning includes gait transitions or other more difficult conditions, this should be stated explicitly. The inconsistency between the learning curve and the pretrained-policy performance needs to be resolved for the finetuning benefit to be convincing.","section":"Section V-B, Table I"}],"minor_comments":[{"comment":"The manuscript contains several typos and stylistic inconsistencies, e.g., 'First We' in the Figure 1 caption and 'preform' instead of 'perform'; the abstract uses 'finetuning' while the main text uses both 'finetuning' and 'fine-tuning'.","section":"General"},{"comment":"References [17] and [18] are identical (both list 'Learning quadrupedal locomotion over challenging terrain'); one should be removed or replaced with a distinct relevant work.","section":"References"},{"comment":"Table I reports point estimates without standard deviations, confidence intervals, or the number of seeds; the 100.00 success rates across all four gaits are suspiciously clean and would be more credible with per-seed variation.","section":"Table I"},{"comment":"The definition of 'success' in the velocity-tracking task is vague; the paper should specify the threshold on tracking error, the required time horizon, and whether the robot must stay within a bounded region.","section":"Section V-A"},{"comment":"The finetuning hyperparameter 'Sampling denoising std 0.04' is not tied to the statement in Section IV.3 that 'we employ a relatively high diffusion noise scheduling to encourage exploration'; the paper should state the pretraining sampling std and clarify how the finetuning value is relatively high.","section":"Table III"},{"comment":"The description of the GPT-4-generated language dataset lacks details such as the prompt template, filtering criteria, and examples of the 1,000 instructions; additional detail would improve reproducibility of the language-conditioning results.","section":"Section IV.4"}],"recommendation":"major_revision","confidential_remarks":"The primary technical concern is the gait-identity gap: the finetuning reward and evaluation metrics do not rule out mode collapse to a single gait. This is a correctable issue, but it requires new experiments or an explicit regularization term. The transition trial-count inconsistency and missing PPO derivation also need attention. If addressed, the paper could be a solid contribution; the current version is too preliminary for acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth a look. The integration of multi-task diffusion pretraining with online PPO finetuning is real, and the real-robot results show the recipe works. But the paper never verifies that the robot performs the commanded gait on the real system, which is a larger gap than the authors acknowledge. The finetuning reward is only speed tracking plus fall detection, and the reported metrics—success rate and MSE velocity error—cannot distinguish trotting from bounding. So the central multi-gait transition claim is not actually supported by the evidence; a policy that collapsed to one stable gait could still pass Table I and the transition trials.\n\nWhat's genuinely new: prior work either used diffusion imitation alone (DiffuseLoco) or RL with crafted rewards. DMLoco combines the two, uses single-step action prediction instead of chunking, and shows a careful path to 50 Hz deployment with DDIM and TensorRT. The real-robot finetuning improvements (45-70% higher success over the pretrained policy) are substantive, and the ablations on state horizon, action horizon, and DDIM vs DDPM are informative and relevant to deployment.\n\nWhere the soft spots are, in order of severity:\n1. Gait identity is not measured anywhere. No foot-contact pattern, no phase, no gait classifier. This is the load-bearing gap. The stress-test note is right.\n2. Missing uncertainty: no error bars in Table I or Figure 4. The 5-trial real-robot transition test is thin.\n3. Baseline handling: BeT and DMLoco-lang are excluded from transition plots. The reason (poor performance, language latency) may be valid, but as reported it is post-hoc.\n4. The PPO section is under-developed: it shows a REINFORCE-style gradient and refers to prior work, but never derives the clipped objective or explains how the diffusion denoising MDP is handled in PPO update.\n5. Reproducibility: reward weights are not reported, and the language variant's 0.05 m^2/s^2 error increase is called \"marginal\" though it is a 36% relative increase. Calling that marginal is overstated.\n\nNone of these, except the gait-identity issue, break the paper. The authors are honest about the simplicity of the reward and they do ship a code link. For a practitioner who wants a working diffusion-PPO quadruped pipeline, this is a useful reference.\n\nSend to peer review—it deserves a serious referee—but with a recommendation for major revision, chiefly to add a direct gait-identity check on both simulated and real-robot rollouts.","headline":"A practical diffusion-PPO recipe with real-robot gains, but the multi-gait claim is not yet proven because gait identity is never measured.","tokens_in":10627,"tokens_out":5653,"would_cite":false,"duration_ms":61657,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"DMLoco claims diffusion policies can master quadruped gaits and language commands when offline imitation is followed by a simple two-term PPO finetune.","keywords":["diffusion policy","quadruped locomotion","PPO finetuning","language-conditioned control","gait transitions","DDIM sampling","sim-to-real transfer","multi-task imitation learning"],"falsifier":"Run the exact two-term PPO finetuning recipe (same noise schedule, learning rate, five DDIM steps, and reward weights) on a second quadruped model in simulation, then test the same four gait transitions at 1.0 m/s in at least 20 trials per transition. If success rates fall well below the reported near-100% or the policy catastrophically forgets one of the pretrained gaits, the finetuning phase—not the pretraining—is not the source of the claimed robustness.","tokens_in":9612,"feed_emoji":"🦿","tokens_out":5319,"duration_ms":53620,"temperature":0.7,"pith_summary":"This paper tries to show that diffusion-based imitation learning, which has lagged behind in legged locomotion because of compounding errors and missing transition data, becomes practical when followed by a short online reinforcement-learning phase. The proposed DMLoco first pretrains a diffusion policy on offline expert data for four quadruped gaits, conditioned on either structured velocity commands or natural language. It then finetunes that policy with PPO in simulation using only two reward terms—speed-tracking error and fall detection—which lets the robot switch between gaits without any explicit transition data. The authors report that this two-stage recipe reaches perfect success across gaits, beats four baselines on tracking error, and runs onboard at 50 Hz using deterministic DDIM sampling and hardware acceleration. If true, it offers a low-reward-engineering route to language-guided, multi-skill locomotion on resource-constrained robots.","feed_headline":"Two-stage diffusion-PPO control hits 100% gait success","feed_subtitle":"Offline imitation plus a two-term online PPO finetune yields language-commanded gait switching at 50 Hz on the robot.","key_machinery":"The load-bearing mechanism is the formulation of a DDPM policy as a two-layer MDP, whose inner denoising loop can be optimised by standard policy-gradient algorithms; this is what lets PPO finetune the pretrained diffusion policy directly. Around that core sit three supporting design choices: conditioning the denoiser on a window of past states and goals (state horizon 30) while predicting only the current action rather than action chunks; a two-term finetuning reward (speed-tracking error and fall detection) with a higher denoising noise schedule to encourage exploration; and DDIM deterministic sampling with five denoising steps, which removes the stochastic delays of DDPM and, with hardware acceleration, reaches the 50 Hz control frequency needed for real-time quadruped control.","core_discovery":"On its own terms, the paper's discovery is that the gap between offline diffusion policies and deployable quadruped control is closed by a particular division of labour. Offline pretraining supplies multi-modal coverage of four gaits (trotting, bounding, pacing, pronking) and aligns language embeddings with structured commands; online PPO finetuning supplies robustness and gait-transition ability that pure imitation lacks. Crucially, finetuning needs no transition demonstrations and no complex reward stack—two terms, speed-tracking error and fall detection, suffice. The paper further finds that predicting a single action per timestep rather than action chunks is important for dynamic locomotion, and that DDIM's deterministic few-step sampling (5 steps) gives both the speed and stability that DDPM cannot match at low step counts. Together these choices yield 100% success across gaits in simulation, the lowest tracking error among the compared methods, and stable transitions on the physical robot, with language-conditioned performance nearly matching structured-command performance.","pith_inferences":["The two-term-reward finetuning recipe is stated generally enough that it should transfer to other quadruped platforms and other skill sets; a direct test would be applying the same hyperparameters to a different robot model without reward redesign.","If the division of labour between imitation pretraining and RL finetuning is the real cause of robustness, then the same recipe should also improve non-diffusion policies—for instance, a transformer policy pretrained on the same dataset and finetuned with the same two-term PPO reward.","The language interface's apparent generalization suggests the diffusion policy stores skills in a shared representation aligned to both structured and free-form commands; an untested corollary is that novel combinations of commands and gaits might be composable, such as requesting a new gait at speeds outside the training ranges.","The dependence on DDIM's determinism for stability implies a sensitivity to sampling stochasticity; a useful stress test would be perturbing the denoising trajectory at deployment and measuring how much the gait-transition success degrades."],"forward_implications":["A diffusion policy pretrained by imitation can be made robust enough for real-robot deployment with a finetuning phase that needs no gait-transition demonstrations and only two reward terms.","The same policy can follow both structured velocity commands and natural-language instructions, with language-conditioned control staying close to structured-command performance on unseen test instructions.","Gait transitions that are absent from the training distribution—trotting to bounding, bounding to pacing, pacing to pronking, pronking to trotting—become reliable after online finetuning, even at 1.0 m/s.","DDIM with five sampling steps replaces 100-step DDPM sampling without losing stability, cutting inference cost enough to run the policy onboard at 50 Hz on embedded hardware.","Predicting single actions instead of action chunks is preferable in locomotion because it avoids delayed responses to rapidly changing states."],"supporting_citations":[{"why":"Supplies the imitation-only diffusion baseline whose lack of online finetuning is the comparison point for DMLoco's robustness gains.","marker":"[28]"},{"why":"Provides the policy-gradient formulation for diffusion policies that DMLoco uses to run PPO finetuning.","marker":"[8]"},{"why":"Supplies the diffusion-policy architecture and conditioning approach that the pretraining stage adapts to locomotion.","marker":"[6]"},{"why":"Serves as the autoregressive imitation baseline (BeT) that DMLoco must beat.","marker":"[39]"},{"why":"Provides the multi-skill RL baseline (WTW) that relies on complex reward design, against which the two-term reward is contrasted.","marker":"[19]"},{"why":"Provides the AMP-style skill-discovery baseline (Cassi) that needs a skill discriminator, the comparison point for latent-variable skill guidance.","marker":"[25]"},{"why":"Defines DDPM, the diffusion formalism whose denoising process is reformulated as a two-layer MDP and replaced by DDIM at deployment.","marker":"[36]"},{"why":"Supplies the lightweight pretrained language model used as the starting point for the language-embedding module.","marker":"[38]"}],"fun_headline_variants":["Diffusion-PPO two-stage training achieves 100% gait success","Language-commanded quadruped control hits 100% via two-stage training","Offline diffusion plus online PPO yields robust 50Hz gait switching","Single-step action prediction enables perfect quadruped gait transitions","Two reward terms suffice for robust diffusion-PPO quadruped control"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a short PPO finetuning phase using only speed-tracking error and fall detection can turn an imitation-only diffusion policy into one that transfers to a real robot with stable gait transitions, despite never seeing transition data and without forgetting the pretrained skills.","fun_headline_variants_meta":{"raw":{"variants":["Diffusion-PPO two-stage training achieves 100% gait success","Language-commanded quadruped control hits 100% via two-stage training","Offline diffusion plus online PPO yields robust 50Hz gait switching","Single-step action prediction enables perfect quadruped gait transitions","Two reward terms suffice for robust diffusion-PPO quadruped control"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000427,"raw_usage":{"total_tokens":2203,"prompt_tokens":980,"completion_tokens":1223,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":596,"completion_tokens_details":{"reasoning_tokens":1133}},"tokens_in":596,"tokens_out":1223,"duration_ms":12802,"temperature":1.0,"reasoning_tokens":1133,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T19:20:38.298265+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the exact two-term PPO finetuning recipe (same noise schedule, learning rate, five DDIM steps, and reward weights) on a second quadruped model in simulation, then test the same four gait transitions at 1.0 m/s in at least 20 trials per transition. If success rates fall well below the reported near-100% or the policy catastrophically forgets one of the pretrained gaits, the finetuning phase—not the pretraining—is not the source of the claimed robustness.","supporting_citations":[{"cited_title":"Diffusion Policy: Visuomotor Policy Learning via Action Diffusion","cited_arxiv_id":null,"evidence_quote":"Supplies the diffusion-policy architecture and conditioning approach that the pretraining stage adapts to locomotion."},{"cited_title":"Behavior transform- ers: Cloningkmodes with one stone","cited_arxiv_id":null,"evidence_quote":"Serves as the autoregressive imitation baseline (BeT) that DMLoco must beat."},{"cited_title":"Walk these ways: Tuning robot control for generalization with multiplicity of behavior","cited_arxiv_id":null,"evidence_quote":"Provides the multi-skill RL baseline (WTW) that relies on complex reward design, against which the two-term reward is contrasted."},{"cited_title":"De- noising diffusion probabilistic models","cited_arxiv_id":null,"evidence_quote":"Defines DDPM, the diffusion formalism whose denoising process is reformulated as a two-layer MDP and replaced by DDIM at deployment."}],"review_version":1}