{"id":"483c61a5-cd85-481c-a5f6-accb0aac16b8","arxiv_id":"2411.08832","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A diffusion-based quadruped locomotion policy uses classifier-free guidance to adapt to new velocity-tracking rewards after training, enabling multi-skill interpolation and onboard CPU deployment.","lead":"This paper trains a diffusion model to control a four-legged robot, then uses classifier-free guidance to adjust the robot's behavior after training, without retraining, for example to walk or crawl at a target speed. The method runs on the robot's onboard CPU and is demonstrated on a real ANYmal quadruped robot.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Most load-bearing issue is that Table I's conclusion rests on lambda values selected using the test reward itself; this selection bias can make the reported parity with the expert optimistic.","rationale":"The paper's central contribution is that classifier-free guidance enables offline adaptation to velocity-tracking rewards without ground truth commands. The strongest quantitative evidence is Table I, where DDPM and DDIM samplers reach 0.84-0.86 average reward versus 0.87-0.90 for the expert. The reader's identified weakest assumption concerns skill interpolation, which is a secondary contribution and only qualitatively verified. I find the more load-bearing issue to be the evaluation protocol: the table caption states results are taken at the lambda value that gave the best rewards. Since lambda is the key hyperparameter controlling the trade-off between reward maximization and dynamics consistency, selecting it on the test reward makes the reported comparison an upper envelope over guidance strengths. This is not necessarily fatal, but it means the main claim is not yet established as a claim about a fixed, reproducible method. The reader's verdict is CONDITIONAL, which remains appropriate; the concern reinforces the need for a cleaner evaluation rather than overturning the approach. The concrete test would settle the issue by re-evaluating with fixed lambda and standard errors.","tokens_in":9722,"tokens_out":7215,"duration_ms":69830,"concrete_test":"Re-run Table I with lambda fixed to 1.5 and 2.0, selected without peeking at the test rewards (e.g., using a validation split of the 100 evaluation episodes), and report per-task means with standard errors over the same environments. Also disclose the lambda values used in the original table. If the fixed-lambda DDPM/DDIM rewards are more than one standard error below the expert for several tasks, the headline parity claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section IV-B's headline comparison (Table I) is reported with results 'taken at the value of lambda that gave the best rewards' (table caption). Because the rows for our method are evaluated after choosing lambda to maximize the exact reward that is the evaluation metric, the comparison to the expert is not a fair test of a fixed hyperparameter configuration. It is an oracle selection over guidance strength, and the paper does not report which lambda values were chosen for each of the four reward functions or whether any held-out episodes were used. This directly affects the strongest claim: without this selection, the DDPM/DDIM rows may fall below the expert, and the claim of comparable velocity tracking would be weakened. The concern is independent of the skill-interpolation question; it targets the quantitative evidence for the offline-adaptation contribution.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a diffusion-based policy for quadruped locomotion trained entirely offline on datasets collected by two RL policies, one for walking and one for crawling. The policy is a transformer denoiser trained with return conditioning and classifier-free guidance; at inference, the return is set to its maximum and CFG adapts the generated action trajectories to a velocity-tracking reward defined after data collection. The authors evaluate the method in simulation for four target velocities against an expert policy conditioned on ground-truth velocity commands (Table I), ablate guidance strength and return-temperature/scaling (Sections IV-D and IV-E), and demonstrate skill transitions and CPU-only inference with three denoising steps on an ANYmal. The central claim is that CFG yields velocity tracking comparable to a command-conditioned expert without ever observing velocity commands at training or inference time.","tokens_in":9926,"tokens_out":6908,"duration_ms":65112,"significance":"The combination of SDE-based diffusion, return-conditioned CFG, multi-skill interpolation, and CPU-only hardware deployment is a useful and timely systems contribution for legged locomotion. The paper is honest about its limitations (no out-of-distribution extrapolation, need for broad data) and releases code and videos. I also agree that the conditioning signal is a return computed from a hand-designed reward rather than a constant fit to the evaluation metric, so this is standard offline RL and not circular. If the comparison in Table I can be made without oracle selection of the guidance strength, the simulation evidence would support the main claim; the skill-interpolation claim, however, needs substantially more methodological and quantitative support.","major_comments":[{"comment":"The caption states that results are 'taken at the value of lambda that gave the best rewards.' Because lambda is a free method hyperparameter and the evaluation metric is exactly the reward used for selection, the reported parity with the expert model is an oracle-selected comparison. Please report the lambda value chosen for each of the four reward functions, choose lambda on a held-out set independent of the test evaluation, and include per-lambda curves for all tasks (Figure 4 provides this only for the forward-velocity reward). This is necessary to support the claim of 'comparable velocity tracking' in Section IV-B.","section":"Section IV-B, Table I"},{"comment":"The claim that the model 'was able to learn interpolations between these two skills' is not adequately supported. The datasets contain 'no transitions present,' and the manuscript never specifies how skill interpolation is implemented at inference: the conditioning includes a one-hot skill vector and no interpolation scheme for it is described. The only evidence is the qualitative snapshots in Figure 3. Please describe the inference-time mechanism and provide quantitative measures of intermediate behavior (e.g., commanded continuous sweeps of body height or velocity, with tracking errors) to distinguish true interpolation from discrete switching.","section":"Section IV-C"}],"minor_comments":[{"comment":"The abstract uses 'adaption' while the title and other sections use 'adaptation'; please standardize the terminology.","section":"Global"},{"comment":"The phrase 'have have' appears in the sentence beginning 'Previous works have have looked at'; please remove the duplicate.","section":"Section IV-C"},{"comment":"In the first paragraph, 'the mode would struggle' should read 'the model would struggle.'","section":"Section IV-E"},{"comment":"The termination axis is not defined; please state whether the plotted quantity is a count, a rate, or a normalized frequency.","section":"Figure 4"},{"comment":"The statement that the return is 'linearly scale[d] between [0,1] based on the batch statistics' is underspecified; state whether these statistics come from the full dataset or from each minibatch and how they are applied at inference.","section":"Section III-C"},{"comment":"The hardware evaluation is only qualitative (snapshots in Figure 3); quantitative hardware tracking metrics would strengthen the deployment claim.","section":"Section IV-C"},{"comment":"The DPM++ (2M) row is clearly outside the expert's error bars, so the 'comparable' conclusion should be explicitly scoped to the DDPM and DDIM samplers rather than stated in broad terms.","section":"Table I"},{"comment":"The remark that the optimal lambda decreases from about 10 to 1.5-2 during training is not supported by any curve or protocol; please provide evidence or remove the observation.","section":"Section IV-D"},{"comment":"The sentence 'While one other prior work has looked at performing Offline RL using CFG' does not explicitly name that work; please add a citation there.","section":"Section II"}],"recommendation":"major_revision","confidential_remarks":"For the editor: the paper is a solid systems contribution in the style of RA-L/ICRA, and the core idea is sound, but the quantitative evaluation needs tightening before journal publication. The main risk is the lambda-selection procedure; if the authors cannot provide a held-out selection, the parity claim may not survive. The skill-interpolation claim is also under-specified both methodologically and quantitatively. I do not see novelty-disclosure issues; the related work appears adequately covered."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a genuine empirical contribution—first CFG-guided diffusion policy for quadruped locomotion with offline reward adaptation and onboard CPU inference—but the headline \"comparable to expert\" claim in Table I rests on oracle selection of lambda, and the skill interpolation is only shown qualitatively. Neither flaw sinks the paper; both need addressing before the strongest claims are taken at face value.\n\nWhat's new and good: they build on DiffuseLoco but move from DDPM to SDE-based sampling, add classifier-free guidance to steer toward velocity-tracking rewards after training, and show 3-step deterministic sampling runs on the ANYmal's CPU. The ablations of lambda and return scaling are informative. The authors are candid that they cannot extrapolate out-of-distribution, which is the right scope. The central idea—reward-guidance as a post-hoc adapter for a multi-skill diffusion policy—is well motivated and the hardware results support that it works.\n\nSoft spots: Table I reports results at the lambda that gave best rewards for each reward function. That's selecting a hyperparameter on the test metric, so the comparison to the expert is optimistic. They don't report which lambdas were chosen or whether they used held-out episodes. This doesn't make the result false, but it does weaken the \"comparable\" claim. They also didn't compare to standard offline RL baselines (IQL, CQL, or a return-conditioned diffusion policy without CFG), so the advantage of CFG over simpler conditioning is not quantified. Section IV-C's skill interpolation claim is based on a qualitative figure; the data had no transitions between walk and crawl, so claiming the score function bridges disjoint modes needs more than one trajectory plot. Minor: the DPM++ sampler is much worse, which is fine but unexplained.\n\nThe math is standard diffusion theory; no red flags. Self-citation to their own RL policy [35] for data collection is legitimate—it's their data generator and is standard practice.\n\nVerdict: worth a serious referee. The paper is scoped, reproducible (code and videos), and the core capability is practically interesting. A revision should fix the lambda-selection reporting and add at least one non-CFG baseline. This is a reasonable conference submission, not a breakthrough.","headline":"Useful empirical combo of CFG diffusion and quadruped hardware, but the Table I comparison is cherry-picked over lambda and the skill interpolation is only qualitative; worth reviewing nonetheless.","tokens_in":10428,"tokens_out":1857,"would_cite":true,"duration_ms":16359,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Classifier-free guidance lets a diffusion policy adapt a quadruped to new velocity commands after training, entirely on the onboard CPU.","keywords":["diffusion models","quadruped locomotion","classifier-free guidance","offline reinforcement learning","skill interpolation","ANYmal","score-based generative models","offline adaptation"],"falsifier":"Run the model over a skill switch while recording base height and foot-contact phase, and check whether the generated trajectories form a continuous bridge between walk and crawl; a bimodal distribution with an empty gap between the two modes would falsify the interpolation claim.","tokens_in":9523,"feed_emoji":"🤖","tokens_out":6781,"duration_ms":61454,"temperature":0.7,"pith_summary":"This paper argues that a single diffusion-based policy can do two things at once for quadruped locomotion: switch between distinct gaits (walking and crawling) with smooth interpolation, and adapt to new velocity-tracking rewards after training without retraining or access to the original velocity commands. The adaptation is achieved by treating a discounted return computed from the robot's velocity as a conditioning variable and applying classifier-free guidance at inference time to push sampled action trajectories toward maximum-return behaviour. On the ANYmal platform the guided policy reaches velocity-tracking rewards comparable to an expert model that was given the true commands during training, while running entirely on the onboard CPU with three denoising steps. The significance is that the same trained model can be reshaped at test time, removing the need for hierarchical skill policies or relabelled datasets.","feed_headline":"Diffusion policy adapts quadruped gaits offline, no retraining","feed_subtitle":"Reward guidance matches an expert that saw true velocity commands, using 3 denoising steps on the robot's CPU.","key_machinery":"The central object is the score function of a diffusion model over action trajectories, trained with denoising score matching and cast in the k-diffusion SDE framework so that a probability-flow ODE enables fast sampling. The mechanism that carries the adaptation claim is classifier-free guidance: the conditioning variable y is defined as a discounted sum of future velocity-tracking rewards, exponentially rescaled and normalised to [0,1], and the model is trained with y randomly masked so that one network can provide both ∇x log p(x) and ∇x log p(x|y). At test time the user sets y = 1 and chooses λ above 1 to trade dynamics consistency for reward maximisation; the paper's ablations show λ ≈ 1.5–2 gives the best reward with fewest terminations.","core_discovery":"On its own terms, the paper's central claim is that classifier-free guidance turns an unlabelled offline dataset of quadruped trajectories into a policy that can be adapted after training to a new reward. The method labels the dataset with a velocity-tracking reward, scales the discounted returns to [0,1], randomly masks the return conditioning during training so the model also learns the unconditional distribution, and then at test time sets the return to 1 and uses λ in the guidance formula to steer generation. With the DDIM sampler needing only three denoising steps, the resulting policy tracks commanded velocities at levels statistically indistinguishable from an expert that was conditioned on the true commands, and does so on real hardware. The paper further claims that the same model interpolates between walking and crawling despite the two modes having been collected as separate datasets with no transitions, and that this interpolation survives when reward guidance is applied.","pith_inferences":["The paper does not test whether the same test-time steering works when the target reward is far outside the training return distribution; its own conclusion says the model cannot extrapolate, so a natural next check is how gracefully performance degrades as the target approaches the distribution's edge.","If the skill-interpolation claim holds in general, the same mechanism could be used to blend more than two gaits or to morph between speeds continuously, potentially replacing gait schedulers in existing locomotion stacks.","The return-labelling scheme suggests a broader recipe: any offline locomotion dataset, including motion-capture clips without command labels, could be post-hoc steered toward user-defined objectives by computing a reward and applying the same guidance.","Since λ is chosen after training, one could vary λ online based on terrain or operator input, using the same weights for conservative and aggressive behaviour."],"forward_implications":["A single diffusion policy can handle multiple locomotion skills, so hierarchical skill-switching layers become unnecessary for gaits with similar state distributions.","New locomotion behaviours can be specified after training by designing a reward function and scaling its returns, without collecting new data or retraining.","Because the model runs in three denoising steps with deterministic sampling, deployment on quadruped hardware does not require a GPU.","Guidance strength is a test-time dial: changing λ from 1.5 to 5 trades reward for stability, giving the operator a live safety and performance knob.","Return-distribution scaling is a first-order design choice; a properly normalised, broad return range is necessary for guidance to work without falls."],"supporting_citations":[{"why":"Supplies the classifier-free guidance equation the method uses to steer trajectories toward high-return behaviour.","marker":"[14]"},{"why":"Establishes the return-as-conditioning paradigm for offline decision making that this paper applies to quadruped locomotion.","marker":"[15]"},{"why":"Provides the preconditioning and ODE framework that lets the model sample in three denoising steps.","marker":"[25]"},{"why":"Is the prior diffusion-on-quadruped deployment this work extends with reward guidance and faster SDE sampling.","marker":"[9]"},{"why":"Motivates predicting multi-step action trajectories for temporal consistency and flexible conditioning.","marker":"[13]"},{"why":"Supplies the score-based generative formulation and Langevin sampling used as the base model.","marker":"[12]"},{"why":"Provides the hand-crafted reward function used to train the data-collection policies for walking and crawling.","marker":"[35]"},{"why":"Defines the DDPM sampler used as one of the three inference-time samplers in the comparison.","marker":"[8]"}],"fun_headline_variants":["Quadruped gaits adapt offline via diffusion, no retraining","Diffusion lets quadrupeds learn new gaits without retraining","Offline diffusion teaches quadrupeds new locomotion skills","No retraining: diffusion adapts quadruped locomotion offline","Diffusion-based offline adaptation for quadruped gaits"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the score function can bridge two disjoint modes in trajectory space: the training data contains separate walking and crawling trajectories with no transitions between them, yet the model is claimed to interpolate continuously and stably between the two skills on hardware.","fun_headline_variants_meta":{"raw":{"variants":["Quadruped gaits adapt offline via diffusion, no retraining","Diffusion lets quadrupeds learn new gaits without retraining","Offline diffusion teaches quadrupeds new locomotion skills","No retraining: diffusion adapts quadruped locomotion offline","Diffusion-based offline adaptation for quadruped gaits"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000229,"raw_usage":{"total_tokens":1417,"prompt_tokens":822,"completion_tokens":595,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":438,"completion_tokens_details":{"reasoning_tokens":512}},"tokens_in":438,"tokens_out":595,"duration_ms":5652,"temperature":1.0,"reasoning_tokens":512,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T21:17:09.653718+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the model over a skill switch while recording base height and foot-contact phase, and check whether the generated trajectories form a continuous bridge between walk and crawl; a bimodal distribution with an empty gap between the two modes would falsify the interpolation claim.","supporting_citations":[{"cited_title":"Is conditional generative modeling all you need for decision- making?","cited_arxiv_id":null,"evidence_quote":"Establishes the return-as-conditioning paradigm for offline decision making that this paper applies to quadruped locomotion."},{"cited_title":"Generative modeling by estimating gradients of the data distribution,","cited_arxiv_id":null,"evidence_quote":"Supplies the score-based generative formulation and Langevin sampling used as the base model."},{"cited_title":"Learning low- frequency motion control for robust and dynamic robot locomotion,","cited_arxiv_id":null,"evidence_quote":"Provides the hand-crafted reward function used to train the data-collection policies for walking and crawling."}],"review_version":1}