{"id":"7d071113-ac99-4aef-904b-ab9fba18dc31","arxiv_id":"2505.08367","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"MA-ROESL cuts training time for learning quadruped gaits from single videos by about 68% using motion-aware frame selection and offline-to-online RL.","lead":"This paper speeds up robot skill learning from a single video by picking motion-rich frames and adding an offline reinforcement learning phase before online fine-tuning. On four quadruped gaits, training time drops by about 68% while the robot reproduces the gaits in simulation and in the real world.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 68.67% training-time reduction is not supported as reported: Algorithm 1's Phase 1 trains a full PPO policy per candidate reward, and Table II gives no per-phase breakdown or values for K1, K2, N1, N2, so the timed protocol behind t_MA-ROESL is unverified.","rationale":"The paper's stated contribution is training efficiency, and Table II is the quantitative evidence for that claim. Before considering whether offline reward ranking matches true fitness, the field has to know what time is being measured. If Phase 1 training is included and is comparable to SDS's per-reward evaluation, the claimed 68.67% reduction cannot hold by the paper's own algorithm, since Phase 1 performs the same full PPO training that SDS performs, and Phases 2 and 3 add extra work. This is not speculation about hidden failures; it is a missing accounting of the exact quantity in the paper's headline result. The same-VLM-as-judge and offline-ranking issues are worth checking, but they affect the quality of the selected reward, not whether the timing comparison is meaningful. The paper deserves credit for reporting real-robot deployment and DTW/contact-pattern analyses, but the efficiency claim needs a per-phase breakdown and hyperparameter disclosure before it can be accepted.","tokens_in":9546,"tokens_out":4405,"duration_ms":45528,"concrete_test":"Request the authors' full wall-clock breakdown for each skill: (a) Phase 1 total time including every TrainAndCollect call, (b) Phase 2 IQL time, (c) Phase 3 fine-tune time, and (d) the equivalent breakdown for the SDS baseline, along with K1, K2, N1, N2, and PPO steps per policy. Recompute Table II's reduction with Phase 1 included. If t_MA-ROESL excludes Phase 1 or uses fewer or faster PPO runs than SDS, the reduction is an artifact of protocol, not of the method.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central efficiency claim rests on Table II, but the paper never states what is included in t_MA-ROESL. In Algorithm 1, Phase 1 (lines 5-14) runs TrainAndCollect(M, R_i) for every candidate reward R_i, with PPO as the online algorithm; Phase 2 additionally trains an IQL policy per relabeled reward (lines 16-25); Phase 3 runs online fine-tuning. Total MA-ROESL wall time therefore contains at least K1*N1 full PPO trainings plus K2*N2 IQL trainings plus a fine-tune, whereas the SDS baseline presumably runs K full PPO trainings. Unless Phase 1 is much cheaper than SDS runs (fewer candidate rewards, fewer PPO steps, or is excluded from Table II), the 68.67% reduction is not an apples-to-apples comparison. The hyperparameters needed to check this - K1, K2, N1, N2, and number of PPO steps per policy - are absent from Table I, and the text 'executed over four iterations' does not disambiguate N1 and N2. The offline-ranking assumption identified by the reader is secondary: even if phase-2 ranking were perfect, the time saved could be an artifact of an unfair timing protocol.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MA-ROESL, a framework for learning quadruped locomotion skills from a single video using vision-language models (VLMs). It introduces motion-aware frame selection based on optical flow to choose behaviorally salient frames for the VLM, and a three-phase training pipeline: (1) train PPO policies for VLM-generated candidate rewards and collect an offline dataset; (2) use IQL on relabeled rewards to rapidly select the best reward-policy pair; (3) fine-tune the selected policy online. The authors report a 68.67% average reduction in training time relative to the SDS baseline across trot, pace, bound, and hop, lower DTW motion-alignment values, and successful zero-shot sim-to-real deployment on a Unitree Go2.","tokens_in":9865,"tokens_out":2411,"duration_ms":25298,"significance":"If the reported efficiency gains are real, the paper addresses an important practical bottleneck in video-based reward learning for legged robots: the cost of evaluating many candidate rewards by full online training. The pipeline idea of using an offline dataset and IQL to rank rewards before online fine-tuning is timely and plausible, and the paper provides a clear problem formulation, explicit pseudocode, and a real-robot deployment, which are strengths. However, the central efficiency claim rests on an incompletely specified timing protocol and on single-number comparisons without statistical support, and the skill-quality evidence is largely qualitative. The significance is therefore conditional: the framework is potentially useful, but the current evidence does not substantiate the headline numbers.","major_comments":[{"comment":"The timing protocol behind t_MA-ROESL is not defined, so the 68.67% reduction is not verifiable. Algorithm 1 phase 1 runs TrainAndCollect(M, R_i) for K1 reward functions per iteration for N1 iterations, i.e., K1*N1 full PPO trainings, each with 4000 environments; phase 2 runs K2*N2 IQL trainings; phase 3 runs an additional online fine-tune. The SDS baseline presumably also trains full PPO policies for each candidate reward. Unless phase 1 uses far fewer policy updates than the SDS runs, or phases are excluded from t_MA-ROESL, the comparison is not apples-to-apples. The paper never states which components are included in t_MA-ROESL, nor the values of K1, K2, N1, N2 or the number of PPO steps per policy. Please provide a per-phase timing breakdown and all protocol hyperparameters.","section":"Algorithm 1, Section V-C.1, Table II"},{"comment":"Table II reports a single training time per skill and per method, with no standard deviation, no number of seeds, and no statistical test. The claimed reduction of 63--73% is therefore a point estimate whose reliability is unknown. Since the main contribution is an efficiency claim, the evaluation needs multiple runs (at least 3--5 seeds) for both t_MA-ROESL and t_SDS, with the mean and spread reported.","section":"Table II, Section V-C.1"},{"comment":"The SDS baseline is not described: its environment code, reward prompt, frame sampling method, training algorithm settings, and hardware are not given. The paper only says SDS is the state-of-the-art method of [7]. Without a description of the baseline configuration, the reader cannot tell whether the time reduction reflects the proposed method or differences in implementation (e.g., number of reward candidates, PPO steps, or hardware). Please specify the exact SDS protocol used for the numbers in Table II.","section":"Section V-A, Section V-C.1"},{"comment":"The reward selection loop is self-referential: GPT-4V both generates candidate rewards and evaluates rollouts to select the best reward, in both phase 1 and phase 2. Since the same VLM is judge and generator, the selected reward may simply reflect the VLM's prior preference rather than actual task fitness F under the true MDP. The paper should discuss this circularity and provide a control experiment, e.g., comparing VLM-selected rewards against an independent fitness metric (such as task success rate or gait metrics) or against random selection.","section":"Section IV-B.2, Algorithm 1 lines 13 and 24"},{"comment":"The offline ranking assumption is load-bearing: phase 2 assumes that IQL policies trained on relabeled offline dataset D rank reward functions in the same order as their true fitness F after online training. The paper does not justify or test this assumption. A direct test would be to take the rewards selected by phase 2, train them fully online, and compare their final performance against rewards that phase 2 ranked lower; without such evidence, the efficiency gain could be an artifact of the offline ranking being wrong, with the final fine-tune masking the error.","section":"Section IV-B.2, Eqs. (3)--(4)"},{"comment":"The skill-quality evaluation is qualitative: Fig. 4 shows DTW curves but no numerical DTW values or confidence intervals, and the claim that MA-ROESL has 'a lower DTW value' is not backed by numbers in the text or table. Similarly, Figs. 5--7 show contact patterns and joint angles but no quantitative success metric (e.g., percentage of steps with correct footfall sequence, cycle frequency error, or velocity tracking error). Please add quantitative metrics with uncertainty for both the simulation and real-world deployment.","section":"Section V-C.2, Figs. 4--7"}],"minor_comments":[{"comment":"Equation (3) is written as 'f = F Mun(π)' with inconsistent notation: the function name F appears on both sides, and the meaning of 'Mun' as the unknown MDP is not typeset clearly. Please rewrite to make the mapping explicit, e.g., f(π) = F_{M_un}(π).","section":"Section III, Eq. (3)"},{"comment":"Equation (6) defines K_motion as a set of frame indices but the text below says 'K is the target number of motion-salient frames to be selected'; the paper should clarify whether K is a count or a rank threshold, and how the supplementary uniformly sampled frames are combined with the top-K selection.","section":"Section IV-A, Eq. (6)"},{"comment":"Table I lists PPO and IQL hyperparameters but omits the new method-specific parameters K, K1, K2, N1, N2, and the offline dataset size. These are needed to reproduce the experiments and to interpret the timing numbers.","section":"Section V-B, hyperparameters"},{"comment":"The sentence about the amplitude discrepancy in thigh joints attributes it to 'the limitation of the input' but does not specify which input (the video frame selection, the VLM prompt, or the optical flow) or how this limitation could be mitigated. Please clarify.","section":"Section V-C.2"},{"comment":"The hardware description says the on-board Jetson Orin NX operates 'independently without external computational support,' but it is unclear whether this refers to running the trained policy or to training; training is stated to run on the workstation. Please disambiguate.","section":"Section V-A"}],"recommendation":"major_revision","confidential_remarks":"The paper's central claim is an efficiency gain, but the reported gain is not yet supported because the timing protocol is not fully specified, the comparison baseline is not described, and there are no repeated runs. The circularity of using the same VLM for generation and selection is a real methodological concern that should be addressed with an independent evaluation or a control. These issues are fixable within the scope of a revision, so I do not recommend rejection, but the current version is not ready for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear Colleague,\n\nQuick take: MA-ROESL is a reasonable incremental contribution—optical-flow frame selection plus an offline IQL screening phase before online fine-tuning for VLM-generated locomotion rewards. The qualitative results (DTW curves, gait diagrams, real-robot deployment) suggest the pipeline works. But the central efficiency claim—68.67% average training-time reduction over SDS—is not supported by the evidence as reported.\n\nWhat's new: the specific three-phase combination is not in the SDS paper, and the motion-aware sampling is a sensible, cheap fix to uniform frame sampling. They also try to address the real cost of VLM reward optimization by moving reward ranking to offline RL. That's a legitimate idea.\n\nWhere it gets soft. Table II reports single numbers without seeds or error bars. More importantly, the paper never states what goes into t_MA-ROESL. Algorithm 1's Phase 1 trains a full PPO policy for every candidate reward, exactly like SDS does; Phase 2 adds IQL trainings per relabeled reward; Phase 3 fine-tunes online. Unless Phase 1 is much cheaper than SDS (fewer candidate rewards, fewer PPO steps, or excluded from the table), the total wall time should be higher, not lower. The reader's concern about the same VLM (GPT-4V) generating and judging rewards is real but secondary—even if the ranking were perfect, we can't verify the time saving without a breakdown. There are also no ablations isolating the frame-selection contribution, and no code or data release.\n\nIs the central argument dead? Not necessarily. The offline-ranking idea could work, and the qualitative motion alignment is encouraging. But as written, the efficiency claim is unverifiable. The paper needs per-phase timing, hyperparameters K1, K2, N1, N2, PPO steps, seeds, and an apples-to-apples baseline description.\n\nWho's it for: people working on VLM-based reward generation or learning-from-video for locomotion. They'd get a useful design suggestion and a cautionary tale about evaluation protocol.\n\nRecommendation: worth sending to peer review—a solid reviewer could force the authors to pin down the timing and add error bars. It's not a desk-reject, but it needs major revisions before the efficiency claim is credible.","headline":"Plausible incremental pipeline for VLM-based skill learning, but the headline efficiency gain rests on an unspecified timing protocol and single-run numbers.","tokens_in":10399,"tokens_out":3412,"would_cite":false,"duration_ms":30041,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Motion-aware frame selection and offline reward screening let a quadruped learn gaits from one video with 68.67 percent less training time.","keywords":["robot skill learning","vision-language models","reward design","offline reinforcement learning","sim-to-real transfer","quadruped locomotion","motion-aware frame selection","single video demonstration"],"falsifier":"Run the same set of candidate reward functions through both the phase-2 offline relabeling procedure and full online training, then evaluate every candidate online; if the offline-selected reward is consistently outperformed by an online-selected reward, the central efficiency claim fails.","tokens_in":1360,"feed_emoji":"🤖","tokens_out":4359,"duration_ms":93737,"temperature":0.7,"pith_summary":"The paper tries to establish that a quadruped robot can learn a locomotion skill from a single video demonstration with far less training time than existing methods, while still reproducing the demonstrated gait. Its proposal, MA-ROESL, changes the video-to-reward pipeline in two ways: it feeds a vision-language model frames selected by motion salience rather than uniform sampling, and it screens candidate rewards with offline reinforcement learning before a short online fine-tuning stage. On trot, pace, bound, and hop, the authors report an average 68.67 percent reduction in training time compared to the SDS baseline, with lower dynamic-time-warping mismatch to the demonstrations. The learned policies transfer zero-shot to a physical quadruped. If the claim holds, video-only skill acquisition becomes a practical route for legged locomotion rather than a costly research exercise.","feed_headline":"Single-video robot gait training cut by 68.67%","feed_subtitle":"Motion-aware frames and offline reward screening reproduce trot, pace, bound, and hop with zero-shot sim-to-real transfer.","key_machinery":"The load-bearing object is a motion-aware frame-selection score, the average dense optical-flow magnitude between consecutive frames, $$\\sigma_k = \\frac{1}{W H}\\sum_{i,j} \\|d_k(i,j)\\|_2,$$ which picks frames that capture the stance-swing transitions that uniform sampling misses. The second mechanism is reward relabeling in an offline dataset: phase-1 trajectories are relabeled with each candidate reward, an offline policy is trained per relabeled dataset, and the vision-language model evaluates the resulting rollouts. This replaces several full online training runs with one offline screening pass before the final online fine-tuning stage.","core_discovery":"The central claim is that the bottleneck in learning skills from a single video is not the vision-language model's ability to propose useful rewards, but how frames are sampled and how expensive reward evaluation is. MA-ROESL claims that a motion-aware frame selection step, based on dense optical flow, yields reward functions that better reflect stance-swing transitions, and that a three-phase pipeline—online data collection, offline reward relabeling with Implicit Q-Learning for cheap screening, then online fine-tuning—lets the method identify the best reward faster. The supporting evidence is the training-time table (16.83 to 6.15 hours for trot, 12.20 to 3.64 for pace, 13.10 to 4.16 for bound, 15.83 to 4.21 for hop), lower DTW alignment than SDS, matching contact patterns across the four gaits, and deployment of the policies on a physical quadruped.","pith_inferences":["A cheap test implied by the paper is to take one candidate reward and compare its offline relabeling ranking against its full online fitness; the efficiency argument predicts they agree, and a reader can check this directly.","Motion-aware sampling likely matters most for gaits with distinct stance and swing phases; for smooth, continuous motions the gain could shrink, so the four-skill result invites testing on non-periodic or long-horizon videos.","If offline reward screening were replaced by an even cheaper learned critic, the three-phase pipeline could be compressed further; the paper does not explore that route.","The reported times are per-skill on a single GPU workstation, so replication across other simulators and robot platforms would determine how much of the gain comes from offline screening speed versus the motion-aware frames."],"forward_implications":["A single video can serve as the only demonstration source for learning a new quadruped gait, with training time short enough for iterative reward experimentation.","The 68.67 percent reduction makes it practical to compare many vision-language-proposed rewards in simulation before committing to online training, so reward search can be broader at the same compute budget.","Motion-aware frame selection is task-agnostic within the pipeline, so any vision-language-based reward generator can ingest better input frames without extra annotation.","Successful zero-shot deployment on a physical quadruped suggests policies trained this way preserve the dynamics of the demonstrated skill, not just its appearance."],"supporting_citations":[{"why":"Defines the SDS baseline whose online reward-evaluation loop MA-ROESL accelerates and whose DTW comparison protocol is reused.","marker":"[7]"},{"why":"Supplies Implicit Q-Learning, the offline RL algorithm used in phase 2 to relabel rewards and train candidate policies quickly.","marker":"[38]"},{"why":"Supplies PPO, the online RL algorithm used for phase-1 data collection and phase-3 fine-tuning.","marker":"[37]"},{"why":"Supplies the vision-language backbone used to generate and evaluate candidate reward functions from the selected frames.","marker":"[39]"},{"why":"Supplies the dense optical-flow method used to compute per-frame motion scores in motion-aware frame selection.","marker":"[34]"},{"why":"Provides the offline RL rationale that reward evaluation can be done without expensive environment interaction.","marker":"[29]"},{"why":"Supports the claim that online fine-tuning after offline pretraining improves robustness and generalization.","marker":"[36]"}],"fun_headline_variants":["MA-ROESL: single-video robot gaits 2.7x faster","Motion-aware frames cut single-video robot training by 68%","Offline reward screening cuts robot skill training time","Zero-shot sim-to-real: robot gaits from one video"],"cache_read_input_tokens":12544,"weakest_assumption_plain":"The offline dataset gathered in phase 1 must be rich enough that relabeling it with a candidate reward ranks candidates in the same order as full online training would; if that ranking is wrong, the accelerated screening selects a suboptimal reward and the efficiency gain is an artifact of the protocol.","fun_headline_variants_meta":{"raw":{"variants":["MA-ROESL: single-video robot gaits 2.7x faster","Motion-aware frames cut single-video robot training by 68%","Offline reward screening cuts robot skill training time","Zero-shot sim-to-real: robot gaits from one video"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001301,"raw_usage":{"total_tokens":5285,"prompt_tokens":900,"completion_tokens":4385,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":516,"completion_tokens_details":{"reasoning_tokens":4310}},"tokens_in":516,"tokens_out":4385,"duration_ms":29991,"temperature":1.0,"reasoning_tokens":4310,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:56:12.221955+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same set of candidate reward functions through both the phase-2 offline relabeling procedure and full online training, then evaluate every candidate online; if the offline-selected reward is consistently outperformed by an online-selected reward, the central efficiency claim fails.","supporting_citations":[{"cited_title":"Gpt-4v(ision) system card,","cited_arxiv_id":null,"evidence_quote":"Supplies the vision-language backbone used to generate and evaluate candidate reward functions from the selected frames."}],"review_version":1}