{"id":"8aafc471-4a7e-4446-b3eb-70daa715ccc7","arxiv_id":"2505.02094","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"Stitching demonstration trajectories and augmenting state neighborhoods with masked buffers lets reinforcement learning acquire robust, generalizable interaction skills from sparse noisy demonstrations.","lead":"This paper presents data augmentation and training techniques that let physics-simulated characters learn robust interaction skills from sparse, noisy demonstrations. It reports large gains in skill transitions and generalization on basketball and household tasks, which could make learning-from-demonstration methods more practical.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The core augmentation claim rests on an untested assumption: Eq. 20 caps masked-buffer bridges at 10 simulation steps, yet no experiment verifies the policy reaches the visible target state within that budget.","rationale":"The reader's weakest assumption identifies the same load-bearing point: STG/STF depend on the ability to repair stitched or neighborhood-perturbed trajectories through masked-buffer inpainting, and this is asserted rather than demonstrated. My reading sharpens this into a quantitative, checkable claim: the buffer length is set by a static kinematic similarity formula with a hard cap of 10 frames, and no experiment verifies that the learned policy can physically reach the visible target state within that budget. This matters because the ablation attributes the largest performance gains to STG and STF; if the bridge timing is wrong for a substantial fraction of edges, the mechanism as described would not produce the reported transition and generalization improvements. I do not see this as a reason to reject the paper: the empirical results are broad and internally consistent, and the concern is addressable with diagnostic experiments and a sensitivity analysis over Nmax. The reader already conditioned the verdict on exactly this kind of missing evidence, so my assessment does not move the verdict. I also considered whether the 'zero-shot' transition claim is the more serious issue, since STG explicitly trains on stitched transitions and TSR then tests those same types of edges; that is a real overstatement, but it concerns interpretation of the metric rather than the validity of the core learning mechanism, and the reader already flagged it. The masked-buffer reachability assumption is more load-bearing because STF/STG are the paper's main novelty and the mechanism by which the reported empirical gains are supposed to arise.","tokens_in":20919,"tokens_out":6606,"duration_ms":100153,"concrete_test":"Instrument the authors' implementation (or a minimal re-implementation on BallPlay-M) to record, for every STF/STG training sample, the simulated state at the first visible frame after the N masked steps; compute the fraction of samples where that state lies outside the epsilon tolerance (Eq. 5) relative to the reference state s_j. Repeat the full training with Nmax = 5, 10, 20, and 40, and also with the masked states removed so reward is applied directly from s_new to s_j. If the failure fraction is large, or if TSR/epsilon-NSR shifts materially with Nmax, then the masked-buffer inpainting mechanism is not doing the work attributed to it and the Tab. 3 ablation attributions need reinterpretation. If the policy reliably reaches s_j within tolerance and the results are Nmax-invariant, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim depends on STF/STG's masked-state bridging: a sampled or stitched start state s_new is connected to a reference state s_j by N masked frames, and RLID is expected to 'inpaint' the missing transition using imitation reward only on visible states (Eq. 7, Sec. 4.3.2, and B.4). The number of buffer frames is set by N = min(-floor(log10(beta)), Nmax) with Nmax=10 (0.167 s at 60 Hz), where beta is a kinematic similarity score (Eq. 19). Nothing in the paper checks that this is enough time for the physics simulator to move the humanoid and object from s_new to within epsilon of s_j. For STG edges, beta can be as low as the validity threshold tau=1e-10, meaning near-zero kinematic similarity is still accepted with only 10 masked frames. If the minimal feasible transition time exceeds N for many edges, those training clips have unachievable targets at the first visible frame, and the reported TSR and epsilon-NSR gains cannot be attributed to the claimed inpainting mechanism. Because the physical simulator does guarantee that generated trajectories are physically plausible, the unvalidated part is not feasibility of the rollout but reachability of the visible reference state within the fixed buffer. The ablation shows STG and STF are the largest contributors (Tab. 3: SM+STF=68.67% SR, SM+STG=74.74% vs SM=53.30%), so this assumption is load-bearing. The paper provides no error analysis over generated bridge samples, no sensitivity study over Nmax, and no code to inspect whether the learned policy actually closes the gap.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper presents SkillMimic-V2, a data-augmentation and training framework for reinforcement learning from interaction demonstrations (RLID). Given sparse, noisy reference trajectories, the method (i) constructs a Stitched Trajectory Graph (STG) that connects states from different skill demonstrations, and (ii) builds a State Transition Field (STF) that assigns to each state in an epsilon-neighborhood of a reference state a unique directed transition to the most similar reference state, connected by a buffer of N masked (reward-excluded) frames that RLID is expected to 'inpaint.' Training uses Adaptive Trajectory Sampling (ATS) to up-weight hard clips and a pretrained History Encoder (HE) to provide temporal context. Experiments on the BallPlay-M basketball dataset and the ParaHome household dataset report large gains over the SkillMimic (SM) and DeepMimic (DM) baselines: on BallPlay-M, average SR rises from 53.3% to 96.9% and average TSR from 15.1% to 93.8%. Ablations (Tab. 3) attribute most of the gain to STG and STF, and additional experiments cover data noise, data efficiency, locomotion transfer, and in-hand reorientation.","tokens_in":21337,"tokens_out":18094,"duration_ms":198441,"significance":"If the reported results are reproducible, this is a substantial contribution: the masked-buffer idea is a clean mechanism for converting sparse interaction demonstrations into dense transition coverage, and the evaluation is unusually broad (two datasets, per-component ablations with negative controls SM+HS and SM+IAE, noise sweeps, data-efficiency curves, locomotion transfer, and a zero-shot reorientation application). The paper also states its limitation with heavily corrupted demonstrations in the conclusion. However, the significance is bounded by three gaps: the core inpainting mechanism rests on an unverified reachability assumption about the masked buffer (Sec. 4.3.2 and B.4); the headline TSR metric is measured on transition classes that STG itself injects into training, so the 'zero-shot' framing overstates what is tested; and all results are single-seed with no code release, so the claims are not yet independently checkable. These gaps are fixable but essential.","major_comments":[{"comment":"The masked-buffer bridge length is N = min(-floor(log10(beta)), N_max) with N_max = 10 (Eq. 20, Tab. 9), i.e., at most 10 policy steps (about 0.167 s at 60 Hz), and the validity threshold is tau = 1e-10 (Tab. 9), so near-dissimilar STG edges are accepted with the full 10-frame buffer. The reward is computed only on visible states (Eq. 2; Sec. 4.3 states masked states are 'excluded from reward computation'), so if the physics simulator cannot move the humanoid and object from the sampled start state s_new to the visible reference state s_j within N steps, the first visible reward is near zero and the bridge supplies no learning signal; yet the paper reports no reachability check, no sensitivity study over N_max, and no error analysis of the generated bridge samples. Because Tab. 3 attributes the largest gains to STF and STG (SM+STF: 68.67% SR, SM+STG: 74.74% SR vs. SM: 53.30% SR), this mechanism is load-bearing, and the present evidence does not establish that the gains come from inpainting rather than from simple exposure to the target skill's own frames. I ask for (i) a sensitivity sweep over N_max (e.g., 2, 5, 10, 20, 40), (ii) a distributional analysis of achievable transition times for randomly sampled STG/STF edges under the simulator, or (iii) an explicit reachability filter at data-generation time.","section":"Sec. 4.3.2, Eq. (20), Sec. B.4"},{"comment":"TSR is evaluated on the inter-skill pairs DL-DR, DF-DR, DF-Shot, and Layup-DFA (Tab. 1), which are exactly the classes of transitions that STG constructs during training: Sec. 4.4 and Algorithm 1 connect states from every other skill to each skill A, and Tab. 3 shows that STG alone raises TSR from 15.11% to 71.67%. The label 'zero-shot skill transition' (Fig. 3) is therefore misleading: the transitions are untrained with respect to the original demonstrations but trained-for with respect to the augmented corpus, and the only held-out variation is the epsilon-NSI perturbation. Please either evaluate TSR on transitions whose source skill was excluded from STG construction during training, or explicitly re-frame TSR as generalization to unseen instances of trained transition classes.","section":"Sec. 5.1, Tab. 1, Sec. 4.4"},{"comment":"All tables report a single number per entry with no standard deviation and no statement of the number of training seeds. The 10,000-trial evaluation (Sec. 5.1) averages over initial states only and does not capture the variance of PPO training, which is generally the dominant source of variance in this setting; entries such as the epsilon-NSR gap between SM+STG (28.91%) and SM+STF (36.96%) in Tab. 3, or the NR column in Tab. 1, are not interpretable without that variance. No code or pretrained models are provided. I request mean +/- std over at least three seeds for the main tables (Tabs. 1-3), or, failing that, a code release with a documented evaluation script so that the results can be reproduced independently.","section":"Sec. 5.2, Tabs. 1-7"},{"comment":"The paper uses 'theoretically' for both the coverage claim ('random initialization within the entire epsilon-neighborhood theoretically ensures complete coverage of states in S', Sec. 4.2) and the failure of naive NSI ('convergence issues both theoretically and empirically', Sec. 4.3), and asserts that the benefit of ATS follows 'as implied by Eq. 1,' but no formal statement or proof is given anywhere in the paper or appendix. The two claims are also in tension: if naive NSI fails because overlapping neighborhoods create non-unique transition targets, the uniqueness imposed by the STF connection rule (Eq. 6) must be shown to resolve that failure, not merely asserted. Please either provide a formal analysis of an idealized setting or remove the theoretical framing and label these arguments as heuristic motivations.","section":"Sec. 4.2-4.3"}],"minor_comments":[{"comment":"Sec. 1 contains 'noisy and parse' (should be 'sparse'), the Sec. 5.3 heading reads 'Datase' (should be 'Dataset'), and the ACM reference block spells the author name 'Hok Wai Tusi' while the author list spells it 'Hok Wai Tsui'.","section":"Sec. 1, Sec. 5.3, author block"},{"comment":"In the provided text the table entries run together (e.g., 'DM + Ours83.2' and '0.0993.4'), making the tables difficult to parse; please ensure clear cell separation in the final version.","section":"Tabs. 1-2"},{"comment":"The text says 'each proposed component yields substantial performance gains,' but adding HE alone decreases epsilon-NSR from 18.26% to 4.33%, and adding ATS to SM+STG+STF slightly lowers SR (77.12 to 76.44) and TSR (73.18 to 70.23); at least one sentence explaining these non-monotonic interactions is needed.","section":"Tab. 3, Sec. 5.4"},{"comment":"The ATS weights use average per-frame rewards r-bar_i, but the text does not state when these rewards are measured (current-policy rollouts vs. one-time estimates) or how they are updated during training; Fig. 3 mentions 'Reward Statistics' but the main text is silent on the update procedure.","section":"Eq. (8)"},{"comment":"The headline '40-50% improvement' is ambiguous between absolute percentage points and relative improvement; in Tab. 1 the average SR increases from 53.3% to 96.9% (+43.6 points, about +82% relative), so please state the convention explicitly.","section":"Sec. 1"},{"comment":"The limitation statement ('our framework shows limitations with heavily corrupted demonstrations') sits in tension with the noise experiments in Tab. 4 (sigma up to 30 mm), which show strong robustness; please clarify what 'heavily corrupted' means relative to the tested sigma range.","section":"Conclusion"}],"recommendation":"major_revision","confidential_remarks":"The reward function (Eq. 2) and the primary baseline (SkillMimic) come from the authors' own earlier work, and with no code release and no seed variance the independent verification burden is higher than usual; I would weigh reproducibility requirements accordingly in the editorial decision. Note also that the manuscript is formatted as a SIGGRAPH '25 paper with a DOI, while the arXiv version under review is dated 4 May 2025; the editorial office should confirm which version is the object of review."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Good paper to know about if you work on imitation learning for physics-based characters. The contribution is a data augmentation and training system: STG stitches short demonstrations into transitions, STF samples neighborhood states and bridges them to reference states with masked buffer frames, ATS reweights hard samples, and a history encoder gives temporal context. The combination is new, and the results are strong: near-perfect SR on BallPlay-M, big TSR jumps, and consistent gains on ParaHome plus noise and data-efficiency tests. The ablations show each piece does something. That is real empirical work.\n\nThe main weakness is that the central mechanism is under-tested. The bridge length is capped at 10 masked frames (0.167 s at 60 Hz), and the similarity threshold accepts connections with beta just above 1e-10. Nothing checks that the policy can actually reach the visible reference state within that budget for low-beta edges. The ablation credits STG and STF with most of the gain, so that assumption is load-bearing. I don't think it sinks the paper—the empirical improvements are broad enough that the method likely works even if the inpainting story is incomplete—but it needs a sensitivity study over Nmax and an analysis of whether bridges are feasible.\n\nOther soft spots: no error bars or multiple seeds, no code release; 'zero-shot' transition is overclaimed since STG stitches those exact transitions into training; and the 'theoretically' language in Sec 4.2–4.3 is not backed by proofs. These are all addressable. The citation pattern is fine; self-citing SkillMimic is legitimate since it is the direct baseline.\n\nI would take this paper seriously. It deserves a proper referee rather than a desk reject. If I were handling it, I would ask for the reproducibility fixes and sharper claims before acceptance, but the core contribution is real.","headline":"A solid, system-level improvement for RLID that earns a real look, but the masked-buffer bridging story is less verified than the ablation numbers imply.","tokens_in":21841,"tokens_out":2974,"would_cite":true,"duration_ms":38458,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Sparse, noisy demonstrations can be turned into generalizable robot interaction skills by augmenting them with a field of physically plausible transitions.","keywords":["Character Animation","Human-Object Interaction","Reinforcement Learning","Manipulation","Data Augmentation","Skill Transition","Imitation Learning","Physics-Based Simulation"],"falsifier":"Take two skill clips whose only physical bridge passes through an obstacle or through a pose unreachable in one simulation step, stitch them with the paper's connection rules, and train the full method; if skill-transition success remains at baseline while the visible-state rewards are high, the inpainting assumption fails. A minimal version: corrupt a middle segment of a single noisy clip, then test whether a policy initialized inside the masked region recovers to within the epsilon tolerance of the reference—if recovery stays near zero, the field has not bridged the gap it was designed to fill.","tokens_in":20746,"feed_emoji":"🤖","tokens_out":10046,"duration_ms":90766,"temperature":0.7,"pith_summary":"The paper sets out to show that the information bottleneck in learning interaction skills from demonstrations is not the noise or sparsity of the data, but the failure to exploit the continuous space of physically feasible trajectories that surrounds any demonstration. Its proposal is to convert sparse clips into a dense training field: a Stitched Trajectory Graph (STG) connects states from different skill clips, and a State Transition Field (STF) assigns every state in an epsilon-neighborhood of the reference a unique transition back to the clip, with masked buffer states absorbing the long jumps. Training on this augmented field, it argues, lets a single Reinforcement Learning from Interaction Demonstration (RLID) policy master skills that never co-occur in the data, recover from perturbed states, and execute longer than the reference clips. The reported results support the claim on basketball and household tasks, with success rates rising from around 53% to 97% on the basketball benchmark and from 5.5% to 100% on the household benchmark.","feed_headline":"Sparse noisy demos can be stitched into reliable robot skills","feed_subtitle":"Stitched trajectory graphs bridge gaps between clips, lifting skill success 40-50% and generalization 35%.","key_machinery":"The load-bearing mechanism is the masked-buffer stitched trajectory: given two states judged similar, the method inserts N masked states as temporal buffers, where N is set by a similarity score, turning a long or physically impossible jump into a missing-data inpainting problem that RLID can solve from the imitation reward on visible states alone. STF builds a field of unique transition directions from every epsilon-neighborhood state back to its most similar reference state, avoiding the convergence ambiguity of naive neighborhood noise; STG expands this field across different skills' clips so transitions absent from the demonstrations become trainable. Adaptive Trajectory Sampling (ATS) reweights training clips according to their per-frame reconstruction reward, so hard segments are sampled more often and the chain break around corrupted frames is repaired. The History Encoder, pretrained by behavioral cloning and frozen during RLID, compresses the past k states into a compact embedding (dimension 3) so that memory-dependent transitions—states that look identical but require different futures—are disambiguated without manual phase specification.","core_discovery":"The central claim is that infinitely many valid but uncaptured trajectories exist around any demonstration—bridging between demonstrated skills or emerging from their neighboring states—and that explicitly constructing them as training data is what separates robust skill acquisition from fragile imitation. The paper asserts that a masked-buffer trajectory, built by sampling an epsilon-neighborhood start, connecting it to its most similar reference state, and inserting mask tokens where transitions are too distant, is a well-posed inpainting problem for RLID: the imitation reward is computed only on visible states and the policy must fill in the masked gaps. On top of this, the Stitched Trajectory Graph (STG) adds transitions between different skills' clips, so skills that never appear together in the demonstrations become chainable, and the State Transition Field (STF) ensures each neighborhood state has a unique transition direction to avoid ambiguous convergence. The paper reports that this combination yields 96.9% average success rate and 93.8% average skill-transition success on BallPlay-M, versus 53.3% and 15.1% for the prior RLID baseline, and raises average success on the ParaHome household tasks from 5.5% to 100%.","pith_inferences":["An editorial extension: the masked-buffer inpainting recipe is not tied to RLID—any trajectory optimizer or motion-inpainting model with a per-state fidelity cost could consume the same stitched graph, so the method's core idea may transfer to non-RL generation pipelines.","A testable prediction implied by the paper's construction: the similarity threshold tau and the maximum mask count N_max should control a phase transition in skill-transition success; sweeping those two hyperparameters over a fixed dataset would reveal the gap lengths the inpainting mechanism can bridge.","The paper's own noted limitation on heavily corrupted demonstrations suggests that the next version would need a learned dynamics prior to keep stitched trajectories physically plausible when the gaps are long—an extension the authors do not test.","If the in-hand reorientation result generalizes, the practical recipe for dexterous manipulation changes: capture a few grasp poses, exploit object symmetry to expand them, and let the transition field synthesize the motion, which could cut data collection costs substantially."],"forward_implications":["A single, brief demonstration clip (1-3 seconds) suffices to train a policy that can sustain a skill for at least 10 seconds; the paper reports Layup success rising from 0% to 91.5% with one noisy clip.","Skills never co-occurring in the reference data can be chained at test time; average skill-transition success on BallPlay-M reaches 93.8% versus 15.1% for the baseline.","Policies become robust to being initialized anywhere within an epsilon-neighborhood of the reference, not just at the reference states themselves, which is the basis for recovery from perturbations and object-pose generalization on ParaHome.","The augmentation machinery transfers beyond interaction: the same recipe improves locomotion skills on BallPlay-M (Run-to-Getup transition 100% versus 44.9% for the baseline) and turns discretely sampled grasp poses into continuous in-hand reorientation of a cube.","Data efficiency scales with augmentation: on ball pickup with 10 clips the method reaches 85.68% success versus 32.26% for the baseline, and with 40 clips 96.32% versus 82.84%."],"supporting_citations":[{"why":"Gives the RLID formulation, the unified interaction imitation reward, and the BallPlay-M basketball dataset that this paper builds on and benchmarks against.","marker":"[Wang et al. 2024c]"},{"why":"Provides the reference-state initialization scheme and imitation reward terms (DeepMimic) that RLID and the DM baseline are built on.","marker":"[Peng et al. 2018]"},{"why":"Shows that random state initialization can learn generalized transitions across sparse motion segments, the idea STF extends to interaction.","marker":"[Peng et al. 2022]"},{"why":"Introduces motion graphs, the stitching paradigm that STG adapts from locomotion to human-object interaction.","marker":"[Kovar et al. 2002]"},{"why":"Supplies the ParaHome household interaction dataset used to evaluate generalization outside the basketball domain.","marker":"[Kim et al. 2024]"},{"why":"Supplies the physics simulation platform used for all reported training and evaluation.","marker":"[Makoviychuk et al. 2021]"},{"why":"Provides the PPO algorithm used to optimize the learned policies.","marker":"[Schulman et al. 2017]"},{"why":"Motivates the hard-sample weighting scheme that ATS adapts for curriculum generation.","marker":"[Won and Lee 2019]"}],"fun_headline_variants":["Stitched trajectory graphs turn sparse demos into robust skills","Bridging gaps in noisy demos lifts skill success to 96.9%","Robust skills from sparse noisy demos via trajectory stitching","From sparse to robust: stitching demo gaps for generalization"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"That a trajectory created by stitching two states together with masked buffer frames in between will be physically plausible and learnable: the RLID imitation reward, computed only on visible frames, is assumed to be enough for the policy to discover a valid transition across the masked gap, no matter how the gap was constructed.","fun_headline_variants_meta":{"raw":{"variants":["Stitched trajectory graphs turn sparse demos into robust skills","Bridging gaps in noisy demos lifts skill success to 96.9%","Robust skills from sparse noisy demos via trajectory stitching","From sparse to robust: stitching demo gaps for generalization"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000281,"raw_usage":{"total_tokens":1688,"prompt_tokens":996,"completion_tokens":692,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":612,"completion_tokens_details":{"reasoning_tokens":620}},"tokens_in":612,"tokens_out":692,"duration_ms":7383,"temperature":1.0,"reasoning_tokens":620,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T01:01:47.315189+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take two skill clips whose only physical bridge passes through an obstacle or through a pose unreachable in one simulation step, stitch them with the paper's connection rules, and train the full method; if skill-transition success remains at baseline while the visible-state rewards are high, the inpainting assumption fails. A minimal version: corrupt a middle segment of a single noisy clip, then test whether a policy initialized inside the masked region recovers to within the epsilon tolerance of the reference—if recovery stays near zero, the field has not bridged the gap it was designed to fill.","supporting_citations":[],"review_version":1}