{"id":"dd8e886b-9866-4cd5-8b37-bd3390c991ed","arxiv_id":"2508.14379","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":7.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"DTWIL creates a surrogate demonstration set by aligning expert and constrained state trajectories through model predictive control and dynamic time warping, then trains an imitator on it.","lead":"This paper introduces action-constrained imitation learning, where an imitator with a restricted action space must learn from an expert with a wider action space, and presents DTWIL, which replaces expert demonstrations with surrogate trajectories the constrained agent can follow. It matters because many real systems, from robot arms to resource schedulers, must copy skilled behavior under hard safety or capacity limits.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The load-bearing risk is that DTW-aligned surrogate demonstrations can minimize state distance while failing the actual task; with method/experiment sections missing, the paper offers no evidence against this.","rationale":"The reader's weakest_assumption already identifies the same core risk: surrogate demonstrations may be feasible but task-incompetent. My analysis sharpens this by noting that the DTW objective itself discards timing and force information, and that the missing experiment section leaves no evidence that the surrogates achieve expert-level returns. The verdict should remain UNVERDICTED because the provided text lacks the method, theory, and experiment sections needed to assess whether this concern lands. I am not recommending a change in verdict; I am reinforcing the reader's caution with a concrete mechanism and a direct test.","tokens_in":4053,"tokens_out":3836,"duration_ms":50299,"concrete_test":"Using the released code, compute the ground-truth task return of the surrogate demonstrations themselves (the MPC rollouts that constitute the dataset) on each benchmark environment. If the mean surrogate return is substantially below the mean expert return, then the surrogate dataset is not a valid expert replacement and the central claim fails. Run this on at least one contact-rich or time-critical task (e.g., peg insertion or a limited-torque locomotion task) to specifically stress the DTW state-alignment assumption.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that replacing expert demonstrations with DTW-aligned, action-constrained surrogate trajectories yields a dataset from which a constrained imitator can learn a policy that is both feasible and competent at the task. The paper's stated objective for generating those surrogates is minimizing DTW distance to the expert's state trajectories (Abstract, Introduction), not maximizing task reward. Two gaps make this precarious. First, DTW is invariant to time warping, and state trajectories do not encode contact forces or action profiles. For contact-rich or time-critical tasks, feasible trajectories can shadow the expert's state path while failing the task (e.g., approaching a peg but not inserting it, or tracking a balancing trajectory without stabilizing it). The paper provides no theorem bounding task suboptimality in terms of DTW distance, and the version under review omits the experiment section, so there is no evidence that the surrogate demonstrations even achieve expert-level returns. Second, the MPC planner requires a dynamics model; no accuracy or robustness discussion appears. If the model is approximate, the surrogate states may be feasible only under the model, not the true environment, making the dataset dynamically infeasible. Because the entire pipeline inherits the quality of the surrogate dataset, the absence of evidence that the surrogates are task-competent is the single most load-bearing concern.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces a new problem setting, Action-Constrained Imitation Learning (ACIL), in which an imitator with a restricted action space must learn from demonstrations of a constraint-free expert. The authors argue that existing IL methods fail under action constraints because the constraint-induced mismatch in occupancy measures prevents the imitator from matching the expert. They propose DTWIL, a method that replaces the expert demonstrations with a surrogate dataset: an MPC planner generates trajectories that are feasible under the imitator's action constraints and that minimize Dynamic Time Warping (DTW) distance to the expert's state trajectories. A standard IL algorithm is then trained on this surrogate dataset. The abstract claims that DTWIL significantly improves performance across multiple robot control tasks and achieves better sample efficiency than several benchmark IL algorithms. The manuscript as provided contains an introduction, references, and brief closing matter, but no method section, no experiments, no tables, and no figures.","tokens_in":4408,"tokens_out":4440,"duration_ms":47995,"significance":"If the claims are correct, ACIL is a genuinely novel and useful problem formulation: it addresses a common practical gap where an agent must imitate an expert with a larger action set, e.g., a lower-torque robot learning from a higher-torque demonstrator. The proposed approach of using MPC and DTW to generate action-feasible surrogate trajectories is creative and clearly distinguishes DTWIL from projection-based adaptations of existing constrained-RL methods. The paper also publicly releases code, which is a positive step for reproducibility. However, because the submitted text omits the method and experiments, the central empirical claim is not auditable and the significance cannot be verified from the manuscript itself. The DTW-based surrogate generation also raises a correctness-risk concern: minimizing state-trajectory distance does not by itself guarantee task competence, especially for contact-rich or time-critical tasks. The paper needs both a full description of the method and targeted experiments or analysis to substantiate the central claim.","major_comments":[{"comment":"The manuscript as submitted is incomplete: after the Introduction, the text jumps directly to a closing fragment, the Impact Statement, acknowledgements, and references. There is no method section describing the DTWIL algorithm, no MPC formulation, no action-constraint handling, and no experimental section. The abstract's claim of 'extensive experiments' and 'outperforms various benchmark imitation learning algorithms' is therefore unsupported. This is the most load-bearing issue: without the full method and results, the paper's central contribution cannot be evaluated.","section":"Overall structure (between Introduction and References)"},{"comment":"DTWIL's stated objective is to generate surrogate trajectories that follow 'similar state trajectories' to the expert while adhering to action constraints. DTW distance is invariant to time warping, and state trajectories do not encode contact forces, action profiles, or timing information. For contact-rich or time-critical tasks, a feasible surrogate can shadow the expert's state path while failing the actual task (e.g., approaching a peg without inserting it). The manuscript provides no theorem or analysis bounding task suboptimality in terms of DTW distance, and no experimental evidence that surrogate demonstrations achieve expert-level returns. This gap undermines the claim that learning from the surrogate dataset yields a competent constrained policy.","section":"Abstract and Introduction"},{"comment":"DTWIL relies on an MPC planner to generate surrogate trajectories. MPC requires a dynamics model, but the manuscript does not describe the model, its accuracy, or how model error is handled. If the model is approximate, the generated surrogate trajectories may be feasible only under the model, not in the true environment, which would violate the premise that the dataset adheres to the action constraints and resembles feasible expert behavior. The paper should specify the model class, its source, and any robustness measures taken.","section":"MPC / dynamics model (implicit in method)"}],"minor_comments":[{"comment":"The reference 'Pomerleau & A, 1991' should be formatted correctly (the author is Dean A. Pomerleau). Please check all reference entries for consistency.","section":"References"},{"comment":"The sentence fragment 'lenges that previous methods have not effectively addressed' appears immediately after the Introduction, suggesting a missing paragraph or a broken page boundary. Please proofread the full text.","section":"Text fragment after Introduction"},{"comment":"The code repository is named 'ACRL-Baselines'. Since ACIL is presented as a distinct problem from ACRL, a different repository name would reduce confusion about the relationship between the two lines of work.","section":"Repository name"}],"recommendation":"major_revision","confidential_remarks":"The submitted text appears to be an incomplete or corrupted version of the paper, as it contains no method or evaluation sections. I recommend asking the authors to provide the complete manuscript. If this is indeed the full submission, it is not ready for review."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: I think the authors have correctly identified a real gap in the imitation learning literature—what to do when the expert has a larger action space than the imitator. The idea of generating surrogate demonstrations that respect the imitator's constraints while tracking the expert's state trajectory is natural, and using DTW to handle temporal misalignment is sensible. That's the good part.\n\nThe paper also does a decent job situating itself against ACRL and state-only IL. The code link is a plus. No red flags in the citation pattern; they build on their own prior ACRL work (Hung et al., 2025) without hiding it.\n\nNow the soft spots. As I can see from the provided text, the method and experiment sections are missing. The abstract promises 'extensive experiments' that 'significantly enhances performance,' but that data is not visible. So I can't audit the headline claim. That's the biggest issue.\n\nThe deeper concern, which I think is legitimate even with the full paper, is that DTW alignment on state trajectories does not by itself guarantee task competence. DTW is invariant to time warping, and state trajectories don't encode contact forces or action profiles. A constrained imitator could produce a surrogate trajectory that shadows the expert's path but fails at the actual objective (e.g., approaching the peg without inserting it). The authors don't provide a bound or even a discussion connecting DTW distance to task reward. And the MPC planner needs a dynamics model; if that model is approximate, the surrogate states might only be feasible under the model. These are open questions rather than demonstrated flaws, but they're load-bearing because the whole pipeline inherits the quality of the surrogate dataset.\n\nI can't give a verdict on correctness because the evidence isn't here. But as a problem formulation, ACIL is worth taking seriously. I'd send it to review and insist that the referees see the full experiments plus an explicit treatment of the task-competence issue. Maybe give it to a student who works on imitation learning as a good example of how to frame a new problem.","headline":"The ACIL framing is a genuine and useful reframing, but as submitted here I can't verify the central empirical claim—and the DTW-alignment idea has a load-bearing gap that the missing experiments may or may not close.","tokens_in":4828,"tokens_out":2941,"would_cite":true,"duration_ms":33806,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper introduces DTWIL, an imitation-learning method for agents whose action space is smaller than an expert's: it replaces the expert demos with feasible surrogate trajectories aligned to the expert's states via dynamic time warping a","keywords":["Action-Constrained Imitation Learning","dynamic time warping","model predictive control","surrogate demonstrations","occupancy measure","imitation learning","action constraints","robot control"],"falsifier":"Run a task whose success requires a precise high-force action at a specific moment (for example, striking or pushing a stuck object), train a DTWIL imitator under a torque limit, and check whether the surrogate trajectories are feasible but fail to accomplish the task. If they are feasible yet task-incompetent, the sufficiency of state-only alignment is refuted.","tokens_in":4035,"feed_emoji":"🤖","tokens_out":13357,"duration_ms":121799,"temperature":0.7,"pith_summary":"The paper introduces a new problem, Action-Constrained Imitation Learning (ACIL): an agent with limited action capabilities must learn from demonstrations produced by an expert with a larger action space. It argues that simply projecting expert actions into the feasible set distorts the occupancy measure, so the imitator can never match the expert's behavior. Its proposed method, DTWIL, instead builds a surrogate dataset of feasible trajectories that visit the same states as the expert, using dynamic time warping to decide which expert states matter and model predictive control to plan constraint-respecting paths. Learning from this surrogate data, standard imitation algorithms perform better across several robot-control tasks and do so more sample-efficiently than the benchmark IL methods compared in the paper. If this holds, imitation under action constraints becomes a matter of aligning states rather than copying actions.","feed_headline":"Constrained robots learn from experts by warping trajectories","feed_subtitle":"By planning feasible surrogate demos that visit the same states as an expert, DTWIL outperforms projection-based imitation.","key_machinery":"Dynamic Time Warping (DTW) distance is the core object: it measures how cheaply one time series can be warped onto another, so a constrained trajectory can match expert states at shifted timestamps. Model Predictive Control (MPC) is the planner that produces surrogate trajectories: given a dynamics model, it searches over feasible action sequences to minimize the DTW distance to the expert demonstration while respecting the action constraints. The output is a surrogate dataset that replaces the original expert demos.","core_discovery":"The central claim is that ACIL fails when the imitator tries to match the expert's occupancy measure directly, because a restricted action set makes the required state-action distribution unreachable. DTWIL sidesteps this by changing the target: instead of imitating the expert's actions, it imitates a feasible path through the expert's states. Each expert demonstration is turned into a surrogate trajectory through MPC, with the DTW distance as the alignment objective, and the resulting surrogate set is used to train any downstream imitation learner. The paper reports that this approach yields better performance and sample efficiency on multiple robot-control benchmarks than projection-based","pith_inferences":["If state alignment is the right level of abstraction, DTWIL's surrogate datasets could also seed offline reinforcement learning or serve as a pretraining source for related constrained tasks.","The MPC step depends on an accurate dynamics model; replacing it with an online-learned model or a model-free planner is a natural extension, and the method's success will hinge on how well the planner covers the constrained region.","Tasks where success depends on precise timing or force profiles rather than visited states would stress-test state-only alignment; a constrained pushing or insertion task with a force threshold is a concrete probe.","Augmenting the DTW cost with action- or torque-weighted penalties could preserve more of the expert's timing intent while still respecting constraints."],"forward_implications":["ACIL is tractable: a constrained agent can learn useful policies from a less-constrained expert without ever seeing feasible expert actions.","Because DTWIL outputs a standard demonstration dataset, existing imitation-learning algorithms can be applied directly to the constrained setting.","Projection-based action-constrained methods are not a sufficient recipe for imitation; their occupancy-measure distortion is an identified failure mode.","DTWIL's sample-efficiency gains suggest that state-level alignment carries more usable information than action-level projection when constraints bind.","The DTW-MPC pipeline gives a general way to compare experts and imitators whose action spaces differ."],"supporting_citations":[{"why":"Defines the occupancy-measure matching objective of imitation learning that action constraints are argued to break.","marker":"(Ho & Ermon, 2016)"},{"why":"Represents the projection-layer approach to action-constrained policy learning that the paper identifies as inadequate for imitation.","marker":"(Chow et al., 2018)"},{"why":"Supplies the dynamic-time-warping distance that measures alignment between expert and surrogate state trajectories.","marker":"(Hiroaki & Chiba, 1978)"},{"why":"Establishes state-only behavioral cloning, the style of imitation that DTWIL's state-aligned surrogate data build on.","marker":"(Torabi et al., 2018a)"},{"why":"Provides the learned probabilistic dynamics model used by MPC to plan feasible surrogate trajectories.","marker":"(Chua et al., 2018)"},{"why":"Frames action-constrained reinforcement learning benchmarks, the setting ACIL extends to imitation.","marker":"(Kasaura et al., 2023)"}],"fun_headline_variants":["Warp expert paths to teach constrained robots","Feasible demo paths: key to constrained imitation","DTWIL: Align trajectories, not actions","Robots learn better by matching states, not actions","Action limits? Warp expert trajectories"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The method assumes that a feasible trajectory that passes through roughly the same states as the expert, even with different timing and forces, is enough to teach the constrained agent the task, and that the MPC planner's dynamics model is accurate enough to find such trajectories.","fun_headline_variants_meta":{"raw":{"variants":["Warp expert paths to teach constrained robots","Feasible demo paths: key to constrained imitation","DTWIL: Align trajectories, not actions","Robots learn better by matching states, not actions","Action limits? Warp expert trajectories"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00013,"raw_usage":{"total_tokens":948,"prompt_tokens":718,"completion_tokens":230,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":462,"completion_tokens_details":{"reasoning_tokens":159}},"tokens_in":462,"tokens_out":230,"duration_ms":3067,"temperature":1.0,"reasoning_tokens":159,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T18:34:24.077663+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a task whose success requires a precise high-force action at a specific moment (for example, striking or pushing a stuck object), train a DTWIL imitator under a torque limit, and check whether the surrogate trajectories are feasible but fail to accomplish the task. If they are feasible yet task-incompetent, the sufficiency of state-only alignment is refuted.","supporting_citations":[{"cited_title":"A Lyapunov-based approach to safe reinforcement learning","cited_arxiv_id":null,"evidence_quote":"Represents the projection-layer approach to action-constrained policy learning that the paper identifies as inadequate for imitation."},{"cited_title":"and Chiba, S","cited_arxiv_id":null,"evidence_quote":"Supplies the dynamic-time-warping distance that measures alignment between expert and surrogate state trajectories."},{"cited_title":"Deep reinforcement learning in a handful of trials using probabilistic dynamics models","cited_arxiv_id":null,"evidence_quote":"Provides the learned probabilistic dynamics model used by MPC to plan feasible surrogate trajectories."},{"cited_title":"Benchmarking actor-critic deep reinforcement learning algorithms for robotics control with action constraints","cited_arxiv_id":null,"evidence_quote":"Frames action-constrained reinforcement learning benchmarks, the setting ACIL extends to imitation."}],"review_version":1}