{"id":"7454759a-423b-46aa-9b99-d66e44157987","arxiv_id":"2508.14383","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":2,"one_line_summary":"Pre-training dynamics representations on arbitrary same-dynamics data lets offline imitation learning succeed with as few as one expert trajectory.","lead":"Robots typically need many expert demonstrations to learn how to imitate, and those are expensive to collect. This paper proposes learning a low-dimensional representation of the dynamics from any available data, then imitating from a handful of expert examples inside that space.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Unstated assumption: IL divergence determined by dynamics representation; action-indistinguishable MDPs may violate it","rationale":"The paper's central promise is a reduction of offline IL to a dynamics-derived representation. The reader's weakest assumption identifies that the IL objective may depend on action statistics not determined by dynamics; we focus on this because it sits at the logical core of the claimed theorem. A minimal MDP with action-independent dynamics would reveal whether the representation preserves enough action information for policy recovery. If it does not, the theorem as stated is false or requires assumptions missing from the abstract. If it does, the factorization must encode action identity, and the abstract's wording should be clarified. The sim-to-real transfer is a secondary concern because the theory explicitly targets same-dynamics data. We retain UNVERDICTED because the full proof and experiments are unavailable; the proposed test would move the verdict to ACCEPT or REJECT based on the outcome. The pre-training idea is plausible and the empirical claims are meaningful, but the central theoretical assertion remains insufficiently supported. We do not accuse the authors of any misconduct; the concern is purely about the argument's assumptions.","tokens_in":834,"tokens_out":5704,"duration_ms":71618,"concrete_test":"Construct a two-state MDP with states A and B and actions L,R where P(next|A,L)=P(next|A,R)=B and P(next|B,L)=P(next|B,R)=A. Pre-train the representation on random same-dynamics data with the proposed NCE loss. Then run the downstream offline IL with a single expert trajectory that always takes L. If the recovered policy does not reliably prefer L over R, the representation space has discarded action information, contradicting the claim. If it does recover L, examine the learned dynamics factorization to see whether it encodes action identity; if so, determine whether the theorem's proof explicitly relies on such encoding. Both outcomes settle whether the central assumption holds.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract's theoretical justification states that the optimal decision variable of offline IL lies in the representation space induced by factorizations of the transition dynamics. This is only true if the IL objective (e.g., a divergence between learner and expert) is fully determined by that representation. In general, an expert's action distribution contains information not present in the transition dynamics: in any MDP where two actions have identical next-state distributions from every state, the dynamics are the same for both actions, yet an expert may consistently select one of them. If the learned representation is based solely on dynamics factorization, it will not distinguish these actions, so the induced representation space cannot contain the optimal policy. The abstract gives no conditions (loss function, expert policy class, coverage assumptions) under which the theorem holds. Additionally, the sim-to-real experiment uses a change in dynamics contrary to the 'same dynamics' guarantee, so the transfer claim is unsupported by the theory.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a pre-training stage for offline imitation learning (IL) that learns representations of transition dynamics from arbitrary data collected under the same dynamics. The abstract claims a theoretical result that the optimal decision variable of offline IL lies in the representation space induced by such dynamics factorizations, thereby reducing the number of downstream parameters. A noise-contrastive-estimation (NCE) loss is introduced to learn these representations. Experiments on MuJoCo are claimed to show expert policy imitation from as few as a single trajectory, and real quadrupeds are claimed to learn to walk from a few real-world demonstrations by leveraging simulator-pretrained representations.","tokens_in":1047,"tokens_out":2446,"duration_ms":31852,"significance":"If the central claims hold, the paper would make a strong contribution: it would show that non-expert data can be harnessed to learn a dynamics representation that provably reduces the IL decision problem, potentially mitigating the small-data bottleneck in offline IL. The idea of reusing massive arbitrary data through a dynamics factorization is attractive and would be a meaningful step. However, the abstract alone provides no proof, no derivation of the NCE loss from the factorization, no baselines, and no error bars. No machine-checked proofs, reproducible code, or parameter-free derivations are visible from the abstract. The significance therefore rests entirely on the strength of the theoretical result and the empirical validation, neither of which can be assessed at this level of detail.","major_comments":[{"comment":"The central assertion that 'the optimal decision variable of offline IL lies in the representation space' is stated without any conditions on the IL loss, the expert policy class, or the MDP structure. This is not true in general: in an MDP where two actions have identical next-state distributions from every state but the expert selects only one of them, any representation learned purely from transition dynamics will identify the two actions, so the induced representation space cannot contain a policy that distinguishes them. The proof must either restrict the class of IL divergences or show that the representation retains action-specific information. This is a load-bearing point; without a precise theorem statement, the theoretical justification is incomplete.","section":"Abstract (theoretical claim)"},{"comment":"The paper claims dynamics representations can be learned from 'arbitrary data collected with the same dynamics.' No coverage assumptions are stated. If the arbitrary data cover only part of the state-action space, the learned factorization may be undefined or incorrect on expert-relevant regions, and the representation may not support the downstream IL objective. The manuscript needs to state the coverage conditions under which the pre-training representation remains valid for the expert distribution.","section":"Abstract (pre-training data coverage)"},{"comment":"The theoretical guarantee is phrased for 'the same dynamics,' but the real-quadruped experiment changes dynamics (simulator to real). The abstract's transfer claim is therefore not supported by the stated theory. Either an extension of the theory to distribution shift is needed, or the sim-to-real result should be presented as an empirical observation without claiming the same theoretical guarantee.","section":"Abstract (sim-to-real transfer)"}],"minor_comments":[{"comment":"The title says 'Arbitrary Demonstrations,' while the abstract says 'arbitrary data.' Clarify whether these are the same source or different, and whether 'demonstrations' includes non-expert trajectories.","section":"Title and abstract"},{"comment":"The phrase 'as few as a single trajectory' is not quantified against any baseline or success criterion. Please define the metric, the comparison, and the number of seeds.","section":"Abstract (empirical claims)"},{"comment":"The abstract mentions a 'tractable loss function inspired by noise contrastive estimation' but does not describe how the factorization maps to the NCE objective. A one-sentence derivation would help the reader understand the link.","section":"Abstract (NCE loss)"}],"recommendation":"uncertain","confidential_remarks":"The abstract is too sparse for a definitive recommendation. The theoretical claim, as stated, is likely false without additional assumptions, and the empirical claims lack the detail needed to verify. I recommend that the editor obtain the full manuscript before making a decision; based on the abstract alone, I cannot determine whether the paper is sound."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"If you see this one on a reading list, it's worth taking seriously, but don't take the abstract's word for it. The core idea—pre-train a dynamics representation from arbitrary same-dynamics data, then run offline IL in that compressed space—is genuinely not a restatement of something I've seen. The theoretical claim that the optimal IL decision variable lives in the representation space is the load-bearing piece, and if it holds, it's an elegant way to attack the limited-data problem. The MuJoCo single-trajectory result and the sim-to-real quadruped transfer would be real wins. I also appreciate the unpretentious writing: the abstract states the mechanism, the claim, and the experiments plainly.\n\nThat said, the soft spot is exactly where the stress-test note lands. The representation is learned from transition dynamics only. If two actions have identical next-state distributions from every state, the dynamics factorization cannot tell them apart, yet an expert may consistently pick one. Unless the IL objective is carefully chosen to depend only on state visitation or the theorem includes an identifiability condition, the representation space will not contain the optimal policy. The abstract gives no conditions on the loss, the expert policy class, or the coverage assumptions. This is not a fatal objection by itself—the full paper may well have the missing conditions—but it's the first thing I'd ask the authors to make precise.\n\nTwo smaller things. The sim-to-real experiment explicitly changes the dynamics, which is outside the \"same dynamics\" guarantee that justifies the pre-training theory. That transfer has to stand as an empirical claim on its own. And as an abstract-only review, I can't check whether the NCE loss actually follows from the factorization, or whether baseline comparisons include standard offline IL methods and multiple seeds. Those are normal limitations of reviewing an abstract, not signs of fraud.\n\nBottom line: this paper deserves a serious referee. The idea is novel enough and the potential payoff is large. A referee should pressure-test the theorem's assumptions, especially the identifiability condition, and ask for baselines with variance. If the proof holds up, it's a useful contribution; if it doesn't, the empirical results on their own may still be interesting. But the theoretical claim as stated is currently under-specified.","headline":"Worth a careful look, but the abstract alone leaves the key theorem uncheckable and the implicit identifiability assumption is likely to bite.","tokens_in":1493,"tokens_out":1676,"would_cite":false,"duration_ms":22523,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Offline imitation learning's optimal decision variable lies in the representation space induced by factorizations of the transition dynamics, so pre-training on arbitrary same-dynamics data greatly reduces the downstream parameters needed t","keywords":["offline imitation learning","dynamics representation","factorization of transition dynamics","noise contrastive estimation","pre-training","single-trajectory imitation","sim-to-real transfer","quadruped locomotion"],"falsifier":"Construct an environment with two expert policies that induce identical state-transition probabilities but different action distributions; if the learned dynamics representation is identical for both while the imitation loss differs, then the optimal decision variable cannot lie in the representation space, and the paper's central reduction would be refuted.","tokens_in":738,"feed_emoji":"🤖","tokens_out":3889,"duration_ms":38820,"temperature":0.7,"pith_summary":"The paper claims that the hard part of offline imitation learning—figuring out what to do from expert demonstrations—can be moved into a pre-training stage that only needs data about the environment's dynamics, not expert behavior. It argues that the optimal decision variable of offline IL lies in the representation space obtained by factorizing the transition dynamics, so downstream imitation can be solved with far fewer parameters. Because this representation can be learned from arbitrary data collected under the same dynamics, massive non-expert datasets become reusable. Experiments show that after such pre-training, MuJoCo policies can be mimicked from a single expert trajectory, and a real quadruped can learn to walk from a few real demonstrations using simulator-pretrained representations. The payoff, if true, is that imitation learning's data bottleneck shifts from expert availability to the much easier problem of collecting any interaction data.","feed_headline":"Dynamics pretraining lets imitation learn from one demo","feed_subtitle":"Theory puts the optimal policy in a compressed dynamics space, so non-expert data does the heavy lifting.","key_machinery":"The central object is a dynamics representation learned from a factorization of the transition dynamics: a map from state-action pairs into a low-dimensional space that captures exactly those features of the dynamics that determine the offline IL objective. The load-bearing property is that the optimal decision variable lies in this representation space, which carries the argument by reducing the downstream IL search to that space. The representation is trained with a noise contrastive estimation loss on arbitrary data sharing the same dynamics, which is what allows non-expert and simulator data to be reused.","core_discovery":"On its own terms, the paper establishes a structural reduction: for offline imitation learning, the optimal decision variable lies in the representation space induced by factorizations of the transition dynamics. This means the imitation objective is fully determined by a compressed description of how states evolve, so a learner that first learns that description from arbitrary same-dynamics data only needs to search over the representation space in the downstream IL stage. The paper derives a tractable noise-contrastive-estimation loss for learning the representation and validates the claim empirically: with the pretrained representation, MuJoCo control tasks can be imitated from a single e","pith_inferences":["If the IL objective really is a function of the dynamics representation alone, then two experts who induce the same transition distributions but differ in action randomness would be indistinguishable to this method; that is testable and would mark the boundary of the claim.","The same logic suggests the pretraining data can be highly heterogeneous in behavior—suboptimal, exploratory, or even adversarial—as long as it covers the state-action space; a natural extension is measuring how coverage quality affects representation fidelity.","The sim-to-real success hints that dynamics factorizations transfer better than policy features across domain shifts, which could be tested by comparing representation transfer against end-to-end imitation across increasing dynamics mismatch.","A direct extension would be to use the dynamics representation not just to initialize downstream IL but to evaluate candidate policies without expert action labels, since the objective is claimed to be determined by the representation."],"forward_implications":["Offline IL can be pretrained on any data with matching dynamics, not just expert demos, so large non-expert datasets become a resource instead of noise.","The downstream IL parameter count shrinks to the dimension of the representation space, making single-trajectory imitation plausible in high-dimensional control.","Simulation-pretrained dynamics representations transfer to real hardware, so a few real demonstrations can suffice for learning to walk.","The NCE loss gives a practical, label-free objective for representation learning without reward or action supervision."],"supporting_citations":[],"fun_headline_variants":["One demo imitation via dynamics pretraining","Pretrain dynamics, imitate from a single demo","Offline imitation needs just one demo after pretraining","Dynamics pretraining unlocks one-shot imitation","Non-expert data pretraining shrinks imitation to one demo"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The pre-training recovers a representation in which the imitation objective depends only on the dynamics; if the objective also requires information that the factorization discards, the reduction and the single-trajectory result would not hold.","fun_headline_variants_meta":{"raw":{"variants":["One demo imitation via dynamics pretraining","Pretrain dynamics, imitate from a single demo","Offline imitation needs just one demo after pretraining","Dynamics pretraining unlocks one-shot imitation","Non-expert data pretraining shrinks imitation to one demo"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000225,"raw_usage":{"total_tokens":1262,"prompt_tokens":667,"completion_tokens":595,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":411,"completion_tokens_details":{"reasoning_tokens":533}},"tokens_in":411,"tokens_out":595,"duration_ms":6570,"temperature":1.0,"reasoning_tokens":533,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T18:35:10.616205+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Construct an environment with two expert policies that induce identical state-transition probabilities but different action distributions; if the learned dynamics representation is identical for both while the imitation loss differs, then the optimal decision variable cannot lie in the representation space, and the paper's central reduction would be refuted.","supporting_citations":[],"review_version":1}