{"id":"f16810a6-59b8-4c57-a43c-03002cab5f17","arxiv_id":"2605.09183","paper_version":2,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":7.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"SeqRejectron constructs a stopping rule with a small set of validator policies to achieve horizon-free sample complexity for selective imitation learning under arbitrary dynamics shifts.","lead":"The paper introduces SeqRejectron, an algorithm that lets an imitation learner stop acting when it detects a dynamics shift between training and test environments using validator policies. A smart generalist might read it to see how to make copied behaviors safer in real deployments where the world changes after training.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Horizon-free Õ(log|Π|/ε²) bound requires sparse costs; without it the stopping-time regret bound reintroduces linear horizon dependence under arbitrary shift.","rationale":"The reader's weakest_assumption already flags the sparse-cost premise as central to the horizon-free bound. Full-text inspection confirms this is the precise point where the argument would fail for general costs, so the concern is load-bearing and the reader's identification is accurate. No other internal inconsistency (e.g., validator-set size or unlabeled-trajectory usage) appears more fundamental once sparsity is granted.","tokens_in":1735,"tokens_out":365,"duration_ms":42030,"concrete_test":"Locate the proof of the deterministic main theorem (likely Theorem 4.1 or equivalent). Re-derive the regret bound while replacing the sparse-cost assumption with a dense-cost model (e.g., per-step cost bounded by 1). If the resulting expression contains an explicit factor of H or the stopping time becomes Ω(H), the horizon-free claim does not hold without sparsity.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The strongest claim states that SeqRejectron yields horizon-free sample complexity for deterministic policies 'assuming sparse costs.' This assumption enters the soundness argument: the validator-based stopping rule must keep expected cumulative cost low before termination even when test dynamics are arbitrarily different. Sparsity (presumably that per-step cost is zero except on a small-measure set of states) is used to prevent the pre-stopping regret from scaling with horizon length. If costs are dense, the same validator construction can only guarantee a bound that grows with H, collapsing the horizon-free property. The paper's extension to misspecified experts and differing train/test policies also degrades gracefully only under the same sparsity premise.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper introduces a model and algorithm (SeqRejectron) for selective imitation learning under arbitrary train-test dynamics shift. Given expert demonstrations from training and unlabeled state trajectories from the same expert in the test environment, the learner outputs a selective policy that is complete (rarely stops on training data) and sound (low regret before stopping on test data). The algorithm uses a small set of validator policies (size independent of horizon and policy class) to construct a stopping rule. For deterministic policies it claims horizon-free Õ(log|Π|/ε²) sample complexity under sparse costs; for stochastic policies it uses a cumulative Hellinger stopping time. Extensions to misspecified experts and differing train/test expert policies are also provided, with graceful degradation.","tokens_in":1906,"tokens_out":597,"duration_ms":24687,"significance":"If the derivations and bounds hold, the work offers a principled way to handle arbitrary dynamics shift in imitation learning without committing to actions in uninformative states. The horizon-free sample complexity (conditional on sparse costs) and validator construction independent of horizon are technically notable strengths for long-horizon settings. The framework's extensions to misspecification add practical value, and the emphasis on both completeness and soundness provides clear, falsifiable guarantees.","major_comments":[{"comment":"The horizon-free Õ(log|Π|/ε²) bound for deterministic policies is stated to hold under the sparse-costs assumption (abstract and §4). The manuscript should explicitly derive or cite the step where sparsity prevents pre-stopping regret from scaling linearly with H under arbitrary shift; without this, the stopping-time regret bound appears to reintroduce horizon dependence, undermining the central claim.","section":"abstract and §4"},{"comment":"The soundness guarantee for the validator-based stopping rule (SeqRejectron) relies on unlabeled test trajectories from the expert. The paper should clarify whether the sample complexity remains horizon-free if these trajectories are unavailable or if the test expert policy differs substantially from training, as the extension in §5 appears to degrade only under the same sparsity premise.","section":"§5"}],"minor_comments":[{"comment":"Notation for the validator set size and its independence from H and |Π| should be introduced earlier (e.g., before the main theorem) to improve readability.","section":"§3"},{"comment":"The abstract mentions 'sparse costs' without a formal definition; add a precise statement (e.g., per-step cost zero except on a small-measure set) in the model section.","section":"§2"}],"recommendation":"major_revision","confidential_remarks":"The citation pattern appears light on recent selective IL or stopping-time work in RL; the editor may wish to check whether the authors adequately position against concurrent arXiv submissions on dynamics shift in imitation."},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for their careful reading and constructive feedback. We address each major comment below and indicate planned revisions to strengthen the clarity of the claims.","responses":[{"response":"We agree that an explicit derivation is needed for full transparency. Under the sparse-costs assumption, the analysis in the proof of the main theorem for deterministic policies bounds the pre-stopping regret by showing that costs are incurred only on a measure-zero or low-probability set of states; combined with the validator-based stopping rule, this ensures the accumulated regret before stopping remains independent of H even under arbitrary dynamics shifts. We will add a dedicated lemma and step-by-step derivation in the revised §4 (with a forward reference from the abstract) to make this explicit and cite the sparsity lemma directly.","revision_made":"yes","referee_comment":"[abstract and §4] The horizon-free Õ(log|Π|/ε²) bound for deterministic policies is stated to hold under the sparse-costs assumption (abstract and §4). The manuscript should explicitly derive or cite the step where sparsity prevents pre-stopping regret from scaling linearly with H under arbitrary shift; without this, the stopping-time regret bound appears to reintroduce horizon dependence, undermining the central claim."},{"response":"The framework is defined to require unlabeled test trajectories from the expert for constructing the validator set and stopping rule; without them the algorithm cannot be executed as stated, so we make no horizon-free claim in that setting. For differing train/test expert policies, §5 already shows graceful degradation of the bounds while retaining horizon-freeness under sparsity. We will add an explicit clarifying paragraph in §5 and the discussion section stating the necessity of test trajectories and confirming that horizon-freeness holds only under the stated assumptions including sparsity.","revision_made":"yes","referee_comment":"[§5] The soundness guarantee for the validator-based stopping rule (SeqRejectron) relies on unlabeled test trajectories from the expert. The paper should clarify whether the sample complexity remains horizon-free if these trajectories are unavailable or if the test expert policy differs substantially from training, as the extension in §5 appears to degrade only under the same sparsity premise."}],"tokens_in":1454,"tokens_out":472,"duration_ms":68828,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"This paper shows how to make imitation learning selective: the policy can stop instead of acting when the test dynamics have shifted in unknown ways. The main contribution is SeqRejectron, which uses a small set of validator policies to build the stopping rule. That set size does not depend on the horizon or the size of the policy class. For deterministic policies the sample complexity is horizon-free at Õ(log|Π|/ε²), but only when costs are sparse. They get similar results for stochastic policies via a cumulative Hellinger stopping time. The framework also covers misspecified experts and different train/test expert policies, with bounds that degrade based on the misspecification level. The approach is grounded in standard concentration inequalities plus the validator construction. There is no obvious circularity in the bounds. The abstract is clear on the claims and the assumptions. The main limitation is the sparse costs requirement. If costs are not sparse, the regret before stopping can scale with the horizon even with the validators, which removes the horizon-free advantage. Access to unlabeled expert state trajectories in the test environment is also needed, and that may not always be available. This is worth reading for people working on imitation learning in robotics or control with changing conditions. It gives a concrete way to avoid bad actions under shift instead of forcing a policy everywhere. I would send this to peer review. The problem is practical and the technical approach looks promising, though the assumptions will need careful discussion in the full version.","headline":"The paper introduces selective stopping for imitation learning under arbitrary dynamics shifts using a small set of validator policies, delivering horizon-free sample complexity when costs are sparse.","tokens_in":2369,"tokens_out":369,"would_cite":true,"duration_ms":46239,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":{"model":"grok-4.3","evidence":[],"headline":"Selective imitation via validator stopping times under dynamics shift","alignment":"orthogonal","rationale":"The paper's core machinery (SeqRejectron, sparse validator distributions from minimax games over version spaces, disagreement/Hellinger stopping times, horizon-free bounds under sparse costs) operates entirely within statistical imitation learning and selective classification under arbitrary shift. It has no structural resemblance to RS primitives such as the single-distinction forcing chain, reciprocal cost J(x) = ½(x + x⁻¹) − 1, golden-ratio ladder, or 8-tick/3D emergence. Sparse costs here are a technical assumption for sample-complexity control and do not parallel RS J-cost reasoning.","tokens_in":60425,"confidence":"high","tokens_out":160,"duration_ms":8972,"cache_read_input_tokens":38528,"cache_creation_input_tokens":0},"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"A selective imitation learner stops in uncertain states under dynamics shift and achieves horizon-free sample complexity using a small fixed set of validator policies.","keywords":["selective imitation learning","dynamics shift","stopping rule","validator policies","horizon-free sample complexity","behavior cloning","imitation learning"],"falsifier":"An experiment or counterexample in which costs have dense support or no unlabeled test trajectories are supplied, causing either the sample complexity to grow with the horizon or the low-regret guarantee before stopping to fail.","tokens_in":2641,"feed_emoji":"","tokens_out":708,"duration_ms":67049,"temperature":0.7,"pith_summary":"Standard behavior cloning forces the learner to output an action at every state, which can cause arbitrary performance loss when the test environment has different transitions than the training environment. The paper studies selective imitation, where the learner may stop and avoid acting on states where its demonstrations are uninformative. Given labeled expert trajectories from training and unlabeled state trajectories from the same expert in testing, the learner must produce a policy that rarely stops on training data yet incurs low regret before stopping on test data. The key technical contribution is an algorithm that builds the stopping decision from a validator set whose size stays independent of task length and policy class size. This produces sample complexity that scales only with the logarithm of the policy class size over the accuracy parameter when costs are sparse.","feed_headline":"Stopping rule achieves horizon-free selective imitation","feed_subtitle":"Unlabeled test trajectories and sparse costs let sample needs depend only on policy class size, not task length.","key_machinery":"The SeqRejectron algorithm that builds a stopping rule from a small set of validator policies whose cardinality is bounded independently of horizon and policy class size.","core_discovery":"The paper establishes that the SeqRejectron algorithm constructs a stopping rule from a validator set of size independent of the horizon and policy class, yielding a selective policy that is complete on the training distribution and sound on the test distribution under arbitrary dynamics shift. For deterministic policies and sparse costs this gives horizon-free sample complexity Õ(log|Π|/ε²). Analogous horizon-free guarantees hold for stochastic policies via a cumulative Hellinger stopping time. The same framework extends to misspecified experts and differing expert policies across environments, with performance degrading gracefully in the amount of misspecification.","pith_inferences":["The selective stopping mechanism could be paired with a separate fallback policy or human query to handle the states where the learner elects to stop.","The validator-based construction may transfer to other sequential tasks that experience gradual distribution shift over long horizons.","Empirical tests with controlled amounts of dynamics mismatch could quantify how often the learned policy chooses to stop as a function of shift severity."],"forward_implications":["For deterministic policies the sample complexity remains horizon-free and scales as Õ(log|Π|/ε²) under sparse costs.","For stochastic policies analogous horizon-free bounds hold through a cumulative Hellinger stopping time.","The framework extends to misspecified experts and different expert policies across train and test, with graceful degradation.","The validator set size stays independent of horizon and policy class size."],"fun_headline_variants":["Validator set constructs stopping rule for dynamics shift","Selective imitation complete in train sound in test environments","Horizon-free sample needs for selective imitation under shift","SeqRejectron enables stopping under arbitrary dynamics change"],"cache_read_input_tokens":64,"weakest_assumption_plain":"The costs are sparse and unlabeled state trajectories from the same expert are available in the test environment.","fun_headline_variants_meta":{"raw":{"variants":["Validator set constructs stopping rule for dynamics shift","Selective imitation complete in train sound in test environments","Horizon-free sample needs for selective imitation under shift","SeqRejectron enables stopping under arbitrary dynamics change"]},"model":"grok-4.3","cost_usd":0.010245,"raw_usage":{"total_tokens":4482,"prompt_tokens":714,"num_sources_used":0,"completion_tokens":57,"cost_in_usd_ticks":102453000,"prompt_tokens_details":{"text_tokens":714,"audio_tokens":0,"image_tokens":0,"cached_tokens":64},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":3711,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":714,"tokens_out":57,"duration_ms":33548,"temperature":1.0,"reasoning_tokens":3711,"cache_read_input_tokens":64,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-05-20T21:59:54.123022+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"An experiment or counterexample in which costs have dense support or no unlabeled test trajectories are supplied, causing either the sample complexity to grow with the horizon or the low-regret guarantee before stopping to fail.","supporting_citations":[],"review_version":2}