Noisy expert imitation learning requires exponential samples for offline methods but polynomial for a variant of on-policy distillation under a noise condition.
Reinforcement and Imitation Learning via Interactive No-Regret Learning
12 Pith papers cite this work. Polarity classification is still indexing.
12
Pith papers citing it
abstract
Recent work has demonstrated that problems-- particularly imitation learning and structured prediction-- where a learner's predictions influence the input-distribution it is tested on can be naturally addressed by an interactive approach and analyzed using no-regret online learning. These approaches to imitation learning, however, neither require nor benefit from information about the cost of actions. We extend existing results in two directions: first, we develop an interactive imitation learning approach that leverages cost information; second, we extend the technique to address reinforcement learning. The results provide theoretical support to the commonly observed successes of online approximate policy iteration. Our approach suggests a broad new family of algorithms and provides a unifying view of existing techniques for imitation and reinforcement learning.
citation-role summary
method 1
citation-polarity summary
roles
method 1polarities
use method 1representative citing papers
Online IL overcomes an information-theoretic bottleneck that offline IL faces in non-realizable settings even at horizon 1, under a new structural characterization of reward-relative misspecification.
Introduces a feasible-reward-set approach to IRL with multiple heterogeneous suboptimal demonstrators, proving monotonic shrinkage of the joint set and two recovery guarantees for the ground-truth optimal reward.
DAgger-style training with turn-level policy interpolation raises 4B and 8B LLM agents to 27.3% and 29.8% on SWE-bench Verified, beating several larger published systems.
Process advantage verifiers trained to predict step-level progress under a distinct prover policy improve LLM reasoning accuracy by over 8% and sample efficiency by 5-6x over outcome reward models.
BMIL learns belief modules jointly with policies for GAIL-style imitation learning in POMDPs, outperforming separate training and standard GAIL on continuous control tasks.
Imitation learning yields provably stable feedback policies for partially observed Vlasov-Poisson instabilities, with error floors determined by minimal behavior-cloning loss characterized via entropy of the initial distribution.
Sparse directional VLA guidance jump-starts PPO for long-horizon and poorly rewarded manipulation, improving sample efficiency over PPO and distillation while enabling zero-shot real-robot transfer.
The primary OL-CL gap in end-to-end autonomous driving arises from objective mismatch creating structural inability to model reactive behaviors, which a test-time adaptation method can mitigate.
SCAL aligns source and target latent features conditioned on system state, reducing target imitation loss to a source loss plus a conditional-KL term, and reports strong sample efficiency in BARC-CARLA.
SDB balances behavioral diversity and learning stability in VLN self-improvement by expanding decisions into latent hypotheses, performing reliability-aware aggregation, and applying a regularizer, yielding gains such as SPL 33.73 to 35.93 on REVERIE val-unseen.
citing papers explorer
-
Behavior Cloning is Not All You Need: The Optimality of On-Policy Distillation for Noisy Expert Feedback
Noisy expert imitation learning requires exponential samples for offline methods but polynomial for a variant of on-policy distillation under a noise condition.
-
When Does Online Imitation Learning Help in LLM Post-Training? The Role of (Non-)Realizability Beyond Horizon
Online IL overcomes an information-theoretic bottleneck that offline IL faces in non-realizable settings even at horizon 1, under a new structural characterization of reward-relative misspecification.
-
Inverse Reinforcement Learning without an Optimal Demonstrator: A Feasible Reward Set Approach
Introduces a feasible-reward-set approach to IRL with multiple heterogeneous suboptimal demonstrators, proving monotonic shrinkage of the joint set and two recovery guarantees for the ground-truth optimal reward.
-
Revisiting DAgger in the Era of LLM-Agents
DAgger-style training with turn-level policy interpolation raises 4B and 8B LLM agents to 27.3% and 29.8% on SWE-bench Verified, beating several larger published systems.
-
Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoning
Process advantage verifiers trained to predict step-level progress under a distinct prover policy improve LLM reasoning accuracy by over 8% and sample efficiency by 5-6x over outcome reward models.
-
Learning Belief Representations for Imitation Learning in POMDPs
BMIL learns belief modules jointly with policies for GAIL-style imitation learning in POMDPs, outperforming separate training and standard GAIL on continuous control tasks.
-
Provable imitation learning for control of instability in partially-observed Vlasov--Poisson equations
Imitation learning yields provably stable feedback policies for partially observed Vlasov-Poisson instabilities, with error floors determined by minimal behavior-cloning loss characterized via entropy of the initial distribution.
-
Vision-Language-Action Jump-Starting for Reinforcement Learning Robotic Agents
Sparse directional VLA guidance jump-starts PPO for long-horizon and poorly rewarded manipulation, improving sample efficiency over PPO and distillation while enabling zero-shot real-robot transfer.
-
BridgeSim: Unveiling the OL-CL Gap in End-to-End Autonomous Driving
The primary OL-CL gap in end-to-end autonomous driving arises from objective mismatch creating structural inability to model reactive behaviors, which a test-time adaptation method can mitigate.
-
State-Conditional Adversarial Learning: An Off-Policy Visual Domain Transfer Method for End-to-End Imitation Learning
SCAL aligns source and target latent features conditioned on system state, reducing target imitation loss to a source loss plus a conditional-KL term, and reports strong sample efficiency in BARC-CARLA.
-
The Essence of Balance for Self-Improving Agents in Vision-and-Language Navigation
SDB balances behavioral diversity and learning stability in VLN self-improvement by expanding decisions into latent hypotheses, performing reliability-aware aggregation, and applying a regularizer, yielding gains such as SPL 33.73 to 35.93 on REVERIE val-unseen.
- Speculative Rollback Correction for Quality-Diverse Web Agent Imitation