Pith. sign in

REVIEW 3 major objections 5 minor 39 references

Domain-Invariant Per-Frame Feature Extraction for Cross-Domain Imitation Learning with Visual Observations

T0 review · 3 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Cross-domain imitation learning succeeds when domain-invariant features are extracted per frame and expert behavior is rewarded in temporal order, allowing a target-domain agent to imitate a source expert from images alone.

desk verdict Genuine D3IL extension with consistent gains; the state-based SAC training and the time-label transfer assumption are the two things to fix. read the letter →

arxiv 2502.02867 v2 pith:PKGYO4IZ submitted 2025-02-05 cs.CV cs.AIcs.LG

Minung Kim , Kawon Lee , Jungmo Kim , Sungho Choi , Seungyul Han This is my paper · ORCID
classification cs.CVcs.AIcs.LG
keywords domain-invariantfeatureextractionper-framefeaturesframe-wisetimelabelingcross-domainimitationlearningfromobservationadversarialvisualobservationsWassersteinGAN
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

DIFF-IL is a method for imitation learning from visual observations when the expert and the learner live in different domains, such as different bodies, dynamics, or viewpoints. The paper's claim is that aligning individual image frames across domains, rather than aligning multi-frame sequences, removes domain-specific appearance and physics while preserving task-relevant state, and that a frame-wise time label then tells the learner which frames are closer to the goal. If true, an agent can learn a target task by watching a source expert's videos plus random interaction in its own domain, without needing the expert's actions or a shared state space. The paper reports that this combination produces higher mean returns than four existing cross-domain imitation methods on most of 14 pendulum and locomotion tasks, and that target frames map almost one-to-one onto source expert frames.

What carries the argument

The central object is the domain-invariant per-frame latent space produced by a shared encoder $p$ and domain-specific decoders $q_S$ and $q_T$, with adversarial alignment by a frame-level WGAN discriminator $D_f$ and a sequence-level discriminator $D_s$. Two label networks carry the reward: $F_{label,s}$, trained to classify expert versus non-expert feature sequences, and $F_{label,f}$, trained on a linear time label so later expert frames score higher. The reward multiplies both labels, so the target learner must simultaneously look like the expert and be progressing toward the goal. The machinery's job is to make the latent code ignore background, camera, and embodiment-specific appearance while preserving the expert's state, such as positions, angles, and velocities, so that expertise can be evaluated frame by frame.

What would settle it

Run DIFF-IL on a task where the source expert reaches the goal midway through the episode and then maintains it, for example a swing-up-and-balance task, and inspect the frame label $F_{label,f}$ along the demonstration: if the label keeps increasing after the goal state is reached instead of saturating, or if the learned target policy is worse than one trained without the time label, that would show the label is encoding episode time rather than task progress.

Watch

Extended reading notes

Core claim

The central claim is that the correct unit for cross-domain alignment in visual imitation is the individual frame, not the trajectory or sequence. A shared encoder maps each image from either domain into a latent code, while domain-specific decoders reconstruct the image, so the code must retain content; a Wasserstein GAN frame discriminator tries to tell source from target codes and the encoder tries to fool it, pushing domain-specific appearance out. A cycle-consistency term sends a code through the opposite domain's decoder and re-encodes it, requiring the code to survive the transfer. After this, the paper aligns sequences of these frame features for expertise labeling, and adds a frame-wise time label $y_t = ((t/H)+1)/2$ for source expert frames, trained only in the source domain. The reward to the target learner is $R_t = -\log(1 - F_{label,s}(z^T_{seq,t+1}) F_{label,f}(z^T_{t+1}))$, so the learner is rewarded for looking expert and for being temporally close to the goal. The experiments report that this pipeline outperforms D3IL, DeGAIL, GWIL, and TPIL in most of 14 tasks.

Load-bearing premise

The load-bearing premise is that in the source expert's demonstrations, later timesteps are always closer to the task goal, and that the same temporal ordering is correct for the target domain; if an expert reaches the goal early and holds it, or makes non-monotonic progress, the time label rewards timing rather than goal achievement.

Editorial extensions

If this is right

  • Single-frame alignment gives a t-SNE feature map in which source expert and target learner frames overlap almost completely, whereas sequence-level methods leave residual domain structure; this is the paper's evidence for seamless alignment.
  • Frame-wise time labels assign increasing reward to later expert frames, letting the learner distinguish expert behavior from random behavior even when the two distributions barely overlap.
  • In the reported results, DIFF-IL achieves higher mean final returns than D3IL, DeGAIL, GWIL, and TPIL in most of the 14 pendulum and MuJoCo tasks, with faster convergence on the MuJoCo learning curves.
  • Because the source expert's ability bounds the target, tasks with a physically slow source, such as Hopper, yield capped target performance, which is a consequence of imitation learning rather than a failure of adaptation.
  • The learned frame features support nearest-neighbor image mapping between target learner frames and source expert frames, giving a way to visualize what the learner is imitating.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The linear time label encodes a monotone-progress prior: it assumes later source frames are closer to the goal and that the target optimal behavior has the same ordering; the paper's tasks satisfy this, but tasks where an expert reaches the goal mid-episode and then holds it would likely make the reward favor lateness over goal achievement, so a learned or distance-based progress label would be a
  • The ablation shows the full method beats both sequence-mapping-only and no-frame-time-label variants, suggesting the gains come from the interaction of per-frame alignment with time labels; a further ablation replacing the time label with a constant would reveal how much of the gain is temporal shaping versus pure per-frame alignment.
  • The per-frame encoder incurs the memory overhead the paper lists as a limitation, so smaller feature dimensions, frame subsampling, or a recurrent encoder could extend the method to longer horizons or higher-resolution observations.
  • The same per-frame alignment plus time-label reward could be applied to other adversarial imitation objectives, for example replacing the binary expert/non-expert sequence label with a continuous progress score, without changing the core mechanism.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes DIFF-IL, a cross-domain imitation learning method that operates on per-frame visual observations. It trains a shared encoder with domain-specific decoders, uses Wasserstein GAN alignment at both frame and sequence levels to remove domain-specific information, and introduces a frame-wise time labeling scheme in which a discriminator assigns higher rewards to frames from later timesteps of the source expert's episodes. The final reward is the product of a sequence label discriminator and a frame time-label discriminator, and the target policy is trained with SAC. The method is evaluated on eight Pendulum/DMC tasks and six MuJoCo locomotion tasks against TPIL, DeGAIL, GWIL, and D3IL, with reported final returns and learning curves, plus ablations of the main components and of WGAN hyperparameters.

Significance. If the empirical protocol were sound, DIFF-IL would be a useful and clearly presented contribution to cross-domain imitation from images. The per-frame alignment idea and the time-label reward are simple, plausible mechanisms, and the paper provides broad comparisons, component ablations, and qualitative image-mapping analyses. The reported gains over prior methods are large and consistent across most of the 14 tasks. However, two load-bearing issues weaken the current claims: the policy optimization appears to use true simulator states rather than visual observations, and the time-label reward encodes source-episode timing rather than goal progress, which may fail in source-faster-than-target settings. Both issues need to be addressed before the empirical claims can be taken at face value.

major comments (3)
  1. [Appendix A.3, Eqs. (A.10)-(A.11)] The SAC critic and actor are trained using true states s_t and s_{t+1} drawn from B_TL, as written in L_Q and L_pi. This directly contradicts Section 3.3, which states that the offline datasets are visual trajectories 'which lack access to true states,' and it contradicts the paper's framing as imitation from visual observations. If s_t is the simulator's true state, then the learner has privileged state information, and the comparison against TPIL, DeGAIL, and D3IL is not a fair test of vision-only imitation. If s_t denotes an encoded visual feature, the notation must be defined and used consistently. This is load-bearing for the central claim and must be resolved, either by clarifying the source of s_t or by rerunning the policy optimization on visual features alone.
  2. [Section 4.3, Eq. (4) and reward definition] The frame-wise time label y_t = (t/H_tilde + 1)/2 is defined as a function of source-episode time, and the final reward is monotone increasing in Flabel_f(z^T), so a target frame earns high reward only if it resembles a late-source frame. This assumes that 'late in the source expert episode' equals 'close to the goal for the target learner.' That identification fails when the source is faster than the target, as in Cheetah-to-Walker, where late-source Cheetah states may be unreachable for the Walker; the reward then pushes the learner toward infeasible states. Section 5.3 discusses the Hopper-as-source cap but not this speed asymmetry, and the ablation in Figure 9 does not include Cheetah-to-Walker. Please add an experiment or analysis that separates episode-time from goal progress, for example by comparing y_t with a state-based progress label or by analyzing reachability of late-source frames in the target domain, and report the time-label ablation for Cheetah-to-Walker.
  3. [Section 5.1 and Tables B.3-B.4] The WGAN hyperparameters lambda_disc and lambda_gen are tuned per task for DIFF-IL, as shown in Tables B.3 and B.4, while the baselines are not given the same per-task tuning or sensitivity analysis. Because the primary evidence for the method's advantage is the final-return comparison in Table 1, the protocol should tune baselines under the same budget or provide evidence that the baseline results are not artifacts of fixed hyperparameters. Without this, the claim that DIFF-IL 'significantly outperforms' the baselines is not fully supported.
minor comments (5)
  1. [Section 4.2, Eq. (2)] The unified WGAN loss is written as LWGAN = lambda_disc Ldisc + lambda_gen Ldisc; the second term should presumably be lambda_gen Lgen. Please fix the typo.
  2. [Section 4.3 vs. Eq. (A.6)] The main text defines the reward using F_label,f(z^T_{t+1}), while Eq. (A.6) uses F_chi_f(z^T_t). The indexing inconsistency matters for reproducibility; please clarify which observation time is used for the frame-label term.
  3. [Appendix B.2.1, Reacher tasks] The target range is written as 'r ∈ 0.15, 0.2', which is missing set notation; it should be 'r ∈ {0.15, 0.2}' or a continuous interval, as intended.
  4. [Appendix E.2] The heading says 'Generator Loss Coefficient λdisc' but the text discusses λgen; the heading should read λgen.
  5. [Appendix F, Limitations] The limitations section mentions memory usage, hyperparameter tuning, and the source-expert cap, but it does not mention the structural assumption behind the time-label reward that later source timesteps correspond to target goal progress; this dependency should be acknowledged and discussed.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the time-label reward is an explicit inductive bias, not a fitted target, and self-citations to D3IL are not load-bearing.

full rationale

The paper's derivation chain is self-contained rather than circular. The encoder p, decoders q, discriminators D_f and D_s, and label discriminators F_label,s and F_label,f are all defined by explicit loss functions (Eqs. 1, 2, and the frame-label loss in Sec. 4.3), and the reward is constructed from the learned discriminators as -log(1 - F_label,s * F_label,f). No component is defined in terms of the final performance metric or the target learner's return. The frame-wise time label y_t = (t/H_tilde + 1)/2 for source expert frames and 0 otherwise is a hand-specified, monotone function of the source expert's episode timestep; it is an inductive bias used to shape rewards, not a parameter fitted to the target domain's success. The paper's claim that later frames are 'closer to the goal' is an assumption about monotonic task progress, and the Limitations section explicitly acknowledges that target performance is constrained by the source expert's capabilities. Whether this assumption fails in speed-asymmetric tasks such as Cheetah-to-Walker is a correctness and robustness concern, not a circularity: the method does not secretly use target goal information to define the label. Self-citations to D3IL (Choi et al., 2024) appear for the consistency-loss inspiration and as a comparison baseline, but the central per-frame alignment and time-labeling mechanism is independently specified and empirically tested against external baselines. Thus there is no step where a prediction reduces by construction to a fitted input or to an unverified self-citation chain.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim rests on three categories of unpaid premises: task-specific assumptions (shared objective, monotonic time labels), architectural trust in adversarial domain alignment, and a hidden set of per-task free parameters (λdisc, λgen, plus exceptions in λrecon and λfcon). The method introduces no new physical or theoretical entities, so the invented-entity ledger is empty.

free parameters (5)
  • WGAN discriminator weight λdisc = Per-task: 1, 50, 0.5, 0.02, 0.1, 0.05 (Tables B.3, B.4)
    Searched per environment in [0.01, 50] to balance feature separation; varied across all 14 tasks, indicating task-specific fitting.
  • WGAN generator weight λgen = Per-task: 0.05, 1, 0.5, 10, 0.01, 0.1 (Tables B.3, B.4)
    Searched per environment in [0.01, 10]; inversely controls alignment strength; also task-specific.
  • WGAN control coefficient α = 0.5
    Ablated on two tasks (Fig. 10 and Appendix E.3); fixed at 0.5 globally, so a hand-chosen constant supported by limited evidence.
  • Reconstruction loss weight λrecon = 0.5 (Pendulum), 1 (MuJoCo), 0.5 (Hopper-to-Walker)
    Table B.2 shows per-domain exceptions, indicating tuning beyond the three headline hyperparameters.
  • Feature consistency weight λfcon = 1 default, 0.1 for IP-to-IDP and IDP-to-IP
    Table B.2 exception, not mentioned as tuned in the main text, so the paper understates the total tuning budget.
assumptions (4)
  • domain assumption Source and target domains share a common task objective, so reward templates derived from source expert temporal labels transfer to the target.
    Sec. 4.3 defines frame labels only from source expert episodes and uses them to reward the target learner; if the goals differ, the reward is meaningless.
  • ad hoc to paper Later timesteps in expert trajectories monotonically approach task completion, making the linear time label a valid progress signal.
    y_t = (t/H+1)/2 in Sec. 4.3; verified only on selected MuJoCo and DMC tasks, not generally, and the paper does not test tasks where the expert regresses or settles early.
  • domain assumption Adversarial domain alignment with reconstruction preserves task-relevant information while removing domain-specific information in the latent space.
    Standard domain-adversarial assumption (DANN, CycleGAN); underlies L_disc,f and L_gen,f with L_enc-dec in Sec. 4.1, and can fail if the reconstruction and adversarial losses conflict.
  • domain assumption The source expert data quality and the source environment reward suffice to define the behavior to imitate.
    Source expert collected with SAC in Sec. 5.2; no noise or suboptimality analysis, yet the whole method assumes the source expert is a reliable oracle.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Domain-Invariant Per-Frame Feature Extraction for Cross-Domain Imitation Learning with Visual Observations." pith.science (2026). https://pith.science/paper/PKGYO4IZ

@misc{pith2026250202867,
  author       = {Pith},
  title        = {Pith review of: Domain-Invariant Per-Frame Feature Extraction for Cross-Domain Imitation Learning with Visual Observations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PKGYO4IZ}},
  note         = {Machine review of arXiv:2502.02867}
}
read the original abstract

Imitation learning (IL) enables agents to mimic expert behavior without reward signals but faces challenges in cross-domain scenarios with high-dimensional, noisy, and incomplete visual observations. To address this, we propose Domain-Invariant Per-Frame Feature Extraction for Imitation Learning (DIFF-IL), a novel IL method that extracts domain-invariant features from individual frames and adapts them into sequences to isolate and replicate expert behaviors. We also introduce a frame-wise time labeling technique to segment expert behaviors by timesteps and assign rewards aligned with temporal contexts, enhancing task performance. Experiments across diverse visual environments demonstrate the effectiveness of DIFF-IL in addressing complex visual tasks.

Figures

Figures reproduced from arXiv: 2502.02867 by the authors.

Figure 1
Figure 1. t-SNE visualization of features extracted from: (a) Existing sequence-based IL methods, (b) DIFF-IL (ours). aims to move forward quickly by mimicking expert demon￾strations from Walker agents (source expert), despite differ￾ing dynamics [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Image mappings of DIFF based on aligned latent features in (a) Reacher, (b) Pendulum, and (c) MuJoCo tasks. Ldisc,f := Ez S t ∼p(·|o S t ), z T t ∼p(·|o T t ) h −Df (z S t ) + Df (z T t ) i + λgp,f · GP, Lgen,f := Ez S t ∼p(·|o S t ), z T t ∼p(·|o T t ) h Df (z S t ) − Df (z T t ) i , (1) Lenc-dec := X d=S,T Ez d t ∼p(·|o d t ) [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Illustration of frame-wise time labeling in [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (6 more)
Figure 5
Figure 5. Figure 5: Pendulum environments [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: MuJoCo environments 5.1. Experimental Setup For comparison, we evaluate cross-domain IL methods us￾ing images: TPIL (Stadie et al., 2017), which extracts domain-invariant features from image sequences; DeGAIL (Cetin & Celiktutan, 2021), which enhances domain in￾formati…
Figure 7
Figure 7. Figure 7: Performance comparison: Learning curves on MuJoCo over 5 random seeds, with results reported as means and standard deviations (shaded areas in graphs and ± values in tables). Additional details on the environments and offline data construction are provided in Appendix …
Figure 8
Figure 8. Figure 8: Image mapping and reward analysis on Walker-to-Cheetah task [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: Ablation study: Component evaluation [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]
Figure 10
Figure 10. Figure 10: Ablation study: WGAN control factor α including image mappings and reward evaluations for other environments, are detailed in Appendix D, demonstrating similar results consistent with the Walker-to-Cheetah case. 5.5. Ablation Studies Component Evaluation: To evaluate …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

39 extracted references · 30 canonical work pages

  1. [1]

    Details of the loss scale coefficients for all losses are summarized in Appendix B.3. A.2. Implementation of GP To ensure stable training of the adversarial network, the WGAN framework (Gulrajani et al., 2017a) incorporates a gradient penalty (GP) to enforce 1-Lipschitz continuity for the discriminator. In the redefined discriminator loss in Eq. (A.1), th...

    work page 2025

  2. [2]

    𝒕: 1.56Frame Label: 0.64Sequence Label: 0.88Estimatedreward 𝑹

    DMC Pendulum task: Figures D.4 and D.5 present the DMC Pend-to-CS and Pend-to-Acrobot tasks, highlighting the mapping and reward estimation in these challenging Pendulum environments. Initially, both frame and sequence label values are low, but they progressively increase as the agent approaches the goal, resulting in higher reward estimates. The rightmos...

    work page 2025

  3. [4]

    𝒕: 1.45FrameLabel: 0.76Sequence Label: 1.00Estimatedreward 𝑹

    For each task-specific figure, images are aligned by processing the Target Learner (TL) and Source Expert (SE) data through the encoder to extract domain-invariant features. The closest features between SE and TL are then matched for alignment. The images are arranged sequentially from left to right, showing the progression of timesteps. The bottom row di...

    work page 2025

  4. [5]

    Similarly, Fig. E.2, highlighting the impact of λdisc on MuJoCo tasks, indicates that Hopper-to-Cheetah is relatively unaffected by λdisc, whereas Walker-to-Cheetah exhibits notable sensitivity, achieving its best performance at λdisc = 0.5. These findings highlight the importance of balancing feature separation in adversarial training. Excessively large ...

    work page 2025

  5. [9]

    Generative adversarial nets

    9 Submission and Formatting Instructions for ICML 2025 Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, Sherjil axu2022discriminatornd Courville, A., and Ben- gio, Y . Generative adversarial nets. In Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N. D., and Weinberger, K. Q. (eds.), Advances in Neural Information Processi...

    work page 2025

  6. [11]

    Robust imitation learning for mobile manipulator focusing on task-related viewpoints and regions

    Ishida, Y ., Noguchi, Y ., Kanai, T., Shintani, K., and Bito, H. Robust imitation learning for mobile manipulator focusing on task-related viewpoints and regions. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2885–2892. IEEE,

    work page 2024

  7. [12]

    Adversar- ial imitation learning from video using a state observer

    Karnan, H., Torabi, F., Warnell, G., and Stone, P. Adversar- ial imitation learning from video using a state observer. In 2022 International Conference on Robotics and Au- tomation (ICRA), pp. 2452–2458. IEEE, 2022a. Karnan, H., Warnell, G., Xiao, X., and Stone, P. V oila: Visual-observation-only imitation learning for au- tonomous navigation. In 2022 Int...

    work page 2022

  8. [13]

    Efficient exploration via state marginal matching

    Lee, L., Eysenbach, B., Parisotto, E., Xing, E., Levine, S., and Salakhutdinov, R. Efficient exploration via state marginal matching. arXiv preprint arXiv:1906.05274 ,

    arXiv 1906

Show all 39 references
  1. [14]

    L., and Ghanem, B

    Li, G., Mueller, M., Casser, V ., Smith, N., Michels, D. L., and Ghanem, B. Oil: Observational imitation learning. arXiv preprint arXiv:1803.01129,

    arXiv

  2. [15]

    Imitation from observation: Learning to imitate behaviors from raw video via context translation

    Liu, Y ., Gupta, A., Abbeel, P., and Levine, S. Imitation from observation: Learning to imitate behaviors from raw video via context translation. In 2018 IEEE Interna- tional Conference on Robotics and Automation (ICRA), pp. 1118–1125. IEEE,

    2018

  3. [16]

    Versatile offline imitation from observations and examples via reg- ularized state-occupancy matching

    10 Submission and Formatting Instructions for ICML 2025 Ma, Y ., Shen, A., Jayaraman, D., and Bastani, O. Versatile offline imitation from observations and examples via reg- ularized state-occupancy matching. In International Con- ference on Machine Learning, pp. 14639–14663. PMLR,

    2025

  4. [18]

    Time-contrastive networks: Self-supervised learning from video

    Sermanet, P., Lynch, C., Chebotar, Y ., Hsu, J., Jang, E., Schaal, S., Levine, S., and Brain, G. Time-contrastive networks: Self-supervised learning from video. In 2018 IEEE international conference on robotics and automa- tion (ICRA), pp. 1134–1141. IEEE,

    2018

  5. [20]

    Tassa, Y ., Doron, Y ., Muldal, A., Erez, T., Li, Y ., Casas, D. d. L., Budden, D., Abdolmaleki, A., Merel, J., Lefrancq, A., et al. Deepmind control suite. arXiv preprint arXiv:1801.00690,

    arXiv

  6. [21]

    Mujoco: A physics engine for model-based control

    Todorov, E., Erez, T., and Tassa, Y . Mujoco: A physics engine for model-based control. In Intelligent Robots and Systems (IROS), 2012 IEEE/RSJ International Con- ference on, pp. 5026–5033. IEEE,

    2012

  7. [23]

    Decomposing the generalization gap in imitation learning for visual robotic manipulation

    Xie, A., Lee, L., Xiao, T., and Finn, C. Decomposing the generalization gap in imitation learning for visual robotic manipulation. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pp. 3153–3160. IEEE,

    2024

  8. [24]

    Empirical evaluation of rectified activations in con- volutional network

    Xu, B. Empirical evaluation of rectified activations in con- volutional network. arXiv preprint arXiv:1505.00853 ,

    arXiv

  9. [26]

    Cross domain robot imitation with invariant representa- tion

    Yin, Z.-H., Sun, L., Ma, H., Tomizuka, M., and Li, W.-J. Cross domain robot imitation with invariant representa- tion. In 2022 International Conference on Robotics and Automation (ICRA), pp. 455–461. IEEE,

    2022

  10. [29]

    The GP term enforces Lipschitz continuity on the discriminator, stabilizing adversarial training by mitigating extreme gradients and promoting smooth convergence

    serves as the interpolation factor. The GP term enforces Lipschitz continuity on the discriminator, stabilizing adversarial training by mitigating extreme gradients and promoting smooth convergence. Additionally, we maintain a 5:1 training ratio between the discriminator and g...

    2018

  11. [30]

    The final output is flattened and passed through a dense layer with 32 units

    to mitigate vanishing gradient issues. The final output is flattened and passed through a dense layer with 32 units. • Decoders (qS ψS , qT ψT ): Reconstructs data from encoded features usingConvTranspose (transposed convolutional lay- ers) with 64 and 32 filters. It upsamples...

    2025

  12. [31]

    Architectural specifications of the proposed networks

    Dense(32) WGAN discriminators (Dζf , Dζs ) BatchNorm() Label discriminators (Fχf , Fχs ) BatchNorm() Dense(400, LeakyReLU) Dense(400, LeakyReLU) Dense(300, LeakyReLU) Dense(300, LeakyReLU) Dense(1) Flatten Dense(1, Sigmoid) Critic (Qθ) Dense(256, ReLU) Target learner policy (π...

    2025

  13. [32]

    For BSR and BT R, random policies are used for data collection

    and collect samples from πSE . For BSR and BT R, random policies are used for data collection. In tasks like IP, IDP, Pendulum, CS, and Acrobot, where random policies can sustain extended downward pole positions, episode lengths vary between expert and random policies. Detaile...

    2021

  14. [35]

    It uses an encoder to extract domain-independent features, a domain discriminator to differentiate domains, and a label discriminator to classify expert and non-expert behaviors

    with GAIL (Ho & Ermon, 2016). It uses an encoder to extract domain-independent features, a domain discriminator to differentiate domains, and a label discriminator to classify expert and non-expert behaviors. A gradient reversal layer optimizes these components simultaneously,...

    2016

  15. [36]

    A discriminator generates rewards by distinguishing between expert and learner behaviors

    enhances feature extraction using dual encoders for domain-specific and behavior-specific features, with discriminators refining extraction accuracy through cycle-consistency and reconstruction. A discriminator generates rewards by distinguishing between expert and learner beh...

    2018

  16. [200]

    Image resolution for each task was configured to the minimum level required for clear agent distinction, optimizing memory usage while maintaining sufficient visual detail

    The state dimensions, action dimensions, image sizes, and episode lengths for all environments are summarized in Table B.1. Image resolution for each task was configured to the minimum level required for clear agent distinction, optimizing memory usage while maintaining suffic...

    2025

  17. [1000]

    The target position is defined in polar coordinates, with r ∈ 0.15, 0.2 and 16 Submission and Formatting Instructions for ICML 2025 φ ∈ 0, π/4, π/2,

    Reacher Tasks (RE2, RE3) : These tasks involve controlling a robotic arm with two (RE2) or three (RE3) joints to reach one of 16 randomly assigned targets. The target position is defined in polar coordinates, with r ∈ 0.15, 0.2 and 16 Submission and Formatting Instructions for...

    2025

  18. [2000]

    ISBN 1558607072

    Mor- gan Kaufmann Publishers Inc. ISBN 1558607072. Peng, X. B., Kanazawa, A., Toyer, S., Abbeel, P., and Levine, S. Variational discriminator bottleneck: Improving imita- tion learning, inverse rl, and gans by constraining infor- mation flow. arXiv preprint arXiv:1810.00821,

    arXiv

  19. [2004]

    Environmental and behavioral imitation for autonomous navigation

    Aoki, J., Sasaki, F., Matsumoto, K., Yamashina, R., and Kurazume, R. Environmental and behavioral imitation for autonomous navigation. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 7779–7786. IEEE,

    2024

  20. [2006]

    and Celiktutan, O

    Cetin, E. and Celiktutan, O. Domain-robust visual imita- tion learning with mutual information constraints. arXiv preprint arXiv:2103.05079,

    arXiv

  21. [2008]

    12 Submission and Formatting Instructions for ICML 2025 A

    ISBN 9781577353683. 12 Submission and Formatting Instructions for ICML 2025 A. Detailed Implementation of DIFF-IL In this section, we detail the implementation of the proposed methods in DIFF-IL. Section A.1 redefines the loss functions, incorporating loss scales and network p...

    2025

  22. [2012]

    Behavioral cloning from observation

    Torabi, F., Warnell, G., and Stone, P. Behavioral cloning from observation. In Proceedings of the 27th Inter- national Joint Conference on Artificial Intelligence , IJ- CAI’18, pp. 4950–4957. AAAI Press, 2018a. ISBN 9780999241127. Torabi, F., Warnell, G., and Stone, P. Generat...

    2025 arXiv

  23. [2014]

    Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V ., and Courville, A. C. Improved training of wasserstein gans. Advances in neural information processing systems, 30, 2017a. Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V ., and Courville, A. C. Improved training of was...

    arXiv

  24. [2015]

    A policy-guided imi- tation approach for offline reinforcement learning

    Xu, H., Jiang, L., Li, J., and Zhan, X. A policy-guided imi- tation approach for offline reinforcement learning. In Oh, A. H., Agarwal, A., Belgrave, D., and Cho, K. (eds.), Advances in Neural Information Processing Systems , 2022a. URL https://openreview.net/forum? id=CKbqDtZ...

    arXiv

  25. [2017]

    Learning robust rewards with adverserial inverse reinforcement learning

    Fu, J., Luo, K., and Levine, S. Learning robust rewards with adverserial inverse reinforcement learning. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenReview.net,

    2018

  26. [2018]

    Imitation learning from observations under transition model disparity

    Gangwani, T., Zhou, Y ., and Peng, J. Imitation learning from observations under transition model disparity. arXiv preprint arXiv:2204.11446,

    arXiv

  27. [2019]

    An integrated frame- work for human–robot collaborative manipulation

    Sheng, W., Thobbi, A., and Gu, Y . An integrated frame- work for human–robot collaborative manipulation. IEEE transactions on cybernetics, 45(10):2030–2041,

  28. [2020]

    Model-based inverse reinforcement learning from visual demonstrations

    Das, N., Bechtle, S., Davchev, T., Jayaraman, D., Rai, A., and Meier, F. Model-based inverse reinforcement learning from visual demonstrations. In Conference on Robot Learning, pp. 1930–1942. PMLR,

    1930

  29. [2022]

    Chang, W.-D., Higuera, J. C. G., Fujimoto, S., Meger, D., and Dudek, G. Il-flow: Imitation learning from observation using normalizing flows. arXiv preprint arXiv:2205.09251,

    arXiv

  30. [2023]

    Learning robust rewards with adversarial inverse reinforcement learning

    Fu, J., Luo, K., and Levine, S. Learning robust rewards with adversarial inverse reinforcement learning. arXiv preprint arXiv:1710.11248,

    arXiv

  31. [2024]

    Primal wasserstein imitation learning

    Dadashi, R., Hussenot, L., Geist, M., and Pietquin, O. Primal wasserstein imitation learning. arXiv preprint arXiv:2006.04678,

    2006 arXiv

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.