Offline Learning of Controllable Diverse Behaviors

Ludovic Denoyer; Mathieu Petitbois; R\'emy Portelas; Sylvain Lamprier

REVIEW 4 major objections 3 minor 1 cited by

Offline Learning of Controllable Diverse Behaviors

T0 review · 4 major / 3 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read By conditioning imitation on per-trajectory style codes, offline agents can reproduce the full mix of human demonstration behaviors and steer generation by trajectory properties.

desk verdict Reasonable core idea (WZBC's similarity-weighted regression) but the main diversity claim rests on an in-sample memorization comparison; the evaluation needs a redesign before the claims hold. read the letter →

arxiv 2504.18160 v1 pith:WMB3SA4S submitted 2025-04-25 cs.LG cs.AIcs.RO

Mathieu Petitbois , Rémy Portelas , Sylvain Lamprier , Ludovic Denoyer This is my paper · ORCID

classification cs.LGcs.AIcs.RO

keywords imitationlearningofflinediversebehaviorstrajectory-leveldiversitystyleconditioningcontrollablegenerationsimilarityweightedregressionbehaviorhistograms

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes that capturing the diversity of human demonstrations offline requires conditioning a behavior-cloning policy on a learned latent 'style' vector associated with each demonstration trajectory, rather than reproducing the action distribution at each state independently. It introduces ZBC, which gives every trajectory its own trainable embedding, and WZBC, a relaxed version that blends styles according to trajectory similarity so the policy stays robust under environment stochasticity. Across maze and robot-manipulation datasets, the authors report that these methods generate trajectories whose behavior histogram (checkpoint sequences or task metrics) is closer to the demonstration histogram than prior multimodal imitation methods, while also allowing users to condition generation on properties such as trajectory length. If true, the significance is a simple, fully offline route to controllable generation of a specified mixture of human-like behaviors, useful for game bots, robot pre-training, and simulation.

What carries the argument

The central object is the style code: a low-dimensional trainable embedding associated with each demonstration trajectory, which the policy conditions on. The key mechanism is similarity-weighted regression (SWR): with ZBC, the policy maximizes the likelihood of each trajectory's actions under its own style embedding; with WZBC, actions from one trajectory are cloned under another trajectory's style, weighted by an exponential kernel exp(−β·ν(·,·)) based on a Euclidean trajectory dissimilarity over padded states. This weighting, controlled by β and a stop-gradient on relabeled styles, lets the model interpolate between a single global policy and fully per-trajectory conditioning, yielding controllability and robustness in one framework.

What would settle it

Train ZBC and WZBC on a maze dataset with two behavior modes that visit the same checkpoint sequence but differ in an unmeasured property such as speed profile, then evaluate both the checkpoint-sequence histogram and a finer measure like state-visitation frequency over a dense grid; if the checkpoint L1 distance is near zero while the state-visitation distance is large, the evaluation metric fails to capture trajectory distribution and the paper's claim of reproducing pM,μ(τ) is unsupported.

Watch

Extended reading notes

Core claim

The central discovery is that trajectory-scale diversity in imitation learning can be captured by a simple conditioning trick: assign each demonstration a trainable latent embedding z (the 'style'), learn a policy π(a|s,z) that clones that demonstration's actions, and at generation time sample styles according to the demonstration distribution. ZBC instantiates this with a look-up table of embeddings; WZBC relaxes it by regressing actions from trajectory i against the style of trajectory j, weighted by exp(−β·ν(τ_i,τ_j)) for a dissimilarity ν, interpolating between behavioral cloning and hard style conditioning. The authors claim this reproduces the actual trajectory distribution (measured by L1 distance of discretized behavior histograms) better than previous transition-level and trajectory-level multimodal methods, while providing controllability by conditioning the style-sampling distribution on trajectory properties.

Load-bearing premise

The load-bearing premise is that the L1 distance between hand-defined behavior histograms, computed with zero-padding over checkpoint sequences or task metrics, faithfully measures whether the generated trajectory distribution matches the demonstration distribution; if those metrics ignore properties that matter (e.g., speed profiles, subtle path variations), the reported diversity scores do not establish the method's central claim.

Editorial extensions

If this is right

If the histogram-L1 results hold, ZBC offers a drop-in offline baseline: a dataset of indexed demonstrations plus a small MLP policy can reproduce the behavior mixture without reward labels or environment interaction.
Style conditioning gives a free control interface: by sampling styles only from trajectories satisfying a property (e.g., length in [70,80]), the generated trajectories concentrate on that property without retraining.
WZBC's robustness gains in stochastic environments suggest that style relaxation, rather than hard per-trajectory codes, is the right operating point for real-world deployments with state noise or variable initial states.
The L1 histogram evaluation, if adopted, shifts assessment of diverse imitation from entropy maximization to faithful reproduction of a given behavior distribution, which is closer to practical needs in games and robotics.
Because ZBC requires no encoder network, it scales cheaply to large offline datasets with many trajectories, at the cost of a lookup table.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural extension the authors do not pursue is combining style conditioning with trajectory-generation models such as transformers or diffusion, so controllability applies to long-horizon generation without per-timestep policy unrolling.
The controllability mechanism, restricting the style-sampling distribution, is closely analogous to classifier-free guidance in diffusion models; one could test whether the same selection rule yields calibrated conditional distributions under metric filters other than length, such as reward or safety constraints.
Because the behavior histogram is hand-defined per task, the method's apparent diversity capture is only as good as the chosen metrics; a stronger test would measure the full state-visitation or action-sequence distribution, which the paper does not do.
The dissimilarity weighting suggests a principled way to handle dataset imbalance: rare behaviors could be up-weighted by tuning β or by re-sampling styles, which might improve the unbalanced-dataset scores reported.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The shortest take: the paper has a genuinely useful nugget, but its headline claim is not supported by the evidence as presented. The useful nugget is WZBC, the similarity-weighted regression objective. Per-trajectory conditioning itself (ZBC) is a simple extension of existing trajectory-conditioning ideas, but the weighting scheme that interpolates between BC and ZBC is a sensible and clean twist. The histogram-based L1 evaluation metric is also a step forward over entropy-based diversity metrics, and the paper is clearly written. The problem is Table 1. ZBC is evaluated by sampling the exact style codes it was trained on and rolling out in a deterministic environment. A policy that stores one embedding per training trajectory can essentially replay each trajectory, which obviously matches the training histogram. The diffusion and energy-based baselines don't have that per-trajectory lookup, so the comparison is not apples-to-apples. The authors themselves note in Section 3.3 that 'ZBC tends to overfit,' and the robustness table (Table 2) confirms it: under random initialization or noisy transitions, ZBC's L1 jumps from 0.256 to 1.556 or 0.729, meaning it cannot transfer its indexed behaviors to unseen states. WZBC is more robust, but Table 2 only compares ZBC and WZBC — no baselines — so we still don't know how it stacks up in realistic conditions. There's also a smaller internal inconsistency: the conclusion says 'ZBC and WZBC displayed the best performance in diversity capture,' but in Table 1 WZBC is worst on d3ilunbalancedavoiding and worse than BESO on several rows. Only ZBC consistently tops the table. That's a minor overstatement, but combined with the memorization issue it matters. The controllability section (conditioning on trajectory length) works, but almost trivially: since each code corresponds to one training trajectory, filtering codes by a property simply selects the relevant training trajectories. It demonstrates control, but not a learned, compact behavior space. And there is no released code, which makes the quantitative results hard to verify. Who is this for? Researchers in diverse imitation learning or game AI who want a quick, simple baseline idea. As evidence for generalizable offline diversity capture, it's not there yet. I would send this to peer review rather than desk-reject, but with strong requests: add a held-out style test (train on a subset of trajectories, evaluate on the rest), include the baselines in the robustness study, and release code. The SWR idea deserves further development; the current evaluation doesn't yet show it delivers.

Referee Report

4 major / 3 minor

Summary. The paper studies offline imitation learning from demonstrations containing multiple distinct behaviors. It proposes ZBC, which clones each training trajectory conditioned on a per-trajectory embedding, and WZBC, which weights behavioral cloning by a trajectory-dissimilarity term to trade off between memorization and robustness. The evaluation uses L1 distances between behavior histograms on Maze2D and D3IL tasks, together with controllability and stochastic-robustness experiments. The authors claim state-of-the-art diversity capture and robustness for their methods relative to prior multi-modal imitation learning approaches.

Significance. The problem is relevant, and the idea of conditioning a policy on a style embedding to capture trajectory-level diversity is simple and practical. If the diversity-reconstruction claim were properly established, the paper would be a useful contribution to offline imitation learning. The clear problem formulation, the proposed histograms-of-behaviors evaluation, and the visualizations of conditional state distributions are strengths. However, the reported experiments are insufficient: the primary diversity metric can be minimized by memorizing training trajectories, several comparative claims are not supported by the tables, and the controllability and robustness statements are evaluated in-sample or without baselines. These issues are addressable with additional experiments and a more careful framing.

major comments (4)

[Sec. 3.2, Eq. (1)-(2); Sec. 4.3, Table 1] The low L1 distances in Table 1 do not establish that ZBC captures the demonstration distribution in a generalizable sense. ZBC learns a separate embedding z_i for each training trajectory and samples uniformly over these embeddings at test time; in a deterministic environment this is effectively replaying training trajectories. Because the histogram in Table 1 is computed on the same training behaviors, a policy that memorizes each trajectory will score well by construction. The baselines, which do not receive a per-trajectory key, are therefore not compared on equal footing. The stochastic-evaluation rows in Table 2 corroborate the memorization interpretation: ZBC's L1 degrades from 0.256 to 1.556 under random initialization, showing that the low scores do not reflect transferable diversity capture. A held-out evaluation, such as training on a subset of trajectories and testing on the rest, or a comparison that removes the per-trajectory index, is needed before the central claim can be accepted.
[Sec. 5; Table 1] The conclusion that 'ZBC and WZBC displayed the best performance in diversity capture compared to previous multi-modal imitation learning methods' is not supported for WZBC. On d3ilunbalancedavoiding, WZBC has L1 = 1.457 +/- 0.087, which is worse than BESO's 1.283 +/- 0.067; on d3ilunbalancedaligning, WZBC has L1 = 0.364 +/- 0.037, worse than BESO's 0.256 +/- 0.066 and DDPM-ACT's 0.212 +/- 0.053. Only ZBC consistently ranks first in the diversity rows. The conclusion should be revised to claim best performance for ZBC alone, or the authors should provide additional evidence for WZBC's competitiveness.
[Sec. 4.5, Table 2; Sec. 5] The statement that 'WZBC displayed some robustness to environment stochasticity compared to the baselines' is not supported by the reported experiments, because Table 2 compares only ZBC and WZBC under stochastic configurations. No baseline is evaluated under pseudo-r-init, r-init, or noise-transi, so no conclusion relative to BESO, DDPM-ACT, or DDPM-GPT can be drawn. The robustness claim should be restricted to a comparison between ZBC and WZBC, or the missing baseline experiments should be added.
[Sec. 4.4, Eq. (9)] The controllability evaluation in Section 4.4 also relies on the training style codes: the conditional style distribution e(z|Psi(tau)) is computed by filtering training trajectories by the property and then sampling uniformly among their embeddings. In a deterministic environment, this tests whether the policy can replay the subset of training trajectories satisfying the property; it does not test whether the learned latent space supports control over unseen or interpolated behaviors. The authors should either frame the controllability result as a selection mechanism over the training set or add experiments that condition on property values not present in the training trajectories.

minor comments (3)

[Sec. 3.3, Eq. (5)] The dissimilarity defined in Eq. (5) is not symmetric despite the text calling it symmetric, because the denominator depends on the first argument. If the stored nu_{i,j} values are used with ordered pairs, this should be stated; otherwise the definition should be changed, for instance by normalizing with a dataset-wide maximum.
[Sec. 4.3, Table 1] No statistical significance tests are reported, and several standard-deviation intervals overlap, such as the pseudo-r-init L1 distances in Table 2 for ZBC (1.152 +/- 0.094) and WZBC (0.828 +/- 0.349). Reporting confidence intervals or significance tests would help the reader assess the claimed rankings.
[Sec. 3.1 and Sec. 3.2] The abstract and introduction emphasize 'Temporal Consistency', but the policy is Markovian in state and style; temporal consistency is represented only through the style code. The paper should clarify this point to avoid overclaiming the degree to which the method itself encodes temporal structure.

Circularity Check

1 steps flagged · score 6.0 of 10

ZBC's per-trajectory codebook (Eqs. 1-2) makes the Table 1 diversity-reconstruction result an in-sample fit; the claimed 'best performance in diversity capture' is not established as a generalizable prediction.

fitted input called prediction [Section 3.2 (Eqs. 1-2) with evaluation in Section 4.3, Table 1]
"Rather, we build on a simpler approach where the encoder is replaced by an embedding matrix that deterministically associates a trainable embedding vector to each trajectory index for a given dataset of indexed trajectories D = {τi}: eϕ(z|τi) = δzi(z). ... With ZBC, we can simply regenerate pM,π(τ) ≈ pM,μ(τ) by sampling uniformly a latent vector from the style cookbook {zi, i = 0, ..., |D| − 1}. ... To quantify the distance between pM,π(τ) and pM,μ(τ), we use the L1-distance between these corresponding behavior histograms with zero-padding."

Each z_i is a trainable embedding fitted exclusively to trajectory τ_i (Eq. 1), and Eq. 2 defines the predicted distribution as the uniform mixture over exactly those per-trajectory codes. The §4.3 metric then compares the rollout histogram with the histogram of the same training trajectories. In the deterministic evaluation environments, a policy that clones each training trajectory under its own code reproduces the training histogram by construction, so Table 1 primarily measures in-sample codebook memorization rather than discovery of a transferable behavior space. Table 2 supports this reading: under random initialization ZBC's L1 jumps to 1.556, showing the per-trajectory codes do not index behaviors at unseen states.

full rationale

The paper contains no load-bearing self-citation chain: the baselines (BESO, DDPM-ACT, DDPM-GPT) and D3IL datasets come from external prior work, and no uniqueness theorem or cited ansatz is used to force the method. The central issue is that the headline diversity-capture result is computed on the same per-trajectory codes that were fit to the training trajectories. Eq. 1 gives each demonstration its own trainable style vector; Eq. 2 samples uniformly over those vectors; and the histogram metric in §4.3 compares the resulting rollouts to the histograms of those same demonstrations. Hence Table 1's low L1 scores are close to a memorization/replay check, not an out-of-sample test of whether the latent space generalizes to new states or novel styles. The robustness table (Table 2) inadvertently confirms this: perturbing initialization or transitions degrades ZBC's L1 substantially. I therefore score 6 rather than 8 because WZBC's off-diagonal weighting is a genuine generalization mechanism and the paper includes robustness experiments that expose the limitation; the conclusion's overstatement about WZBC's Table 1 rank is an internal consistency issue, not circularity.

Assumptions & free parameters 3 free parameters · 4 assumptions · 1 invented entities

The central method rests on a small set of modeling choices: a per-trajectory finite mixture assumption about the data, a Euclidean trajectory dissimilarity, and a histogram-based evaluation metric. There are hand-chosen hyperparameters (beta, p, style dimension) and no formal verification. The style latent is a learned representation with only in-distribution validation.

free parameters (3)

beta = 10.0
Bandwidth in the WZBC weighting exp(-beta * nu) (Eq. 4). It controls the tradeoff between BC (beta = 0) and ZBC (large beta) and is chosen by hand.
p = 0.8
Probability of sampling an off-diagonal trajectory pair in Algorithm 1. This hyperparameter governs the balance between per-trajectory and cross-trajectory updates and is chosen by hand.
style_dim = 10
Dimension of the per-trajectory latent style embeddings. Chosen by hand for all environments.

assumptions (4)

domain assumption The demonstration dataset was generated by a finite set of stylized expert policies Pi_e with unknown number K and unknown sampling distribution mu(pi).
Section 3.1 introduces this generative model; the method and the evaluation by behavior histograms assume it.
ad hoc to paper The Euclidean distance on padded state sequences (Eq. 5) is a valid dissimilarity measure for behavior similarity.
WZBC's weighting depends on this metric. The authors themselves note in Section 5 that this distance might struggle in high-dimensional data.
ad hoc to paper L1 distance between hand-defined behavior histograms, with zero-padding, measures similarity between pM,pi(tau) and pM,mu(tau).
Section 4.3 uses this as the primary diversity metric. It is not shown to be a sufficient statistic for trajectory distributions.
domain assumption Filtering style codes by a trajectory property (Eq. 8) yields a good conditional style distribution e(z|Psi).
Section 4.4 assumes this without derivation; the paper demonstrates it only in low-stochasticity settings.

invented entities (1)

style latent z
purpose: A low-dimensional code per trajectory that conditions the policy to generate a specific behavior; central to ZBC and WZBC.
The style representation is validated through in-distribution diversity and controllability experiments on the training set. No external, out-of-domain falsifiable handle is provided.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Offline Learning of Controllable Diverse Behaviors." pith.science (2026). https://pith.science/paper/WMB3SA4S

@misc{pith2026250418160,
  author       = {Pith},
  title        = {Pith review of: Offline Learning of Controllable Diverse Behaviors},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WMB3SA4S}},
  note         = {Machine review of arXiv:2504.18160}
}

read the original abstract

Imitation Learning (IL) techniques aim to replicate human behaviors in specific tasks. While IL has gained prominence due to its effectiveness and efficiency, traditional methods often focus on datasets collected from experts to produce a single efficient policy. Recently, extensions have been proposed to handle datasets of diverse behaviors by mainly focusing on learning transition-level diverse policies or on performing entropy maximization at the trajectory level. While these methods may lead to diverse behaviors, they may not be sufficient to reproduce the actual diversity of demonstrations or to allow controlled trajectory generation. To overcome these drawbacks, we propose a different method based on two key features: a) Temporal Consistency that ensures consistent behaviors across entire episodes and not just at the transition level as well as b) Controllability obtained by constructing a latent space of behaviors that allows users to selectively activate specific behaviors based on their requirements. We compare our approach to state-of-the-art methods over a diverse set of tasks and environments. Project page: https://mathieu-petitbois.github.io/projects/swr/

Figures

Figures reproduced from arXiv: 2504.18160 by the authors.

**Figure 2.** (Left) Pictures of the environments. (Right) Histograms of the behavior distribution of [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

**Figure 3.** Dissimilarity values of trajectories ν(τ ∗ , τ ) for different reference trajectories τ ∗ in red. Blue trajectories are the most similar, green the most dissimilar. β = 0 β = 3.0 β = 100.0 [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: Values of the conditional input sample distributions ρ(˜s|z). β = 0 give similar input sample distribution as BC, while β = 100 give a similar input sampling distribution as ZBC. We can find a middle ground with β = 3.0, allowing all the support of BC but with a signif…

**Figure 5.** Figure 5: (Left) L1 distance between the training behavior histogram and respectively: the prop [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

When Reasoning Narrows the Move: Diversity Collapse in LLM Game Play
cs.CL 2026-07 conditional novelty 6.0 of 10

Supervised fine-tuning collapses LLM action diversity in board-game play beyond what the accuracy–diversity tradeoff requires; augmenting SFT data with all optimal actions per state partially prevents this.

Reference graph

Works this paper leans on

40 extracted references · 24 canonical work pages · cited by 1 Pith paper

[1]

Tenenbaum, Tommi S

Anurag Ajay, Yilun Du, Abhi Gupta, Joshua B. Tenenbaum, Tommi S. Jaakkola, and Pulkit Agrawal. Is conditional generative modeling all you need for decision making? In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=sP1fo2K9DFG

work page 2023
[2]

Testing, validation, and verification of robotic and autonomous systems: A systematic review

Hugo Araujo, Mohammad Reza Mousavi, and Mahsa Varshosaz. Testing, validation, and verification of robotic and autonomous systems: A systematic review. ACM Trans. Softw. Eng. Methodol., 32 0 (2), mar 2023. ISSN 1049-331X. doi:10.1145/3542945. URL https://doi.org/10.1145/3542945

doi:10.1145/3542945 2023
[3]

Champandard

Alex J. Champandard. Ai Game Development. 2003

work page 2003
[4]

Decision transformer: Reinforcement learning via sequence modeling, 2021

Lili Chen, Kevin Lu, Aravind Rajeswaran, Kimin Lee, Aditya Grover, Michael Laskin, Pieter Abbeel, Aravind Srinivas, and Igor Mordatch. Decision transformer: Reinforcement learning via sequence modeling, 2021

work page 2021
[5]

Diffusion policy: Visuomotor policy learning via action diffusion, 2024

Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion, 2024

2024
[6]

Implicit behavioral cloning, 2021

Pete Florence, Corey Lynch, Andy Zeng, Oscar Ramirez, Ayzaan Wahid, Laura Downs, Adrian Wong, Johnny Lee, Igor Mordatch, and Jonathan Tompson. Implicit behavioral cloning, 2021

2021
[7]

Off-policy deep reinforcement learning without exploration, 2019

Scott Fujimoto, David Meger, and Doina Precup. Off-policy deep reinforcement learning without exploration, 2019

2019
[8]

Multi-modal imitation learning from unstructured demonstrations using generative adversarial nets, 2017

Karol Hausman, Yevgen Chebotar, Stefan Schaal, Gaurav Sukhatme, and Joseph Lim. Multi-modal imitation learning from unstructured demonstrations using generative adversarial nets, 2017

work page 2017

Show all 40 references

[9]

Generative adversarial imitation learning, 2016

Jonathan Ho and Stefano Ermon. Generative adversarial imitation learning, 2016

2016
[10]

ArtificialIntelligenceforGames

Millington Ian and Funge John. ArtificialIntelligenceforGames. 2009

2009
[11]

Offline reinforcement learning as one big sequence modeling problem

Michael Janner, Qiyang Li, and Sergey Levine. Offline reinforcement learning as one big sequence modeling problem. In Advances in Neural Information Processing Systems, 2021

2021
[12]

Tenenbaum, and Sergey Levine

Michael Janner, Yilun Du, Joshua B. Tenenbaum, and Sergey Levine. Planning with diffusion for flexible behavior synthesis, 2022

2022
[13]

Towards diverse behaviors: A benchmark for imitation learning with human demonstrations, 2024

Xiaogang Jia, Denis Blessing, Xinkai Jiang, Moritz Reuss, Atalay Donat, Rudolf Lioutikov, and Gerhard Neumann. Towards diverse behaviors: A benchmark for imitation learning with human demonstrations, 2024

2024
[14]

Offline reinforcement learning with implicit q-learning, 2021

Ilya Kostrikov, Ashvin Nair, and Sergey Levine. Offline reinforcement learning with implicit q-learning, 2021

2021
[15]

Conservative q-learning for offline reinforcement learning, 2020

Aviral Kumar, Aurick Zhou, George Tucker, and Sergey Levine. Conservative q-learning for offline reinforcement learning, 2020

2020
[16]

When should we prefer offline reinforcement learning over behavioral cloning?, 2022

Aviral Kumar, Joey Hong, Anikait Singh, and Sergey Levine. When should we prefer offline reinforcement learning over behavioral cloning?, 2022

2022
[17]

Human-level ai’s killer application: Interactive computer games

John Laird and Michael VanLent. Human-level ai’s killer application: Interactive computer games. AI Magazine, 22 0 (2): 0 15, Jun. 2001. doi:10.1609/aimag.v22i2.1558. URL https://ojs.aaai.org/aimagazine/index.php/aimagazine/article/view/1558

2001 doi
[18]

Cic: Contrastive intrinsic control for unsupervised skill discovery, 2022

Michael Laskin, Hao Liu, Xue Bin Peng, Denis Yarats, Aravind Rajeswaran, and Pieter Abbeel. Cic: Contrastive intrinsic control for unsupervised skill discovery, 2022

2022
[19]

Infogail: Interpretable imitation learning from visual demonstrations, 2017

Yunzhu Li, Jiaming Song, and Stefano Ermon. Infogail: Interpretable imitation learning from visual demonstrations, 2017

2017
[20]

What matters in learning from offline human demonstrations for robot manipulation, 2021

Ajay Mandlekar, Danfei Xu, Josiah Wong, Soroush Nasiriany, Chen Wang, Rohun Kulkarni, Li Fei-Fei, Silvio Savarese, Yuke Zhu, and Roberto Martín-Martín. What matters in learning from offline human demonstrations for robot manipulation, 2021

2021
[21]

Stylized offline reinforcement learning: Extracting diverse high-quality behaviors from heterogeneous datasets

Yihuan Mao, Chengjie Wu, Xi Chen, Hao Hu, Ji Jiang, Tianze Zhou, Tangjie Lv, Changjie Fan, Zhipeng Hu, Yi Wu, Yujing Hu, and Chongjie Zhang. Stylized offline reinforcement learning: Extracting diverse high-quality behaviors from heterogeneous datasets. In The Twelfth Internati...

2024
[22]

Behavioral Mathematics for Game AI

Dave Mark. Behavioral Mathematics for Game AI. 2009

2009
[23]

Three states and a plan: The a.i

Jeff Orkin. Three states and a plan: The a.i. of f.e.a.r. 2006. URL https://api.semanticscholar.org/CorpusID:62493110

2006
[24]

Imitating human behaviour with diffusion models, 2023

Tim Pearce, Tabish Rashid, Anssi Kanervisto, Dave Bignell, Mingfei Sun, Raluca Georgescu, Sergio Valcarcel Macua, Shan Zheng Tan, Ida Momennejad, Katja Hofmann, and Sam Devlin. Imitating human behaviour with diffusion models, 2023

2023
[25]

Pomerleau

Dean A. Pomerleau. Alvinn: An autonomous land vehicle in a neural network. In D. Touretzky (ed.), Advances in Neural Information Processing Systems, volume 1. Morgan-Kaufmann, 1988. URL https://proceedings.neurips.cc/paper_files/paper/1988/file/812b4ba287f5ee0bc9d43bbf5bbe87fb...

1988
[26]

Goal-conditioned imitation learning using score-based diffusion policies, 2023

Moritz Reuss, Maximilian Li, Xiaogang Jia, and Rudolf Lioutikov. Goal-conditioned imitation learning using score-based diffusion policies, 2023

2023
[27]

Artificial Intelligence: A Modern Approach

Stuart Russell and Peter Norvig. Artificial Intelligence: A Modern Approach. 2016

2016
[28]

Behavior transformers: Cloning k modes with one stone, 2022

Nur Muhammad Mahi Shafiullah, Zichen Jeff Cui, Ariuntuya Altanzaya, and Lerrel Pinto. Behavior transformers: Cloning k modes with one stone, 2022

2022
[29]

A mathematical theory of communication

Claude Elwood Shannon. A mathematical theory of communication. The Bell System Technical Journal, 27: 0 379--423, 1948. URL http://plan9.bell-labs.com/cm/ms/what/shannonday/shannon1948.pdf

1948
[30]

Diverse behavior is what game ai needs: Generating varied human-like playing styles using evolutionary multi-objective deep reinforcement learning, 2020

Ruimin Shen, Yan Zheng, Jianye Hao, Yinfeng Chen, and Changjie Fan. Diverse behavior is what game ai needs: Generating varied human-like playing styles using evolutionary multi-objective deep reinforcement learning, 2020

2020
[31]

Skill decision transformer, 2023

Shyam Sudhakaran and Sebastian Risi. Skill decision transformer, 2023

2023
[32]

A note on the evaluation of generative models, 2016

Lucas Theis, Aäron van den Oord, and Matthias Bethge. A note on the evaluation of generative models, 2016

2016
[33]

Braviner, Panteha Naderian, Chris J

Valentin Villecroze, Harry J. Braviner, Panteha Naderian, Chris J. Maddison, and Gabriel Loaiza-Ganem. Bayesian nonparametrics for offline skill discovery, 2022

2022
[34]

Robust imitation of diverse behaviors, 2017

Ziyu Wang, Josh Merel, Scott Reed, Greg Wayne, Nando de Freitas, and Nicolas Heess. Robust imitation of diverse behaviors, 2017

2017
[35]

Diverse policies recovering via pointwise mutual information weighted imitation learning

Hanlin Yang, Jian Yao, Weiming Liu, Qing Wang, Hanmin Qin, Kong hansheng, Kirk Tang, Jiechao Xiong, Chao Yu, Kai Li, Junliang Xing, Hongwu Chen, Juchao Zhuo, QIANG FU, Yang Wei, and Haobo Fu. Diverse policies recovering via pointwise mutual information weighted imitation learn...

2025
[36]

Zhao, Vikash Kumar, Sergey Levine, and Chelsea Finn

Tony Z. Zhao, Vikash Kumar, Sergey Levine, and Chelsea Finn. Learning fine-grained bimanual manipulation with low-cost hardware, 2023

2023
[37]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...
[38]

@esa (Ref

\@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...
[39]

\@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...
[40]

For robotics, learning from human experts allows to reach human-level performance without any controller hard coding or expensive interaction with simulated or real environments

@open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.

[1] [1]

Tenenbaum, Tommi S

Anurag Ajay, Yilun Du, Abhi Gupta, Joshua B. Tenenbaum, Tommi S. Jaakkola, and Pulkit Agrawal. Is conditional generative modeling all you need for decision making? In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=sP1fo2K9DFG

work page 2023

[2] [2]

Testing, validation, and verification of robotic and autonomous systems: A systematic review

Hugo Araujo, Mohammad Reza Mousavi, and Mahsa Varshosaz. Testing, validation, and verification of robotic and autonomous systems: A systematic review. ACM Trans. Softw. Eng. Methodol., 32 0 (2), mar 2023. ISSN 1049-331X. doi:10.1145/3542945. URL https://doi.org/10.1145/3542945

doi:10.1145/3542945 2023

[3] [3]

Champandard

Alex J. Champandard. Ai Game Development. 2003

work page 2003

[4] [4]

Decision transformer: Reinforcement learning via sequence modeling, 2021

Lili Chen, Kevin Lu, Aravind Rajeswaran, Kimin Lee, Aditya Grover, Michael Laskin, Pieter Abbeel, Aravind Srinivas, and Igor Mordatch. Decision transformer: Reinforcement learning via sequence modeling, 2021

work page 2021

[5] [5]

Diffusion policy: Visuomotor policy learning via action diffusion, 2024

Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion, 2024

2024

[6] [6]

Implicit behavioral cloning, 2021

Pete Florence, Corey Lynch, Andy Zeng, Oscar Ramirez, Ayzaan Wahid, Laura Downs, Adrian Wong, Johnny Lee, Igor Mordatch, and Jonathan Tompson. Implicit behavioral cloning, 2021

2021

[7] [7]

Off-policy deep reinforcement learning without exploration, 2019

Scott Fujimoto, David Meger, and Doina Precup. Off-policy deep reinforcement learning without exploration, 2019

2019

[8] [8]

Multi-modal imitation learning from unstructured demonstrations using generative adversarial nets, 2017

Karol Hausman, Yevgen Chebotar, Stefan Schaal, Gaurav Sukhatme, and Joseph Lim. Multi-modal imitation learning from unstructured demonstrations using generative adversarial nets, 2017

work page 2017