{"id":"5f5aadb8-46d1-4258-8c33-46300b451e6c","arxiv_id":"2504.18160","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"An offline imitation method that assigns each demonstration a learned style code and blends similar codes can reproduce varied behaviors and allow a user to select the behavior.","lead":"This paper teaches an offline agent to copy many different human behaviors by giving each demonstration a learned 'style tag' and conditioning the agent's actions on that tag. A second variant blends similar styles to stay reliable in noisy environments; a generalist would read it to see whether this simple trick really captures demonstration diversity or mostly memorizes the training set.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 1's in-sample comparison rewards ZBC's per-trajectory codebook memorization (Eq. 1), so the claimed 'best performance in diversity capture' is not established; WZBC is also not best on several D3IL rows.","rationale":"The reader identified the L1 histogram metric as the weakest assumption, but the deeper issue is that the evaluation protocol itself rewards memorization: ZBC stores a per-trajectory embedding, so under the deterministic evaluation conditions, it can reproduce the training histogram rather than learning a transferable distribution. This is confirmed by the sharp degradation in Table 2 under stochasticity, where ZBC's L1 jumps to near-baseline levels. As a result, Table 1 does not establish that ZBC outperforms prior methods in capturing the actual trajectory distribution; it only shows that index-based retrieval can match a training histogram when the environment is deterministic. The reader's rationale does mention 'in-sample evaluation that largely measures memorization,' but the formal weakest_assumption does not, so my agreement is partial. I recommend UNVERDICTED rather than CONDITIONAL because the central claim is not currently testable from the reported evaluation; a properly designed held-out or generalization test could still support the method, but the present evidence does not. The internal inconsistency in the conclusion regarding WZBC's performance on D3IL datasets further undermines the paper as written, but the primary concern remains the validity of the diversity-capture comparison.","tokens_in":11285,"tokens_out":6589,"duration_ms":71649,"concrete_test":"Hold out 20% of trajectories: train ZBC/WZBC and baselines on the remaining 80%, then sample style embeddings only from training indices and generate rollouts; compute Table 1's L1 histogram distance to (a) the training histogram and (b) the held-out histogram. If ZBC/WZBC's L1 on held-out trajectories rises substantially (e.g., to BC-level) while baselines degrade less, the low in-sample L1 is a memorization artifact. Report the same split for all six datasets with 5 seeds.","verdict_should_be":"UNVERDICTED","load_bearing_attack":"ZBC (Eq. 1) learns a distinct embedding z_i for each training trajectory, and sampling z uniformly at test time (Eq. 2) lets the policy replay near-identical rollouts of the training set in a deterministic environment. The diversity metric in §4.3 compares the histogram of these rollouts to the training histogram; a memorizing agent therefore scores near zero by construction. The comparison to BESO, DDPM-ACT, and DDPM-GPT is not apples-to-apples: those baselines must generalize across states, while ZBC has a per-trajectory key. The strongest evidence that Table 1 measures memorization is Table 2: under random initialization (r-init) or noisy transitions, ZBC's L1 jumps from 0.256 to 1.556 and 0.729, showing it cannot transfer the indexed behaviors to unseen states. Additionally, the conclusion (§5) says 'ZBC and WZBC' displayed best performance, but in Table 1 WZBC is worst on d3ilunbalancedavoiding (1.457 vs BESO 1.283) and worse than BESO/DDPM-ACT on d3ilunbalancedaligning; only ZBC consistently tops the table. So the central claim is both confounded by memorization and internally overstated for WZBC.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies offline imitation learning from demonstrations containing multiple distinct behaviors. It proposes ZBC, which clones each training trajectory conditioned on a per-trajectory embedding, and WZBC, which weights behavioral cloning by a trajectory-dissimilarity term to trade off between memorization and robustness. The evaluation uses L1 distances between behavior histograms on Maze2D and D3IL tasks, together with controllability and stochastic-robustness experiments. The authors claim state-of-the-art diversity capture and robustness for their methods relative to prior multi-modal imitation learning approaches.","tokens_in":11609,"tokens_out":7867,"duration_ms":78016,"significance":"The problem is relevant, and the idea of conditioning a policy on a style embedding to capture trajectory-level diversity is simple and practical. If the diversity-reconstruction claim were properly established, the paper would be a useful contribution to offline imitation learning. The clear problem formulation, the proposed histograms-of-behaviors evaluation, and the visualizations of conditional state distributions are strengths. However, the reported experiments are insufficient: the primary diversity metric can be minimized by memorizing training trajectories, several comparative claims are not supported by the tables, and the controllability and robustness statements are evaluated in-sample or without baselines. These issues are addressable with additional experiments and a more careful framing.","major_comments":[{"comment":"The low L1 distances in Table 1 do not establish that ZBC captures the demonstration distribution in a generalizable sense. ZBC learns a separate embedding z_i for each training trajectory and samples uniformly over these embeddings at test time; in a deterministic environment this is effectively replaying training trajectories. Because the histogram in Table 1 is computed on the same training behaviors, a policy that memorizes each trajectory will score well by construction. The baselines, which do not receive a per-trajectory key, are therefore not compared on equal footing. The stochastic-evaluation rows in Table 2 corroborate the memorization interpretation: ZBC's L1 degrades from 0.256 to 1.556 under random initialization, showing that the low scores do not reflect transferable diversity capture. A held-out evaluation, such as training on a subset of trajectories and testing on the rest, or a comparison that removes the per-trajectory index, is needed before the central claim can be accepted.","section":"Sec. 3.2, Eq. (1)-(2); Sec. 4.3, Table 1"},{"comment":"The conclusion that 'ZBC and WZBC displayed the best performance in diversity capture compared to previous multi-modal imitation learning methods' is not supported for WZBC. On d3ilunbalancedavoiding, WZBC has L1 = 1.457 +/- 0.087, which is worse than BESO's 1.283 +/- 0.067; on d3ilunbalancedaligning, WZBC has L1 = 0.364 +/- 0.037, worse than BESO's 0.256 +/- 0.066 and DDPM-ACT's 0.212 +/- 0.053. Only ZBC consistently ranks first in the diversity rows. The conclusion should be revised to claim best performance for ZBC alone, or the authors should provide additional evidence for WZBC's competitiveness.","section":"Sec. 5; Table 1"},{"comment":"The statement that 'WZBC displayed some robustness to environment stochasticity compared to the baselines' is not supported by the reported experiments, because Table 2 compares only ZBC and WZBC under stochastic configurations. No baseline is evaluated under pseudo-r-init, r-init, or noise-transi, so no conclusion relative to BESO, DDPM-ACT, or DDPM-GPT can be drawn. The robustness claim should be restricted to a comparison between ZBC and WZBC, or the missing baseline experiments should be added.","section":"Sec. 4.5, Table 2; Sec. 5"},{"comment":"The controllability evaluation in Section 4.4 also relies on the training style codes: the conditional style distribution e(z|Psi(tau)) is computed by filtering training trajectories by the property and then sampling uniformly among their embeddings. In a deterministic environment, this tests whether the policy can replay the subset of training trajectories satisfying the property; it does not test whether the learned latent space supports control over unseen or interpolated behaviors. The authors should either frame the controllability result as a selection mechanism over the training set or add experiments that condition on property values not present in the training trajectories.","section":"Sec. 4.4, Eq. (9)"}],"minor_comments":[{"comment":"The dissimilarity defined in Eq. (5) is not symmetric despite the text calling it symmetric, because the denominator depends on the first argument. If the stored nu_{i,j} values are used with ordered pairs, this should be stated; otherwise the definition should be changed, for instance by normalizing with a dataset-wide maximum.","section":"Sec. 3.3, Eq. (5)"},{"comment":"No statistical significance tests are reported, and several standard-deviation intervals overlap, such as the pseudo-r-init L1 distances in Table 2 for ZBC (1.152 +/- 0.094) and WZBC (0.828 +/- 0.349). Reporting confidence intervals or significance tests would help the reader assess the claimed rankings.","section":"Sec. 4.3, Table 1"},{"comment":"The abstract and introduction emphasize 'Temporal Consistency', but the policy is Markovian in state and style; temporal consistency is represented only through the style code. The paper should clarify this point to avoid overclaiming the degree to which the method itself encodes temporal structure.","section":"Sec. 3.1 and Sec. 3.2"}],"recommendation":"major_revision","confidential_remarks":"This manuscript reads as a workshop contribution, but the core diversity-capture claim is not yet substantiated for a journal audience. The memorization confound in Table 1 is the central issue, and it is addressable with held-out trajectory evaluation and unbiased comparisons. I would not reject outright because the proposed methods are simple and potentially useful, and the flaws are localizable to the evaluation protocol and the framing of the conclusions."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The shortest take: the paper has a genuinely useful nugget, but its headline claim is not supported by the evidence as presented. The useful nugget is WZBC, the similarity-weighted regression objective. Per-trajectory conditioning itself (ZBC) is a simple extension of existing trajectory-conditioning ideas, but the weighting scheme that interpolates between BC and ZBC is a sensible and clean twist. The histogram-based L1 evaluation metric is also a step forward over entropy-based diversity metrics, and the paper is clearly written. The problem is Table 1. ZBC is evaluated by sampling the exact style codes it was trained on and rolling out in a deterministic environment. A policy that stores one embedding per training trajectory can essentially replay each trajectory, which obviously matches the training histogram. The diffusion and energy-based baselines don't have that per-trajectory lookup, so the comparison is not apples-to-apples. The authors themselves note in Section 3.3 that 'ZBC tends to overfit,' and the robustness table (Table 2) confirms it: under random initialization or noisy transitions, ZBC's L1 jumps from 0.256 to 1.556 or 0.729, meaning it cannot transfer its indexed behaviors to unseen states. WZBC is more robust, but Table 2 only compares ZBC and WZBC — no baselines — so we still don't know how it stacks up in realistic conditions. There's also a smaller internal inconsistency: the conclusion says 'ZBC and WZBC displayed the best performance in diversity capture,' but in Table 1 WZBC is worst on d3ilunbalancedavoiding and worse than BESO on several rows. Only ZBC consistently tops the table. That's a minor overstatement, but combined with the memorization issue it matters. The controllability section (conditioning on trajectory length) works, but almost trivially: since each code corresponds to one training trajectory, filtering codes by a property simply selects the relevant training trajectories. It demonstrates control, but not a learned, compact behavior space. And there is no released code, which makes the quantitative results hard to verify. Who is this for? Researchers in diverse imitation learning or game AI who want a quick, simple baseline idea. As evidence for generalizable offline diversity capture, it's not there yet. I would send this to peer review rather than desk-reject, but with strong requests: add a held-out style test (train on a subset of trajectories, evaluate on the rest), include the baselines in the robustness study, and release code. The SWR idea deserves further development; the current evaluation doesn't yet show it delivers.","headline":"Reasonable core idea (WZBC's similarity-weighted regression) but the main diversity claim rests on an in-sample memorization comparison; the evaluation needs a redesign before the claims hold.","tokens_in":719,"tokens_out":1959,"would_cite":false,"duration_ms":40745,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"By conditioning imitation on per-trajectory style codes, offline agents can reproduce the full mix of human demonstration behaviors and steer generation by trajectory properties.","keywords":["imitation learning","offline learning","diverse behaviors","trajectory-level diversity","style conditioning","controllable generation","similarity weighted regression","behavior histograms"],"falsifier":"Train ZBC and WZBC on a maze dataset with two behavior modes that visit the same checkpoint sequence but differ in an unmeasured property such as speed profile, then evaluate both the checkpoint-sequence histogram and a finer measure like state-visitation frequency over a dense grid; if the checkpoint L1 distance is near zero while the state-visitation distance is large, the evaluation metric fails to capture trajectory distribution and the paper's claim of reproducing pM,μ(τ) is unsupported.","tokens_in":11095,"feed_emoji":"🤖","tokens_out":8150,"duration_ms":68800,"temperature":0.7,"pith_summary":"The paper proposes that capturing the diversity of human demonstrations offline requires conditioning a behavior-cloning policy on a learned latent 'style' vector associated with each demonstration trajectory, rather than reproducing the action distribution at each state independently. It introduces ZBC, which gives every trajectory its own trainable embedding, and WZBC, a relaxed version that blends styles according to trajectory similarity so the policy stays robust under environment stochasticity. Across maze and robot-manipulation datasets, the authors report that these methods generate trajectories whose behavior histogram (checkpoint sequences or task metrics) is closer to the demonstration histogram than prior multimodal imitation methods, while also allowing users to condition generation on properties such as trajectory length. If true, the significance is a simple, fully offline route to controllable generation of a specified mixture of human-like behaviors, useful for game bots, robot pre-training, and simulation.","feed_headline":"Per-trajectory style codes capture human behavior diversity offline","feed_subtitle":"Style-conditioned offline cloning matches the real mix of human behaviors, no environment interaction.","key_machinery":"The central object is the style code: a low-dimensional trainable embedding associated with each demonstration trajectory, which the policy conditions on. The key mechanism is similarity-weighted regression (SWR): with ZBC, the policy maximizes the likelihood of each trajectory's actions under its own style embedding; with WZBC, actions from one trajectory are cloned under another trajectory's style, weighted by an exponential kernel exp(−β·ν(·,·)) based on a Euclidean trajectory dissimilarity over padded states. This weighting, controlled by β and a stop-gradient on relabeled styles, lets the model interpolate between a single global policy and fully per-trajectory conditioning, yielding controllability and robustness in one framework.","core_discovery":"The central discovery is that trajectory-scale diversity in imitation learning can be captured by a simple conditioning trick: assign each demonstration a trainable latent embedding z (the 'style'), learn a policy π(a|s,z) that clones that demonstration's actions, and at generation time sample styles according to the demonstration distribution. ZBC instantiates this with a look-up table of embeddings; WZBC relaxes it by regressing actions from trajectory i against the style of trajectory j, weighted by exp(−β·ν(τ_i,τ_j)) for a dissimilarity ν, interpolating between behavioral cloning and hard style conditioning. The authors claim this reproduces the actual trajectory distribution (measured by L1 distance of discretized behavior histograms) better than previous transition-level and trajectory-level multimodal methods, while providing controllability by conditioning the style-sampling distribution on trajectory properties.","pith_inferences":["A natural extension the authors do not pursue is combining style conditioning with trajectory-generation models such as transformers or diffusion, so controllability applies to long-horizon generation without per-timestep policy unrolling.","The controllability mechanism, restricting the style-sampling distribution, is closely analogous to classifier-free guidance in diffusion models; one could test whether the same selection rule yields calibrated conditional distributions under metric filters other than length, such as reward or safety constraints.","Because the behavior histogram is hand-defined per task, the method's apparent diversity capture is only as good as the chosen metrics; a stronger test would measure the full state-visitation or action-sequence distribution, which the paper does not do.","The dissimilarity weighting suggests a principled way to handle dataset imbalance: rare behaviors could be up-weighted by tuning β or by re-sampling styles, which might improve the unbalanced-dataset scores reported."],"forward_implications":["If the histogram-L1 results hold, ZBC offers a drop-in offline baseline: a dataset of indexed demonstrations plus a small MLP policy can reproduce the behavior mixture without reward labels or environment interaction.","Style conditioning gives a free control interface: by sampling styles only from trajectories satisfying a property (e.g., length in [70,80]), the generated trajectories concentrate on that property without retraining.","WZBC's robustness gains in stochastic environments suggest that style relaxation, rather than hard per-trajectory codes, is the right operating point for real-world deployments with state noise or variable initial states.","The L1 histogram evaluation, if adopted, shifts assessment of diverse imitation from entropy maximization to faithful reproduction of a given behavior distribution, which is closer to practical needs in games and robotics.","Because ZBC requires no encoder network, it scales cheaply to large offline datasets with many trajectories, at the cost of a lookup table."],"supporting_citations":[{"why":"Supplies the D3IL benchmark datasets (Avoiding, Aligning) and the baseline implementations used for comparison.","marker":"Jia et al., 2024"},{"why":"BESO, a diffusion-based goal-conditioned imitation baseline whose diversity scores ZBC/WZBC are compared against.","marker":"Reuss et al., 2023"},{"why":"DDPM-ACT, a diffusion action-chunking baseline in the comparison table.","marker":"Chi et al., 2024"},{"why":"DDPM-GPT, a transformer-diffusion imitation baseline in the comparison table.","marker":"Pearce et al., 2023"},{"why":"Establishes that entropy is maximized by the uniform distribution, justifying the paper's shift from entropy-based to L1-histogram diversity evaluation.","marker":"Shannon, 1948"},{"why":"Prior stylized offline RL work whose entropy-based diversity measure the paper argues fails to measure faithful reproduction.","marker":"Mao et al., 2024"},{"why":"Shows trajectory-scale style conditioning via variational encoders, the approach the paper deliberately bypasses for computational cost.","marker":"Wang et al., 2017"},{"why":"Defines behavioral cloning, the base method ZBC extends by conditioning on style embeddings.","marker":"Pomerleau, 1988"},{"why":"GAIL, a canonical inverse-RL method that the paper notes fails on multimodal demonstration data.","marker":"Ho & Ermon, 2016"}],"fun_headline_variants":["Style codes per trajectory clone diverse behaviors offline","Controllable behavior diversity from offline style conditioning","Latent style embeddings reproduce true behavior distribution","Sampling learned styles yields controllable diverse imitation","Offline imitation with per-demo style codes for control"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the L1 distance between hand-defined behavior histograms, computed with zero-padding over checkpoint sequences or task metrics, faithfully measures whether the generated trajectory distribution matches the demonstration distribution; if those metrics ignore properties that matter (e.g., speed profiles, subtle path variations), the reported diversity scores do not establish the method's central claim.","fun_headline_variants_meta":{"raw":{"variants":["Style codes per trajectory clone diverse behaviors offline","Controllable behavior diversity from offline style conditioning","Latent style embeddings reproduce true behavior distribution","Sampling learned styles yields controllable diverse imitation","Offline imitation with per-demo style codes for control"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000201,"raw_usage":{"total_tokens":1348,"prompt_tokens":882,"completion_tokens":466,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":498,"completion_tokens_details":{"reasoning_tokens":396}},"tokens_in":498,"tokens_out":466,"duration_ms":4847,"temperature":1.0,"reasoning_tokens":396,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T10:22:51.661855+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train ZBC and WZBC on a maze dataset with two behavior modes that visit the same checkpoint sequence but differ in an unmeasured property such as speed profile, then evaluate both the checkpoint-sequence histogram and a finer measure like state-visitation frequency over a dense grid; if the checkpoint L1 distance is near zero while the state-visitation distance is large, the evaluation metric fails to capture trajectory distribution and the paper's claim of reproducing pM,μ(τ) is unsupported.","supporting_citations":[{"cited_title":"A mathematical theory of communication","cited_arxiv_id":null,"evidence_quote":"Establishes that entropy is maximized by the uniform distribution, justifying the paper's shift from entropy-based to L1-histogram diversity evaluation."},{"cited_title":"Stylized offline reinforcement learning: Extracting diverse high-quality behaviors from heterogeneous datasets","cited_arxiv_id":null,"evidence_quote":"Prior stylized offline RL work whose entropy-based diversity measure the paper argues fails to measure faithful reproduction."},{"cited_title":"Robust imitation of diverse behaviors, 2017","cited_arxiv_id":null,"evidence_quote":"Shows trajectory-scale style conditioning via variational encoders, the approach the paper deliberately bypasses for computational cost."},{"cited_title":"Generative adversarial imitation learning, 2016","cited_arxiv_id":null,"evidence_quote":"GAIL, a canonical inverse-RL method that the paper notes fails on multimodal demonstration data."}],"review_version":1}