Pith. sign in

REVIEW 3 major objections 4 minor 14 references

Behavioral Entropy-Guided Dataset Generation for Offline Reinforcement Learning

T0 review · 3 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that offline RL algorithms trained on datasets collected by a behavioral-entropy-maximizing policy outperform those trained on datasets from Shannon entropy, Rényi entropy, SMM, and RND on all five tasks considered.

desk verdict Solid empirical comparison, but the central claim that the reward actually maximizes behavioral entropy rests on an unjustified approximation, so as written the experiments compare a heuristic. read the letter →

arxiv 2502.04141 v1 pith:2IEMHEDG submitted 2025-02-06 cs.LG

Wesley A. Suttle , Aamodh Suresh , Carlos Nieto-Granda This is my paper · ORCID
classification cs.LG
keywords behavioralentropyofflinereinforcementlearningdatasetgenerationk-nearestneighborestimationexplorationobjectivePrelecprobabilityweightingRényiShannon
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to establish that behavioral entropy (BE)—an entropy that composes Shannon's formula with a probability weighting function borrowed from behavioral economics—is a practical and effective exploration objective for continuous control. It extends BE to continuous distributions, gives k-nearest-neighbor estimators with consistency and finite-sample guarantees, and turns the estimator into a reward that standard RL agents can maximize. Using that reward to collect offline datasets, the paper claims, makes downstream offline RL policies perform better than datasets collected with Shannon entropy, Rényi entropy, state-marginal matching, and random network distillation on every one of five locomotion tasks tested. The payoff, if true, is that a single exploration objective with one tunable parameter can generate diverse offline datasets that support a range of downstream tasks more efficiently than existing alternatives.

What carries the argument

The central object is differential behavioral entropy, $H_{B,\alpha,\beta}(f) = \beta \int e^{-\beta(-\log f(x))^\alpha} (-\log f(x))^\alpha dx$, which is Shannon entropy evaluated under Prelec's probability weighting $w(x)=e^{-\beta(-\log x)^\alpha}$. The estimator is the importance-sampling-corrected k-NN plug-in, and the load-bearing simplification is the reduction from the full estimator to the reward $r(s,a) = \|s - NN_k(s)\|_2 e^{-\beta(\log(\|s-NN_k(s)\|_2 + c))^\alpha} (\log(\|s-NN_k(s)\|_2 + c))^\alpha$ with $d=1$; maximizing this reward is claimed to approximately maximize BE of the state occupancy measure.

What would settle it

Compute the omitted constant $D_{k,n}$ for the experiment's settings ($d=1$, $n=500{,}000$, $k=12$) and compare it with the distribution of $\log \|s - NN_k(s)\|_2$ in a generated dataset; if the constant is comparable to or larger than the typical log-distance, the approximation in Eq. (22) is not negligible, and a direct comparison of Eq. (21) and Eq. (24) as policy-ranking objectives would show whether the experimental results hold for the true BE estimator.

Watch

Extended reading notes

Core claim

Behavioral entropy in continuous spaces is defined by applying Prelec's probability weighting $w(x)=e^{-\beta(-\log x)^\alpha}$ to the density and then computing Shannon entropy of the weighted density. The paper derives a k-NN plug-in estimator for this quantity, corrected by importance sampling, and proves it converges and has controlled bias and variance. From the estimator it extracts a per-step reward that is approximately proportional to BE improvement, so that any RL algorithm trained on that reward approximately maximizes BE of the state occupancy measure. In experiments, policies trained on the BE reward collected 500K-step datasets for Walker and Quadruped, and offline RL on those datasets outperformed baselines in the majority of task-algorithm combinations. The paper's stated conclusion is that BE-generated datasets lead to superior offline RL performance and better data efficiency compared with Shannon, Rényi, SMM, and RND objectives.

Load-bearing premise

The reward used in the experiments drops a constant $D_{k,n}$ that depends on $n$, $k$, and the dimension $d$, and the central claim depends on this drop being negligible, since otherwise the policies are maximizing a heuristic derived from behavioral entropy rather than behavioral entropy itself.

Editorial extensions

If this is right

  • If the central claim holds, offline RL practitioners can replace hand-tuned exploration rewards with a single BE reward and get better downstream task performance on the same computational budget.
  • The k-NN estimator's consistency and bias/variance guarantees mean BE can be used in high-dimensional state representations if the representation dimension is kept moderate.
  • BE's smooth variation in $\alpha$ gives dataset designers a knob that interpolates between broad and focused coverage, something the paper shows Rényi entropy does not offer stably for $q > 1$.
  • The paper shows comparable downstream performance with 500K-element datasets to that of prior work using 10M-element datasets, suggesting BE collection reduces both data and training requirements.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the approximation dropping $D_{k,n}$ deserves scrutiny; if it is not negligible, the paper's experiments should be read as evidence for a BE-inspired heuristic reward rather than for BE itself, and the gap between Eq. (21) and Eq. (24) is a direct way to test this.
  • Editorial inference: because the paper notes the extension to state-action occupancy measures is straightforward, the same machinery could generate datasets targeting action diversity, which may matter for tasks like manipulation where multiple actions reach the same state.
  • Editorial inference: the smooth dependence of coverage on $\alpha$ suggests an automatic schedule of $\alpha$ during data collection could produce a single dataset with both coarse and fine coverage, something the paper does not explore.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper extends behavioral entropy (BE), previously defined for discrete distributions, to continuous state spaces. It proposes k-nearest-neighbor estimators for differential BE, states consistency and bias-variance guarantees (Theorems 1 and 2), and derives a practical reward function intended to induce BE-maximizing policies. These policies are then used with the APT algorithm to generate datasets for offline RL in Walker and Quadruped MuJoCo environments, and the datasets are compared against those generated with Rényi entropy, Shannon entropy, RND, and SMM. The headline empirical result is that offline RL algorithms trained on BE-generated datasets outperform all baselines on five tasks, and outperform Rényi-generated datasets on four of the five tasks.

Significance. If the connection between the implemented reward and the BE objective were established, the paper would offer a principled and flexible exploration objective for dataset generation, with substantial empirical support. The experimental effort is a real strength: the authors train 1,275 offline RL policies, report clear aggregate results in Table 1, and include ablations and coverage visualizations. The theoretical results for the importance-sampling k-NN estimator in Theorems 1 and 2 are also valuable as a first step for BE estimation in continuous spaces. However, the central derivation linking the practical reward in Eq. (24) to behavioral-entropy maximization contains an unverified approximation and a dimension mismatch. Because that link is the stated reason the generated datasets should reflect BE coverage, the significance of the empirical comparison is currently conditional on a point that the paper does not validate.

major comments (3)
  1. [§4, Eqs. (21)–(22)] The step from Eq. (21) to Eq. (22) replaces d log R_{i,k,n} + D_{k,n} with d log R_{i,k,n} in both the exponent and the power, on the assertion that the contribution of D_{k,n} is negligible. This step is load-bearing: it is the only derivation connecting the trained policies to the stated objective H^{B,α,β}(d_π). In the experimental setup n = 500,000 and k = 12, and with the d = 1 used in Eq. (24), D_{k,n} = log(nπ^{1/2}/(kΓ(3/2))) ≈ 11.9. This is not small compared with the per-sample log-distance terms, which are of order zero or negative in typical normalized feature spaces. Dropping D can change the sign of the argument of the α-power, and for non-integer α the expression in Eq. (21) is not even defined when d log R_{i,k,n} + D_{k,n} < 0. Theorems 1 and 2 are asymptotic consistency and bias results for Eq. (13), not for the approximation in Eq. (22), and no numerical check is reported to show that Eq. (22) tracks Eq. (21) on actual state samples. Without such a check, Eq. (24) is a heuristic exploration bonus, and the abstract's causal attribution of the Table 1 results to behavioral-entropy maximization is not established.
  2. [§4, Eqs. (23)–(24), and Table 2] Eq. (23) carries the dimension d explicitly in the exponent and in the log-power, but Eq. (24) sets d = 1, while Table 2 reports that the APT representation dimension is 512. This is not a cosmetic numerical-stability change: moving from a d-dimensional log distance to a one-dimensional one changes the reward surface by a factor of d inside the exponent and inside the log-power, and the additive constant c in Eq. (24) introduces a further modification that has no counterpart in the estimator of Eq. (21). The paper should either derive Eq. (24) from the d-dimensional k-NN estimator without this substitution, or provide empirical evidence that the ranking of policies induced by Eq. (24) matches the ranking induced by Eq. (21) on samples from the relevant feature distribution. As it stands, the implemented reward is not shown to be approximately proportional to the BE estimator it is claimed to encode.
  3. [Appendix A.1, Lemma 1] Lemma 1 as stated bounds |E[H_n^{B,w}(f)] − H^{B,w}(f)| by O((k/n)^{ξ/d}) for the estimator H_n^{B,w} defined in Eq. (26), which is a sample average of g(f(X_i)) and is exactly unbiased for H^{B,w}(f). The stated bound is therefore vacuous, and the sentence attributing it to Singh & Póczos, Theorem 5 is misleading because Eq. (26) contains no k-NN density estimate at all. The proof of Theorem 2 does not actually need this bound—the first term in Eq. (33) is controlled through Lemma 2 and the second term is zero—but the supporting lemma should be corrected, redefined in terms of the plug-in estimator, or removed so that the theoretical contribution is stated accurately.
minor comments (4)
  1. [Appendix A.1, Eqs. (25)–(27)] The displayed equations are garbled: Eq. (25) appears to combine two separate equalities, and Eq. (26) uses "1/f(x)" where the argument should be X_i. Please rewrite these definitions so that the integrand and the sample average are unambiguous.
  2. [§5, Dataset Generation and Visualization] The sentence "Altogether we considered 17 datasets: eight BE, five BE, two SE, and one each for RND and SMM" should read "eight BE, five RE, two SE, and one each for RND and SMM"; as written it mislabels the Rényi datasets.
  3. [§5, Experimental Setup] The comparison with the 10M-element ExORL datasets is reported as evidence of data- and sample-efficiency, but the evaluation protocols, environment versions, and offline training budgets may differ from those used to produce the ExORL numbers. A short paragraph listing the exact source of each reference number and any differences in evaluation would make this comparison reproducible.
  4. [Appendix A.6, Figure 13] The reward-function visualization would be more informative if it showed the behavior of Eq. (24) as a function of the raw distance for the actual values of α and β used in the experiments, rather than only for the α range shown; this would also help readers gauge the effect of the additive constant c.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the BE reward is derived from a BE estimator and tested against external baselines; the Eq. 21 to Eq. 22 approximation is a validity concern, not a circular reduction.

full rationale

The derivation chain is not circular. Behavioral entropy is defined independently in Eq. 8 from Prelec's weighting function, and the k-NN estimator in Eq. 13 is a plug-in estimator of that definition whose consistency is anchored to external results (Devroye and Wagner 1977; Singh and Poczos 2016; Zhao and Lai 2022). The practical reward in Eq. 24 is derived from the estimator, not fitted to the experimental outcomes, and the experimental comparisons are against external baselines (Shannon, Renyi, SMM, RND) in standard MuJoCo environments. The claim that BE-generated datasets outperform these baselines is an empirical downstream result, not a quantity defined in terms of the input. The self-citation to Suresh et al. (2024) for the beta conditioning is load-bearing for the theoretical admissibility claim, but that prior work is a published, independently checkable result rather than an unverified premise imported solely to force the present conclusion. The main weakness is the approximation in Eq. 22 and the implementation choice d=1 in Eq. 24: these threaten the fidelity of the reward to the stated BE objective, as does the paper's own admission that the volumetric-coverage relationships are not conclusive. However, a broken or heuristic approximation is a correctness risk, not a circular reduction, because the claimed 'prediction' is not equivalent to any fitted parameter or to the estimator input by construction. No step in the paper defines the output in terms of the claimed input or renames a known result as a derivation. The score is therefore 0.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim rests on several hyperparameters (alpha, k, c, M) and on the unproven negligibility of D_{k,n}. No new physical entities are introduced. The main conceptual input, the BE functional and its admissibility, comes from the authors' prior published work.

free parameters (5)
  • alpha (α) = grid {0.2, 0.5, 0.7, 0.9, 1.5, 2.0, 3.0, 5.0}; best per task selected for headline
    Controls the shape of the Prelec weighting function in the BE reward. The paper's headline comparison uses the best alpha per task, so the performance claim is conditional on this parameter being tuned.
  • M (representation dimension for beta conditioning) = 512
    Used in beta = e^{(1-alpha) log log M} to guarantee admissibility of BE; M is the APT representation dimension, chosen as a hyperparameter.
  • k (k-NN neighbors) = 12
    k-NN estimator hyperparameter in reward computation; affects bias-variance tradeoff.
  • c (additive constant in Eq 24) = not reported
    Added inside logarithms for numerical stability; the value is not reported, yet it changes the reward numerically.
  • q (Rényi order) for baseline = grid {0.2, ..., 1.1} for offline RL; q greater than 1 excluded
    Baseline parameter; not central to the BE claim but part of the comparison that was post hoc trimmed.
assumptions (4)
  • standard math k-NN density estimator consistency (Devroye-Wagner; Zhao-Lai sup-norm bounds)
    Used in Theorems 1 and 2 to establish consistency and bias/variance bounds of the BE estimator.
  • domain assumption The state occupancy measure d_pi has a density that is bounded away from zero on a compact support
    Required for Theorem 2 (c1 <= f <= c2). In continuous RL state spaces, occupancy measures can have thin tails or concentrated support, so this may not hold.
  • ad hoc to paper D_{k,n} is negligible in the reward derivation (Eq 21 to Eq 22)
    The paper asserts this under suitable conditions without proof. With n=500K, k=12, d=1, D = log(n pi / k) is about 11.9, which is not negligible relative to log nearest-neighbor distances. This is the key unsupported step connecting the reward to BE.
  • domain assumption Beta conditioning from Suresh et al. 2024 makes BE an admissible entropy
    The paper adopts this self-cited prior result as a premise for the reward design.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Behavioral Entropy-Guided Dataset Generation for Offline Reinforcement Learning." pith.science (2026). https://pith.science/paper/2IEMHEDG

@misc{pith2026250204141,
  author       = {Pith},
  title        = {Pith review of: Behavioral Entropy-Guided Dataset Generation for Offline Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2IEMHEDG}},
  note         = {Machine review of arXiv:2502.04141}
}
abstract

Entropy-based objectives are widely used to perform state space exploration in reinforcement learning (RL) and dataset generation for offline RL. Behavioral entropy (BE), a rigorous generalization of classical entropies that incorporates cognitive and perceptual biases of agents, was recently proposed for discrete settings and shown to be a promising metric for robotic exploration problems. In this work, we propose using BE as a principled exploration objective for systematically generating datasets that provide diverse state space coverage in complex, continuous, potentially high-dimensional domains. To achieve this, we extend the notion of BE to continuous settings, derive tractable $k$-nearest neighbor estimators, provide theoretical guarantees for these estimators, and develop practical reward functions that can be used with standard RL methods to learn BE-maximizing policies. Using standard MuJoCo environments, we experimentally compare the performance of offline RL algorithms for a variety of downstream tasks on datasets generated using BE, R\'{e}nyi, and Shannon entropy-maximizing policies, as well as the SMM and RND algorithms. We find that offline RL algorithms trained on datasets collected using BE outperform those trained on datasets collected using Shannon entropy, SMM, and RND on all tasks considered, and on 80% of the tasks compared to datasets collected using R\'{e}nyi entropy.

Figures

Figures reproduced from arXiv: 2502.04141 by the authors.

Figure 1
Figure 1. (Left) Comparison of Shannon entropy, Renyi entropy, and behavioral entropy (ours) and their ´ effects on dataset generation, shown in PHATE plots, when used as an exploration objective. (Right) Perfor￾mance comparison of an offline RL algorithm (CQL) for three downstream tasks on datasets generated using Shannon, behavioral entropy (ours), and Renyi entropy for the parameter ´ q = 1.1 shown in the left-hand figure.… view at source ↗
Figure 2
Figure 2. Visualizations of probability weightings (left) and superior expressiveness of BE (right). Our main contributions are: • Behavioral entropy estimation in continuous spaces. We propose a version of BE applicable to continuous probability distributions, derive k-nearest neighbor (k-NN) estimators for BE with general probability weighting functions, and provide convergence guarantees and probabilistic bounds characteri… view at source ↗
Figure 3
Figure 3. PHATE plots for Walker tasks. Dataset Generation and Visualization. For dataset generation, we used the Active Pre-Training (APT) algorithm (Liu & Abbeel, 2021) implemented in the URLB framework to maximize BE using the reward proposed in equation 24 for various val￾ues of α, RE using the reward proposed in Zhang et al. (2021) for various values of q, and SE using the default reward from Liu & Abbeel (2021). Specifi… view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: Comparison of offline RL performance over the entropy objectives used in dataset generation. Plots show mean and standard deviation over five seeds. Dotted line shows performance of RL policy trained online until approximate optimality. Offline RL Experiments. We compa…
Figure 5
Figure 5. Figure 5: Offline RL results for all α and q values evaluated. Initial trials showed q ∈ {2.0, 3.0, 5.0} led to performance no better (and usually worse) than q = 1.1, so offline RL training for these q values was not performed. 16 [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]
Figure 6
Figure 6. Figure 6: Ablation result comparing the effect of performing 100K vs. 200K offline RL training steps on a 3M-element dataset generated using Shannon entropy as exploration objective. These results suggest that performing additional offline RL training has only a marginal effect …
Figure 7
Figure 7. Figure 7: Offline RL results averaged over all α, q values. A.4 QUANTITATIVE COVERAGE EXPERIMENTS 18 [PITH_FULL_IMAGE:figures/full_fig_p018_7.png]
Figure 8
Figure 8. Figure 8: Visualization of evolution of smallest hypersphere radius r (normalized by the maximum radius achieved over all datasets) over the course of data generation training step T for the Walker and Quadruped domains. We refer to this coverage metric as volumetric coverage. W…
Figure 9
Figure 9. Figure 9: TSNE plots for Walker 20 [PITH_FULL_IMAGE:figures/full_fig_p020_9.png]
Figure 10
Figure 10. Figure 10: PHATE plots for Quadruped 21 [PITH_FULL_IMAGE:figures/full_fig_p021_10.png]
Figure 11
Figure 11. Figure 11: TSNE plots for Quadruped 22 [PITH_FULL_IMAGE:figures/full_fig_p022_11.png]
Figure 12
Figure 12. Figure 12: Qualitative visualization of SMM and RND for data generation A.6 BE REWARD FUNCTION VISUALIZATION [PITH_FULL_IMAGE:figures/full_fig_p023_12.png]
Figure 13
Figure 13. Figure 13: Visualization of the BE reward function equation 24 by varying the parameter α with β conditioned according to (4) from (Suresh et al., 2024) with M = 512, denoting the representation dimensions. These visualizations highlight the diversity and variety of rewards that…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

14 extracted references · 10 canonical work pages

  1. [2]

    (31) 13 Published as a conference paper at ICLR 2025 We are now in a position to prove our main result. Theorem

    work page 2025

  2. [7]

    Don’t change the algorithm, change the data: Exploratory data for offline reinforcement learning

    Denis Yarats, David Brandfonbrener, Hao Liu, Michael Laskin, Pieter Abbeel, Alessandro Lazaric, and Lerrel Pinto. Don’t change the algorithm, change the data: Exploratory data for offline reinforcement learning. In ICLR 2022 Workshop on Generalizable Policy Learning in Physical World,

    work page 2022

  3. [9]

    Let µ denote the Lebesgue measure and Br(x) = {x′ ∈ Rd | ∥x′ − x∥2 < r}. Define H B,w (f ) = − Z X log(w(f (x)))w(f (x))dx Z X g(f (x))f (x)dx, (25) H B,w n (f ) = − nX i=1 1 f (x) log(w(f (Xi)))w(f (Xi)) = 1 n nX i=1 g(f (Xi)), (26) bH B,w k,n (f ) = − nX i=1 1 ˆf (x) log(w( ˆf (Xi)))w( ˆf (Xi)) = 1 n nX i=1 g( ˆf (Xi)). (27) Our goal is to establish a b...

    work page 2003

  4. [11]

    For a given set S ⊂ X, radius r, and m > 0, let N (S, r) denote the covering number, the minimum number of balls of radius r needed to cover S

    We first recall another useful result from the literature. For a given set S ⊂ X, radius r, and m > 0, let N (S, r) denote the covering number, the minimum number of balls of radius r needed to cover S. Let ∥·∥op denote the operator norm. Lemma 2 ((Zhao & Lai, 2022)). Suppose there exist C1, C2, C3, N0 > 0 and β ∈ (0, 1] such that the following conditions...

    work page 2022

  5. [13]

    First notice that E h bH B,w k,n (f ) i − H B,w (f ) ≤ E h bH B,w k,n (f ) − H B,w n (f ) i + E H B,w n (f ) − H B,w (f )

    (32) Proof. First notice that E h bH B,w k,n (f ) i − H B,w (f ) ≤ E h bH B,w k,n (f ) − H B,w n (f ) i + E H B,w n (f ) − H B,w (f ) . (33) The second term can be bounded using Lemma 1, so it just remains to bound the first term. Recall that X is compact, f is bounded strictly away from 0 on X , and w is Lipschitz. We therefore have that g is the product...

    work page 2025

  6. [28]

    Lemma 1 ((Singh & P ´oczos, 2016))

    We first recall the following result. Lemma 1 ((Singh & P ´oczos, 2016)). Suppose that, for some ξ ∈ (0, 2], f is ξ-H¨older continuous and strictly positive on X . Suppose furthermore that there exists a function f∗ : X →R+ and a constant f ∗ such that 0 < f∗(x) ≤ R Br(x) f (y)dy/µ(Br(x)) ≤ f ∗ < ∞, for all x ∈ X, r∈ (0, √ d], and assume that R ∞ 0 e−xxkf...

    work page 2016

  7. [512]

    Initial trials showed q ∈ {2.0, 3.0, 5.0} led to performance no better (and usually worse) than q = 1 .1, so offline RL training for these q values was not performed

    → 1024 → dim(A) ReLU MLP representation dimension 512 k in NN approximator 12 average top k in NN True RND hyperparameter representation dimension 512 predictor, target network architecture dim(S) → 1024 → 1024 → 512 ReLU MLP normalized observation clipping 5 SMM hyperparameter skill dimension 4 skill discriminator learning rate 10−3 V AE learning rate 10...

    work page 2025

  8. [2008]

    Offline reinforcement learning: Tuto- rial, review, and perspectives on open problems

    Sergey Levine, Aviral Kumar, George Tucker, and Justin Fu. Offline reinforcement learning: Tuto- rial, review, and perspectives on open problems. arXiv preprint arXiv:2005.01643,

    arXiv 2005

Show all 14 references
  1. [2016]

    Diversity is all you need: Learning skills without a reward function

    Benjamin Eysenbach, Abhishek Gupta, Julian Ibarz, and Sergey Levine. Diversity is all you need: Learning skills without a reward function. arXiv preprint arXiv:1802.06070,

    arXiv

  2. [2018]

    nearest neighbor

    Kevin Beyer, Jonathan Goldstein, Raghu Ramakrishnan, and Uri Shaft. When is “nearest neighbor” meaningful? In Database Theory—ICDT’99: 7th International Conference Jerusalem, Israel, January 10–12, 1999 Proceedings 7, pp. 217–235. Springer,

    1999

  3. [2019]

    Curiosity-driven exploration by self-supervised prediction

    11 Published as a conference paper at ICLR 2025 Deepak Pathak, Pulkit Agrawal, Alexei A Efros, and Trevor Darrell. Curiosity-driven exploration by self-supervised prediction. In International conference on machine learning, pp. 2778–2787. PMLR,

    2025

  4. [2020]

    Urlb: Unsupervised reinforcement learning benchmark

    Michael Laskin, Denis Yarats, Hao Liu, Kimin Lee, Albert Zhan, Kevin Lu, Catherine Cang, Lerrel Pinto, and Pieter Abbeel. Urlb: Unsupervised reinforcement learning benchmark. arXiv preprint arXiv:2110.15191,

    arXiv

  5. [2021]

    Efficient exploration via state marginal matching

    Lisa Lee, Benjamin Eysenbach, Emilio Parisotto, Eric Xing, Sergey Levine, and Ruslan Salakhutdi- nov. Efficient exploration via state marginal matching. arXiv preprint arXiv:1906.05274,

    1906 arXiv

  6. [2022]

    Fix a p.d.f

    12 Published as a conference paper at ICLR 2025 A A PPENDIX A.1 P ROOFS Fix a probability weighting function w and let g(y) = − 1 y log(w(y))w(y). Fix a p.d.f. f ∈ ∆(X ), where X ⊂Rd is compact. Fix n, k∈ N, and let X1, . . . , Xn ∼ f (·). Recall the definition of ˆf from equation

    2025

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.