Pith. sign in

REVIEW 3 major objections 6 minor 67 references

Random Feature Information Gain approximates information-gain exploration in deep RL with a closed-form random-Fourier-feature bonus, and is shown to match neural-network baselines across twelve tasks.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 16:29 UTC pith:AWNKUGLS

load-bearing objection Clean, plausible exploration bonus with a real theory–practice mismatch: the error bound only holds under λ = nλ0, but every experiment fixes λ = 1. the 3 major comments →

arxiv 2607.17981 v1 pith:AWNKUGLS submitted 2026-07-20 cs.LG

Information-Based Exploration via Random Features for Reinforcement Learning

Waris Radji , Odalric-Ambrym Maillard This is my paper
classification cs.LG
keywords random featuresexploration bonusinformation gaindeep reinforcement learningkernel methodsoptimism in the face of uncertaintyGaussian processesrandom Fourier features
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that information-gain exploration bonuses in deep reinforcement learning can be computed in closed form from random Fourier features, without any learned representation. It derives the bonus from Gaussian-process entropy reduction, gives approximation error bounds, and shows empirically that the bonus is competitive with neural-network exploration methods across control, navigation, and MinAtar tasks. If this is right, exploration in continuous or otherwise uncountable state spaces no longer requires black-box uncertainty estimates; a fixed random feature map and a D-by-D matrix inverse provide a principled alternative. The theoretical results also provide guidance for choosing the number of random features.

Core claim

The paper's central claim is that the information gain of querying a state in a Gaussian-process model, IG(x*|D_n) = ½ log(1 + σ²_n(x*)/σ²), can be approximated by the random-feature bonus r⁺(s) = ½ log(1 + φ(s)ᵀ(ΦᵀΦ + λI)⁻¹φ(s)), where φ is a random Fourier feature map and Φ is the feature matrix of previously visited states. The paper proves (Proposition 7) that, under uniform kernel approximation, bounded kernel, and regularization scaling λ = nλ₀, the approximation error is bounded by [ϵ(λ₀+κ)² + ϵ²λ₀] / (2nλ₀³). It then derives a feature-dimension requirement D = O(d/ε_k² · log(σ_p·diam(X)/(ε_k·δ))) to guarantee a target error with high probability. Finally, it shows that this bonus, in

What carries the argument

The central object is the RFIG bonus r⁺(s) = ½ log(1 + φ(s)ᵀ(ΦᵀΦ + λI)⁻¹φ(s)), where φ is a random Fourier feature map approximating a shift-invariant kernel (typically RBF), Φ is the feature matrix of visited states, and λ is a regularization parameter. This formula converts the information gain of a Gaussian-process posterior into a closed-form expression that requires only a D×D matrix inverse instead of an n×n kernel inverse, reducing complexity from O(n³) to O(D³) via the Woodbury identity. The random feature map does the work of approximating the kernel, and the inverse matrix encodes the accumulated uncertainty of all past observations.

Load-bearing premise

The error bound only holds if the regularization parameter grows proportionally to the number of visited states, but the experiments keep λ fixed, so the proven guarantee does not apply to the configuration that is actually tested.

What would settle it

Plug the experimental setting into Proposition 7: with λ = 1 and n = 10⁶, the bound is ϵ(λ₀+κ)²/(2nλ₀³) + ϵ²λ₀/(2nλ₀³) where λ₀ = 1/n; this simplifies to roughly ϵκ²n²/2 + ϵ²n/2, which diverges as n grows. A bound that increases with sample size contradicts the stated consistency, so computing that value and observing it explode is a direct falsification of the theoretical guarantee in the tested regime.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Exploration in uncountable state spaces can be driven by a closed-form information-gain bonus that requires no learned representation and no neural-network uncertainty estimates.
  • The approximation error of the bonus is controlled by the kernel approximation error and, under the stated regularization scaling, decreases with the number of samples, providing a consistency guarantee.
  • The feature-dimension requirement D = O(d/ε² log(1/δ)) gives a principled way to set the number of random features based on the desired information-gain accuracy, state dimension, and kernel bandwidth.
  • When integrated into a standard policy-gradient agent, the bonus matches or exceeds neural-network baselines on control, navigation, and MinAtar tasks, suggesting that representation learning is not necessary for effective optimism-based exploration.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The theoretical guarantee is stated under λ = nλ₀, but the experiments fix λ = 1 while n grows to millions; plugging the experimental configuration into Proposition 7 makes the bound grow like O(ϵ n²) rather than vanish, so the paper's own guarantee does not currently cover the setup it evaluates.
  • A direct test of the theory would be to run the same experiments with λ proportional to n; if performance degrades, the empirical competitiveness may depend on the unproven constant-λ regime rather than the analyzed one.
  • The closed-form information-gain formula could be reused as a general uncertainty score in active learning or out-of-distribution detection, since it only needs a feature map and an accumulated matrix inverse.
  • The subsampling ratio ρ appears to be load-bearing in practice: without it, the bonus may decay too quickly as the covariance shrinks, meaning the practical bonus is a regularized variant of information gain rather than the exact one.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces RFIG, an exploration bonus for deep RL based on approximating GP information gain with random Fourier features. The bonus is r+(s) = ½ log(1 + φ(s)ᵀ(ΦᵀΦ + λI)⁻¹φ(s)). The authors derive error bounds (Prop. 6–7) connecting uniform kernel-approximation error to information-gain error, give a feature-dimension requirement (Cor. 9), and integrate RFIG with PPO using subsampling, Newton-Schulz inversion, and fixed hyperparameters. Experiments on control, navigation, and MinAtar tasks report that RFIG is competitive with RND, VIME, and #Explo. The abstract and conclusion claim that RFIG provides 'superior theoretical interpretation' on top of its empirical performance.

Significance. If the theoretical claims were valid in the evaluated regime, this would be a valuable contribution: a closed-form, kernel-based exploration bonus with finite-sample guarantees and no learned representation, in the spirit of making exploration interpretable. The paper has several strengths: it ships code, uses 32 seeds on a diverse set of tasks, and explicitly connects the bonus to Bayesian kernel methods. However, the advertised guarantees currently do not cover the experimental configuration: Assumption 4 (λ = nλ₀) is not satisfied by the fixed λ=1 used in Algorithm 1 and §5.2, so Prop. 7's bound diverges along the evaluated trajectory. The proof of Prop. 2 also contains a Woodbury-identity error and an unstated feature-normalization assumption. These are fixable, but until fixed, the 'superior theoretical interpretation' claim is overstated.

major comments (3)
  1. [§4.2 and §5.2 (Assumption 4 vs Algorithm 1)] The error bounds in Propositions 6–7 and Corollary 9 all invoke Assumption 4 (λ = nλ₀), but Algorithm 1 fixes λ at initialization (Σ₀ ← λI) and never changes it, and §5.2 sets λ = 1 while cumulative timesteps reach 10⁶–2.5×10⁶. If λ₀ is read as λ/n, the RHS of Eq. (9) becomes ε(1/n+κ)²/(2n(1/n)³) + ε²(1/n)/(2n(1/n)³) ≈ ε κ² n²/2, which diverges. If λ₀ is read as a fixed constant, Assumption 4 is simply not satisfied. The paper never flags this mismatch, and §4.3/§5.1 present Corollary 9 as practical guidance. Since the abstract and conclusion rest on these bounds, this is a load-bearing gap: the 'superior theoretical interpretation' is not supported for the evaluated configuration.
  2. [Appendix A.1, Proposition 2 proof] The Woodbury identity is misstated: the second term should be (1/σ²)Φ(ΦᵀΦ + σ²I)⁻¹Φᵀ, not (1/σ⁴)Φ(ΦᵀΦ + σ²I)⁻¹Φᵀ. The displayed formula and the substituted expression in Eq. (12) are inconsistent. More importantly, the step from Eq. (12) to Eq. (13) silently requires φ(x)ᵀφ(x) = 1 (or an equivalent condition), which is not stated and is only approximately true for random Fourier features. The final formula is the standard Bayesian linear regression information gain when features are normalized, but the proof as written is invalid. Please correct the identity and explicitly state the normalization assumption on φ.
  3. [Algorithm 1 vs §4.2 (subsampling and approximate inversion)] The theoretical analysis assumes exact posterior variance computed from all n samples. Algorithm 1, however, subsamples a fraction ρ=6.25% of states and uses a fixed K=20 Newton-Schulz iteration for matrix inversion (Appendix B.1). These algorithmic details are not reflected in Propositions 6–7; at minimum, n in the bounds must be the number of subsampled points, and the Newton-Schulz approximation error is unaccounted for. The paper presents these as 'Details that Matter' without explaining their effect on the guarantees, further weakening the claim that the implemented system has the theoretical properties proved in §4.
minor comments (6)
  1. [Appendix A.1] Typo: 'Poof of Proposition 2' should be 'Proof of Proposition 2'.
  2. [Appendix A.3] Lemma 10 is referred to as 'Lemma X' in the proof of Proposition 7; fix the cross-reference.
  3. [Appendix A.4] The proof of the feature-dimension result is labeled 'Proof of Proposition 9', but it is Corollary 9. Also, the step 'Setting κ=1 ends the proof' is too abrupt; spell out how the quadratic root leads to Eq. (11).
  4. [§5.2 vs Appendix B.1] The Discussion states 'full matrix inversion used in our final experiments', while Appendix B.1 says K=20 Newton-Schulz iterations were used. Please reconcile or clarify which was actually used.
  5. [§5.2, Figure 1] Consider adding a table with aggregate scores (e.g., interquartile mean and standard error) to support the 'competitive' claim quantitatively; the current figure is visually informative but no aggregate statistics are tabulated.
  6. [Throughout] Minor language issues: 'recommandations' should be 'recommendations'; 'theorically' should be 'theoretically'; and 'Assumption 4 ensures that regularization term remain properly balanced' has a subject-verb agreement issue.

Circularity Check

0 steps flagged

No significant circularity: RFIG's approximation target (GP information gain) is an external quantity and the error bounds are standard perturbation results.

full rationale

RFIG's derivation is not circular. Definition 1 defines exact GP information gain from the posterior variance σ_n^2(x). Proposition 2 replaces the kernel with a random-feature approximation and obtains the bonus formula via the Woodbury identity; the quantity being approximated is the well-defined external GP information gain, not the algorithm's own output. Propositions 6 and 7 then bound the difference between the approximate and exact variances using standard matrix perturbation arguments under explicit assumptions (Assumptions 3-5), and Proposition 8 supplies the RFF uniform-convergence bound from Rahimi and Recht (2007), an external result. Corollary 9 is a corollary of those bounds, not an input. No fitted parameter is renamed as a prediction: D=1024, λ=1, ρ=6.25%, and ℓ=√d̄ are fixed in Algorithm 1 and §5.2, and the comparison against RND, VIME, and #Explo is an external benchmark evaluation. The self-citations present (Pesquerel and Maillard 2022; Radji 2025) are background and environment references and carry no load in the derivation. The mismatch between Assumption 4 (λ=nλ0) and the experimental λ=1 is a practice-theory validity gap, not circularity: it makes the stated error bound inapplicable or divergent along the evaluated trajectory, but it does not make any predicted quantity equivalent to an input by construction.

Axiom & Free-Parameter Ledger

6 free parameters · 6 axioms · 0 invented entities

The central derivation draws only on standard GP/kernel facts plus three stated assumptions; the genuinely ad hoc ingredient is Assumption 4 (λ = nλ₀), introduced to make the bounds vanish and contradicted by the algorithm's fixed λ. The unexamined domain assumption — that state-space GP information gain of an unspecified function f drives reward-maximizing exploration — is what connects the approximation theory to RL performance, and it is never proven. No new entities (particles, forces, dimensions) are postulated.

free parameters (6)
  • length-scale ℓ = ℓ = √d̄ (effective dimension estimated from samples)
    Hand-chosen heuristic, §5.1; proportionality constant unspecified; no ablation. Determines the RFF spectral density and hence the entire bonus geometry.
  • number of random features D = 1024
    Fixed across all environments (§5.2); Corollary 9 gives a tentative guideline but no target ε, δ are stated, so D is not derived from the bound.
  • subsample ratio ρ = 0.0625
    Chosen to stop the bonus vanishing as n grows (§5.1, §5.2); directly shapes the discounting of novelty; no ablation.
  • regularization λ = 1
    Fixed at 1 in Algorithm 1 and all experiments; the theory (Assumption 4) demands λ = nλ₀, so the guarantees do not apply to this configuration.
  • exploration coefficient β = 0.5
    Shared across all methods and applied to normalized bonuses (§5.2), so it does not change relative comparisons.
  • Newton-Schulz iterations K = 20
    Fixed iteration count (Appendix B.1); the paper notes final experiments actually use full matrix inversion, so this parameter belongs to a described-but-not-final mechanism.
axioms (6)
  • domain assumption Uniform kernel approximation by the random feature map (Assumption 3): sup |φ(x)ᵀφ(x′)−k(x,x′)| ≤ ε with high probability, over a compact domain X of bounded diameter.
    Invoked for Propositions 6–7 and Corollary 9; the paper itself notes diam(X) is generally unknown in RL (§4.3), so the D requirement cannot be instantiated in practice.
  • ad hoc to paper Regularization scales linearly with sample size: λ = nλ₀ (Assumption 4).
    Introduced solely to make the variance/IG error bounds well-behaved; contradicted by Algorithm 1 and §5.2 (λ=1 fixed). With λ fixed, the Proposition 6 bound grows as ϵκ²n²/λ².
  • standard math Bounded kernel: |k(x,x′)| ≤ κ (Assumption 5).
    Satisfied by RBF and Matérn kernels, as stated in §4.2.
  • domain assumption State-space GP information gain of an unspecified function f is a sound exploration signal for reward maximization in MDPs.
    §3 introduces f abstractly ('an unknown function f: X → ℝ') but never ties it to reward, dynamics, or value; the entire RL justification rests on this link, which is asserted rather than proven.
  • ad hoc to paper The feature-space diagonal is treated as exactly the kernel diagonal: φ(x)ᵀφ(x) = k(x,x) = 1.
    Used implicitly in Appendix A.1 Eq (12)–(13) to obtain σ̂²_n = σ²φᵀ(ΦᵀΦ+σ²I)⁻¹φ; for RFF this holds only in expectation, and the equality additionally requires σ² = 1.
  • domain assumption State normalization and the Valko et al. (2013) effective-dimension estimator yield a well-conditioned kernel with ℓ = √d̄.
    §5.1. The recommended length-scale and its proportionality constant are heuristics; the cited estimator is not described, so ℓ is under-specified.

pith-pipeline@v1.3.0-alltime-deepseek · 15769 in / 31600 out tokens · 253259 ms · 2026-08-01T16:29:15.219149+00:00 · methodology

0 comments
read the original abstract

Representation learning has enabled classical exploration strategies to be extended to deep Reinforcement Learning (RL), but often makes algorithms more complex and theoretical guarantees harder to establish. We introduce Random Feature Information Gain (RFIG), grounded in Bayesian kernel methods theory, which uses random Fourier features to approximate information gain and compute exploration bonuses in non-countable spaces. We provide error bounds on information gain approximation and avoid the black-box aspects of neural network-based uncertainty estimation, for optimism-based exploration. We present practical details that make RFIG scalable to deep RL scenarios, enabling smooth integration into standard deep RL algorithms. Experimental evaluation across diverse control and navigation tasks demonstrates that RFIG achieves competitive performance with well-established deep exploration methods while offering superior theoretical interpretation.

Figures

Figures reproduced from arXiv: 2607.17981 by Odalric-Ambrym Maillard, Waris Radji.

Figure 1
Figure 1. Figure 1: Comparing exploration strategies. Solid lines represent the interquartile mean, with shaded areas indicating the 25th-75th percentiles across 32 random seeds. Discussion. The experimental results across tasks of varying dimensionality reveal dis￾tinct performance patterns that highlight fundamental tradeoffs in exploration strategy design ( [PITH_FULL_IMAGE:figures/full_fig_p010_1.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

67 extracted references · 17 linked inside Pith

  1. [1]

    135 , author=

    Introduction to reinforcement learning, vol. 135 , author=. 1998 , publisher=

    1998

  2. [2]

    2009 , publisher=

    Active learning literature survey , author=. 2009 , publisher=

    2009

  3. [3]

    Journal of Computer and System Sciences , volume=

    An analysis of model-based interval estimation for Markov decision processes , author=. Journal of Computer and System Sciences , volume=. 2008 , publisher=

    2008

  4. [4]

    Machine learning , volume=

    Finite-time analysis of the multiarmed bandit problem , author=. Machine learning , volume=. 2002 , publisher=

    2002

  5. [5]

    Proceedings of the 26th annual international conference on machine learning , pages=

    Near-Bayesian exploration in polynomial time , author=. Proceedings of the 26th annual international conference on machine learning , pages=

  6. [6]

    Advances in neural information processing systems , pages=

    Unifying count-based exploration and hashing , author=. Advances in neural information processing systems , pages=

  7. [7]

    International conference on machine learning , pages=

    Curiosity-driven exploration by self-supervised prediction , author=. International conference on machine learning , pages=. 2017 , organization=

    2017

  8. [8]

    arXiv preprint arXiv:2002.06038 , year=

    Never give up: Learning directed exploration strategies , author=. arXiv preprint arXiv:2002.06038 , year=

    Pith/arXiv arXiv 2002

  9. [9]

    arXiv preprint arXiv:1810.12894 , year=

    Exploration by random network distillation , author=. arXiv preprint arXiv:1810.12894 , year=

    Pith/arXiv arXiv

  10. [10]

    arXiv preprint arXiv:0912.3995 , year=

    Gaussian process optimization in the bandit setting: No regret and experimental design , author=. arXiv preprint arXiv:0912.3995 , year=

    Pith/arXiv arXiv

  11. [11]

    International Conference on Artificial Intelligence and Statistics , pages=

    Efficient kernelized ucb for contextual bandits , author=. International Conference on Artificial Intelligence and Statistics , pages=. 2022 , organization=

    2022

  12. [12]

    arXiv preprint arXiv:1309.6869 , year=

    Finite-time analysis of kernelised contextual bandits , author=. arXiv preprint arXiv:1309.6869 , year=

    Pith/arXiv arXiv

  13. [13]

    Advances in neural information processing systems , volume=

    Random features for large-scale kernel machines , author=. Advances in neural information processing systems , volume=

  14. [14]

    Biometrika , volume=

    On the likelihood that one unknown probability exceeds another in view of the evidence of two samples , author=. Biometrika , volume=. 1933 , publisher=

    1933

  15. [15]

    Advances in neural information processing systems , volume=

    An empirical evaluation of thompson sampling , author=. Advances in neural information processing systems , volume=

  16. [16]

    Advances in neural information processing systems , volume=

    Learning to optimize via information-directed sampling , author=. Advances in neural information processing systems , volume=

  17. [17]

    , author=

    An Asymptotically Optimal Bandit Algorithm for Bounded Support Models. , author=. COLT , pages=. 2010 , organization=

    2010

  18. [18]

    Advances in neural information processing systems , volume=

    Vime: Variational information maximizing exploration , author=. Advances in neural information processing systems , volume=

  19. [19]

    Advances in neural information processing systems , volume=

    Deep exploration via bootstrapped DQN , author=. Advances in neural information processing systems , volume=

  20. [20]

    2018 Information Theory and Applications Workshop (ITA) , pages=

    Efficient exploration through bayesian deep q-networks , author=. 2018 Information Theory and Applications Workshop (ITA) , pages=. 2018 , organization=

    2018

  21. [21]

    International conference on machine learning , pages=

    Self-supervised exploration via disagreement , author=. International conference on machine learning , pages=. 2019 , organization=

    2019

  22. [22]

    Advances in neural information processing systems , volume=

    \# exploration: A study of count-based exploration for deep reinforcement learning , author=. Advances in neural information processing systems , volume=

  23. [23]

    arXiv preprint arXiv:2408.03029 , year=

    Highly efficient self-adaptive reward shaping for reinforcement learning , author=. arXiv preprint arXiv:2408.03029 , year=

    Pith/arXiv arXiv

  24. [24]

    arXiv preprint arXiv:1911.08701 , year=

    Bayesian curiosity for efficient exploration in reinforcement learning , author=. arXiv preprint arXiv:1911.08701 , year=

    Pith/arXiv arXiv 1911

  25. [25]

    Bayesian

    Morere, Philippe and Ramos, Fabio , booktitle=. Bayesian

  26. [26]

    International conference on computational learning theory , pages=

    A generalized representer theorem , author=. International conference on computational learning theory , pages=. 2001 , organization=

    2001

  27. [27]

    Transactions of the American mathematical society , volume=

    Theory of reproducing kernels , author=. Transactions of the American mathematical society , volume=

  28. [28]

    1999 , publisher=

    Elements of information theory , author=. 1999 , publisher=

    1999

  29. [29]

    Ridge regression learning algorithm in dual variables , author=

  30. [30]

    Seventh International Workshop on Artificial Intelligence and Statistics , year=

    Probabilistic kernel regression models , author=. Seventh International Workshop on Artificial Intelligence and Statistics , year=

  31. [31]

    Advances in neural information processing systems , volume=

    Gaussian processes for regression , author=. Advances in neural information processing systems , volume=

  32. [32]

    1950 , publisher=

    Inverting modified matrices , author=. 1950 , publisher=

    1950

  33. [33]

    Advances in neural information processing systems , volume=

    Fast sparse Gaussian process methods: The informative vector machine , author=. Advances in neural information processing systems , volume=

  34. [34]

    arXiv preprint arXiv:1506.02785 , year=

    On the error of random Fourier features , author=. arXiv preprint arXiv:1506.02785 , year=

    Pith/arXiv arXiv

  35. [35]

    arXiv preprint arXiv:2402.02229 , year=

    Vanilla bayesian optimization performs great in high dimensions , author=. arXiv preprint arXiv:2402.02229 , year=

    Pith/arXiv arXiv

  36. [36]

    ZAMM-Journal of Applied Mathematics and Mechanics/Zeitschrift f

    Iterative berechung der reziproken matrix , author=. ZAMM-Journal of Applied Mathematics and Mechanics/Zeitschrift f. 1933 , publisher=

    1933

  37. [37]

    arXiv preprint arXiv:2405.19548 , year=

    Rlexplore: Accelerating research in intrinsically-motivated reinforcement learning , author=. arXiv preprint arXiv:2405.19548 , year=

    Pith/arXiv arXiv

  38. [38]

    arXiv preprint arXiv:1707.06347 , year=

    Proximal policy optimization algorithms , author=. arXiv preprint arXiv:1707.06347 , year=

    Pith/arXiv arXiv

  39. [39]

    Daniel Freeman and Erik Frey and Anton Raichuk and Sertan Girgin and Igor Mordatch and Olivier Bachem , title =

    C. Daniel Freeman and Erik Frey and Anton Raichuk and Sertan Girgin and Igor Mordatch and Olivier Bachem , title =

  40. [40]

    Robert Tjarko Lange , title =

  41. [41]

    arXiv preprint arXiv:2407.19396 , year=

    NAVIX: Scaling MiniGrid Environments with JAX , author=. arXiv preprint arXiv:2407.19396 , year=

    Pith/arXiv arXiv

  42. [42]

    arXiv preprint arXiv:2410.20092 , year=

    Ogbench: Benchmarking offline goal-conditioned rl , author=. arXiv preprint arXiv:2410.20092 , year=

    Pith/arXiv arXiv

  43. [43]

    1992 , publisher=

    Efficient exploration in reinforcement learning , author=. 1992 , publisher=

    1992

  44. [44]

    Density-based bonuses on learned representations for reward-free exploration in deep reinforcement learning , author=

  45. [45]

    arXiv preprint arXiv:2007.13442 , year=

    Fast active learning for pure exploration in reinforcement learning , author=. arXiv preprint arXiv:2007.13442 , year=

    Pith/arXiv arXiv 2007

  46. [46]

    1959 , publisher=

    Lectures on Fourier integrals , author=. 1959 , publisher=

    1959

  47. [47]

    arXiv preprint arXiv:2412.12098 , year=

    MaxInfoRL: Boosting exploration in reinforcement learning through information gain maximization , author=. arXiv preprint arXiv:2412.12098 , year=

    Pith/arXiv arXiv

  48. [48]

    Proceedings of the thiry-fourth annual ACM symposium on Theory of computing , pages=

    Similarity estimation techniques from rounding algorithms , author=. Proceedings of the thiry-fourth annual ACM symposium on Theory of computing , pages=

  49. [49]

    Communications of the ACM , volume=

    Near-optimal hashing algorithms for approximate nearest neighbor in high dimensions , author=. Communications of the ACM , volume=. 2008 , publisher=

    2008

  50. [50]

    arXiv preprint arXiv:2401.09750 , year=

    Exploration and anti-exploration with distributional random network distillation , author=. arXiv preprint arXiv:2401.09750 , year=

    Pith/arXiv arXiv

  51. [51]

    Advances in neural information processing systems , volume=

    Near-optimal regret bounds for reinforcement learning , author=. Advances in neural information processing systems , volume=

  52. [52]

    Proceedings of the thirteenth international conference on artificial intelligence and statistics , pages=

    Understanding the difficulty of training deep feedforward neural networks , author=. Proceedings of the thirteenth international conference on artificial intelligence and statistics , pages=. 2010 , organization=

    2010

  53. [53]

    2013 IEEE international conference on acoustics, speech and signal processing , pages=

    Advances in optimizing recurrent networks , author=. 2013 IEEE international conference on acoustics, speech and signal processing , pages=. 2013 , organization=

    2013

  54. [54]

    Proceedings of the AAAI conference on artificial intelligence , volume=

    Deep reinforcement learning that matters , author=. Proceedings of the AAAI conference on artificial intelligence , volume=

  55. [55]

    Advances in Neural Information Processing Systems , volume=

    (More) efficient reinforcement learning via posterior sampling , author=. Advances in Neural Information Processing Systems , volume=

  56. [56]

    Advances in Neural Information Processing Systems , volume=

    IMED-RL: Regret optimal learning of ergodic Markov decision processes , author=. Advances in Neural Information Processing Systems , volume=

  57. [57]

    International conference on machine learning , pages=

    Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor , author=. International conference on machine learning , pages=. 2018 , organization=

    2018

  58. [58]

    arXiv preprint arXiv:1312.5602 , year=

    Playing atari with deep reinforcement learning , author=. arXiv preprint arXiv:1312.5602 , year=

    Pith/arXiv arXiv

  59. [59]

    The Thirteenth International Conference on Learning Representations , year=

    Standard gaussian process is all you need for high-dimensional bayesian optimization , author=. The Thirteenth International Conference on Learning Representations , year=

  60. [60]

    Rodrigo de Lazcano and Kallinteris Andreas and Jun Jet Tai and Seungjae Ryan Lee and Jordan Terry , title =

  61. [61]

    2025 , url=

    Pointax: JAX-Native PointMaze Environment , author=. 2025 , url=

    2025

  62. [62]

    MinAtar: An Atari-Inspired Testbed for Thorough and Reproducible Reinforcement Learning Experiments , journal =. 2019

    2019

  63. [63]

    arXiv preprint arXiv:1812.07544 , year=

    Information-directed exploration for deep reinforcement learning , author=. arXiv preprint arXiv:1812.07544 , year=

    Pith/arXiv arXiv

  64. [64]

    The 22nd International Conference on Artificial Intelligence and Statistics , pages=

    Online learning in kernelized markov decision processes , author=. The 22nd International Conference on Artificial Intelligence and Statistics , pages=. 2019 , organization=

    2019

  65. [65]

    Kernel-Based Reinforcement Learning in Robust

    Lim, Shiau Hong and Autef, Arnaud , booktitle =. Kernel-Based Reinforcement Learning in Robust. 2019 , editor =

    2019

  66. [66]

    International Conference on Artificial Intelligence and Statistics , pages=

    A kernel-based approach to non-stationary reinforcement learning in metric spaces , author=. International Conference on Artificial Intelligence and Statistics , pages=. 2021 , organization=

    2021

  67. [67]

    Cognitive Neuroscience , volume=

    Active inference and epistemic value , author=. Cognitive Neuroscience , volume=. 2015 , publisher=

    2015