Pith. sign in

REVIEW 4 major objections 4 minor 33 references

Informed Asymmetric Actor-Critic: Leveraging Privileged Signals Beyond Full-State Access

T0 review · 4 major / 4 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Conditioning a critic on any state-dependent privileged signal—not just the full state—leaves the expected policy gradient unchanged, so asymmetric training can exploit partial internal information without biasing updates.

desk verdict Clean theory, weak empirical validation: a modest but legitimate generalization of asymmetric RL that deserves review but needs a stronger experimental case. read the letter →

arxiv 2509.26000 v3 pith:ZTGKR6AF submitted 2025-09-30 cs.LG stat.ML

Daniel Ebi , Damien Ernst , Klemens Böhm , Gaspard Lambrechts This is my paper
classification cs.LGstat.ML
keywords reinforcementlearningpartiallyobservableMarkovdecisionprocessesasymmetricactor-criticprivilegedinformationpolicygradientunbiasednessconditionalindependencekernelmethods
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that an actor-critic agent does not need full-state access during training to obtain unbiased policy gradients. Its informed asymmetric actor-critic framework conditions the critic on an arbitrary state-dependent privileged signal, drawn from an information channel added to a POMDP, and proves that the resulting informed asymmetric policy gradient equals the standard history-based policy gradient. This matters because many training setups expose some internal variables without exposing the entire state, and prior asymmetric methods required full state. The paper also proposes two ways to judge whether a candidate signal is worth adding: a kernel-based conditional-independence test usable before training and a post-training bound on return-prediction error. In benchmark navigation tasks and synthetic informed POMDPs, a well-chosen partial signal matches or beats full-state baselines while using less state information.

What carries the argument

The load-bearing object is the informed history Q-function Qπ(h,i,a) and its unbiasedness identity Ei|h[Qπ(h,i,a)] = Qπ(h,a), proved by the law of total expectation. Substituting this identity into the standard policy-gradient expression yields the informed asymmetric policy gradient ∇IAACθ J(πθ) = E[Σt γt Qπ(ht,it,at) ∇ log πθ(at|ht)] and shows it equals ∇θ J(πθ). The companion informed value function Vπ(h,i) satisfies a Bellman recursion that supports temporal-difference learning. To choose signals, the paper introduces a Hilbert-Schmidt Conditional Independence Criterion (HSCIC)—a kernel measure of whether the signal carries information about returns beyond what the history already provid

What would settle it

On a small informed POMDP where the exact informed Q-function is computable by dynamic programming, train the proposed neural critic, then compare the sampled informed-asymmetric policy gradient with the exact policy gradient computed from the true Q-function. If the expected difference grows with the critic's temporal-difference error and exceeds sampling noise, the learned-critic approximation has introduced bias, contradicting the practical claim that partial signals preserve unbiased gradients.

Watch

Extended reading notes

Core claim

The central claim is that any state-dependent privileged signal suffices for unbiased critic conditioning. Formally, the paper defines the informed history value function Vπ(h,i) and informed history Q-function Qπ(h,i,a) in an informed POMDP, proves that Ei|h[Qπ(h,i,a)] = Qπ(h,a) for all histories and actions, and derives that ∇IAACθ J(πθ) = ∇θ J(πθ). Since the actor still conditions only on observable histories, the critic can be given any variable that depends on the underlying state—an internal sensor reading, a simulator variable, a partial location map—without changing the expected gradient. This recovers the history-state critic as the special case i = s and extends asymmetric RL to se

Load-bearing premise

The unbiased-gradient theorem assumes the critic evaluates the exact informed expected-return function; the implementation replaces that exact function with a neural network trained by bootstrapped temporal-difference errors, and the paper supplies no bound or argument showing that this learned replacement preserves unbiasedness.

Editorial extensions

If this is right

  • Asymmetric RL can operate with partial privileged information; any state-dependent training signal is admissible without biasing the actor's gradient.
  • The history-state critic of prior asymmetric methods becomes a special case (signal equals full state), so the theory extends rather than replaces existing results.
  • Practitioners gain two concrete screening tools: a pre-training conditional-independence test and a post-training confidence bound on value-prediction improvement.
  • Training environments that expose only some internal variables—sensors, simulator internals, partial maps—can now be used in asymmetric training without requiring full state.
  • Empirically, more state information is not automatically better: informative partial signals can match or outperform full-state critics on several benchmark tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: because every state-dependent signal leaves the expected gradient unchanged, the remaining question is gradient variance, not bias; the paper's informativeness tests do not directly measure variance, so choosing signals to minimize update variance is a natural next step.
  • Editorial inference: the identity suggests a generative design rule—sample the signal distribution I(·|s) to maximize value-prediction improvement, treating the signal channel itself as part of the architecture.
  • Editorial inference: the empirical finding that a noisy partial signal can beat full state suggests signal relevance, not state coverage, drives value estimation; a direct test would be to rank candidate signals by HSCIC before training and check whether that ranking predicts final performance.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes informed asymmetric actor-critic (IAAC), a method for partially observable RL in which the critic is conditioned on an arbitrary state-dependent privileged signal i_t (not necessarily the full state) while the actor uses only the observation-action history h_t. The central theoretical contribution is Theorem 4.1, which states that, under an exact informed Q-function Q^π(h,i,a), the informed asymmetric policy gradient equals the standard policy gradient. The paper also introduces two informativeness criteria for selecting privileged signals: a pre-training kernel-based conditional-independence test (HSCIC) and a post-training criterion based on reduction in return prediction error ((ε,δ)-informativeness). Experiments on six benchmark navigation tasks and synthetic informed POMDPs compare IAAC with symmetric and asymmetric baselines and evaluate the two criteria.

Significance. If Theorem 4.1 is taken at face value, it cleanly generalizes the Baisero–Amato history-state asymmetric actor-critic result to arbitrary state-dependent privileged signals, and the proof is a simple, parameter-free application of the law of total expectation. This is a useful conceptual extension: it shows that full-state access is not necessary for the unbiasedness of the gradient estimator. The paper also makes a concrete attempt to operationalize signal selection, albeit with empirical validation that is currently too weak to support the claims. The theoretical contribution is likely correct and of value to the asymmetric-RL community, but the practical method as implemented replaces the exact Q^π with a learned TD critic, and no bias bound is supplied; the empirical evaluation of the informativeness criteria does not yet demonstrate their usefulness. These gaps are significant enough to require revision before the paper can be accepted.

major comments (4)
  1. [Eq. (7)] The unbiasedness theorem is proved for the exact informed Q-function Q^π(h,i,a). The implemented estimator, however, replaces this quantity by a learned TD critic: δ_t = r_t + γV̂(h_{t+1},i_{t+1};ϑ) − V̂(h_t,i_t;ϑ). For arbitrary V̂, E[δ_t | h_t,i_t,a_t] = Q^π(h_t,i_t,a_t) − (V̂(h_t,i_t) − γE[V̂(h_{t+1},i_{t+1}) | h_t,i_t,a_t]), and this Bellman-error term is generally a function of i_t. Thus the statement “the critic can incorporate additional training-time signals without biasing policy updates” is not established for the practical algorithm. Please either provide a bound or a consistency condition for the approximation, or explicitly restrict the unbiasedness claim to the exact-Q setting and discuss the bias introduced by function approximation.
  2. [Table 1, §5.1] The Bellman recursion for Q^π(h,i,a) is written as Q^π(h,i,a) = R(h,i,a) + γ E_{o',i'|i,a}[V^π(h',i')], with h' = hao'. The expectation over o',i' generally depends on h as well as i,a because the distribution of the next state, and hence of the next observation and information, depends on the belief induced by h and i. The equation should condition on h,i,a (i.e., E_{o',i'|h,i,a}). As written, the recursion is not well-defined and the statement that V^π(h,i) is the time-invariant informed value function is questionable. Fixing this does not change Theorem 4.1, but it is needed for a correct theoretical treatment of the value function and TD target.
  3. [Figure 2, §5.2] The empirical validation of the HSCIC-based informativeness criterion is weak. For informative signals, the reported p-values average between 0.108 and 0.239 with standard deviations of 0.144–0.251, and no rejection rates at any α are reported. With only B=30 permutations, the test is severely underpowered, and the claim that “noiseless partial input is more informative than full state” is not supported by these p-values, which all exceed conventional significance levels. Moreover, the permutation test permutes y_i over samples with varying z_i; this does not in general generate a valid null distribution for the conditional independence hypothesis. The authors should report rejection rates, use a larger B (or an analytic approximation), and either justify the permutation procedure or replace it with a conditional-permutation approach.
  4. [Figure 2, §5.2] The post-training (ε,δ)-informativeness criterion is not convincingly validated. Across the 20 synthetic instances, the distribution of ε centers near zero and exceeds zero only for δ=0.1. This suggests that the criterion rarely provides positive evidence of informativeness in the very setting where it is supposed to be useful. The paper should report the fraction of instances in which H0 is rejected for each δ, compare against a baseline with i_t=∅ (which should ideally never be deemed informative), and discuss why the median ε is not robustly positive. Without such evidence, the practical utility of this test as a signal-selection tool is not demonstrated.
minor comments (4)
  1. [Abstract] Also, the paper says “significantly outperforms” in several places (e.g., Figure 1 caption), but no statistical tests are reported. Add significance tests or confidence intervals for the learning-curve comparisons.
  2. [§3.2 and §6.1] In Definition 5.1, the formula for ε is ambiguous: it is unclear whether the term − 2C log(2/δ)/(3T) is inside or outside the square root. Please fix the typesetting.
  3. [§5.1] The HSCIC test is computed under a random or exploratory policy, but the conditional dependence of G_t on i_t may be policy-dependent. This is a limitation that should be acknowledged explicitly.
  4. [§4.2] The paper would benefit from a short discussion of the variance of the informed policy gradient estimator. The theorem only shows unbiasedness; it does not imply that an arbitrary i reduces variance, and some of the experimental results (e.g., Figure 2) suggest high variance across instances.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular derivation: Theorem 4.1 follows from the law of total expectation; the only caveat is a theory-to-implementation gap (TD/function approximation), which is a correctness risk, not circularity.

full rationale

The load-bearing derivation is Theorem 4.1, which equates the informed asymmetric policy gradient with the standard policy gradient. Reading the proof, the equality follows by substituting Lemma 4.2 into Eq. (1); Lemma 4.2 is itself an application of the law of total expectation to the definition Q^π(h,i,a)=E_{s0:∞,a0:∞|h,i,a}[Σ γ^j R(s_j,a_j)]. No fitted parameter, learned critic, or external empirical value is used in the theorem; the only inputs are the POMDP model and the definition of i∼I(·|s). The recovery of Baisero and Amato's history-state gradient when i=s is explicitly a corollary, not an assumption. The self-citations to the informed POMDP [19] and to prior asymmetric theory [10,23] supply vocabulary and baselines, but the theorem does not invoke them as evidence. The main caveat is an approximation gap, not circularity: Section 4.2 replaces exact Q^π by a learned TD critic V̂(h,i;ϑ) and δ_t, for which no bias bound is provided; the unbiasedness theorem therefore does not directly apply to the implemented estimator. This is a correctness/robustness concern that would be flagged under a different pass, but it does not make the derivation equivalent to its inputs. Similarly, Eq. (7)'s omission of h in E_{o',i'|i,a} is a technical issue, not a self-referential definition. Therefore no circular step is present; the score remains at the floor of the no-significant-circularity band to reflect the minor unquantified practical gap.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central unbiasedness theorem is parameter-free and rests only on standard probability laws and the informed-POMDP structure. No fitted constants enter the derivation. The two selection criteria introduce user-chosen kernel bandwidths, permutation count, and confidence level δ, but these are not fitted to the target claim. No new physical or formal entities are postulated.

free parameters (3)
  • HSCIC kernel bandwidths = median pairwise distance (median heuristic)
    Bandwidths for Gaussian RBF kernels k_G, k_I, k_Z are chosen via median heuristic (Appendix C), affecting HSCIC values and p-values; not fitted to the target result but a hand-selected hyperparameter.
  • Permutation count B and significance level α = B=30; α not pre-specified in Table 1
    The HSCIC hypothesis test uses B=30 permutations and reports p-values without a fixed rejection threshold; this influences whether a signal is deemed informative.
  • Confidence parameter δ in (ε,δ)-informativeness = δ ∈ {0.01, 0.05, 0.1}
    User-chosen confidence level; the ε test statistic and conclusions in Figure 2 depend on it.
assumptions (4)
  • domain assumption Informed POMDP observation function: p(o_t | i_t, s_t) = eO(o_t | i_t), i.e., the observation is conditionally independent of the state given the privileged information.
    Used in Section 3.2 to define the informed POMDP and in the Q/Bellman recursions; if the privileged signal does not mediate observations, the recursive definitions are not justified.
  • domain assumption The policy π(a | h) does not depend on the privileged information i, so p(i | h, a) = p(i | h).
    Required in Lemma 4.2 and Theorem 4.1 (steps b/c) for the law of total expectation to yield Q^π(h,a); this holds by construction since the actor conditions only on h.
  • standard math Standard policy gradient theorem for history-dependent policies (Eq. 1).
    The unbiasedness proof starts from E[Σ γ^t Q^π(h_t,a_t) ∇ log π(a_t|h_t)] = ∇J(πθ), a standard result for POMDPs with history policies.
  • standard math HSCIC equals zero iff conditional independence under characteristic kernels and regular conditional distribution (Theorem C.1, Park & Muandet).
    The pre-training criterion's validity depends on this equivalence between HSCIC and G_t ⊥⊥ i_t | h_t,a_t.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Informed Asymmetric Actor-Critic: Leveraging Privileged Signals Beyond Full-State Access." pith.science (2026). https://pith.science/paper/ZTGKR6AF

@misc{pith2026250926000,
  author       = {Pith},
  title        = {Pith review of: Informed Asymmetric Actor-Critic: Leveraging Privileged Signals Beyond Full-State Access},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZTGKR6AF}},
  note         = {Machine review of arXiv:2509.26000}
}
read the original abstract

Asymmetric reinforcement learning leverages privileged information available during training to improve learning under partial observability. Existing asymmetric actor-critic methods typically assume access to the full environment state to condition the critic during training, which is often unrealistic in practice. We introduce the informed asymmetric actor-critic framework that allows the critic to be conditioned on arbitrary state-dependent privileged signals, and show that any such signal yields unbiased policy gradient estimates. This substantially expands the set of admissible privileged information and raises the problem of selecting the most informative signals for learning. To this end, we propose two novel informativeness criteria: a dependence-based test that can be applied prior to training, and a test based on improvements in value prediction that can be applied post hoc. Experiments on partially observable benchmarks and synthetic environments demonstrate that carefully selected privileged signals can match or outperform full-state asymmetric baselines while relying on strictly less state information.

Figures

Figures reproduced from arXiv: 2509.26000 by the authors.

Figure 1
Figure 1. Learning performance on six benchmark navigation tasks. Curves show episodic returns [PITH_FULL_IMAGE:figures/full_fig_p010_1.png] view at source ↗
Figure 2
Figure 2. Boxplot distributions of ϵ over 1,000 test episodes for synthetic POMDP instances with privileged signal it (ς = 0.1), computed for (a) different δ across 20 instances; (b) fixed δ = 0.05 for five randomly sampled instances. (ϵ, δ)-informativeness. We validate the post-hoc informativeness criterion on 20 synthetic informed POMDP instances with noisy privileged signals (ς = 0.1). Specifically, we compare critic varia… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

33 extracted references · 3 linked inside Pith

  1. [1]

    Reinforcement learning for HVAC control in intelligent buildings: A technical and conceptual review

    Khalil Al Sayed, Abhinandana Boodi, Roozbeh Sadeghian Broujeny, and Karim Beddiar. Reinforcement learning for HVAC control in intelligent buildings: A technical and conceptual review. Journal of Building Engineering, 95, 2024. ISSN 2352-7102

    2024

  2. [2]

    MicroPPO: Safe power flow management in decentralized micro-grids with proximal policy optimization

    Daniel Ebi, Edouard Fouché, Marco Heyden, and Klemens Böhm. MicroPPO: Safe power flow management in decentralized micro-grids with proximal policy optimization. In2024 IEEE 11th International Conference on Data Science and Advanced Analytics (DSAA), pages 1–10, 2024

    2024

  3. [3]

    Deep reinforcement learning solutions for energy microgrids management

    Vincent François-Lavet, David Taralla, Damien Ernst, and Raphaël Fonteneau. Deep reinforcement learning solutions for energy microgrids management. InEuropean Workshop on Reinforcement Learning (EWRL 2016), 2016

    2016

  4. [4]

    Deep reinforcement learning framework for autonomous driving.Electronic Imaging, 2017:70–76, 01 2017

    Ahmad Sallab, Mohammed Abdou, Etienne Perot, and Senthil Yogamani. Deep reinforcement learning framework for autonomous driving.Electronic Imaging, 2017:70–76, 01 2017

    2017

  5. [5]

    Deep reinforcement learning for robotics: A survey of real-world successes

    Chen Tang, Ben Abbatematteo, Jiaheng Hu, Rohan Chandra, Roberto Martín-Martín, and Peter Stone. Deep reinforcement learning for robotics: A survey of real-world successes. Proceedings of the AAAI Conference on Artificial Intelligence, 39(27):28694–28698, Apr. 2025

    2025

  6. [6]

    Planning and acting in partially observable stochastic domains.Artificial intelligence, 101(1-2):99–134, 1998

    Leslie Pack Kaelbling, Michael L Littman, and Anthony R Cassandra. Planning and acting in partially observable stochastic domains.Artificial intelligence, 101(1-2):99–134, 1998. 13

    1998

  7. [7]

    Deep recurrent Q-learning for partially observable MDPs

    Matthew Hausknecht and Peter Stone. Deep recurrent Q-learning for partially observable MDPs. In 2015 AAAI fall symposium series, 2015

    2015

  8. [8]

    Learning deep neural network policies with continuous memory states

    Marvin Zhang, Zoe McCarthy, Chelsea Finn, Sergey Levine, and Pieter Abbeel. Learning deep neural network policies with continuous memory states. In2016 IEEE international conference on robotics and automation (ICRA), pages 520–527. IEEE, 2016

    2016

Show all 33 references
  1. [9]

    Asymmetric actor critic for image-based robot learning.arXiv preprint arXiv:1710.06542, 2017

    Lerrel Pinto, Marcin Andrychowicz, Peter Welinder, Wojciech Zaremba, and Pieter Abbeel. Asymmetric actor critic for image-based robot learning.arXiv preprint arXiv:1710.06542, 2017

    2017 arXiv

  2. [10]

    Unbiased asymmetric reinforcement learning under partial observability

    Andrea Baisero and Christopher Amato. Unbiased asymmetric reinforcement learning under partial observability. InProceedings of the Conference on Autonomous Agents and Multiagent Systems, 2022

    2022

  3. [11]

    On improving deep reinforcement learning for PODMPs.arXiv preprint arXiv:1704.07978, 2017

    Pengfei Zhu, Xin Li, Pascal Poupart, and Guanghui Miao. On improving deep reinforcement learning for PODMPs.arXiv preprint arXiv:1704.07978, 2017

    2017 arXiv

  4. [12]

    Recurrent policy gradients

    Daan Wierstra, Alexander Förster, Jan Peters, and Jürgen Schmidhuber. Recurrent policy gradients. Logic Journal of IGPL, 18(5):620–634, 2010

    2010

  5. [13]

    Reinforcement learning with long short-term memory

    Bram Bakker. Reinforcement learning with long short-term memory. In T. Dietterich, S. Becker, and Z. Ghahramani, editors,Advances in Neural Information Processing Systems, volume 14. MIT Press, 2001

    2001

  6. [14]

    Recurrent natural policy gradient for POMDPs

    Semih Cayci and Atilla Eryilmaz. Recurrent natural policy gradient for POMDPs. InICML 2024 Workshop: Foundations of Reinforcement Learning and Control – Connections and Perspectives, 2024

    2024

  7. [15]

    Bridging state and history representations: Understanding self-predictive rl

    Tianwei Ni, Benjamin Eysenbach, Erfan Seyedsalehi, Michel Ma, Clement Gehring, Aditya Mahajan, and Pierre-Luc Bacon. Bridging state and history representations: Understanding self-predictive rl. arXiv preprint arXiv:2401.08898, 2024

    2024 arXiv

  8. [16]

    Approximate information state for approximate planning and reinforcement learning in partially observed systems

    Jayakumar Subramanian, Amit Sinha, Raihan Seraj, and Aditya Mahajan. Approximate information state for approximate planning and reinforcement learning in partially observed systems. Journal of Machine Learning Research, 23(12):1–83, 2022

    2022

  9. [17]

    Data-driven planning via imitation learning.The International Journal of Robotics Research, 37(13-14):1632–1672, 2018

    Sanjiban Choudhury, Mohak Bhardwaj, Sankalp Arora, Ashish Kapoor, Gireeja Ranade, Sebastian Scherer, and Debadeepta Dey. Data-driven planning via imitation learning.The International Journal of Robotics Research, 37(13-14):1632–1672, 2018

    2018

  10. [18]

    Robust asymmetric learning in POMDPs

    Andrew Warrington, Jonathan W Lavington, Adam Scibior, Mark Schmidt, and Frank Wood. Robust asymmetric learning in POMDPs. InInternational Conference on Machine Learning, pages 11013–11023. PMLR, 2021

    2021

  11. [19]

    Informed POMDP: Leveraging additional information in model-based RL.Reinforcement Learning Journal, 2024

    Gaspard Lambrechts, Adrien Bolland, and Damien Ernst. Informed POMDP: Leveraging additional information in model-based RL.Reinforcement Learning Journal, 2024

    2024

  12. [20]

    The wasserstein believer: Learning belief updates for partially observable environments through reliable latent space models

    Raphaël Avalos, Florent Delgrange, Ann Nowe, Guillermo Perez, and Diederik M Roijers. The wasserstein believer: Learning belief updates for partially observable environments through reliable latent space models. InThe Twelfth International Conference on Learning Representation...

    2024

  13. [21]

    Hu, James Springer, Oleh Rybkin, and Dinesh Jayaraman

    Edward S. Hu, James Springer, Oleh Rybkin, and Dinesh Jayaraman. Privileged sensing scaffolds reinforcement learning. In The Twelfth International Conference on Learning Representations, 2024

    2024

  14. [22]

    Neural policy gradient methods: Global optimality and rates of convergence

    Lingxiao Wang, Qi Cai, Zhuoran Yang, and Zhaoran Wang. Neural policy gradient methods: Global optimality and rates of convergence. In8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020

    2020

  15. [23]

    A theoretical justification for asymmetric actor-critic algorithms

    Gaspard Lambrechts, Damien Ernst, and Aditya Mahajan. A theoretical justification for asymmetric actor-critic algorithms. InForty-second International Conference on Machine Learning, 2025

    2025

  16. [24]

    A hilbert space embedding for distributions

    Alex Smola, Arthur Gretton, Le Song, and Bernhard Schölkopf. A hilbert space embedding for distributions. In Marcus Hutter, Rocco A. Servedio, and Eiji Takimoto, editors,Algorithmic Learning Theory, pages 13–31, Berlin, Heidelberg, 2007. Springer Berlin Heidelberg

    2007

  17. [25]

    Measuring statistical dependence with hilbert-schmidt norms

    Arthur Gretton, Olivier Bousquet, Alex Smola, and Bernhard Schölkopf. Measuring statistical dependence with hilbert-schmidt norms. In Sanjay Jain, Hans Ulrich Simon, and Etsuji Tomita, editors,Algorithmic Learning Theory, pages 63–77, Berlin, Heidelberg, 2005. Springer Berlin ...

    2005

  18. [26]

    A measure-theoretic approach to kernel conditional mean embeddings

    Junhyung Park and Krikamol Muandet. A measure-theoretic approach to kernel conditional mean embeddings. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 21247–21259. Curran Associate...

    2020

  19. [27]

    On overfitting and asymptotic bias in batch reinforcement learning with partial observability

    Vincent François-Lavet, Guillaume Rabusseau, Joelle Pineau, Damien Ernst, and Raphael Fonteneau. On overfitting and asymptotic bias in batch reinforcement learning with partial observability. J. Artif. Int. Res., 65(1):1–30, 2019

    2019

  20. [28]

    Solving large POMDPs using real time dynamic programming

    Hector Geffner and Blai Bonet. Solving large POMDPs using real time dynamic programming. In Working Notes Fall AAAI Symposium on POMDPs, 1998

    1998

  21. [29]

    gym-pomdps: Gym environments from POMDP files.https://github

    Andrea Baisero. gym-pomdps: Gym environments from POMDP files.https://github. com/abaisero/gym-pomdps, 2019. Accessed: 2025-08-01

    2019

  22. [30]

    POMDP Robot Domains

    Hai Nguyen. POMDP Robot Domains. https://github.com/hai-h-nguyen/ pomdp-domains, 2021. Accessed: 2025-08-01

    2021

  23. [31]

    Multi-agent reinforcement learning with directed exploration and selective memory reuse

    Shuo Jiang and Christopher Amato. Multi-agent reinforcement learning with directed exploration and selective memory reuse. InProceedings of the ACM Symposium on Applied Computing, pages 777–784, 03 2021

    2021

  24. [32]

    gym-gridverse: Gridworld domains for fully and partially observable settings.https://github.com/abaisero/gym-gridverse, 2021

    Andrea Baisero and Sammie Katt. gym-gridverse: Gridworld domains for fully and partially observable settings.https://github.com/abaisero/gym-gridverse, 2021. Accessed: 2025- 08-01

    2021

  25. [33]

    asym-porl: Asymmetric methods for partially observable reinforcement learning

    Andrea Baisero and Sammie Katt. asym-porl: Asymmetric methods for partially observable reinforcement learning. https://github.com/abaisero/asym-rlpo, 2021. Accessed: 2025- 08-01. 15 A AUXILIARY RESULTS This section collects our auxiliary results. Corollary A.1(Relation of V π(...

    2021

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.