Pith. sign in

REVIEW 3 major objections 2 minor 1 cited by

Tail-Risk-Safe Monte Carlo Tree Search under PAC-Level Guarantees

T0 review · 3 major / 2 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Monte Carlo Tree Search can be made tail-safe by embedding Conditional Value-at-Risk into the backup rule, and a Wasserstein ambiguity set preserves this safety under finite samples, with PAC guarantees and regret bounds.

desk verdict Plausible risk-aware MCTS extension, but the abstract's headline PAC guarantee for W-MCTS is uncheckable without the missing specification of how the Wasserstein radius is set. read the letter →

arxiv 2508.05441 v1 pith:R2OO6OPT submitted 2025-08-07 cs.LG cs.AI

Zuyuan Zhang , Arnob Ghosh , Tian Lan This is my paper
classification cs.LGcs.AI
keywords MonteCarloTreeSearchConditionalValue-at-RisktailriskWassersteinambiguitysetPACguaranteeregretboundrisk-awaredecisionmakingdistributionalrobustness
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to fix a blind spot in Monte Carlo Tree Search (MCTS): choosing actions by expected return ignores rare but catastrophic outcomes. It proposes CVaR-MCTS, which replaces the expected-return backup with Conditional Value-at-Risk at level $\alpha$, so the search explicitly optimizes the average loss over the worst $(1-\alpha)\%$ of simulated returns. Because CVaR estimates from a handful of simulations are biased, the paper adds W-MCTS, which places a first-order Wasserstein ambiguity set around each empirical return distribution and optimizes against the worst distribution inside that set. For both variants the paper proves PAC tail-safety guarantees — a high-probability bound on the expected loss in the worst tail — and establishes regret bounds. Simulated experiments in diverse environments show the methods outperform baselines in reward and stability.

What carries the argument

The load-bearing object is the Conditional Value-at-Risk (CVaR) at level $\alpha$, the average loss in the worst $(1-\alpha)\%$ of the return distribution, used as the MCTS backup and selection statistic. To handle estimation bias, W-MCTS wraps the empirical return distribution in a first-order Wasserstein ambiguity set of radius $\varepsilon_s$; the algorithm evaluates the worst-case CVaR over all distributions within that ball. The PAC guarantees rely on the radius $\varepsilon_s$ being chosen so that, with high confidence, the true distribution lies inside the ball at every state-action pair.

What would settle it

Run W-MCTS on a simple two-armed bandit with heavy-tailed rewards, using the paper's prescribed radius $\varepsilon_s$ for a given confidence $\delta$; if, over many repeated experiments, the true distribution lies outside the ball—or the realized worst-$(1-\alpha)$-loss exceeds the asserted bound—more often than $\delta$, the PAC claim is refuted.

Watch

Extended reading notes

Core claim

The central claim is that tail-risk can be made a first-class objective in MCTS without sacrificing probabilistic soundness. Concretely, CVaR-MCTS with parameter $\alpha$ gives an explicit guarantee: with high probability, the policy's expected loss over the worst $(1-\alpha)\%$ of outcomes is bounded by a value the algorithm controls. W-MCTS extends this to the finite-sample regime by assuming the true return distribution at each state-action pair lies within a Wasserstein ball of radius $\varepsilon_s$ centered at the empirical estimate; optimizing the worst-case CVaR inside that ball preserves the PAC tail-safety guarantee. The paper also derives regret bounds for both algorithms, showing

Load-bearing premise

The PAC tail-safety guarantee for W-MCTS stands on the assumption that the true return distribution is inside the Wasserstein ball around the empirical distribution, with the radius set by a concentration inequality rather than tuned to the observed data.

Editorial extensions

If this is right

  • Decision makers can choose $\alpha$ directly to cap the expected loss in the worst $(1-\alpha)\%$ of outcomes, making MCTS usable in high-stakes settings.
  • W-MCTS extends the guarantee to realistic sample sizes, since the Wasserstein ball absorbs the estimation error of the tail.
  • The regret bounds imply that the added safety does not prevent convergence to good average performance.
  • The approach replaces hard cost constraints with a tunable risk measure, which may yield less conservative policies than threshold-based safety methods.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same ambiguity-set argument could generalize to other coherent risk measures, suggesting a recipe for risk-aware planning beyond CVaR.
  • The concentration-based radius requirement implies that deeper states, which see fewer samples, may need very large ambiguity sets; this could make W-MCTS conservative in wide or deep trees—a testable limitation.
  • The paper does not address exploration bonuses explicitly; whether the PAC guarantees extend to risk-aware exploration (e.g., UCB-style bonuses on the tail) is an open extension.
  • A natural empirical check is whether the realized tail-loss coverage matches the confidence level across a spectrum of MDPs, which would separate the concentration assumption from the algorithm's construction.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 2 minor

Summary. The paper proposes two Monte Carlo Tree Search variants, CVaR-MCTS and W-MCTS, aimed at providing tail-risk safety guarantees. CVaR-MCTS embeds Conditional Value-at-Risk into the search objective, while W-MCTS adds a first-order Wasserstein ambiguity set with radius epsilon_s to handle finite-sample bias in tail-risk estimation. The abstract claims PAC tail-safety guarantees and regret bounds for both methods, plus empirical outperformance over baselines. The full text was not provided to the referee, so this assessment is based on the abstract and the accompanying review materials.

Significance. If the theoretical guarantees are established rigorously, the paper addresses a real gap in safety-aware MCTS: expected-return objectives ignore tail risk, and prior constrained approaches lack finite-sample guarantees. The PAC framing and the use of distributional robustness via Wasserstein ambiguity sets are appropriate and potentially valuable. The paper also promises a tractable algorithm, which would make the theory actionable. Credit is due for targeting explicit tail-risk control rather than heuristic penalties, and for considering estimation bias in risk measures.

major comments (3)
  1. [Abstract (CVaR-MCTS claim)] The PAC guarantee for W-MCTS is stated as 'preserves such guarantees under finite-sample estimation bias,' but the abstract does not specify how the Wasserstein radius epsilon_s is chosen. For a PAC statement to be non-vacuous, epsilon_s must be set via a concentration inequality so that, with probability at least 1-delta, the true return distribution lies within the ball around the empirical distribution for every state-action pair. This requires an explicit functional form epsilon_s(n, delta, d) and a union bound over the number of visited nodes, which grows with tree size. If epsilon_s is a hyperparameter or calibrated on the same data, the guarantee is either vacuous or circular. The revision must state the exact construction and prove the uniform coverage bound over the tree.
  2. [Abstract (CVaR-MCTS claim)] The sentence 'achieves explicit tail-risk control over the expected loss in the worst (1-alpha)% scenarios' is essentially the definition of CVaR. As stated, it is tautological rather than a formal guarantee. The paper needs to articulate the nontrivial finite-sample PAC statement: for example, a high-probability bound on the CVaR of the return distribution induced by the policy selected by the search, with explicit dependence on the number of samples, the tree depth, and alpha. Without this, the headline 'guarantee' adds nothing beyond the choice of objective.
  3. [Abstract (regret claim)] The abstract states 'establish their regret' without giving any rates, assumptions, or problem setting. A reader cannot evaluate whether the regret bounds are novel, tight, or even of the expected order. The revision should explicitly state the regret theorems, including dependencies on the horizon, the number of iterations, the CVaR level alpha, and the Wasserstein radius epsilon_s.
minor comments (2)
  1. [Abstract (notation)] The notation 'W-MCTS' is defined only by 'Wasserstein-MCTS (or W-MCTS)' later; place the abbreviation at first use. Also, 'worst (1-alpha)% scenarios' is imprecise: it should be 'the worst (1-alpha) fraction of scenarios' or similar, since alpha is a probability level.
  2. [Abstract (clarity)] The phrase 'with respect to just the expected returns' is slightly awkward. Suggest 'using only expected returns' for readability.

Circularity Check

1 steps flagged · score 2.0 of 10

CVaR framing is definitional, but PAC and regret analyses remain independent; W-MCTS epsilon_s omission is a verifiability gap, not circularity.

  1. self definitional [Abstract (second sentence describing CVaR-MCTS)]
    "Our CVaR-MCTS with parameter $\alpha$ achieves explicit tail-risk control over the expected loss in the 'worst $(1-\alpha)\%$ scenarios.'"

    CVaR at level α is defined as the expected loss in the worst (1−α) tail. The quoted guarantee is therefore a restatement of the objective's definition, not a derived property. This is a framing tautology: any method that uses CVaR as its selection criterion trivially 'achieves' control over that tail statistic. However, the paper's separate PAC and regret claims concern sample complexity and high-probability bounds, which are not contained in the definition and are not shown to be circular here.

full rationale

The only potentially tautological passage is the abstract's statement that CVaR-MCTS 'achieves explicit tail-risk control over the expected loss in the worst (1−α)% scenarios.' Since CVaR is by definition the expected loss in that tail, this sentence restates the objective rather than a derived theorem. The central advertised results are PAC tail-safety guarantees and regret bounds for CVaR-MCTS and W-MCTS; those are nontrivial claims that do not reduce to the definition of CVaR. The objection that the Wasserstein radius ε_s is not specified in the abstract is a rigor/completeness concern, not circularity: it concerns whether the PAC bound can be instantiated, not whether the result is assumed. No self-citation chain, fitted-parameter-called-prediction, or ansatz-smuggling is visible from the provided text. Thus circularity is minor and confined to the framing sentence.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

Everything the central claim rests on: two input parameters (alpha and epsilon_s), the MDP/sampling model, the coherent-risk-measure definition of CVaR, the metric structure of the first-order Wasserstein distance, and the concentration inequalities needed for the PAC statement. No invented entities are introduced. The selection of epsilon_s is the least-specified input.

free parameters (2)
  • alpha (CVaR tail parameter) = user-specified, value not disclosed in abstract
    Defines the worst (1-alpha) fraction of outcomes the planner protects against. It is a design choice, standard for CVaR, but the guarantee explicitly depends on it.
  • epsilon_s (Wasserstein radius per state-action pair) = not specified in abstract
    Sets the size of the ambiguity set around the empirical return distribution at each (s,a). The PAC tail-safety guarantee depends on the true distribution lying inside this ball, so how the radius is chosen is load-bearing.
assumptions (4)
  • domain assumption The decision problem is a Markov decision process or bandit model with well-defined state-action return distributions.
    MCTS operates on such a model; the abstract assumes expected returns and state-action pairs exist and can be sampled.
  • domain assumption CVaR is a coherent risk measure whose alpha-level value captures the worst (1-alpha)% scenarios.
    This is the standard definition of CVaR, imported from risk theory; the paper's tail-risk language depends on it.
  • standard math First-order Wasserstein distance defines a valid ambiguity set structure for perturbing the empirical distribution.
    The ambiguity set P_epsilon_s(s,a) is a Wasserstein ball; existence and metric properties of Wasserstein distances are assumed as background.
  • domain assumption Concentration inequalities apply to the empirical return estimates so that the true distribution lies in the ambiguity ball with controllable probability.
    The PAC guarantee requires a probabilistic relationship between sample size, radius epsilon_s, and tail-risk estimation error; the abstract does not state the required sample-complexity conditions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Tail-Risk-Safe Monte Carlo Tree Search under PAC-Level Guarantees." pith.science (2026). https://pith.science/paper/R2OO6OPT

@misc{pith2026250805441,
  author       = {Pith},
  title        = {Pith review of: Tail-Risk-Safe Monte Carlo Tree Search under PAC-Level Guarantees},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/R2OO6OPT}},
  note         = {Machine review of arXiv:2508.05441}
}
abstract

Making decisions with respect to just the expected returns in Monte Carlo Tree Search (MCTS) cannot account for the potential range of high-risk, adverse outcomes associated with a decision. To this end, safety-aware MCTS often consider some constrained variants -- by introducing some form of mean risk measures or hard cost thresholds. These approaches fail to provide rigorous tail-safety guarantees with respect to extreme or high-risk outcomes (denoted as tail-risk), potentially resulting in serious consequence in high-stake scenarios. This paper addresses the problem by developing two novel solutions. We first propose CVaR-MCTS, which embeds a coherent tail risk measure, Conditional Value-at-Risk (CVaR), into MCTS. Our CVaR-MCTS with parameter $\alpha$ achieves explicit tail-risk control over the expected loss in the "worst $(1-\alpha)\%$ scenarios." Second, we further address the estimation bias of tail-risk due to limited samples. We propose Wasserstein-MCTS (or W-MCTS) by introducing a first-order Wasserstein ambiguity set $\mathcal{P}_{\varepsilon_{s}}(s,a)$ with radius $\varepsilon_{s}$ to characterize the uncertainty in tail-risk estimates. We prove PAC tail-safety guarantees for both CVaR-MCTS and W-MCTS and establish their regret. Evaluations on diverse simulated environments demonstrate that our proposed methods outperform existing baselines, effectively achieving robust tail-risk guarantees with improved rewards and stability.

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Operator-Guided Invariance Learning for Continuous Reinforcement Learning

    cs.LG 2026-05 unverdicted novelty 7.0 of 10

    VPSD-RL discovers exact and approximate value-preserving Lie-group operators in continuous RL to stabilize learning via transition augmentation and consistency regularization.

Reference graph

Works this paper leans on

54 extracted references · 37 canonical work pages · cited by 1 Pith paper

  1. [1]

    Modification of UCT with Patterns in Monte-Carlo Go

    Sylvain Gelly, Yizao Wang, R \'e mi Munos, and Olivier Teytaud. Modification of UCT with Patterns in Monte-Carlo Go. PhD thesis, INRIA, 2006

    work page 2006

  2. [2]

    Mastering the game of go with deep neural networks and tree search

    David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al. Mastering the game of go with deep neural networks and tree search. nature, 529 0 (7587): 0 484--489, 2016

    2016

  3. [3]

    A general reinforcement learning algorithm that masters chess, shogi, and go through self-play

    David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, et al. A general reinforcement learning algorithm that masters chess, shogi, and go through self-play. Science, 362 0 (6419): 0 1140--1144, 2018

    2018

  4. [4]

    Mastering atari, go, chess and shogi by planning with a learned model

    Julian Schrittwieser, Ioannis Antonoglou, Thomas Hubert, Karen Simonyan, Laurent Sifre, Simon Schmitt, Arthur Guez, Edward Lockhart, Demis Hassabis, Thore Graepel, et al. Mastering atari, go, chess and shogi by planning with a learned model. Nature, 588 0 (7839): 0 604--609, 2020

    2020

  5. [5]

    Progressive strategies for monte-carlo tree search

    Guillaume M Jb Chaslot, Mark HM Winands, H Jaap van den Herik, Jos WHM Uiterwijk, and Bruno Bouzy. Progressive strategies for monte-carlo tree search. New Mathematics and Natural Computation, 4 0 (03): 0 343--357, 2008

    work page 2008

  6. [6]

    A survey on model-based reinforcement learning

    Fan-Ming Luo, Tian Xu, Hang Lai, Xiong-Hui Chen, Weinan Zhang, and Yang Yu. A survey on model-based reinforcement learning. Science China Information Sciences, 67 0 (2): 0 121101, 2024

    work page 2024

  7. [7]

    Lipschitz lifelong monte carlo tree search for mastering non-stationary tasks

    Zuyuan Zhang and Tian Lan. Lipschitz lifelong monte carlo tree search for mastering non-stationary tasks. arXiv preprint arXiv:2502.00633, 2025

    arXiv 2025

  8. [8]

    Decision making for autonomous driving considering interaction and uncertain prediction of surrounding vehicles

    Constantin Hubmann, Marvin Becker, Daniel Althoff, David Lenz, and Christoph Stiller. Decision making for autonomous driving considering interaction and uncertain prediction of surrounding vehicles. In 2017 IEEE intelligent vehicles symposium (IV), pages 1671--1678. IEEE, 2017

    work page 2017

Show all 54 references
  1. [9]

    Cooperation-aware reinforcement learning for merging in dense traffic

    Maxime Bouton, Alireza Nakhaei, Kikuo Fujimura, and Mykel J Kochenderfer. Cooperation-aware reinforcement learning for merging in dense traffic. In 2019 IEEE Intelligent Transportation Systems Conference (ITSC), pages 3441--3447. IEEE, 2019

    2019

  2. [10]

    Learning to collaborate with unknown agents in the absence of reward

    Zuyuan Zhang, Hanhan Zhou, Mahdi Imani, Taeyoung Lee, and Tian Lan. Learning to collaborate with unknown agents in the absence of reward. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 14502--14511, 2025 a

    2025

  3. [11]

    A cluster-based weighted feature similarity moving target tracking algorithm for automotive fmcw radar

    Rongqian Chen, Yingquan Zou, Anyong Gao, and Leshi Chen. A cluster-based weighted feature similarity moving target tracking algorithm for automotive fmcw radar. In 2022 IEEE 95th Vehicular Technology Conference:(VTC2022-Spring), pages 1--5. IEEE, 2022

    2022

  4. [12]

    Stability improvement of pulse power supply with dual-inductance active storage unit using hysteresis current control

    Ping Yang, Xi Chen, Rongqian Chen, Yusheng Peng, Songrong Wu, and Jianping Xu. Stability improvement of pulse power supply with dual-inductance active storage unit using hysteresis current control. IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 11 0 (1):...

    2021

  5. [13]

    Look-ahead robust network optimization with generative state predictions

    Fei Xu Yu, Zuyuan Zhang, Emily Grob, Gina Adam, Sean Coffey, Nathaniel D Bastian, and Tian Lan. Look-ahead robust network optimization with generative state predictions. In AAAI 2025 Workshop on Artificial Intelligence for Wireless Communications and Networking (AI4WCN)

    2025

  6. [14]

    Design and characterization of a pneumatic tunable-stiffness bellows actuator

    Rongqian Chen, Jun Kwon, Wei-Hsi Chen, and Cynthia Sung. Design and characterization of a pneumatic tunable-stiffness bellows actuator. In 2024 IEEE 7th International Conference on Soft Robotics (RoboSoft), pages 997--1003. IEEE, 2024

    2024

  7. [15]

    A distributed abstract mac layer for cooperative learning on internet of vehicles

    Yifei Zou, Zuyuan Zhang, Congwei Zhang, Yanwei Zheng, Dongxiao Yu, and Jiguo Yu. A distributed abstract mac layer for cooperative learning on internet of vehicles. IEEE Transactions on Intelligent Transportation Systems, 25 0 (8): 0 8972--8983, 2024

    2024

  8. [16]

    A crowdsensing service pricing method in vehicular edge computing

    Zheng Li, Sizhe Tang, Hao Tian, Haolong Xiang, Xiaolong Xu, and Wanchun Dou. A crowdsensing service pricing method in vehicular edge computing. In 2024 IEEE International Symposium on Parallel and Distributed Processing with Applications (ISPA), pages 82--89. IEEE, 2024

    2024

  9. [17]

    Cnn partitioning and offloading for vehicular edge networks in web3

    Xiaolong Xu, Sizhe Tang, Lianyong Qi, Xiaokang Zhou, Fei Dai, and Wanchun Dou. Cnn partitioning and offloading for vehicular edge networks in web3. IEEE Communications Magazine, 61 0 (8): 0 36--42, 2023 a

    2023

  10. [18]

    Hayes, Mathieu Reymond, Diederik M

    Conor F. Hayes, Mathieu Reymond, Diederik M. Roijers, Enda Howley, and Patrick Mannion. Monte carlo tree search algorithms for risk-aware and multi-objective reinforcement learning, 2022. URL https://arxiv.org/abs/2211.13032

    2022 arXiv

  11. [19]

    Sim \ a o, Alessandro Farinelli, and Matthijs T

    Alberto Castellini, Federico Bianchi, Edoardo Zorzi, Thiago D. Sim \ a o, Alessandro Farinelli, and Matthijs T. J. Spaan. Scalable safe policy improvement via monte carlo tree search. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonat...

    2023

  12. [20]

    Modeling other players with bayesian beliefs for games with incomplete information

    Zuyuan Zhang, Mahdi Imani, and Tian Lan. Modeling other players with bayesian beliefs for games with incomplete information. arXiv preprint arXiv:2405.14122, 2024 a

    2024 arXiv

  13. [21]

    Threshold uct: Cost-constrained monte carlo tree search with pareto curves

    Martin Kure c ka, V \'a clav Nevyho s t e n \`y , Petr Novotn \`y , and V \' t Un c ovsk \`y . Threshold uct: Cost-constrained monte carlo tree search with pareto curves. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 26569--26578, 2025

    2025

  14. [22]

    Monte-carlo tree search for constrained pomdps

    Jongmin Lee, Geon-Hyeong Kim, Pascal Poupart, and Kee-Eung Kim. Monte-carlo tree search for constrained pomdps. Advances in Neural Information Processing Systems, 31, 2018

    2018

  15. [23]

    Conditional value-at-risk for general loss distributions

    R Tyrrell Rockafellar and Stanislav Uryasev. Conditional value-at-risk for general loss distributions. Journal of banking & finance, 26 0 (7): 0 1443--1471, 2002

    2002

  16. [24]

    Q-learning

    Christopher JCH Watkins and Peter Dayan. Q-learning. Machine learning, 8: 0 279--292, 1992

    1992

  17. [25]

    Playing atari with deep reinforcement learning

    Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602, 2013

    2013 arXiv

  18. [26]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  19. [27]

    Advantage actor-critic with reasoner: Explaining the agent's behavior from an exploratory perspective

    Muzhe Guo, Feixu Yu, Tian Lan, and Fang Jin. Advantage actor-critic with reasoner: Explaining the agent's behavior from an exploratory perspective. arXiv preprint arXiv:2309.04707, 2023

    2023 arXiv

  20. [28]

    Br-defedrl: Byzantine-robust decentralized federated reinforcement learning with fast convergence and communication efficiency

    Jing Qiao, Zuyuan Zhang, Sheng Yue, Yuan Yuan, Zhipeng Cai, Xiao Zhang, Ju Ren, and Dongxiao Yu. Br-defedrl: Byzantine-robust decentralized federated reinforcement learning with fast convergence and communication efficiency. In Ieee infocom 2024-ieee conference on computer com...

    2024

  21. [29]

    Distributed age-of-information scheduling with noma via deep reinforcement learning

    Congwei Zhang, Yifei Zou, Zuyuan Zhang, Dongxiao Yu, Jorge Torres G \'o mez, Tian Lan, Falko Dressler, and Xiuzhen Cheng. Distributed age-of-information scheduling with noma via deep reinforcement learning. IEEE Transactions on Mobile Computing, 2024 b

    2024

  22. [30]

    Network diffuser for placing-scheduling service function chains with inverse demonstration

    Zuyuan Zhang, Vaneet Aggarwal, and Tian Lan. Network diffuser for placing-scheduling service function chains with inverse demonstration. In IEEE INFOCOM 2025-IEEE Conference on Computer Communications, pages 1--10. IEEE, 2025 b

    2025

  23. [31]

    Cooperative backdoor attack in decentralized reinforcement learning with theoretical guarantee

    Mengtong Gao, Yifei Zou, Zuyuan Zhang, Xiuzhen Cheng, and Dongxiao Yu. Cooperative backdoor attack in decentralized reinforcement learning with theoretical guarantee. arXiv preprint arXiv:2405.15245, 2024

    2024 arXiv

  24. [32]

    Implementing first-person shooter game ai in wild-scav with rule-enhanced deep reinforcement learning

    Zeyu Fang, Jian Zhao, Wengang Zhou, and Houqiang Li. Implementing first-person shooter game ai in wild-scav with rule-enhanced deep reinforcement learning. In 2023 IEEE Conference on Games (CoG), pages 1--8. IEEE, 2023

    2023

  25. [33]

    Learning from random demonstrations: Offline reinforcement learning with importance-sampled diffusion models

    Zeyu Fang and Tian Lan. Learning from random demonstrations: Offline reinforcement learning with importance-sampled diffusion models. arXiv preprint arXiv:2405.19878, 2024

    2024 arXiv

  26. [34]

    Near-minimax-optimal risk-sensitive reinforcement learning with cvar

    Kaiwen Wang, Nathan Kallus, and Wen Sun. Near-minimax-optimal risk-sensitive reinforcement learning with cvar. In International Conference on Machine Learning, pages 35864--35907. PMLR, 2023

    2023

  27. [35]

    On reward-free reinforcement learning with linear function approximation

    Ruosong Wang, Simon S Du, Lin Yang, and Russ R Salakhutdinov. On reward-free reinforcement learning with linear function approximation. Advances in neural information processing systems, 33: 0 17816--17826, 2020

    2020

  28. [36]

    Risk-sensitive reinforcement learning: near-optimal risk-sample tradeoff in regret

    Yingjie Fei, Zhuoran Yang, Yudong Chen, Zhaoran Wang, and Qiaomin Xie. Risk-sensitive reinforcement learning: near-optimal risk-sample tradeoff in regret. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS '20, Red Hook, NY, USA,...

    2020

  29. [37]

    Constrained policy optimization

    Joshua Achiam, David Held, Aviv Tamar, and Pieter Abbeel. Constrained policy optimization. In International conference on machine learning, pages 22--31. PMLR, 2017

    2017

  30. [38]

    A lyapunov-based approach to safe reinforcement learning

    Yinlam Chow, Ofir Nachum, Edgar Duenez-Guzman, and Mohammad Ghavamzadeh. A lyapunov-based approach to safe reinforcement learning. Advances in neural information processing systems, 31, 2018 a

    2018

  31. [39]

    Natural policy gradient primal-dual method for constrained markov decision processes

    Dongsheng Ding, Kaiqing Zhang, Tamer Basar, and Mihailo R Jovanovic. Natural policy gradient primal-dual method for constrained markov decision processes. In NeurIPS, 2020

    2020

  32. [40]

    Provably efficient model-free constrained rl with linear function approximation

    Arnob Ghosh, Xingyu Zhou, and Ness Shroff. Provably efficient model-free constrained rl with linear function approximation. arXiv preprint arXiv:2206.11889, 2022

    2022 arXiv

  33. [41]

    Exploration-exploitation in constrained mdps

    Yonathan Efroni, Shie Mannor, and Matteo Pirotta. Exploration-exploitation in constrained mdps. arXiv preprint arXiv:2003.02189, 2020

    2003 arXiv

  34. [42]

    Learning policies with zero or bounded constraint violation for constrained mdps

    Tao Liu, Ruida Zhou, Dileep Kalathil, PR Kumar, and Chao Tian. Learning policies with zero or bounded constraint violation for constrained mdps. arXiv preprint arXiv:2106.02684, 2021

    2021 arXiv

  35. [43]

    A provably-efficient model-free algorithm for constrained markov decision processes

    Honghao Wei, Xin Liu, and Lei Ying. A provably-efficient model-free algorithm for constrained markov decision processes. arXiv preprint arXiv:2106.01577, 2021

    2021 arXiv

  36. [44]

    Risk-constrained reinforcement learning with percentile risk criteria

    Yinlam Chow, Mohammad Ghavamzadeh, Lucas Janson, and Marco Pavone. Risk-constrained reinforcement learning with percentile risk criteria. Journal of Machine Learning Research, 18 0 (167): 0 1--51, 2018 b

    2018

  37. [45]

    Online learning in risk sensitive constrained mdp

    Arnob Ghosh and Mehrdad Moharrami. Online learning in risk sensitive constrained mdp. In Forty-second International Conference on Machine Learning

  38. [46]

    Constrained risk-averse markov decision processes

    Mohamadreza Ahmadi, Ugo Rosolia, Michel D Ingham, Richard M Murray, and Aaron D Ames. Constrained risk-averse markov decision processes. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 11718--11725, 2021

    2021

  39. [47]

    Reinforcement learning of risk-constrained policies in markov decision processes

    Tom \'a s Br \'a zdil, Krishnendu Chatterjee, Petr Novotn \`y , and Ji r \' Vahala. Reinforcement learning of risk-constrained policies in markov decision processes. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 9794--9801, 2020

    2020

  40. [48]

    Design and control of a tunable-stiffness coiled-spring actuator

    Shivangi Misra, Mason Mitchell, Rongqian Chen, and Cynthia Sung. Design and control of a tunable-stiffness coiled-spring actuator. In 2023 IEEE International Conference on Robotics and Automation (ICRA), number 72, 2023

    2023

  41. [49]

    Mastering chess and shogi by self-play with a general reinforcement learning algorithm

    David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, et al. Mastering chess and shogi by self-play with a general reinforcement learning algorithm. arXiv preprint arXiv:17...

    2017 arXiv

  42. [50]

    Multi-objective monte-carlo tree search

    Weijia Wang and Michele Sebag. Multi-objective monte-carlo tree search. In Asian conference on machine learning, pages 507--522. PMLR, 2012

    2012

  43. [51]

    C-mcts: Safe planning with monte carlo tree search

    Dinesh Parthasarathy, Georgios Kontes, Axel Plinge, and Christopher Mutschler. C-mcts: Safe planning with monte carlo tree search. arXiv preprint arXiv:2305.16209, 2023

    2023 arXiv

  44. [52]

    Improved sample complexity bounds for distributionally robust reinforcement learning

    Zaiyan Xu, Kishan Panaganti, and Dileep Kalathil. Improved sample complexity bounds for distributionally robust reinforcement learning. In International Conference on Artificial Intelligence and Statistics, pages 9728--9754. PMLR, 2023 b

    2023

  45. [53]

    An environment for autonomous driving decision-making

    Edouard Leurent. An environment for autonomous driving decision-making. https://github.com/eleurent/highway-env, 2018

    2018

  46. [54]

    Optimization of conditional value-at-risk

    R Tyrrell Rockafellar, Stanislav Uryasev, et al. Optimization of conditional value-at-risk. Journal of risk, 2: 0 21--42, 2000

    2000

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.