Strategy Masking: A Method for Guardrails in Value-based Reinforcement Learning Agents

Jeremy Kedziora; Jonathan Keane; Sam Keyser

REVIEW 4 major objections 5 minor 20 references

Strategy Masking: A Method for Guardrails in Value-based Reinforcement Learning Agents

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Strategy masking suppresses learned lying in RL agents after training, with win rate nearly unchanged.

desk verdict A plausible inference-time guardrail idea in value-based RL, but the central win-rate preservation claim is not empirically supported as presented. read the letter →

arxiv 2501.05501 v2 pith:D5FPSTRL submitted 2025-01-09 cs.AI cs.LGcs.MA

Jonathan Keane , Sam Keyser , Jeremy Kedziora This is my paper · ORCID

classification cs.AIcs.LGcs.MA

keywords strategymaskingrewarddecompositionguardrailsreinforcementlearninglyingsocialdeceptionDQNvalue-basedRL

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces strategy masking, a way to make a value-based reinforcement learning agent change its behavior after training by decomposing its reward into separate dimensions and tuning a mask vector that picks which dimensions the agent may use when choosing actions. The authors apply it to lying in the social-deception game Coup, where a lie is well-defined: completing an action or block without holding the corresponding card. They show that an agent trained only to win can be steered at inference time by giving the 'lie' reward dimension a negative weight, driving lying actions to near zero while leaving win rate roughly unchanged. The method matters because it offers a way to impose guardrails on already-trained agents without retraining or extra compute, provided the unwanted behavior can be observed and encoded as a reward channel. A convergence theorem is proved for masked Q-learning, showing the masked update converges to the optimal Q-values under standard stochastic-approximation conditions.

What carries the argument

The strategy mask is a coefficient vector $\vec{m}$ whose dot product with the decomposed state-action value vector $\vec{Q}(s,a)$ selects and weights reward dimensions in the agent's action choice: $a^*(s) = \arg\max_a \vec{Q}(s,a) \cdot \vec{m}$. During training, masked SARSA, masked expected SARSA, and masked Q-learning incorporate $\vec{m}$ into the temporal-difference target, so the agent learns future values under the constraint that its own subsequent decisions will respect the mask. At inference the mask can be changed without retraining, which lets the user suppress or punish behaviors that were learned during training. The masked Q-learning update is shown to be a contraction mapping in the sup-norm, so the masked values converge with probability one to the optimal Q-values of the masked problem.

What would settle it

Play the same post-training sweep in a variant of Coup where lying is frequently optimal, for example a rule change that makes bluffs much cheaper to attempt; if negative lie weights then cut win rate substantially, the claim that masking suppresses lying without compromising performance would fail. Alternatively, an environment where the lie reward is only partially observable, such as estimating hallucination in language-model outputs, would test whether the method still works when the reward channel must itself be inferred.

Watch

Extended reading notes

Core claim

The paper's central claim is that a value-based agent's learned incentives can be explicitly decomposed into reward dimensions, and that by masking or negatively weighting a dimension during inference the experimenter can suppress the associated behavior without materially harming performance on the primary task. Evidence comes from Coup: a win-only agent, when its lie-reward dimension is weighted negative after training, takes lying actions in nearly zero percent of its decisions while its win rate drops only slightly; positive weights, by contrast, push lying up to roughly 70 percent of actions and drag win rate down. The authors interpret this asymmetry as lying being advantageous in only a limited set of game states, so removing the incentive forces changes in few decisions, while inflating it pushes the agent into riskier lies. The result is presented as a general guardrail technique for any value-based RL setting with a decomposable reward.

Load-bearing premise

The whole demonstration depends on the unwanted behavior—here, lying—being unambiguously detectable and scored as its own reward channel; in settings where the behavior cannot be observed and encoded that cleanly, strategy masking cannot be applied as stated.

Editorial extensions

If this is right

A trained agent's lying rate can be driven to near zero by a negative mask weight on the lie dimension, with win rate minimally affected.
Positive mask weights increase lying up to roughly 70 percent of actions and lower win rate, showing the mask can also amplify behaviors.
The same mask-sweep result holds when opponent-selection priorities are updated, though the overall win rate drops under harder opponents.
Masked Q-learning converges with probability one to the optimal Q-values under standard learning-rate conditions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

In real applications like LLM hallucination, the limiting step will be constructing a reliable reward channel for the unwanted behavior; if that channel can be built, the same post-training mask sweep could serve as a cheap alignment intervention.
The asymmetry between positive and negative weights suggests a general recipe: suppressing a behavior only changes decisions where that behavior was marginally preferred, so it should generally be cheaper than forcing a behavior.
The technique could be extended to policy-gradient methods by using the mask to reweight a decomposed advantage or by reshaping the policy distribution, though the paper does not show how.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Strategy masking is a reasonable, clearly written extension of reward decomposition: train with decomposed Q-values, then adjust a weight vector at inference to suppress a behavior. The Coup experiments show the expected mechanical effect—negative weights on the Lie dimension drive lying actions toward zero—and the authors are honest that direct observability of the target behavior is a precondition. The convergence theorem is a standard contraction argument and is fine for the tabular case.

What is genuinely new is the post-training inference-time mask sweep, which I don't think prior reward decomposition work does. The paper is also well organized and the limitations section is candid.

The soft spots are empirical. The central claim that win rate is 'minimally impacted' rests on Figure 4, which has no error bars, no seeds, and no comparison baseline. In a three-player game with baseline win around 33%, a drop of even a few points without variance estimates is uninterpretable. The robustness run with updated PFSP priorities has the same problem. The internal inconsistency the stress test flags is real: Table IV shows the top two collected reward dimensions for the Win-Lie agent are Win and Challenge, not Win and Lie, and for Win-Challenge it's Win and Lie, not Win and Challenge. That undercuts the 'controllable behavior shaping' sub-claim, though not the suppression direction, which is mechanistic.

The suppression direction itself is partly by construction—changing the scalarized Q-values changes action selection—so the independent content is the win-rate claim, and that is not yet demonstrated. I'd also like to see a baseline like retraining with a fixed negative weight, and some analysis of how mask weights interact with the league's opponent distribution.

Who is this for? Researchers working on reward decomposition, interpretable RL, or inference-time guardrails in AI safety. It deserves a serious referee because the idea is novel and the paper is honestly written, but it needs major revision: statistical rigor, baselines, and ideally released code before the performance-preservation claim can be accepted.

Referee Report

4 major / 5 minor

Summary. The paper proposes "strategy masking," a method that decomposes a value-based RL agent's reward into multiple scalar dimensions, trains the agent with access to a masked subset of those dimensions, and then adjusts the mask at inference time to suppress or penalize an unwanted behavior. The method is demonstrated in the social deception game Coup, where the unwanted behavior is lying (taking an action while not holding the matching card). The authors train DQN/DRQN agents with four reward dimensions (Win, Challenge, Lie, Bait), show that different training masks produce agents with different behavioral tendencies, and then show that varying the Lie mask weight post-training can drive lying actions to near zero while claiming that win rate is only minimally impacted. A convergence proof is provided for masked tabular Q-learning in Appendix C.

Significance. If the central performance-preservation claim holds, strategy masking would be a lightweight, post-training guardrail for value-based RL systems, and the Coup environment is a well-chosen testbed because lying is unambiguous and observable. The paper is clearly written, provides pseudocode, and includes a tabular convergence proof with a standard contraction argument. The strength of the contribution, however, is currently limited by the weak empirical support for the headline claim: the key figure has no statistical uncertainty quantification, and one of the training-behavior claims is contradicted by the paper's own table.

major comments (4)

[V-C, Figure 4] The central claim that lying can be suppressed "without compromising agent ability to perform effectively" rests entirely on Figure 4, which reports a single sweep over lie-dimension weights with no error bars, no confidence intervals, no multiple seeds, and no statistical test. In a three-player game the baseline win rate is about 33%, so a difference of a few percentage points is hard to interpret without variance information. The robustness rerun with updated PFSP priorities also lacks error bars. The manuscript should report means and variances over at least several independent training runs (or clearly state if only one agent was used), and ideally include a paired comparison (e.g., win rate of the same agent with lie weight 0 vs. -1 on the same set of episodes) or a statistical test to support the phrase "minimally impacted." Without this, the main conclusion is not quantitatively supported.
[V-A, Table IV] The text in Section V-A asserts that "for either agent their top two dimensions by reward were the dimensions we biased them towards," but Table IV shows the opposite. For the Win-Lie agent, the top two collected rewards are Win (78.22%) and Challenge (9.09%), with Lie (7.41%) third. For the Win-Challenge agent, the top two are Win (83.72%) and Lie (9.10%), with Challenge (0.79%) last. This internal inconsistency weakens the sub-claim that reward decomposition and strategy masking create agents with predictable, controllable behavioral priorities during training. The authors should either correct the claim or discuss why the top-two reward distributions do not match the intended mask.
[II-C, Appendix C] The convergence theorem in Section II-C and Appendix C proves convergence only for the scalarized quantity Q(s,a)\cdot m under a restricted update rule, and only for tabular MDPs. It does not cover convergence of the individual decomposed components, nor does it cover the DQN/DRQN function-approximation setting used in all the experiments. The paper should explicitly state that the theoretical guarantee applies only to the scalarized tabular update, and should not imply that the experimental algorithm inherits these guarantees. The proof also contains small errors (e.g., the variance expression is missing a closing parenthesis, and an extra (s,a) appears in the definition of Delta_{t+1}) that should be fixed.
[VI] The authors themselves list a critical scope limitation: strategy masking as demonstrated relies on the unwanted behavior being unambiguous and directly observable, so it can be encoded as a reward dimension. The paper's abstract and introduction frame the method as a general guardrail mechanism, but the only fully demonstrated case is one where a clean ground-truth signal (holding a specific card in Coup) is available. The manuscript should more prominently temper the generalization claims and discuss what would be needed to apply the method when the target behavior (e.g., LLM hallucination) must first be detected or estimated. This is not a correctness error in the presented experiments, but it is load-bearing for the paper's stated significance.

minor comments (5)

[Abstract] There is a typo in the abstract: the quotation mark after "unethical" is not closed, and the following sentence begins with "Without" but should be a new sentence or be connected grammatically.
[II-A, Algorithm 1] In Algorithm 1 (Masked DQN), the loss in line 11 sums over all K dimensions but the target y is defined in Eq. (1) as a vector; it would be clearer to explicitly write y^{(j)}_k = r_k(s^{(j)}, a^{(j)}, s'^{(j)}) + gamma Q^{(k)}(s'^{(j)}, a^*_m(s'^{(j)}) | w') for non-terminal states.
[Table III] The eight strategy masks in Table III are presented without explicit column numbers; the text refers to "column 1," "column 2," etc., which is understandable but could be made more reader-friendly by numbering the columns.
[V-A, Table IV] The Win-Lie agent's win rate in Table IV (30.56%) is below the 33% expected against uniform random play in a three-player game, and the Win-Challenge agent's win rate is 42.58%. The authors may want to comment on whether the Win-Lie agent's below-chance win rate indicates a weak agent or a strong league, since this affects interpretation of the subsequent post-training results.
[Appendix C] The proof of Lemma C.1 uses the notation a^{*,i}_m(s) and then switches to argmax; the argument is standard but a few steps are compressed, and the transition from the triangle inequality to the max-norm bound would be clearer with an explicit statement that max_a |Q1(s,a) - Q2(s,a)| <= ||Q1 - Q2||_inf.

Circularity Check

1 steps flagged · score 1.0 of 10

No significant circularity: only the behavior-suppression direction is definitional; the performance-preservation claim is measured independently.

self definitional [Section II-A (strategy mask definition) and Section V-C (Figure 4 discussion)]
"Setting the kth entry of ⃗m to 0 would mean suppressing the contribution that the factor measured by rk(·) makes. Setting it to a negative value would amount to punishing the agent for taking an action possessed of that factor. ... when we disincentivized lying through the strategy mask (with weights less than 0), we see that lying actions decreased to near zero while the win rate was minimally impacted."

Action selection is defined as a*(s) = argmax_a Q(s,a)·m (Section II-A). Lowering m_Lie mechanically lowers the scalarized score of actions with large Q_Lie, so the observed fall in lying is a direct consequence of the mask definition, not an independent empirical prediction. The genuinely non-circular content is the measured win-rate impact in Figure 4, which is an external simulation outcome rather than an arithmetic consequence of the equations.

full rationale

The paper's derivation chain is largely self-contained. Reward decomposition and the mask are defined in Section II-A, and the masked DQN update (Eq. 1) is a straightforward adaptation of standard Q-learning; the convergence theorem uses a standard contraction argument. The experimental claims are simulation-based measurements, not fitted parameters renamed as predictions. There are no self-citations, and no load-bearing appeal to the authors' prior work. The only mildly definitional element is that suppressing a behavior by setting its mask weight negative directly alters the argmax in action selection, so the qualitative direction of the lying reduction is by construction. However, the quantitative near-zero lying rate and, more importantly, the 'minimally impacted win rate' result are empirical measurements from simulated games, so the central claim retains independent content. The score is low because no derived prediction reduces to its inputs by construction beyond this intended mechanism.

Assumptions & free parameters 4 free parameters · 5 assumptions · 1 invented entities

The central results rest on hand-chosen reward magnitudes and mask configurations, on an unambiguously observable Lie channel, and on win rate against a self-built league as the performance measure. These are not fitted to data, so the circularity burden is low, but they are free choices that determine the reported effect sizes.

free parameters (4)

Reward magnitudes for Win, Challenge, Lie, Bait dimensions = 10, 1, 1, 1
Set a priori in Section IV-C; the scaling between Win and other dimensions controls the relative incentive and shapes the reported trade-offs in Figure 4.
Training mask configuration = Column 1 of Table III for the main agent: Win=1, Challenge=0, Lie=0, Bait=0
The choice of training mask determines which Q-dimensions influence action selection during training and is central to the post-training behavior change.
Core RL hyperparameters = Not reported
Learning rate, discount factor, epsilon schedule, batch size, replay buffer size, and LSTM hidden dimensions are not given in the text, so the exact operating point is unreproducible.
League play hyperparameters = p=0.3, z=6, checkpoint every 50k episodes, exploiter training 50k episodes, w=1000, p=39 league players, n=3 players…
Appendix B specifies these values; they control opponent distribution and therefore affect all win-rate measurements.

assumptions (5)

standard math Finite state and action spaces, bounded rewards, and bounded mask coefficients for the convergence theorem
Theorem II.1 and its proof in Appendix C assume these conditions; they do not hold in the DQN/DRQN function-approximation experiments.
domain assumption Lying is unambiguously detectable and can be assigned a scalar reward channel
Section IV-C and Table II define the Lie reward when a player completes an action or block without holding the required card; this requires ground-truth private information that is not available in most real deployments.
ad hoc to paper The four reward dimensions (Win, Challenge, Lie, Bait) are sufficient to represent behavior relevant to winning and honesty
The authors chose this decomposition to balance model complexity against capturing a minimal representation of information management in Coup, Section IV-C. Other behaviors such as deception by omission are not modeled.
domain assumption Q-estimates for a masked dimension remain meaningful enough post-training to guide inference-time behavior changes
In Algorithm 1 all dimensions including Lie are updated even when the Lie mask is 0, but actions are chosen by the masked scalar. The post-training manipulation depends on these values being usable. The authors note that the optimality of training/inference mask alignment is open in the footnote to Section IV-C.
domain assumption Win rate against the StarLite league is a valid measure of the agent's ability to perform effectively
Section V-C uses win percentage against league opponents selected by PFSP; league composition depends on training dynamics and priority updates, so absolute win rate is not a fixed external benchmark.

invented entities (1)

Strategy mask vector m
purpose: Weights reward dimensions at training and inference to suppress, remove, or penalize behavior channels.
Introduced as the central control mechanism. Its effects are demonstrated in the Coup experiments, but it is a design artifact without external benchmark support.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Strategy Masking: A Method for Guardrails in Value-based Reinforcement Learning Agents." pith.science (2026). https://pith.science/paper/D5FPSTRL

@misc{pith2026250105501,
  author       = {Pith},
  title        = {Pith review of: Strategy Masking: A Method for Guardrails in Value-based Reinforcement Learning Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/D5FPSTRL}},
  note         = {Machine review of arXiv:2501.05501}
}

read the original abstract

The use of reward functions to structure AI learning and decision making is core to the current reinforcement learning paradigm; however, without careful design of reward functions, agents can learn to solve problems in ways that may be considered "undesirable" or "unethical." Without thorough understanding of the incentives a reward function creates, it can be difficult to impose principled yet general control mechanisms over its behavior. In this paper, we study methods for constructing guardrails for AI agents that use reward functions to learn decision making. We introduce a novel approach, which we call strategy masking, to explicitly learn and then suppress undesirable AI agent behavior. We apply our method to study lying in AI agents and show that it can be used to effectively modify agent behavior by suppressing lying post-training without compromising agent ability to perform effectively.

Figures

Figures reproduced from arXiv: 2501.05501 by the authors.

**Figure 2.** Comparison of distribution of actions that would have been taken [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

**Figure 3.** Comparison of distribution of actions that would have been taken [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: Across 5000 games, win percent and percentage of actions that [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

**Figure 5.** Figure 5: DQN function-approximator architecture used for training Coup [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

**Figure 6.** Figure 6: StarLite league play structure. We depict how this league play develops over time in [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

20 extracted references · 10 canonical work pages

[1]

A. P. Badia, P. Sprechmann, A. Vitvitskyi, D. Guo, B. Piot, S. Kapturowski, O. Tieleman, M. Arjovsky, A. Pritzel, A. Bolt, and C. Blundell. Never give up: Learning directed exploration strategies. 2020. URL http://arxiv.org/abs/2002.06038

arXiv 2020
[2]

Brockman, V

G. Brockman, V. Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba. Openai gym. CoRR, abs/1606.01540, 2016. URL http://arxiv.org/abs/1606.01540

arXiv 2016
[3]

Brown and T

N. Brown and T. Sandholm. Libratus: The superhuman ai for no-limit poker. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17 , pages 5226--5228, 2017. doi:10.24963/ijcai.2017/772. URL https://doi.org/10.24963/ijcai.2017/772

doi:10.24963/ijcai.2017/772 2017
[4]

Brown and T

N. Brown and T. Sandholm. Superhuman ai for multiplayer poker. Science, 365 0 (6456): 0 885--890, 2019. doi:10.1126/science.aay2400. URL https://www.science.org/doi/abs/10.1126/science.aay2400

doi:10.1126/science.aay2400 2019
[5]

Dietterich

T. Dietterich. Hierarchical reinforcement learning with the maxq value function decomposition. Journal of Artifcial Intelligence Research, 13, 2000

work page 2000
[6]

M. J. Hausknecht and P. Stone. Deep recurrent q-learning for partially observable mdps. CoRR, abs/1507.06527, 2015. URL http://arxiv.org/abs/1507.06527

arXiv 2015
[7]

Hochreiter and J

S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural Comput., 9 0 (8): 0 1735–1780, nov 1997. ISSN 0899-7667. doi:10.1162/neco.1997.9.8.1735. URL https://doi.org/10.1162/neco.1997.9.8.1735

doi:10.1162/neco.1997.9.8.1735 1997
[8]

R. T. Icarte, T. Klassen, R. Valenzano, and S. McIlraith. Using reward machines for high-level task specification and decomposition in reinforcement learning. In J. Dy and A. Krause, editors, Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 2107--2116. PMLR, 10--15 Jul 2018....

work page 2018

Show all 20 references

[9]

Jaakkola, M

T. Jaakkola, M. Jordan, and S. Singh. Convergence of stochastic iterative dynamic programming algorithms. Neural Computation, 6: 0 1185--1201, 1994

1994
[10]

Juozapaitis, A

Z. Juozapaitis, A. Koul, A. Fern, M. Erwig, and F. Doshi-Velez. Explainable reinforcement learning via reward decomposition. In in proceedings at the International Joint Conference on Artificial Intelligence. A Workshop on Explainable Artificial Intelligence., 2019

2019
[11]

Karlsson

J. Karlsson. Task decomposition in reinforcement learning. In 1994 AAAI Spring Symposium, 1994. URL https://aaai.org/papers/0006-ss94-02-006-task-decomposition-in-reinforcement-learning/

1994
[12]

V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. A. Riedmiller. Playing atari with deep reinforcement learning. CoRR, abs/1312.5602, 2013. URL http://arxiv.org/abs/1312.5602

2013 arXiv
[13]

S. J. Russell and A. Zimdars. Q-decomposition for reinforcement learning agents. In International Conference on Machine Learning, 2003. URL https://api.semanticscholar.org/CorpusID:5376984

2003
[14]

D. Shi, X. Guo, Y. Liu, and W. Fan. Optimal policy of multiplayer poker via actor-critic reinforcement learning. 24 0 (6), 2022. ISSN 1099-4300. doi:10.3390/e24060774. URL https://www.mdpi.com/1099-4300/24/6/774

2022 doi
[15]

T. Shu, C. Xiong, , and R. Socher. Hierarchical and interpretable skill acquisition in multi-task reinforcement learning. In International Conference on Learning Representations, 2018

2018
[16]

R. S. Sutton and A. G. Barto. Reinforcement Learning: An Introduction. A Bradford Book, Cambridge, MA, USA, 2018. ISBN 0262039249

2018
[17]

C. K. Tham and R. W. Prager. A modular q-learning architecture for manipulator task decomposition. In In Machine Learning Proceedings, 1994

1994
[18]

van Seijen, M

H. van Seijen, M. Fatemi, J. Romoff, R. Laroche, T. Barnes, and J. Tsang. Hybrid reward architecture for reinforcement learning. CoRR, abs/1706.04208, 2017

2017 arXiv
[19]

Vinyals, I

O. Vinyals, I. Babuschkin, W. M. Czarnecki, M. Mathieu, A. Dudzik, J. Chung, D. H. Choi, R. Powell, T. Ewalds, P. Georgiev, J. Oh, D. Horgan, M. Kroiss, I. Danihelka, A. Huang, L. Sifre, T. Cai, J. P. Agapiou, M. Jaderberg, A. S. Vezhnevets, R. Leblond, T. Pohlen, V. Dalibard,...

2019
[20]

Z. Wang, X. Li, L. Sun, H. Zhang, H. Liu, and J. Wang. Learning state-specific action masks for reinforcement learning. Algorithms, 17 0 (2), 2024. ISSN 1999-4893. doi:10.3390/a17020060. URL https://www.mdpi.com/1999-4893/17/2/60

2024 doi

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

A. P. Badia, P. Sprechmann, A. Vitvitskyi, D. Guo, B. Piot, S. Kapturowski, O. Tieleman, M. Arjovsky, A. Pritzel, A. Bolt, and C. Blundell. Never give up: Learning directed exploration strategies. 2020. URL http://arxiv.org/abs/2002.06038

arXiv 2020

[2] [2]

Brockman, V

G. Brockman, V. Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba. Openai gym. CoRR, abs/1606.01540, 2016. URL http://arxiv.org/abs/1606.01540

arXiv 2016

[3] [3]

Brown and T

N. Brown and T. Sandholm. Libratus: The superhuman ai for no-limit poker. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17 , pages 5226--5228, 2017. doi:10.24963/ijcai.2017/772. URL https://doi.org/10.24963/ijcai.2017/772

doi:10.24963/ijcai.2017/772 2017

[4] [4]

Brown and T

N. Brown and T. Sandholm. Superhuman ai for multiplayer poker. Science, 365 0 (6456): 0 885--890, 2019. doi:10.1126/science.aay2400. URL https://www.science.org/doi/abs/10.1126/science.aay2400

doi:10.1126/science.aay2400 2019

[5] [5]

Dietterich

T. Dietterich. Hierarchical reinforcement learning with the maxq value function decomposition. Journal of Artifcial Intelligence Research, 13, 2000

work page 2000

[6] [6]

M. J. Hausknecht and P. Stone. Deep recurrent q-learning for partially observable mdps. CoRR, abs/1507.06527, 2015. URL http://arxiv.org/abs/1507.06527

arXiv 2015

[7] [7]

Hochreiter and J

S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural Comput., 9 0 (8): 0 1735–1780, nov 1997. ISSN 0899-7667. doi:10.1162/neco.1997.9.8.1735. URL https://doi.org/10.1162/neco.1997.9.8.1735

doi:10.1162/neco.1997.9.8.1735 1997

[8] [8]

R. T. Icarte, T. Klassen, R. Valenzano, and S. McIlraith. Using reward machines for high-level task specification and decomposition in reinforcement learning. In J. Dy and A. Krause, editors, Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 2107--2116. PMLR, 10--15 Jul 2018....

work page 2018