Pith. sign in

REVIEW 3 major objections 3 minor 84 references

Knowledge- and Gradient-Guided Reinforcement Learning for Parametrized Action Markov Decision Processes

T0 review · 3 major / 3 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read By pruning the decision space with a Datalog knowledge base and refining parameters with projected gradient ascent on the Q-function, KGRL achieves higher returns and better sample efficiency than five PAMDP baselines.

desk verdict Useful neuro-symbolic RL for parametrized action spaces, but the gains largely come from handcrafted per-domain rules; the gradient-guided component's benefit is not supported by their own ablations. read the letter →

arxiv 2607.12924 v2 pith:HILGXV5F submitted 2026-07-14 cs.AI

Jonas Ehrhardt , René Heesch , Oliver Niggemann This is my paper · ORCID
classification cs.AI
keywords reinforcementlearningparametrizedactionMDPneuro-symbolicreasoningDatalogpruninggradient-guidedparameteroptimizationsampleefficiencyexplainable
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that explicit but incomplete symbolic domain knowledge can make reinforcement learning in parametrized action spaces dramatically more sample-efficient. In a parametrized action MDP (PAMDP), each decision is a discrete action plus continuous parameters, and existing methods estimate parameters with a single forward pass. KGRL instead evaluates a Datalog knowledge base at every step to prune actions that are known to be inapplicable and to constrain parameters to feasible ranges, then refines parameter guesses by projected gradient ascent on the Q-function. The authors argue this guides exploration toward feasible decisions and leads to higher episodic returns with lower variance than state-of-the-art baselines on four benchmark domains. A sympathetic reader would take away that symbolic rules and gradient-based parameter search are complementary, and that even incomplete rules can substantially improve both training and deployment.

What carries the argument

The central object is the knowledge-constrained PAMDP, a PAMDP enriched with a Datalog knowledge base K and an abstraction function α mapping continuous states to symbolic facts. From these, the set of applicable actions and, for each action, a feasible parameter region are derived as hard constraints. The PARAMOPT loop — projected gradient ascent on the differentiable Q-function with respect to the parameters, projecting back into the feasible region — is the mechanism that turns one-shot parameter estimates into iteratively refined ones. KGRL is the DQN-based algorithm that couples rule-based action pruning with this parameter refinement at every step.

What would settle it

Take a PAMDP where the rules are deliberately corrupted to prune a random subset of actions with the same pruning rate as the true rules. If KGRL with random rules matches the AULC of KGRL with true rules, the reported advantage could be due to action-space reduction rather than semantic guidance; if true rules clearly outperform random rules of equal pruning rate, the knowledge content is doing the causal work.

Watch

Extended reading notes

Core claim

KGRL extends DQN with two coupled mechanisms: a Datalog knowledge base that, together with a symbolic abstraction of the state, computes the set of applicable actions and the feasible parameter region for each action; and PARAMOPT, a projected gradient ascent loop that optimizes the action's parameter by ascending the action-value function's gradient within the feasible region. At decision time, KGRL evaluates the rule base, prunes non-applicable actions, runs PARAMOPT for each remaining action, and greedily selects the action-parameter pair with the highest Q-value. The paper reports that across HardMoveX, CatchPoint, HardGoal, and Platform, KGRL achieves the highest average Area Under the

Load-bearing premise

The load-bearing premise is that the handcrafted Datalog rules and the symbolic abstraction are correct enough to prune only suboptimal decisions; a wrong rule that prunes an optimal action or parameter range directly harms the agent's performance.

Editorial extensions

If this is right

  • If KGRL's claims hold, RL agents can safely embed incomplete expert rules as hard constraints without losing the ability to learn the unspecified part of the policy.
  • Sample efficiency in PAMDPs improves because exploration is restricted to feasible action-parameter regions, so fewer environment interactions are wasted on clearly invalid decisions.
  • Gradient-based parameter refinement at decision time can substitute for learned one-shot parameter estimators, yielding better parameters with the same Q-network.
  • The approach produces local procedural explanations as a byproduct of the decision mechanism, which is relevant for safety-critical deployment.
  • Knowledge guidance matters most at deployment, suggesting that rule bases act as a hard feasibility filter even when the policy is already trained.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's own ablations imply that enabling gradient-guided refinement only after the Q-function is well fitted could preserve the deployment benefit while avoiding the training-time noise it observes.
  • If the knowledge base were learned from data or demonstrations instead of handcrafted, the same algorithm could transfer to domains where experts cannot articulate rules; the hard-pruning design would then need softening to tolerate rule errors.
  • Systematically corrupting the rule base (e.g., randomly flipping a fraction of pruning rules) and measuring AULC would isolate whether the gains come from the semantic content of the knowledge or merely from shrinking the action space.
  • The explanation traces suggest a post-hoc aggregation step could produce global explanations — e.g., which rules fire most often in which state regions — turning local justifications into policy-level insight.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. The paper proposes Knowledge- and Gradient-Guided Reinforcement Learning (KGRL), a DQN extension for Parametrized Action MDPs. KGRL evaluates a handcrafted Datalog knowledge base at each state to prune non-applicable actions and constrain feasible parameter regions, and uses PARAMOPT, a projected gradient-ascent loop over the Q-function, to refine parameter estimates. The authors formalize knowledge-constrained PAMDPs, provide an algorithm with explanation traces, and evaluate on HardMove4/8/10, CatchPoint, HardGoal, and Platform against QPAMDP, PA-DDPG, P-DQN, MP-DQN, and HyAR. They report that KGRL achieves the highest average AULC across all domains and seeds, and that ablations show the knowledge base matters at deployment while gradient guidance has mixed or negative effects during training.

Significance. If the empirical claims are established, the paper contributes a useful neuro-symbolic extension to PAMDP RL: a formal integration of incomplete Datalog knowledge with action pruning and parameter constraints, a practical gradient-based parameter refinement, public code, and a demonstration of local procedural explanation traces. The experimental design (eight seeds, 16 evaluation runs, standard benchmarks, ablations) is a strength. However, the central claim that KGRL outperforms all baselines is not yet cleanly separated from the handcrafted, near-optimal knowledge injected into KGRL, and the paper's own ablations undermine the training-time benefit of the gradient component. These issues must be addressed before the headline claims are acceptable.

major comments (3)
  1. [§1 RQ2; §5.2 Table 1; §6] RQ2 asks whether gradient-guided parameter refinement increases sample efficiency and performance compared with one-shot estimators. This is contradicted by the paper's own Table 1: KGRL (train nogg) outperforms or matches full KGRL on HardMove4 (-1.43 vs -1.64), HardMove8 (-1.34 vs -1.44), and CatchPoint (-1.69 vs -1.70), and is essentially tied on HardGoal and Platform. The Discussion (§6) concedes that "integrating the gradient-guidance during training has a negative effect." Since the Abstract and RQ2 claim a training-time sample-efficiency gain, the current evidence supports at most a deployment-time benefit. Please reconcile the claims and clarify whether the nogg ablation also removes PARAMOPT from the target computation in Alg. 1 lines 18-21.
  2. [§5.1; Table 2; Table 1; §6] The handcrafted knowledge bases encode strong, near-optimal task knowledge: HardMoveX rules prune actions pointing away from the goal by >90° and cap parameters by distance; HardGoal rules prune segments blocked by the keeper. Table 2 shows ~49% action pruning in HardMoveX, and Table 1 shows that removing the knowledge base at evaluation degrades AULC dramatically (e.g., HardMove4 from -1.64 to -4.33). No baseline receives equivalent knowledge, and, despite §6 claiming that KGRL can operate with learned knowledge bases, no experiment with learned, noisy, or imperfect rules is presented. The headline advantage may therefore reflect the injected rules rather than the KGRL mechanism itself. Please add baselines that receive the same action/parameter masks, or a sensitivity study over corrupted/learned rules, to establish the separable contribution of the algorithm.
  3. [§5.2; Table 1] The AULC comparison reports only means ± one standard deviation over eight seeds, with no significance tests, paired comparisons, or confidence intervals. In the Platform row, the best baseline MP-DQN (0.19±0.06) overlaps with KGRL (0.22±0.02), and similar overlap may exist for other rows. The claim in §5.2 that KGRL "consistently achieves the highest average AULC" needs statistical support across the eight seeds, or at least a clear statement of effect sizes and variability.
minor comments (3)
  1. [§4.1, Eq. (15)] Eq. (15) writes y = r + γ(1-d') arg max_{a'} max_ψ Q; the target should be the maximum value, not an arg max action. Algorithm 1 line 21 is correct, so this appears to be a typo in the displayed equation.
  2. [§5.2] Typo: "algortihm" should be "algorithm."
  3. [Figure 3] The learning curves show only HardMove4, CatchPoint, HardGoal, and Platform; adding HardMove8 and HardMove10 would make the cross-domain claim fully visible.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: KGRL's reported gains come from an empirical comparison with handcrafted knowledge inputs, not from a formal derivation that reduces to its own assumptions.

full rationale

The paper proposes KGRL, a DQN extension that prunes actions and constrains parameters using a Datalog knowledge base and refines parameters by projected gradient ascent. There is no theorem or derived prediction whose conclusion is identical by construction to an input; the knowledge base is an explicit input, and its contribution is tested through ablations rather than assumed. The handcrafted rules in §5.1 (e.g., HardMoveX pruning actions pointing away from the goal and capping parameters by distance) are a potential experimental confound, because the baselines do not receive equivalent domain priors, and the paper itself acknowledges in §6 that wrong rules can prune optimal actions and that learned knowledge bases are left to future work. However, an unfair or confounded comparison is a validity concern, not circularity under the stated criteria. The self-citations — Ehrhardt et al. 2025 for gradient-based refinement and Vranješ et al. 2024 for experimental design — are attributions to prior work and are not load-bearing; the algorithm, ablations, and evaluation are presented in this paper. Thus the derivation chain is self-contained, and no specific circular step can be exhibited.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central empirical claim does not rest on fitted physical constants, but it rests on a per-domain handcrafted infrastructure: abstraction thresholds, Datalog rule sets, and PARAMOPT/DQN hyperparameters. These are inputs authored by the paper's authors for the evaluation domains, not outputs of a theory, and exact definitions are not fully reported.

free parameters (4)
  • Abstraction thresholds α
    The threshold tests that map numeric states to symbolic facts (e.g., 'tank near full iff level ≥ 41' in §3.2) are hand-chosen per domain. The exact thresholds for the evaluation domains are not reported, but the method's behavior depends on them.
  • Knowledge-rule constants and parameter caps
    Rules such as HardMoveX 'cap parameters based on the distance to the goal' and Platform constraints based on 'enemy closeness, high velocity, or near gaps' contain manually chosen numeric bounds. These are not specified in the paper and directly shape the feasible parameter regions.
  • PARAMOPT hyperparameters (ζ, ξ, U_max, initial guess)
    The projected-gradient-ascent learning rate, stopping threshold, maximum iterations, and initialization strategy are algorithm hyperparameters that affect whether refinement improves or harms training. They are not reported.
  • DQN hyperparameters
    Network architecture, replay buffer size, exploration schedule, batch size, and target-sync period are not reported. They govern all algorithms and are needed to reproduce the numbers.
assumptions (5)
  • standard math Stratified Datalog knowledge base K has a unique intended model for every state
    Invoked in §3.2 to define entailment in Eq. (6). It is a standard result for stratified Datalog with negation-as-failure.
  • domain assumption A correct symbolic abstraction function α from numeric states to a finite proposition set exists and is provided
    Eq. (4) assumes a given abstraction function. The paper notes in §6 that threshold-based abstractions are risky near thresholds, especially in noisy environments.
  • ad hoc to paper The handcrafted rules are incomplete but do not exclude optimal decisions
    The empirical claim rests on rules that prune many suboptimal actions while retaining optimal ones. Section 5.1 describes per-domain rules authored for the benchmarks, and §6 concedes that wrong rules can prune optimal actions or parameter ranges.
  • domain assumption Projected gradient ascent on the non-convex Qθ finds near-optimal parameters
    Used in Algorithm 2 and Eq. (15). Section 6 acknowledges 'there is no guarantee that ψ̂ will refine into a global optimum'.
  • domain assumption Actions not provably non-applicable are applicable by default
    Eq. (8) adopts a default applicability scheme: app(a) holds unless napp(a) can be derived. If the abstraction or rules fail to fire, unsafe or infeasible actions remain available.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Knowledge- and Gradient-Guided Reinforcement Learning for Parametrized Action Markov Decision Processes." pith.science (2026). https://pith.science/paper/HILGXV5F

@misc{pith2026260712924,
  author       = {Pith},
  title        = {Pith review of: Knowledge- and Gradient-Guided Reinforcement Learning for Parametrized Action Markov Decision Processes},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HILGXV5F}},
  note         = {Machine review of arXiv:2607.12924}
}
read the original abstract

In this paper, we study Reinforcement Learning in Parametrized Action Markov Decision Processes (PAMDP), where each decision consists of a symbolic action and numerical parameters. In such settings Reinforcement Learning algorithms typically determine parameters with one-shot estimators, which makes their training sample inefficient. Though in most PAMDP environments explicit but incomplete knowledge (e.g., rules, safety constraints, or expert heuristics) is available, it is rarely directly used to increase the sample-efficiency of training Reinforcement Learning agents. We step into this gap and propose our novel Neuro-Symbolic Knowledge- and Gradient-Guided Reinforcement Learning (KGRL) algorithm. KGRL uses domain knowledge in a Datalog knowledge base to derive the set of applicable actions and feasible parameters for a given state. This allows it to prune non-applicable actions from the decision-space and constrain the parameter spaces of the remaining actions. We then use a gradient-based parameter refinement loop to estimate the optimal parameters during training and deployment of the agent. By recording activated rules along the trajectory, KGRL additionally provides local procedural explanations on the pruning of actions and constraining of parameters. Overall, KGRL guides the agent's exploration and deployment toward feasible and constraint-aware decisions, while increasing sample efficiency during training. KGRL outperforms state-of-the-art RL baselines for PAMDPs in both, sample efficiency and episodic return.

Figures

Figures reproduced from arXiv: 2607.12924 by the authors.

Figure 1
Figure 1. We propose the novel Neuro-Symbolic Knowledge [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. The Figure shows our evaluation PAMDP domains. We [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. The Figure shows the learning curves of KGRL and the baselines on the [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: The Figure shows an exemplary excerpt from the [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

84 extracted references · 12 canonical work pages

  1. [1]

    Achiam, J.; Held, D.; Tamar, A.; and Abbeel, P. 2017. Constrained policy optimization. In Precup, D., and Teh, Y. W., eds., Proceedings of the 34th International Conference on Machine Learning , volume 70 of Proceedings of Machine Learning Research , 22--31. PMLR

    2017

  2. [2]

    Alshiekh, M.; Bloem, R.; Ehlers, R.; K\" o nighofer, B.; Niekum, S.; and Topcu, U. 2018. Safe reinforcement learning via shielding. Proceedings of the AAAI Conference on Artificial Intelligence 32(1)

    2018

  3. [3]

    Anderson, G.; Verma, A.; Dillig, I.; and Chaudhuri, S. 2020. Neurosymbolic reinforcement learning with formally verified exploration. In Larochelle, H.; Ranzato, M.; Hadsell, R.; Balcan, M.; and Lin, H., eds., Advances in Neural Information Processing Systems , volume 33, 6172--6183. Curran Associates, Inc

    2020

  4. [4]

    Arora, A.; Fiorino, H.; Pellier, D.; Métivier, M.; and Pesty, S. 2018. A review of learning planning action models. The Knowledge Engineering Review 33

    2018

  5. [5]

    J.; James, S

    Bester, C. J.; James, S. D.; and Konidaris, G. D. 2019. Multi-pass q-networks for deep reinforcement learning with parameterised action spaces. arXiv preprint arXiv:1905.04388

    arXiv 2019

  6. [6]

    Ciatto, G.; Sabbatini, F.; Agiollo, A.; Magnini, M.; and Omicini, A. 2024. Symbolic knowledge extraction and injection with sub-symbolic predictors: A systematic literature review. ACM Computing Surveys 56(6):1–35

    2024

  7. [7]

    data-hungry

    Duan, Y., and Wainwright, M. J. 2024. Taming "data-hungry" reinforcement learning? stability in continuous state-action spaces. In Globerson, A.; Mackey, L.; Belgrave, D.; Fan, A.; Paquet, U.; Tomczak, J.; and Zhang, C., eds., Advances in Neural Information Processing Systems , volume 37, 71352--71380. Curran Associates, Inc

    2024

  8. [8]

    J.; Li, J.; Paduraru, C.; Gowal, S.; and Hester, T

    Dulac-Arnold, G.; Levine, N.; Mankowitz, D. J.; Li, J.; Paduraru, C.; Gowal, S.; and Hester, T. 2021. Challenges of real-world reinforcement learning: definitions, benchmarks and analysis. Machine Learning 110(9):2419–2468

    2021

Show all 84 references
  1. [9]

    Ehrhardt, J.; Schmidt, J.; Heesch, R.; and Niggemann, O. 2025. Using gradient-based optimization for planning with deep q-networks in parametrized action spaces. In Proceedings of the ECAI Workshop on AI-based Planning for Complex Real-World Applications (CAIPI 2025) , volume ...

    2025

  2. [10]

    Fan, Z.; Su, R.; Zhang, W.; and Yu, Y. 2019. Hybrid actor-critic reinforcement learning in parameterized action space. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19 , 2279--2285. International Joint Conferences on Artif...

    2019

  3. [11]

    Feng, Y.; Zhu, J.; Platzer, A.; and Laurent, J. 2025. Adaptive shielding via parametric safety proofs. Proceedings of the ACM on Programming Languages 9:816–843

    2025

  4. [12]

    Fulton, N., and Platzer, A. 2018. Safe reinforcement learning via formal methods: Toward safe control through proof and learning. Proceedings of the AAAI Conference on Artificial Intelligence 32(1)

    2018

  5. [13]

    Ghanadbashi, S., and Golpayegani, F. 2022. Using ontology to guide reinforcement learning agents in unseen situations: A traffic signal control system case study. Applied Intelligence 52(2):1808–1824

    2022

  6. [14]

    Han, S.; Dastani, M.; and Wang, S. 2026. Neuro-symbolic action masking for deep reinforcement learning. In Proceedings of the 25th International Conference on Autonomous Agents and Multiagent Systems , AAMAS. International Foundation for Autonomous Agents and Multiagent Systems

    2026

  7. [15]

    J., and Stone, P

    Hausknecht, M. J., and Stone, P. 2016. Deep reinforcement learning in parameterized action space. In Bengio, Y., and LeCun, Y., eds., 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings

    2016

  8. [16]

    Heesch, R.; Cimatti, A.; Ehrhardt, J.; Diedrich, A.; and Niggemann, O. 2024. A lazy approach to neural numerical planning with control parameters. In Proceedings of the 27th European Conference on Artificial Intelligence (ECAI 2024) . IOS Press

    2024

  9. [17]

    K\" o nighofer, B.; Lorber, F.; Jansen, N.; and Bloem, R. 2020. Shield Synthesis for Reinforcement Learning . Springer International Publishing. 290–306

    2020

  10. [18]

    K\" o nighofer, B.; Bloem, R.; Jansen, N.; Junges, S.; and Pranger, S. 2025. Shields for safe reinforcement learning. Communications of the ACM 68(11):80–90

    2025

  11. [19]

    Krarup, B.; Cashmore, M.; Magazzeni, D.; and Miller, T. 2019. Towards model-based contrastive explanations for explainable planning. In ICAPS Workshop on Explainable AI Planning (XAIP)

    2019

  12. [21]

    Levine, S.; Kumar, A.; Tucker, G.; and Fu, J. 2020. Offline reinforcement learning: Tutorial, review, and perspectives on open problems. arXiv preprint arXiv:2005.01643

    2020 arXiv

  13. [22]

    Levitin, E., and Polyak, B. 1966. Constrained minimization methods. USSR Computational Mathematics and Mathematical Physics 6(5):1–50

    1966

  14. [23]

    Li, B.; Tang, H.; Zheng, Y.; Hao, J.; Li, P.; Wang, Z.; Meng, Z.; and Wang, L. 2022. Hyar: Addressing discrete-continuous action reinforcement learning via hybrid action representation. In International Conference on Learning Representations

    2022

  15. [24]

    B.; and Wu, J

    Mao, J.; Gan, C.; Kohli, P.; Tenenbaum, J. B.; and Wu, J. 2019. The neuro-symbolic concept learner: Interpreting scenes, words, and sentences from natural supervision. In International Conference on Learning Representations

    2019

  16. [25]

    Masson, W.; Ranchod, P.; and Konidaris, G. 2016. Reinforcement learning with parameterized actions. Proceedings of the AAAI Conference on Artificial Intelligence 30(1)

    2016

  17. [26]

    Mazumder, S.; Liu, B.; Wang, S.; Zhu, Y.; Yin, X.; Liu, L.; and Li, J. 2022. Knowledge-guided exploration in deep reinforcement learning. arXiv preprint arXiv:2210.15670

    2022 arXiv

  18. [27]

    Mazzi, G.; Meli, D.; Castellini, A.; and Farinelli, A. 2023. Learning logic specifications for soft policy guidance in POMCP . In Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems , 373--381

    2023

  19. [28]

    Meli, D.; Castellini, A.; and Farinelli, A. 2024. Learning logic specifications for policy guidance in pomdps: an inductive logic programming approach. Journal of Artificial Intelligence Research 79:725–776

    2024

  20. [29]

    Milani, S.; Topin, N.; Veloso, M.; and Fang, F. 2024. Explainable reinforcement learning: A survey and comparative review. ACM Computing Surveys 56(7):1–36

    2024

  21. [30]

    A.; Veness, J.; Bellemare, M

    Mnih, V.; Kavukcuoglu, K.; Silver, D.; Rusu, A. A.; Veness, J.; Bellemare, M. G.; Graves, A.; Riedmiller, M.; Fidjeland, A. K.; Ostrovski, G.; Petersen, S.; Beattie, C.; Sadik, A.; Antonoglou, I.; King, H.; Kumaran, D.; Wierstra, D.; Legg, S.; and Hassabis, D. 2015. Human-leve...

    2015

  22. [31]

    M.; Broekens, J.; Plaat, A.; and Jonker, C

    Moerland, T. M.; Broekens, J.; Plaat, A.; and Jonker, C. M. 2023. Model-based reinforcement learning: A survey. Foundations and Trends in Machine Learning 16(1):1–118

    2023

  23. [32]

    Silver, D.; Schrittwieser, J.; Simonyan, K.; Antonoglou, I.; Huang, A.; Guez, A.; Hubert, T.; Baker, L.; Lai, M.; Bolton, A.; Chen, Y.; Lillicrap, T.; Hui, F.; Sifre, L.; van den Driessche, G.; Graepel, T.; and Hassabis, D. 2017. Mastering the game of go without human knowledg...

    2017

  24. [33]

    Silver, D.; Hubert, T.; Schrittwieser, J.; Antonoglou, I.; Lai, M.; Guez, A.; Lanctot, M.; Sifre, L.; Kumaran, D.; Graepel, T.; Lillicrap, T.; Simonyan, K.; and Hassabis, D. 2018. A general reinforcement learning algorithm that masters chess, shogi, and go through self-play. S...

    2018

  25. [34]

    S., and Barto, A

    Sutton, R. S., and Barto, A. G. 2018. Reinforcement learning: An introduction . Cambridge, Massachusetts: MIT Press

    2018

  26. [35]

    Veronese, C.; Farinelli, A.; and Meli, D. 2026. Sample-efficient neurosymbolic deep reinforcement learning

    2026

  27. [36]

    S.; and Niggemann, O

    Vranje s , D.; Ehrhardt, J.; Heesch, R.; Moddemann, L.; Steude, H. S.; and Niggemann, O. 2024. Design Principles for Falsifiable, Replicable and Reproducible Empirical Machine Learning Research . In Pill, I.; Natan, A.; and Wotawa, F., eds., 35th International Conference on Pr...

    2024

  28. [37]

    Xiong, J.; Wang, Q.; Yang, Z.; Sun, P.; Han, L.; Zheng, Y.; Fu, H.; Zhang, T.; Liu, J.; and Liu, H. 2018. Parametrized deep q-networks learning: Reinforcement learning with discrete-continuous hybrid action space

    2018

  29. [38]

    Yu, Y. 2018. Towards sample efficient reinforcement learning. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence , IJCAI-2018, 5739–5743. International Joint Conferences on Artificial Intelligence Organization

    2018

  30. [39]

    Zhu, J.; Wu, F.; and Zhao, J. 2021. An overview of the action space for deep reinforcement learning. In 2021 4th International Conference on Algorithms, Computing and Artificial Intelligence , ACAI’21. ACM

    2021

  31. [40]

    Proceedings of the 34th International Conference on Machine Learning , pages =

    Constrained Policy Optimization , author =. Proceedings of the 34th International Conference on Machine Learning , pages =. 2017 , editor =

    2017

  32. [41]

    Safe Reinforcement Learning via Shielding , volume =

    Alshiekh, Mohammed and Bloem, Roderick and Ehlers, R\". Safe Reinforcement Learning via Shielding , volume =. Proceedings of the AAAI Conference on Artificial Intelligence , publisher =. doi:10.1609/aaai.v32i1.11797 , number =

    doi

  33. [42]

    Neurosymbolic Reinforcement Learning with Formally Verified Exploration , url =

    Anderson, Greg and Verma, Abhinav and Dillig, Isil and Chaudhuri, Swarat , booktitle =. Neurosymbolic Reinforcement Learning with Formally Verified Exploration , url =

  34. [43]

    A review of learning planning action models , volume =

    Arora, Ankuj and Fiorino, Humbert and Pellier, Damien and Métivier, Marc and Pesty, Sylvie , year =. A review of learning planning action models , volume =. doi:10.1017/s0269888918000188 , journal =

    doi

  35. [44]

    and James, Steven D

    Bester, Craig J. and James, Steven D. and Konidaris, George D. , title =. arXiv preprint arXiv:1905.04388 , year =. doi:10.48550/ARXIV.1905.04388 , url =

    1905 doi arXiv

  36. [45]

    and Moré, Jorge J

    Calamai, Paul H. and Moré, Jorge J. , year =. Projected gradient methods for linearly constrained problems , volume =. Mathematical Programming , publisher =. doi:10.1007/bf02592073 , number =

    doi

  37. [46]

    Symbolic Knowledge Extraction and Injection with Sub-symbolic Predictors: A Systematic Literature Review , volume =

    Ciatto, Giovanni and Sabbatini, Federico and Agiollo, Andrea and Magnini, Matteo and Omicini, Andrea , year =. Symbolic Knowledge Extraction and Injection with Sub-symbolic Predictors: A Systematic Literature Review , volume =. ACM Computing Surveys , publisher =. doi:10.1145/...

    doi

  38. [47]

    data-hungry

    Duan, Yaqi and Wainwright, Martin J , booktitle =. Taming "data-hungry" reinforcement learning? Stability in continuous state-action spaces , url =. doi:10.52202/079017-2279 , editor =

    doi

  39. [48]

    and Li, Jerry and Paduraru, Cosmin and Gowal, Sven and Hester, Todd , year =

    Dulac-Arnold, Gabriel and Levine, Nir and Mankowitz, Daniel J. and Li, Jerry and Paduraru, Cosmin and Gowal, Sven and Hester, Todd , year =. Challenges of real-world reinforcement learning: definitions, benchmarks and analysis , volume =. Machine Learning , publisher =. doi:10...

    doi

  40. [49]

    Proceedings of the ECAI Workshop on AI-based Planning for Complex Real-World Applications (CAIPI 2025) , url=

    Using Gradient-based Optimization for Planning with Deep Q-Networks in Parametrized Action Spaces , author=. Proceedings of the ECAI Workshop on AI-based Planning for Complex Real-World Applications (CAIPI 2025) , url=. 2025 , volume=

    2025

  41. [50]

    Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence,

    Hybrid Actor-Critic Reinforcement Learning in Parameterized Action Space , author =. Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence,. 2019 , month =. doi:10.24963/ijcai.2019/316 , url =

    2019 doi

  42. [51]

    Adaptive Shielding via Parametric Safety Proofs , volume =

    Feng, Yao and Zhu, Jun and Platzer, André and Laurent, Jonathan , year =. Adaptive Shielding via Parametric Safety Proofs , volume =. doi:10.1145/3720450 , journal =

    doi

  43. [52]

    Proceedings of the 36th International Conference on Machine Learning , pages =

    Off-Policy Deep Reinforcement Learning without Exploration , author =. Proceedings of the 36th International Conference on Machine Learning , pages =. 2019 , editor =

    2019

  44. [53]

    Safe Reinforcement Learning via Formal Methods: Toward Safe Control Through Proof and Learning , volume=

    Fulton, Nathan and Platzer, André , year=. Safe Reinforcement Learning via Formal Methods: Toward Safe Control Through Proof and Learning , volume=. Proceedings of the AAAI Conference on Artificial Intelligence , publisher=. doi:10.1609/aaai.v32i1.12107 , number=

    doi

  45. [54]

    Using ontology to guide reinforcement learning agents in unseen situations: A traffic signal control system case study , volume =

    Ghanadbashi, Saeedeh and Golpayegani, Fatemeh , year =. Using ontology to guide reinforcement learning agents in unseen situations: A traffic signal control system case study , volume =. Applied Intelligence , publisher =. doi:10.1007/s10489-021-02449-5 , number =

    doi

  46. [55]

    Hausknecht and Peter Stone , editor =

    Matthew J. Hausknecht and Peter Stone , editor =. Deep Reinforcement Learning in Parameterized Action Space , booktitle =. 2016 , url =

    2016

  47. [56]

    Neuro-symbolic Action Masking for Deep Reinforcement Learning , year =

    Han, Shuai and Dastani, Mehdi and Wang, Shihan , collection =. Neuro-symbolic Action Masking for Deep Reinforcement Learning , year =. doi:10.65109/jwph6906 , booktitle =

    doi

  48. [57]

    A Lazy Approach to Neural Numerical Planning with Control Parameters , ISBN =

    Heesch, René and Cimatti, Alessandro and Ehrhardt, Jonas and Diedrich, Alexander and Niggemann, Oliver , year =. A Lazy Approach to Neural Numerical Planning with Control Parameters , ISBN =. doi:10.3233/faia241000 , booktitle =

    doi

  49. [58]

    Proceedings of the ECAI Workshop on AI-based Planning for Complex Real-World Applications (CAIPI 2025) , url=

    Learning Sound and Complete Preconditions in Complex Real-World Domains , author=. Proceedings of the ECAI Workshop on AI-based Planning for Complex Real-World Applications (CAIPI 2025) , url=. 2025 , volume=

    2025

  50. [59]

    International Conference on Learning Representations , year=

    Model Based Reinforcement Learning for Atari , author=. International Conference on Learning Representations , year=

  51. [60]

    Shield Synthesis for Reinforcement Learning , ISBN =

    K\". Shield Synthesis for Reinforcement Learning , ISBN =. 2020 , pages =. doi:10.1007/978-3-030-61362-4_16 , booktitle =

    2020 doi

  52. [61]

    Shields for Safe Reinforcement Learning , volume =

    K\". Shields for Safe Reinforcement Learning , volume =. Communications of the ACM , publisher =. 2025 , month = Oct, pages =. doi:10.1145/3715958 , number =

    2025 doi

  53. [62]

    ICAPS Workshop on Explainable AI Planning (XAIP) , year=

    Towards model-based contrastive explanations for explainable planning , author=. ICAPS Workshop on Explainable AI Planning (XAIP) , year=

  54. [63]

    arXiv preprint arXiv:2005.00904 , year=

    The ilasp system for inductive learning of answer set programs , author=. arXiv preprint arXiv:2005.00904 , year=

    2005 arXiv

  55. [64]

    doi:10.48550/ARXIV.2005.01643 , author =

    Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems , journal =. doi:10.48550/ARXIV.2005.01643 , author =

    2005 doi arXiv

  56. [65]

    and Polyak, B.T

    Levitin, E.S. and Polyak, B.T. , year =. Constrained minimization methods , volume =. USSR Computational Mathematics and Mathematical Physics , publisher =. doi:10.1016/0041-5553(66)90114-5 , number =

    doi

  57. [66]

    International Conference on Learning Representations , year =

    Boyan Li and Hongyao Tang and Yan Zheng and Jianye Hao and Pengyi Li and Zhen Wang and Zhaopeng Meng and Li Wang , title =. International Conference on Learning Representations , year =

  58. [67]

    International Conference on Learning Representations , year=

    The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervision , author=. International Conference on Learning Representations , year=

  59. [68]

    Reinforcement Learning with Parameterized Actions , volume =

    Masson, Warwick and Ranchod, Pravesh and Konidaris, George , year =. Reinforcement Learning with Parameterized Actions , volume =. Proceedings of the AAAI Conference on Artificial Intelligence , publisher =. doi:10.1609/aaai.v30i1.10226 , number =

    doi

  60. [69]

    Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems , year =

    Giulio Mazzi and Daniele Meli and Alberto Castellini and Alessandro Farinelli , title =. Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems , year =

    2023

  61. [70]

    Learning Logic Specifications for Policy Guidance in POMDPs: an Inductive Logic Programming Approach , volume =

    Meli, Daniele and Castellini, Alberto and Farinelli, Alessandro , year =. Learning Logic Specifications for Policy Guidance in POMDPs: an Inductive Logic Programming Approach , volume =. doi:10.1613/jair.1.15826 , journal =

    doi

  62. [71]

    doi:10.48550/ARXIV.2210.15670 , author =

    Knowledge-Guided Exploration in Deep Reinforcement Learning , journal =. doi:10.48550/ARXIV.2210.15670 , author =

    doi arXiv

  63. [72]

    Explainable Reinforcement Learning: A Survey and Comparative Review , volume =

    Milani, Stephanie and Topin, Nicholay and Veloso, Manuela and Fang, Fei , year =. Explainable Reinforcement Learning: A Survey and Comparative Review , volume =. ACM Computing Surveys , publisher =. doi:10.1145/3616864 , number =

    doi

  64. [73]

    and Veness, Joel and Bellemare, Marc G

    Mnih, Volodymyr and Kavukcuoglu, Koray and Silver, David and Rusu, Andrei A. and Veness, Joel and Bellemare, Marc G. and Graves, Alex and Riedmiller, Martin and Fidjeland, Andreas K. and Ostrovski, Georg and Petersen, Stig and Beattie, Charles and Sadik, Amir and Antonoglou, I...

    doi

  65. [74]

    Learning Safe Numeric Action Models , volume =

    Mordoch, Argaman and Juba, Brendan and Stern, Roni , year =. Learning Safe Numeric Action Models , volume =. Proceedings of the AAAI Conference on Artificial Intelligence , publisher =. doi:10.1609/aaai.v37i10.26424 , number =

    doi

  66. [75]

    and Broekens, Joost and Plaat, Aske and Jonker, Catholijn M

    Moerland, Thomas M. and Broekens, Joost and Plaat, Aske and Jonker, Catholijn M. , year =. Model-based Reinforcement Learning: A Survey , volume =. Foundations and Trends in Machine Learning , publisher =. doi:10.1561/2200000086 , number =

    doi

  67. [76]

    Mastering the game of Go without human knowledge , volume =

    Silver, David and Schrittwieser, Julian and Simonyan, Karen and Antonoglou, Ioannis and Huang, Aja and Guez, Arthur and Hubert, Thomas and Baker, Lucas and Lai, Matthew and Bolton, Adrian and Chen, Yutian and Lillicrap, Timothy and Hui, Fan and Sifre, Laurent and van den Dries...

    doi

  68. [77]

    A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play , volume =

    Silver, David and Hubert, Thomas and Schrittwieser, Julian and Antonoglou, Ioannis and Lai, Matthew and Guez, Arthur and Lanctot, Marc and Sifre, Laurent and Kumaran, Dharshan and Graepel, Thore and Lillicrap, Timothy and Simonyan, Karen and Hassabis, Demis , year =. A general...

    doi

  69. [78]

    2018 , publisher=

    Reinforcement learning: An introduction , author=. 2018 , publisher=

    2018

  70. [79]

    doi:10.48550/ARXIV.2601.02850 , author =

    Sample-Efficient Neurosymbolic Deep Reinforcement Learning , publisher =. doi:10.48550/ARXIV.2601.02850 , author =

    doi arXiv

  71. [80]

    35th International Conference on Principles of Diagnosis and Resilient Systems (DX 2024) , pages =

    Vranje. 35th International Conference on Principles of Diagnosis and Resilient Systems (DX 2024) , pages =. 2024 , volume =. doi:10.4230/OASIcs.DX.2024.7 , annote =

    2024 doi

  72. [81]

    Scalable Planning with Tensorflow for Hybrid Nonlinear Domains , volume =

    Wu, Ga and Say, Buser and Sanner, Scott , booktitle =. Scalable Planning with Tensorflow for Hybrid Nonlinear Domains , volume =

  73. [82]

    2018 , copyright =

    Parametrized Deep Q-Networks Learning: Reinforcement Learning with Discrete-Continuous Hybrid Action Space , publisher =. 2018 , copyright =. doi:10.48550/ARXIV.1810.06394 , author =

    2018 doi arXiv

  74. [83]

    Towards Sample Efficient Reinforcement Learning , url =

    Yu, Yang , year =. Towards Sample Efficient Reinforcement Learning , url =. doi:10.24963/ijcai.2018/820 , booktitle =

    2018 doi

  75. [84]

    An Overview of the Action Space for Deep Reinforcement Learning , DOI =

    Zhu, Jie and Wu, Fengge and Zhao, Junsuo , year =. An Overview of the Action Space for Deep Reinforcement Learning , DOI =. 2021 4th International Conference on Algorithms, Computing and Artificial Intelligence , publisher =

    2021

  76. [85]

    Model-lite planning: Case-based vs

    Zhuo, Hankz Hankui and Kambhampati, Subbarao , year =. Model-lite planning: Case-based vs. model-based approaches , volume =. doi:10.1016/j.artint.2017.01.004 , journal =

    2017 doi

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.