Pith. sign in

REVIEW 3 major objections 5 minor 47 references

Neural DNF-MT: A Neuro-symbolic Approach for Learning Interpretable and Editable Policies

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Trained neural DNF-MT actors translate directly into editable ProbLog or ASP policies, with a bidirectional neural-logic mapping.

desk verdict A genuine increment in neuro-symbolic policy learning whose main extraction claim is only proven after a thresholding step that the authors themselves show can break mutual exclusivity. read the letter →

arxiv 2501.03888 v4 pith:UXYTK7QN submitted 2025-01-07 cs.AI cs.LGcs.LO

Kexin Gu Baugh , Luke Dickens , Alessandra Russo This is my paper · ORCID
classification cs.AIcs.LGcs.LO MSC 68T0768T2768T05
keywords neuro-symboliclearningreinforcementinterpretablepoliciesprobabilisticlogicprogramsanswersetprogrammingdisjunctivenormalformmutex-tanhactivationpolicydistillation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Neural DNF-MT is a fully differentiable neural architecture for reinforcement learning that is designed so a trained actor can be read out as a standard logic program: ProbLog annotated disjunctions for stochastic policies and ASP rules for deterministic policies. The paper's central claim is that this gives the performance of a black-box neural actor together with an interpretable, manually editable policy, and that for bivalent logic the translation is bidirectional, so an edited ASP program can be written back into the neural model and executed with equivalent truth values. The architecture uses a disjunctive-normal-form network whose final activation, mutex-tanh, maps outputs to probabilities that sum to one, and auxiliary losses push activations and weights toward values that make logical extraction faithful. Evaluations on corridor tasks, Blackjack, Taxi, and Door Corridor show extracted ASP policies matching the neural actors in deterministic settings, while ProbLog extraction in Blackjack and Taxi loses some performance. The paper isolates the post-training thresholding step, which snaps continuous weights to $\{-6, 0, 6\}$, as the cause, and gives a concrete Door Corridor counterexample where thresholding flips a disjunctive output and creates two simultaneously true actions.

What carries the argument

The load-bearing object is the mutex-tanh activation, defined as $\mathrm{mutex\text{-}tanh}(\mathbf{d})_k = 2\,\mathrm{softmax}(\mathbf{d})_k - 1$, applied to the disjunctive layer of a neural DNF model. It maps any real pre-activation vector to $(-1,1)^N$ in a way that makes the interpreted probabilities $(\tilde{y}_i+1)/2$ sum to $1$, satisfying probabilistic mutual exclusivity; with auxiliary losses that drive conjunctive activations and disjunctive weights toward $\pm1$ and $\pm6$, the same network can also satisfy logical mutual exclusivity after thresholding. The conjunctive layer remains a tanh-activated semi-symbolic layer whose bias is computed from the max-absolute weight, and the whole actor can be preceded by a trainable encoder that invents predicate-like features from complex observations. This combination is what lets one network serve both as a differentiable policy and as a source of extractable ProbLog or ASP rules.

What would settle it

Evaluate the Door Corridor actor from Listing 6 on the input $x_2=-1, x_7=1, x_9=1, x_{13}=-1$: before thresholding only action 1 is true, whereas after thresholding both action 1 and action 3 are true, which directly violates logical mutual exclusivity. More generally, one can scan any trained actor for a state where thresholding changes the sign of any disjunctive node or makes two disjunctive nodes positive; finding a single such state is enough to show the extraction pipeline is not faithful for that actor without additional correction.

Watch

Extended reading notes

Core claim

The authors claim that a neural DNF-MT actor, a semi-symbolic network with a conjunctive layer followed by a disjunctive layer, can be trained end-to-end with actor-critic PPO (or distilled from an MLP actor) and then translated, without retraining, into a close logical approximation of the learned policy. For stochastic policies the translation is a ProbLog program whose annotated disjunctions carry the action probabilities, with the mutex-tanh activation guaranteeing that the probabilities sum to one. For deterministic policies the translation is an ASP program whose rules are read off after weights are thresholded to $\{-6, 0, 6\}$ and activations are replaced by step functions; the paper proves in Appendix A that with such weights the truth value of the ASP program equals the bivalent interpretation of the neural activations, in both directions. This bidirectional property is what makes policy intervention possible: a user can change the ASP rules to adapt to a new termination condition, port the edited program back into a neural actor, and run it without further training.

Load-bearing premise

The load-bearing premise is that snapping the trained continuous weights to the three values $-6$, $0$, $6$ preserves the sign of every node output (and hence the chosen action and the exclusivity of the policy); the paper's own Listing 6 shows a Door Corridor actor where thresholding flips a disjunctive output from negative to positive, making two actions true at once.

Editorial extensions

If this is right

  • A trained neural DNF-MT actor yields an inspectable policy: the extracted ASP rules for deterministic environments are short human-readable clauses, e.g., `action(left) :- in_s_1.`, and ProbLog rules carry explicit action probabilities.
  • The bidirectional neural-to-bivalent translation means manual policy intervention does not require retraining: editing the ASP program and porting it back produced optimal behaviour in two Door Corridor variants where an MLP actor failed.
  • Inference stays fast because execution happens in the neural actor; running the extracted ProbLog program is orders of magnitude slower, so the logic program serves as the interpretable specification rather than the runtime engine.
  • Because the model trains with a standard MLP critic and PPO, and supports end-to-end predicate invention, it avoids rule templates or mode declarations used by several prior neuro-symbolic approaches.
  • The extracted logic policy is only a close approximation, not a guaranteed copy, of the neural policy; in Blackjack and Taxi the ProbLog extraction degraded performance, which the paper attributes to thresholding.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • I infer that if thresholding were replaced by a sign-preserving discretisation, for instance using the min-based bias derived in Appendix F or per-node scaling, the extraction failure described in Listing 6 would be avoidable and the bidirectional edit loop would be reliable for any trained actor; the paper leaves this as future work.
  • I infer that the same softmax-derived exactly-one activation could be applied to other neural classifiers that need both probabilistic calibration and logical mutual exclusivity, not just RL policies.
  • I infer that the demonstrated policy intervention suggests a practical workflow for environment-rule changes: patch a few ASP clauses instead of fine-tuning a network, then port the edited program back for parallel execution.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes neural DNF-MT, a differentiable neuro-symbolic architecture for reinforcement learning policies, built on pix2rule's semi-symbolic DNF layers but adding a mutex-tanh activation that makes the action distribution mutually exclusive by construction. The authors claim that trained neural DNF-MT actors can be directly translated into ProbLog programs for stochastic policies and ASP programs for deterministic policies, that this translation is bidirectional, and that edited ASP policies can be ported back into the neural model. They evaluate on Switcheroo Corridor, Blackjack, Taxi, and Door Corridor environments, reporting performance at the level of MLP baselines while providing interpretable logic programs, and they demonstrate policy intervention on two Door Corridor variants. The paper includes a formal equivalence proof in Appendix A for the neural-to-logic and logic-to-neural translation for thresholded weights in {-6,0,6}.

Significance. If the central extraction claim held, this would be a valuable contribution: it offers a principled way to obtain standard ProbLog/ASP policies from differentiable RL actors, with a bidirectional translation that supports manual policy editing. The mutex-tanh activation is a clean mechanism for enforcing probabilistic mutual exclusivity, and the proof in Appendix A is a genuine formalization for the thresholded case. The policy intervention demonstration (DC-T, DC-OT) is a compelling proof of concept. However, the significance is currently conditional because the post-training thresholding step—the bridge between the trained continuous model and the provably translatable discrete model—is shown by the authors' own experiments to fail on a concrete instance (Listing 6), and the reported experimental results omit failed runs. The paper is honest about this limitation, but the central 'direct translation' contribution (Contribution 2) is not yet supported at the level claimed.

major comments (3)
  1. [Section 3.3, Step 2; Appendix A; Section 5, Listing 6] The core extraction claim rests on an unproven thresholding step. Propositions A.1 and A.2 prove truth-value equivalence only under Condition (10), w ∈ {-6,0,6}^I, i.e., after thresholding; they do not establish that thresholding preserves the policy. Section 5 and Listing 6 give a concrete counterexample: for x2=-1, x7=1, x9=1, x13=-1, the pre-threshold disjunctive output is y3=-0.86 (only action 1 true) while after thresholding y3=1.00, producing two true actions and violating P2. This directly contradicts the claim that a trained actor 'can be directly translated' into a close approximation of the learned policy. The manuscript needs either a guarantee or a probabilistic tolerance bound for thresholding, or a substantial reframing of the contribution as a heuristic extraction with failure cases.
  2. [Section 4.4 and Table 6] The reporting of Door Corridor results is inconsistent and the headline parity result omits failed runs. Section 4.4 states that out of 32 runs, 6 cannot finish training and 1 of the remaining 26 fails to maintain mutual exclusivity after thresholding, leaving 25 interpretable runs. Yet Table 6 reports 16 runs per model for Door Corridor and lists Neural DNF-MT* as -8.000 ± 0.000, which appears to average only the successful 25 runs. The discrepancy between 32 and 16 runs needs clarification, and the abstract's claim of performing 'at the level of competing black-box methods' must be reported with the success rate (25/32) and the two distinct failure modes explicitly shown, rather than as an unconditional parity result.
  3. [Appendix F and Eq. (1)] Appendix F's analysis shows why thresholding is not semantics-preserving: the max-based bias in Eq. (1) encodes input importance, and thresholding to {−6,0,6} discards that weighting. The example with weights [3,1,1] and inputs [1,-1,1] yields a positive node output under the max bias, although the intended conjunction is false; this means the trained continuous model does not necessarily respect the DNF semantics that the extracted rules assume. This is not a minor technicality but a fundamental mismatch between the training objective (which uses importance-weighted inputs) and the post-training interpretation (which treats all weights as ±6). The paper should either adopt the min-based bias (or an alternative) to align training with the extraction semantics, or explicitly characterize which trained models are safe to extract.
minor comments (5)
  1. [Abstract/Keywords] The keywords line contains a typo: 'Neuro-symobilc' should be 'Neuro-symbolic'.
  2. [Table 6 caption and Section 4.4] The number of runs for Door Corridor is inconsistent (16 in Table 6 caption versus 32 in Section 4.4). Please reconcile these numbers and state the run counts for each environment precisely.
  3. [Listing 6] Listing 6 leaves bias terms uncalculated, which makes the thresholding counterexample difficult to verify. Please include the full learned weights and biases for that run, or provide a link to the exact trained model checkpoint used in the example.
  4. [Definition 3.2] The logical formula for logical mutual exclusivity uses the symbols 'Ü' and 'Û' which appear to be rendering artifacts for OR (∨) and AND (∧). Please fix the notation.
  5. [Appendix E] The ProbLog run-time comparison for Taxi is based on a synthetic program format rather than on actual extracted programs; the authors acknowledge this, but the conclusion would be stronger with timings for the real extracted ProbLog programs from Section 4.3, even if approximate.

Circularity Check

1 steps flagged · score 2.0 of 10

Minor circularity: the ProbLog evaluation is proxied by the same neural actor from which the ProbLog rules were read off; the thresholding failure is a correctness gap rather than a circular step.

  1. fitted input called prediction [Section 4, Experiments, first paragraph]
    "We do not directly evaluate the extracted ProbLog policies because of the long ProbLog query time. Instead, we evaluate their final neural DNF-MT actors before logical rule extraction (i.e. after step 3, re-pruning) as an approximation. The approximation is acceptable because a ProbLog policy’s action distribution is the same as its corresponding neural DNF-MT’s action distribution to 3 decimal places."

    The ProbLog rules are constructed in post-training step (4.a) by reading probabilities and truth values off the very same neural DNF-MT actor: Eq (3) sets p_i = (y_i+1)/2 from the mutex-tanh output, and the rule bodies are the unique conjunctive activations of that actor. Therefore the asserted 'same action distribution to 3 decimal places' holds by construction over the enumerated activations, not by an independent evaluation of the extracted program. Using the source neural actor as a proxy for the extracted ProbLog policy assumes the fidelity that the post-training pipeline is supposed to establish. The circularity is partial because the paper also directly evaluates some ProbLog policies (e.g.

full rationale

The paper's central derivation is largely self-contained and not circular: the neural-to-ASP translation is a compiler-style correctness theorem (Appendix A) that shows, under the post-threshold conditions w in {-6,0,6} and x in {-1,1}, the ASP rule and the thresholded node agree by construction. That is a legitimate verification rather than a prediction from fitted inputs. The main weakness of the paper — thresholding to {-6,0,6} can flip the sign of a disjunctive output and break logical mutual exclusivity (Section 5, Listing 6, Table 3) — is a correctness and robustness gap, not a circularity, because the paper acknowledges the failure and does not use the equivalence proof to justify the thresholding step. Self-citations to pix2rule [7] and neural DNF-EO [3] are heavy, but they supply the base semi-symbolic architecture and do not function as an unverified uniqueness theorem; the novel mutex-tanh activation and the editing loop are evaluated independently. The one genuine circular moment is the use of the pre-extraction neural actor as a stand-in for the extracted ProbLog policy, justified by an equality that is definitional rather than empirical. Hence the overall score is 2.

Assumptions & free parameters 5 free parameters · 6 assumptions · 1 invented entities

The central extraction pipeline rests on a set of hand-set thresholds and loss coefficients, plus the assumption that thresholding preserves logical behaviour, which the paper itself shows can fail. The mathematical core (mutex-tanh normalization and translation equivalence on {-6,0,6} weights) is self-contained.

free parameters (5)
  • Thresholding parameter tau = not reported; chosen per run
    Post-training step 2 (Section 3.3) selects tau to maintain trajectory or action probabilities, making extraction fidelity dependent on a data-dependent choice.
  • Pruning threshold tau_prune = 1e-3
    Appendix B.2 sets this for edge removal checks; it determines how aggressively rules are simplified before extraction.
  • Auxiliary loss coefficients lambda_i = varied per environment, e.g. 0, 1e-6, 1e-5, 1e-3, 3e-15
    Appendix B and D list values tuned per environment; they shape whether weights approach +/-6 so thresholding can succeed.
  • Delta scheduling hyperparameters = e.g. initial_delta=0.1, delta_decay_delay=30-1000, delta_decay_steps=5-100, delta_decay_rate=1.1
    Appendix B controls how fast the logical bias strengthens; the schedule affects whether interpretable rules form.
  • Number of conjunctive nodes C = 4 (SC/LC), 64 (Blackjack/Taxi), 12 (Door Corridor)
    Architecture capacity for rule search, chosen per environment; it bounds the size and complexity of extracted programs.
assumptions (6)
  • domain assumption Semi-symbolic node semantics and the max-version bias 𝛽 = max_i |w_i| - sum |w_i| from pix2rule are adopted without revalidation.
    Used in Section 2.2 and throughout; Appendix F shows the max bias can make a conjunction whose input is false still fire, which is the root of the thresholding failure.
  • domain assumption Observations can be binarized to values in {-1,1} without losing task-relevant information.
    Used in Section 3.3 and Figure 1; for complex observations an encoder is added, and its outputs are discretized by sign in post-processing, which may discard information.
  • ad hoc to paper There exists a single threshold tau such that snapping weights to {-6,0,6} preserves the trajectory or action probabilities within tolerance.
    Post-training step 2 in Section 3.3; the paper's Listing 6 and Table 3 provide a concrete counterexample, so this axiom is load-bearing and not universally true.
  • ad hoc to paper Auxiliary loss L^(4) drives the model to satisfy logical mutual exclusivity P2 for deterministic policies.
    Equation (8) and Section 3.2; no proof is given, and the Door Corridor failing run shows P2 can be violated after thresholding.
  • domain assumption PPO with MLP critic converges to a near-optimal policy for the tested environments when hyperparameters are tuned.
    Standard RL assumption invoked in Section 3.3; Taxi training actually failed to find working hyperparameters for neural DNF-MT, requiring distillation instead.
  • standard math ASP and ProbLog semantics are well-defined and can be used to evaluate extracted policies.
    Used in Section 3.3 and Appendix E; ProbLog inference is slow, so extracted stochastic policies were not directly evaluated, only their neural equivalents.
invented entities (1)
  • Mutex-tanh activation function
    purpose: Map disjunctive layer logits to a vector in (-1,1) whose linear rescaling is a probability distribution, satisfying probabilistic mutual exclusivity.
    The function is defined and used internally; its only support is mathematical identity and experimental performance, with no externally falsifiable prediction.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Neural DNF-MT: A Neuro-symbolic Approach for Learning Interpretable and Editable Policies." pith.science (2026). https://pith.science/paper/UXYTK7QN

@misc{pith2026250103888,
  author       = {Pith},
  title        = {Pith review of: Neural DNF-MT: A Neuro-symbolic Approach for Learning Interpretable and Editable Policies},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UXYTK7QN}},
  note         = {Machine review of arXiv:2501.03888}
}
read the original abstract

Although deep reinforcement learning has been shown to be effective, the model's black-box nature presents barriers to direct policy interpretation. To address this problem, we propose a neuro-symbolic approach called neural DNF-MT for end-to-end policy learning. The differentiable nature of the neural DNF-MT model enables the use of deep actor-critic algorithms for training. At the same time, its architecture is designed so that trained models can be directly translated into interpretable policies expressed as standard (bivalent or probabilistic) logic programs. Moreover, additional layers can be included to extract abstract features from complex observations, acting as a form of predicate invention. The logic representations are highly interpretable, and we show how the bivalent representations of deterministic policies can be edited and incorporated back into a neural model, facilitating manual intervention and adaptation of learned policies. We evaluate our approach on a range of tasks requiring learning deterministic or stochastic behaviours from various forms of observations. Our empirical results show that our neural DNF-MT model performs at the level of competing black-box methods whilst providing interpretable policies.

Figures

Figures reproduced from arXiv: 2501.03888 by the authors.

Figure 2
Figure 2. Neural DNF-MT model as an actor in actor-critic [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Post-training processing to extract an interpretable logical policy from a trained neural DNF-MT actor. There are two [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 5
Figure 5. Small corridor (SC), same as the one from [33]. The start, goal, and spe￾cial states are customis￾able but fixed once created throughout training and in￾ference. We create three corridors based on differ￾ent configurations: Small Corridor (SC) as shown in [PITH_FULL_IMAGE:figures/full_fig_p005_5.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Mean episodic return (y-axis) ± standard error of the baselines and neural DNF-MT models, together with the ProbLog/ASP programs extracted from their corresponding neural DNF-MT models. All Q-tables are trained using Q-learning, and all MLP actors are trained with acto…
Figure 6
Figure 6. Figure 6: Door Corridor (DC): the agent needs to turn right first, and tog￾gle and go through three doors to reach the end of the corridor. To evaluate an extracted ASP program in the environment, we first pass the 3 × 3 observation to the encoder, convert invented predicates wi…
Figure 8
Figure 8. Figure 8: Long Corridor-11 environment, created according [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 7
Figure 7. Figure 7: Long Corridor-5 environment, created according to [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 9
Figure 9. Figure 9: Taxi Environment: the taxi needs to pick up a pas [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]
Figure 10
Figure 10. Figure 10: Policy grid of a neural DNF-MT actor in the Black [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]
Figure 11
Figure 11. Figure 11: Extracted ProbLog policy grid of the same neural [PITH_FULL_IMAGE:figures/full_fig_p018_11.png]
Figure 12
Figure 12. Figure 12: Time taken vs the number of annotated disjunc [PITH_FULL_IMAGE:figures/full_fig_p021_12.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 23 canonical work pages

  1. [1]

    EU AI Act

    2024. EU AI Act. https://artificialintelligenceact.eu/article/13/

    work page 2024

  2. [2]

    Jason Ansel, Edward Yang, Horace He, Natalia Gimelshein, Animesh Jain, Michael Voznesensky, Bin Bao, Peter Bell, David Berard, Evgeni Burovski, Geeta Chauhan, Anjali Chourdia, Will Constable, Alban Desmaison, Zachary DeVito, Elias Ellison, Will Feng, Jiong Gong, Michael Gschwind, Brian Hirsh, Sherlock Huang, Kshiteej Kalambarkar, Laurent Kirsch, Michael L...

    arXiv 2024

  3. [3]

    Kexin Gu Baugh, Nuri Cingillioglu, and Alessandra Russo. 2023. Neuro-symbolic Rule Learning in Real-world Classification Tasks. InProceedings of the AAAI 2023 Spring Symposium on Challenges Requiring the Combination of Machine Learning and Knowledge Engineering (AAAI-MAKE 2023) , Andreas Martin, Hans-Georg Fill, Aurona Gerber, Knut Hinkelmann, Doug Lenat,...

    work page 2023

  4. [4]

    Christopher Berner, Greg Brockman, Brooke Chan, Vicki Cheung, Przemys- law Debiak, Christy Dennison, David Farhi, Quirin Fischer, Shariq Hashme, Christopher Hesse, Rafal Józefowicz, Scott Gray, Catherine Olsson, Jakub Pa- chocki, Michael Petrov, Henrique Pondé de Oliveira Pinto, Jonathan Raiman, Tim Salimans, Jeremy Schlatter, Jonas Schneider, Szymon Sido...

    arXiv 2019

  5. [5]

    Yushi Cao, Zhiming Li, Tianpei Yang, Hao Zhang, Yan Zheng, Yi Li, Jianye Hao, and Yang Liu. 2024. GALOIS: boosting deep reinforcement learning via generalizable logic synthesis. In Proceedings of the 36th International Conference on Neural Information Processing Systems (New Orleans, LA, USA) (NIPS ’22). Curran Associates Inc., Red Hook, NY, USA, Article ...

    work page 2024

  6. [6]

    Maxime Chevalier-Boisvert, Bolun Dai, Mark Towers, Rodrigo de Lazcano, Lu- cas Willems, Salem Lahlou, Suman Pal, Pablo Samuel Castro, and Jordan Terry

  7. [7]

    Nuri Cingillioglu and Alessandra Russo. 2021. pix2rule: End-to-end Neuro- symbolic Rule Learning. In Proceedings of the 15th International Workshop on Neural-Symbolic Learning and Reasoning (NeSy 2021) as part of the 1st International Joint Conference on Learning & Reasoning (IJCLR 2021) , Artur d’Avila Garcez and Ernesto Jiménez-Ruiz (Eds.). CEUR Worksho...

    work page 2021

  8. [8]

    Muggle- ton

    Andrew Cropper, Sebastijan Dumančić, Richard Evans, and Stephen H. Muggle- ton. 2022. Inductive logic programming at 30. Machine Learning 111, 1 (01 Jan 2022), 147–172. https://doi.org/10.1007/s10994-021-06089-1

    doi:10.1007/s10994-021-06089-1 2022

Show all 47 references
  1. [9]

    Luc De Raedt, Angelika Kimmig, and Hannu Toivonen. 2007. ProbLog: a prob- abilistic prolog and its application in link discovery. In Proceedings of the 20th International Joint Conference on Artifical Intelligence (Hyderabad, India) (IJ- CAI’07). Morgan Kaufmann Publishers Inc...

    2007

  2. [10]

    Quentin Delfosse, Hikaru Shindo, Devendra Dhami, and Kristian Kersting. 2023. Interpretable and Explainable Logical Policies via Neurally Guided Symbolic Abstraction. In Advances in Neural Information Processing Systems , A. Oh, T. Nau- mann, A. Globerson, K. Saenko, M. Hardt,...

    2023

  3. [11]

    Thomas G Dietterich. 2000. Hierarchical reinforcement learning with the MAXQ value function decomposition. Journal of artificial intelligence research 13 (2000), 227–303

    2000

  4. [12]

    Honghua Dong, Jiayuan Mao, Tian Lin, Chong Wang, Lihong Li, and Denny Zhou. 2019. Neural Logic Machines. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019 . OpenReview.net. https://openreview.net/forum?id=B1xY-hRctX

    2019

  5. [13]

    Sašo Džeroski, Luc De Raedt, and Kurt Driessens. 2001. Relational Reinforcement Learning. Machine Learning 43, 1 (01 Apr 2001), 7–52. https://doi.org/10.1023/A: 1007694015589

    2001 doi

  6. [14]

    Richard Evans and Edward Grefenstette. 2018. Learning explanatory rules from noisy data. Journal of Artificial Intelligence Research 61 (2018), 1–64

    2018

  7. [15]

    Xiaojie Gao, Yueming Jin, Qi Dou, and Pheng-Ann Heng. 2020. Automatic Gesture Recognition in Robot-assisted Surgery with Reinforcement Learning and Tree Search. In 2020 IEEE International Conference on Robotics and Automation (ICRA) . 8440–8446. https://doi.org/10.1109/ICRA409...

    2020

  8. [16]

    Baxter, Jie Xu, Jiming Xu, Xingtao Zhou, and Kang Zhang

    Jianxing He, Sally L. Baxter, Jie Xu, Jiming Xu, Xingtao Zhou, and Kang Zhang

  9. [17]

    Shengyi Huang, Rousslan Fernand Julien Dossa, Chang Ye, Jeff Braga, Dipam Chakraborty, Kinal Mehta, and João G.M. Araújo. 2022. CleanRL: High-quality Single-file Implementations of Deep Reinforcement Learning Algorithms.Journal of Machine Learning Research 23, 274 (2022), 1–18...

    2022

  10. [18]

    Zhengyao Jiang and Shan Luo. 2019. Neural Logic Reinforcement Learning. In Proceedings of the 36th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 97) , Kamalika Chaudhuri and Ruslan Salakhut- dinov (Eds.). PMLR, 3110–3119. https://...

    2019

  11. [19]

    Littman, and Anthony R

    Leslie Pack Kaelbling, Michael L. Littman, and Anthony R. Cassandra. 1998. Plan- ning and acting in partially observable stochastic domains. Artificial Intelligence 101, 1 (1998), 99–134. https://doi.org/10.1016/S0004-3702(98)00023-X

    1998 doi

  12. [20]

    Daiki Kimura, Masaki Ono, Subhajit Chaudhury, Ryosuke Kohita, Akifumi Wachi, Don Joven Agravante, Michiaki Tatsubori, Asim Munawar, and Alexander Gray

  13. [21]

    Vladimir Lifschitz. 2019. Answer set programming . Springer Nature, Cham, Switzerland. https://doi.org/10.1007/978-3-030-24658-7

    2019 doi

  14. [22]

    Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Tim- othy Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. 2016. Asyn- chronous Methods for Deep Reinforcement Learning. In Proceedings of The 33rd International Conference on Machine Learning (Proc...

    2016

  15. [23]

    Rusu, Joel Veness, Marc G

    Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, Alex Graves, Martin Riedmiller, Andreas K. Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, ...

    2015 doi

  16. [24]

    Stephen Muggleton. 1995. Inverse entailment and progol. New Generation Computing 13, 3 (01 Dec 1995), 245–286. https://doi.org/10.1007/BF03037227

    1995 doi

  17. [25]

    Stephen Muggleton and Luc de Raedt. 1994. Inductive Logic Programming: Theory and methods. The Journal of Logic Programming 19-20 (1994), 629–679. https://doi.org/10.1016/0743-1066(94)90035-3 Special Issue: Ten Years of Logic Programming

    1994 doi

  18. [26]

    Potassco, the Potsdam Answer Set Solving Collection. 2022. Clingo: A grounder and solver for logic programs. University of Potsdam. https://github.com/potassco/ clingo

    2022

  19. [27]

    Puterman

    Martin L. Puterman. 1990. Markov decision processes. In Stochastic Models . Handbooks in Operations Research and Management Science, Vol. 2. Elsevier, 331–434. https://doi.org/10.1016/S0927-0507(05)80172-0

    1990 doi

  20. [28]

    Gray, Francois P

    Ryan Riegel, Alexander G. Gray, Francois P. S. Luus, Naweed Khan, Ndivhuwo Makondo, Ismail Yunus Akhalwaya, Haifeng Qian, Ronald Fagin, Francisco Bara- hona, Udit Sharma, Shajith Ikbal, Hima Karanam, Sumit Neelam, Ankita Likhyani, and Santosh K. Srivastava. 2020. Logical Neura...

    2020 arXiv

  21. [29]

    Cynthia Rudin. 2019. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead.Nature Machine Intelligence 1, 5 (01 May 2019), 206–215. https://doi.org/10.1038/s42256-019-0048-x

    2019 doi

  22. [30]

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov

  23. [31]

    Hikaru Shindo, Masaaki Nishino, and Akihiro Yamamoto. 2021. Differentiable inductive logic programming for structured examples. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 5034–5041

    2021

  24. [32]

    David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, Yutian Chen, Timothy Lillicrap, Fan Hui, Laurent Sifre, George van den Driessche, Thore Graepel, and Demis Hassabis. 2017. Ma...

    2017

  25. [33]

    Sutton and Andrew G

    Richard S. Sutton and Andrew G. Barto. 2018. Reinforcement Learning: An Intro- duction. A Bradford Book, Cambridge, MA, USA

    2018

  26. [34]

    Mark Towers, Ariel Kwiatkowski, Jordan K Terry, John U. Balis, Gianluca de Cola, Tristan Deleu, Manuel Goulão, Andreas Kallinteris, Markus Krimmel, Arjun KG, Rodrigo Perez-Vicente, Andrea Pierré, Sander Schulhoff, Jun Jet Tai, Hannah Jin Shen Tan, and Omar G. Younis. 2024. Gym...

    2024

  27. [35]

    Le, Yisong Yue, and Swarat Chaudhuri

    Abhinav Verma, Hoang M. Le, Yisong Yue, and Swarat Chaudhuri. 2019. Imitation- projected programmatic reinforcement learning. In Proceedings of the 33rd Inter- national Conference on Neural Information Processing Systems . Curran Associates Inc., Red Hook, NY, USA, Article 141...

    2019

  28. [36]

    Abhinav Verma, Vijayaraghavan Murali, Rishabh Singh, Pushmeet Kohli, and Swarat Chaudhuri. 2018. Programmatically Interpretable Reinforcement Learning. In Proceedings of the 35th International Conference on Machine Learning (Proceed- ings of Machine Learning Research, Vol. 80)...

    2018

  29. [37]

    Christopher J. C. H. Watkins and Peter Dayan. 1992. Q-learning. Machine Learning 8, 3 (01 May 1992), 279–292. https://doi.org/10.1007/BF00992698

    1992 doi

  30. [38]

    Williams

    Ronald J. Williams. 1992. Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning. Mach. Learn. 8, 3–4 (May 1992), 229–256. https://doi.org/10.1007/BF00992696

    1992 doi

  31. [39]

    XiaoDan Wu, RuiChang Li, Zhen He, TianZhi Yu, and ChangQing Cheng. 2023. A value-based deep reinforcement learning model with human expertise in optimal treatment of sepsis. npj Digital Medicine 6, 1 (02 Feb 2023), 15. https: //doi.org/10.1038/s41746-023-00755-5

    2023 doi

  32. [40]

    Reichert, Timothy P

    Vinícius Flores Zambaldi, David Raposo, Adam Santoro, Victor Bapst, Yujia Li, Igor Babuschkin, Karl Tuyls, David P. Reichert, Timothy P. Lillicrap, Edward Lockhart, Murray Shanahan, Victoria Langston, Razvan Pascanu, Matthew M. Botvinick, Oriol Vinyals, and Peter W. Battaglia....

    2018 arXiv

  33. [41]

    Matthieu Zimmer, Xuening Feng, Claire Glanois, Zhaohui Jiang, Jianyi Zhang, Paul Weng, Jianye Hao, Dong Li, and Wulong Liu. 2021. Differentiable Logic Machines. CoRR abs/2102.11529 (2021). arXiv:2102.11529 https://arxiv.org/abs/ 2102.11529

    2021 arXiv

  34. [42]

    K.J Åström. 1965. Optimal control of Markov processes with incomplete state information. J. Math. Anal. Appl. 10, 1 (1965), 174–205. https://doi.org/10.1016/ 0022-247X(65)90154-X A NEURAL-BIV ALENT-LOGIC TRANSLATION This section focuses on proving that the neural-bivalent-logi...

    1965

  35. [47]

    We modify the pruning and rule extraction stages to be better fitted for policy learning

    and [3]. We modify the pruning and rule extraction stages to be better fitted for policy learning. We provide additional information for some stages in the post-training processing below. (1) Pruning: In experiments, we pass over conj.-to-disj. edges (weights) first before the...

  36. [2017]

    CoRR abs/1707.06347 (2017)

    Proximal Policy Optimization Algorithms. CoRR abs/1707.06347 (2017). arXiv:1707.06347 http://arxiv.org/abs/1707.06347

    2017 arXiv

  37. [2019]

    Nature Medicine 25, 1 (01 Jan 2019), 30–36

    The practical implementation of artificial intelligence technologies in medicine. Nature Medicine 25, 1 (01 Jan 2019), 30–36. https://doi.org/10.1038/ s41591-018-0307-0

    2019

  38. [2021]

    InProceed- ings of the 2021 Conference on Empirical Methods in Natural Language Processing , Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (Eds.)

    Neuro-Symbolic Reinforcement Learning with First-Order Logic. InProceed- ings of the 2021 Conference on Empirical Methods in Natural Language Processing , Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (Eds.). Association for Computational Linguistic...

    2021 doi

  39. [2023]

    CoRR abs/2306.13831 (2023)

    Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Tasks. CoRR abs/2306.13831 (2023)

    2023 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.