Pith. sign in

REVIEW 4 major objections 5 minor 52 references

Co-Activation Graph Analysis of Safety-Verified and Explainable Deep Reinforcement Learning Policies

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that co-activation graph analysis, with datasets labeled by model checking, identifies the neurons and features that determine whether a deep RL policy satisfies safety properties.

desk verdict A plausible first step toward semi-global safety explanations for RL, but small state sets and absent stability checks leave the strong claims unbacked. read the letter →

arxiv 2501.03142 v1 pith:ZYWMUUN2 submitted 2025-01-06 cs.AI cs.LG

Dennis Gross , Helge Spieker This is my paper · ORCID
classification cs.AIcs.LG
keywords ExplainableReinforcementLearningModelCheckingCo-activationGraphAnalysisSafetyVerificationPageRankLouvainCommunityDetectionDeepQ-LearningSemi-GlobalExplanations
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes a framework that combines RL policy model checking with co-activation graph analysis to explain why a trained deep reinforcement learning policy is safe. Instead of explaining one state at a time or the whole policy at once, the method produces semi-global explanations: it builds a graph of which neurons tend to fire together over the states that satisfy a user-specified safety property, then ranks neurons and state features and detects functional neuron communities. The authors demonstrate on taxi and cleaning-robot environments that the top-ranked features match what actually breaks safety, since pruning those feature connections causes the verified safety probability to collapse. If the framework works as claimed, it gives safety engineers a way to turn formal verification results into interpretable maps of the network.

What carries the argument

The central object is the co-activation graph, an undirected graph whose nodes are neurons and whose edge weights are the Pearson correlations between the neurons' activations over a labeled set of inputs. Labels come from the new step: an RL policy and an environment MDP are turned into a fully deterministic induced DTMC by querying the policy's action in each reachable state, a probabilistic model checker verifies a PCTL safety property, and the states relevant to that property become the dataset. PageRank on the weighted graph ranks neuron importance, community detection finds functional modules, and comparing graphs built under different labels (different safety properties, or critical versus non-critical states) exposes which neurons and features carry each property. This carries the argument because the ranked neurons and features are then checked by pruning their connections and re-verifying the property.

What would settle it

Recompute the PageRank neuron rankings and community structures after deleting one state at a time from the 12-state one-job dataset, or after bootstrap resampling; if the top-neuron set or the claimed top features change materially, the conclusion that the method identifies safety-relevant neurons does not hold.

Watch

Extended reading notes

Core claim

The central claim is that co-activation graph analysis, when fed datasets labeled by model checking, identifies the neurons and state features that determine whether an RL policy satisfies safety properties. On the taxi domain, the most important feature neurons for completing two jobs are the passenger's destination and the number of jobs completed, and pruning those connections drops the two-job reachability probability to 0.25; the most important features for completing one job are fuel level, destination x-coordinate, and jobs completed, and pruning them drops the probability to zero. Comparing critical against non-critical states under the same safety property reveals different top neurons and different feature importances, while Louvain communities show roughly 93-97% overlap, indicating largely shared processing regions. The paper's validation is that these pruning results "confirm that our method correctly identifies significant and highly relevant neurons for the safety properties."

Load-bearing premise

The method assumes that Pearson correlations computed over the model-checked state sets—sometimes just 12 states, with a critical/non-critical split of 22 versus 184—are stable enough for PageRank and community-detection rankings to be meaningful, yet it does not test this stability.

Editorial extensions

If this is right

  • For the taxi policy, the top features for finishing two jobs are passenger destination and jobs completed; pruning them reduces the two-job reachability probability from 1 to 0.25.
  • For finishing one job, top features are fuel level, destination x-coordinate, and jobs completed; pruning them drops one-job reachability to zero.
  • Critical and non-critical states under the same property produce different top neurons and feature rankings, so local state importance labels can be lifted to semi-global explanations.
  • Community overlap of 93-97% across labels suggests most of the network is shared and only a small set of neurons and features differentiates safety-relevant behavior.
  • The same method transfers to a second environment (cleaning robot), where feature importance rankings were identical for two safety properties and modularities were 0.37 and 0.35.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension not pursued here is to use the PageRank rankings directly as a pruning criterion, removing low-ranked neurons while re-verifying the safety property to obtain smaller, still-safe policies.
  • Because the label datasets are tiny (12 states in one case, 22 critical states in another), a bootstrap or leave-one-state-out stability test would tell whether the rankings are signal or artifacts of a few states.
  • The semi-global explanation idea could be applied to other policy properties besides safety, such as robustness or fairness constraints, as long as a labeler over reachable states exists.
  • One could test whether the co-activation graphs predict safety violations in a new environment region by using the ranked features to construct counterexamples.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a framework that combines probabilistic model checking of reinforcement learning (RL) policies with co-activation graph analysis. For a user-specified safety property, the framework builds the deterministic transition system induced by the trained policy, uses the Storm model checker to identify the states relevant to the property, optionally labels those states with a local explainability metric such as critical/non-critical status, and then constructs a neuron co-activation graph from activations over the resulting state set. PageRank and Louvain community detection are applied to rank neurons, identify important state features, and compare modularity and community overlap across labels. Experiments are reported on a taxi environment and a cleaning-robot environment, with two comparative settings: different safety properties (Section 5.1) and critical versus non-critical states (Section 5.2).

Significance. If the experimental concerns are resolved, the framework fills a legitimate gap between local and global explainable RL: it uses model checking to define semantically meaningful state subsets and transfers co-activation graph analysis from classification to sequential decision-making, with publicly available code and a coherent pipeline built on established tools. The main scientific value is the proposal of semi-global safety explanations and the demonstration that neuron-level graph analysis can be coupled with formal verification. However, the central claim that the method 'correctly identifies significant and highly relevant neurons for the safety properties' is currently supported only by small, unreplicated datasets and an external pruning reference, so the significance hinges on the robustness of the reported rankings.

major comments (4)
  1. [Section 5.1 and Section 5.2] The co-activation graph is computed from Pearson correlations over very small state sets: Section 5.1 reports 12 data points for P=1(♦ jobs=1) versus 206 for P=1(♦ jobs=2), and Section 5.2 reports 22 critical versus 184 non-critical states. With n=12, the standard error of a Pearson correlation is about 0.33, so the 2048-neuron correlation matrix is dominated by sampling noise and many large spurious correlations will appear. Since PageRank and Louvain are deterministic functions of that matrix, the reported top-50 rankings, feature importance lists, modularity differences (0.29 vs 0.25 and 0.25 vs 0.23), and community overlaps (97% and 93%) may shift substantially with one or two states. The paper provides no bootstrap, permutation, random-baseline, or replication analysis to establish stability, and the comparison in Section 5.1 is additionally confounded by the unequal sample sizes (12 vs 206). This directly undermines the claim in Section 5.3 that the method 'correctly identifies significant and highly relevant neurons'.
  2. [Section 5.1 and Section 5.3] The pruning validation used to support the feature rankings is cited from prior work (Gross and Spieker, 2024b) rather than performed and reported in this manuscript. The statements that pruning the identified feature neurons drops the reachability probability to 0.25 or 0 are presented as evidence, but no pruning experiment, ablation details, or quantitative results appear in this paper. Without reproducing the pruning check here or adding a random-feature control (e.g., pruning equally many randomly selected input features and showing they do not degrade the safety property), the conclusion that the co-activation rankings identify safety-relevant neurons is not supported by the evidence in this manuscript.
  3. [Section 5.2] The critical/non-critical labeling depends on a single threshold of 100 on the distance between the highest and lowest predicted Q-values, but no sensitivity analysis is provided for this threshold. Because the threshold determines the 22/184 split, the downstream neuron rankings, modularity values, and community structures could be artifacts of the chosen threshold rather than of the critical-state distinction. The paper should report results for at least two alternative thresholds or justify why the chosen threshold is not driving the conclusions.
  4. [Section 4.3] The limitation statement says the method 'supports labeled datasets of different sizes' without giving any guidance on minimum sample sizes or stability diagnostics. Given that the experiments use n=12 and n=22 for key subsets, this claim is too strong. The paper should either provide quantitative guidance (e.g., bootstrap confidence intervals or a minimum-correlation threshold) or explicitly downgrade the claim to describe the method as exploratory for very small state sets.
minor comments (5)
  1. [Section 3.3, Eq. (1)] Equation (1) appears to have a typo: the second correlation argument should be A(j,l,S) rather than A(j,k,S), since the right-hand side currently does not depend on the layer index l.
  2. [Section 4 heading] The heading 'Methodolodgy' is misspelled; it should be 'Methodology'.
  3. [Section 5.3] The sentence 'These findings conform that our method correctly identifies...' should read 'confirm' rather than 'conform'.
  4. [Figures 3 and 4] The captions of Figures 3 and 4 would be clearer if they explicitly stated which dataset label corresponds to each color, using the safety property notation from the text (e.g., P=1(♦ jobs=1) versus P=1(♦ jobs=2)).
  5. [Sections 5.1 and 5.2] The paper does not describe how neuron-level PageRank values are aggregated into state-feature importance rankings. Since features are attributes of states rather than neurons, the mapping from the highest-ranked neurons to the reported feature lists should be explained explicitly.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: co-activation rankings are computed from activations and model-checked labels, with pruning/model-checking used as independent validation.

full rationale

The derivation chain is not circular. Co-activation graphs are built from neuron activations on states labeled by model checking (Section 4.1), and PageRank/Louvain are deterministic graph algorithms applied to the resulting correlation matrices (Section 4.2); no parameter is fitted to the claimed rankings, and the labels are not derived from the graph analysis. The model-checked state sets come from COOL-MC (Gross et al., 2022) with Storm as an external model checker, and the pruning validation in Sections 5.1 and 5.3 cites the authors' prior pruning work (Gross and Spieker, 2024b) as a separate empirical check via model checking, not as an equation that makes the PageRank result true by construction. The only self-citations are tool/validation references, and none defines the target result into existence; the pruning check is externally falsifiable and does not rely on the present paper's fitted values. The limitations passage in Section 4.3 notes support for labeled datasets of different sizes without minimum-sample guidance, and the 12-state and 22-state datasets in Sections 5.1 and 5.2 raise a real stability concern, but instability is a correctness/robustness limitation, not circularity: the paper does not provide bootstrap or random baselines, but the absence of such checks does not by itself make the derivation equivalent to its inputs. Overall, the co-activation analysis is self-contained with respect to its inputs, so the circularity burden is not met.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The method has one hand-set analysis parameter (the critical-state threshold) and relies on three assumptions inherited from prior work: that co-activation correlations reflect functional structure, that Storm verification is exact, and that the policies and environments are finite and deterministic. No genuinely new entities are introduced; 'semi-global safety explanations' is a category name, not an invented physical or formal object.

free parameters (1)
  • Critical state threshold = 100 (difference between highest and lowest Q-values)
    Used in Section 5.2 to label states as critical or non-critical; changing this threshold changes the two co-activation graphs and all resulting rankings.
assumptions (3)
  • domain assumption Pairwise Pearson correlations of neuron activations over a label set summarize the policy's functional structure, and PageRank/Louvain on that graph reflect the roles of neurons and features.
    Equation (1) and Section 4.2 apply Horta et al. (2021) to RL without demonstrating that correlation-based centrality is stable in small RL state sets.
  • domain assumption Storm model checking of the induced DTMC returns the exact set of states satisfying the specified PCTL safety property.
    Section 4.1 builds on COOL-MC and Storm; the paper treats exactness as given and does not test verifier sensitivity.
  • domain assumption The trained policies are memoryless deterministic and the induced models are finite and fully explorable within the available resources.
    Sections 3.1 and 4.1 require a deterministic policy and finite reachable state space; the taxi and cleaning robot environments are small enough for this, but the assumption limits generality.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Co-Activation Graph Analysis of Safety-Verified and Explainable Deep Reinforcement Learning Policies." pith.science (2026). https://pith.science/paper/ZYWMUUN2

@misc{pith2026250103142,
  author       = {Pith},
  title        = {Pith review of: Co-Activation Graph Analysis of Safety-Verified and Explainable Deep Reinforcement Learning Policies},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZYWMUUN2}},
  note         = {Machine review of arXiv:2501.03142}
}
read the original abstract

Deep reinforcement learning (RL) policies can demonstrate unsafe behaviors and are challenging to interpret. To address these challenges, we combine RL policy model checking--a technique for determining whether RL policies exhibit unsafe behaviors--with co-activation graph analysis--a method that maps neural network inner workings by analyzing neuron activation patterns--to gain insight into the safe RL policy's sequential decision-making. This combination lets us interpret the RL policy's inner workings for safe decision-making. We demonstrate its applicability in various experiments.

Figures

Figures reproduced from arXiv: 2501.03142 by the authors.

Figure 2
Figure 2. This diagram represents an RL system in which [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The 50 most significant neurons identified for [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

52 extracted references · 43 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence 'output.state := if if FUNCTION not #0 #1 if FUNCTION and 'skip pop #0 if FUNCTIO...

  2. [2]

    and Katoen, J.-P

    Baier, C. and Katoen, J.-P. (2008). Principles of model checking . MIT press

    work page 2008

  3. [3]

    and Li, G

    Ban, J. and Li, G. (2024). Training is execution: A reinforcement learning-based collision avoidance algorithm for volatile scenarios. IEEE Access , 12:116956--116967

    work page 2024

  4. [4]

    Bekkemoen, Y. (2024). Explainable reinforcement learning (XRL): a systematic literature review and taxonomy. Mach. Learn. , 113(1):355--441

    work page 2024

  5. [5]

    D., Guillaume, J., and Lambiotte, R

    Blondel, V. D., Guillaume, J., and Lambiotte, R. (2023). Fast unfolding of communities in large networks: 15 years later. CoRR , abs/2311.06047

    arXiv 2023

  6. [6]

    Z., Parker, D., and Ujma, M

    Br \' a zdil, T., Chatterjee, K., Chmelik, M., Forejt, V., Kret \' nsk \' y , J., Kwiatkowska, M. Z., Parker, D., and Ujma, M. (2014). Verification of markov decision processes using learning algorithms. In ATVA , volume 8837 of LNCS

    work page 2014

  7. [7]

    G., and Lime, D

    Cassez, F., David, A., Fleury, E., Larsen, K. G., and Lime, D. (2005). Efficient on-the-fly algorithms for the analysis of timed games. In CONCUR , volume 3653 of Lecture Notes in Computer Science , pages 66--80. Springer

    2005

  8. [8]

    Corsi, D., Marchesini, E., and Farinelli, A. (2021). Formal verification of neural networks for safety-critical tasks in deep reinforcement learning. In de Campos, C. and Maathuis, M. H., editors, Proceedings of the Thirty-Seventh Conference on Uncertainty in Artificial Intelligence , volume 161 of Proceedings of Machine Learning Research , pages 333--343. PMLR

    work page 2021

Show all 52 references
  1. [9]

    G., Larsen, K

    David, A., Jensen, P. G., Larsen, K. G., Mikucionis, M., and Taankvist, J. H. (2015). Uppaal stratego. In TACAS , volume 9035 of Lecture Notes in Computer Science , pages 206--211. Springer

    2015

  2. [10]

    Z., Parker, D., and Ujma, M

    Dr \" a ger, K., Forejt, V., Kwiatkowska, M. Z., Parker, D., and Ujma, M. (2015). Permissive controller synthesis for probabilistic systems. Log. Methods Comput. Sci. , 11(2)

    2015

  3. [11]

    Eliyahu, T., Kazak, Y., Katz, G., and Schapira, M. (2021). Verifying learning-augmented systems. In SIGCOMM , pages 305--318. ACM

    2021

  4. [12]

    E., Noguez, J., and Reyes, A

    Elizalde, F., Sucar, L. E., Noguez, J., and Reyes, A. (2009). Generating explanations based on markov decision processes. In MICAI , volume 5845 of Lecture Notes in Computer Science , pages 51--62. Springer

    2009

  5. [13]

    E., Reyes, A., and deBuen, P

    Elizalde, F., Sucar, L. E., Reyes, A., and deBuen, P. (2007). An MDP approach for explanation generation. In ExaCt , volume WS-07-06 of AAAI Technical Report , pages 28--33. AAAI Press

    2007

  6. [14]

    P., Hermanns, H., Hoffmann, J., Klauck, M., K \" o hl, M

    Gros, T. P., Hermanns, H., Hoffmann, J., Klauck, M., K \" o hl, M. A., and Wolf, V. (2022). Mogym: Using formal models for training and verifying decision-making agents. In CAV (2) , volume 13372 of Lecture Notes in Computer Science , pages 430--443. Springer

    2022

  7. [15]

    Gross, D., Jansen, N., Junges, S., and P \' e rez, G. A. (2022). COOL-MC: A comprehensive tool for reinforcement learning and model checking. In SETTA , volume 13649 of Lecture Notes in Computer Science , pages 41--49. Springer

    2022

  8. [16]

    and Spieker, H

    Gross, D. and Spieker, H. (2024a). Enhancing rl safety with counterfactual llm reasoning. In ICTSS 2024, 36th International Conference on Testing Software and Systems

    2024

  9. [17]

    and Spieker, H

    Gross, D. and Spieker, H. (2024b). Safety-oriented pruning and interpretation of reinforcement learning policies. In Proceedings of the 32nd European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN 2024)

    2024

  10. [18]

    M., Perez, M., Schewe, S., Somenzi, F., Trivedi, A., and Wojtczak, D

    Hahn, E. M., Perez, M., Schewe, S., Somenzi, F., Trivedi, A., and Wojtczak, D. (2019). Omega-regular objectives in model-free reinforcement learning. In TACAS (1) , volume 11427 of LNCS , pages 395--412. Springer

    2019

  11. [19]

    and Jonsson, B

    Hansson, H. and Jonsson, B. (1994). A logic for reasoning about time and reliability. Formal Aspects Comput. , 6(5):512--535

    1994

  12. [20]

    Hasanbeig, M., Kroening, D., and Abate, A. (2020). Deep reinforcement learning with temporal logics. In FORMATS , volume 12288 of LNCS

    2020

  13. [21]

    Hensel, C., Junges, S., Katoen, J., Quatmann, T., and Volk, M. (2022). The probabilistic model checker Storm . Int. J. Softw. Tools Technol. Transf. , 24(4):589--610

    2022

  14. [22]

    R., Mueller, S

    Hoffman, R. R., Mueller, S. T., Klein, G., and Litman, J. (2018). Metrics for explainable AI: challenges and prospects. CoRR , abs/1812.04608

    2018 arXiv

  15. [23]

    R., Mueller, S

    Hoffman, R. R., Mueller, S. T., Klein, G., and Litman, J. (2023). Measures for explainable AI: explanation goodness, user satisfaction, mental models, curiosity, trust, and human-ai performance. Frontiers Comput. Sci. , 5

    2023

  16. [24]

    Horta, V. A. C. and Mileo, A. (2019). Towards explaining deep neural networks through graph analysis. In DEXA Workshops , volume 1062 of Communications in Computer and Information Science , pages 155--165. Springer

    2019

  17. [25]

    Horta, V. A. C. and Mileo, A. (2021). Generating local textual explanations for cnns: A semantic approach based on knowledge graphs. In AI*IA , volume 13196 of Lecture Notes in Computer Science , pages 532--549. Springer

    2021

  18. [26]

    Horta, V. A. C., Sobczyk, R., Stol, M. C., and Mileo, A. (2023). Semantic interpretability of convolutional neural networks by taxonomy extraction. In NeSy , volume 3432 of CEUR Workshop Proceedings , pages 118--127. CEUR-WS.org

    2023

  19. [27]

    Horta, V. A. C., Tiddi, I., Little, S., and Mileo, A. (2021). Extracting knowledge from deep neural networks through graph analysis. Future Gener. Comput. Syst. , 120:109--118

    2021

  20. [28]

    H., Bhatia, K., Abbeel, P., and Dragan, A

    Huang, S. H., Bhatia, K., Abbeel, P., and Dragan, A. D. (2018). Establishing appropriate trust via critical states. In 2018 IEEE/RSJ international conference on intelligent robots and systems (IROS) , pages 3929--3936. IEEE

    2018

  21. [29]

    Ji, Z., Liu, G., Xu, W., Yao, B., Liu, X., and Zhou, Z. (2024). Deep reinforcement learning on variable stiffness compliant control for programming-free robotic assembly in smart manufacturing. Int. J. Prod. Res. , 62(19):7073--7095

    2024

  22. [30]

    Jin, P., Wang, Y., and Zhang, M. (2022). Efficient LTL model checking of deep reinforcement learning systems using policy extraction. In SEKE , pages 357--362. KSI Research Inc

    2022

  23. [31]

    W., Katz, G., and Schapira, M

    Kazak, Y., Barrett, C. W., Katz, G., and Schapira, M. (2019). Verifying deep-rl-driven systems. In NetAI@SIGCOMM , pages 83--89. ACM

    2019

  24. [32]

    Liu, L., Yang, J., and Yan, B. (2024). A dynamic mission abort policy for transportation systems with stochastic dependence by deep reinforcement learning. Reliab. Eng. Syst. Saf. , 241:109682

    2024

  25. [33]

    Miao, C., Zeng, Z., Wu, Q., Yu, H., and Leung, C. (2018). Humanized artificial intelligence: What, why and how. International Journal of Information Technology , 24(2)

    2018

  26. [34]

    Milani, S., Topin, N., Veloso, M., and Fang, F. (2024). Explainable reinforcement learning: A survey and comparative review. ACM Comput. Surv. , 56(7):168:1--168:36

    2024

  27. [35]

    Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. A. (2013). Playing atari with deep reinforcement learning. CoRR , abs/1312.5602

    2013 arXiv

  28. [36]

    A., Veness, J., Bellemare, M

    Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M. A., Fidjeland, A., Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wierstra, D., Legg, S., and Hassabis, D. (2015). Human-le...

    2015

  29. [37]

    Page, L. (1999). The pagerank citation ranking: Bringing order to the web. Technical report, Technical Report

    1999

  30. [38]

    PRISM Manual

    PRISM ( 2023 ). PRISM Manual . www.prismmodelchecker.org. Accessed: 03/14/2024

    2023

  31. [39]

    Schilling, C., Lukina, A., Demirovic, E., and Larsen, K. G. (2023). Safety verification of decision-tree policies in continuous time. In NeurIPS

    2023

  32. [40]

    and Tiddi, I

    Selani, D. and Tiddi, I. (2021). Knowledge extraction from auto-encoders on anomaly detection tasks using co-activation graphs. In K-CAP , pages 65--71. ACM

    2021

  33. [41]

    and Guzdial, M

    Sieusahai, A. and Guzdial, M. (2021). Explaining deep reinforcement learning agents in the atari domain through a surrogate model. In AIIDE , pages 82--90. AAAI Press

    2021

  34. [42]

    L., Diuk, C., and Littman, M

    Strehl, A. L., Diuk, C., and Littman, M. L. (2007). Efficient structure learning in factored-state mdps. In AAAI , pages 645--650. AAAI Press

    2007

  35. [43]

    Sutton, R. S. and Barto, A. G. (2018). Reinforcement learning: An introduction . MIT press

    2018

  36. [44]

    Termine, A., Primiero, G., and D'Asaro, F. A. (2021). Modelling accuracy and trustworthiness of explaining agents. In LORI , volume 13039 of Lecture Notes in Computer Science , pages 232--245. Springer

    2021

  37. [45]

    and Taylor, M

    Torrey, L. and Taylor, M. E. (2013). Teaching on a budget: agents advising agents in reinforcement learning. In AAMAS , pages 1053--1060. IFAAMAS

    2013

  38. [46]

    a llstr \

    Vamplew, P., Smith, B. J., K \" a llstr \" o m, J., de Oliveira Ramos, G., Radulescu, R., Roijers, D. M., Hayes, C. F., Heintz, F., Mannion, P., Libin, P. J. K., Dazeley, R., and Foale, C. (2022). Scalar reward is not enough: a response to silver, singh, precup and sutton (202...

    2022

  39. [47]

    Vouros, G. A. (2023). Explainable Deep Reinforcement Learning : State of the Art and Challenges . ACM Computing Surveys , 55(5):1--39. arXiv:2301.09937 [cs]

    2023 arXiv

  40. [48]

    Wang, Y., Liu, A., Yang, J., Wang, L., Xiong, N., Cheng, Y., and Wu, Q. (2024). Clinical knowledge-guided deep reinforcement learning for sepsis antibiotic dosing recommendations. Artif. Intell. Medicine , 150:102811

    2024

  41. [49]

    Wang, Y., Roohi, N., West, M., Viswanathan, M., and Dullerud, G. E. (2020). Statistically model checking PCTL specifications on markov decision processes via reinforcement learning. In CDC , pages 1392--1397. IEEE

    2020

  42. [50]

    Watkins, C. J. and Dayan, P. (1992). Q-learning. Machine learning , 8:279--292

    1992

  43. [51]

    Zhu, C., Dastani, M., and Wang, S. (2024). A survey of multi-agent deep reinforcement learning with communication. Auton. Agents Multi Agent Syst. , 38(1):4

    2024

  44. [52]

    Zhu, H., Xiong, Z., Magill, S., and Jagannathan, S. (2019). An inductive synthesis framework for verifiable reinforcement learning. In PLDI , pages 686--701. ACM

    2019

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.