Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

Causality-informed Anomaly Detection in Partially Observable Sensor Networks: Moving beyond Correlations

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Causal DQ claims that adding a causal mask to Q-network training yields tighter error bounds and faster convergence for partially observable sensor placement.

desk verdict A novel causal-regularized DQN for sensor placement with an honest ablation and broad experiments, but the tighter-bound proofs are invalid and one table contradicts the empirical headline. read the letter →

arxiv 2507.09742 v1 pith:V2R6SBXJ submitted 2025-07-13 cs.AI

Xiaofeng Xiao , Bo Shen , Xubo Yue This is my paper
classification cs.AI MSC 68T0568T3790C4062L10
keywords anomalydetectionpartialobservabilitysensorplacementcausalreinforcementlearningdeepQ-networkentropysequentialchangemeanshift
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that injecting causal information into a deep Q-network---specifically a binary causal mask that flags which data streams are truly mean-shifted---makes sensor-placement learning for anomaly detection converge faster and with provably tighter error bounds than a correlation-only Q-network. The authors build a 'Causal DQ' framework that folds causality into the state, the reward, and a causal-entropy regularizer in the loss. They prove that the regularized optimal Q-value is bounded by a term depending on the number of causally relevant streams, not the full combinatorial action space. If correct, this means partial-observability anomaly detection can be both faster and more reliable without performing risky interventions to discover causal effects.

What carries the argument

The load-bearing object is the causal entropy regularizer $H_c(\pi(\cdot|s)) = -\mathbb{E}_{a_i\in A}[\sum C_{a_i\rightarrow r|s}\;\pi(a_i|s)\log\pi(a_i|s)]$, which is subtracted from the Q-learning loss. It is built from the causal mask $C_{a_i\rightarrow r|s}$, a binary oracle-like indicator of which streams truly shifted; the mask also enters the reward through $M^{a\rightarrow r}$. As a result, all the paper's bounds---the contraction lemma, the optimal-value bound, the finite-time error bound, and the convergence-time bound---have $\log\sum C$ in place of $\log|A|$, which is what produces the claimed tighter error and faster convergence.

What would settle it

Run the full Causal DQ pipeline on the Tennessee Eastman data with $m=5$ and $\delta=0.25$, but compute the causal mask only from the estimated causal graph, with no access to true shift locations. If, over 100 replications, the average detection delay is not below the Non-Causal baseline of 193.5, the practical claim that causality-informed masks improve detection would fail. A sharper test is to vary the causal-discovery quality and check whether ADD tracks the mask's false-discovery rate as the theory implies.

Watch

Extended reading notes

Core claim

The central claim is that the causal mask $C_{a_i\rightarrow r|s}$, a binary indicator of whether action $a_i$ selects a mean-shifted stream, can be woven into Q-network training so that every theoretical guarantee of the non-causal DQN survives, but with the action-space cardinality $|A|$ replaced by the strictly smaller sum of the mask. Theorem 5.1 gives $Q^*(s,a) \leq r(s,a) + \gamma\,\mathbb{E}[\max_{a'} Q(s',a')] + \frac{\gamma}{1-\gamma}\frac{\log\sum C_{a_i\rightarrow r|s}}{\tau}$, and Corollaries 5.1--5.2 and Lemma 5.2 translate this into an error bound and convergence-time bound that are tighter than the non-causal versions. The paper also reports Average Detection Delay improvements over four baselines in simulations and two real datasets, with the largest gains at weak mean shifts. The authors are careful to note that the causal-entropy regularizer introduces an asymptotic bias, but argue that the bias shrinks the error because it is driven by the causal mask rather than the full action space.

Load-bearing premise

The load-bearing assumption is that training knows exactly which data streams have truly shifted, because that knowledge feeds both the reward and the proved bounds; in real deployment those locations must be inferred, and the causal-discovery step sits outside the theory.

Editorial extensions

If this is right

  • If the bounds hold, Causal DQ converges to the optimal Q-function in $O(\log(1/(\varepsilon - \frac{\gamma}{1-\gamma}\frac{\log\sum C}{\tau})))$ steps, exponentially faster than the non-causal bound driven by $|A|$.
  • The asymptotic error floor shrinks because $\log\sum C$ replaces $\log|A|$; in high-dimensional settings where $|A|=\binom{p}{m}$ is enormous, the gap is substantial.
  • Average detection delay is shortest at weak mean shifts ($\delta=0.25$--$0.5$), exactly where correlation-based methods degrade, because causal information lets the agent focus on ancestor streams.
  • The framework is intervention-free, so it can be applied where deliberately injecting faults is unsafe, and the same causal-entropy regularizer can be dropped into other RL algorithms.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The theory's oracle assumption suggests a test the paper does not run: replace the ground-truth mask with the estimated-graph mask and measure the degradation; the paper's own Table 10 indicates ADD can jump from 9 to 200 when recovery fails.
  • The same mask-based entropic regularization could be applied to actor-critic or policy-gradient methods, where the mask would modulate the policy gradient instead of the Q-loss.
  • A natural extension is a learned, continuous causal mask instead of a binary oracle, so uncertain causal estimates are down-weighted rather than thresholded.
  • If causal discovery runs online while the sensor budget changes, the causal statistic could adaptively re-rank streams, potentially extending the approach to non-stationary causal graphs.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes Causal DQ, a deep Q-network for partially observable sensor placement in anomaly detection, in which causal discovery is used to construct a causal state, a causal mask is inserted into the reward, and a causal-entropy regularizer is added to the DQN loss. The central theoretical claims are that the causal-entropy-regularized Bellman operator remains a contraction, that the optimal Q-function admits a tighter upper bound than the non-causal softmax bound, that the error between the learned and optimal Q-functions is smaller, and that convergence is exponentially faster. These claims are supported by simulation studies and two real-world case studies (Tennessee Eastman Process and Solar Flare Detection).

Significance. If the theoretical claims were correct, the paper would make a useful contribution: it addresses a practically important problem, proposes an intervention-free way to combine causal information with deep RL for sensor placement, and backs the method with an extensive empirical study including an ablation that explicitly tests sensitivity to causal-discovery quality. However, the central theoretical result is invalid as stated: the proof of Theorem 5.1 relies on a false inequality for the causal entropy, and the causal mask cancels when the claimed causal value function is written out, so the tighter-bound argument does not go through. In addition, the theory assumes an oracle causal mask that equals the ground-truth mean-shift indicator, which is unavailable in deployment and is not the output of the causal-discovery step that the paper highlights as its novelty. The empirical results may still indicate practical value, but the abstract and Section 5 make the theoretical contribution load-bearing, and that contribution is not established.

major comments (4)
  1. [Appendix A.1, proof of Theorem 5.1] The proof uses the assertion 'As 0 ≤ Hc(π(·|s)) ≤ log Σ_{ai} C_{ai→r|s}' to replace log|A| by log ΣC in the softmax bound. This inequality is false. For C=(1,0) and π=(0.5,0.5), Hc=0.3466 > log 1=0; in general, because π is not renormalized to the causal support, Hc can exceed log ΣC. Moreover, substituting the definition of Hc into the displayed expression for Vc(s) gives Vc(s)=(1/τ) log Σ_a exp(τ Q(s,a)), so the causal mask cancels and Vc is exactly the ordinary softmax value. Consequently, the claimed tighter upper bound in Theorem 5.1 is not established, and Remarks 5.1–5.2, Corollaries 5.1–5.2, and Lemma 5.2 all inherit this error. This invalidates the central claim of tighter theoretical error bounds for Causal DQ.
  2. [§4.1, §4.3, §5; §6.5 Table 10] The causal mask C_{ai→r|s} is defined to be 1 exactly when action ai selects a data stream with a mean shift, i.e., it is the ground-truth anomaly indicator. The same oracle mask enters the reward (§4.3) and every theoretical bound in Section 5, but in deployment the true mean-shift locations are unknown. The causal discovery graph, which is the paper's stated novelty, does not appear in Theorem 5.1, Corollary 5.1, or Theorem 5.2. The paper's own ablation study shows the practical consequence: when causal discovery recovers no edges (mTPR=0), Causal DQ fails to detect any shift (ADD=200), whereas perfect recovery gives ADD=9. Thus the theoretical improvement is not attributable to the causal-discovery component, and the proven bounds hold only under an oracle assumption that the method cannot satisfy at test time.
  3. [§4.3 and Assumption 5.2] Theorem 5.2 assumes |r(s,a,s′)| ≤ Rmax ≤ 1, but the reward defined in Section 4.3 includes the large negative constant U (e.g., −20) and positive reward constants yi, wi that are not scaled by Rmax. The implemented training therefore violates the assumption under which the finite-time error bound is proved, and the paper provides no reward rescaling argument. As a result, the finite-time bound in Theorem 5.2 is not applicable to the algorithm as run in Sections 6 and 7.
  4. [§4.2, §4.4, Remark 5.2] The action representation is inconsistent between the theoretical comparison and the implementation. In §4.2 an action is a p-dimensional binary vector with exactly m ones, and Remark 5.2 takes |A|=C(p,m); but in §4.4 the network outputs per-stream Q-values and selects the top-m streams, which corresponds to a softmax over p individual actions. The inequality Σ C_{ai→r|s} < |A| and the claim that the causal bound is tighter than the non-causal bound depend on which action space is meant. This ambiguity makes the tightness comparison in Remarks 5.1–5.2 and 5.7 not well-defined.
minor comments (4)
  1. [Throughout] There are numerous typographical errors, including 'F oundations' in Table 1, 'q=100' instead of 'p=100' in §6.1, 'T o' in §4, and 'the the' in §3.2. A thorough proofread is needed.
  2. [§4.1] The continuous-action definition of Hc writes Pc(a|s)=C(a,s)π(a|s)/∫ C(ã,s)π(ã|s)dã, but the discrete definition used in the proofs sums C_iπ_i without renormalizing. These two definitions behave differently, and the paper does not state which one is used in Section 5.
  3. [§6, Figures 2–4] The convergence comparisons appear to be based on single training runs. Reporting multiple seeds with error bars would substantiate the claim that Causal DQ 'converges faster and more stably' than the non-causal baseline.
  4. [§5, Theorem 5.2] The proof of Theorem 5.2 introduces a sampling matrix Ω and changes notation relative to the statement of the theorem; in particular, the transition probability matrix P in the theorem statement is replaced by a replay-buffer sampling construction in the proof. The relation between these objects should be made explicit.

Circularity Check

1 steps flagged · score 6.0 of 10

The 'tighter causal bound' is built from the oracle mean-shift mask C, not from estimated causal structure, so the claimed advantage reduces by construction to the target labels.

  1. self definitional [Section 4.1 (causal mask definition) and Remark 5.2 / Theorem 5.1]
    "we utilize a binary vector to represent Cai→r|s, assigning a value of 1 to an action ai if it selects a data stream with a mean shift, and 0 otherwise (Sec. 4.1); "The magnitude of causal mask in reward computing satisfies P ai Cai→r|s ≤ m < |A| = C p m ... This implies that the error bounds for the optimal value function Q∗ in Causal DQ are tighter than those for Q∗ nc in non-causal Q-network" (Remark 5.2)."

    The improvement advertised by Theorem 5.1 and the corollaries is exactly the replacement of log|A| with log ΣC. But C is not obtained from the causal discovery step; the paper defines C as the ground-truth indicator of which selected streams have a mean shift, and the same indicator is inserted into the reward function and the causal entropy regularizer. Thus the tighter error bound and the resulting faster-convergence claim are forced by construction: the bound is a function of the oracle number of anomalous streams supplied as an input, rather than a consequence of the estimated causal graph.

full rationale

The main circular step is the use of the oracle mean-shift mask as the source of theoretical tightness. Section 4.1 defines C as the true mean-shift indicator, Section 4.3 feeds the same indicator into the reward, and Theorem 5.1/Remarks 5.1-5.2 express every bound through log ΣC and then assert tightness because ΣC < |A|. The estimated causal graph is never used in the proof, so the claimed causal advantage is an artifact of plugging the target labels into the regularizer. This is a self-definitional reduction of the central theoretical claim rather than an empirical or externally derived result, so the circularity score is 6 rather than 0-2. Separately, the Appendix's asserted inequality 0 ≤ Hc ≤ log ΣC is mathematically doubtful and the substitution of Hc into Vc cancels the causal mask, collapsing Vc to the ordinary softmax value; I treat this as a correctness risk that further weakens the proof, but it is not itself an additional circular step. The empirical sections are benchmarked against external baselines and provide independent content, and the ablation in Section 6.5 honestly reports that completely wrong causal graphs make Causal DQ fail to detect shifts (Table 10), which weighs as a limitation rather than as circularity. Self-citations in the paper are illustrative or methodological and are not load-bearing for the derivation.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claim rests on four families of assumptions: Gaussian streaming data with a recursively computable posterior, identifiability of the causal graph from observational data, availability of the ground-truth anomaly mask during training, and a reward-boundedness condition that the paper's own reward violates. The free parameters include several hyperparameters that are tuned per scenario or left entirely unspecified (lambda, reward constants, simulation edge probability). No new physical entities are introduced.

free parameters (6)
  • lambda (time-decay parameter in local statistic) = not reported
    Section 4.2 defines the recursive posterior update p(mu|...) = p(mu|...)^(1-lambda) * p(X(n)|mu); lambda controls the forgetting rate and is never specified, yet it determines the local statistic used in the state.
  • causal entropy decay rate alpha = 0.05 (p=10), 0.1 (p=50,100)
    Table 13 lists per-scenario values tuned by the authors; alpha directly weights the causal entropy term in the loss (Eq. 4).
  • temperature tau = 0.65 to 0.9 per scenario
    Table 13 lists tuned values controlling exploration; tau appears in the denominator of the claimed bounds.
  • reward constants y_i and w_i = unspecified
    Section 4.3 defines g = sum a_i*y_i + w_i*s_i with positive values if stream i has a mean shift, but no numeric values are given, so the reward scale is a free choice.
  • negative reward U = -20
    Section 4.3 fixes U=-20 by hand; it contradicts Assumption 5.2 that |r|<=1.
  • ER graph edge probability alpha (simulation) = unspecified
    Section 6 draws a DAG with each edge probability alpha but never reports alpha, so the causal graph density in the simulation is unconstrained.
assumptions (5)
  • domain assumption The data streams follow a Gaussian distribution with mean mu and covariance Sigma.
    Invoked in Sections 3.1 and 4.2 to construct the local statistic Lambda(n) and its chi-squared distribution; the recursive posterior update assumes Gaussian conjugacy.
  • domain assumption Causal Markov condition and faithfulness hold so PC/FCI recover a CPDAG consistent with the data.
    Section 3.2 relies on standard causal discovery assumptions; partial observability and latent confounders are not addressed.
  • ad hoc to paper The binary causal mask C_{ai->r|s} is known during training and equals the ground-truth mean-shift indicator for each stream.
    Section 4.1 defines C(a,s)=1 if action ai selects a mean-shifted stream; Section 4.3 uses the same mask for rewards. The theoretical bounds in Theorem 5.1 and Corollaries depend on sum C, the true number of anomalies, so this is an oracle assumption.
  • ad hoc to paper Reward magnitude is bounded by R_max <= 1 (Assumption 5.2).
    Assumption 5.2 in Section 5 is required by Theorem 5.2, but Section 4.3 sets U=-20, a reward with magnitude 20, so the assumption is violated by the paper's own reward design.
  • domain assumption The estimated causal graph is accurate enough that the causal statistic phi_i remains informative for state construction.
    Section 6.5 ablation shows ADD degrades when TPR is below 0.4 and detection fails when the graph is completely wrong, so empirical performance depends on this assumption.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Causality-informed Anomaly Detection in Partially Observable Sensor Networks: Moving beyond Correlations." pith.science (2026). https://pith.science/paper/V2R6SBXJ

@misc{pith2026250709742,
  author       = {Pith},
  title        = {Pith review of: Causality-informed Anomaly Detection in Partially Observable Sensor Networks: Moving beyond Correlations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/V2R6SBXJ}},
  note         = {Machine review of arXiv:2507.09742}
}
read the original abstract

Nowadays, as AI-driven manufacturing becomes increasingly popular, the volume of data streams requiring real-time monitoring continues to grow. However, due to limited resources, it is impractical to place sensors at every location to detect unexpected shifts. Therefore, it is necessary to develop an optimal sensor placement strategy that enables partial observability of the system while detecting anomalies as quickly as possible. Numerous approaches have been proposed to address this challenge; however, most existing methods consider only variable correlations and neglect a crucial factor: Causality. Moreover, although a few techniques incorporate causal analysis, they rely on interventions-artificially creating anomalies-to identify causal effects, which is impractical and might lead to catastrophic losses. In this paper, we introduce a causality-informed deep Q-network (Causal DQ) approach for partially observable sensor placement in anomaly detection. By integrating causal information at each stage of Q-network training, our method achieves faster convergence and tighter theoretical error bounds. Furthermore, the trained causal-informed Q-network significantly reduces the detection time for anomalies under various settings, demonstrating its effectiveness for sensor placement in large-scale, real-world data streams. Beyond the current implementation, our technique's fundamental insights can be applied to various reinforcement learning problems, opening up new possibilities for real-world causality-informed machine learning methods in engineering applications.

Figures

Figures reproduced from arXiv: 2507.09742 by the authors.

Figure 1
Figure 1. Workflow of our proposed causality-informed anomaly detection under par [PITH_FULL_IMAGE:figures/full_fig_p010_1.png] view at source ↗
Figure 2
Figure 2. Total Reward Comparison between Causal and Non-Causal DQ with differ [PITH_FULL_IMAGE:figures/full_fig_p024_2.png] view at source ↗
Figure 3
Figure 3. Tennessee Eastman Process (TEP); Total Reward Comparison between [PITH_FULL_IMAGE:figures/full_fig_p031_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Solar Flare Detection (SFD); Total Reward Comparison between Causal and [PITH_FULL_IMAGE:figures/full_fig_p033_4.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Causal Transfer in Medical Image Analysis

    cs.CV 2026-03 accept novelty 5.0 of 10

    Causal Transfer Learning unifies structural causal models, invariant risk minimisation and counterfactuals with transfer learning to produce domain-robust medical image models.

Reference graph

Works this paper leans on

74 extracted references · 66 canonical work pages · cited by 1 Pith paper

  1. [1]

    Boosting Soft Q-Learning by Bounding

    Adamczyk, J., V. Makarenko, S. Tiomkin, and R. V. Kulkarni (2024). Boosting soft q-learning by bounding. arXiv preprint arXiv:2406.18033\/

    work page Pith review arXiv 2024

  2. [2]

    Aydin, and D

    Ahmadzadeh, A., B. Aydin, and D. Kempton (2020). Bigdata cup challenge 2020: Flare prediction. https://kaggle.com/competitions/bigdata2020-flare-prediction

    work page 2020

  3. [3]

    Arshad, K., R. F. Ali, A. Muneer, I. A. Aziz, S. Naseer, N. S. Khan, and S. M. Taib (2022). Deep reinforcement learning for anomaly detection: A systematic review. Ieee Access\/ 10 , 124017--124035

    work page 2022

  4. [4]

    Cao, H., F. Feng, T. Yang, J. Huo, and Y. Gao (2025). Causal information prioritization for efficient reinforcement learning. arXiv preprint arXiv:2502.10097\/

    work page Pith review arXiv 2025

  5. [5]

    Federated Learning of Dynamic Bayesian Network via Continuous Optimization from Time Series Data

    Chen, J., Y. Ma, and X. Yue (2024). Temporal causal discovery in dynamic bayesian networks using federated learning. arXiv preprint arXiv:2412.09814\/

    work page Pith review arXiv 2024

  6. [6]

    Cheng, L., R. Guo, R. Moraffah, P. Sheth, K. S. Candan, and H. Liu (2022). Evaluation methods and measures for causal learning algorithms. IEEE Transactions on Artificial Intelligence\/ 3\/ (6), 924--943

    work page 2022

  7. [7]

    Chong, E. K., C. M. Kreucher, and A. O. Hero III (2009). Partially observable markov decision process approximations for adaptive sensing. Discrete Event Dynamic Systems\/ 19\/ (3), 377--422

    work page 2009

  8. [8]

    Kroizer, and T

    Dabush, L., A. Kroizer, and T. Routtenberg (2023). State estimation in partially observable power systems via graph signal processing tools. Sensors\/ 23\/ (3), 1387

    work page 2023

Show all 74 references
  1. [9]

    Jiang, G

    Deng, Z., J. Jiang, G. Long, and C. Zhang (2023). Causal reinforcement learning: A survey. arXiv preprint arXiv:2307.01452\/

    2023 arXiv

  2. [10]

    Entner, D. and P. O. Hoyer (2010). On causal discovery from time series data using fci. Probabilistic graphical models\/ 16

    2010

  3. [11]

    Fan, J., Z. Wang, Y. Xie, and Z. Yang (2020). A theoretical analysis of deep q-learning. In Learning for dynamics and control , pp.\ 486--489. PMLR

    2020

  4. [12]

    Frauen, V

    Feuerriegel, S., D. Frauen, V. Melnychuk, J. Schweisthal, K. Hess, A. Curth, S. Bauer, N. Kilbertus, I. S. Kohane, and M. van der Schaar (2024). Causal machine learning for predicting treatment outcomes. Nature Medicine\/ 30\/ (4), 958--968

    2024

  5. [13]

    Addanki, T

    Gao, S., R. Addanki, T. Yu, R. A. Rossi, and M. Kocaoglu (2024). Causal discovery-driven change point detection in time series. arXiv preprint arXiv:2407.07290\/

    2024 arXiv

  6. [14]

    Grasset, G

    Gasse, M., D. Grasset, G. Gaudron, and P.-Y. Oudeyer (2021). Causal reinforcement learning using observational and interventional data. arXiv preprint arXiv:2106.14421\/

    2021 arXiv

  7. [15]

    Scherrer, and O

    Geist, M., B. Scherrer, and O. Pietquin (2019). A theory of regularized markov decision processes. In International conference on machine learning , pp.\ 2160--2169. PMLR

    2019

  8. [16]

    Zhang, and P

    Glymour, C., K. Zhang, and P. Spirtes (2019). Review of causal discovery methods based on graphical models. Frontiers in genetics\/ 10 , 524

    2019

  9. [17]

    Grimbly, S. J., J. Shock, and A. Pretorius (2021). Causal multi-agent reinforcement learning: Review and open problems. arXiv preprint arXiv:2111.06721\/

    2021 arXiv

  10. [18]

    Yan, and C

    Guo, J., H. Yan, and C. Zhang (2024). Thompson sampling-based partially observable online change detection for exponential families. INFORMS Journal on Data Science\/ 3\/ (2), 145--161

    2024

  11. [19]

    Guo, R., L. Wang, C. Zhang, L. Gu, D. Li, and X. Qian (2024). A causality-informed graph convolutional network for video assessment of parkinsonian leg agility. IEEE Transactions on Circuits and Systems for Video Technology\/

    2024

  12. [20]

    Harris, N. and M. Drton (2013). Pc algorithm for nonparanormal graphical models. The Journal of Machine Learning Research\/ 14\/ (1), 3365--3383

    2013

  13. [21]

    Vecerik, O

    Hester, T., M. Vecerik, O. Pietquin, M. Lanctot, T. Schaul, B. Piot, D. Horgan, J. Quan, A. Sendonaris, I. Osband, et al. (2018). Deep q-learning from demonstrations. In Proceedings of the AAAI conference on artificial intelligence , Volume 32

    2018

  14. [22]

    Peters, and N

    Huang, S., J. Peters, and N. Pfister (2024). Causal change point detection and localization. arXiv preprint arXiv:2403.12677\/

    2024 arXiv

  15. [23]

    Jeong, N. and D. Lee (2024). Finite-time error analysis of soft q-learning: Switching system approach. arXiv preprint arXiv:2403.06366\/

    2024 arXiv

  16. [24]

    Liang, Y

    Ji, T., Y. Liang, Y. Zeng, Y. Luo, G. Xu, J. Guo, R. Zheng, F. Huang, F. Sun, and H. Xu (2024). Ace: Off-policy actor-critic with causality-aware entropy regularization. arXiv preprint arXiv:2402.14528\/

    2024 arXiv

  17. [25]

    Lynch, Q

    Kaddour, J., A. Lynch, Q. Liu, M. J. Kusner, and R. Silva (2022). Causal machine learning: A survey and open problems. arXiv preprint arXiv:2206.15475\/

    2022 arXiv

  18. [26]

    Kalisch, M. and P. B \"u hlman (2007). Estimating high-dimensional directed acyclic graphs with the pc-algorithm. Journal of Machine Learning Research\/ 8\/ (3)

    2007

  19. [27]

    Kitson, N. K., A. C. Constantinou, Z. Guo, Y. Liu, and K. Chobtham (2023). A survey of bayesian network structure learning. Artificial Intelligence Review\/ 56\/ (8), 8721--8814

    2023

  20. [28]

    Shakkottai, A

    Kocaoglu, M., S. Shakkottai, A. G. Dimakis, C. Caramanis, and S. Vishwanath (2020). Applications of common entropy for causal inference. Advances in neural information processing systems\/ 33 , 17514--17525

    2020

  21. [29]

    Korb, K. B. and A. E. Nicholson (2010). Bayesian artificial intelligence . CRC press

    2010

  22. [30]

    Kurniawati, H. (2022). Partially observable markov decision processes and robotics. Annual Review of Control, Robotics, and Autonomous Systems\/ 5\/ (1), 253--277

    2022

  23. [31]

    Hsu, and J

    Lauri, M., D. Hsu, and J. Pajarinen (2022). Partially observable markov decision processes in robotics: A survey. IEEE Transactions on Robotics\/ 39\/ (1), 21--40

    2022

  24. [32]

    Lechner, M. (2023). Causal machine learning and its use for public policy. Swiss Journal of Economics and Statistics\/ 159\/ (1), 8

    2023

  25. [33]

    Hu, and N

    Lee, D., J. Hu, and N. He (2023). A discrete-time switching system analysis of q-learning. SIAM Journal on Control and Optimization\/ 61\/ (3), 1861--1880

    2023

  26. [34]

    Ye, J.-R

    Li, H., H. Ye, J.-R. C. Cheng, and K. Liu (2024). Online monitoring of heterogeneous partially observable data streams based on q-learning. IEEE Transactions on Automation Science and Engineering\/

    2024

  27. [35]

    Zheng, and K

    Li, H., Z. Zheng, and K. Liu (2025). Online monitoring of high-dimensional data streams with deep q-network. IEEE Transactions on Automation Science and Engineering\/

    2025

  28. [36]

    Jin, and J

    Li, J., J. Jin, and J. Shi (2008). Causation-based t 2 decomposition for multivariate process monitoring and diagnosis. Journal of Quality Technology\/ 40\/ (1), 46--58

    2008

  29. [37]

    Luo, and X

    Li, J., Y. Luo, and X. Zhang (2021). Causal reinforcement learning: An instrumental variable approach. arXiv preprint arXiv:2103.04021\/

    2021 arXiv

  30. [38]

    Li, J. and J. Shi (2007). Knowledge discovery from observational data for process control using causal bayesian networks. IIE transactions\/ 39\/ (6), 681--690

    2007

  31. [39]

    Li, X., R. Guo, H. Zhu, T. Chen, and X. Qian (2024). A causality-informed graph intervention model for pancreatic cancer early diagnosis. IEEE Transactions on Artificial Intelligence\/

    2024

  32. [40]

    Liao, L., Z. Fu, Z. Yang, Y. Wang, D. Ma, M. Kolar, and Z. Wang (2024). Instrumental variable value iteration for causal offline reinforcement learning. Journal of Machine Learning Research\/ 25\/ (303), 1--56

    2024

  33. [41]

    Chung, C

    Liu, Q., A. Chung, C. Szepesv \'a ri, and C. Jin (2022). When is partially observable reinforcement learning not scary? In Conference on Learning Theory , pp.\ 5175--5220. PMLR

    2022

  34. [42]

    Szepesv \'a ri, and C

    Liu, Q., C. Szepesv \'a ri, and C. Jin (2022). Sample-efficient reinforcement learning of partially observable markov games. Advances in Neural Information Processing Systems\/ 35 , 18296--18308

    2022

  35. [43]

    Liu, Y., H. Wang, Z. Wang, X. Zhu, J. Liu, P. Sun, R. Tang, J. Du, V. C. Leung, and L. Song (2025). Crcl: Causal representation consistency learning for anomaly detection in surveillance videos. IEEE Transactions on Image Processing\/

    2025

  36. [44]

    H \"u tter, J

    Lopez, R., J.-C. H \"u tter, J. Pritchard, and A. Regev (2022). Large-scale differentiable causal discovery of factor graphs. Advances in Neural Information Processing Systems\/ 35 , 19290--19303

    2022

  37. [45]

    Meisami, and A

    Lu, Y., A. Meisami, and A. Tewari (2022). Efficient reinforcement learning with prior causal knowledge. In Conference on Causal Learning and Reasoning , pp.\ 526--541. PMLR

    2022

  38. [46]

    Marcot, B. G. and T. D. Penman (2019). Advances in bayesian network modelling: Integration of modelling technologies. Environmental modelling & software\/ 111 , 386--393

    2019

  39. [47]

    Naser, M. (2022). Causality, causal discovery, and causal inference in structural engineering. arXiv preprint arXiv:2204.01543\/

    2022 arXiv

  40. [48]

    Oh, M.-h. and G. Iyengar (2019). Sequential anomaly detection using inverse reinforcement learning. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & data mining , pp.\ 1480--1490

    2019

  41. [49]

    Pan, L., Q. Cai, Q. Meng, W. Chen, L. Huang, and T.-Y. Liu (2019). Reinforcement learning with dynamic boltzmann softmax updates. arXiv preprint arXiv:1903.05926\/

    2019 arXiv

  42. [50]

    Pearl, J. (1995). From bayesian networks to causal networks. In Mathematical models for handling partial knowledge in artificial intelligence , pp.\ 157--182. Springer

    1995

  43. [51]

    Rieth, C. A., B. D. Amsel, R. Tran, and M. B. Cook (2017). Additional tennessee eastman process simulation data for anomaly detection evaluation. Harvard Dataverse\/ 1 , 2017

    2017

  44. [52]

    Zhang, X

    Ruan, K., J. Zhang, X. Di, and E. Bareinboim (2023). Causal imitation learning via inverse reinforcement learning. In The Eleventh International Conference on Learning Representations

    2023

  45. [53]

    Sch \"o lkopf, and G

    Seitzer, M., B. Sch \"o lkopf, and G. Martius (2021). Causal influence detection for improving efficiency in reinforcement learning. Advances in Neural Information Processing Systems\/ 34 , 22905--22918

    2021

  46. [54]

    Shao, Y., Q. Cao, S. C. Liew, and H. Chen (2021). Partially observable minimum-age scheduling: The greedy policy. IEEE Transactions on Communications\/ 70\/ (1), 404--418

    2021

  47. [55]

    Shi, C., X. Wang, S. Luo, H. Zhu, J. Ye, and R. Song (2023). Dynamic causal effects evaluation in a/b testing with a reinforcement learning framework. Journal of the American Statistical Association\/ 118\/ (543), 2059--2071

    2023

  48. [56]

    Succar, R. and M. Porfiri (2025). Detecting directional coupling in network dynamical systems via kalman’s observability. Physical Review Letters\/ 134\/ (7), 077401

    2025

  49. [57]

    Tang, X., R. Guo, C. Zhang, and X. Qian (2024). A causal counterfactual graph neural network for arising-from-chair abnormality detection in parkinsonians. Medical Image Analysis\/ 97 , 103266

    2024

  50. [58]

    Tsamardinos, I., L. E. Brown, and C. F. Aliferis (2006). The max-min hill-climbing bayesian network structure learning algorithm. Machine learning\/ 65 , 31--78

    2006

  51. [59]

    Guez, and D

    Van Hasselt, H., A. Guez, and D. Silver (2016). Deep reinforcement learning with double q-learning. In Proceedings of the AAAI conference on artificial intelligence , Volume 30

    2016

  52. [60]

    Rinaldi, F

    Viti, F., M. Rinaldi, F. Corman, and C. M. Tamp \`e re (2014). Assessing partial observability in network sensor location problems. Transportation research part B: methodological\/ 70 , 65--89

    2014

  53. [61]

    Wang, X., Y. Du, S. Zhu, L. Ke, Z. Chen, J. Hao, and J. Wang (2021). Ordering-based causal discovery with reinforcement learning. arXiv preprint arXiv:2105.06631\/

    2021 arXiv

  54. [62]

    Li, and K

    Xian, X., J. Li, and K. Liu (2018). Causation-based monitoring and diagnosis for multivariate categorical processes with ordinal information. IEEE Transactions on Automation Science and Engineering\/ 16\/ (2), 886--897

    2018

  55. [63]

    Alharbi, P

    Xiao, X., K. Alharbi, P. Zhang, H. Qin, and X. Yue (2025). Explainable federated bayesian causal inference and its application in advanced manufacturing. arXiv preprint arXiv:2501.06077\/

    2025 arXiv

  56. [64]

    Wang, and W

    Xing, S., Y. Wang, and W. Liu (2025). Multi-dimensional anomaly detection and fault localization in microservice architectures: A dual-channel deep learning approach with causal inference for intelligent sensing. Sensors (Basel, Switzerland)\/ 25\/ (11), 3396

    2025

  57. [65]

    Yang, D., G. Yu, J. Wang, Z. Wu, and M. Guo (2023). Reinforcement causal structure learning on order graph. In Proceedings of the AAAI Conference on Artificial Intelligence , Volume 37, pp.\ 10737--10744

    2023

  58. [66]

    Yang, L., H. Rao, M. Lin, Y. Xu, and P. Shi (2022). Optimal sensor scheduling for remote state estimation with limited bandwidth: A deep reinforcement learning approach. Information Sciences\/ 588 , 279--292

    2022

  59. [67]

    Xian, and C

    Yao, J., X. Xian, and C. Wang (2023). Adaptive sampling for monitoring multi-profile data with within-and-between profile correlation. Technometrics\/ 65\/ (3), 375--387

    2023

  60. [68]

    Yoav, B. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. JR Statist. Soc. B\/ 57 , 289--300

    1995

  61. [69]

    Zeng, Y., R. Cai, F. Sun, L. Huang, and Z. Hao (2024). A survey on causal reinforcement learning. IEEE Transactions on Neural Networks and Learning Systems\/

    2024

  62. [70]

    Zha, D., K.-H. Lai, M. Wan, and X. Hu (2020). Meta-aad: Active anomaly detection with deep reinforcement learning. In 2020 IEEE International Conference on Data Mining (ICDM) , pp.\ 771--780. IEEE

    2020

  63. [71]

    Zhang, C. and S. C. Hoi (2019). Partially observable multi-sensor sequential change detection: A combinatorial multi-armed bandit approach. In Proceedings of the AAAI Conference on Artificial Intelligence , Volume 33, pp.\ 5733--5740

    2019

  64. [72]

    Zhang, W. and Y. Mei (2023). Bandit change-point detection for real-time monitoring high-dimensional data under sampling control. Technometrics\/ 65\/ (1), 33--43

    2023

  65. [73]

    Aragam, P

    Zheng, X., B. Aragam, P. K. Ravikumar, and E. P. Xing (2018). Dags with no tears: Continuous optimization for structure learning. Advances in neural information processing systems\/ 31

    2018

  66. [74]

    Ng, and Z

    Zhu, S., I. Ng, and Z. Chen (2019). Causal discovery with reinforcement learning. arXiv preprint arXiv:1906.04477\/

    2019 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.