REVIEW 4 major objections 3 minor 48 references
Causal Policy Learning in Reinforcement Learning: Backdoor-Adjusted Soft Actor-Critic
T0 review · 4 major / 3 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read DoSAC: RL policies that act on causal effects, not correlations.
desk verdict DoSAC's backdoor adjustment is built on a conditional-marginal mix-up: Eq. (4) samples the past from p(z|s), so the method estimates the observational policy, not the interventional one. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The backdoor adjustment formula is the load-bearing identity: $\pi(a|\mathrm{do}(s)) = \mathbb{E}_{p(a_{t-1}, s_{t-1})}[p(a|s, a_{t-1}, s_{t-1})]$, which turns an intervention into an average of observational conditionals. DoSAC's new component is the Backdoor Reconstructor, a neural network $p_\phi(a_{t-1}, s_{t-1}|s_t)$ trained to produce pseudo-past variables from the current state, which the paper treats as a sampling proxy for the marginal $p(a_{t-1}, s_{t-1})$. Together with the Interventional Actor $p_\theta(a_t|s_t, a_{t-1}, s_{t-1})$, the pair enables forward sampling from $\pi(a|\mathrm{do}(s))$ using replay-buffer data.
What would settle it
In a synthetic MDP with a known latent confounder, compute the true $\pi(a|\mathrm{do}(s))$ by physically intervening on the state, then compare it with the distribution produced by DoSAC's two-stage sampler (Reconstructor followed by Interventional Actor). If the two distributions diverge systematically as the dependence of $s_t$ on $(a_{t-1}, s_{t-1})$ increases, the reconstructor is not implementing backdoor adjustment and the central claim is falsified.
Extended reading notes
Core claim
The central claim is that the interventional policy $\pi(a_t|\mathrm{do}(s_t))$ can be estimated in an off-policy actor-critic setting without access to true confounders or causal labels. Using the backdoor criterion, the paper writes $\pi(a_t|\mathrm{do}(s_t)) = \mathbb{E}_{p(a_{t-1}, s_{t-1})}[p(a_t|s_t, a_{t-1}, s_{t-1})]$, and replaces the marginal over the past with samples from a learned conditional $p_\phi(a_{t-1}, s_{t-1}|s_t)$, called the Backdoor Reconstructor. Actions are then drawn by first sampling a pseudo-past pair and then sampling from the conditional actor. The paper claims that this two-stage sampling estimates the interventional policy, and that the resulting soft actor-critic objective with causal entropy generalizes SAC and recovers it exactly in clean environments.
Load-bearing premise
The load-bearing premise is that sampling pseudo-past variables from the learned conditional distribution $p_\phi(a_{t-1}, s_{t-1}|s_t)$ is an acceptable stand-in for the marginal $p(a_{t-1}, s_{t-1})$ required by the backdoor formula; because the current state is itself caused by the past, these two distributions generally differ, and if the difference is large the algorithm is not actually estimating the interventional policy.
Editorial extensions
If this is right
- Policies learned with DoSAC are invariant to the confounder that generated the training data, so they transfer to settings where the confounder is absent or different.
- The method integrates into standard SAC pipelines with no additional data requirements, only two extra learned distributions.
- When no confounders are present, DoSAC reduces exactly to SAC, so the causal machinery introduces no overhead in clean environments.
- The causal entropy bonus $H(a|\mathrm{do}(s))$ replaces the standard entropy bonus and drives exploration toward actions that are robust to confounding.
- Empirically, DoSAC reports higher returns than SAC and the ACE baseline in confounded continuous-control benchmarks, both under persistent confounding and in clean evaluations.
Reading between the lines
- Editorial extension: the identification step replaces a marginal with a conditional, so the unbiasedness result is an empirical hypothesis unless the proxy is shown to be faithful; the paper's own Lemma 2 only covers the clean case.
- Editorial extension: the same Backdoor Reconstructor could be dropped into other off-policy actor-critic algorithms, since the adjustment is not specific to the SAC objective.
- Editorial extension: the experiments inject additive Gaussian confounders; testing temporally correlated or structured confounders would show whether the method handles the non-stationary confounding the paper lists as a limitation.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes DoSAC, an extension of Soft Actor-Critic that aims to learn the interventional policy π(a|do(s)) rather than the observational policy π(a|s), in order to remove bias from hidden confounders in off-policy RL. The method introduces a learnable Backdoor Reconstructor q_φ(a_{t-1},s_{t-1}|s_t) that samples pseudo-past variables, uses these samples in a backdoor-adjustment formula, and adds a causal entropy regularizer to the SAC objective. The authors report experiments on continuous control benchmarks with synthetically injected confounding, comparing DoSAC against SAC, ACE, TD3, DDPG, PPO, and RPO, and claim both theoretical guarantees (Lemmas 1 and 2) and improved robustness and generalization.
Significance. If the central claim were correct, DoSAC would be a practically valuable contribution: a method that estimates interventional policies from observational replay data without access to true confounders or causal labels, integrated into a standard actor-critic framework. The paper addresses an important problem, and the experimental comparison covers several baselines and a sensitivity analysis over confounder strength. However, the load-bearing estimation step in Section 4.3 is not a valid backdoor adjustment: the implemented sampler replaces the required marginal p(a_{t-1},s_{t-1}) with a conditional q_φ(a_{t-1},s_{t-1}|s_t), and the causal entropy used in the Q-target is the entropy of the conditional actor rather than of the interventional policy. The paper therefore does not establish that DoSAC estimates π(a|do(s)), and the empirical results do not compensate for this because no experiment checks the estimated quantity against a known causal target. The theoretical and algorithmic core would need to be redesigned, not locally corrected.
major comments (4)
- [Section 4.3, Eq. (4) and Algorithm 1] The implemented estimator does not compute the backdoor-adjusted policy. Eq. (3) requires an expectation over the marginal p(a_{t-1},s_{t-1}). Eq. (4) is a valid marginalization only if s̃_t is drawn from p(s̃_t) and then (a_{t-1},s_{t-1}) is drawn from p(·|s̃_t). The paper instead says it approximates this by setting s̃_t = s_t and learns q_φ(a_{t-1},s_{t-1}|s_t), and Algorithm 1 draws (ã, s̃) ∼ q_φ(·|s) for the query state s only. Since s_t is a descendant of (a_{t-1},s_{t-1}) under the assumed dynamics, q_φ(z|s) is generally unequal to p(z), and the action distribution produced is ∫ p_θ(a|s,z) q_φ(z|s) dz, which equals the observational p(a|s) when the learned conditionals match the data, not π(a|do(s)). Conditioning on the current state thus reintroduces the dependence that backdoor adjustment is supposed to remove. Section 6 concedes that pseudo-past inference 'may prove insufficient,' but the issue here is not approximation error; it is that Eq. (4) is not implemented as stated. This is the load-bearing step of the paper, so the claim that DoSAC estimates interventional policies is unsupported.
- [Section 4.3, Eqs. (7)–(9)] The quantity used as causal entropy in the Q-target is not the entropy of the interventional policy. In Eq. (8), the bonus is α log p(ã′|s′, s̃, ã), where (s̃, ã) ∼ q_φ(·|s′) and ã′ ∼ p_θ(·|s′, s̃, ã). This is the log-density of the conditional actor for one sampled z, not log π(ã′|do(s′)), whose density under Algorithm 1 is the mixture ∫ p_θ(a|s′,z) q_φ(z|s′) dz. Consequently the objective optimized by the algorithm is not Eq. (2), and H(a|do(s)) from Eq. (1) is never actually computed. This compounds the first issue throughout the policy evaluation and policy improvement steps.
- [Section 4.3, 'Training the model'] No loss function or training objective is given for the Backdoor Reconstructor q_φ. The text states it is 'a neural network trained to approximate the inverse dynamics,' and Algorithm 1 relies on samples from q_φ, but the only explicit loss in the paper is the Q-loss in Eq. (9), which does not constrain q_φ to estimate p(z|s) or any marginal p(z). Without a defined objective for q_φ, the procedure is underspecified and cannot be reproduced or evaluated as written.
- [Section 5 and Appendix A] The empirical evaluation does not test whether the learned policy equals or approximates the interventional policy. Confounding is injected as additive action noise and methods are compared by return, but there is no experiment with a known causal target—for example, a synthetic SCM where π(a|do(s)) can be computed exactly—or a check that the backdoor adjustment removes a specific spurious correlation. Because the theory in Section 4 is in question, the improved returns alone do not establish that any causal adjustment is taking place; they could result from the added entropy regularizer or extra model capacity.
minor comments (3)
- [Table 2 caption] The caption says 'All agents were trained in the presence of hidden confounders, but evaluated in clean environments,' which appears to be copied from Table 1; Table 2 reports evaluation with confounders, so the caption and the surrounding sentence should be corrected.
- [Section 5 and Eq. (2)] There are typographical errors: 'Humanroid' should be 'Humanoid' in Section 5, and 'inerventional' should be 'interventional' in Eq. (2).
- [Eq. (4)] The notation is ambiguous: the dummy variable s̃_t is not distinguished from the query state s_t, and it is unclear how the outer expectation over p(s̃_t) is estimated from the replay buffer; this should be clarified independently of the correctness issue.
Circularity Check
DoSAC's 'interventional policy' is the observational policy by construction: Eq. (4) replaces the marginal p(a_{t-1},s_{t-1}) required by Eq. (3) with the conditional p_phi(·|s_t), so Algorithm 1 samples ∫ p(a|s,z)q(z|s)dz = p(a|s).
-
self definitional
[Section 4.3, Eq. (4) and Algorithm 1 (see also Eq. (8))]
"To approximate the marginal distribution p(a_{t-1}, s_{t-1}), we leverage a model-based approach: we infer pseudo-past variables from the current state. Specifically, we define a dummy variable \tilde{s}_t representing the current state, and write: p(a_{t-1}, s_{t-1}) = \int p(a_{t-1}, s_{t-1}|\tilde{s}_t)p(\tilde{s}_t)d\tilde{s}_t. (4) In practice, we approximate this by setting \tilde{s}_t = s_t using states sampled from the replay buffer, and learn a conditional model p_\phi(a_{t-1}, s_{t-1}|s_t) to predict the pseudo-past from the current state."
Eq. (3) defines the interventional policy as E_{p(a_{t-1},s_{t-1})}[p(a_t|s_t,a_{t-1},s_{t-1})], which requires the marginal of the past variables. Eq. (4) rewrites that marginal as \int p(a_{t-1},s_{t-1}|\tilde{s}_t)p(\tilde{s}_t)d\tilde{s}_t and then, "in practice," sets \tilde{s}_t=s_t and learns q_\phi(\cdot|s_t) from replay data. Algorithm 1 therefore draws a sample via \int p_\theta(a|s,z)q_\phi(z|s)dz. When q_\phi and p_\theta are the true observational conditionals, this equals \int p(a,z|s)dz = p(a|s) by the law of total probability. Thus the object called \pi(a|do(s)) is, by construction, the observational policy \pi(a|s): conditioning on s_t reintroduces the very backdoor path the adjustment was meant to block. Eq.
full rationale
The core circularity is in the construction of the interventional sampler. The backdoor formula (Eq. 3) requires the marginal p(a_{t-1},s_{t-1}); Eq. (4) introduces that marginal through an integral over a dummy state \tilde{s}_t, but the implementation fixes \tilde{s}_t to the query state s_t and fits q_\phi(a_{t-1},s_{t-1}|s_t) to observational replay data. Algorithm 1's forward sampling then implements \int p_\theta(a|s,z)q_\phi(z|s)dz, which, at the true conditionals, is exactly p(a|s). So the quantity named \pi(a|do(s)) is mathematically the same as the observational policy \pi(a|s) that Section 1 says DoSAC is designed to correct. The same substitution appears in Eq. (8), where the alleged causal-entropy bonus is the log of a conditional density drawn from the fitted model. The experiments compare DoSAC against external baselines (SAC, TD3, ACE, etc.) and are not themselves circular; the lemmas are standard SAC arguments restated with do(s), and no load-bearing self-citation chain is present. The circularity is confined to, but fatal for, the causal identification claim: no experiment checks the estimated do-policy against a known interventional distribution, and the conclusion's limitation that pseudo-past inference "may prove insufficient" understates the issue, since by construction the adjusted sampler is the observational policy rather than an approximation to it. Score 6: the central causal prediction reduces by construction, while the empirical comparison retains independent content.
Assumptions & free parameters
free parameters (2)
- Confounder standard deviation sigma =
1.0 (Ant, Walker2d, Humanoid), 0.2 (LunarLander)
- Confounder mean mu =
0.0
assumptions (4)
- domain assumption The causal graph in Figure 1(a) has (a_{t-1}, s_{t-1}) as a set that blocks all backdoor paths from s_t to a_t.
- ad hoc to paper The learned conditional p_phi(a_{t-1}, s_{t-1}|s_t) is a good approximation to the marginal p(a_{t-1}, s_{t-1}) needed in Eq. (3).
- standard math Do-calculus Rule 2 applies, so pi(a|do(s)) = pi(a|s) when no hidden confounders exist.
- standard math The objective J(pi) = E_s E_{a~pi(a|do(s))}[Q - alpha log pi(a|do(s))] is strictly concave in pi at fixed Q.
invented entities (2)
-
Pseudo-past variables (s~_{t-1}, a~_{t-1})
-
Backdoor Reconstructor
Cite this review
Pith. "Pith review of Causal Policy Learning in Reinforcement Learning: Backdoor-Adjusted Soft Actor-Critic." pith.science (2026). https://pith.science/paper/7K6RHFIN
@misc{pith2026250605445,
author = {Pith},
title = {Pith review of: Causal Policy Learning in Reinforcement Learning: Backdoor-Adjusted Soft Actor-Critic},
year = {2026},
howpublished = {\url{https://pith.science/paper/7K6RHFIN}},
note = {Machine review of arXiv:2506.05445}
}
abstract
Hidden confounders that influence both states and actions can bias policy learning in reinforcement learning (RL), leading to suboptimal or non-generalizable behavior. Most RL algorithms ignore this issue, learning policies from observational trajectories based solely on statistical associations rather than causal effects. We propose DoSAC (Do-Calculus Soft Actor-Critic with Backdoor Adjustment), a principled extension of the SAC algorithm that corrects for hidden confounding via causal intervention estimation. DoSAC estimates the interventional policy $\pi(a | \mathrm{do}(s))$ using the backdoor criterion, without requiring access to true confounders or causal labels. To achieve this, we introduce a learnable Backdoor Reconstructor that infers pseudo-past variables (previous state and action) from the current state to enable backdoor adjustment from observational data. This module is integrated into a soft actor-critic framework to compute both the interventional policy and its entropy. Empirical results on continuous control benchmarks show that DoSAC outperforms baselines under confounded settings, with improved robustness, generalization, and policy reliability.
Figures
Figures from the paper (1 more)
Reference graph
Works this paper leans on
-
[1]
Bareinboim, E., Forney, A., and Pearl, J. (2015). Bandits with unobserved confounders: A causal approach. Advances in Neural Information Processing Systems , 28
work page 2015
-
[2]
Bareinboim, E. and Pearl, J. (2016). Causal inference and the data-fusion problem. In Proceedings of the National Academy of Sciences
work page 2016
-
[3]
Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., and Zaremba, W. (2016). Openai gym. arXiv preprint arXiv:1606.01540
arXiv 2016
-
[4]
Cai, T., Li, H., Gu, Q., and Bareinboim, E. (2024). Learning by doing: An online causal reinforcement learning framework with causal-aware policy. In International Conference on Learning Representations (ICLR)
work page 2024
-
[5]
Forney, A. and Bareinboim, E. (2019). Counterfactual randomization: rescuing experimental studies from obscured confounding. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 33, pages 2454--2461
work page 2019
-
[6]
Fujimoto, S., Hoof, H., and Meger, D. (2018). Addressing function approximation error in actor-critic methods. In International conference on machine learning , pages 1587--1596. PMLR
work page 2018
-
[7]
Ghassami, A., Salehkaleybar, S., Kiyavash, N., and Bareinboim, E. (2018). Budgeted experiment design for causal structure learning. In International Conference on Machine Learning , pages 1724--1733. PMLR
work page 2018
-
[8]
Haarnoja, T., Zhou, A., Abbeel, P., and Levine, S. (2018). Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In International conference on machine learning , pages 1861--1870. PMLR
work page 2018
Show all 48 references
-
[9]
Huang, S., Dossa, R. F. J., Ye, C., Braga, J., Chakraborty, D., Mehta, K., and Araújo, J. G. (2022). Cleanrl: High-quality single-file implementations of deep reinforcement learning algorithms. Journal of Machine Learning Research , 23(274):1--18
2022
-
[10]
Jaber, A., Kocaoglu, M., Shanmugam, K., and Bareinboim, E. (2020). Causal discovery from soft interventions with unknown targets: Characterization and learning. Advances in neural information processing systems , 33:9551--9561
2020
-
[11]
Ji, T., Liang, Y., Zeng, Y., Luo, Y., Xu, G., Guo, J., Zheng, R., Huang, F., Sun, F., and Xu, H. (2024). Ace: Off-policy actor-critic with causality-aware entropy regularization. In International Conference on Machine Learning , pages 21620--21647. PMLR
2024
-
[12]
Jin, H., Peng, Y., Yang, W., Wang, S., and Zhang, Z. (2022). Federated reinforcement learning with environment heterogeneity. In International Conference on Artificial Intelligence and Statistics , pages 18--37. PMLR
2022
-
[13]
Kumor, D., Zhang, J., and Bareinboim, E. (2021). Sequential causal imitation learning with unobserved confounders. Advances in Neural Information Processing Systems , 34:14669--14680
2021
-
[14]
and Bareinboim, E
Lee, S. and Bareinboim, E. (2019). Structural causal bandits with non-manipulable variables. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 33, pages 4164--4172
2019
-
[15]
and Bareinboim, E
Lee, S. and Bareinboim, E. (2020). Characterizing optimal mixed policies: Where to intervene and what to observe. Advances in neural information processing systems , 33:8565--8576
2020
-
[16]
P., Hunt, J
Lillicrap, T. P., Hunt, J. J., Pritzel, A., et al. (2016). Continuous control with deep reinforcement learning. In ICLR
2016
-
[17]
P., Hunt, J
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2015). Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971
2015 arXiv
-
[18]
and Krishnamurthy, A
Lu, X. and Krishnamurthy, A. (2021). Sample-efficient reinforcement learning is not enough: Better exploration with conservative causal models. In ICML
2021
-
[19]
Lu, Y., Zhang, R., and Bareinboim, E. (2020). Sample-efficient reinforcement learning via counterfactual-based data augmentation. arXiv preprint arXiv:2002.11089
2020 arXiv
-
[20]
V., Sima, K., and Leong, T
Ma, H., Luo, Z., Vo, T. V., Sima, K., and Leong, T. (2025). Highly efficient self-adaptive reward shaping for reinforcement learning. In The Thirteenth International Conference on Learning Representations
2025
-
[21]
V., Fu, D., and Leong, T.-Y
Ma, H., Sima, K., Vo, T. V., Fu, D., and Leong, T.-Y. (2024a). Reward shaping for reinforcement learning with an assistant reward agent. In International Conference on Machine Learning , pages 33925--33939. PMLR
2024
-
[22]
V., and Leong, T.-Y
Ma, H., Vo, T. V., and Leong, T.-Y. (2024b). Mixed-initiative bayesian sub-goal optimization in hierarchical reinforcement learning. In Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems , pages 1328--1336
2024
-
[23]
Namkoong, H., Keramati, R., Yadlowsky, S., and Brunskill, E. (2020). Off-policy policy evaluation for sequential decisions under unobserved confounding. Advances in Neural Information Processing Systems , 33:18819--18831
2020
-
[24]
and Sontag, D
Oberst, M. and Sontag, D. (2019). Counterfactual off-policy evaluation with gumbel-max structural causal models. In International Conference on Machine Learning , pages 4881--4890. PMLR
2019
-
[25]
o lkopf, B., R \
Pace, A., Y \`e che, H., Sch \"o lkopf, B., R \"a tsch, G., and Tennenholtz, G. (2024). Delphic offline reinforcement learning under nonidentifiable hidden confounding. In Proceedings of the Twelfth International Conference on Learning Representations (ICLR)
2024
-
[26]
Pearl, J. (2009a). Causal inference in statistics: An overview. Statistics Surveys , 3
2009
-
[27]
Pearl, J. (2009b). Causality . Cambridge university press
2009
-
[28]
Rahman, M. M. and Xue, Y. (2022). Robust policy optimization in deep reinforcement learning. arXiv preprint arXiv:2212.07536
2022 arXiv
-
[29]
Ruan, K., Zhang, J., Di, X., and Bareinboim, E. (2023). Causal imitation learning via inverse reinforcement learning. In The Eleventh International Conference on Learning Representations
2023
-
[30]
Sch\" o lkopf, B. (2022). Causality for machine learning. In Probabilistic and Causal Inference: The Works of Judea Pearl , page 765–804. Association for Computing Machinery, New York, NY, USA, 1 edition
2022
-
[31]
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347
2017 arXiv
-
[32]
Shi, C., Uehara, M., Huang, J., and Jiang, N. (2022). A minimax learning approach to off-policy evaluation in confounded partially observable markov decision processes. In International Conference on Machine Learning , pages 20057--20094. PMLR
2022
-
[33]
A., Mehrjou, A., Itti, L., and Sch \"o lkopf, B
Sontakke, S. A., Mehrjou, A., Itti, L., and Sch \"o lkopf, B. (2021). Causal curiosity: Rl agents discovering self-supervised experiments for causal representation learning. In International conference on machine learning , pages 9848--9858. PMLR
2021
-
[34]
Sutton, R. S. and Barto, A. G. (2018). Reinforcement learning: An introduction . MIT press
2018
-
[35]
Swamy, G., Choudhury, S., Bagnell, D., and Wu, S. (2022). Causal imitation learning under temporally correlated noise. In International Conference on Machine Learning , pages 20877--20890. PMLR
2022
-
[36]
V., Bhattacharyya, A., Lee, Y., and Leong, T.-Y
Vo, T. V., Bhattacharyya, A., Lee, Y., and Leong, T.-Y. (2022a). An adaptive kernel approach to federated learning of heterogeneous causal effects. Advances in Neural Information Processing Systems , 35:24459--24473
2022
-
[37]
V., Lee, Y., Hoang, T
Vo, T. V., Lee, Y., Hoang, T. N., and Leong, T.-Y. (2022b). Bayesian federated estimation of causal effects from observational data. In Uncertainty in Artificial Intelligence , pages 2024--2034. PMLR
2022
-
[38]
Wang, L., Yang, Z., and Wang, Z. (2021). Provably efficient causal reinforcement learning with confounded observational data. In Advances in Neural Information Processing Systems , volume 34, pages 21164--21175
2021
-
[39]
T., and Athey, S
Xiong, R., Koenecke, A., Powell, M., Shen, Z., Vogelstein, J. T., and Athey, S. (2023). Federated causal inference in heterogeneous observational data. Statistics in Medicine , 42(24):4418--4439
2023
-
[40]
Yu, S., Fang, S., Peng, R., Qi, Z., Zhou, F., and Shi, C. (2024). Two-way deconfounder for off-policy evaluation in causal reinforcement learning. In Advances in Neural Information Processing Systems , volume 37
2024
-
[41]
Zhang, J. (2020). Designing optimal dynamic treatment regimes: A causal reinforcement learning approach. In International conference on machine learning , pages 11012--11022. PMLR
2020
-
[42]
and Bareinboim, E
Zhang, J. and Bareinboim, E. (2016). Markov decision processes with unobserved confounders: A causal approach. Purdue AI Lab, West Lafayette, IN, USA, Tech. Rep
2016
-
[43]
and Bareinboim, E
Zhang, J. and Bareinboim, E. (2019). Near-optimal reinforcement learning in dynamic treatment regimes. Advances in Neural Information Processing Systems , 32
2019
-
[44]
and Bareinboim, E
Zhang, J. and Bareinboim, E. (2021). Bounding causal effects on continuous outcome. In Proceedings of the AAAI Conference on Artificial Intelligence , pages 12207--12215
2021
-
[45]
and Bareinboim, E
Zhang, J. and Bareinboim, E. (2022a). Can humans be out of the loop? In Conference on Causal Learning and Reasoning , pages 1010--1025. PMLR
2022
-
[46]
and Bareinboim, E
Zhang, J. and Bareinboim, E. (2022b). Online reinforcement learning for mixed policy scopes. Advances in Neural Information Processing Systems , 35:3191--3202
2022
-
[47]
Zhang, J., Kumor, D., and Bareinboim, E. (2020). Causal imitation learning with unobserved confounders. Advances in neural information processing systems , 33:12263--12274
2020
-
[48]
Zhu, W., Yu, C., and Zhang, Q. (2023). Causal deep reinforcement learning using observational data. In Proceedings of the 32nd International Joint Conference on Artificial Intelligence (IJCAI) , pages 3771--3777
2023
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.