Pith. sign in

REVIEW 4 major objections 5 minor 35 references

Competitive Multi-Agent Deep Reinforcement Learning with Counterfactual Thinking

T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Counterfactual-thinking agents out-earn MADDPG and COMA rivals in competitive multi-agent games.

desk verdict A sign error in Algorithm 1 reverses the CFT agent's action-selection rule, so the paper's reported wins cannot validate the method as written. read the letter →

arxiv 1908.04573 v2 pith:G4M2M5LT submitted 2019-08-13 cs.LG cs.AIcs.MAstat.ML

Yue Wang , Yao Wan , Chenwei Zhang , Lixin Cui , Lu Bai , Philip S. Yu This is my paper · ORCID
classification cs.LGcs.AIcs.MAstat.ML
keywords multi-agentreinforcementlearningcounterfactualthinkingactor-criticregretparallelpoliciescompetitivegamesscenario-baseddeep
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a multi-agent reinforcement learning agent becomes more competitive if it mimics human counterfactual thinking: before acting, the agent generates several candidate 'intent' actions in parallel, estimates the reward each would have produced, and computes a regret for every intent it did not choose. The agent then picks its actual action from those regrets, and the regrets are continually revised using the gap between estimated and received rewards. If the claim holds, a Counterfactual Thinking (CFT) agent accumulates more reward in the same number of episodes than opponents trained with MADDPG or CMPG, with only a modest increase in computation. The paper supports the claim with experiments on a simulated underwater pursuit game and on pricing tasks built from real retail and hotel datasets.

What carries the argument

The load-bearing object is the scenario-regret matrix $M = \{m_{k,l}\}$ of size $K \times L$, which pairs K parallel policies with L state-scenarios obtained by clustering observed states. Each entry $m_{k,l}$ is the prior regret of policy k under scenario l, and the actor uses these regrets to weight the intent actions (sampling with a SOFTMIN distribution or taking a regret-weighted sum). The critic supplies the counterfactual Q-value $q^k_i$ for each intent, computes the posterior regret $m^{*}_{k,l} = \max(q_i) - q^k_i$, and updates the matrix toward that posterior via KL divergence. This mechanism lets the agent evaluate actions it never executed and search several policy subspaces at once.

What would settle it

Take a trained CFT agent and freeze the opponent policies. On a fresh set of states, execute each of the K intent actions and record the true subsequent rewards. If the rank order of intents by the critic's counterfactual Q-values does not match the rank order by true rewards, the regret signal is not tracking reality, and replacing the regret-weighted selection with uniform random selection among intents should preserve performance.

Watch

Extended reading notes

Core claim

The central claim is that a single agent can be made more competitive by a K-parallel policy layer that emits K intent actions for a state, evaluates all of them with a centralized critic, and records in a scenario-regret matrix how much worse each intent is than the best intent for the current scenario. The regret for intent k is defined as $m^{*}_{k,l} = \max(q_i) - q^k_i$, where $q^k_i = Q(s, a_1, \ldots, I^k_i, \ldots, a_N)$ is the critic's estimate of the counterfactual Q-value when agent $i$ takes intent $k$ while the other agents' actions are held fixed. The actor selects or samples actions using these regrets, and the critic updates both the Q-function and the regret matrix, so the two components supervise each other in a max-min training loop. The paper reports that in the tested environments the CFT agent obtains more accumulative reward than its opponents, both when replacing a DDPG-based agent and in direct competition between CFT and CMPG agents.

Load-bearing premise

The method assumes the centralized critic can accurately estimate the reward an intent action would have produced even when that intent was never actually executed in the environment; if those estimates are noisy, the regret-weighted action choice is driven by noise and the agent degrades to random behavior.

Editorial extensions

If this is right

  • A CFT agent should accumulate more reward than a MADDPG agent within the same number of episodes in mixed cooperative-competitive tasks, because the parallel intent search covers more policy subspaces per iteration.
  • In competitive pricing markets, a seller or hotel brand that switches to CFT policies should improve its sales-rank position relative to competitors that keep using MADDPG, given the same historical demand data.
  • Any off-policy actor-critic MARL method can be augmented with a parallel intent layer and scenario-regret matrix without replacing the centralized critic, so the mechanism is a drop-in component rather than a new training paradigm.
  • Training time grows linearly with the number of intent policies $K$, so the competitive advantage does not require exponential extra computation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The regret update is a form of baseline normalization inside the actor: subtracting $\max(q_i)$ from each intent's Q-value stabilizes the policy gradient, so part of the reported gain may come from variance reduction rather than from genuinely counterfactual reasoning.
  • The scenario clustering suggests a transfer opportunity: regrets learned under one scenario could be used to initialize policies in similar scenarios of a different task, e.g. from simulated pursuit-evasion to real pricing.
  • If the critic's estimates for never-executed intents are accurate, the method effectively performs off-policy evaluation at decision time, which would make it applicable beyond competitive games to any sequential decision problem with a learned world model.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes the Counterfactual Thinking (CFT) agent for competitive multi-agent deep reinforcement learning. The CFT actor maintains K parallel policies that generate K intent actions, and a scenario-regret matrix is used to select among these intents. The CFT critic, a centralized critic, evaluates each intent with a Q-value and computes a regret as the gap between the maximum Q-value and each intent's Q-value. The paper claims that this regret-based mechanism lets agents explore more policy subspaces, converge faster, and earn more cumulative reward than MADDPG and CMPG/COMA opponents in a multi-agent water-world environment and in a multi-seller marketing environment built on RETAIL and HOTEL datasets. The central empirical claim is that CFT agents are more competitive than their opponents under fair information.

Significance. If the proposed mechanism worked as claimed, the idea of deliberately generating multiple intents and using a centralized critic to estimate counterfactual regrets would be a practically interesting contribution to competitive MARL, potentially improving sample efficiency and policy-space exploration. The paper also attempts to validate the method on real-world retail and hotel datasets, which is valuable in principle. However, the contribution is not established by the manuscript as written: the core action-selection algorithm contradicts the stated minimum-regret decision rule, and the critic's training objective uses a KL divergence on vectors that are not probability distributions. These are load-bearing technical issues, and the reported experiments cannot be interpreted as evidence for the described CFT mechanism. The paper also lacks the experimental detail needed to reproduce or evaluate the results.

major comments (4)
  1. [Section III-B, Algorithm 1 (line 8) and Eq. (16)] The deterministic branch of Algorithm 1 outputs a = Σ_{k∈[1,K]} m_{k,l} I_k, weighting each intent action by its raw regret. Since Eq. (16) defines regret as m*_{k,l} = max(q_i) − q_k_i, higher regret means the intent is worse than the best intent. Weighting by m therefore amplifies exactly the intents the agent should avoid, contradicting the text's statement that the actor 'outputs the intent with the minimum regret as the final action' and the ε-branch's softmin behavior, which gives the minimum-regret intent the largest weight. Moreover, the m_{k,l} values are not normalized, so the deterministic output is not a valid convex combination and its magnitude depends on the arbitrary sum of regrets. This is a central internal inconsistency: if the implementation follows Algorithm 1, the method should select bad actions; if it deviates from the text, the described mechanism is not what was tested. Either way, the experiments cannot validate the claimed CFT mechanism.
  2. [Section III-C, Eq. (17)] The critic's objective minimizes λ|q^{t−1}_i − q^t_i|^2/n + (1−λ)KL(m_{:,l}, m*_{:,l}). The KL term requires both arguments to be probability distributions, but m* from Eq. (16) is a vector of nonnegative Q-value gaps that does not sum to 1, and m_{:,l} is a raw regret vector from the scenario-regret matrix that is also not normalized. KL divergence is undefined for such inputs. The paper calls these 'regret distributions' but provides no normalization or justification, so the critic training objective is not fully specified and the max-min training procedure is incomplete.
  3. [Section III-C, Eq. (15) and surrounding text] The entire counterfactual regret signal is computed from Q-values Q(s, a_1, ..., I_k, ..., a_N) for K intents, but only one of these intents is ever executed. The critic is therefore asked to evaluate action vectors that are off-policy and may lie far outside the training distribution. The paper provides no analysis, diagnostic, or empirical evidence that these extrapolated Q-values are accurate enough for the regret values to be meaningful. If they are noisy or biased, regret-based action selection simply amplifies critic approximation error. Since the claimed competitive advantage depends directly on this extrapolation, this is a load-bearing correctness-risk concern that is not addressed.
  4. [Section IV, Table II and Figures 3-5] Because Algorithm 1's deterministic branch is inconsistent with the stated minimum-regret decision rule, the experimental results in Table II and Figures 3-5 cannot be attributed to the described CFT mechanism. The paper also omits crucial experimental details: the ε decay schedule, learning rates, network architectures, number of random seeds, and standard deviations are reported only partially (Table II), and the exploration settings are described only briefly. These omissions make the reported advantages irreproducible and prevent a reader from determining whether the results would hold with the corrected algorithm.
minor comments (5)
  1. [Equation (12)] The formula r′_i(s,a) = (1−α)r_i(s,a) + α−r̂_i(s,a)/(N−1) appears garbled; it should likely be r′_i = (1−α)r_i + α(−r̂_i)/(N−1). The following sentence about α > 0.5 is also inconsistent with that expression and needs clarification.
  2. [Equation (7)] The return expression R^t_i uses s_{t+1}, s_{t+3}, and action a in several places; the indices should be consistent (e.g., s_{t+1} with a_{t+1}, s_{t+2} with a_{t+2}) for a standard discounted-return definition.
  3. [Section IV-C, cross-references] The text refers to 'Table 2' when the relevant table is Table II, and it refers to 'Figure 3' and 'Figure 4' in places discussing Figures 4 and 5; these cross-references should be corrected to not confuse the reader.
  4. [Section IV-A, baseline naming] The baseline is introduced as CMPG but later figures and text use 'COMA' interchangeably; the manuscript should clarify whether CMPG and COMA are the same method and consistently use one name.
  5. [Section IV, training setup] Hyperparameters for the environments (learning rate, network sizes, ε schedule, batch size, random seeds, number of trials) are not fully reported outside the scalability experiment, limiting reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: internal regret bootstrap, external reward validation.

full rationale

The paper's derivation chain is self-contained. The regret signal m*_{k,l} = max(q_i) - q_k_i (Eq. 16) is computed from the centralized critic's Q estimates for the K intent actions (Eq. 15), and the actor is trained by arg max q_i (Eq. 14) while the critic is fitted to the TD target of Eq. 11 (Eq. 17). This is an internal bootstrap of exactly the kind used in standard actor-critic and off-policy RL, not a prediction that is forced by its inputs in the sense of self-definitional circularity. The central empirical claim—CFT agents obtain more cumulative reward than opponents—is evaluated against the environment's external reward signal in Section IV, so it is not an artifact of the internal definitions. I checked the enumerated patterns: there are no load-bearing self-citations; the cited prior work (MADDPG [8], DDPG [23], CMPG [9]) supplies standard machinery and is not used to justify the CFT mechanism; no uniqueness theorem is imported; and the 'counterfactual' framing, while motivated psychologically, is implemented with explicit equations rather than by renaming a known result. Note: Algorithm 1's deterministic branch (a = Σ m_{k,l} I_k) appears to select high-regret intents if m is interpreted as in Eq. 16, but that is an internal consistency/correctness concern, not a circularity, and does not affect this scoring.

Assumptions & free parameters 6 free parameters · 4 assumptions · 1 invented entities

The paper introduces a scenario-regret matrix as a new model component, and it relies on several unstated or unjustified numerical choices (L, K, alpha, lambda, epsilon, gamma). The critic's generalization to never-executed intents is a domain assumption central to the method. The KL loss is not mathematically well-defined as stated.

free parameters (6)
  • Scenario number L = 16
    Set to 16 in Sections IV-B and IV-D; no justification or sensitivity analysis.
  • Parallel policy number K = 4 (MAWW), 6 (MSM)
    Chosen per environment; no sweep or justification.
  • Competition weight alpha (Eq. 12) = not reported
    Controls the degree of competition in reward shaping; value never stated.
  • Loss trade-off lambda (Eq. 17) = not reported
    Balances TD loss and KL regret loss; value never stated.
  • Random degree epsilon (Algorithm 1) = not reported
    Epsilon-greedy parameter in action selection; not specified.
  • Discount factor gamma = not reported
    Standard RL discount; not given in the paper.
assumptions (4)
  • domain assumption The multi-agent environment is a Markov game with fixed transition function T and rewards r_i.
    Section II-A defines the problem as a Markov game; requires stationarity and Markov property.
  • domain assumption The centralized critic Q(s, a1, ..., aN) generalizes correctly to arbitrary intent actions that may never have been executed.
    Equation 15 computes q_k_i for each intent; the regret matrix depends on these extrapolated values.
  • ad hoc to paper The KL divergence in Eq. 17 is well-defined for regret vectors m and m* even though m is not normalized as a probability distribution.
    Regret values m_{k,l} in [0,1] are not normalized; KL requires normalized distributions. The loss is therefore not a proper KL divergence.
  • ad hoc to paper The reward shaping in Eq. 12 correctly implements the intended degree of competition, with alpha controlling own vs. others' rewards as described.
    The text states alpha > 0.5 means the agent cares more about its own reward, but the equation gives others' negative rewards more weight for alpha > 0.5, contradicting the text.
invented entities (1)
  • Scenario-regret matrix M
    purpose: Stores a regret value for each of K policies under each of L clustered scenarios; used to weight actions.
    Introduced as a new data structure in Section III-B; has no external validation and its update relies entirely on the critic's internal Q estimates.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Competitive Multi-Agent Deep Reinforcement Learning with Counterfactual Thinking." pith.science (2026). https://pith.science/paper/G4M2M5LT

@misc{pith2026190804573,
  author       = {Pith},
  title        = {Pith review of: Competitive Multi-Agent Deep Reinforcement Learning with Counterfactual Thinking},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/G4M2M5LT}},
  note         = {Machine review of arXiv:1908.04573}
}
read the original abstract

Counterfactual thinking describes a psychological phenomenon that people re-infer the possible results with different solutions about things that have already happened. It helps people to gain more experience from mistakes and thus to perform better in similar future tasks. This paper investigates the counterfactual thinking for agents to find optimal decision-making strategies in multi-agent reinforcement learning environments. In particular, we propose a multi-agent deep reinforcement learning model with a structure which mimics the human-psychological counterfactual thinking process to improve the competitive abilities for agents. To this end, our model generates several possible actions (intent actions) with a parallel policy structure and estimates the rewards and regrets for these intent actions based on its current understanding of the environment. Our model incorporates a scenario-based framework to link the estimated regrets with its inner policies. During the iterations, our model updates the parallel policies and the corresponding scenario-based regrets for agents simultaneously. To verify the effectiveness of our proposed model, we conduct extensive experiments on two different environments with real-world applications. Experimental results show that counterfactual thinking can actually benefit the agents to obtain more accumulative rewards from the environments with fair information by comparing to their opponents while keeping high performing efficiency.

Figures

Figures reproduced from arXiv: 1908.04573 by the authors.

Figure 1
Figure 1. Explore and exploit the environments as RL processes. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The framework counterfactual actor-critic reinforcement learning, where [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Comparison of accumulative rewards obtained by agents on MAWW environment. (a) directly use the framework of [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Comparison of different actors thinking with counterfactual actor-critic reinforcement learning on MSM with RETAIL [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Comparison of different actors thinking with counterfactual actor-critic reinforcement learning on MSM with HOTEL [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Comparison of scalability for all methods. [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

35 extracted references · 29 canonical work pages

  1. [9]

    Foerster, Gregory Farquhar, Triantafyllos Afouras, Nantas Nardelli, and Shimon Whiteson

    Jakob N. Foerster, Gregory Farquhar, Triantafyllos Afouras, Nantas Nardelli, and Shimon Whiteson. Counterfactual multi-agent policy gradients. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in ...

    work page 2018

  2. [1]

    Journal of Economic Dynamics and Control , 27(11):2207 – 2218, 2003

    Learning competitive pricing strategies by multi-agent reinforcement learning. Journal of Economic Dynamics and Control , 27(11):2207 – 2218, 2003. Computing in economics and finance

    work page 2003

  3. [2]

    P. J. ’t Hoen and J. A. La Poutr ´e. A decommitment strategy in a competitive multi-agent transportation setting. In Peyman Faratin, David C. Parkes, Juan A. Rodr ´ıguez-Aguilar, and William E. Walsh, editors, Agent-Mediated Electronic Commerce V . Designing Mechanisms and Systems , pages 56–72, Berlin, Heidelberg, 2004. Springer Berlin Heidelberg

    work page 2004

  4. [3]

    Deep reinforcement learning framework for autonomous driving

    Ahmad El Sallab, Mohammed Abdou, Etienne Perot, and Senthil Yoga- mani. Deep reinforcement learning framework for autonomous driving. CoRR, abs/1704.02532, 2017

    arXiv 2017

  5. [4]

    Learning exploration/exploitation strategies for single trajectory reinforcement learning

    Michael Castronovo, Francis Maes, Raphael Fonteneau, and Damien Ernst. Learning exploration/exploitation strategies for single trajectory reinforcement learning. In Proceedings of the Tenth European Workshop on Reinforcement Learning, EWRL 2012, Edinburgh, Scotland, UK, June, 2012, pages 1–10, 2012. 0 500 1000 1500 2000 2500 3000 3500 0 1 02 03 04 0 Time ...

    work page 2012

  6. [5]

    Papadimitriou and John N

    Christos H. Papadimitriou and John N. Tsitsiklis. The complexity of markov decision processes. Mathematics of Operations Research , 12(3):441–450, 1987

    work page 1987

  7. [6]

    Littman, and Andrew W

    Leslie Pack Kaelbling, Michael L. Littman, and Andrew W. Moore. Reinforcement learning: A survey. J. Artif. Intell. Res., 4:237–285, 1996

    1996

  8. [7]

    Foerster, Yannis M

    Jakob N. Foerster, Yannis M. Assael, Nando de Freitas, and Shimon Whiteson. Learning to communicate with deep multi-agent reinforce- ment learning. In Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain , pages 2137–2145, 2016

    work page 2016

Show all 35 references
  1. [8]

    Multi-agent actor-critic for mixed cooperative-competitive environments

    Ryan Lowe, Yi Wu, Aviv Tamar, Jean Harb, Pieter Abbeel, and Igor Mordatch. Multi-agent actor-critic for mixed cooperative-competitive environments. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4-9 Dec...

    2017

  2. [10]

    Counterfactual data- fusion for online reinforcement learners

    Andrew Forney, Judea Pearl, and Elias Bareinboim. Counterfactual data- fusion for online reinforcement learners. In Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017 , pages 1156–1164, 2017

    2017

  3. [11]

    Gupta, Maxim Egorov, and Mykel J

    Jayesh K. Gupta, Maxim Egorov, and Mykel J. Kochenderfer. Co- operative multi-agent control using deep reinforcement learning. In Autonomous Agents and Multiagent Systems - AAMAS 2017 Workshops, Best Papers, S ˜ao Paulo, Brazil, May 8-12, 2017, Revised Selected Papers, pages 6...

    2017

  4. [12]

    Reinforcement Learning and Markov Decision Processes , pages 3–42

    Martijn van Otterlo and Marco Wiering. Reinforcement Learning and Markov Decision Processes , pages 3–42. Springer Berlin Heidelberg, Berlin, Heidelberg, 2012

    2012

  5. [13]

    Hausknecht and Peter Stone

    Matthew J. Hausknecht and Peter Stone. Deep recurrent q-learning for partially observable mdps. In 2015 AAAI Fall Symposia, Arlington, Virginia, USA, November 12-14, 2015 , pages 29–37, 2015

    2015

  6. [14]

    Reinforcement learning to play an optimal nash equilibrium in team markov games

    Xiaofeng Wang and Tuomas Sandholm. Reinforcement learning to play an optimal nash equilibrium in team markov games. In Advances in Neural Information Processing Systems 15 [Neural Information Pro- cessing Systems, NIPS 2002, December 9-14, 2002, Vancouver, British Columbia, Ca...

    2002

  7. [15]

    An algorithm for distributed reinforcement learning in cooperative multi-agent systems

    Martin Lauer and Martin Riedmiller. An algorithm for distributed reinforcement learning in cooperative multi-agent systems. In In Proceedings of the Seventeenth International Conference on Machine Learning, pages 535–542. Morgan Kaufmann, 2000

    2000

  8. [16]

    Reinforcement Learning and Dynamic Programming Using Function Approximators

    de Schutter B Busoniu L, Babuka R and Ernst D. Reinforcement Learning and Dynamic Programming Using Function Approximators . FL: CRC Pressp, Boca Ratone, 2010

    2010

  9. [17]

    Christopher J. C. H. Watkins and Peter Dayan. Technical note q-learning. Machine Learning, 8:279–292, 1992

    1992

  10. [18]

    Sutton, David A

    Richard S. Sutton, David A. McAllester, Satinder P. Singh, and Yishay Mansour. Policy gradient methods for reinforcement learning with function approximation. In Advances in Neural Information Processing Systems 12, [NIPS Conference, Denver, Colorado, USA, November 29 - Decemb...

    1999

  11. [19]

    H. J. Kushner and G. Yin. Stochastic approximation and recursive algorithms and applications . Springer, 2003

    2003

  12. [20]

    Riedmiller

    David Silver, Guy Lever, Nicolas Heess, Thomas Degris, Daan Wierstra, and Martin A. Riedmiller. Deterministic policy gradient algorithms. In Proceedings of the 31th International Conference on Machine Learning, ICML 2014, Beijing, China, 21-26 June 2014 , pages 387–395, 2014

    2014

  13. [21]

    Multi-agent sys- tems by incremental gradient reinforcement learning

    Alain Dutech, Olivier Buffet, and Franc ¸ois Charpillet. Multi-agent sys- tems by incremental gradient reinforcement learning. In Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence, IJCAI 2001, Seattle, Washington, USA, August 4-10, 2001 , ...

    2001

  14. [22]

    Riedmiller

    V olodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin A. Riedmiller. Playing atari with deep reinforcement learning. CoRR, abs/1312.5602, 2013

    2013 arXiv

  15. [23]

    Lillicrap, Jonathan J

    Timothy P. Lillicrap, Jonathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. Continuous control with deep reinforcement learning. In 4th International Confer- ence on Learning Representations, ICLR 2016, San Juan, Puerto Ric...

    2016

  16. [24]

    Carlos Garcia-Monco, Elena Astigarraga, Ainara Gonzalez, and Jordan Grafman

    Marian Gomez Beldarrain, J. Carlos Garcia-Monco, Elena Astigarraga, Ainara Gonzalez, and Jordan Grafman. Only spontaneous counterfactual thinking is impaired in patients with prefrontal cortex lesions. Cognitive Brain Research, 24(3):723–726, 8 2005

    2005

  17. [25]

    R. Polikar. Ensemble based systems in decision making. IEEE Circuits and Systems Magazine , 6(3):21–45, Third 2006

    2006

  18. [26]

    A softmin-based neural model for causal reason- ing

    Lotfi Ben Romdhane. A softmin-based neural model for causal reason- ing. IEEE Trans. Neural Networks , 17(3):732–744, 2006

    2006

  19. [27]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In 3rd International Conference on Learning Represen- tations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015

    2015

  20. [28]

    Stewart, and Jimeng Sun

    Edward Choi, Siddharth Biswal, Bradley Malin, Jon Duke, Walter F. Stewart, and Jimeng Sun. Generating multi-label discrete patient records using generative adversarial networks. In Proceedings of the Machine Learning for Health Care Conference, MLHC 2017, Boston, Massachusetts...

    2017

  21. [29]

    Cooperative multi-agent control using deep reinforcement learning

    Jayesh K Gupta, Maxim Egorov, and Mykel Kochenderfer. Cooperative multi-agent control using deep reinforcement learning. In International Conference on Autonomous Agents and Multiagent Systems , pages 66–

  22. [30]

    Ilya Sutskever, Oriol Vinyals, and Quoc V . Le. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, December 8-13 2014, Montreal, Quebec, Canada, pages 310...

    2014

  23. [31]

    Causal inference in statistics: An overview

    Judea Pearl. Causal inference in statistics: An overview. Statist. Surv., 3:96–146, 2009

    2009

  24. [32]

    Causal inference based on counterfactuals

    Hfler M. Causal inference based on counterfactuals. BMC Med Res Methodol, 28, 5 2005

    2005

  25. [33]

    Jack S. Levy. Counterfactuals, causal inference, and historical analysis. Security Studies, 24, 07 2015

    2015

  26. [34]

    Deep reinforcement learning for multi-agent systems: A review of challenges, solutions and applications

    Thanh Thi Nguyen, Ngoc Duy Nguyen, and Saeid Nahavandi. Deep reinforcement learning for multi-agent systems: A review of challenges, solutions and applications. CoRR, abs/1812.11794, 2018

    2018 arXiv

  27. [35]

    WOLPERT and KAGAN TUMER.Optimal Payoff Functions for Members of Collectives , pages 355–369

    DA VID H. WOLPERT and KAGAN TUMER.Optimal Payoff Functions for Members of Collectives , pages 355–369

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.