Pith. sign in

REVIEW 4 major objections 5 minor 12 references

The paper argues that treating opponents' intentions as hidden states, rather than observed actions, determines whether cooperation survives execution noise, and derives a noise threshold at which mutual intention inference collapses cooper

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 07:15 UTC pith:JALRZBKP

load-bearing objection A clean, honest POMDP formulation of intention inference in the noisy IPD; but the central 'representation matters' claim is not yet established because the POMDP sees the true noise level and the MDP doesn't. the 4 major comments →

arxiv 2608.02440 v1 pith:JALRZBKP submitted 2026-08-03 cs.GT cs.LG

Intention Inference Under Execution Noise: Separating Aleatoric and Epistemic Uncertainty in Social Dilemmas

Kival Mahadew , Jonathan Shock This is my paper
classification cs.GT cs.LG MSC 91A2091A26
keywords execution noiseintention inferencePOMDPactive inferenceIterated Prisoner's Dilemmaaleatoric vs epistemic uncertaintycritical noise thresholdDirichlet learning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

In noisy social dilemmas, an observed defection can mean betrayal or a slipped hand. Standard reinforcement-learning formulations treat the executed action as the state, so the agent cannot even frame the question, and systematically over-retaliates. This paper proposes a POMDP in which opponent intentions are latent states and actions are noisy observations, solved with active inference, and claims this representational choice is decisive: the POMDP outperforms the MDP against conditionally cooperative opponents at every noise level tested. The paper then derives a critical noise threshold, expressed as a fixed point on the learned cooperative prior, at which the cooperative hypothesis becomes the minority explanation over the planning horizon, and shows this threshold produces a sharp, correlated cooperation collapse in self-play. The value of intention inference is context-dependent: it helps exactly in games with a temptation to defect (T>R), and not in coordination games like Stag Hunt.

Core claim

The paper's central claim is that separating aleatoric execution noise (a known rate ε) from epistemic uncertainty about opponent intent is a structural prerequisite for principled forgiveness, not a modeling detail. A POMDP with latent intention states and a fixed, known observation matrix absorbs noise at the likelihood level, while the opponent's reactive policy is learned online via Dirichlet counts. From this, Proposition 1 gives the maximum number of observed defections attributable to noise over h steps, and the cooperation regime is governed by the cooperative survival probability p_t(CC)^h. When it crosses 0.5 at the fixed point ε̃ = 1 − p_∞(ε̃), the agent retaliates because hostile

What carries the argument

The key object is the latent-intention POMDP: hidden states are joint (self, opponent) intended actions, observations are the executed joint actions corrupted by a binary symmetric channel with known parameter ε (the A matrix), and the transition matrix B encodes the opponent's reactive policy, learned online through Dirichlet prior counts. The planning objective is the expected free energy, whose decomposition into pragmatic value, state information gain, and parameter information gain is what lets the agent both infer current intent and actively probe to learn how intent evolves. The analytical workhorse is Proposition 1, an h-step forgiveness condition k*(h,p,ε), together with the fixed-p

Load-bearing premise

The POMDP agent is given the true execution-noise rate ε through a fixed, known observation matrix, while the MDP baseline has no access to ε; if a noise-aware MDP recovered most of the advantage, the paper's claim that the latent intention representation is the decisive factor would be substantially weakened.

What would settle it

Run an MDP baseline that receives the same ε and discounts observed defections accordingly (e.g., using a noise-aware likelihood or a softened reward); if it matches the POMDP's cooperation rates against Tit-for-Tat and exhibits the same self-play threshold, the central representation claim is wrong. Alternatively, measure p_∞(ε) at fine noise steps between 0.10 and 0.15 to test whether the fixed point ε̃ = 1 − p_∞(ε̃) is actually satisfied.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • MDP-based agents in noisy social dilemmas will systematically over-retaliate; the latent-intention formulation is the principled alternative.
  • The h-step forgiveness rule gives a quantitative, evidence-based answer to how much forgiveness is warranted, replacing engineered forgiveness like Generous Tit-for-Tat.
  • The value of intention inference is bounded by the game's payoff structure: it helps in dilemmas with a temptation payoff (T>R) and is near-neutral in coordination games such as Stag Hunt.
  • Self-play under noise can produce correlated belief-driven collapse even with perfectly accurate inference; stability requires mechanisms such as forgetting, change-point detection, or opponent-shaping objectives.
  • A persistently unresolved epistemic drive in a stationary environment is a diagnostic signal that the state representation conflates reducible and irreducible uncertainty.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the noise level ε is unknown, a hierarchical POMDP that infers ε jointly with intention may push the threshold: the fixed point ε̃ = 1 − p_∞(ε̃) suggests that misestimating ε biases p_∞ in a self-consistent way. This is a direct, testable extension the paper leaves open.
  • The stationarity assumption for Dirichlet learning is violated precisely in self-play, where collapse occurs; this hints that the collapse may be partly a model-misspecification artifact, and non-stationary opponent models or change-point detection might soften it—an inference beyond the paper's claims.
  • The threshold's form—halfway between learned cooperation prior and noise—recalls a ratio-symmetric fixed point; a mean-field or stochastic-approximation analysis of p_t(CC) could predict the exact critical ε from the payoff matrix and horizon, not just bracket it.
  • The explore-then-commit pattern (parameter information gain spikes then vanishes) suggests a general design principle for multi-agent RL: representation quality can be diagnosed by whether epistemic drive resolves in a stationary environment.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a Partially Observable Markov Decision Process (POMDP) formulation of the iterated prisoner's dilemma with symmetric execution noise, in which latent intentions are hidden states and executed actions are noisy observations, solved within the active inference (AIF) framework. The core claims are: (i) an MDP with executed actions as states structurally conflates aleatoric and epistemic uncertainty, leading to systematic over-retaliation; (ii) the POMDP's latent-intention representation, combined with the expected-free-energy decomposition into pragmatic and epistemic terms, enables principled, adaptive forgiveness; and (iii) when both agents use this intention inference, a critical noise threshold exists, given by a fixed point on the learned cooperative prior, beyond which cooperation collapses even though the agent can still discriminate noise from hostility. Experiments compare POMDP-AIF against MDP-AIF in self-play, against Tit-for-Tat, and against Win-Stay Lose-Shift, plus a Stag Hunt control and ablation studies. The paper concludes that the representational choice is the decisive factor and that the value of intention inference is context-dependent.

Significance. If the central claims were fully established, the paper would make a useful contribution: it connects the game-theoretic literature on noise-induced collapse with Bayesian uncertainty decomposition and active inference, and offers a mechanistic explanation of the sharp self-play transition. Proposition 1 is a clean conditional result, and the EFE decomposition in Section 6.3 provides an intuitive account of explore-then-commit learning. The Stag Hunt and WSLS results are valuable scope conditions. However, the empirical support for the headline 'representation matters' claim is currently undermined by a known confound (the POMDP is given the true noise level via the A matrix, while the MDP is not), and the threshold 'prediction' is validated in-sample using the same runs that define the threshold. The paper's own limitations section acknowledges both issues and leaves the controlled baseline to future work, so the central message is not yet established.

major comments (4)
  1. [Section 7; Section 6.1] The headline POMDP-vs-MDP comparison confounds representation with information. The POMDP is handed the true execution-noise parameter ε through the fixed observation matrix A, while the MDP has no noise information at all. Section 7 acknowledges that 'an MDP with a noise-representative prior recovers much of the POMDP's advantage,' which directly undercuts the claim that the latent intention layer is the decisive factor. The controlled noise-aware MDP baseline is left to future work, so the central claim currently rests on an unmeasured confound. Please run the noise-aware MDP baseline in the main experiments and report the magnitude of the recovered advantage.
  2. [Section 6.4; Appendix D, Table 12] The empirical validation of the critical threshold is circular. The threshold ε̃ = 1 − p_∞(ε̃) uses p_∞(ε), the converged cooperative prior learned from the same self-play runs whose cooperation rates are later compared with the threshold. Section 6.4 takes p_500(CC) from those runs to 'verify' Proposition 1, and Appendix D, Table 12 compares p_500(CC)^5 to the same-run empirical cooperation rate. As the paper itself notes, three co-monotonic data points are insufficient to distinguish the proposed relationship from other monotonic functions. A proper test would use an analytically derived p_∞ or a held-out calibration set, then evaluate on different noise levels or different seeds.
  3. [Section 5, Eq. (5); Appendix D] The critical threshold is defined inconsistently. The main text states that the threshold is the noise level at which the cooperative survival probability crosses 0.5 for the planning horizon (i.e., p_∞(ε)^h = 0.5), but Eq. (5) gives the myopic fixed point ε̃ = 1 − p_∞(ε̃), which corresponds to h = 1. The empirical verification uses h = 5, so Eq. (5) is not the threshold being tested. Please state the h-dependent fixed-point condition explicitly and show that the empirically used threshold matches it.
  4. [Appendix A] Hyperparameters are tuned on the same evaluation grid (noise levels 0.00–0.35) and the same metrics (score, cooperation rate) that are later reported in the main experiments. The selected configurations are the best on that grid, which risks overfitting the reported curves and inflating the apparent advantage. Please provide a train/test split (e.g., tune on a subset of noise levels and test on held-out levels) or demonstrate that the main conclusions are robust across a range of reasonable hyperparameters.
minor comments (5)
  1. [Section 5, Eq. (2)] The displayed formula uses a floor, but the proof (Appendix D, Eq. 9) states a strict inequality k < X. The floor is only correct when X is not an integer; if X is an integer, the maximum forgivable k should be X−1. Please add a caveat or define k* using ceil(X)−1.
  2. [Appendix D, Proposition 3] The claim that the gap between binary and mixed thresholds is at most one is supported by computational verification over the finite set in Table 10, not by a general proof. If this bound is used in the main text, state it as a numerical observation over the evaluated parameter range.
  3. [Section 6.2] The paper reports 'consistent advantage' and 'sharp collapse' based on descriptive statistics and 95% confidence intervals, but does not provide significance tests or effect sizes. Adding formal comparisons (e.g., bootstrap or permutation tests) would strengthen the empirical claims.
  4. [Appendix D; Section 4.3] There are minor formatting errors: missing spaces in phrases such as 'theτcooperative rounds' and 'the agent’s component ofs t+1'. Please proofread the appendix and Section 4.3.
  5. [Section 3; Section 7] The paper argues that in self-play the Dirichlet stationarity assumption is violated, and that this is the intended source of the collapse. It would be helpful to explicitly separate the role of this model misspecification from the threshold mechanism; currently the two are intertwined.

Circularity Check

2 steps flagged

Critical-noise-threshold 'derivation' is an implicit equation in the empirically learned cooperative prior, and its Section 6.4 verification compares same-run quantities.

specific steps
  1. fitted input called prediction [Section 6.4 / Table 12; Appendix D (System-Level Fixed Point, Eq. 5)]
    "To validate the critical noise threshold derived in Section 5, we extract the Dirichlet cooperative prior p500(CC) at t=500 (10% of the trajectory) ... The empirical cooperation rates coincide closely with the cooperative survival probabilities at all three noise levels (Table 12)."

    The threshold is defined through p∞(ε), the converged Dirichlet cooperative prior learned from the same self-play runs whose empirical cooperation rates are then matched to p500(CC)^5. The Dirichlet counts are accumulated from those runs, so the learned prior and the empirical cooperation rate are monotone functions of the same trajectory data; the 'prediction' is therefore a restatement of the fitted prior rather than an independent derivation. The paper itself concedes that 'three co-monotonic data points are insufficient to distinguish this relationship from other monotonic functions of the same quantities,' which is exactly the fitted-input-as-prediction pattern.

  2. self definitional [Section 5, paragraph after Proposition 1]
    "The critical threshold is the noise level at which the cooperative survival probability crosses below 0.5 for the agent's planning horizon: the point where the hostile hypothesis becomes the typical case over the planning window regardless of discriminability."

    This sentence defines the critical threshold as p_t(CC)^h = 0.5. The later 'empirical verification' then checks that p500(CC)^5 falls below 0.5 at ε=0.15 and that cooperation is low. That is checking a defining condition against a measured prior, not deriving the threshold from the generative model. The threshold's location is never solved from payoffs and noise alone; it inherits the empirically measured p∞, making the central 'derived threshold' a definitional restatement in practice.

full rationale

The paper contains genuine, self-contained empirical work: the POMDP/MDP comparisons, the Stag Hunt control, and the epistemic-ablation experiments are real simulations, and Proposition 1 is a valid closed-form statement of the h-step forgiveness rule given a fixed cooperative prior. There are no load-bearing self-citations, imported uniqueness theorems, or ansatz-by-citation problems. The circularity is concentrated in the 'critical noise threshold' contribution. The threshold is defined as an implicit equation in the empirically learned cooperative prior, ε̃ = 1 − p∞(ε̃), and Section 6.4 validates it using priors extracted from the same self-play runs whose cooperation rates are then matched to p500(CC)^5. Because the Dirichlet counts are learned from those very runs, the close agreement in Table 12 is partly a comparison of a quantity with a monotone transform of itself. The authors' own caveat that 'three co-monotonic data points are insufficient' supports this reading. The separate confound noted in Section 7—the POMDP is given the true ε while the MDP is not—is a real threat to the headline comparison, but it is an experimental-correctness issue rather than a circularity, so it is not scored here. Overall, the central threshold claim partially reduces to its inputs, while the rest of the paper's contributions remain independent; hence a partial-circularity score of 6.

Axiom & Free-Parameter Ledger

4 free parameters · 6 axioms · 0 invented entities

No new physical entities are introduced; latent intention states are standard POMDP latent variables, not independently testable objects. The central 'derived threshold' depends on four fitted/selected quantities: planning horizon, update interval, Dirichlet prior, and the empirical cooperative prior p_500(CC). The axioms are mostly standard AIF and game-theory assumptions, but the known-ε and stationarity assumptions are load-bearing and explicitly acknowledged by the authors.

free parameters (4)
  • Planning horizon h = 5 (POMDP self-play); 3 (POMDP vs TFT)
    Chosen by grid search over {1,3,5,10} maximizing mean score on the same tasks/noise levels; directly enters Proposition 1 and the threshold value.
  • B-matrix update interval Δt = 50 (POMDP self-play); 10 (POMDP vs TFT)
    Selected by grid search over {5,10,20,50}; controls Dirichlet count accumulation and cooperation stability.
  • Dirichlet concentration α0 = 1
    Fixed symmetric uniform (add-one) prior; hand-chosen default, sensitivity only partially tested in Appendix E.
  • Converged cooperative prior p_∞(ε) / p_500(CC) = 0.954 at ε=0.05; 0.907 at ε=0.10; 0.761 at ε=0.15
    Measured from self-play Dirichlet counts at t=500; used in the fixed-point ε̃=1−p_∞(ε̃), making the 'critical threshold' an empirical fit rather than a parameter-free prediction.
axioms (6)
  • domain assumption Execution noise is a binary symmetric channel with known flip probability ε, and the POMDP agent knows the true A matrix.
    Section 3; Section 7 admits the MDP baseline lacks this information, making the comparison partly informational.
  • domain assumption Opponent intentions evolve according to a stationary reactive policy that can be learned with Dirichlet counts.
    Section 4.4; Section 7 states 'the Dirichlet learning assumes stationary opponent policies,' which is violated in self-play.
  • domain assumption Preferences over observations are log preferences proportional to IPD payoffs (T=5,R=3,P=1,S=0).
    Section 3: C(o)=ln P̃(o), with ordering C(DC)>C(CC)>C(DD)>C(CD).
  • domain assumption The binary hypothesis partition (all-cooperative vs all-hostile) suffices for the forgiveness threshold; mixed-sequence corrections are bounded with gap ≤1.
    Appendix D, Proposition 3; computationally verified only on diagonal prior–noise pairs, with off-diagonal entries hypothetical.
  • domain assumption Rational IPD play concentrates on symmetric equilibria CC/DD, justifying the binary partition.
    Appendix D: 'The binary partition reflects the game's strategic structure' and the bistable equilibrium argument.
  • standard math Expected free energy decomposition into pragmatic, state information gain, and parameter information gain is the correct planning objective.
    Section 4.3, Eq. (1); standard AIF machinery, but the balance/weighting of terms is assumed.

pith-pipeline@v1.3.0-daily-deepseek · 22509 in / 12297 out tokens · 124465 ms · 2026-08-04T07:15:41.344796+00:00 · methodology

0 comments
read the original abstract

In noisy social dilemmas, intended actions are stochastically corrupted before execution, so an observed defection may reflect hostile intent or action error. Standard Markov Decision Process (MDP) formulations treat executed actions as states, structurally precluding this distinction and causing systematic over-retaliation. We introduce a Partially Observable MDP (POMDP) formulation encoding opponent intentions as latent states and executed actions as noisy observations, solved within the active inference (AIF) framework with a cost function that decomposes into epistemic and pragmatic components that jointly address inferring current intent and learning how intent evolves. In the Iterated Prisoner's Dilemma with symmetric noise, we derive a critical noise threshold governing cooperation collapse, connecting it to a fixed-point condition on learned priors. Experiments reveal that the value of intention inference is context-dependent: the POMDP provides consistent advantages against conditionally cooperative opponents, but mutual intention inference under sufficient noise produces correlated belief-driven collapse. The advantage is specific to games where intent attribution is decision-relevant.

Figures

Figures reproduced from arXiv: 2608.02440 by Jonathan Shock, Kival Mahadew.

Figure 1
Figure 1. Figure 1: Mean cumulative score (± 95% CI, 30 seeds, 5000 timesteps) as a function of noise level ϵ. Left: self-play. Centre: vs TFT. Right: vs WSLS. The POMDP outperforms at low noise in self-play but collapses sharply between ϵ = 0.10 and 0.15; against TFT, it maintains a consistent advantage across all noise levels. 0.00 0.05 0.10 0.15 0.20 0.25 Noise Level ( ) 0.0 0.2 0.4 0.6 0.8 1.0 Mutual Cooperation Rate (CC … view at source ↗
Figure 2
Figure 2. Figure 2: Mutual cooperation rate (fraction of CC outcomes, ± 95% CI) as a function of ϵ. Same layout as [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Expected free energy decomposition for POMDP-AIF (left, blue) and MDP-AIF (right, red) vs TFT at ϵ = 0.10, averaged over 30 seeds with 95% confidence intervals for the first 100 turns. Each row shows the difference D−C (positive favours defection) for one EFE component: pragmatic value (row 1), state information gain (row 2), parameter information gain (row 3), and the resulting P(Cooperate) (row 4). 6.4. … view at source ↗
Figure 4
Figure 4. Figure 4: Mean cumulative score (± 95% CI) as a function of noise level ϵ, with the epistemic (parame￾ter/state information gain) term removed from the EFE. 0.00 0.05 0.10 0.15 0.20 0.25 Noise Level ( ) 0.0 0.2 0.4 0.6 0.8 1.0 Mutual Cooperation Rate (CC / turns) Self-play 0.00 0.05 0.10 0.15 0.20 0.25 Noise Level ( ) vs TFT 0.00 0.05 0.10 0.15 0.20 0.25 Noise Level ( ) vs WSLS Mutual Cooperation Rate Mean ± 95% CI … view at source ↗
Figure 5
Figure 5. Figure 5: Mutual cooperation rate as a function of ϵ, with the epistemic term removed from the EFE. The MDP with full EFE (Section 6.3) shows that epistemic drive under an inadequate representation yields only gradual, uncommitted improvement. The POMDP with pragmatic value only shows that the correct representation without epistemic drive yields no improvement at all. The cooperative advantages we report require bo… view at source ↗
Figure 6
Figure 6. Figure 6: Mean cumulative score (± 95% CI) as a function of noise level ϵ in the Stag Hunt game. 0.00 0.05 0.10 0.15 0.20 0.25 Noise Level ( ) 0.0 0.2 0.4 0.6 0.8 1.0 Mutual Cooperation Rate (CC / turns) Self-play 0.00 0.05 0.10 0.15 0.20 0.25 Noise Level ( ) vs TFT 0.00 0.05 0.10 0.15 0.20 0.25 Noise Level ( ) vs WSLS Mutual Cooperation Rate Mean ± 95% CI by Noise Level and Matchup Formulation POMDP MDP [PITH_FULL… view at source ↗
Figure 7
Figure 7. Figure 7: Mutual cooperation rate as a function of ϵ in the Stag Hunt game. Figures 6 and 7 show the mean cumulative score and mutual cooperation rate across noise levels. The POMDP advantage that was prominent in the IPD largely disappears. In self-play, the two formulations track closely, with the POMDP marginally ahead at ϵ = 0.05 to 0.10 but within overlapping confidence intervals. Against TFT, the MDP slightly … view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

12 extracted references · 3 canonical work pages

  1. [1]

    In self-play, however, the benefit ofh >3 is marginal

    collapses entirely because it cannot value reciprocation, whereash≥3 recovers near-optimal cooperation. In self-play, however, the benefit ofh >3 is marginal. Third, at low-to-moderate noise with matched hyperparameters (e.g.,h= 3, ∆t= 50,ε= 0.10), the POMDP sustains higher cooperation than the MDP (0.74 vs. 0.57). This advantage reverses beyond the self-...

    2000

  2. [9]

    Alex Kendall and Yarin Gal

    doi: 10.1073/pnas.1621239114. Alex Kendall and Yarin Gal. What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?, October

    doi:10.1073/pnas.1621239114

  3. [1985]

    doi: 10.1177/0022002785029004004

    ISSN 1552-8766. doi: 10.1177/0022002785029004004. Intention Inference 12 Neil C. Rabinowitz, Frank Perbet, H. Francis Song, Chiyuan Zhang, S. M. Ali Eslami, and Matthew Botvinick. Machine Theory of Mind, March

    work page doi:10.1177/0022002785029004004

  4. [1997]

    doi: 10.1006/jtbi.1996.0326

    ISSN 0022-5193. doi: 10.1006/jtbi.1996.0326. Daphne Demekas, Conor Heins, and Brennan Klein. An analytical model of active inference in the Iterated Prisoner’s Dilemma, August

    arXiv 1996

  5. [2005]

    doi: 10.1613/jair.1579

    ISSN 10769757. doi: 10.1613/jair.1579. Christian Hilbe, Luis A. Martinez-Vaquero, Krishnendu Chatterjee, and Martin A. Nowak. Memory- n strategies of direct reciprocity.Proceedings of the National Academy of Sciences, 114(18):4715– 4720, May

    doi:10.1613/jair.1579

  6. [2006]

    ISBN 978-1-59593-303-4

    Association for Computing Machinery. ISBN 978-1-59593-303-4. doi: 10.1145/1160633.1160736. Chris L. Baker, Rebecca Saxe, and Joshua B. Tenenbaum. Action understanding as inverse planning. Cognition, 113(3):329–349, December

    arXiv

  7. [2008]

    doi: 10.1371/journal.pcbi.1000254

    ISSN 1553-7358. doi: 10.1371/journal.pcbi.1000254. A. Hyperparameter Configuration Active Inference Agent: Search Procedure and Selected V alues A.1 Overview Hyperparameters were selected via an exhaustive grid search over the parameters and ranges listed in Table

    doi:10.1371/journal.pcbi.1000254

  8. [2009]

    doi: 10.1016/j.cognition.2009.07.005

    ISSN 1873-7838. doi: 10.1016/j.cognition.2009.07.005. Maarten C. Boerlijst, Martin A. Nowak, and Karl Sigmund. The Logic of Contrition.Journal of Theoretical Biology, 185(3):281–293, April

    doi:10.1016/j.cognition.2009.07.005 2009

  9. [2014]

    doi: 10.1371/journal.pcbi.1003810

    ISSN 1553-734X, 1553-7358. doi: 10.1371/journal.pcbi.1003810. Jakob N. Foerster, Richard Y. Chen, Maruan Al-Shedivat, Shimon Whiteson, Pieter Abbeel, and Igor Mordatch. Learning with Opponent-Learning Awareness, September

    work page doi:10.1371/journal.pcbi.1003810

  10. [2015]

    doi: 10.1016/j.concog.2014.12.003

    ISSN 1090-2376. doi: 10.1016/j.concog.2014.12.003. Karl J. Friston, Marco Lin, Christopher D. Frith, Giovanni Pezzulo, J. Allan Hobson, and Sasha On- dobaka. Active Inference, Curiosity and Insight.Neural Computation, 29(10):2633–2683, October

    doi:10.1016/j.concog.2014.12.003 2014

  11. [2017]

    doi: 10.1162/neco a 00999

    ISSN 0899-7667, 1530-888X. doi: 10.1162/neco a 00999. Dibya Ghosh, Jad Rahme, Aviral Kumar, Amy Zhang, Ryan P. Adams, and Sergey Levine. Why Generalization in RL is Difficult: Epistemic POMDPs and Implicit Partial Observability, July

    doi:10.1162/neco

  12. [2024]

    doi: 10.1371/journal.pcbi.1012644

    ISSN 1553- 734X. doi: 10.1371/journal.pcbi.1012644. P. J. Gmytrasiewicz and P. Doshi. A Framework for Sequential Planning in Multi-Agent Settings. pages 49–79,

    work page doi:10.1371/journal.pcbi.1012644