Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Position: Theory of Mind Benchmarks are Broken for Large Language Models

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Current LLM theory-of-mind benchmarks mostly measure whether a model can predict another agent's actions; this paper argues they miss whether the model acts on those predictions, and shows the two can diverge sharply.

desk verdict Useful literal–functional ToM distinction and clean game results, but the title overclaims and Definition 2.2 is a general decision-quality metric, not a ToM-specific one. read the letter →

arxiv 2412.19726 v4 pith:YE257MR6 submitted 2024-12-27 cs.AI

Matthew Riemer , Zahra Ashktorab , Djallel Bouneffouf , Payel Das , Miao Liu , Justin D. Weisz , Murray Campbell This is my paper · ORCID
classification cs.AI
keywords theoryofmindlargelanguagemodelsfunctionalliteralrepeatedgamesregretin-contextadaptationbenchmarkevaluation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Most LLM theory-of-mind benchmarks ask a model to predict another agent's next action and reward it for accuracy. The paper argues those benchmarks are broken because they assume the model's predictions are wired into its own decisions; with LLMs this process consistency cannot be taken for granted. It introduces functional theory of mind, defined as the regret an agent incurs by failing to act optimally in response to a partner's observed behavior. In simple 100-round games against partners that always play one action, open-source LLMs score over 90 percent prediction accuracy while missing most of the reward, because they predict correctly and then fail to exploit their predictions. If the paper is right, high literal theory-of-mind scores say little about whether an LLM can adapt to a new partner in-context.

What carries the argument

The machinery is the paired definitions in Section 2. Definition 2.1 measures literal theory of mind as a loss $\mathbb{D}(\phi(\mathbf{a}_{-i}^{1},\ldots,\mathbf{a}_{-i}^{T}),\phi(\hat{\mathbf{a}}_{-i}^{1},\ldots,\hat{\mathbf{a}}_{-i}^{T}))$ between predicted and actual partner actions under an abstraction mapping $\phi$. Definition 2.2 measures functional theory of mind as $\sum_{t=1}^{T}(r^{i*}_t - r^i_t)$, the cumulative reward forgone compared with the optimal $T$-step response to the actual partner policy. The load-bearing comparison is between $\Delta_{\mathrm{ToM}}/T$, the regret of a rational policy that faithfully follows the literal model, and $\Delta_{\mathrm{Functional}}/T$, the regret the LLM actually incurs; the gap between them isolates the failure to convert predictions into decisions. A derived interesting-task score computes the expected regret of an optimal responder to one partner policy when played against another, giving benchmark designers a quantitative criterion for whether a task tests adaptation at all.

What would settle it

Replace the social partner with a non-social stochastic process that generates the same sequence of actions (for example, a fixed pseudo-random schedule), holding prompt length, payoff table, and interaction history identical; if functional regret stays at the same high level, the deficit is general long-context reasoning, while a large drop would confirm the theory-of-mind-specific account.

Watch

Extended reading notes

Core claim

On the paper's own terms, literal theory of mind is the $T$-step loss between the partner actions an agent predicts and the actions the partner actually takes (Definition 2.1), and functional theory of mind is the $T$-step regret of the agent's policy relative to the optimal response to the partner's real policy (Definition 2.2). The central discovery is that these two quantities decouple in current LLMs: across Rock-Paper-Scissors, Iterated Battle of the Sexes, and Iterated Prisoner's Dilemma, models reach 90 percent or higher literal prediction accuracy while incurring large functional regret against partners that play one fixed action for 100 rounds. The gap persists even when the partner's actual next action is provided as oracle input, and in the opposite direction a trained reasoning model can match a tabular near-optimal baseline on functional regret while its literal predictions are relatively weak. The pith is that theory-of-mind capability, defined as adapting one's own behavior to another agent, is not implied by prediction accuracy.

Load-bearing premise

The load-bearing premise is that the gap is a theory-of-mind-specific failure rather than a general weakness in reasoning over long interaction histories or following complex instructions; the authors' own oracle and payoff-ablation results show how much weight this assumption carries.

Editorial extensions

If this is right

  • Existing theory-of-mind benchmark scores overstate agent readiness for multi-agent interaction, since high literal accuracy can coexist with near-maximal functional regret.
  • Meaningful LLM theory-of-mind evaluation should report both ToM% and $\Delta_{\mathrm{Functional}}/T$ (or accumulated reward) and the gap between them.
  • Supplying the partner's true action as oracle input does not fix the functional deficit, so improvements cannot come from better prediction alone.
  • The two capabilities are not ordered: a trained reasoning model can show strong functional theory of mind with weak literal theory of mind, and vice versa.
  • Simple repeated matrix games with single-action or tit-for-tat partners are a sufficient minimal testbed to expose the failure.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's own ablations point to a confound the theory-of-mind framing does not resolve: removing the payoff table from the oracle prompt nearly closes the Rock-Paper-Scissors regret, so long-context payoff reasoning may be the true bottleneck.
  • A direct extension would run the same prompts with a non-social stochastic partner matched for action statistics; if regret stays high, functional theory of mind is not a distinct capability.
  • The definitions should transfer to non-game deployments: for a tutoring or coding assistant, the metric that matters is whether the interaction improves, not whether the next user action is predicted.
  • Because the game-theoretic and theory-of-mind solutions diverge in adversarial settings, evaluation suites should separate cooperative, mixed, and adversarial partners rather than averaging them together.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This position paper argues that most existing theory of mind (ToM) benchmarks for large language models are "broken" because they measure only literal ToM—the ability to predict another agent's behavior—and not functional ToM—the ability to adapt one's own behavior rationally to a partner. The authors formalize both notions: literal ToM as a prediction loss (Definition 2.1) and functional ToM as T-step regret against the optimal response to the partner's policy (Definition 2.2). They then report experiments in Rock-Paper-Scissors, Iterated Battle of the Sexes, and Iterated Prisoner's Dilemma against simple single-action and tit-for-tat partners, using several prompting strategies and open-source LLMs. The empirical pattern is that LLMs often show high literal prediction accuracy (ToM %) while incurring large functional regret, with a tabular RMax baseline achieving much lower regret. The paper concludes that strong literal ToM performance does not imply strong functional ToM performance, and that benchmarks should include interactive, functional evaluations.

Significance. If the position is accepted, it would reshape how the community evaluates ToM in LLMs: instead of passive question-answering tests, benchmarks would need interactive tasks in which the agent's decisions, not just its predictions, are scored. The formal definitions and the regret-based metric are clean and reusable, and the empirical study is careful in several respects: results are reported with confidence intervals, a tabular RMax baseline anchors the comparison, and the prompting ablations are extensive. The finding that high literal ToM accuracy coexists with poor functional performance in simple games is a useful cautionary result. However, the central claim is broader than the evidence: the experiments demonstrate a gap in three matrix games, but the ablations suggest that much of the gap may be due to general long-context reasoning or instruction-following rather than ToM-specific reasoning. The paper therefore makes a strong case for including decision-oriented social benchmarks, but the specific assertion that existing ToM benchmarks are broken because they omit functional ToM needs additional support or a narrower framing.

major comments (3)
  1. [Section 4.2 / Table 3] The paper's own ablations undermine the ToM-specific attribution of the functional gap. For LLAMA-3 70B Instruct on RPS, regret per step drops from 0.444 under QA prompting to 0.103 under Oracle + Max -Payoffs and to 0.121 under CoT + 3-Shot, close to the tabular RMax baseline of 0.083. These interventions supply no additional information about the partner's mental state; they change payoff salience, instruction emphasis, and reasoning format. The text itself states that "reasoning over long contexts is at the core of the issue." This suggests the observed failure may be a general long-context reasoning or instruction-following deficit rather than a specifically theory-of-mind deficit. Since the central claim is that ToM benchmarks are broken because they omit functional ToM, the manuscript should either decompose the gap (for example, with non-social control tasks matched in context length and payoff reasoning) or explicitly reframe the claim as advocating for social decision-making benchmarks more broadly.
  2. [Abstract and Section 1] The claim that "the majority" of existing ToM benchmarks are broken is not directly supported by the evidence. No standard ToM benchmark (for example, Sally-Anne, FANToM, or the tasks used by Bubeck et al., Kosinski, or Strachan et al.) is run through the proposed functional metric; the empirical support consists of three matrix games with single-action or tit-for-tat partners. A case study can motivate a position, but the title-level claim requires either a systematic mapping of benchmark item types to Definitions 2.1 and 2.2 or a stated narrowing of the claim to "benchmarks that report only literal ToM should not be used to infer functional competence." As written, the paper overgeneralizes from its own game-based setup to the entire benchmark literature.
  3. [Definition 2.2 and Appendix C.4] Definition 2.2 defines functional ToM as T-step regret against the optimal response to the partner's policy, with no term that references the partner's beliefs, intentions, or mental state. As the authors note in Section 1, this makes the metric satisfiable by model-free policies. Consequently, high ΔFunctional can arise from non-ToM causes: failure to parse the payoff table, failure to follow the reward-maximization instruction, action-label biases, or inability to reason over the history. Appendix C.4 documents that action-label inductive bias systematically changes regret, and Table 3 shows that removing payoffs while providing oracle actions can almost close the gap. The metric may still be a valuable measure of partner-adaptive behavior, but calling it "theory of mind" without further justification conflates social cognition with general decision-making. The paper should either justify the terminological choice or weaken the inference from high regret to a ToM deficit.
minor comments (5)
  1. [Section 4.1] The authors state that for Reflexion they "always report the best value" over memory sizes 1 to 3. This is a mild optimistic selection bias; it is disclosed, but the tables would be clearer if the selected memory size appeared alongside the Reflexion rows rather than only in Appendix C.2.
  2. [Table 6 caption] The caption reads "Iterated Batte of Sexes" and should be "Iterated Battle of Sexes."
  3. [Section 4.2] The sentence containing "the LLM must struggle" is misspelled as "strugle"; please correct this typo.
  4. [Figure 15 caption] The caption says "The only difference with Figure 15 in the emphasis on maximizing reward," but the comparison should be with Figure 14 (Oracle Prompting), not with itself.
  5. [Definition 2.1] The sentence introducing the literal ToM loss is grammatically incomplete ("The loss from start state s1 with respect to a joint policy ..."). Rewriting it as a proper definition would improve readability.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: literal and functional ToM are independently operationalized and measured directly; the paper's central claim is a normative position supported by self-contained experiments, not a reduction to its own inputs.

full rationale

The derivation chain is self-contained. Definition 2.1 (literal ToM loss) is prediction error over the partner's actions; Definition 2.2 (functional ToM regret) is the T-step reward gap against the optimal response to the partner's policy. Both quantities are measured directly from LLM rollouts (ToM %, ΔFunctional/T, ΔToM/T), so the observed dissociation between high literal accuracy and high functional regret is an empirical result, not a fitted parameter renamed as a prediction. The only tuning choice is the per-game Reflexion memory size ('we always report the best value', Section 4.1), which is an evaluation-protocol hyperparameter and does not determine the central comparison. Self-citations (e.g., Kim et al. 2022a, Khetarpal et al. 2020, Riemer et al. 2024a) appear in related-work and appendix framing, but the definitions, experiments, and ablations do not load-bear on them. The paper's own Table 3 ablations (Oracle + Max -Payoffs RPS 0.103; CoT + 3-Shot RPS 0.121) weaken the attribution of the gap to theory-of-mind-specific failure, suggesting long-context reasoning or instruction-following may be involved; this is a construct-validity limitation rather than circularity. The 'broken benchmarks' conclusion follows from the normative premise that functional ToM is what matters for deployed agents, which is a position, not a self-justifying derivation. No equation reduces to its own input and no cited result is doing unseen load-bearing work.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the POSG formalism, the regret-based definitions, and the transfer of matrix-game results to natural-language theory-of-mind benchmarks. No physical or conceptual entities are invented; the only model selection parameter is the Reflexion memory size. The main unstated burden is the assumption that the observed gap in simple games generalizes to the richer benchmarks the paper criticizes.

free parameters (1)
  • Reflexion memory size = 1 for RPS and IBS, 3 for IPD in Table 3
    The paper reports the best value among memory sizes 1, 2, and 3 for each game, a mild favorable model selection that can slightly overstate Reflexion performance but is not used to establish the central claim.
assumptions (4)
  • domain assumption Agent interaction can be modeled as a partially observable stochastic game with text-encoded histories for LLM policies.
    Section 2 defines the setting and assumes the LLM's history representation is encoded as LLM tokens; this is a modeling choice that may not cover all real deployments.
  • domain assumption Existing LLM theory-of-mind benchmarks are accurately characterized as measuring only literal theory of mind and lacking an interactive functional component.
    Section 3 surveys prior benchmarks and labels them literal; this is a premise of the argument, not something the paper empirically demonstrates on those benchmarks.
  • domain assumption Regret against the optimal response to the partner's policy is the right normative standard for functional theory of mind.
    Definition 2.2 uses T-step regret; computing it requires knowing the partner's policy and payoff structure, which limits the metric to settings where the optimal response is computable.
  • domain assumption Stochastic sampling from next-token probabilities is a faithful way to obtain the LLM's action policy.
    The LM prompting strategy in Section 4.1 derives action probabilities from the model's next-token distribution; if this extraction is not faithful, some measured regrets could be artifacts, though QA and CoT prompting provide partial checks.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Position: Theory of Mind Benchmarks are Broken for Large Language Models." pith.science (2026). https://pith.science/paper/YE257MR6

@misc{pith2026241219726,
  author       = {Pith},
  title        = {Pith review of: Position: Theory of Mind Benchmarks are Broken for Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YE257MR6}},
  note         = {Machine review of arXiv:2412.19726}
}
read the original abstract

Our paper argues that the majority of theory of mind benchmarks are broken because of their inability to directly test how large language models (LLMs) adapt to new partners. This problem stems from the fact that theory of mind benchmarks for LLMs are overwhelmingly inspired by the methods used to test theory of mind in humans and fall victim to a fallacy of attributing human-like qualities to AI agents. We expect that humans will engage in a consistent reasoning process across various questions about a situation, but this is known to not be the case for current LLMs. Most theory of mind benchmarks only measure what we call literal theory of mind: the ability to predict the behavior of others. However, this type of metric is only informative when agents exhibit self-consistent reasoning. Thus, we introduce the concept of functional theory of mind: the ability to adapt to agents in-context following a rational response to their behavior. We find that many open source LLMs are capable of displaying strong literal theory of mind capabilities, but seem to struggle with functional theory of mind -- even with exceedingly simple partner policies. Simply put, strong literal theory of mind performance does not necessarily imply strong functional theory of mind performance or vice versa. Achieving functional theory of mind, particularly over long interaction horizons with a partner, is a significant challenge deserving a prominent role in any meaningful LLM theory of mind evaluation.

Figures

Figures reproduced from arXiv: 2412.19726 by the authors.

Figure 1
Figure 1. LM Prompting Example. The LLM is given this prompt for each action for the respective game action0 , action1 , and action2 (for RPS only) and we compare its probabilities computed for the token representation of the actions to determine which action it selects [PITH_FULL_IMAGE:figures/full_fig_p019_1.png] view at source ↗
Figure 2
Figure 2. LM Literal Theory of Mind Prompting - Agent First. The LLM is given this prompt for each action that its partner will take for the respective game action0 , action1 , and action2 (for RPS only) and we compare its probabilities computed for the token representation of the actions to determine which action prediction it selects [PITH_FULL_IMAGE:figures/full_fig_p019_2.png] view at source ↗
Figure 3
Figure 3. LM Literal Theory of Mind Prompting - Partner First. The LLM is given this prompt for each action that its partner will take for the respective game action0 , action1 , and action2 (for RPS only) and we compare its probabilities computed for the token representation of the actions to determine which action prediction it selects. 19 [PITH_FULL_IMAGE:figures/full_fig_p019_3.png] view at source ↗
Figures from the paper (40 more)
Figure 4
Figure 4. Figure 4: QA Prompting. The LLM is given this prompt and generates an answer with stochastic decoding. If the provided response matches the template and includes an action from the action list, the response is accepted. If not, we query the LLM for more stochastic responses unti…
Figure 5
Figure 5. Figure 5: QA Literal Theory of Mind Prompting. The LLM is given this prompt and generates an answer with stochastic decoding. If the provided response matches the template and includes an action from the action list, the response is accepted. If not, we query the LLM for more st…
Figure 6
Figure 6. Figure 6: CoT Prompting. The LLM is given this prompt and generates an answer with stochastic decoding. If the provided response matches the template and includes an action from the action list, the response is accepted. If not, we query the LLM for more stochastic responses unt…
Figure 7
Figure 7. Figure 7: CoT Literal Theory of Mind Prompting. The LLM is given this prompt and generates an answer with stochastic decoding. If the provided response matches the template and includes an action from the action list, the response is accepted. If not, we query the LLM for more s…
Figure 8
Figure 8. Figure 8: Plans+Insights Prompting. The LLM is given this prompt and generates an answer with stochastic decoding. If the provided response matches the template and includes an action from the action list, the response is accepted. If not, we query the LLM for more stochastic re…
Figure 9
Figure 9. Figure 9: Plans+Insights Literal Theory of Mind Prompting. The LLM is given this prompt and generates an answer with stochastic decoding. If the provided response matches the template and includes an action from the action list, the response is accepted. If not, we query the LLM…
Figure 10
Figure 10. Figure 10: Reflection Module Prompting for Reflexion Prompting. The plan output from this prompt fills in the most recent memory for the Reflexion prompts in Figures 11 and 12 [PITH_FULL_IMAGE:figures/full_fig_p022_10.png]
Figure 11
Figure 11. Figure 11: Reflexion Prompting (3 Memories). The LLM is given this prompt and generates an answer with stochastic decoding. If the provided response matches the template and includes an action from the action list, the response is accepted. If not, we query the LLM for more stoc…
Figure 12
Figure 12. Figure 12: Reflexion Literal Theory of Mind Prompting (3 Memories). The LLM is given this prompt and generates an answer with stochastic decoding. If the provided response matches the template and includes an action from the action list, the response is accepted. If not, we quer…
Figure 13
Figure 13. Figure 13: CoT 3-Shot Prompting. The LLM is given this prompt and generates an answer with stochastic decoding. If the provided response matches the template and includes an action from the action list, the response is accepted. If not, we query the LLM for more stochastic respo…
Figure 14
Figure 14. Figure 14: Oracle Prompting. The provided action that the other agent will choose is true ’oracle’ knowledge in all cases. The LLM is given this prompt and generates an answer with stochastic decoding. If the provided response matches the template and includes an action from the…
Figure 15
Figure 15. Figure 15: Oracle Prompting Emphasizing Maximization. The provided action that the other agent will choose is true ’oracle’ knowledge in all cases. The LLM is given this prompt and generates an answer with stochastic decoding. If the provided response matches the template and in…
Figure 16
Figure 16. Figure 16: S2A Prompting. We accept the LLM’s output for this prompt in all cases. This output replaces the rules and history in the subsequent QA or CoT prompt [PITH_FULL_IMAGE:figures/full_fig_p024_16.png]
Figure 17
Figure 17. Figure 17: S2A Literal Theory of Mind Prompting. We accept the LLM’s output for this prompt in all cases. This output replaces the rules and history in the subsequent QA or CoT prompt. 24 [PITH_FULL_IMAGE:figures/full_fig_p024_17.png]
Figure 18
Figure 18. Figure 18: Social Prompting. The actions that it is predicted that the other player will choose is the actual output of a prior application of literal theory of mind oriented prompt to the same LLM model. If the provided response matches the template and includes an action from …
Figure 19
Figure 19. Figure 19: IPD Regret for Mistral Large 2 with a Tit for Tat Partner Across Action Types. We compare performance across action spaces with the Tabular RMax algorithm to test the influence of different levels of inductive bias. with 20 repetitions to take up a longer portion of t…
Figure 20
Figure 20. Figure 20: RPS Regret for LLAMA-3 70B Instruct with a Single Action Partner Across Action Types. We compare performance across action spaces with the Tabular RMax algorithm to test the influence of different levels of inductive bias [PITH_FULL_IMAGE:figures/full_fig_p029_20.png]
Figure 21
Figure 21. Figure 21: RPS ToM % for LLAMA-3 70B Instruct with a Single Action Partner Across Action Types. We compare accuracy across action spaces with a Tabular counting algorithm to test the influence of different levels of inductive bias. 29 [PITH_FULL_IMAGE:figures/full_fig_p029_21.png]
Figure 22
Figure 22. Figure 22: IBS Regret for LLAMA-3 70B Instruct with a Single Action Partner Across Action Types. We compare performance across action spaces with the Tabular RMax algorithm to test the influence of different levels of inductive bias [PITH_FULL_IMAGE:figures/full_fig_p030_22.png]
Figure 23
Figure 23. Figure 23: IBS ToM % for LLAMA-3 70B Instruct with a Single Action Partner Across Action Types. We compare accuracy across action spaces with a Tabular counting algorithm to test the influence of different levels of inductive bias. 30 [PITH_FULL_IMAGE:figures/full_fig_p030_23.png]
Figure 24
Figure 24. Figure 24: IPD Regret for LLAMA-3 70B Instruct with a Single Action Partner Across Action Types. We compare performance across action spaces with the Tabular RMax algorithm to test the influence of different levels of inductive bias [PITH_FULL_IMAGE:figures/full_fig_p031_24.png]
Figure 25
Figure 25. Figure 25: IPD ToM % for LLAMA-3 70B Instruct with a Single Action Partner Across Action Types. We compare accuracy across action spaces with a Tabular counting algorithm to test the influence of different levels of inductive bias. 31 [PITH_FULL_IMAGE:figures/full_fig_p031_25.png]
Figure 26
Figure 26. Figure 26: RPS Regret for LLAMA-3 70B Instruct with a Tit for Tat Partner Across Action Types. We compare performance across action spaces with the Tabular RMax algorithm to test the influence of different levels of inductive bias [PITH_FULL_IMAGE:figures/full_fig_p032_26.png]
Figure 27
Figure 27. Figure 27: RPS ToM % for LLAMA-3 70B Instruct with a Tit for Tat Partner Across Action Types. We compare accuracy across action spaces with a Tabular counting algorithm to test the influence of different levels of inductive bias. 32 [PITH_FULL_IMAGE:figures/full_fig_p032_27.png]
Figure 28
Figure 28. Figure 28: IBS Regret for LLAMA-3 70B Instruct with a Tit for Tat Partner Across Action Types. We compare performance across action spaces with the Tabular RMax algorithm to test the influence of different levels of inductive bias [PITH_FULL_IMAGE:figures/full_fig_p033_28.png]
Figure 29
Figure 29. Figure 29: IBS ToM % for LLAMA-3 70B Instruct with a Tit for Tat Partner Across Action Types. We compare accuracy across action spaces with a Tabular counting algorithm to test the influence of different levels of inductive bias. 33 [PITH_FULL_IMAGE:figures/full_fig_p033_29.png]
Figure 30
Figure 30. Figure 30: IPD Regret for LLAMA-3 70B Instruct with a Tit for Tat Partner Across Action Types. We compare performance across action spaces with the Tabular RMax algorithm to test the influence of different levels of inductive bias [PITH_FULL_IMAGE:figures/full_fig_p034_30.png]
Figure 31
Figure 31. Figure 31: IPD ToM % for LLAMA-3 70B Instruct with a Tit for Tat Partner Across Action Types. We compare accuracy across action spaces with a Tabular counting algorithm to test the influence of different levels of inductive bias. 34 [PITH_FULL_IMAGE:figures/full_fig_p034_31.png]
Figure 32
Figure 32. Figure 32: RPS Regret for Mistral Large 2 with a Single Action Partner Across Action Types. We compare performance across action spaces with the Tabular RMax algorithm to test the influence of different levels of inductive bias [PITH_FULL_IMAGE:figures/full_fig_p035_32.png]
Figure 33
Figure 33. Figure 33: RPS ToM % for Mistral Large 2 with a Single Action Partner Across Action Types. We compare accuracy across action spaces with a Tabular counting algorithm to test the influence of different levels of inductive bias. 35 [PITH_FULL_IMAGE:figures/full_fig_p035_33.png]
Figure 34
Figure 34. Figure 34: IBS Regret for Mistral Large 2 with a Single Action Partner Across Action Types. We compare performance across action spaces with the Tabular RMax algorithm to test the influence of different levels of inductive bias [PITH_FULL_IMAGE:figures/full_fig_p036_34.png]
Figure 35
Figure 35. Figure 35: IBS ToM % for Mistral Large 2 with a Single Action Partner Across Action Types. We compare accuracy across action spaces with a Tabular counting algorithm to test the influence of different levels of inductive bias. 36 [PITH_FULL_IMAGE:figures/full_fig_p036_35.png]
Figure 36
Figure 36. Figure 36: IPD Regret for Mistral Large 2 with a Single Action Partner Across Action Types. We compare performance across action spaces with the Tabular RMax algorithm to test the influence of different levels of inductive bias [PITH_FULL_IMAGE:figures/full_fig_p037_36.png]
Figure 37
Figure 37. Figure 37: IPD ToM % for Mistral Large 2 with a Single Action Partner Across Action Types. We compare accuracy across action spaces with a Tabular counting algorithm to test the influence of different levels of inductive bias. 37 [PITH_FULL_IMAGE:figures/full_fig_p037_37.png]
Figure 38
Figure 38. Figure 38: RPS Regret for Mistral Large 2 with a Tit for Tat Partner Across Action Types. We compare performance across action spaces with the Tabular RMax algorithm to test the influence of different levels of inductive bias [PITH_FULL_IMAGE:figures/full_fig_p038_38.png]
Figure 39
Figure 39. Figure 39: RPS ToM % for Mistral Large 2 with a Tit for Tat Partner Across Action Types. We compare accuracy across action spaces with a Tabular counting algorithm to test the influence of different levels of inductive bias. 38 [PITH_FULL_IMAGE:figures/full_fig_p038_39.png]
Figure 40
Figure 40. Figure 40: IBS Regret for Mistral Large 2 with a Tit for Tat Partner Across Action Types. We compare performance across action spaces with the Tabular RMax algorithm to test the influence of different levels of inductive bias [PITH_FULL_IMAGE:figures/full_fig_p039_40.png]
Figure 41
Figure 41. Figure 41: IBS ToM % for Mistral Large 2 with a Tit for Tat Partner Across Action Types. We compare accuracy across action spaces with a Tabular counting algorithm to test the influence of different levels of inductive bias. 39 [PITH_FULL_IMAGE:figures/full_fig_p039_41.png]
Figure 42
Figure 42. Figure 42: IPD Regret for Mistral Large 2 with a Tit for Tat Partner Across Action Types. We compare performance across action spaces with the Tabular RMax algorithm to test the influence of different levels of inductive bias [PITH_FULL_IMAGE:figures/full_fig_p040_42.png]
Figure 43
Figure 43. Figure 43: IPD ToM % for Mistral Large 2 with a Tit for Tat Partner Across Action Types. We compare accuracy across action spaces with a Tabular counting algorithm to test the influence of different levels of inductive bias. 40 [PITH_FULL_IMAGE:figures/full_fig_p040_43.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. MeetingToM: Evaluating Multimodal LLMs on Theory-of-Mind Reasoning in Multi-Party Meetings

    cs.CL 2026-07 conditional novelty 6.0 of 10

    A new benchmark shows MLLMs underperform humans on meeting Theory-of-Mind tasks, especially detecting pseudo-consensus and hidden dissent.

Reference graph

Works this paper leans on

124 extracted references · 33 canonical work pages · cited by 1 Pith paper

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Abdulhai, M., Kim, D.-K., Riemer, M., Liu, M., Tesauro, G., and How, J. P. Context-specific representation abstraction for deep option learning. arXiv preprint arXiv:2109.09876, 2021

    work page Pith review arXiv 2021

  3. [3]

    J., Bethge, M., and Schulz, E

    Akata, E., Schulz, L., Coda-Forno, J., Oh, S. J., Bethge, M., and Schulz, E. Playing repeated games with large language models. arXiv preprint arXiv:2305.16867, 2023

    arXiv 2023

  4. [4]

    Evolutionary optimization of model merging recipes

    Akiba, T., Shing, M., Tang, Y., Sun, Q., and Ha, D. Evolutionary optimization of model merging recipes. arXiv preprint arXiv:2403.13187, 2024

    arXiv 2024

  5. [5]

    Efficient black-box planning using macro-actions with focused effects

    Allen, C., Katz, M., Klinger, T., Konidaris, G., Riemer, M., and Tesauro, G. Efficient black-box planning using macro-actions with focused effects. arXiv preprint arXiv:2004.13242, 2020

    arXiv 2004

  6. [6]

    The falcon series of open language models

    Almazrouei, E., Alobeidli, H., Alshamsi, A., Cappelli, A., Cojocaru, R., Debbah, M., Goffinet, \'E ., Hesslow, D., Launay, J., Malartic, Q., et al. The falcon series of open language models. arXiv preprint arXiv:2311.16867, 2023

    arXiv 2023

  7. [7]

    Hindsight experience replay

    Andrychowicz, M., Wolski, F., Ray, A., Schneider, J., Fong, R., Welinder, P., McGrew, B., Tobin, J., Pieter Abbeel, O., and Zaremba, W. Hindsight experience replay. Advances in neural information processing systems, 30, 2017

    2017

  8. [8]

    V., Dugan, C., Johnson, J., Pan, Q., Zhang, W., Kumaravel, S., and Campbell, M

    Ashktorab, Z., Liao, Q. V., Dugan, C., Johnson, J., Pan, Q., Zhang, W., Kumaravel, S., and Campbell, M. Human-ai collaboration in a cooperative game setting: Measuring social perception and outcomes. Proceedings of the ACM on Human-Computer Interaction, 4 0 (CSCW2): 0 1--20, 2020

    2020

Show all 124 references
  1. [9]

    More effective choice in the prisoner's dilemma

    Axelrod, R. More effective choice in the prisoner's dilemma. Journal of conflict resolution, 24 0 (3): 0 379--403, 1980

    1980

  2. [10]

    The option-critic architecture

    Bacon, P.-L., Harb, J., and Precup, D. The option-critic architecture. In Thirty-First AAAI Conference on Artificial Intelligence, 2017

    2017

  3. [11]

    N., Chandar, S., Burch, N., Lanctot, M., Song, H

    Bard, N., Foerster, J. N., Chandar, S., Burch, N., Lanctot, M., Song, H. F., Parisotto, E., Dumoulin, V., Moitra, S., Hughes, E., et al. The hanabi challenge: A new frontier for ai research. Artificial Intelligence, 280: 0 103216, 2020

    2020

  4. [12]

    theory of mind

    Baron-Cohen, S., Leslie, A. M., and Frith, U. Does the autistic child have a “theory of mind”? Cognition, 21 0 (1): 0 37--46, 1985

    1985

  5. [13]

    Barto, A. G. and Dietterich, T. G. Reinforcement learning and its relationship to supervised learning. Handbook of Learning and Approximate Dynamic Programming, pp.\ 45--63, 2004

    2004

  6. [14]

    Continual learning with self-organizing maps

    Bashivan, P., Schrimpf, M., Ajemian, R., Rish, I., Riemer, M., and Tu, Y. Continual learning with self-organizing maps. arXiv preprint arXiv:1904.09330, 2019

    1904 arXiv

  7. [15]

    Improving cooperation in language games with bayesian inference and the cognitive hierarchy

    Bills, J., Archibald, C., and Blaylock, D. Improving cooperation in language games with bayesian inference and the cognitive hierarchy. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp.\ 13658--13666, 2025

    2025

  8. [16]

    and Schulz, E

    Binz, M. and Schulz, E. Using cognitive psychology to understand gpt-3. Proceedings of the National Academy of Sciences, 120 0 (6): 0 e2218523120, 2023

    2023

  9. [17]

    Brafman, R. I. and Tennenholtz, M. R-max-a general polynomial time algorithm for near-optimal reinforcement learning. Journal of Machine Learning Research, 3 0 (Oct): 0 213--231, 2002

    2002

  10. [18]

    and Sandholm, T

    Brown, N. and Sandholm, T. Superhuman ai for multiplayer poker. Science, 365 0 (6456): 0 885--890, 2019

    2019

  11. [19]

    T., Li, Y., Lundberg, S., et al

    Bubeck, S., Chandrasekaran, V., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E., Lee, P., Lee, Y. T., Li, Y., Lundberg, S., et al. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712, 2023

    2023 arXiv

  12. [20]

    D., Jurafsky, D., et al

    Cases, I., Rosenbaum, C., Riemer, M., Geiger, A., Klinger, T., Tamkin, A., Li, O., Agarwal, S., Greene, J. D., Jurafsky, D., et al. Recursive routing networks: Learning to compose modules for language understanding. In Proceedings of the 2019 Conference of the North American C...

    2019

  13. [21]

    B., Gupta, A., Levine, S., and Griffiths, T

    Chang, M. B., Gupta, A., Levine, S., and Griffiths, T. L. Automatically composing representation transformations as a means for generalization. 2019

    2019

  14. [22]

    Colman, A. M. Cooperation, psychological game theory, and limitations of rationality in social interaction. Behavioral and brain sciences, 26 0 (2): 0 139--153, 2003

    2003

  15. [23]

    D., Riemer, M., Chenthamarakshan, V., Padhi, I., and Mojsilovic, A

    Das, P., Wadhawan, K., Chang, O., Sercu, T., Santos, C. D., Riemer, M., Chenthamarakshan, V., Padhi, I., and Mojsilovic, A. Pepcvae: Semi-supervised targeted design of antimicrobial peptide sequences. arXiv preprint arXiv:1810.07743, 2018

    arXiv

  16. [24]

    Qlora: Efficient finetuning of quantized llms

    Dettmers, T., Pagnoni, A., Holtzman, A., and Zettlemoyer, L. Qlora: Efficient finetuning of quantized llms. Advances in Neural Information Processing Systems, 36, 2024

    2024

  17. [25]

    D., Liu, M., Sattigeri, P., Nagireddy, M., Varshney, K

    Dognin, P., Rios, J., Luss, R., Padhi, I., Riemer, M. D., Liu, M., Sattigeri, P., Nagireddy, M., Varshney, K. R., and Bouneffouf, D. Contextual moral value alignment through context-based aggregation. arXiv preprint arXiv:2403.12805, 2024

    2024 arXiv

  18. [26]

    The llama 3 herd of models

    Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  19. [27]

    predict, then optimize

    Elmachtoub, A. N. and Grigas, P. Smart “predict, then optimize”. Management Science, 68 0 (1): 0 9--26, 2022

    2022

  20. [28]

    Can large language models serve as rational players in game theory? a systematic analysis

    Fan, C., Chen, J., Jin, Y., and He, H. Can large language models serve as rational players in game theory? a systematic analysis. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp.\ 17960--17967, 2024

    2024

  21. [29]

    A., and Shorrer, R

    Fish, S., Gonczarowski, Y. A., and Shorrer, R. I. Algorithmic collusion by large language models. arXiv preprint arXiv:2404.00806, 2024

    2024

  22. [30]

    and Sandholm, T

    Ganzfried, S. and Sandholm, T. Safe opponent exploitation. ACM Transactions on Economics and Computation (TEAC), 3 0 (2): 0 1--28, 2015

    2015

  23. [31]

    Correlated q-learning

    Greenwald, A., Hall, K., Serrano, R., et al. Correlated q-learning. In ICML, volume 3, pp.\ 242--249, 2003

    2003

  24. [32]

    Clochat: Understanding how people customize, interact, and experience personas in large language models

    Ha, J., Jeon, H., Han, D., Seo, J., and Oh, C. Clochat: Understanding how people customize, interact, and experience personas in large language models. In Proceedings of the CHI Conference on Human Factors in Computing Systems, pp.\ 1--24, 2024

    2024

  25. [33]

    F., Hull, R., Khabiri, E., Riemer, M., Sukaviriya, N., and Vacul \' n, R

    Heath, F. F., Hull, R., Khabiri, E., Riemer, M., Sukaviriya, N., and Vacul \' n, R. Alexandria: Extensible framework for rapid exploration of social media. In 2015 IEEE International Congress on Big Data, pp.\ 483--490. IEEE, 2015

    2015

  26. [34]

    J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W

    Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021

    2021 arXiv

  27. [35]

    T., Wortsman, M., Gururangan, S., Schmidt, L., Hajishirzi, H., and Farhadi, A

    Ilharco, G., Ribeiro, M. T., Wortsman, M., Gururangan, S., Schmidt, L., Hajishirzi, H., and Farhadi, A. Editing models with task arithmetic. arXiv preprint arXiv:2212.04089, 2022

    2022 arXiv

  28. [36]

    Q., Sablayrolles, A., Roux, A., Mensch, A., Savary, B., Bamford, C., Chaplot, D

    Jiang, A. Q., Sablayrolles, A., Roux, A., Mensch, A., Savary, B., Bamford, C., Chaplot, D. S., Casas, D. d. l., Hanna, E. B., Bressand, F., et al. Mixtral of experts. arXiv preprint arXiv:2401.04088, 2024

    2024 arXiv

  29. [37]

    P., Littman, M

    Kaelbling, L. P., Littman, M. L., and Cassandra, A. R. Planning and acting in partially observable stochastic domains. Artificial intelligence, 101 0 (1-2): 0 99--134, 1998

    1998

  30. [38]

    B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., and Amodei, D

    Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., and Amodei, D. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361, 2020

    2001 arXiv

  31. [39]

    F., and Hull, R

    Khabiri, E., Riemer, M., Heath III, F. F., and Hull, R. Domain scoping for subject matter experts. In 2015 AAAI Fall Symposium Series, 2015

    2015

  32. [40]

    Towards continual reinforcement learning: A review and perspectives

    Khetarpal, K., Riemer, M., Rish, I., and Precup, D. Towards continual reinforcement learning: A review and perspectives. arXiv preprint arXiv:2012.13490, 2020

    2012 arXiv

  33. [41]

    Kim, D.-K., Liu, M., Omidshafiei, S., Lopez-Cot, S., Riemer, M., Habibi, G., Tesauro, G., Mourad, S., Campbell, M., and How, J. P. Learning hierarchical teaching policies for cooperative agents. arXiv preprint arXiv:1903.03216, 2019 a

    1903 arXiv

  34. [42]

    Kim, D.-K., Liu, M., Omidshafiei, S., Lopez-Cot, S., Riemer, M., Tesauro, G., Campbell, M., Mourad, S., Habibi, G., and How, J. P. Heterogeneous knowledge transfer via hierarchical teaching in cooperative multiagent reinforcement learning. 2019 b

    2019

  35. [43]

    K., Liu, M., Riemer, M

    Kim, D. K., Liu, M., Riemer, M. D., Sun, C., Abdulhai, M., Habibi, G., Lopez-Cot, S., Tesauro, G., and How, J. A policy gradient algorithm for learning to learn in multiagent reinforcement learning. In International Conference on Machine Learning, pp.\ 5541--5550. PMLR, 2021

    2021

  36. [44]

    Kim, D.-K., Riemer, M., Liu, M., Foerster, J., Everett, M., Sun, C., Tesauro, G., and How, J. P. Influencing long-term behavior in multiagent reinforcement learning. Advances in Neural Information Processing Systems, 35: 0 18808--18821, 2022 a

    2022

  37. [45]

    N., Tesauro, G., and How, J

    Kim, D.-K., Riemer, M., Liu, M., Foerster, J. N., Tesauro, G., and How, J. P. Game-theoretical perspectives on active equilibria: A preferred solution concept over nash equilibria. arXiv preprint arXiv:2210.16175, 2022 b

    2022 arXiv

  38. [46]

    L., Kim, G., Choi, Y., and Sap, M

    Kim, H., Sclar, M., Zhou, X., Bras, R. L., Kim, G., Choi, Y., and Sap, M. Fantom: A benchmark for stress-testing machine theory of mind in interactions. arXiv preprint arXiv:2310.15421, 2023

    2023 arXiv

  39. [47]

    A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al

    Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114 0 (13): 0 3521--3526, 2017

    2017

  40. [48]

    S., and Campbell, M

    Klinger, T., Adjodah, D., Marois, V., Joseph, J., Riemer, M., Pentland, A. S., and Campbell, M. A study of compositional generalization in neural models, 2020. URL https://arxiv.org/abs/2006.09437

    2020 arXiv

  41. [49]

    Theory of mind may have spontaneously emerged in large language models

    Kosinski, M. Theory of mind may have spontaneously emerged in large language models. arXiv preprint arXiv:2302.02083, 4: 0 169, 2023

    2023 arXiv

  42. [50]

    Asynchronous coagent networks

    Kostas, J., Nota, C., and Thomas, P. Asynchronous coagent networks. In III, H. D. and Singh, A. (eds.), Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pp.\ 5426--5435. PMLR, 13--18 Jul 2020. URL htt...

    2020

  43. [51]

    Kuhn, H. W. Extensive games and the problem of information. Contributions to the Theory of Games, 2 0 (28): 0 193--216, 1953

    1953

  44. [52]

    Mechanism design for computationally limited agents

    Larson, K. Mechanism design for computationally limited agents. Carnegie Mellon University, 2004

    2004

  45. [53]

    and Yuan, Y

    Leng, Y. and Yuan, Y. Do llm agents exhibit social behavior? arXiv preprint arXiv:2312.15198, 2023

    2023 arXiv

  46. [54]

    Large language model-empowered agents for simulating macroeconomic activities

    Li, N., Gao, C., Li, Y., and Liao, Q. Large language model-empowered agents for simulating macroeconomic activities. Available at SSRN 4606937, 2023

    2023

  47. [55]

    and Hoiem, D

    Li, Z. and Hoiem, D. Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence, 40 0 (12): 0 2935--2947, 2017

    2017

  48. [56]

    Littman, M. L. Markov games as a framework for multi-agent reinforcement learning. In Proceedings of the Eleventh International Conference on International Conference on Machine Learning, ICML'94, pp.\ 157--163. Morgan Kaufmann Publishers Inc., 1994. ISBN 1-55860-335-2. URL ht...

    1994

  49. [57]

    Littman, M. L. et al. Friend-or-foe q-learning in general-sum games. In ICML, volume 1, pp.\ 322--328, 2001

    2001

  50. [58]

    and Heydari, B

    Lor \`e , N. and Heydari, B. Strategic behavior of large language models: Game structure vs. contextual framing. arXiv preprint arXiv:2309.05898, 2023

    2023 arXiv

  51. [59]

    Multi-agent actor-critic for mixed cooperative-competitive environments

    Lowe, R., Tamar, A., Harb, J., Pieter Abbeel, O., and Mordatch, I. Multi-agent actor-critic for mixed cooperative-competitive environments. Advances in neural information processing systems, 30, 2017

    2017

  52. [60]

    Towards a holistic landscape of situated theory of mind in large language models

    Ma, Z., Sansom, J., Peng, R., and Chai, J. Towards a holistic landscape of situated theory of mind in large language models. arXiv preprint arXiv:2310.19619, 2023

    2023 arXiv

  53. [61]

    Malloy, T., Klinger, T., Liu, M., Riemer, M., Tesauro, G., and Sims, C. R. Consolidation via policy information regularization in deep rl for multi-agent games. arXiv preprint arXiv:2011.11517, 2020 a

    2011 arXiv

  54. [62]

    R., Klinger, T., Liu, M., Riemer, M., and Tesauro, G

    Malloy, T., Sims, C. R., Klinger, T., Liu, M., Riemer, M., and Tesauro, G. Deep rl with information constrained policies: Generalization in continuous control. arXiv preprint arXiv:2010.04646, 2020 b

    2010 arXiv

  55. [63]

    Malloy, T., Klinger, T., Liu, M., Tesauro, G., Riemer, M., and Sims, C. R. Rl generalization in a theory of mind game through a sleep metaphor (student abstract). In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp.\ 15841--15842, 2021 a

    2021

  56. [64]

    R., Klinger, T., Liu, M., Riemer, M., and Tesauro, G

    Malloy, T., Sims, C. R., Klinger, T., Liu, M., Riemer, M., and Tesauro, G. Capacity-limited decentralized actor-critic for multi-agent games. In 2021 IEEE Conference on Games (CoG), pp.\ 1--8. IEEE, 2021 b

    2021

  57. [65]

    D., Klinger, T., Tesauro, G., and Sims, C

    Malloy, T., Liu, M., Riemer, M. D., Klinger, T., Tesauro, G., and Sims, C. R. Learning in factored domains with information-constrained visual representations. arXiv preprint arXiv:2303.17508, 2023

    2023 arXiv

  58. [66]

    P., Riemer, M., Bhati, R., and Rish, I

    Memarian, A., Touzel, M. P., Riemer, M., Bhati, R., and Rish, I. Summarizing societies: Agent abstraction in multi-agent reinforcement learning. In From Cells to Societies: Collective Learning across Scales, 2022

    2022

  59. [67]

    R., Pires, B

    Moreno, P., Hughes, E., McKee, K. R., Pires, B. A., and Weber, T. Neural recursive belief states in multi-agent reinforcement learning. arXiv preprint arXiv:2102.02274, 2021

    2021 arXiv

  60. [68]

    E., Bouneffouf, D., and Weisz, J

    Morrison, K., Ashktorab, Z., Gonzalez, G. E., Bouneffouf, D., and Weisz, J. Establishing the cooperative game wavelength as a testbed to explore mutual theory of mind. In AAAI Conference on Artificial Intelligence, 2025

    2025

  61. [69]

    Continuous coordination as a realistic scenario for lifelong learning

    Nekoei, H., Badrinaaraayanan, A., Courville, A., and Chandar, S. Continuous coordination as a realistic scenario for lifelong learning. In International Conference on Machine Learning, pp.\ 8016--8024. PMLR, 2021

    2021

  62. [70]

    Towards few-shot coordination: Revisiting ad-hoc teamplay challenge in the game of hanabi

    Nekoei, H., Zhao, X., Rajendran, J., Liu, M., and Chandar, S. Towards few-shot coordination: Revisiting ad-hoc teamplay challenge in the game of hanabi. In Conference on Lifelong Learning Agents, pp.\ 861--877. PMLR, 2023

    2023

  63. [71]

    M., Zi, Y., Guha, A., Anderson, C

    Nguyen, S., Babe, H. M., Zi, Y., Guha, A., Anderson, C. J., and Feldman, M. Q. How beginning programmers and code llms (mis) read each other. In Proceedings of the CHI Conference on Human Factors in Computing Systems, pp.\ 1--26, 2024

    2024

  64. [72]

    D., Hurtado, J., Khetarpal, K., Lindeborg, R., Cecchi, L., Lesort, T., et al

    Normandin, F., Golemo, F., Ostapenko, O., Rodriguez, P., Riemer, M. D., Hurtado, J., Khetarpal, K., Lindeborg, R., Cecchi, L., Lesort, T., et al. Sequoia: A software framework to unify continual learning research. arXiv preprint arXiv:2108.01005, 2021

    2021 arXiv

  65. [73]

    Omidshafiei, S., Kim, D.-K., Liu, M., Tesauro, G., Riemer, M., Amato, C., Campbell, M., and How, J. P. Learning to teach in cooperative multiagent reinforcement learning. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pp.\ 6128--6136, 2019

    2019

  66. [74]

    Game theory

    Owen, G. Game theory. Emerald Group Publishing, 2013

    2013

  67. [75]

    R., et al

    Padhi, I., Dognin, P., Rios, J., Luss, R., Achintalwar, S., Riemer, M., Liu, M., Sattigeri, P., Nagireddy, M., Varshney, K. R., et al. Comvas: Contextual moral values alignment system. Proceedings of the International Joint Conference on Artificial Intelligence, 2024

    2024

  68. [76]

    S., Zou, C

    Park, J. S., Zou, C. Q., Shaw, A., Hill, B. M., Cai, C., Morris, M. R., Willer, R., Liang, P., and Bernstein, M. S. Generative agent simulations of 1,000 people. arXiv preprint arXiv:2411.10109, 2024

    2024 arXiv

  69. [77]

    Markov decision processes

    Puterman, M. Markov decision processes. 1994. Jhon Wiley & Sons, New Jersey, 1994

    1994

  70. [78]

    A., and Botvinick, M

    Rabinowitz, N., Perbet, F., Song, F., Zhang, C., Eslami, S. A., and Botvinick, M. Machine theory of mind. In International conference on machine learning, pp.\ 4218--4227. PMLR, 2018

    2018

  71. [79]

    Distributed computing in social media analytics

    Riemer, M. Distributed computing in social media analytics. Distributed Computing in Big Data Analytics: Concepts, Technologies and Applications, pp.\ 121--135, 2017

    2017

  72. [80]

    A deep learning and knowledge transfer based architecture for social media user characteristic determination

    Riemer, M., Krasikov, S., and Srinivasan, H. A deep learning and knowledge transfer based architecture for social media user characteristic determination. In Proceedings of the third International Workshop on Natural Language Processing for Social Media, pp.\ 39--47, 2015

    2015

  73. [81]

    Correcting forecasts with multifactor neural attention

    Riemer, M., Vempaty, A., Calmon, F., Heath, F., Hull, R., and Khabiri, E. Correcting forecasts with multifactor neural attention. In International Conference on Machine Learning, pp.\ 3010--3019. PMLR, 2016

    2016

  74. [82]

    Generative knowledge distillation for general purpose function compression

    Riemer, M., Franceschini, M., Bouneffouf, D., and Klinger, T. Generative knowledge distillation for general purpose function compression. NIPS 2017 Workshop on Teaching Machines, Robots, and Humans, 5: 0 30, 2017 a

    2017

  75. [83]

    Representation stability as a regularizer for improved text analytics transfer learning

    Riemer, M., Khabiri, E., and Goodwin, R. Representation stability as a regularizer for improved text analytics transfer learning. arXiv preprint arXiv:1704.03617, 2017 b

    2017 arXiv

  76. [84]

    Learning to learn without forgetting by maximizing transfer and minimizing interference

    Riemer, M., Cases, I., Ajemian, R., Liu, M., Rish, I., Tu, Y., and Tesauro, G. Learning to learn without forgetting by maximizing transfer and minimizing interference. arXiv preprint arXiv:1810.11910, 2018 a

    2018 arXiv

  77. [85]

    Learning abstract options

    Riemer, M., Liu, M., and Tesauro, G. Learning abstract options. Advances in neural information processing systems, 31, 2018 b

    2018

  78. [86]

    Scalable recollections for continual lifelong learning

    Riemer, M., Klinger, T., Bouneffouf, D., and Franceschini, M. Scalable recollections for continual lifelong learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pp.\ 1352--1359, 2019

    2019

  79. [87]

    On the role of weight sharing during deep option learning

    Riemer, M., Cases, I., Rosenbaum, C., Liu, M., and Tesauro, G. On the role of weight sharing during deep option learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp.\ 5519--5526, 2020

    2020

  80. [88]

    C., Cases, I., Subbaraj, G., Touzel, M

    Riemer, M., Raparthy, S. C., Cases, I., Subbaraj, G., Touzel, M. P., and Rish, I. Continual learning in environments with polynomial mixing times. Advances in Neural Information Processing Systems, 2022

    2022

  81. [89]

    Balancing context length and mixing times for reinforcement learning at scale

    Riemer, M., Khetarpal, K., Rajendran, J., and Chandar, S. Balancing context length and mixing times for reinforcement learning at scale. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024 a

    2024

  82. [90]

    Enabling realtime reinforcement learning at scale with staggered asynchronous inference, 2024 b

    Riemer, M., Subbaraj, G., Berseth, G., and Rish, I. Enabling realtime reinforcement learning at scale with staggered asynchronous inference, 2024 b . URL https://arxiv.org/abs/2412.14355

    2024 arXiv

  83. [91]

    Realtime reinforcement learning: Towards rapid asynchronous deployment of large models

    Riemer, M., Subbaraj, G., Berseth, G., and Rish, I. Realtime reinforcement learning: Towards rapid asynchronous deployment of large models. In ICML 2024 Workshop: Aligning Reinforcement Learning Experimentalists and Theorists, 2024 c

    2024

  84. [92]

    C., and Hovy, D

    Rooein, D., Curry, A. C., and Hovy, D. Know your audience: Do llms adapt to different age and education levels? arXiv preprint arXiv:2312.02065, 2023

    2023 arXiv

  85. [93]

    Routing networks: Adaptive selection of non-linear functions for multi-task learning

    Rosenbaum, C., Klinger, T., and Riemer, M. Routing networks: Adaptive selection of non-linear functions for multi-task learning. In International Conference on Learning Representations. International Conference on Learning Representations, ICLR, 2018

    2018

  86. [94]

    D., Jurafsky, D., and Potts, C

    Rosenbaum, C., Cases, I., Riemer, M., Geiger, A., Karttunen, L., Greene, J. D., Jurafsky, D., and Potts, C. Dispatched routing networks. Technical report, 2019 a

    2019

  87. [95]

    Routing networks and the challenges of modular and compositional computation

    Rosenbaum, C., Cases, I., Riemer, M., and Klinger, T. Routing networks and the challenges of modular and compositional computation. arXiv preprint arXiv:1904.12774, 2019 b

    1904 arXiv

  88. [96]

    Universal value function approximators

    Schaul, T., Horgan, D., Gregor, K., and Silver, D. Universal value function approximators. In International conference on machine learning, pp.\ 1312--1320. PMLR, 2015

    2015

  89. [97]

    Mastering atari, go, chess and shogi by planning with a learned model

    Schrittwieser, J., Antonoglou, I., Hubert, T., Simonyan, K., Sifre, L., Schmitt, S., Guez, A., Lockhart, E., Hassabis, D., Graepel, T., et al. Mastering atari, go, chess and shogi by planning with a learned model. Nature, 588 0 (7839): 0 604--609, 2020

    2020

  90. [98]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models

    Shao, Z., Wang, P., Zhu, Q., Xu, R., Song, J., Bi, X., Zhang, H., Zhang, M., Li, Y., Wu, Y., et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024

    2024 arXiv

  91. [99]

    Shapley, L. S. Stochastic games. Proceedings of the National Academy of Sciences, 39 0 (10): 0 1095--1100, 1953. ISSN 0027-8424. doi:10.1073/pnas.39.10.1095. URL https://www.pnas.org/content/39/10/1095

    1953 doi

  92. [100]

    Practices for governing agentic ai systems

    Shavit, Y., Agarwal, S., Brundage, M., Adler, S., O’Keefe, C., Campbell, R., Lee, T., Mishkin, P., Eloundou, T., Hickey, A., et al. Practices for governing agentic ai systems. arXiv preprint, 2023

    2023

  93. [101]

    Reflexion: Language agents with verbal reinforcement learning

    Shinn, N., Cassano, F., Gopinath, A., Narasimhan, K., and Yao, S. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 36, 2024

    2024

  94. [102]

    W., Albergo, D., Borghini, G., Pansardi, O., Scaliti, E., Gupta, S., Saxena, K., Rufo, A., Panzeri, S., Manzi, G., et al

    Strachan, J. W., Albergo, D., Borghini, G., Pansardi, O., Scaliti, E., Gupta, S., Saxena, K., Rufo, A., Panzeri, S., Manzi, G., et al. Testing theory of mind in large language models and humans. Nature Human Behaviour, pp.\ 1--11, 2024

    2024

  95. [103]

    O., Keeling, G., Baranes, A., Barnett, B., McKibben, M., Kanyere, T., Lentz, A., Dunbar, R

    Street, W., Siy, J. O., Keeling, G., Baranes, A., Barnett, B., McKibben, M., Kanyere, T., Lentz, A., Dunbar, R. I., et al. Llms achieve adult human performance on higher-order theory of mind tasks. arXiv preprint arXiv:2405.18870, 2024

    2024 arXiv

  96. [104]

    Sutton, R. S. Dyna, an integrated architecture for learning, planning, and reacting. ACM Sigart Bulletin, 2 0 (4): 0 160--163, 1991

    1991

  97. [105]

    Sutton, R. S. and Barto, A. G. Reinforcement learning: An introduction. 2018

    2018

  98. [106]

    S., Precup, D., and Singh, S

    Sutton, R. S., Precup, D., and Singh, S. Between mdps and semi-mdps: A framework for temporal abstraction in reinforcement learning. Artificial intelligence, 112 0 (1-2): 0 181--211, 1999

    1999

  99. [107]

    D., Chen, P.-Y., Zouaq, A., Das, P., and Chandar, S

    Thakkar, M., Fournier, Q., Riemer, M. D., Chen, P.-Y., Zouaq, A., Das, P., and Chandar, S. A deep dive into the trade-offs of parameter-efficient preference alignment techniques, 2024 a . URL https://arxiv.org/abs/2406.04879

    2024 arXiv

  100. [108]

    Combining domain and alignment vectors to achieve better knowledge-safety trade-offs in llms

    Thakkar, M., More, Y., Fournier, Q., Riemer, M., Chen, P.-Y., Zouaq, A., Das, P., and Chandar, S. Combining domain and alignment vectors to achieve better knowledge-safety trade-offs in llms. arXiv preprint arXiv:2411.06824, 2024 b

    2024 arXiv

  101. [109]

    Thomas, P. S. Policy gradient coagent networks. In Shawe-Taylor, J., Zemel, R., Bartlett, P., Pereira, F., and Weinberger, K. (eds.), Advances in Neural Information Processing Systems, volume 24. Curran Associates, Inc., 2011. URL https://proceedings.neurips.cc/paper_files/pap...

    2011

  102. [110]

    Llama 2: Open foundation and fine-tuned chat models

    Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

  103. [111]

    P., Memarian, A., Riemer, M., Mircea, A., Williams, A

    Touzel, M. P., Memarian, A., Riemer, M., Mircea, A., Williams, A. R., Ahlstrand, E., Lehnert, L., Bhati, R., Dumas, G., and Rish, I. Scalable approaches for a theory of many minds. In Agentic Markets Workshop at ICML 2024, 2024

    2024

  104. [112]

    and Goel, A

    Wang, Q. and Goel, A. K. Mutual theory of mind for human-ai communication. arXiv preprint arXiv:2210.03842, 2022

    2022 arXiv

  105. [113]

    and Sandholm, T

    Wang, X. and Sandholm, T. Reinforcement learning to play an optimal nash equilibrium in team markov games. Advances in neural information processing systems, 15, 2002

    2002

  106. [114]

    Watkins, C. J. C. H. Learning from delayed rewards. 1989

    1989

  107. [115]

    V., Zhou, D., et al

    Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q. V., Zhou, D., et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35: 0 24824--24837, 2022

    2022

  108. [116]

    and Sukhbaatar, S

    Weston, J. and Sukhbaatar, S. System 2 attention (is something you might need too). arXiv preprint arXiv:2311.11829, 2023

    2023 arXiv

  109. [117]

    Can large language model agents simulate human trust behaviors? arXiv preprint arXiv:2402.04559, 2024

    Xie, C., Chen, C., Jia, F., Ye, Z., Shu, K., Bibi, A., Hu, Z., Torr, P., Ghanem, B., and Li, G. Can large language model agents simulate human trust behaviors? arXiv preprint arXiv:2402.04559, 2024

    2024 arXiv

  110. [118]

    Walking in others' shoes: How perspective-taking guides large language models in reducing toxicity and bias

    Xu, R., Zhou, Z., Zhang, T., Qi, Z., Yao, S., Xu, K., Xu, W., and Qiu, H. Walking in others' shoes: How perspective-taking guides large language models in reducing toxicity and bias. arXiv preprint arXiv:2407.15366, 2024

    2024 arXiv

  111. [119]

    A., and Bansal, M

    Yadav, P., Tam, D., Choshen, L., Raffel, C. A., and Bansal, M. Ties-merging: Resolving interference when merging models. Advances in Neural Information Processing Systems, 36, 2024

    2024

  112. [120]

    Cm3: Cooperative multi-goal multi-stage multi-agent reinforcement learning

    Yang, J., Nakhaei, A., Isele, D., Fujimura, K., and Zha, H. Cm3: Cooperative multi-goal multi-stage multi-agent reinforcement learning. arXiv preprint arXiv:1809.05188, 2018

    2018 arXiv

  113. [121]

    Mutual theory of mind in human-ai collaboration: An empirical study with llm-driven ai agents in a real-time shared workspace task

    Zhang, S., Wang, X., Zhang, W., Chen, Y., Gao, L., Wang, D., Zhang, W., Wang, X., and Wen, Y. Mutual theory of mind in human-ai collaboration: An empirical study with llm-driven ai agents in a real-time shared workspace task. arXiv preprint arXiv:2409.08811, 2024

    2024 arXiv

  114. [122]

    S., Pedramfar, M., Riemer, M., Moradipari, A., and Liu, M

    Zini, M. S., Pedramfar, M., Riemer, M., Moradipari, A., and Liu, M. Coagent networks revisited. arXiv preprint arXiv:2001.10474, 2020

    2001 arXiv

  115. [123]

    Cyclic equilibria in markov games

    Zinkevich, M., Greenwald, A., and Littman, M. Cyclic equilibria in markov games. Advances in neural information processing systems, 18, 2005

    2005

  116. [124]

    Regret minimization in games with incomplete information

    Zinkevich, M., Johanson, M., Bowling, M., and Piccione, C. Regret minimization in games with incomplete information. Advances in neural information processing systems, 20, 2007

    2007

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.