Pith. sign in

REVIEW 3 major objections 4 minor 2 cited by

Language-Informed Synthesis of Rational Agent Models for Grounded Theory-of-Mind Reasoning On-The-Fly

T0 review · 3 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read LIRAS synthesizes PDDL world and agent models from language and video, then runs Bayesian inverse planning to emit graded judgments of goals, beliefs, rewards, and costs.

desk verdict A well-built LLM-to-PDDL plus Bayesian inverse planning pipeline that hits human noise ceilings on several theory-of-mind tasks, but the near-ceiling correlations rest on a single synthesis sample per stimulus and the abstract overclaims on m-DKG. read the letter →

arxiv 2506.16755 v1 pith:32X6XWN4 submitted 2025-06-20 cs.CL cs.AI

Lance Ying , Ryan Truong , Katherine M. Collins , Cedegao E. Zhang , Megan Wei , Tyler Brooke-Wilson , Tan Zhi-Xuan , Lionel Wong
show 1 more author
Joshua B. Tenenbaum
This is my paper · ORCID
classification cs.CLcs.AI
keywords theoryofmindBayesianinverseplanningmultimodalreasoninglanguage-conditionedmodelsynthesisPDDLrationalagentssocialcognitionsequentialagentmodeling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

LIRAS tries to show that grounded, situation-specific theory-of-mind reasoning can be automated by synthesizing a symbolic rational agent model from language and then inverting it against parsed visual evidence. Given a linguistic description and video frames, a multimodal LLM compiles a PDDL planning domain and an agent configuration (goal space, rewards, costs, observability, temperature), a VLM parses each grid cell into symbolic states, and the Sequential Inverse Agent Modeling engine runs Bayesian inverse planning to output graded probabilities over goals, beliefs, rewards, and costs. The paper reports that this pipeline correlates with human judgments between 0.74 and 0.87 across five cognitive-science domains, reaching the split-half noise ceiling of human data on DKG variants, and that it outperforms GPT-4o, OpenAI o3, and the same VLM used end-to-end. The wider point is that language can restructure social inference through explicit model construction rather than through pattern matching alone.

What carries the argument

The carrying object is the synthesized (PO)MDP agent model: a PDDL domain generated from the linguistic description, agent parameters covering goal predicates, reward and action-cost tables, observability and belief configuration, and Boltzmann temperature, plus VLM-parsed environment states with derived actions. SIAM, Sequential Inverse Agent Modeling, extends Sequential Inverse Plan Search with joint inference over goals, rewards, costs, and beliefs; it enumerates hypotheses over mental states, updates belief states, computes expected utilities via memoized A* path costs, and weights each hypothesis by the Boltzmann probability of the observed action sequence. The mechanism shows how language can change the structure of the inference problem itself.

What would settle it

Take one DKG-Reuse stimulus and deliberately change one generated rule so the PDDL contradicts the intended instructions, for example removing the reusability of keys while keeping visuals and queries fixed; if LIRAS's human correlation does not drop, the synthesized model is not what carries the inference. The paper already reports hallucination and syntax errors that are removed by rejection sampling, so the residual unsampled parses would be the place to look for systematic distortion.

Watch

Extended reading notes

Core claim

The central claim is that the bottleneck in multimodal social reasoning is not perception but the absence of an explicit, task-specific generative model of the agent and environment. LIRAS has the LLM translate the instructions into PDDL and into an agent model with priors over goals, rewards, costs, and observability, parses the video frames cell by cell into matching PDDL states, and then uses SIAM to compute a posterior over mental states by Boltzmann-rational action likelihoods with memoized shortest-path utilities. On the food-truck, astronaut, DKG-Single/Double/Reuse/Inverse, and m-DKG tasks, the full LIRAS pipeline produces human-likeness correlations between about 0.74 and 0.87, while direct VLM prompting and even the ablated symbolic-synthesis-without-Bayes version fall much lower or negative. This is what the authors are trying to establish: a lightweight VLM plus explicit inverse planning can match the graded, language-sensitive social inferences people make, where larger end-to-end models cannot.

Load-bearing premise

The whole pipeline rests on the assumption that the LLM-generated PDDL domain, the LLM-generated agent configuration, and the VLM's cell-by-cell state parses jointly recover the exact task content that human participants read and saw, with any residual errors staying benign enough not to distort the inferred posterior.

Editorial extensions

If this is right

  • Foundation-model baselines, including o3, give uneven and sometimes negative correlations with human judgments, whereas LIRAS with a lightweight VLM stays at 0.74 or above on every domain tested.
  • The ablation shows that synthesized symbolic models alone are not enough: asking the LLM to answer from the same PDDL and states performs at base-VLM level, so the explicit Bayesian inversion carries the human-likeness.
  • Changing the linguistic rule (key double, reuse, inverse) shifts human goal judgments, and LIRAS tracks those shifts while baselines degrade most on the unusual reuse variant.
  • Near-noise-ceiling correlation on the DKG variants means LIRAS accounts for nearly all reliable human variance in those stimulus sets.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the pattern holds beyond gridworlds, language-conditioned model synthesis plus Bayesian inversion could become a probe for how narration restructures mental models in humans, not just a benchmark.
  • A natural test is to feed LIRAS incomplete or implicit rule descriptions; if its posterior degrades the same way humans' does, the framework would help locate where commonsense inference takes over from explicit linguistic information.
  • The cell-by-cell VLM parsing and rejection sampling suggest a scalability bottleneck: continuous domains and visually rich scenes would require a different parser, so the contribution may be best framed as a proof-of-concept for discrete, language-defined environments.
  • The ablated result implies that neural models can assemble the right symbols yet still fail to reason over them; that suggests hybrid neuro-symbolic architectures may benefit human-AI collaboration even when the neural component is small.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces LIRAS, a neuro-symbolic framework that synthesizes a PDDL environment model, a rational agent configuration, and cell-level visual state parses from language and video inputs, and then runs Bayesian inverse planning (SIAM) to output graded theory-of-mind ratings. It evaluates on two published cognitive-science datasets (Food Truck, Astronaut) and five DKG variants, including rule-modified and multi-agent versions, comparing correlations with human judgments against Gemini 2.0 Flash, GPT-4o, OpenAI o3, and an ablation that replaces the Bayesian engine with direct LLM inference. The reported LIRAS correlations (0.74–0.87) are close to or above the split-half reliability of the human data on DKG variants, and the qualitative example in Figure 2 illustrates that LIRAS, unlike baselines, shifts its goal posterior when the language changes from the DKG-Single to the DKG-Reuse rule.

Significance. If the reported results hold, the paper would make a valuable contribution: it demonstrates that a lightweight VLM, coupled with LLM-driven synthesis of structured world models and Bayesian inverse planning, can capture graded human mental-state judgments across diverse tasks without fitting any parameters to the human responses. The external human benchmark and the ablation isolating the Bayesian inference engine are notable strengths, as is the use of paired DKG stimuli to show sensitivity to linguistic rule changes. However, the central quantitative claim currently rests on a single stochastic synthesis sample per stimulus and on an unquantified assumption that residual parsing and synthesis errors are benign, so the significance cannot be fully assessed without additional sampling and parse-accuracy evidence.

major comments (3)
  1. [Section 3.3 and Eqs. (1), (6), (7)] All LIRAS results are generated with k=1 stochastic synthesis sample per stimulus, with no seed variance reported; because the PDDL domain, agent configuration, and visual state parses are all outputs of stochastic LLM/VLM sampling, the correlations in Tables 1 and 2 are one draw from a distribution over compiled models. The statement that 'semantic variation among models that actually compile is minimal' is asserted without quantitative support, and Section 4.1 confirms that hallucination and syntax errors were frequent enough to require resampling. Please report the distribution of correlations over multiple independent samples (or at least inter-sample agreement among ratings), the rejection and compile rates per stimulus, and an analysis showing that the retained samples are not selected in a way that systematically favors human agreement; without this, the near-ceiling correlations could reflect sampling luck rather than a property of the LIRAS architecture.
  2. [Section 4.1] The paper concedes hallucination and syntax errors in world-model synthesis and visual parsing, but provides no quantitative parse-accuracy measure or audit of residual errors. Since SIAM's posterior is computed over the parsed PDDL states and reconstructed actions, the claim that LIRAS matches human judgments at the split-half ceiling requires evidence that the residual errors are benign; for example, a manual audit of state and action sequences on a random stimulus subset, or an analysis of whether stimuli that required resampling produce ratings that differ systematically from those that did not.
  3. [Sections 3.3 and 3.4] The evaluation protocol averages baselines over k=3 samples while reporting only k=1 for LIRAS; because the paper notes that single-sample baseline results are significantly more variable and worse, the single-sample LIRAS correlations are not directly comparable to the averaged baselines under sampling noise. Please report LIRAS results under the same k=3 protocol, or justify statistically that the single-sample estimates are stable, so the comparisons in Tables 1 and 2 are on equal footing.
minor comments (4)
  1. [Tables 1 and 2] The paper says that results 'statistically significant from others are bolded' but never describes the significance test, the unit of analysis, or whether the confidence intervals are bootstrap intervals over stimuli; please specify the procedure.
  2. [Section 3.2] The human data section reports n=20 per variant (100 total) but then states that 13 participants were excluded; please clarify the final sample size per variant and whether the split-half reliability in Table 2 is computed on the post-exclusion sample.
  3. [References and prose] The same reference appears as both 'Buschoff et al. (2025)' and 'Schulze Buschoff et al. (2025)'; please standardize. There is also a typo in Section 6.3 ('to to synthesize code') and a grammatical issue in the Figure 2 caption ('were tested').
  4. [Section 7 and Appendix E] Because the method depends on explicitly provided action spaces and transition rules, and the paper acknowledges that humans can infer such rules implicitly, the title and abstract overstate the 'on-the-fly' generality; a more precise claim would be that LIRAS synthesizes models from explicit linguistic instructions on discrete gridworld tasks.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: LIRAS predictions are produced by Bayesian inverse planning over language- and vision-derived symbolic models, with human judgments used only as an external benchmark.

full rationale

The derivation chain is self-contained against an external benchmark. LIRAS synthesizes a PDDL environment model from language (Eq. 1), synthesizes agent-model parameters from language (Eq. 6), parses visual states with a VLM (Eq. 7), reconstructs actions greedily (Eq. 8), and computes mental-state posteriors via SIAM inverse planning. Human ratings appear only as the evaluation target in Tables 1 and 2; they never enter the synthesis, parsing, or inference equations. Goals, rewards, costs, observability, and temperature are generated by an LLM from the linguistic description, with uniform priors over goal/reward/cost spaces, not fitted to human responses. The paper's self-citations (SIPS/SIAM lineage, Ying et al. 2025a/b) are technical and philosophical lineage, not load-bearing uniqueness claims, and no cited theorem is invoked to forbid alternative models. The Section 3.3 statement that k=1 sample is sufficient because 'semantic variation among models that actually compile is minimal' is an unquantified robustness assumption, and Section 4.1 concedes hallucination and syntax errors requiring resampling; these are correctness and statistical-reliability concerns, not circularity, because the retained samples are selected only for compilation, not for agreement with human judgments. No step reduces a prediction to its input by construction, and no fitted parameter is renamed as a prediction.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

The framework's central claim depends on the rationality assumptions of inverse planning (uniform priors, Boltzmann action selection), on the faithfulness of LLM/VLM-synthesized PDDL and agent configurations, and on the representativeness of a single synthesis sample. No new physical entities are introduced; the method's assumptions are all about representational fidelity and modeling choices inherited from Bayesian theory-of-mind.

free parameters (3)
  • Agent rationality temperature beta = LLM-synthesized; default 1 (Appendix E.3)
    Scales the Boltzmann-rational action likelihood in Eq. 5 and directly controls the sharpness of all posterior inferences. Chosen by the LLM from the language description, not derived from first principles.
  • Action cost profiles = e.g., movement costs 1, 2, or 4 and pickup 5 in the prompt example (Appendix E.3)
    Cost values for each action and terrain enter the expected-utility and shortest-path computations in SIAM; the LLM generates them per stimulus without independent justification.
  • Goal rewards (when inferred) = LLM-synthesized per stimulus
    In food truck and Astronaut domains, reward magnitudes for desires are part of the synthesized agent model; posterior estimates of rewards depend on these values.
assumptions (6)
  • domain assumption Agents are approximately rational: beliefs are consistent with observations and actions are Boltzmann-rational under expected utility minus costs.
    Equations 2-5 in Section 2.2; this generative model is the foundation of the inverse-planning engine and is assumed, not derived.
  • domain assumption The PDDL domain synthesized by the LLM faithfully encodes the transition dynamics described in the linguistic instructions.
    Section 2.1; if the synthesized domain misstates rules such as key reuse, inverse-planning posteriors will be wrong.
  • domain assumption The VLM's cell-by-cell parsing yields an accurate symbolic state sequence with no systematic bias.
    Sections 2.3 and 4.1; hallucination and syntax errors are acknowledged, and mitigation is resampling until compile, which does not guarantee semantic accuracy.
  • ad hoc to paper A single synthesis sample per stimulus is representative; semantic variation among compiling models is minimal.
    Section 3.3; stated without quantitative support; all reported LIRAS correlations depend on this.
  • standard math A* search provides exact shortest-path costs in deterministic PDDL environments, with correct memoization across belief states.
    Appendix D; standard algorithm, correctness is assumed.
  • domain assumption Mean human judgments and split-half correlations are adequate ground-truth measurements for graded social inferences.
    Section 3.2; published domains provide only mean ratings, and new DKG data are averaged over 20 participants per variant.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Language-Informed Synthesis of Rational Agent Models for Grounded Theory-of-Mind Reasoning On-The-Fly." pith.science (2026). https://pith.science/paper/32X6XWN4

@misc{pith2026250616755,
  author       = {Pith},
  title        = {Pith review of: Language-Informed Synthesis of Rational Agent Models for Grounded Theory-of-Mind Reasoning On-The-Fly},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/32X6XWN4}},
  note         = {Machine review of arXiv:2506.16755}
}
read the original abstract

Drawing real world social inferences usually requires taking into account information from multiple modalities. Language is a particularly powerful source of information in social settings, especially in novel situations where language can provide both abstract information about the environment dynamics and concrete specifics about an agent that cannot be easily visually observed. In this paper, we propose Language-Informed Rational Agent Synthesis (LIRAS), a framework for drawing context-specific social inferences that integrate linguistic and visual inputs. LIRAS frames multimodal social reasoning as a process of constructing structured but situation-specific agent and environment representations - leveraging multimodal language models to parse language and visual inputs into unified symbolic representations, over which a Bayesian inverse planning engine can be run to produce granular probabilistic judgments. On a range of existing and new social reasoning tasks derived from cognitive science experiments, we find that our model (instantiated with a comparatively lightweight VLM) outperforms ablations and state-of-the-art models in capturing human judgments across all domains.

Figures

Figures reproduced from arXiv: 2506.16755 by the authors.

Figure 1
Figure 1. Overview of Language-Informed Rational Agent Synthesis (LIRAS). [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. A qualitative example showing how models [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Experiment Interface for the human data collection. [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Scatterplots of model vs human judgments on Foodtruck and Astronaut domains. Error bar indicates [PITH_FULL_IMAGE:figures/full_fig_p018_4.png]
Figure 5
Figure 5. Figure 5: Scatterplots of model vs human judgments on DKG domains and variants. Error bars indicate standard [PITH_FULL_IMAGE:figures/full_fig_p019_5.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Using Theory of Mind to Arbitrate between Social and Non-social Learning

    cs.MA 2026-07 accept novelty 6.0 of 10

    Selective social learning is captured by a Rational Mentalizing model that uses Theory of Mind to estimate observation utility and arbitrates against non-social exploration cost.

  2. Modeling Open-World Cognition as On-Demand Synthesis of Probabilistic Models

    cs.CL 2025-07 conditional novelty 6.0 of 10

    A hybrid language-model and probabilistic-program architecture predicts human judgments on novel open-world reasoning vignettes better than language-model-only baselines.

Reference graph

Works this paper leans on

52 extracted references · 27 canonical work pages · cited by 2 Pith papers

  1. [1]

    Constructions Aeronautiques, Adele Howe, Craig Knoblock, ISI Drew McDermott, Ashwin Ram, Manuela Veloso, Daniel Weld, David Wilkins SRI, Anthony Barrett, Dave Christianson, and 1 others. 1998. PDDL | T he P lanning D omain D efinition language. Technical Report, Tech. Rep

    work page 1998

  2. [2]

    Arwa Alanqary, Gloria Z Lin, Joie Le, Tan Zhi-Xuan, Vikash K Mansinghka, and Joshua B Tenenbaum. 2021. Modeling the mistakes of boundedly rational agents within a bayesian theory of mind. arXiv preprint arXiv:2106.13249

    arXiv 2021

  3. [3]

    Ren \'e e Baillargeon, Rose M Scott, and Lin Bian. 2016. Psychological reasoning in infancy. Annual review of psychology, 67:159--186

    work page 2016

  4. [4]

    Chris Baker, Rebecca Saxe, and Joshua Tenenbaum. 2011. Bayesian theory of mind: Modeling joint belief-desire attribution. In Proceedings of the annual meeting of the cognitive science society, volume 33

    work page 2011

  5. [5]

    Chris L Baker, Julian Jara-Ettinger, Rebecca Saxe, and Joshua B Tenenbaum. 2017. Rational quantitative attribution of beliefs, desires and percepts in human mentalizing. Nature Human Behaviour, 1(4):0064

    work page 2017

  6. [6]

    Chris L Baker, Rebecca Saxe, and Joshua B Tenenbaum. 2009. Action understanding as inverse planning. Cognition, 113(3):329--349

    work page 2009

  7. [7]

    Richard Bellman. 1958. Dynamic programming and stochastic control processes. Information and control, 1(3):228--239

    work page 1958

  8. [8]

    Tyler Brooke-Wilson. 2023. https://philosophy.mit.edu/wp-content/uploads/brookewilson_dissertation.pdf Bounded Rationality as a Strategy for Cognitive Science . Ph.D. thesis, Massachusetts Institute of Technology, Cambridge, MA

    work page 2023

Show all 52 references
  1. [9]

    Luca Buschoff, Elif Akata, Matthias Bethge, and Eric Schulz. 2025. https://doi.org/10.1038/s42256-024-00963-y Visual cognition in multimodal large language models . Nature Machine Intelligence, 7:96--106

    2025 doi

  2. [10]

    Katherine M Collins, Ilia Sucholutsky, Umang Bhatt, Kartik Chandra, Lionel Wong, Mina Lee, Cedegao E Zhang, Tan Zhi-Xuan, Mark Ho, Vikash Mansinghka, and 1 others. 2024. Building machines that learn and think with people. Nature Human Behavior, 8(10):1851--1863

    2024

  3. [11]

    Logan Cross, Violet Xiang, Agam Bhatia, Daniel LK Yamins, and Nick Haber. 2024. Hypothetical minds: Scaffolding theory of mind for multi-agent tasks with large language models. arXiv preprint arXiv:2407.07086

    2024 arXiv

  4. [12]

    Daniel Clement Dennett. 1981. The Intentional Stance. MIT Press

    1981

  5. [13]

    Justin Domke. 2025. https://arxiv.org/abs/2504.14025 Large language bayes . Preprint, arXiv:2504.14025

    2025

  6. [14]

    Carlos Hern \'a ndez, Xiaoxun Sun, Sven Koenig, and Pedro Meseguer. 2011. Tree Adaptive A* . In The 10th International Conference on Autonomous Agents and Multiagent Systems-Volume 1, pages 123--130

    2011

  7. [15]

    Jennifer Hu, Felix Sosa, and Tomer Ullman. 2025. Re-evaluating theory of mind evaluation in large language models. arXiv preprint arXiv:2502.21098

    2025 arXiv

  8. [16]

    Julian Jara-Ettinger, Hyowon Gweon, Laura E Schulz, and Joshua B Tenenbaum. 2016. The na \" ve utility calculus: Computational principles underlying commonsense psychology. Trends in cognitive sciences, 20(8):589--604

    2016

  9. [17]

    Julian Jara-Ettinger, Laura E Schulz, and Joshua B Tenenbaum. 2020. The naive utility calculus as a unified, quantitative framework for action understanding. Cognitive Psychology, 123:101334

    2020

  10. [18]

    Chuanyang Jin, Yutong Wu, Jing Cao, Jiannan Xiang, Yen-Ling Kuo, Zhiting Hu, Tomer Ullman, Antonio Torralba, Joshua B Tenenbaum, and Tianmin Shu. 2024. Mmtom-qa: Multimodal theory of mind question answering. arXiv preprint arXiv:2401.08743

    2024 arXiv

  11. [19]

    Leslie Pack Kaelbling, Michael L Littman, and Anthony R Cassandra. 1998. Planning and acting in partially observable stochastic domains. Artificial intelligence, 101(1-2):99--134

    1998

  12. [20]

    Hyunwoo Kim, Melanie Sclar, Tan Zhi-Xuan, Lance Ying, Sydney Levine, Yang Liu, Joshua B Tenenbaum, and Yejin Choi. 2025. Hypothesis-driven theory-of-mind reasoning for large language models. arXiv preprint arXiv:2502.11881

    2025 arXiv

  13. [21]

    Michal Kosinski. 2023. Theory of mind may have spontaneously emerged in large language models. arXiv preprint arXiv:2302.02083

    2023 arXiv

  14. [22]

    Michal Kosinski. 2024. Evaluating large language models in theory of mind tasks. Proceedings of the National Academy of Sciences, 121(45):e2405460121

    2024

  15. [23]

    Michael Y Li, Emily Fox, and Noah Goodman. 2024. Automated statistical model discovery with language models. In Forty-first International Conference on Machine Learning

    2024

  16. [24]

    Bo Liu, Yuqian Jiang, Xiaohan Zhang, Qiang Liu, Shiqi Zhang, Joydeep Biswas, and Peter Stone. 2023. https://arxiv.org/abs/2304.11477 Llm+p: Empowering large language models with optimal planning proficiency . Preprint, arXiv:2304.11477

    2023 arXiv

  17. [25]

    Shima Rahimi Moghaddam and Christopher J Honey. 2023. Boosting theory-of-mind performance in large language models via prompting. arXiv preprint arXiv:2304.11490

    2023 arXiv

  18. [26]

    OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff ...

    2024 arXiv

  19. [27]

    Linlu Qiu, Fei Sha, Kelsey Allen, Yoon Kim, Tal Linzen, and Sjoerd van Steenkiste. 2025. Bayesian teaching enables probabilistic reasoning in large language models. arXiv preprint arXiv:2503.17523

    2025

  20. [28]

    Maarten Sap, Ronan Le Bras, Daniel Fried, and Yejin Choi. 2022. Neural theory-of-mind? on the limits of social intelligence in large lms. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3762--3780

    2022

  21. [29]

    Luca M Schulze Buschoff, Elif Akata, Matthias Bethge, and Eric Schulz. 2025. Visual cognition in multimodal large language models. Nature Machine Intelligence, pages 1--11

    2025

  22. [30]

    Haojun Shi, Suyu Ye, Xinyu Fang, Chuanyang Jin, Leyla Isik, Yen-Ling Kuo, and Tianmin Shu. 2025. Muma\-tom: Multi-modal multi-agent theory of mind. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 1510--1519

    2025

  23. [31]

    Michael Shum, Max Kleiman-Weiner, Michael L Littman, and Joshua B Tenenbaum. 2019. Theory of minds: Understanding behavior in groups through inverse planning. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 6163--6170

    2019

  24. [32]

    Tenenbaum, Leslie Pack Kaelbling, and Michael Katz

    Tom Silver, Soham Dan, Kavitha Srinivas, Joshua B. Tenenbaum, Leslie Pack Kaelbling, and Michael Katz. 2023. https://arxiv.org/abs/2305.11014 Generalized planning in pddl domains with pretrained large language models . Preprint, arXiv:2305.11014

    2023 arXiv

  25. [33]

    Hao Tang, Darren Key, and Kevin Ellis. 2024. Worldcoder, a model-based llm agent: Building world models by writing code and interacting with the environment. Advances in Neural Information Processing Systems, 37:70148--70212

    2024

  26. [34]

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M. Dai, Anja Hauth, Katie Millican, David Silver, Melvin Johnson, Ioannis Antonoglou, Julian Schrittwieser, Amelia Glaese, Jilin Chen, Emily Pitler, Timothy Lil...

    2025 arXiv

  27. [35]

    Tomer Ullman. 2023. Large language models fail on trivial alterations to theory-of-mind tasks. arXiv preprint arXiv:2302.08399

    2023 arXiv

  28. [36]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed H Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems, volume 35, pages 24824--24837

    2022

  29. [37]

    Lew, Noah D

    Lionel Wong, Gabriel Grand, Alexander K. Lew, Noah D. Goodman, Vikash K. Mansinghka, Jacob Andreas, and Joshua B. Tenenbaum. 2023. https://arxiv.org/abs/2306.12672 From word models to world models: Translating from natural language to the probabilistic language of thought . ar...

    2023 arXiv

  30. [38]

    Tenenbaum, and Jacob Andreas

    Lionel Wong, Jiayuan Mao, Pratyusha Sharma, Zachary S Siegel, Jiahai Feng, Noa Korneev, Joshua B. Tenenbaum, and Jacob Andreas. 2024. https://openreview.net/forum?id=qJ0Cfj4Ex9 Learning grounded action abstractions from language . In The Twelfth International Conference on Lea...

    2024

  31. [39]

    Sarah A Wu, Rose E Wang, James A Evans, Joshua B Tenenbaum, and David C et al Parkes. 2021. Too many cooks: Bayesian inference for coordinating multi-agent collaboration. Topics in Cognitive Science, 13(2):414--432

    2021

  32. [40]

    Yaqi Xie, Chen Yu, Tongyao Zhu, Jinbin Bai, Ze Gong, and Harold Soh. 2023. Translating natural language to planning goals with large-language models. arXiv preprint arXiv:2302.05128

    2023 arXiv

  33. [41]

    Lance Ying, Katherine M Collins, Lionel Wong, Ilia Sucholutsky, Ryan Liu, Adrian Weller, Tianmin Shu, Thomas L Griffiths, and Joshua B Tenenbaum. 2025 a . On benchmarking human-like intelligence in machines. arXiv preprint arXiv:2502.20502

    2025 arXiv

  34. [42]

    Lance Ying, Tan Zhi-Xuan, Vikash Mansinghka, and Joshua B Tenenbaum. 2023. Inferring the goals of communicating agents from actions and instructions. arXiv preprint arXiv:2306.16207, 2(1):arXiv--2306

    2023 arXiv

  35. [43]

    Lance Ying, Tan Zhi-Xuan, Lionel Wong, Vikash Mansinghka, and Joshua B Tenenbaum. 2025 b . Understanding epistemic language with a language-augmented bayesian theory of mind. Transactions of the Association for Computational Linguistics

    2025

  36. [44]

    Zhining Zhang, Chuanyang Jin, Mung Yao Jia, and Tianmin Shu. 2025. Autotom: Automated bayesian inverse planning and model discovery for open-ended theory of mind. In ICLR 2025 Workshop on Foundation Models in the Wild

    2025

  37. [45]

    Tan Zhi-Xuan. 2022. PDDL. jl: An Extensible Interpreter and Compiler Interface for Fast and Flexible AI Planning. Ph.D. thesis, Massachusetts Institute of Technology

    2022

  38. [46]

    Tan Zhi-Xuan, Nishad Gothoskar, Falk Pollok, Dan Gutfreund, Joshua B Tenenbaum, and Vikash K Mansinghka. 2022. Solving the baby intuitions benchmark with a hierarchically bayesian theory of mind. In RSS 2022 Workshop on Social Intelligence in Humans and Robots

    2022

  39. [47]

    Tan Zhi-Xuan, Gloria Kang, Vikash Mansinghka, and Josh Tenenbaum. 2024 a . Infinite ends from finite samples: Open-ended goal inference as top-down bayesian filtering of bottom-up proposals. In Proceedings of the Annual Meeting of the Cognitive Science Society, volume 46

    2024

  40. [48]

    Tan Zhi-Xuan, Jordyn Mann, Tom Silver, Josh Tenenbaum, and Vikash Mansinghka. 2020. Online B ayesian goal inference for boundedly rational planning agents. Advances in neural information processing systems, 33:19238--19250

    2020

  41. [49]

    Tan Zhi-Xuan, Lance Ying, Vikash Mansinghka, and Joshua B Tenenbaum. 2024 b . Pragmatic instruction following and goal assistance via cooperative language-guided inverse planning. In Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems, ...

    2024

  42. [50]

    Jian-Qiao Zhu and Thomas L Griffiths. 2024. Eliciting the priors of large language models using iterated in-context learning. arXiv preprint arXiv:2406.01860

    2024 arXiv

  43. [51]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  44. [52]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.